This paper presents the identification of impact forces acting on sandwich panels, and also the detection of the impact-induced damage based on the identified impact force. In the present study, the impact location and the force history are identified using the radiated sound at impact which is measured with microphones. The impact location is identified using the differences in the arrival times of the sound wave at the microphones. The force history is determined from the measured sound pressure using the identification method based on experimental transfer matrices. Here, the experimental transfer matrix expresses the relationship between the applied impact force and the measured sound pressure, and it is constructed based on the measured data obtained by conducting preliminary impact tests. The damage occurrence is judged by examining the identified force history for the presence of sharp drops and small fluctuations due to damage propagation. The validity of the proposed method is verified experimentally with a sandwich panel composed of CFRP facesheets and an aramid honeycomb core by conducting impact tests using an impulse hammer. The results reveal that the location and force history of the applied impact can be identified accurately by using the radiated sound even when damages are induced within the structure. In addition, it is shown that the occurrence and location of the damage can be estimated in real time from the identified impact force.