An AI-based analysis of vibration time-histories was pointed out in order to assess the damage in structures. In particular, machine learning techniques were applied to extract features that were used to train a regressor based on k-nearest neighbors. A widely accepted Damage Index (DI) computed in terms of decay of the first modal frequency of the building was considered as indicator of the state of damage. The vibration recordings were processed by conventional Experimental Modal Analysis (EMA) techniques. The proposed workflow was validated through the application to shaking table tests of two common construction typologies: a typical Italian rubble masonry building (RMB) and a two-story reinforced-concrete frame (RCF). Seismic tests were performed at gradually increasing intensity from 0.05 g up to specimens failure. They were alternated with dynamic identification white-noise tests (WHN) at 0.05 g. Vibration measurements were carried out through optical markers of a 3D motion capture system (3DVision) with an accuracy of less than 0.1 mm. The first modal frequency of RMB decayed from 11.5 Hz to 2.6 Hz (DI = 95%) and of RCF from 3.5 Hz to 1.1 Hz (DI = 91%). The quality of the obtained regressions were evaluated using the Coefficient of Determination (R2) and the Relative Percent Deviation (RPD). Moreover, also the fit time and the effect of the time sample length in the signals segmentation were evaluated. Results obtained with the two specimens and with 3DVision data were compared. © 2025 The Authors.