A deep learning model has been developed to associate microtremor H/V spectra with soil profiles. This model has the capability to predict soil profiles based on microtremor H/V spectra at any given location. In constructing the model, the first step involves converting acceleration H/V spectra into color spectra (color images), which are then classified into observation sites using deep learning techniques. The dataset utilized in this study comprises microseismic motions with accelerations of 50 gal or less, in total 13,150 waveforms obtained from 87 K-NET stations. The learning model for these color images is tested and demonstrates the ability to classify observation sites with an accuracy of approximately 80%. Additionally, the model is employed to identify the color images of 10 locations within the Kansai University campus, where the soil profiles are known, and compare these profiles with K-NET sites exhibiting similar spectra.