This is the accepted version of the article: Guerrero, Eloi; Brugués, Josep; Verdú, Jordi; [et al.]. «Microwave microphone using a general purpose 24 GHz FMCW radar». IEEE Sensors Letters, Vol. 4, Issue 6 (June 2020), art. 3501004. DOI 10.1109/LSENS.2020.2997978 This version is available at https://ddd.uab.cat/record/232981 under the terms of the license Microwave Microphone Using a General Purpose 24-GHz FMCW Radar Eloi Guerrero, Graduate Student Member, IEEE, Josep Brugués, Jordi Verdú, Senior Member, IEEE, and Pedro de Paco, Senior Member, IEEE, Abstract—The usage of a commercial 24 GHz frequency- modulated continuous-wave (FMCW) radar sensor to retrieve sound signals is investigated in this article. Thanks to the great phase measurement capabilities of current radar sensors, low- PLL frequency audio signals can be recovered by measuring the vibrations they induce in objects due to their pressure wave nature. To prove the concept, a sound emitted by a speaker is recorded by analysing the small-scale displacement of a reflective surface using a radar sensor originally devised for automotive and UAV applications at the 24 GHz ISM band. FFT Index Terms—FMCW, microphone, radar. Sound-vibration FMCW Radar transducer I. I NTRODUCTION Fig. 1: FMCW sound retrieval scheme. ADAR sensors are now available for a wide range R of applications. For example, it is possible that our vehicles feature more than four radar sensors or that traffic aiming to retrieve sound by pointing a radar sensor to the lights at crossroads detect the presence of pedestrians [1], [2] vocal cords of a human being at the 24 GHz and 94 GHz using microwave sensors. Thanks to integration the cost of bands, respectively. Also [10] explores voice recognition from radar systems has decreased rapidly, enabling a long list of micro Doppler data introducing neural networks to classify commercial sensors to deploy. letters and vowels. However, in all cases mentioned above, Many previous applications have covered the use of radar the sensors were continous-wave (CW) radars making use of sensors to retrieve small-scale vibratory motion applied to the well-known CW Doppler approach. structures such as bridges or buildings aiming to monitor In this paper, we approach the measurement of sound by structural health [3], industrial machinery [4] and even loud- means of a frequency-modulated continuous-wave (FMCW) speakers [5]. In this scope, radar sensors compete against radar at the 24 GHz ISM band that is specifically designed for several alternative methods that yield different levels of ac- automotive and UAV applications, thus proving the application curacy. Laser Doppler Vibrometers (LDV), for example, are on a general purpose low-cost sensor. This is possible thanks highly accurate but need higher maintenance and line-of-sight to the great phase measurement capabilities that the FMCW contact. scheme provides. Among vibrations, detection may focus in those generated Fig. 1 depicts the structure of the proposed microwave by sound waves. Since a pressure wave impinging into a body microphone. It is composed of a phase-locked loop (PLL) will trigger very small vibrations over the object’s surface, generating the chirp signal, a transmitter chain for amplifica- detecting them enables turning sensors into microphones. This tion, a transmitting antenna, one (or more) receiving antenna, approach has been investigated by Davis et al. in [6] using a receiver chain to down-convert the received signals using the high-speed video information of an object. They propose the reference chirps and an intermediate frequency (IF) output for so-called visual microphone. However, the use of cameras is further processing. limited to direct line-of-sight with well illuminated objects and need external sources of light. Moreover, sound-triggered II. W ORKING P RINCIPLE vibrations have been previously used for remote sound eaves- As opposed to the continuous-wave approach to radar in dropping in terms of surveillance and security. From a low- which a single unmodulated tone is emitted, the FMCW power millimetre-wave perspective, previous approaches have approach consists in the emission of a train of Np chirp pulses focused on the detection of sound, as in [7], showing how of a certain bandwidth. As introduced in [11], information on to detect heart sounds caused by muscular contraction using the range of present targets can be extracted by mixing the radar systems, or more specifically, the works in [8], [9], received signals with the reference train of chirps. A received chirp will suffer from a small delay (tdelay ) that will translate This work was supported by MCIU-Spain under Grant RTI2018-096019- B-C33. to an IF tone through the mixing process. The IF signal can be Corresponding author: Eloi Guerrero (e-mail: eloi.guerrero@uab.cat). decomposed into its spectral components by means of the Fast Fourier Transform (FFT) on a signal processing module. Using (1), the frequency components can be translated to range in metres, being fIF the IF spectral component, Ts the up-ramp Measure NP chirps time of the chirp pulse, B the swept bandwidth on each chirp and c0 the velocity of light in vacuum. c0 fIF Ts Stack columnwise R= (1) 2B x N frames in nc x Np matrix Considering (1) and the properties of FFT, targets will be resolved as different components in the spectrum if separated at least 1/Ts . Thus, the range resolution can be defined as in FFT over column (2). direction c0 ∆R = (2) 2B Store phase values At the 24 GHz ISM band, the available bandwidth is 250 PSD estimation of k-th row MHz, that is an achievable range resolution of 0.6 metres. Therefore, the sub-millimetre mechanical oscillations caused by sound waves cannot be extracted from the range infor- mation. Consider now a moving target. Not only time delay Fig. 2: Flow diagram of the proposed FMCW sound retrieval with respect to the reference chirp will appear but in fact, a method. nc is the number of samples per chirp and k is the frequency shift due to Doppler effect will be observed. Thus, range position at which the reflective object is found. the signal at the output of the receiver mixing stage will be defined as in (3), being fc the carrier frequency and v the radial to recover the sound that caused the vibrations. In general velocity of the target with respect to the radar. The expression terms, the algorithm in use is shown in Fig. 2. has two unknowns: range and velocity. After receiving Np chirps, each resulting IF tone is trans- B tdelay 2fc v formed to a range profile and the position of the vibrating fIF = ± (3) object is found. Now, the phase value at that range position Ts c0 along the Np chirps is stored in an array and a spectral The output of the initial range FFT is the magnitude estimation algorithm such as Welch [13] is applied to obtain and phase at each of the frequency bins (range bins using the frequency components of the sound that induced the (1)). If the target suffers a small displacement between two vibration. This process can be applied over many frames consecutive chirps, the output of their range FFT will be for a continuous measurement. Since FMCW radar sensors equal in magnitude but different in phase. For an unambiguous measure Np chirps before coherently processing them, a small detection of this displacement, it must fall into the ±λ/4 idle time between frames is necessary and implies a loss of interval between chirps, avoiding cycle slips. Then, if a frame phase coherence when performing a continuous measurement of Np consecutive chirps is emitted and received, the output of of sound. However, our sensor has a total chirp time of 284 µs the range FFT of each chirp can be stacked column-wise. For and its idle time is in the order of 500 µs. Such idle time does a given range bin, the phase evolution can be analysed along not imply a loss of sound information but the appearance of chirps to obtain the velocity of the targets present in that bin. low amplitude components at higher frequencies due to the This is possible by computing another FFT but now in row loss of coherence. These components can be filtered by further direction; the so-called 2D-FFT that yields the well-known processing of the data. Range-Doppler map. Clearly, a relation between a small displacement and a variation in phase can be established as follows: III. E XPERIMENTAL VALIDATION The experimental validation of this article is based on 4π∆r the 24 GHz radar platform DemoRad by Analog Devices ∆φ = (4) λ and Inras. This is a development platform with 2 transmitter In the case of a vibratory motion, consider an object that and 4 receiving antennas that features an Analog Devices vibrates due to a mechanical effort. The object will oscillate RFIC family designed for automotive and UAV applications. with a given amplitude and frequency. In most cases, this The measurement set-up is mounted inside an RF anechoic displacement is almost imperceptible to the naked eye, in the chamber. A 23x13 cm copper plate of 0.5 mm thickness is range of tens of microns, but can be measured by inspection attached to one of the chamber poles and a small speaker cone of the phase evolution along chirps that is detected by the is placed at a distance of 5 cm from the plate. The speaker is radar sensor [12]. From the phase, vibration frequency and connected to a function generator to produce the sound that amplitude can be recovered. What we propose is to use the will induce vibration on the metal plate. At a distance of 2.5 radar to measure the vibrations due to sound waves of a meters, at the other chamber pole, the radar is mounted facing reflective object and then apply a spectral estimation algorithm the plate. The experimental set-up is shown in Fig. 3. 5 0 -5 20 40 60 80 100 120 Chirp Index (a) 5 Fig. 3: Measurement set-up in the anechoic chamber. 0 RX Channel 1 RX Channel 2 -152 -5 -68 20 40 60 80 100 120 -154 -70 Chirp Index -156 -72 (b) -158 -74 -160 Fig. 5: Phase oscillation for an excitation of 440 Hz. (a) direct -76 -162 LoS, (b) behind EPS cover. -78 20 40 60 80 100 120 20 40 60 80 100 120 Chirp index Chirp index 1 RX Channel 3 RX Channel 4 Direct LoS 108 26 EPS Cover 106 24 104 0.5 22 102 20 100 18 98 16 0 20 40 60 80 100 120 20 40 60 80 100 120 200 400 600 800 1000 1200 1400 1600 Chirp index Chirp index Frequency (Hz) Fig. 4: Phase evolution for a 440 Hz vibration with phase drift Fig. 6: Power spectral density of the sound retrieved by the on all four receiving channels of the DemoRad platform. radar for a 440 Hz sound excitation. An initial simple test is carried out using as single frequency of ±0.121 mm. tone of 440 Hz at the speaker. A frame of Np = 128 chirps Having computed the phase evolution along chirps, thanks is measured and the result of analysing the phase at the to the Welch spectral estimation method, the sound spectrum range bin where the plate is resolved is shown in Fig. 4. can be computed as shown in Fig. 6. Note that due to the Clearly, the vibration can be observed but superposed to an differentiation stage only Np − 1 samples of the phase are undesired phase drift. This drift had been previously observed processed. The loss of a sample has an negligible effect in the and reported to the manufacturer. This is a common issue in spectral estimation resolution in this case. many implementations due to slight PLL clock drifts. In the Once the concept is proven for a single frame, the exper- scope of velocity detection is not a considerable issue since it iment is now to analyse how does the radar recover a sound implies a velocity error of ±0.006 m/s, well below the velocity that changes over time, as would be a conversation or a song. resolution of the radar. However, when trying to detect small To do this the analysis takes into account more than a single scale vibrations the drift must be avoided, to reduce its effect frame of chirps. To test these continuous time approach the on the estimation of the frequency. set-up is the same but now configuring the function generator To avoid the phase drift along chirps our procedure implies to change from 440 Hz to 550 Hz and back within a time differentiating the phase evolution with respect to time at of 0.3 seconds. After differentiation, the spectral density can expense of loosing one of the 128 samples. With this, the be computed another time and is plotted with respect to time, phase result of a single receiver channel is now shown in creating the spectrogram in Fig. 7. Fig. 5a and 5b after differentiation. This figure depicts two The validation shown in this paper is performed using a measurements, one with line of sight between radar and plate single receiver channel. In a more realistic scenario related to and the other adding a wall of expanded polystyrene (EPS) automotive radar, it is possible to separate sound information between the two. This is just a simple proof of the capability from different vibration sources applying MIMO techniques of detecting sound without the need of visual contact. Notice thanks to the multiple transmit and receive channels common that the phase excursion is of ±3.5◦ , in turn a displacement in this platforms. Concerning the spectrum occupation of the 1600 [9] S. Li, Y. Tian, G. Lu, Y. Zhang, H. Lv, X. Yu, H. Xue, H. Zhang, J. Wang, and X. Jing, “A 94-GHz millimeter-wave sensor for speech 1400 -20 signal acquisition,” Sensors, vol. 13, no. 11, pp. 14 248–14 260, oct 2013. [10] R. Khanna, D. Oh, and Y. Kim, “Through-wall remote human voice 1200 recognition using doppler radar with transfer learning,” IEEE Sensors Journal, vol. 19, no. 12, pp. 4571–4576, jun 2019. -40 [11] M. Skolnik, Radar Handbook, Third Edition, ser. Electronics electrical 1000 engineering. McGraw-Hill Education, 2008. 550 Hz [12] Y. Xiong, Z. Peng, G. Xing, W. Zhang, and G. Meng, “Accurate and 800 robust displacement measurement for fmcw radar vibration monitoring,” -60 IEEE Sensors Journal, vol. 18, no. 3, pp. 1131–1139, Feb 2018. 600 [13] P. Welch, “The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified 400 -80 periodograms,” vol. 15, pp. 70–73, 1967. [14] M. Gardill, J. Schwendner, and J. Fuchs, “An approach to over-the- 200 440 Hz 440 Hz air synchronization of commercial chirp-sequence automotive radar sensors,” in 2020 IEEE Topical Conference on Wireless Sensors and 0 -100 Sensor Networks (WiSNeT). IEEE, jan 2020. 50 100 150 200 250 300 Time (ms) Fig. 7: Sound spectrum evolution for the 440-550-440 Hz series. 24 GHz ISM band, it is possible to mitigate interference from other automotive radar sensors applying de-ramping techniques as shown in [14]. IV. C ONCLUSION In this paper the use of a commercial FMCW radar to recover sound has been explored and demonstrated. Previous approaches in the literature used full continuous-wave custom radars to measure sound. In this case, the chirp rate used in automotive and UAV applications can achieve a detectable audible range of 3.4 kHz, that of the telephonic line, what indicates that this application can recover voice-like quality sound. The experiments also demonstrate the capability of easily detecting a displacement in the order of hundreds of microns. R EFERENCES [1] R. Pérez, F. Schubert, R. Rasshofer, and E. Biebl, “Single-frame vul- nerable road users classification with a 77 ghz fmcw radar sensor and a convolutional neural network.” IEEE, 2018, pp. 1–10. [2] A. Bartsch, F. Fitzek, and R. H. Rasshofer, “Pedestrian recognition using automotive radar sensors,” Advances in Radio Science, vol. 10, pp. 45– 55, sep 2012. [3] M. Pieraccini and L. Miccinesi, “An interferometric MIMO radar for bridge monitoring,” IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 9, pp. 1383–1387, sep 2019. [4] G. Vinci, S. Lindner, S. Mann, F. Barbon, S. Linz, R. Weigel, and A. Koelpin, “Six-port microwave interferometer radar for mechanical vibration analysis,” in 2013 European Radar Conference, 2013, pp. 287– 290. [5] A. Izzo, L. Ausiello, C. Clemente, and J. J. Soraghan, “Loudspeaker analysis: A radar based approach,” IEEE Sensors Journal, vol. 20, no. 3, pp. 1223–1237, feb 2020. [6] A. Davis, M. Rubinstein, N. Wadhwa, G. Mysore, F. Durand, and W. T. Freeman, “The visual microphone: Passive recovery of sound from video,” ACM Transactions on Graphics (Proc. SIGGRAPH), vol. 33, no. 4, pp. 79:1–79:10, 2014. [7] C. Will, K. Shi, S. Schellenberger, T. Steigleder, F. Michler, J. Fuchs, R. Weigel, C. Ostgathe, and A. Koelpin, “Radar-based heart sound detection,” Scientific Reports, vol. 8, no. 11551, 2018. [8] Heng Zhao, Zhengyu Peng, Hong Hong, Xiaohua Zhu, and Changzhi Li, “A portable 24-ghz auditory radar for non-contact speech sensing with background noise rejection and directional discrimination,” in 2016 IEEE MTT-S International Microwave Symposium (IMS), May 2016, pp. 1–4.