A group of researchers from five US universities devised a new attack method for Android smartphones that allows, to varying degrees, determine the gender and identity of the caller, and it can even decipher private conversations.
Dubbed “EarSpy,” the side-channel attack is designed to capture motion sensor data readings caused by reverberations from ear speakers in mobile devices.
Although eavesdropping attacks have been tested on smartphone speakers, ear speakers were deemed too weak to generate enough vibrations to make a side-channel attack practical. However, modern smartphones have more sophisticated stereo speakers and come with more sensitive motion sensors that can detect even the most subtle vibrations coming from the speakers.
“Among the built-in sensors of smartphones, motion sensors are mostly known as vulnerable to eavesdropping. Adversaries leverage motion sensors to collect audio (e.g., voice conversation), touch screen inputs, and even indoor locations. Eavesdropping through motion sensors is straightforward, as adversaries do not need explicit permission to collect raw data from them,” the researchers wrote in a technical paper.
The team tested their method on a OnePlus 7T and a OnePlus 9 smartphone running Android 11 and Android 12 respectively, using varying sets of pre-recorded audio that was played only through the ear speakers of the two devices. The researchers also used the third-party app called ‘Physics Toolbox Sensor Suite’ to capture accelerometer data during a simulated call and then analyzed the captured data using a program in MATLAB.
For the purpose of recognizing voice content, caller identity, and gender, a machine learning (ML) system was trained using datasets that were easily accessible.
“We found up to 98.6% accuracy on gender detection, up to 92.6% accuracy on speaker detection, and up to 56.42% accuracy on speech detection, which proves the presence of distinguishing speech features in the accelerometer data that the adversaries can leverage for eavesdropping,” the researchers noted.
“Although recent smartphones use larger and more powerful ear speakers, they still reduce the volume at a reasonable level to ensure the comfort of the users during a phone conversation. As a result, they cannot generate a significant impact on raw accelerometer data,” they added. “However, our result indicates that it is sufficient for the adversary to reasonably detect significant speech features (e.g., gender, speaker’s identity, speech).”