A team of researchers devised a new technique called ‘Side Eye’ that can extract audio from static photos and silent (or muted) videos and even determine the gender of someone speaking in the room where a photo was taken.
The new technique takes advantage of the image stabilization technology and the rolling shutter technique to capture images to create a point-of-view (POV) optical-acoustic side channel for acoustic eavesdropping.
“The movement of smartphone camera hardware leaks acoustic information because images unwittingly modulate ambient sound as imperceptible distortions,” the researchers explained in a research paper, adding that “this side channel requires no line of sight and no object within the camera’s field of view.”
As part of the research, the team used a dataset of 10,000 samples of signal-digit utterances from 10 males and 10 females to perform three classification tasks (gender, identity, and digit recognition) and trained their model for each task. They used Google Pixel, Samsung Galaxy, and Apple iPhone devices for the experiments.
“Our evaluation with 10 smartphones on a spoken digit dataset reports 80.66%, 91.28%, and 99.67% accuracies on recognizing 10 spoken digits, 20 speakers, and 2 genders respectively,” the team wrote.
The researchers said that Side Eye currently doesn't work with speech from human voices and was only tested with sound from powerful speakers.
Users can mitigate the risks of such attacks by using devices with low-quality cameras, which would limit information embedded in videos by reducing video resolution and frame rate. Keeping smartphones away from electronic speakers and adding vibration-isolation dampening materials like vibration reduction mats between the device and surface also should help to reduce the surface of the attack.