Dec 8 2020
When a robot vacuum cleaner performs its task around a house, it is also likely to pick up private discussions along with the household dirt and dust.
Assistant Professor Jun Han (left) and doctoral student Mr Sriram Sami (right) from NUS Computing with a robot vacuum cleaner, a monitor showing recovered sound waves, and common household items made from materials that can reflect sound. Image Credit: National University of Singapore.
Computer scientists from the National University of Singapore (NUS) have now demonstrated that private conversations can be certainly spied through a standard robot vacuum cleaner and its integrated Light Detection and Ranging (Lidar) sensor.
Dubbed LidarPhone, the innovative technique repurposes the Lidar sensor that is normally used by a robot vacuum cleaner to find its way around a house into a laser-based microphone to spy on private discussions.
Headed by Jun Han, an Assistant Professor from NUS Computer Science, and his doctoral student Mr Sriram Sami, the researchers were able to recover speech data with excellent precision. The study also involved Mr Dai Yimin and Mr Sean Tan Rui Xiang, both NUS students, and Nirupam Roy, an Assistant Professor from the University of Maryland.
The proliferation of smart devices—including smart speakers and smart security cameras—has increased the avenues for hackers to snoop on our private moments. Our method shows it is now possible to gather sensitive data just by using something as innocuous as a household robot vacuum cleaner. Our work demonstrates the urgent need to find practical solutions to prevent such malicious attacks.
Mr Sriram Sami, Doctoral Student, Department of Computer Science, National University of Singapore
The researchers presented their study at the Association for Computing Machinery’s Conference on Embedded Networked Sensor Systems (SenSys 2020) on November 18th, 2020. They also won the Best Poster Runner Up Award at the conference.
How the Attack Works
The Lidar sensor forms the heart of the LidarPhone attack method. This device emits an invisible scanning laser and generates a map of its environment. By reflecting lasers off regular objects, like a takeaway bag or dustbin placed close to an individual’s television soundbar or computer speaker, the invader could acquire data about the original sound that caused the surfaces of the objects to vibrate.
With the help of deep learning algorithms and applied signal processing, scientists could recover speech from the audio data, and may even acquire sensitive data.
The researchers conducted experiments in which they used a standard robot vacuum cleaner equipped with a couple of sources of sound. One source was music clips from television shows played via a television soundbar, while the other one was the voice of an individual who was reading out numbers played by a computer speaker.
The researchers obtained over 19 hours of recorded audio files and then ran them through deep learning algorithms. These algorithms were trained to detect musical sequences or match human voices. The system effectively detected the digits being read aloud, which could represent the bank account or credit card numbers of a victim.
Similarly, music clips from television shows may also reveal the political orientation or viewing preferences of the victim. The system effectively achieved a classification accuracy rate of 90% when classifying music clips and 91% when recovering the spoken digits. The outcomes are considerably higher when compared to a random guess of 10%.
Besides this, the team worked with regular household materials to find out how well these materials reflected the Lidar laser beam, and they eventually observed that the precision of audio recovery differed between different materials.
The researchers also learned that a glossy polypropylene bag is the most optimal material for reflecting the laser beam, while glossy cardboard was the worst.
Preventing Such Attacks
Hence, to avoid the misuse of Lidars, the researchers have proposed that users should not connect their robot vacuum cleaners to the Internet. They also recommended that manufacturers of Lidar sensors should integrate a mechanism that no one can override. This approach can avoid the internal laser from firing when the Lidar is not revolving.
In the long term, we should consider whether our desire to have increasingly ‘smart’ homes is worth the potential privacy implications. We might have to accept that each new Internet-connected sensing device brought into our homes poses an additional risk to our privacy, and make our choices carefully.
Jun Han, Assistant Professor, Department of Computer Science, National University of Singapore
Future Work
The researchers are looking for ways to apply the concepts learned from LidarPhone to autonomous vehicles—which also make use of Lidar sensors—as they could also be applied to spy on private conversations that take place in adjoining cars through slight vibrations of the vehicle windows. The team is also investigating the susceptibility of active laser sensors integrated into next-generation smartphones, which may disclose more privacy problems.