Track energy use & carbon footprint in real-time

MagnifiSense correctly classified 94% of users’ interactions with 12 common devices after a quick one-time calibration, including microwaves, blenders, remote controls, electric toothbrushes, laptops, light dimmers and even cars and buses. Even without the calibration, MagnifiSense was still correct 83% of the time.

The sensor worn on the wrist uses unique electromagnetic radiation signatures generated by electrical components or motors in those devices to pinpoint when its wearer flicks a light switch, turns on a stove or even boards a train.

“It’s another way to log what you’re interacting with so at the end of the day or month you can see how much energy you used,” said Shwetak Patel, Washington Research Foundation Endowed Professor of Computer Science & Engineering and Electrical Engineering, who directs the UW Ubicomp Lab.

“Right now, we can know that lights are 20% of your energy use. With this, we divvy it up and say who consumed that energy,” said Patel.

Appliances and vehicles such as cars, buses and trains emit a unique pattern of electromagnetic radiation, based on the combination of electrical components that make them run.

In a 24-hour test in which a single user did everything from read on a laptop to cook dinner and take a bus ride, the system correctly identified 25 out of 29 interactions with various devices and vehicles.

MagnifiSense also has potential for other smart home applications, such as recognising a user’s preference for interacting with an appliance or device. By sensing whether an adult or child is turning on a television or tablet, for instance, a system could automatically display their favourite programmes or tailor the device with appropriate selections.

In assisted living settings or nursing homes, the wearable sensor could help keep track of how efficiently elderly people are going about everyday tasks such as cooking or grooming. It could also detect when a stove has been left on for a long period of time and help alert someone to that danger.

“The nice thing with MagnifiSense is that you don’t have to instrument every single appliance in your house, which gets expensive and cumbersome,” said lead author Edward Wang, a UW electrical engineering doctoral student. “It can also sense some of the blank spots that other technologies can’t, like battery-powered devices.”

The team combined three simple, off-the-shelf sensors that use inductors, or coils of wire wound around magnets. Those proved to be the most accurate without being so power-hungry that wearing them would be impractical.

The sensors also capture a broad frequency range that allows the system to differentiate between electromagnetic radiation emanating from the unique combinations of electronic components such as motors, rectifiers and modulators embedded in everyday devices.

These sources of unique electromagnetic radiation patterns enable MagnifiSense to identify what devices its wearer is using.

“When a blender turns on, for instance, modulators change the current profile of the device and create something similar to a vocal cord pattern,” Wang said. “A blender ‘sings’ quite differently than a hairdryer even though to our ears they sound similar.”

The team also developed innovative signal processing and machine learning algorithms to help the system correctly match those patterns with a particular type of device.

One advantage to a wearable option is that anyone concerned about privacy issues can control when they use it, researchers said, or simply take it off.

Next steps include testing MagnifiSense on a wider variety of devices and distinguishing between multiple devices operating in close proximity. In preliminary tests, for instance, MagnifiSense had the most trouble correctly classifying a handful of particular toothbrushes, shavers and cars.

The researchers also plan to work on miniaturising their proof-of-concept device into something that could be embedded into a watch or band. Based on its investigation, the team believes that with slight improvement to the update rate of magnetic sensors in current smartphones and smartwatches, MagnifiSense could soon be enabled on new devices with a simple software upgrade.

“We think it could be integrated into any wrist-sized product,” said Patel. “The next steps are really to look at what other devices we can detect and work on a prototype that’s wearable.”

Co-authors include UW electrical engineering doctoral student Tien-Jui Lee, UW computer science and engineering doctoral students Alex Mariakakis, Mayank Goel and Sidhant Gupta.