Enabling richer data for the applications of the future

The concept of sensor fusion is already widely used in applications such as motor sport, medical and aerospace electronics to gather extremely rich data about the system’s environment and enable it to be acted upon in real time. The basic premise is that combining the data from more than one type of sensor can tell the system more about its environment than the two data sets in isolation. For example, an autonomous driving application might use both a high end image sensor as well as a radar system to detect objects and obstacles in the road around the car.

While the camera can recognise the shape of an oncoming truck, the radar sensor system can tell its speed and direction of travel. The combination of data from these two sensor systems allows the vehicle’s control algorithms to make a decision as to whether the truck is parked or whether to wait before entering a narrow stretch of road in order to let the truck pass through first.

While the basic concept isn’t new, having been used for a long time in everything from military systems to game controllers, advances in technology are enabling sensor fusion for even low end applications today. The main barriers to entry in this field were obtaining accurate, cheap sensors, the cost of processing power for combining the data and the ability to combine the data fast enough to respond to the results in real time. These things are no longer out of reach. The prevalence of consumer smartphones, which offer processing power and internet connectivity, have allowed even the smallest of IoT nodes to take advantage of sensor fusion. Further, MEMS technology has come a long way in increasing sensory accuracy while reducing cost.

Some of the simplest sensor fusion systems, relying on just two data sources, are also the most widely used. Relative humidity sensors, widely used in HVAC systems, combine a moisture sensor and a temperature sensor, and calculate the amount of liquid water present in air expressed as percentage saturation at that particular air temperature. The relative humidity of a building such as a home or office has a big effect on how the inhabitants feel, as it affects their perception of the ambient temperature. HVAC systems use relative humidity measurements to understand how well air is circulating in the building and whether humidifiers or dehumidifiers are needed.

Relative humidity sensors come in a couple of different form factors. ALPS has a humidity and temperature sensor module mounted on a small PCB, the HSHCAL series, which enables a lot of design freedom as it can be mounted in the optimal location within the system. Meanwhile, TE Connectivity’s HTU21D humidity and temperature sensor module comes in a reflow solderable DFN package for automated assembly, which measures just 3x3x0.9mm for compact applications. Both these relative humidity sensor examples operate across the full range of 0 to 100% relative humidity, and provide a digital (I2C) output of the data for direct interface with a microcontroller.

Another classic example of sensor fusion is the combination of multi-axis accelerometers, gyroscopes and magnetometers. All these sensor types are widely available as integrated MEMS devices in small packages. Combining three accelerometers, acting along perpendicular axes, results in a sensor that can detect lateral movement in any direction. This type of sensor is referred to as having three degrees of freedom (3DOF). MEMS gyroscopes are also widely available to measure rotational acceleration, and again, three are typically combined to measure rotational movement in three dimensions (roll, pitch and yaw). Where it gets really interesting is when a 3DOF accelerometer is combined with a 3DOF gyroscope – the combination of linear and rotational movement in three dimensions is enough to describe the exact movement of any rigid object in three-dimensional space. This type of sensor is called a 6DOF sensor.

Taking the principles of sensor fusion a step further, a 3DOF magnetometer is often combined with 6DOF sensors to try to compensate for the inaccuracies of the other two. For example, accelerometers and gyroscopes often suffer from inaccuracies caused by vibration of the system. Adding a magnetometer to measure movement relative to the earth’s magnetic field can help compensate, while the accelerometer and gyroscope can fill in any gaps in magnetometer data caused by interference. This 9DOF sensor works better than the sum of its parts (and is commonly called an IMU: inertial measurement unit), but combine it with a temperature sensor in order to compensate for the effects of atmospheric fluctuations on the other MEMS sensors and it can be made more accurate still. Barometric pressure sensors are also sometimes added to measure altitude for positioning purposes. In this way, 10 and 11DOF sensors are created.

9DOF IMUs often also incorporate a microcontroller to perform filtering and fusion of the data using algorithms. These IMUs output the data in simple format for easy integration into systems and are widely available in compact packaging today, for general applications including GPS systems, consumer devices and industrial robotics. Different combinations of the degrees of freedom are also available for specific applications. For example, Murata has a combined single axis gyro and 3DOF accelerometer solution, the SCC2000 series, which is designed for extremely harsh environments and can produce reliable data even when exposed to shock and vibration. This part is used in industrial machine control, where robotic arms are used to complete manufacturing tasks such as welding, painting and material handling. These robots can move quickly in any direction so they need to have accurate knowledge of the position of their ‘hand’ (end of arm tooling) in order to complete their tasks. These robots often operate in areas that are deemed too dangerous for humans to work in, so it’s important that the sensors used to gather the data for fusion can withstand extreme temperatures, shock and vibration.

Another important application for the SCC2000 is automotive electronic stability control (ESC). ESC is used to monitor the movement of the car to detect skid (aquaplane) situations. This system is a prime example of sensor fusion, as it uses data from many sensors to detect a complex situation and respond in real time. A steering wheel angle sensor detects the direction the driver is intending to go. A wheel speed sensor detects changes in speed that can be caused by loss of traction. And a sensor like the SCC2000 provides data on the car’s linear acceleration and yaw rate.

A complex control algorithm combines this data, and if the result indicates the start of a skid, the brakes can be applied to individual wheels or other safety measures taken. For ESC, an SCC2000 with the z-axis gyro configuration is used; it comes in x- or z-axis configurations, or you can put three on the same board for a 6DOF solution. This configuration is used in applications where single-package 6DOF sensors can’t resist the harsh environment of the application. Other automotive applications that can use the SCC2000’s triple accelerometers and one gyro include hill start assist, to tell if the car is stopped or parked on a hill, roll over detection which detects an accident, and navigation systems.

Other simple applications for multi-DOF movement sensors include the pedometer, which has seen mass adoption as consumers begin to use technology for health and wellness purposes. Pedometer wristbands have improved their accuracy by integrating altitude sensors, which themselves perform the fusion of data from barometric pressure and temperature sensors for accuracy. Fusing altitude data with movement data means the effect of walking uphill or up stairs can be taken into account when the device calculates how many calories have been burned.

The enabling technology for all the applications listed here is MEMS. As MEMS has matured, it has become more accurate, more compact, and much less expensive. Automatic calibration is now often performed in the factory, so MEMS sensors are easier and cheaper to integrate too. Almost any combination of sensors can be had. It is this technology, combined with access to cheap processing power in smartphones and the cloud, that is driving sensor fusion in internet of things (IoT) applications and beyond. In the future, highly complex sensor fusion algorithms will evolve, making machines and electronic systems fully aware of their environment, and able to respond in real time. Data from MEMS movement sensors will be fused with data from camera systems and other sensor types to enable autonomous driving and other extremely complex tasks.

www.avnet.abacus.eu