When Panasonic showed its conceptual design for the cabin of a self-driving car at the Consumer Electronics Show (CES) earlier this year, one of the demonstrations used thermal cameras to paint an image of the occupants’ body heat on an overhead flat-panel display. Although future travellers might enjoy watching thermal images of themselves as they ride along, thermal sensors already have a role to play in vehicle design, and other sensors are joining them.
Joris Roels, temperature sensors marketing manager at Melexis says their use is currently limited to high-end vehicle, but infrared (IR) imagers have advantages over conventional heat sensors. Comfort is one reason. The air-conditioning can tune its output to what the occupants feel rather than aim for a specific temperature.
IR sensors can also supplement the simple designs already in use, such as strain gauges in the front seats. They inform both the seatbelt warning indicator and the airbag-firing controller. But such sensors are easily fooled by a heavy case sitting on the seat. Body-heat sensors help overcome the problem of mistaking a bag for a body, especially when combined with other sensor and camera technologies.
Another imager-based technology likely to take hold is the time-of-flight (ToF) camera, which measures the time it takes for reflections from pulses of light transmitted by a nearby laser or LED to arrive at each pixel within the image sensor. For automotive use, the cameras are designed to measure distances accurately over a range of up to two or three meters in the vehicle cabin.
Dealing with the speed of light in the confined space of a vehicle is challenging, says Norbert Bieler, senior marketing manager for emerging automotive technologies at Analog Devices. But the information that results is a high resolution VGA image and accurate depth map indicating the distance of an object to the camera. That makes it easier for software to discriminate between parts of the vehicle and people and objects sitting or moving in the cabin.
ToF cameras in vehicles will typically use IR for illumination rather than visible light to avoid distracting the driver. Dealing with driver distraction is a key part of the initial wave of applications for the cameras. Bieler points to the increasing complexity of in-car controls as a driving force for a shift to user interfaces that let the driver make changes to the air conditioning or entertainment without forcing them to take their eyes off the road. A simple wave of one hand can fast-forward through a song instead of forcing the driver to look around for the appropriate dial on a dashboard or centre pillar touchscreen.
The camera provides a way for the car to check on the driver in situations that will become more common as the vehicle provides greater levels of autonomous control. At CES, Analog Devices demonstrated a combination of touch sensors in the steering wheel with a ToF camera to help determine whether the driver is able to take control after autonomous driving mode or simply to tell whether the driver is texting when they should be watching the road.
In a similar demonstration at CES, Infineon Technologies showed how a ToF camera can make the car’s driving behaviour respond to movements by the driver and front-seat passenger. “The camera can be employed in a lot of use-cases such as adjusting the airbag,” says Martin Lass, ToF marketing manager at Infineon. The driver might, for example, recline in their seat while the car is driving itself. “So you will want to adjust the airbag pressure in case there is a collision. You can also adjust the rear-view mirror based on the current head position. If I move to the right side, the mirror will follow my position.”
Bieler says: “In the future, there is potential for using these cameras with other sensors for applications such as driver monitoring. The target is a modular, scalable interior sensor set that includes different sensor technologies. We are cooperating with vehicle makers, tier ones and strategic partners to understand the technological requirements.”
Working together
For a supplier like Analog Devices, it means bringing together the work of multiple groups, such as the healthcare, consumer and industrial divisions. The additional technologies include sensors to monitor the vital signs of the driver, such as heartrate and skin conductance, which can indicate levels of stress and body-fluid levels. Bieler says: “Dehydration helps detect drowsiness. But our vital-signs monitoring technology may in the future enable even more break-through functions going beyond health and wellness parameter sensing.
“Overall, the high-level target is to get a clear view of who or what is in the cabin and what is happening there,” Bieler says.
Although both ToF and thermal imagers use IR, they cannot be integrated into one sensor technology. The heat sensors need to work at lower resolution as they work in the far-IR region and employ different sensor elements to the imagers used in ToF applications.
Gualtiero Bagnuoli, optical sensors marketing manager at Melexis, says: “IR in combination can be useful to distinguish between a live person and an object with the apparent shape of a human being. A combination of the two might be necessary for safety reason or to ensure much higher reliability.”
One application for a combination of IR technologies is to detect whether an infant has been left behind in the vehicle. The system can activate the air-conditioning to prevent overheating while sending a message to the driver’s phone.
| "IR in combination can be useful to distinguish between a live person and an object with the apparent shape of a human being." Gualtiero Bagnuoli, optical sensors, marketing manager |
Israel-based Guardian Optical Technologies won $5m in Series A funding last year to work on its own sensing technology. The company claims a roof-mounted sensor can detect movement down to 1µm resolution and can use that as a way of spot how many people are in the car at any time. Like ToF, Guardian expects to use the sensor technology to handle a range of gesture-recognition tasks.
In the short term, manufacturers such as Kia are using simpler proximity sensors, which can be based on IR or capacitive techniques depending on usable range, to pick up on simple gestures. To be launched later this year, the K900 luxury vehicle will adjust the lighting around the dashboard dynamically when an occupant’s hand reaches toward it.
| The Kia K900 interior which uses a range of simple proximity sensors |
Other sensors are focused less on the occupants than on the environment they are in, with air quality a potentially major target although uptake is likely to be regionally focused, depending local concern over pollution and legislation. Recirculating air can lead to the build-up of unwanted gases such as volatile organic chemicals (VOCs), and engine-exhaust gases picked up from the traffic flow and recirculated by the air-conditioning system. If the system detects concentrations approaching a limit it can trigger the intakes to collect more fresh air.
In principle, air-quality detection is another good application for IR sensors. However, metal-oxide semiconductor sensors have advantages for automotive designs. IR chemical sensors need to be focused on specific wavelengths absorbed by a target molecule. The semiconductor sensors detect a broad range of gases because they focus on resistance changes that occur as the molecules adsorb onto the surface.
Although the rise of autonomous driving will lead to a forest of sensors being deployed around the exterior of the car, manufacturers are clearly not neglecting the interior. The will use a variety of sensors to keep a closer eye on what is happening inside the vehicle.
