AdaSky explains why thermal cameras are essential to AV safety
The technology stack for today’s advanced driver assistance and autonomous systems commonly includes a combination of radar, LiDAR, and ultrasonics, as well as cameras that read visible light and infrared radiation. However, Yakov Shaharabani, Chief Executive of Israel-based AdaSky, believes cameras that capture the unique heat signature of roadway objects will soon become a “prime focus” of autonomous technology.
AdaSky, along with Seek Thermal, are looking to challenge the leading provider of thermal cameras, FLIR, in providing another redundant sensor for safer vehicle autonomy. Its devices measure emissivity. Shaharabani explained: “The emissivity of an object is its effectiveness in emitting energy as thermal radiation.”
We received a demonstration of AdaSky’s system in Las Vegas on the sidelines of CES 2020. Dashboard-mounted cameras fed by AdaSky’s thermal camera were able to distinguish people and objects on nearly pitch-dark suburban streets. The clarity was uncanny. The system can also handle quick changes of lighting, such as oncoming headlights and high-contrast light while emerging from a tunnel—the corner cases that throw off cameras using visible light.
The big promise of thermal cameras is the ability to detect common roadway objects—not just heat-producing pedestrians and animals—in very dark environments and during extreme fog, snow, and other harsh weather conditions.
“Two objects can have the same heat,” said Shaharabani. “But they don’t have the same emissivity, which is a combination of heat and the material that it’s made of.”
From the moment of the company’s founding in January 2016, its goal was to build a thermal camera from the ground up specifically for automotive applications. AdaSky’s competitors provide thermal cameras also for military, security, firefighting, marine, hunting, and other industrial applications.
This focus on automotive—both in terms of requirements and supply chain—led AdaSky to set its sights on a thermal camera that’s smaller, requires less power, produces more precise data, and costs less. Shaharabani pegs a final at-scale cost for its thermal camera in the single digits, meaning between $10 and $99.
Jim Hines, an independent consultant working on sensor technologies for connected and autonomous vehicles, said that night-vision systems have been used in luxury models from Audi, BMW, Cadillac, and Mercedes-Benz.
“The feature never migrated to mid-market models because it didn’t deliver a strong safety benefit,” said Hines. “In other words, conventional headlight illumination was sufficient in most cases.” Vehicle autonomy is providing a new opportunity for the technology.
AdaSky’s camera, which has no moving parts, does not use a cooling system or a shutter that requires operations to pause for a half-second or longer—a critical period of blindness for a moving car—to recalibrate the inner components.
“The temperature of some of those components in the camera can change in a non-linear way,” said Shaharabani. “We solve those discrepancies only with algorithms.”
Instead of using hardware to compensate for those fluctuations in temperature, AdaSky’s software probes the system to identify erratic changes and apply corrections to produce crisp, reliable images. That’s made possible via a tailor-made image-sensor processor (ISP) manufactured by STMicroelectronics. The other hardware components include a lens (available with different fields of view), an emissivity detector, and a power/communications board that delivers the signal to the car’s computer.
Proprietary software also enables AdaSky to yield useful data via a library of heat signatures.
“Every object is providing heat,” explained Shaharabani. “The heat signature of a car, the glass, the engine, the tires, are all different.”
The mapping of those signatures, which improves over time with more data, enables the system to quickly and precisely distinguish between different types of objects.
While AdaSky is not yet talking about the specific automakers expected to deploy its thermal cameras, there is media speculation about applications in U.S. pickup trucks. That’s partly because thermal sensors could be a compelling feature for truck owners to help avoid collisions with large animals on dark country roads. Thermal cameras, depending on the field of view, can sense as far as 200 m (656 ft), well beyond the reach of headlights.
Similarly, automakers are considering thermal sensors for backup applications, which don’t benefit from bright headlights. Shaharabani, no surprise, believes thermal cameras should be placed front, back, and even on sides for effective blind-spot detection.
Ultimately, what could catapult thermal sensors to the mass market is changes in global test procedures. Currently, the U.S. National Highway Transportation Safety Administration (NHTSA) and The European New Car Assessment Program (Euro NCAP) do not fully consider safety in dark settings typically encountered in real-world driving. If global auto-safety agencies start to evaluate safety when there is little to no surrounding illumination, then thermal cameras could effectively become required gear to achieve five-star ratings.
“There are many examples of safety features that were initially offered on premium models eventually becoming mandated on all cars,” said Hines, the sensor-industry analyst. “Thermal imaging could follow that path, but it will depend on the cost and the expected impact on road safety.”