Far infrared sensing designed for better AV safety
Autonomous vehicles will need to quickly and reliably detect all obstacles in their path. To date, this task has primarily been undertaken using LiDAR, radar, and visible-light sensing; while all offer useful capabilities, none can guarantee detection under all conditions, especially when factors such as rain, snow, fog, smoke, glare, and low light are involved.
Far infrared (FIR) detectors have two capabilities that make them excellent candidates for use in comprehensive sensing solutions alongside other technologies. First, FIR sensors can “see” the heat emitted by objects in the form of thermal infrared photons, even in the absence of visible light. Second, FIR sensors can help distinguish among different types of objects based on how effectively they emit radiation (their emissivity). These two types of information open new avenues for detecting and identifying obstacles and determining how to respond to them.
Consider, for example, a highway in thick fog with light snow falling as shown in Figure 1. The lower-right image, made with visible light (VIS), offers almost no evidence of the vehicle ahead. The other three frames use different portions of the infrared spectrum to discern its presence.
Two detectors primarily sense the photons emitted by the vehicle’s lights: the near-infrared (NIR) at lower left, which operates in the 0.7-0.95 µm frequency band, and the short-wave infrared (SWIR) at upper left, which operates at 1-1.7 µm. The upper-right image, however, is of particular interest; it is made with an FIR sensor that is sensitive to photons emitted by the car body in the 8-12 µm long-wave infrared (LWIR) band. This demonstrates that the FIR sensor can detect objects without any additional source of light, and also provide important information about size and shape of the vehicle’s body.
Similarly, Figure 2, made in a “fog tunnel” test facility, shows that pedestrians who are barely detectable by the visible-light sensor (lower right) are apparent in the LWIR image.
AWARE Project combined LWIR and visible-light sensors
Both tests were conducted by the All Weather All Roads Enhanced Vision (AWARE) project, a recently concluded three-year collaboration among 10 French companies and research organizations that explored both automotive and aerospace sensor applications. Researchers determined that, under adverse conditions, a pairing of visible-light sensors (ideally with range extended into the NIR spectrum) and FIR sensors is optimal for detecting vehicles and pedestrians.
Onboard algorithms will choose the most appropriate combination of signals for given situations; fortunately, FIR images require much less processing power than visible light images, so vehicle computing resources should not be an issue.
Of course, technical capabilities alone are not sufficient for practical implementation of FIR sensing, and it has been limited until now by affordability and vehicle integration challenges. Cooled FIR sensors (such as HgCdTe photodiodes) are unsuitable for broad use because of very high production costs and their need for cryogenic cooling. On the other hand, although uncooled FIR sensors are available from a number of vendors, they have traditionally lacked the necessary resolution and have also been cost-prohibitive for mass markets.
Microbolometer sensors can be produced using standard chipmaking processes
Fortunately, new generations of IR detectors are poised to alter traditional standards of performance, cost, and size. Especially exciting are uncooled microbolometer sensors that leverage standard semiconductor and micro-electromechanical system (MEMS) materials and production techniques.
One such development effort began in the 1990s at Leti, a government micro- and nanotechnology research institute in France. The technology was spun out in 2002 to ULIS, a private company that has brought many IR sensors to market, while continuing to partner with Leti for R&D.
These microbolometers are becoming fully compatible with mainstream semiconductor production processes and tools, and can thus follow traditional chip industry trends towards smaller size, lower price, and better performance.
The goal: finer resolution, reduced costs & lower power consumption
Current devices have a pixel size of 12 µm, down from an original 45 µm, enabling for example a 320 x 240 resolution in a 16.5-mm square package. Work is progressing towards finer resolution, lower cost, and reduced power consumption, through wafer-level vacuum packaging technologies, improved IR lens fabrication, and new pixel architectures and materials, as well as integration of processing abilities.
Uncooled IR sensors are now established in several sectors, including thermography, predictive maintenance, energy conservation, smart buildings, and firefighting, with 2018 sales estimated at about 1 million units and $3 billion in revenue. A handful of high-end automotive companies (e.g. Rolls-Royce, BMW, Mercedes-Benz, and Cadillac) have integrated single forward-looking FIR sensors into their vehicles, and rapid growth is anticipated. A recent Yole Développement report on autonomous vehicles forecast 3.4 million automotive microbolometer shipments annually by 2030.
Autonomous vehicles have many hurdles to clear before they can offer high-security driving. But IR sensors appear poised to help tackle the essential task of object detection and recognition in challenging situations, with vast potential for ongoing performance improvement and price reduction.