Smart Cities and V2X thermal imaging
Mike Walters and Michael Deruytter of FLIR Systems Inc. discuss how thermal imaging combined with V2X technology can help bridge the gap in the existing sensor suite to provide for consistent connected mobility.
Vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2X) crash-avoidance technologies rely on the communication of information between nearby vehicles and infrastructure to warn drivers about potentially dangerous situations that could lead to crashes. For example, V2X helps warn a drivers that a vehicle ahead is braking and he/she needs to slow down, or it lets a driver know that it is not safe to proceed through an intersection because another car, unseen by the driver, is quickly approaching.
The mainstream adoption of advanced driver-assistance systems (ADAS) and autonomous vehicle (AV) technology will revolutionize how we travel and transport goods while also improving safety and reducing accident rates by removing human error and the ever-increasing threat of distracted driving and distracted pedestrians. These systems paired with technology-enhanced public health, emergency response, and environmental systems will create Smart Cities of the future, where urban areas will be highly connected to improve overall mobility, efficiency, and safety.
To achieve this vision, city infrastructures will require multi-sensor solutions to ensure redundancy and effectiveness 24 hours a day, seven days a week, in any type of weather. The multi-sensing solutions must include thermal imaging, or long-wave infrared, which detects the heat energy emitted and reflected by everything on Earth. The ability for machines to discern heat, particularly for livings things such as pedestrians, bicyclists, pets, or wildlife crossing the road, will be crucial to the success of Smart Cities and driverless cars.
Furthermore, thermal imaging is a passive technology and can serve as a redundant and confirmatory system for existing sensors in the respective ADAS/AV and V2X sensor suites, working in concert with visible cameras and receiving signals such as LiDAR. For example, LiDAR can be problematic in cluttered urban environments where multiple LiDAR systems may be attempting to survey the same object.
Thermal imaging has already proven essential within the sensor suite for ADAS and AV because it provides an additional layer of detection where other sensors may fail, including within challenging lighting conditions such as nighttime, shadows, blinding sun glare, cluttered urban environments, and in inclement weather (i.e., rain, fog, and snow).
Most importantly, thermal imaging is very effectively paired with artificial intelligence and machine learning. CNNs (convolutional neural networks) can be trained via thermal imaging to recognize shapes, such as other cars, pedestrians or bicycles, to help create a more comprehensive and redundant system for identifying and classifying roadway objects. As thermal imaging detects a separate part of the electromagnetic spectrum versus visible cameras, it looks the same in day or night.
Diversifying sensor suites to meet new urban challenges
ADAS and AV cannot succeed in achieving the widespread societal impacts promised without also communicating with the surrounding environment and infrastructure. To truly bring this technology into the mainstream and integrate it into Smart Cities of the future, researchers and developers must consider how to leverage thermal imaging in our wider transportation management and infrastructure systems outside of the car. Using this additional band of the electromagnetic spectrum will further diversify the sensor suite and add an additional layer of perspective for Smart Cities to be effective in virtually all conditions, day or night.
To that end, the automotive industry, along with national, regional, and local governments, are increasingly focusing on connected mobility, which has prompted OEMs to incorporate and connect V2X communication systems into ADAS and AV systems. According to IHS Markit, more than 11.2 million light vehicles equipped with some form of V2X system will be produced globally in 2024, representing 12% of the light-vehicle fleet. Additionally, according to Grand View Research, the market value is expected to reach $26.72 billion by 2025.
Thermal V2X essential for Smart Cities
Thermal imaging combined with V2X technology can help bridge the gap in the existing sensor suite to provide the consistent connected mobility the industry strives for. Thermal V2X-enabled traffic sensors can communicate the thermal data to other V2X-enabled vehicles.
Thermal sensors allow traffic operators to factor pedestrian and bicycle movement into traffic control strategies. Thermal cameras, coupled with artificial intelligence, can distinguish between the heat signatures of bicyclists, pedestrians, and vehicles. This allows for pedestrian-in-crosswalk warnings in all weather conditions 24/7.
Connected vehicles can receive a safety message and will slow down or use warning signals and headup displays to draw the driver’s attention to the crossing, enhancing the awareness of the vehicle’s driver, which results in faster reaction times and reduced speeds when bicyclists or pedestrians are in the path. Traffic signals can then dynamically give bicyclists or pedestrians green time ahead of vehicle traffic, as needed. As a result, intersections and pedestrian crossings become safer and more efficient.
Many cities across the globe have already started to integrate thermal imaging as part of a Smart City infrastructure, primarily for roadway safety purposes. For example, the Arizona Department of Transportation in the U.S. has already installed thermal cameras pointed along a Phoenix-area freeway to spot and quickly alert other drivers and authorities of wrong-way drivers, particularly at night where visibility is greatly reduced. The system has already helped traffic operators to quickly identify cars entering freeways via exit ramps so they can quickly react and warn drivers ahead to help prevent deadly car accidents.
The future of thermal V2X
As demonstrated, there are many benefits to leveraging thermal V2X, including improvements in overall safety and efficiency. More specifically, a thermal V2X-enabled traffic sensor can also process the V2X messages generated by connected vehicles and help prioritize traffic signals for public transport and emergency vehicles, resulting in a rapid movement of those vehicles in an urban environment.
Although the future is promising for the development of Smart Cities and ADAS and AV systems, there remain challenges for researchers and developers to overcome, beyond just developing and deploying the appropriate technology. For example, governments will need to complete legal frameworks that can regulate these technologies, as well as manage and balance the sometimes competing needs of privacy and security.
However, the promise of Smart City potential is too great—considering the improvements in safety, efficiency, and overall quality of life it can provide. To achieve that vision, developers, engineers, and governments must consider thermal V2X as a crucial component to bringing that promise to life.