New interior monitoring system featuring cameras and artificial intelligence developed by Bosch
Bosch announced that it has developed a new interior monitoring system featuring cameras and artificial intelligence (AI) to monitor the road as well as the driver and other passengers.
“If the car knows what its driver and occupants are doing, driving will become safer and more convenient,” said Harald Kroeger, a member of the Robert Bosch GmbH board of management.
The Bosch system may go into production in 2022. In that year, the EU will make safety technology that warns drivers of drowsiness and distraction, for example, a standard feature in new vehicles. The EU Commission expects the new safety requirements for vehicles will save more than 25,000 lives and help prevent at least 140,000 severe injuries by 2038.
By tracking what is happening inside the car, technology aims to help solve a fundamental problem of self-driving cars. For example, if responsibility for driving is to be transferred to the driver following an automated drive, the car needs to be able to prove the driver is neither sleeping, nor reading the newspaper, nor writing emails on their smartphone.
A smart camera constantly monitors the driver
According to the company, if a driver dozes off or looks at a smartphone for just three seconds, a vehicle traveling at approximately 30 mph can cover about 130 ft completely unsupervised, and many people underestimate the associated risk. International studies state that nearly one in 10 accidents are caused by distraction or drowsiness, which has prompted Bosch to develop an interior monitoring system that detects and alerts to this danger and provides driving assistance. A camera integrated in the steering wheel detects when drivers’ eyelids are getting heavy, when they are distracted, and when they turn their head toward passenger or rear seats. Thanks to AI, Bosch states that the system draws the right conclusions from this information: it warns inattentive drivers, recommends a break if they are getting tired, or even reduces the speed of the vehicles—depending on the automaker’s wishes as well as legal requirements.
To achieve this, Bosch engineers have used intelligent image-processing algorithms and machine learning to teach the system to understand what the person in the driving seat is doing. For driver drowsiness, the system is trained using recordings of real driving situations, eyelid position and eye-blink rate to learn how tired the driver really is. This allows the system to give an alert that is appropriate to the situation, so driver assistance can be used to intervene.
Bosch states that warning systems that sound the alert, in the case of distraction and drowsiness, will be so important in the future that NCAP, the European New Car Assessment Program, will include them in the roadmap for its vehicle safety assessment by 2025. Further, only the software in the vehicle itself evaluates the information provided by the interior monitoring system—the information is neither saved nor shared with third parties.
Responsibility for steering passes from car to driver and back
When cars start driving automatically, it is important that they understand their drivers, as cars will drive along freeways without driver intervention. They will also have to hand back control to their drivers in tricky situations such as construction zones or when the exit ramp is drawing near. Drivers need the option to safely take the wheel again at any time during the automated driving phase. The camera makes sure they don’t fall asleep, and if their eyes remain closed for a prolonged period, an alarm is sounded. The system also interprets camera recordings to establish what drivers are currently doing and how ready they are to respond. The transfer of driving responsibility is then timed accordingly.
When the car keeps its camera eyes open
The new Bosch system keeps its eye not only on the driver, it also tracks all the other passengers, whether next to or behind the driver. For this purpose, a camera mounted above or below the rearview mirror monitors the entire passenger compartment. It notices when children in the rear seats have unfastened their seat belts and warns the driver. If someone sitting in the back is leaning too far forward, at an angle, or with their feet up on the seat next to them, the airbags and belt tensioner will not be able to protect them properly in an accident. The interior monitoring camera can tell what position they are sitting in and set the airbags and belt tensioner to ensure the best possible protection.
This technology also aims to aid in the safety of children in parked vehicles. In the U.S. in 2018, parked vehicles claimed the lives of more than 50 children (source: KidsAndCars.org), either because they had been left in the car for a short while or had clambered in unnoticed. The new Bosch system can reportedly recognize this danger and warn parents by sending a message to their smartphone. In an emergency, it also can alert the emergency services.
A camera for more convenience
The Bosch interior monitoring camera can recognize preset driver preferences, such as seat position, rearview mirror adjustment, steering wheel height and infotainment profiles, for increased driver convenience. The camera can be used for eye- and hand-gesture control of the infotainment system, as well.