Nio explains the strategy behind its dashboard robot
For centuries, travelers from around the world have placed small figurines on and about their vehicles. Good-luck charms, saints, and imaginary friends put passengers at ease—helping to ensure safe passage. For Americans, that might be the bobble-head of a favorite athlete. Chinese automaker Nio brought the same concept into the 21st Century with its Nomi dashboard robot.
“Bobble-heads can’t control your air-conditioning,” said AI specialist Tao Liang, Nio’s Engineering Director for Machine Learning. “We take the need for people to bond with their vehicle and add more on top of that.”
Liang explained that one of the core design concepts of the Nio ES8, the company’s flagship all-electric SUV, was to create a connection between car and driver. While Nomi, the cute rounded digital figure, is only about half the size of a baseball, the entire cabin experience is transformed by its presence. (The company would say, “her” presence.)
When the illusion of Nomi works as intended, the vehicle springs to life. But that seamlessness of the experience belies the hard work invested by Nio’s team of technologists and UX (user experience) specialists in designing the every detail of the robot’s form and interactions.
Nio wanted to turn the car into a living thing
“Our first inclination was to make it more [Amazon] Alexa-like with a light ring type of interface,” said Liang. But a glowing light pulsating in sync with a voice didn’t make the connection with Chinese drivers that Nio had in mind. “We wanted Nomi to be the soul of our car,” he said.
Nio models offer the “Nomi Mate,” with the face-like screen, as well as “Nomi Halo,” the simpler design just with a light.
Adding two graphical eyes immediately heightened the emotional connection with passengers. When that was combined with swiveling movements, the Nomi Mate established a “feeling of another living thing,” according to Liang.
For example, when you open the door, Nomi turns toward you, says hello, and blinks at you with good humor. The device was designed in Munich but had to be tailored to Chinese cultural standards. The face is slightly cartoonish. A representation depicted with too much realism would have been less cuddly and, well, somewhat creepy, as Liang acknowledged.
Whereas Western users might have a heightened sense of privacy, not wanting an unfamiliar visitor in their space, Chinese drivers welcome Nomi.
The process of making Nomi was more art than science.
“We used our intuition,” said Liang. “We created different kinds of nodding, different motion curves, and asked our customers to tell us which ones seemed most natural.” In other words, an overly mechanical type of movement would have been less organic and friendly.
Nomi looks simple and cute, but powerful computers and AI make it work
Nio had to overcome engineering obstacles to manufacture Nomi. “Having moving parts inside a vehicle is very challenging for productization,” explained Liang. The team went through 40 different motors, and nearly as many metal and plastic gears, before finding the right combination for fluid movement. The motors needed speed and precision to achieve a person-like motion.
At the same time, the device had to be automotive grade, able to withstand vibrations and a wide range of temperatures. “In certain parts of China, it’s very cold, and the motor can lock up,” said Liang. “Nomi might make a face and use its voice, but not turn your way.” That would be enough to destroy the illusion of a living thing. The same problem occurs if the gearing is too loud, especially considering the hush-quiet of an electric vehicle’s cabin.
Nomi adds an element of sentience based on the AI guiding its actions. Using fog computing, some interactions and natural language protocols are the result of training via remote servers. But so-called inference, the ability to promptly respond in the cabin, happens via powerful processors packaged in the vehicle’s center console. An overreliance on the cloud would be problematic for stretches of the roadway when the car doesn’t have a fast enough connection to remote servers.
Engineers limited Nomi’s functionality to emotion-inducing interactions
Critically, Nio’s roboticists limited Nomi’s functionality to the movements and interactions that produced the most lifelike impressions. For example, when rain is expected, Nomi carries an umbrella.
However, the team resisted the temptation to solve for artificial general intelligence, the theoretical capacity for a machine to understand or learn any intellectual task that a human being can.
It was decided that pressing a button, for such tasks as choosing a song, was more natural than too much back-and-forth decision-making with Nomi. Hard-to-remember hand gestures were avoided, despite both rows being in view of cabin-based cameras.
Users can personalize Nomi’s name, which also serves as its command trigger. But Nio puts limits on that personalization to avoid the risk of breaking tried-and-true interactions.
Nomi is quite capable of suggesting a destination based on past driving patterns, encouraging a driver to keep his or her hands on the wheel, or turning on the A/C with a simple voice command. The focus was on the simplest, yet seemingly most intelligent tasks, rather than “over-engineering the whole thing,” according to Liang.
“We believe that Nomi is enhancing the emotional bond with users,” said Liang. “So, we’re sticking with it for all the future vehicles.”