The ABCs of teleoperation
Ottopia Technologies’ CEO and Founder Amit Rosenzweig has developed this guide to the new terms that will define our autonomous future.
The new age of mobility is coming. There’s much talk in the industry and the popular and trade press about the various commercial and technology trials of autonomous vehicles (AVs). While some writers mourn the end of driving as we know it; others look forward to a world free of traffic accidents.
While all agree that this is the future, the industry consensus is that full autonomy is still many years away. The mundane task of driving a vehicle, especially in urban settings, presents some insurmountable challenges for AV technology. The hardest nut to crack is edge cases; road conditions and situations that challenge what the AV is designed to tackle: road work, accidents, difficult weather conditions, police guiding traffic, unusual obstacles on the road, etc.
Today’s AV technology providers are well aware of the challenges ahead. The CEO of Argo AI, Bryan Salesky, put it well when pressed by the New York Times (https://nyti.ms/2JCvwSY) recently: “You see all kinds of crazy things on the road, and it turns out they’re not all that infrequent, but you have to be able to handle all of them,” he said, referring to edge cases. He continued to explain that 80% of the technology for AV integration already exists, but the 20% needed to anticipate decisions in unpredictable environments is needed before the technology can fully take the wheel. That’s where teleoperation comes into play.
Right now, when an AV faces a challenging situation, it is programmed to stop and ask for human intervention. This intervention can be provided either by a safety driver sitting in the vehicle or by a remote operator using a teleoperation solution to remotely control the vehicle. Since placing a safety driver in each AV undermines the economic viability of any real commercial deployment, teleoperation has become the essential and safe enabler of the new mobility for the foreseeable future. This is also reflected in a rise in the legality of teleoperation. So far sixteen U.S. states and countries around the world have legalized teleoperation.
Despite teleoperation’s essential role as an enabler of the future of mobility in general and AVs in particular, there is a lack of clear definition to define the most important aspects of this burgeoning industry. It’s time for us to take a few minutes to discuss the ABCs of teleoperation and outline key terms that will pave the way for the sector’s continued growth.
Here’s the first-ever (non-comprehensive) alphabetical list of teleoperation terms and their definitions:
ATAS (Advanced Teleoperator Assistance Systems): A teleoperation specific adaptation of the automotive industry’s ADAS. ATAS includes vehicle side collision detection and collision avoidance capabilities to ensure safety during direct control of the vehicle, even when the network performance is especially bad. ATAS is a registered trademark of Ottopia Technologies Ltd.
ATO (Advanced Teleoperation): A platform that offers both direct and indirect methods of vehicle remote control which is safe and cyber-secure. ATO is a registered trademark of Ottopia Technologies Ltd.
Cybersecurity for Teleoperation: A suite of cybersecurity measures that are designed and tested to ensure teleoperation platforms do not add an “attack surface” (read — risk of being hacked) to the vehicle. Teleoperation relies on connectivity and if cybersecurity features are not built into the solution from the first lines of code, it increases the risk that such a solution will jeopardize the entire fleet.
Data Gateway: A vehicle side hardware and software module that enables connecting to and using the vehicle’s drive-by-wire system or a retrofitted actuation system.
Direct Control: A teleoperation method of control that allows an operator to drive the vehicle as if she was in the driver’s seat.
Edge Cases: Road conditions that exceed what the Autonomous Vehicle (AV) is designed or able to tackle, for example road work, accidents, very bad weather conditions, a police officer guiding traffic, an unusual obstacle on the road, etc.
FOV (field of view): The observable area a person can see through her eyes or via an optical device. In teleoperation it’s the operator’s field of view as seen through screens of the teleoperation station.
Handoff: The transfer of control from the AV to the teleoperator, following a request for teleoperator assistance generated by the AV (a trigger). Or, an active action performed by the teleoperator to safely transfer control of the vehicle back to the AV.
Human Intervention: An active action taken by the teleoperator to control the AV either via a direct or indirect method of control.
Indirect Control: An advanced teleoperation method that allows the teleoperator to instruct the AV on how to overcome an obstacle, instead of actually driving it. Using this method, the teleoperator utilizes software tools such as Path-Choice or Path-Drawing. Indirect methods of control are safer, more cyber-secure, more reliable and allow for better scalability than any direct method of control.
ISO 26262: An international standard for functional safety of electrical and/or electronic systems in production automobiles defined by the International Organization for Standardization. Its goal is a unifying safety standard for all automotive E/E (Electrical/Electronic) systems. Any safe teleoperation system must adhere to the industry’s defined standards.
Latency: The amount of time a message takes to traverse a system. Two-way minimal latency, between the vehicle and the teleoperation station, and back, is required to ensure that teleoperators can safely guide the vehicle from remote locations.
Network Bonding: A technology that aggregates data streams from several cellular and/or WiFi modems into one reliable, high-bandwidth and low-latency link. It is considered an essential module required for safe teleoperation.
Network Survey: Prior to commencing teleoperation services in any given operational design domain (ODD), the network quality, coverage and latency in the ODD must be surveyed.
Operational Design Domain (ODD): California’s DMV defines it as “… the specific operating domain(s) in which an automated function or system is designed to properly operate, including but not limited to geographic area, roadway type, speed range, environmental conditions (weather, daytime/nighttime, etc.), and other domain constraints.”
Path-Choice and Path-Drawing: User interfaces and software tools that enable humans to instruct AVs on a preferable path to overcome an obstacle, which then the AV executes autonomously. Path-Choice and Path-Drawing are examples of indirect methods of control.
Retrofit: An aftermarket installation of system/s on a vehicle, such as a teleoperation kit of hardware and software. Retrofit systems can enable a set of capabilities even if said vehicle wasn’t originally manufactured with those capabilities in mind.
Staffing Calculator: A software solution used to predict and manage the number of teleoperators needed on a weekly, daily and hourly basis inside a teleoperation center to meet demand for teleoperation.
Teleoperation (TO): The ability to remotely control vehicles, autonomous or not.
Teleoperation Center: An installation, usually in an office setting, that houses teleoperators and one or more teleoperation stations, used for conducting teleoperation.
Teleoperation Service Provider: A company that provides teleoperation services.
Teleoperation Station: A hardware and software solution used by a teleoperator in a teleoperation center to conduct teleoperation missions.
Teleoperator/Remote Operator: The State of California DMV defines a remote operator as “… a natural person who: possesses the proper class of license for the type of test vehicle being operated; is not seated in the driver’s seat of the vehicle; engages and monitors the autonomous vehicle; is able to communicate with occupants in the vehicle through a communication link. A remote operator may also have the ability to perform the dynamic driving task for the vehicle or cause the vehicle to achieve a minimal risk condition.” See https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/definitions.
A broader definition would omit the word “autonomous,” i.e. a teleoperator in essence can control any type of vehicle, not just an autonomous one.
Trigger: A request for human intervention initiated by the AV.
Video Transport: Smart, end-to-end, video streaming and compression technology, especially when working in sync with network bonding technology, guarantees the best utilization of the available bandwidth and minimal latency to ensure safe teleoperation. It is an essential software and hardware module required for safe teleoperation.
And with that, we conclude our list of teleoperation terms and their definitions. As AV technology evolves and teleoperation expands, we hope these definitions guide meaningful and clear discussions across both business and regulatory activities. We’re ready for the future of transportation and can’t wait to hear these terms used more and more in the months and years ahead.