The video amazes everyone who sees it.
A young man drives his Lexus through Manhattan, hurrying to a meeting. When he reaches his destination, he pulls over, calls a service center, asks the service representative to take his car to the nearest parking lot, gets out of the car, and heads to his meeting. The car drives on to the parking lot, a few blocks away, controlled remotely by the service representative who sits in a faraway office equipped with a steering wheel, gear shift and panoramic screens showing the road in real-time.
The Lexus naturally attracts astonished looks from pedestrians, who see a car navigating Manhattan's crowded streets without a driver. When the man finishes his errands, he calls the center and asks the driver to get the car back to him. Although this is a promotional video, the technology shown is entirely real and also Israeli.
In the spring of 2018, when autonomous vehicles, which drive themselves without a driver, seemed like the next big thing in technology, the extended Rosenzweig family gathered at their parents' home in Zichron Ya'akov for the Passover Seder. Brothers Oren and Amit Rosenzweig are prominent figures in the Israeli hi-tech scene, particularly in the smart automotive technologies sector.
Oren, 43, was a co-founder of Innoviz, a trailblazing manufacturer of LiDAR, laser sensors made for autonomous vehicles; his brother Amit, 38, served 12 years in IDF technological units, where he developed algorithms for the Iron Dome and Arrow missile defense systems and held a command role in cybersecurity in Unit 8200. During his service, he led a project that won the Israel Defense Prize.
Upon his release from the military, after a brief stint at Screenovate (which was sold to Intel) and two years at Microsoft, Amit searched for an interesting technological niche to establish a startup in fields he specialized in.
I asked Oren what was missing in the autonomous car field, where there are problems that still need to be solved. "Look, everyone talks about the wonder of a car that drives itself, but less so about safety," he replied. "If you put your child in such a car today, it's not certain they would reach their destination. It’s clear the field will grow and develop, but it’s also clear there will be a need for some human involvement."
Revolution takes time
Oren Rosenzweig knew what the industry at the time managed to hide: as smart as they may be, autonomous cars need humans who can operate them during malfunctions or confusion in the car’s artificial intelligence (AI) systems.
In many countries, this is even an explicit requirement by regulators and the law. The idea that took shape at the Rosenzweig family home was a system that allows human operators – trained drivers – to drive the car despite being far away from it.
It was then the startup company Ottopia was established, which currently has 35 employees in Tel Aviv, most of whom are software engineers in video, communications, networks, security and more. Its goal, says the CEO Amit Rosenzweig, was to offer autonomous car manufacturers complementary technology that "fills in the gaps" in case of car malfunctions.
Ottopia operated in the field for four years and, in fact, a significant part of its customers – companies specializing mainly in mobile robots for deliveries like Robotrack, Last-Mile Delivery, and more – were made during those years.
All the promising forecasts were dashed. It’s clear today that private autonomous cars roaming the centers of big cities are likely a vision that will only be realized in a decade or two. Even the autonomous taxis that are already operating are far from capturing a significant share of the global public transportation market.
Since 2010, over $100 billion has been invested in developing autonomous vehicles worldwide. But 20 years after the first experiments and eight years after they began to roam the road, such vehicles still operate in a limited format in a very small number of cities.
The accumulated experience shows they still struggle to deal with various challenges, especially in cities, considered particularly chaotic environments: severe weather conditions, construction sites, animals walking on roads, unexpected extreme events and even making basic turns.
Lawmakers around the world are now more cautious than before in approving autonomous vehicles to drive in urban environments. In contrast, vehicles intended for highways – autonomous trucks, for example – are less exposed to extreme situations, and manufacturers are now focusing on them.
The automotive industry outwardly continues to show optimism. Tesla trained its autonomous vehicles on over five billion kilometers of roads. In 2021, Professor Amnon Shashua of Israel's Mobileye promised autonomous taxis in Tel Aviv "by next year." But in practice, the market leader, Waymo, owned by Google-Alphabet, has lost about 80% of its value in the last five years, and other companies in the field, such as Drive.AI, Voyage and Zoox, are struggling.
Fortunately for Mobileye, the technologies it developed not only aim to fully replace the human driver in the future but also to assist them today. The revolution will likely come, but slower than we were led to believe.
Four years after its founding, modest revenues and slow development caused Ottopia to recalculate its course.
"Even if you take all types of autonomous vehicles operating today into account, you can only see a few thousands operating," Amit Rosenzweig said. "I realized the change would take much longer than even the most pessimistic forecasts of the most skeptical forecasters, and that we couldn’t make money from it, and I have dozens of employees to support."
The company’s conclusion was to change its purpose: if initially, Ottopia's system was designed as a backup in case of emergency for a malfunctioning car, it will stand on its own from now on. "We developed the best technology in the world here that allows humans to control cars remotely. We were basically the second fiddle to the autonomous car, now it's time to play the first fiddle."
The rich come first
Last week, Ottopia announced its final product for the first time at the prestigious Startup Autobahn automotive conference in Germany, which is now aimed at regular cars – a "remote driver" service.
There’s a caveat, however. The service will initially be offered to owners of expensive premium cars, those whose time equals high amounts of money, or to rental companies that can charge extra for pampered tourists.
According to Paul Kandel, Ottopia's business development manager, the payment will be rated according to a private driver’s standard rate – between $30 to $50 per hour. "This is how new technologies are integrated into the automotive world. Always,” Rosenzweig said.
“Even Innoviz’s LiDAR didn’t start with a Suzuki Swift but with a BMW 7 Series. Every auto-tech company you see in the world starts with premium products because that's where the early adopters are. Later, it trickles down: the Koreans watch what Mercedes and BMW do, and do the same after two years,” he added.
You're essentially offering a kind of call center. Just like operators, drivers will sit side by side in an office building, taking control of cars for short tasks on demand.
"Exactly. Car manufacturers, for instance, can sell this as a subscription service, charging money per ride. It’s even possible we’ll be the ones operating it for them. There’s a demand for this, and we are considering it."
You’ll operate drivers from Tel Aviv driving cars in San Francisco, like service centers in India providing support to washing machine buyers in London?
"In principle, it's possible but not advisable due to the low quality of communication over such distances. Currently, we achieve safe driving up to 50 km/h, and the distance will inevitably reduce driving speed. It might also be impossible due to foreign regulatory rules."
Implementing the idea turns out to be very technologically demanding. The main challenge is the need for perfect cellular communication between the car and the driver's station, with (almost) zero latency: every steering movement or brake press remotely must be translated in real-time, without delays, into immediate responses within the cat.
Ottopia's system relies on simultaneous reception and transmission over two cellular networks, cleverly managed with two different modems installed in the car. Can cellular networks, especially the widespread 4G, provide the low latency and high reliability needed?
Rosenzweig is certain it can: "With the technology we’ve developed, we achieve an incredible delay of only one-tenth of a second from the moment a specific element is captured by the car’s camera until it appears on the remote driver’s screen."
Ottopia has already registered nine patents for the key elements of their system, most in the field of data analysis and prediction in cellular networks, with another 20 patents pending approval.
But cellular networks don’t provide perfect coverage everywhere today. If there's no cellular reception, you’re stuck. For example, while driving in a tunnel.
"We have a learning technology that continuously gathers data from cars on the road and updates its knowledge base. If the route requested for your task isn’t covered by cellular reception, the system will notify you in advance and won’t enable the service, or offer an alternative route. As for tunnels — some cellular companies' networks today have amplifiers; we were surprised to find the reception in them is sometimes even better."
In order to offer a safe service, you must have the most reliable and up-to-date information on the cellular networks' reception and transmission status everywhere. Your success depends on these data repositories.
"Correct. Unfortunately, we don’t have such information for the entire globe. And this is precisely why we work with the major cellular providers, because they provide us with crucial data on cellular coverage quality in the areas they operate in. But look, Mercedes is the first company in the world to receive regulatory approval and launch a 'Level 3' autonomous car – which means the driver can read a newspaper while driving. They launched it with much fanfare in California, Nevada, and Germany – on very specific roads.
“However, once one of the manufacturers announces the integration of our product, it will be available on a much wider scale in two years. The mapping we have access to today is much broader than one specific highway in California; it spans tens of thousands of square kilometers, including thousands of streets and routes. So, if we have another two years – we’ll be talking about hundreds of thousands of roads and streets."
This could happen faster if cellular providers disclosed these data to you. But why would they?
"They have an interest. I managed to meet with CEOs of some of the largest cellular providers in the world on this matter. Why would AT&T’s CEO decide to dedicate half an hour of his precious time to the CEO of a startup from Tel Aviv? Because he understands remote driving will bring a lot of cellular data consumption – mainly via video – which will generate new revenue for the companies."
A $60 million-$70 million project
The company’s remote driver station resembles the driver's seat in any car, including the steering wheel, pedals, turn signal switches and gear shift. However, instead of a windshield, three closely mounted computer screens are put together that provide the panoramic view seen by the driver on the road.
The mirrors – rear and side view – are represented by small windows within each screen. Part of the challenge faced by the company’s programmers was eliminating camera distortions created by standard vehicle cameras to provide the driver with as realistic a view as possible, as if they were sitting inside the car.
Moreover, AI technology developed at Ottopia can enhance and sharpen the video footage received from the car's cameras in real time. The driver hears the car’s noises through headphones.
The driver's user interface is entirely intuitive, including a GPS map and a dashboard displaying all the real car’s heads-up display, as well as performance indicators – affected by a drop in cellular reception quality or a problem with the car's systems. Control over the car is encrypted, requires the owner's approval, and involves a secure identity verification process. The owner can track the car’s status and location at any stage via a mobile application.
While a driverless autonomous car requires expensive and numerous computers, sensors and radars, some of which are placed in a bulky dome on its roof, Ottopia's system makes do with what’s already installed on most luxury cars' production lines, which typically include 12 different cameras, computing units and internet connections.
The required modifications are minimal – mainly requiring an additional modem – but must be integrated into the car during its production, necessitating close cooperation between Ottopia and major luxury car manufacturers.
Ottopia, considered to be a global leader in the field, currently works with international car manufacturers such as BMW and Hyundai, mainly in the pre-implementation of remote driving capabilities in new cars and trucks they produce, and provides them with consulting on developing standards and regulations being decided on the subject. Incidentally, its main competitor, Germany's Vay, focuses primarily on harnessing its systems for car rental services.
I'm guessing it's easy to translate this technology for military use. It can save lives. The Israel Aerospace Industries (IAI) recently unveiled an unmanned D9 bulldozer operating in Gaza being controlled remotely.
"That’s right, there are applications for this technology in the military field, and we already have defense clients, but I can’t elaborate on that."
Would it be possible to remotely fly fighter jets in the future using your technologies?
"Yes, definitely. Ships as well. As long as a communication medium exists – be it via satellite or cellular networks – it makes sense to utilize technologies like ours that can reduce bandwidth and latency. Is it economically justifiable? For now, probably not."
Ottopia is now essentially waiting for developments in the market. Your future depends on that.
"We’re on the brink of a massive push for this market, based on indications we have from car manufacturers’ managers seriously considering incorporating our system into mass production. Once such a decision is made – it’ll become a $60 million-$70 million project. One project, that's all we need."