Tesla: Hardware 3, AI Chip and 1 Billion Autopilot Miles

Tesla CEO Elon Musk announced in October that Tesla’s custom AI chip is six months away from being launched. According to Musk, the new chip will increase Tesla’s autonomous driving performance between 500% to 2000%. 

Tesla’s current hardware with NVIDIA chips can process 200 frames per second, while the new Tesla chip is expected to handle 2,000 frames per second with full redundancy and fail-over.

“The performance claims are against what they have in the vehicle today, which are three years old,” says NVIDIA’s director of automotive Danny Shapiro. NVIDIA’s latest silicon is at least 10 times faster than that, which would put it on a par with Tesla’s chip. – Wired

To match NVIDIA’s hardware performance after two years of development, does show the progress Tesla has made after parting ways from its ex-partner Mobileye in 2016.

Tesla’s new hardware will cost the same as the old hardware. 

The unending delays show how difficult self-driving tech is:

As per this Fortune report, Elon Musk said in December 2015 that Tesla will start building fully autonomous vehicles in two years.

According to the original plan, Tesla wanted to do a coast to coast autonomous run, from LA to New York by the end of 2017.  The idea was great. Elon Musk expected the trip to be “from home in LA, to dropping you off in Times Square, and then the car will go park itself.”

It would have been a great demonstration of Tesla’s self-driving capability but the company has kept pushing the target date further down the road.

“We could have done the coast-to-coast drive, but it would have required too much specialized code to effectively game it or make it somewhat brittle and that it would work for one particular route, but not the general solution. So I think we would be able to repeat it, but if it’s just not any other route, which is not really a true solution,” said Elon Musk

Simply put, Tesla Autopilot is not yet ready for a prime-time drive from the west coast to the east coast.

Google’s self-driving spin-off Waymo was planning to launch a commercial self-driving taxi service by the end of 2018, but conflicting reports have emerged about the timing of the launch.

According to this report, “Waymo’s autonomous vehicles (AVs) are experiencing a number of technical difficulties that could delay the expected launch of its driverless taxi fleet this year”. But Bloomberg reported this month that Waymo will soon launch its commercial taxi service. With just four more weeks left, we still don’t know if 2018 will see a taxi driving on its own.

Tesla’s Current Hardware: The LiDAR Problem

Automakers are racing each other to develop the world’s first autonomous driving vehicle. If successful, autonomous vehicles will completely change global transportation as we know it. An on-demand point to point robot-taxi service will obliterate trains and even completely replace short distance air travel.

Major auto companies like Nissan, Mercedes Benz, GM, Tesla and many more have already thrown their hat in the ring to develop autonomous driving tech. But unlike other companies, Tesla decided early on that it will not include Lidar as part of its autonomous hardware.

LIDAR, which stands for Light Detection and Ranging, is a remote sensing method that uses light in the form of a pulsed laser to measure ranges (variable distances) to the Earth. A LIDAR instrument principally consists of a laser, a scanner, and a specialized GPS receiver. – National Ocean Service

Uber Self-Driving Vehicle with Lidar

Tesla’s decision may spectacularly backfire but the company has remained steadfast in its decision that it can deliver level 5 autonomy with a hardware suite of 8 cameras, 12 ultrasonic sensors, and a forward facing radar.

But Google’s self-driving Spin-off Waymo thinks differently.

“While driving-assist camera systems used in cars are quickly becoming popular, they have a few critical limitations. Similar to the human eye, cameras have a hard time seeing in difficult lighting conditions,” said Waymo CEO John Krafcik in his keynote address at NAIAS’s inaugural Automobili-D conference in Detroit.

Waymo:LiDARs, vision system, and radars

Waymo’s self-driving hardware suite includes three different types of LiDAR sensors, five radar sensors, and eight cameras.

While Waymo relies heavily on Lidar, Tesla relies heavily on its cameras. Lidar is expensive. It’s so expensive that you can buy a couple of cars for the price of Lidar. But it also offers plenty of advantages. Lidar uses laser light to identify its surroundings. Since Lidar is it’s own source of light, it works well in the dark and during the day.

LiDar is expensive now, but it won’t remain at the same level when auto companies start building them in the millions. Sure, it will take time for the price to drop, but it will happen over time.

Tesla is taking a huge risk with its non-lidar approach, but the payoff would be significant if successful. The cheapest self-driving tech available to the mass market. If it fails, there is always the LiDAR bandwagon to join.

Hardware 3: What we know about it

Tesla’s hardware 3 is all about the processing power it brings to the table. Hardware 3 is expected to offer a tenfold increase in processing capability compared to current hardware with improved redundancy.

Tesla is packing so much computing firepower in its cars because that’s the only way to make self-driving tech work. For a car to be truly autonomous, the onboard computer should have the ability to process the humungous amount of data generated every second, on its own.

NVIDIA DRIVE AGX Pegasus™ achieves an unprecedented 320 TOPS of deep learning with an architecture built on two NVIDIA® Xavier™ processors and two next-generation TensorCore GPUs

Nvidia’s new Drive PX Pegasus board, designed for self-driving vehicles, offers 320 trillion operations per second, which is powerful enough to support level 5 autonomy.

According to NVIDIA “this energy-efficient, high-performance AI computer runs an array of deep neural networks simultaneously and is designed to safely handle highly automated and fully autonomous driving. No steering wheel or pedals required.”

As Alan Kay once said,

“People who are really serious about software should make their own hardware”. 

Tesla has taken those words seriously as the company now designs and builds everything in-house.“We had the benefit of knowing what our neural networks look like, and what they’ll look like in the future,” said Pete Bannon, director of the Hardware 3 project.

By designing its chip, Tesla can perfect the relationship between its hardware and software, similar to what Apple does.

Steve Jobs long believed Apple should own the technologies inside its products rather than rely on mashups of components from other chip makers, including Samsung, Intel and Imagination Technologies. In 2008, the company made a small but significant step in that direction by acquiring boutique chip maker P.A. Semi. – Bloomberg

Tesla currently uses NVIDIA’s Drive PX 2 platform. NVIDIA’s newly launched Pegasus platform offers a ten-fold performance boost over the original PX 2. If Tesla has indeed achieved a ten-fold improvement in performance with its custom AI chip, then it puts them on par with the industry’s leading self-driving chip from NVIDIA.

When Tesla released hardware 2, the company thought it had enough processing power to achieve full self-driving capability. But it took some time to realize that computing power wasn’t enough. That’s why Tesla decided to build a chip on its own and wait for hardware 3 before demonstrating Autopilot’s ability to drive from the west coast to the east coast.

Tesla has realized that edge computing, (strong computing power closer to the point of data generation – in Tesla’s case it will be the vehicle), is crucial in achieving fully autonomous self-driving capabilities.

And if you are a Tesla owner who paid for full-self driving capability, nothing to worry – Tesla will upgrade to hardware 3 free of charge.


In his second “master plan” for Tesla, published in July 2016, Musk wrote ” We expect that worldwide regulatory approval will require something on the order of 6 billion miles.

Tesla announced today that “Tesla owners have driven 1 billion (!) miles with Autopilot engaged”. It’s still a long way to 6 billion miles, but with Tesla adding 6,000 to 7,000 cars to its fleet every week, the company will quickly accumulate the miles.

But the odds of seeing an autonomous car driving from anywhere to anywhere is not going to happen in a hurry.

Tesla’s Autopilot is impressive, their non-Lidar approach is a risky bet, the company is gobbling up millions of Autopilot miles every day and hardware 3 may be a game changer. But even now, Tesla’s autopilot and Waymo’s self-driving tech are far from being fully autonomous and it’s going to be a while before we get a genuinely autonomous system.

For now. Let’s wait and please don’t take your hand off the wheel.