Tesla's "Dojo" supercomputer will train its vision-centric autonomous driving technology

Tesla has gone all out to carry out vision-only autonomous driving, and even phase out radar sensors in some electric vehicles. Now at the CVPR 2021 seminar, Andrej Karpathy, Tesla's senior director of artificial intelligence, explained how it plans to do this by using an internal supercomputer called "Dojo", as reported by TechCrunch.

Karparthy explained that with vision-only tech, computers must respond to new environments with the same speed and acuity as a human. However, doing that requires AI training on a massive dataset with a powerful supercomputer to crunch it. Tesla has one of those in house with "Dojo," a next-gen model with 1.8 exaflops of performance and 10 petabytes of NVME storage running at 1.6 terabytes per second.

While the system hasn't been benchmarked, Karparthy figures it would be one of the fastest in the world. "If you take the total number of FLOPS it would indeed place somewhere around the fifth spot,” Karpathy told TechCrunch. “The fifth spot is currently occupied by NVIDIA with their Selene cluster, which has a very comparable architecture and similar number of GPUs."

To train the system, Tesla's supercomputer collects video from eight cameras on Tesla vehicles each running at 36 frames per second. While that generates a huge amount of data, it's more scalable than building and maintaining high-definition maps around the world. However, it also requires nearly instantaneous processing, which needs to be treated as a supervised learning problem.

So far, the system works well in sparsely populated areas, where cars can drive around with no intervention. However, Tesla has found (like all other autonomous vehicle companies) that navigating densely populated areas is much more difficult. Still, Karpathy said that Tesla's computer has been able to handle new types of traffic warnings, pedestrian collision detections and pedal misapplications, the latter happening when a driver accidentally presses the gas instead of the brakes.

Although Tesla has had several notorious accidents, the autonomous driving system failed to find obstacles or track the route correctly, CEO Elon Musk is firmly committed to only vision. "When radar and vision are inconsistent, which one do you believe? Vision has higher accuracy, so it doubles vision better than sensor fusion," he recently tweeted. The company believes that supercomputers will eventually help vehicles gain advanced autonomous driving capabilities, but it is best to take a wait-and-see attitude because we have heard this before.