For more than two years, Tesla CEO Elon Musk has teased the development of Tesla’s supercomputer called “Dojo.” Last year, he even claimed that Tesla’s Dojo would have a capacity of more than one exaflop, or one quintillion (1,018) of floating-point operations per second.
Tesla first unveiled this Dojo supercomputer on AI Day even though Elon Musk has been talking about it on Twitter for nearly a year now. Tesla claimed that it was the fastest computer in the world for training ML algorithms.
Register for Free AI Summit
This chip was created from scratch internally at Tesla. It is primarily essential for computer vision for self-driving cars using cameras. Tesla collected over a million vehicles to train the neural network using the Dojo supercomputer.
Specifications and claims:
Dojo is powered by Tesla’s custom chip, D1, which packs 7nm technology and 362 teraflops of processing power. Tesla claims its D1 Dojo chip has GPU-level computation and processor-level flexibility, with an IO network switch.
Chips can connect seamlessly without any glue to each other. Tesla took advantage of this by connecting 500,000 nodes. The result is a nine pFLOPS training tile with 36TB per second of bandwidth in a format of less than one cubic foot. Tesla did the above by going against the general industry standard of cutting the wafer into pieces. It leaves 25 SoCs on the wafer and uses high quality silicon. These will allow the chips to communicate with each other without losing speed while maintaining the quality of the motherboard.
Tesla only needs 120 fully functional pads for Dojo. For comparison, in 2014, Intel manufactured over 130,000 300mm wafers. But, since Dojo uses small five-by-five slices, the costs should be significantly lower.
The company also claims that the computer has no RAM outside of the SoC. To illustrate, even a smartphone, the latest hard drives and Tesla’s HW3 have RAM chips outside of the SoC. Tesla uses cache instead, a faster level of RAM.
Dojo versus competitors
Chipmaker Intel, graphics card maker Nvidia, and startup Graphcore are among the companies that make chips used to train AI models.
Download our mobile app
The Intel Mobileye Q4 chip is capable of performing 2.5 peaks while consuming 3 watts of power. The chip is structured to support a fully autonomous driving system. Each EyeQ chip consists of heterogeneous and fully programmable accelerators optimized for its own family of algorithms. The chip has generic multithreaded processor cores, making it a robust computing platform for ADAS / AV applications. It supports 40 Gbps data bandwidth and has the added capability of supporting more sensors through PCIe and Gigabit Ethernet ports.
Until now, Tesla has used the Nvidia Drive PX2 chip to implement its autopilot and speed up production of autonomous vehicles. The chip can understand the vehicle’s surroundings in real time, locate itself on a map, and use the information to plan a safe route. Nvidia claimed that it is the world’s most advanced self-driving car platform that combines deep learning with sensor fusion and surrounding vision. The chip configuration consists of a mobile processor that can run at 10 watts and is converted to a multi-chip configuration with two mobile processors and two discrete GPUs. GPUs can deliver 24 trillion deep learning operations per second.
Another Driver series chip, PX Xavier, consumes just 20 watts of power while delivering 20 TOPS of performance. It is packed with 7 billion transistors. Nvidia Drive Pegasus uses the power of two Xavier SoCs and Nvidia’s Turing architecture to provide 320 TOPS capacity while consuming 500 watts. Nvidia designs the platform for Tier 4 and Tier 5 stand-alone systems.
However, among graphics cards, NVIDIA’s GA100 Ampere SoC still leads the way with 54 billion transistors.
Living up to the hype
Tesla has a history of coming up with brilliant concepts that never materialized. So many of the products Musk showcased at events were barely marketed, not yet available, or have been phased out at the ideation level. These include products like the Model X in 2012, to newer products like Tesla Energy or the solar roof. The company has also not released an official research paper on the supercomputer.
Tesla enthusiasts took to Twitter to express their mixed opinions on the Dojo chip. While some question Tesla’s claims with “Designing a chip is easy”. What is difficult is building the compiler, the runtime scheduler in a large-scale HPC environment ”or“ Dojo is the CPAP fan of machine learning runtimes ”, others think Tesla is giving it away. its money to competitors.
Tesla claims the chip can achieve a 10-fold improvement in the next iteration of the computing device. When it comes to Dojo, it’s critical to remember that only Tesla’s claims and design will be compared to existing and running models from Nvidia, Intel, and Graphcore. Tesla hasn’t assembled the entire system, but Musk believes it will be fully operational next year.
Join our Discord server. Be part of an engaging online community. Join here.
Subscribe to our newsletter
Receive the latest updates and relevant offers by sharing your email.