Tesla is building a custom $1 billion A.I. supercomputer because it cannot get enough Nvidia chips

Elon Musk would not agree. The Tesla CEO is willing to invest well over $1 billion to build an A.I. supercomputer he dubbed Dojo just because he can’t get his hands on enough of Nvidia CEO Jensen Huang’s advanced A100 tensor core GPU clusters.

The EV visionary needs them to train both his “Full Self-Driving” (FSD) software currently sold for $15,000, as well as his Optimus humanoid robots under development to perform useful everyday tasks.  

“Frankly if they could deliver us enough GPUs, we might not need Dojo,” Musk told analysts on Wednesday during Tesla’s Q2 earnings call. “But they can’t, because they have got so many customers.” 

In October, Musk said he wasn’t even sure his latest project would prove itself superior to buying Nvidia chips off the shelf—unusual given he typically demands the best from his team. Instead he was full of praise of Huang and thanked the company for prioritize some of its orders. 

“We’ll actually take Nvidia hardware as fast as Nvidia will deliver it to us,” he said yesterday.

Tesla has a voracious demand for A.I. chips that seemingly outstrips virtually any commercial competitor. It aims to reach an in-house compute capability of 100 exaFLOPS (an extremely high level of supercomputer performance) by the end of next year, a truly staggering figure that would ensconce it easily among the global top five providers known currently, according to the company.

Autonomy to drive car sales ‘through the ceiling’

Musk is in a hurry since he has little intention to be proven wrong yet again with his annual ritual of predicting he would soon achieve full autonomy. (On Wednesday he jokingly called himself “the boy who cried FSD”.)

This increasingly vague term he throws around that can mean everything from a Tesla merely navigating traffic better than an average driver to actually driving on its own without any human supervision at all. 

The latter he predicts would usher in Tesla’s own “ChatGPT moment”, and create in his words on Wednesday the “single biggest step chance in asset value maybe in history” as overnight his cars instantly become robotaxis. That’s why Musk told analysts during the call he is so willing to sacrifice profitability for volume at a time when incumbent rivals like Volkswagen and Mercedes-Benz are taking the exact opposite approach.

“If we had more training compute, we’d get it done faster,” he said on Wednesday. For now, FSD is merely a driver assist system that can make mistakes at the worst time.

When it comes to solving the problem, Musk has adopted a fundamentally different approach to the entire industry, one that necessitates heavy spending on computing power. 

Instead of a sinfully expensive suite of multiple redundant sensor systems like that used by Waymo, Cruise, Mobileye and their partners and rolled out in carefully mapped geo-fenced areas, Musk wants his cars to be so intelligent they can drive anywhere autonomously simply by relying on camera data. To train them he is using 300 millions miles and counting of real-world driving data that he scrapes every day from his customers’ cars.

That data will now be fed to Dojo, which was designed to significantly reduce the cost of neural net training by utilizing custom silicon developed internally at Tesla. It is optimized for processing video data rather than mimicking the large language model employed by an OpenAI or Google DeepMind.

“In the long term, autonomy—we think—is gonna just drive [vehicle sales] volume through the ceiling next level,” Musk told investors on Wednesday.