Inside Tesla's Dojo AI Training Supercomputer

Interestingly, because the computer is designed for in-house workloads and not as a multi-user or cloud server, the architecture dispenses with many of the task partitioning and hardware security measures general-purpose processors incorporate (more or less). In Dojo, these functions are delegated to software, which is under the control of the machine’s designer/owner/operator.

Each “training tile”, the basic unit from which the computer is assembled, mounting 25 D1 chips, each with 354 processing cores and 440 Mb of static RAM, on a wafer providing high-speed interconnect, dissipates 15 kilowatts of heat, fed with 18,000 amperes of power at 0.83 volts.

1 Like

This and similar videos we see here remind me of the inadequacy of my understanding of computer science. The best analogy I can make is with medical education. Thought I doubt it is very similar today (due to social justice consuming the curriculum), I came to realize that the first two years of medical school - the basic sciences - were in reality a broad overview of the historical lines of thought in most all the medical sub-disciplines; it also taught the language of medicine. When finished with these 2 years, one had a decent intellectual sense of how the current state of the art arose and how the parts fit together. It was satisfying and heuristic to be able to read and understand medical literature in many fields and fit such new knowledge into the evolving context of medicine.

I find myself thoroughly incapable of doing anything remotely similar when it comes to computer science. I can tell it is significant, but am unable to follow it at all. As I read, particularly, John Walker’s writings, I see what is possible for those who have lived and worked at the center of the computing revolution, this absolutely astounding explosion of human capabilities - as mediated for the moment at least, on silicon. I wish I were going to be around for future developments (provided we don’t exterminate ourselves first through the political lunacy which is afoot).

1 Like

Actually I think you have an excellent understanding of computer science! The mystery comes because of the speed. I listened to audio of a Richard Feynman lecture depicting computers as very fast filing systems.
(In fact the web link to the lecture may have been on Scanalyst(?))

1 Like