Elon Musk's Sneak Peek at Tesla Full Self-Driving V12, “Nothing But Nets”

On 2023-08-26, Elon Musk live streamed a 45 minute drive in his personal Tesla Model S driving on the streets of California’s San Francisco Peninsula. The car was equipped with “Hardware 3”, not the upgraded “Hardware 4” recently released, with a faster computer and improved cameras. The Full Self Driving software was the forthcoming version 12 (FSD V12), in which all of the previously hand-coded behaviour has been replaced entirely with neural networks trained from video of humans driving and synthetic imagery—“nothing but nets”. In the video he says:

There’s over 300,000 lines of C++ [code] in the exclusive control stack of version 11, and there’s basically none of that in version 12.

During the drive, he observes that when it slows down for speed bumps, it’s because that’s what vehicles which it observed in training do, not because there’s any code that identifies speed bumps. Similarly, for road signs, “It can read signs without ever being told to read”—in other words, it observes drivers’ reaction upon encountering a given kind of sign and learns to emulate it without knowing, for example, the semantic meaning of “Stop”. (This may land Tesla back in hot water with the Safetyland Traffic Nazis, since what it learned to do upon encountering a stop sign is a “California creep” without ever coming to a full stop, behaviour which provoked an earlier recall of a prior version of the Full Self Driving software.)

The video above is an analysis of the demo drive by YouTuber “AI DRIVR” with excerpts from the full video. The video below is a YouTube transcription of the full demo drive (which can only be viewed on 𝕏, not embedded here), slightly edited by Tesla Oracle to improve audio and remove irrelevant material.

Here is Tesla Oracle’s background on FSD V12, its development and testing status, and analysis of the demo video.

Elon musk claims that the V12 software is able to navigate areas which have not been mapped and it has never seen before, and do so entirely autonomously, without online access to Internet resources.

The other voice you’ll hear in the video is Ashok Elluswamy, Tesla director of Autopilot software, sitting in the passenger seat. Elluswamy testified in a 2018 deposition that (quoting the Reuters article below):

A 2016 video that Tesla used to promote its self-driving technology was staged to show capabilities like stopping at a red light and accelerating at a green light that the system did not have, according to testimony by a senior engineer.

The video, which remains archived on Tesla’s website, was released in October 2016 and promoted on Twitter by Chief Executive Elon Musk as evidence that “Tesla drives itself.”

But the Model X was not driving itself with technology Tesla had deployed, Ashok Elluswamy, director of Autopilot software at Tesla, said in the transcript of a July deposition taken as evidence in a lawsuit against Tesla for a 2018 fatal crash involving a former Apple engineer.

The previously unreported testimony by Elluswamy represents the first time a Tesla employee has confirmed and detailed how the video was produced.

The video carries a tagline saying: “The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself.”

To create the video, the Tesla used 3D mapping on a predetermined route from a house in Menlo Park, California, to Tesla’s then-headquarters in Palo Alto, he said.

Drivers intervened to take control in test runs, he said. When trying to show the Model X could park itself with no driver, a test car crashed into a fence in Tesla’s parking lot, he said.

“The intent of the video was not to accurately portray what was available for customers in 2016. It was to portray what was possible to build into the system,” Elluswamy said, according to a transcript of his testimony seen by Reuters.

When Tesla released the video, Musk tweeted, “Tesla drives itself (no human input at all) thru urban streets to highway to streets, then finds a parking spot.”



Meanwhile, what the heck is going on with the Cruise (subsidiary of General Motors) robo-taxis in San Francisco? After a Cruise vehicle collided with a fire truck on 2023-08-17:

The California Department of Motor Vehicles ordered “Cruise to immediately reduce its active fleet of operating vehicles by 50% until the investigation is complete and Cruise takes appropriate corrective actions to improve road safety.” and Cruise announced it would comply, taking half its vehicles off the streets.

Cruise implements their self-driving entirely differently than Tesla, as one might expect for a company whose investors include General Motors and Microsoft. The cars bristle with lidar, radar, and cameras, and rely upon a completely detailed map of the areas in which they operate. The driving behaviour appears to be largely hand-coded, one imagines with Microsoft-style 350 branch case statements making the life-or-death decisions on the road.

Here is AI DRIVR’s take on the situation.


Can the sensors compensate for snow, either snow on the things which must be observed - like road signs and road markings - or on the apertures of the sensors, themselves? It seems to me that this is a big challenge. Parenthetically, I also wonder how battery storage for electric cars fares in the typical low winter temperatures of much of North America and Canada. I have not ever ben tempted to buy one. In fact, since I read that some jurisdictions plan to ban the sale of gas cars by 2030, I bought a new Subaru Crosstrek, which has the AWD necessary for Pittsburgh climate as well as safety features like automatic braking, lane warning and cross traffic sensor/auto braking from the backup camera. That car will likely outlast me.





In addition to reduced battery capacity due to chemical reactions slowing down in cold weather, if the car has a resistive heater, it will account for a substantial chunk of battery power. Unlike a combustion engine where the heater gets its energy from waste heat from the engine’s cooling system, electric motors generate little waste heat and all heating energy drains the battery.

Starting in 2020, Tesla, Kia, Nissan, and Volkswagen have replaced resistive heaters on some models with heat pumps (on some makes and models this is an extra-cost feature). Volkswagen claims their heat pump restores around 30% of the range that would be lost to the resistive heater in weather around freezing outside.

Here is an article about electric vehicle heat pumps from Current Automotive, “Model Y is the first Tesla with a heat pump. Here’s why that’s a big deal.

Can the sensors compensate for snow, either snow on the things which must be observed - like road signs and road markings - or on the apertures of the sensors, themselves?

The sensors on Teslas are just cameras with, I believe, some near infrared capability as well as visual light sensitivity. The front-facing cameras are inside the windshield and take advantage of the wipers and washer. The side cameras are set high to avoid dirt thrown up from the road. The rear camera may require cleaning. All of the cameras have heaters to deal with snow and fog accumulation: this is like the heaters in LED-based traffic lights for the inverse problem.

Obscuration of road signs is precisely as much a problem for the self-driving cameras as it is for human drivers. Presumably the goal is for the software to be trained to do as well as a human driver with the information available.


Interesting if true:

1 Like

Yes, all those little cuts are suspicious. It’s not like there’s an hour of continuous self-driving video.

1 Like