Elon Musk Says Tesla FSD AI Having Exactly The Problem I Thought 2 Years Ago It Was Having

Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News!

During the Q&A session of the 2024 Tesla annual shareholder meeting, in response to one of the questions about Full Self Driving (FSD), CEO Elon Musk explained something about Tesla’s FSD development that completely affirms a suspicion I had two years ago. Also, this is one of the key challenges that made me a bit concerned about whether Tesla’s approach will lead to robotaxi capability.

I’ll start with what Elon said, emphasizing in bold the key part I’m highlighting. “As the system gets better, it gets harder to figure out which AI model is better. Because now, like thousands of miles between interventions, how do we, as quickly as possible, figure out which AI model is better. And when you make these different AI models, they’re obviously not, like, super deterministic. So, you have one model that eliminates one problem, but creates another problem.” I’ll come back to the rest of this segment later and how Tesla is dealing with the issue and trying to get around it, but I just want to emphasize here that it’s not simply the challenge of picking the best model, but the challenge of one problem getting fixed while another problem is created. Let’s return briefly to what I wrote more than two years ago when I started noticing this problem:

“Basically, I’ve been trying to understand reasons why Tesla FSD isn’t as good as I’d hoped it would be by now, and why it sometimes gets significantly worse. One potential issue is what I’m calling the ‘see-saw problem.’ If my theory is correct to any notable degree, it could be a critical fault in Tesla’s approach to widespread, generalized self driving.

“My concern is that as Tesla corrects flagged issues and uploads new software to Tesla customer cars, those corrections create issues elsewhere. In other words, they are just playing software see-saw.”

I don’t know how much that was happening two years ago compared to today, but with Musk acknowledging that it’s happening, let’s take a look at how he says Tesla is trying to solve the problem.

“So, we’re trying to solve this by a combination of simulation; uploading models, having them run in shadow mode — so, it’s actually kind of helpful not everyone has Full Self Driving, because we can see … we can run it in shadow mode and see ‘what would this new model have done compared to what the user did.’ So, since we’ve got millions of cars that we can do this with, that gives us a delta between what the AI model predicted it would do and the user would do. And if you kind of sum up the errors between them, you can see, ‘oh, there was a bigger error stack from this model versus that model’ when you uploaded them into — we each uploaded them into 100,000 cars.”

He then goes into a bit more detail on how they are trying to advance from model to model. “But that’s the biggest limiter right now — it’s not training, it’s not data, it’s actually testing the AI models. And then figuring out clever ways to figure out if a new model is better or not. Like we know like there are particular intersections that are difficult. […] Like, all the models will be great at driving on a well painted road with no intersection. They’ll all be great at that. […] So, we’ve identified several thousand intersections in the United States where we’ll actually take the new model and try the car in those very difficult intersections.”

He also added that they haven’t yet uncorked any of the extra ability of FSD “Hardware 4,” but says they’ll do that later this year.

Overall, the big question with regard to identifying new better and better models, and overcoming the aforementioned “see-saw problem,” is how quickly Tesla can do this in order to roll out unsupervised full self driving and robotaxis. Will it take a year? 10 years? Obviously, Elon Musk is optimistic they’ll get there soon, but as even Musk states, he’s chronically overly optimistic. What’s your estimate?