Autonomy Day

I watched Tesla’s “Autonomy Day” event yesterday and these are my notes.

According to Tesla, a self-driving car is best served via visual/camera input, just like a human driver. “Lidar” and detailed maps are crap compared to simply seeing the world as a series of still-framed images — visual cues are best. But in order to recognize what all these visualized objects are, the car needs access to a vast human-annotated database. In other words, humans must create a collection of labeled images so that the car can meaningfully interpret the shapes it detects.

But because of variances in every object, the car needs to make guesses about what it sees. In order to make guesses, a “neural network” is necessary. By inputing and labeling many pictures, the computer learns to identify images that are more likely to be one thing rather than another, in other words, it’s “close enough”. So now when the car sees an object that’s reminiscent of all the pedestrians it was previously shown, the car can assume that the unique object currently on camera is a pedestrian too.

Tesla’s self-driving technology relies on a vast array of cars-in-the-field sending data back to feed the ever-learning neural-network, which then sends updated instructions back to those cars, in effect teaching them to drive better based on the shared-experience of all the networked cars. When one car encounters a unique experience, a human at Tesla analyzes and annotates the scene and updates the database so that all cars can benefit from the experience of the first car.

A couple other things to note: For times when vision is obscured, such as fog, a forward-facing radar does come into play. For the most heavily congested areas, an aggressive driving mode seems to be necessary. The car will likely engage in a “game of chicken” where it attempts to force itself through — this means that there is a non-zero chance of “fender benders” under these conditions (but the level of aggression is going to be optional per driver).

More things to note: The cars being manufactured right now are NOT fully autonomous but they have all the necessary hardware for self-driving — and will only require software updates when ready (possibly within months). Tesla does have experimental self-driving cars on the road right-now that run with impressive results. Their timeline to a self-driving car has been incredibly fast, perhaps about 2 to 3 years.

Also something to note: Tesla’s taxis. There will be a fleet of self-driving cars available to be summoned via an app. This will be a cheaper option than current solutions from Uber/Lyft. Tesla itself will have dedicated cars for this purpose and Tesla owner’s can also add their own cars to this network (for a financial benefit).

Elon predicts that human-driven cars will become a thing-of-the-past very quickly. He expects the reliability of computer-controlled cars to become so undeniable that human-driven cars will seem reckless in comparison.

And of course: I, for one, welcome our new autonomous overlords.