A piece of tape tricks Tesla autopilot to misread speed limit sign

[Stevicus]

How a Piece of Tape Tricked a Tesla Into Reading a 35MPH Sign as 85MPH

Researchers have identified a potential issue with sensors used in some older Tesla models that misinterpreted speed signs to read the speed limit as 50 miles per hour greater than the sign stated when a piece of electrical tape slightly obscured one of the numbers.

The issue was identified as part of 18 months of research by McAfee researchers Shivangee Trivedi and Mark Bereza, who sought to examine the ways that machine learning could potentially be exploited, specifically in the case of autonomous vehicles. Many autonomous vehicle suppliers use cameras produced by MobilEye to register the limits on speed signs, and McAfee researchers found that using stickers to slightly alter the appearance of the sign had the potential to interfere with the camera’s reading.

Apparently, a piece of electrical tape was put on a speed limit sign:

And this caused the sensor to misread it as "85" instead of "35."

I'm surprised that it would actually have to read signs. I thought all that information regarding roads and their speed limits would be in some GPS database. The company stated that in practice, they would not rely solely on the sensors to read signs:

Reached for comment, a MobilEye spokesperson told Gizmodo by email that stickers on the traffic signs “can confuse a human eye and therefore we do not consider this an adversarial attack.” While this may be true for the stickers that obscured the number, that’s not necessarily true for the electrical tape, which the researchers noted did not necessarily indicate to a human that the sign had been altered. However, the spokesperson added that the system was not meant to support fully autonomous driving.

“Traffic sign fonts are determined by regulators, and so advanced driver assistance systems (ADAS) are primarily focused on other more challenging use cases, and this system in particular was designed to support human drivers—not autonomous driving,” the spokesperson said. “Autonomous vehicle technology will not rely on sensing alone, but will also be supported by various other technologies and data, such as crowdsourced mapping, to ensure the reliability of the information received from the camera sensor and offer more robust redundancies and safety.”

The researchers commended MobilEye for apparent changes to its cameras and for looking to research to inform their product performance. They did note, however, that the camera that misclassified the speed limit is still present in a significant percentage of deployed Teslas. And while drivers are expected to still be alert behind the wheel of any semi-autonomous vehicle, history has shown that is not always the case.

I suppose another way to trick these cars would be to put a cardboard cut-out of a kid crossing the street in the path of an automated vehicle. The sensors would see it as a pedestrian and stop the car, and it would remain stopped until someone determined it wasn't a pedestrian or until someone moved it.

 

[Twilight Hue]

Can you imagine these things glitching out?

Or the sensors getting clogged with debris or snow?

Whoever thought automated cars was a smart idea wasn't very bright to begin with.

 

[BSM1]

I am envisioning a movie about a driverless smart car colliding with a Roomba.....who gets sued?

 

[Heyo]

Can you imagine these things glitching out?

Or the sensors getting clogged with debris or snow?

Whoever thought automated cars was a smart idea wasn't very bright to begin with.

The software doesn't have to be perfect, it just has to be better than the average human - or at least cheaper.

 

[Altfish]

I don't think we'll see these types of car accepted in my lifetime - mind you I am over 60.

The problems are insurance and programming in the event of an accident.

How do you sort out the insurance when a Tesla hits another car?

How do you programme the cars when an accident is inevitable? Save the driverless car 'driver' and kill the 6-year old kid that ran into the road?

 

[Revoltingest]

How a Piece of Tape Tricked a Tesla Into Reading a 35MPH Sign as 85MPH



Apparently, a piece of electrical tape was put on a speed limit sign:

And this caused the sensor to misread it as "85" instead of "35."

I'm surprised that it would actually have to read signs. I thought all that information regarding roads and their speed limits would be in some GPS database. The company stated that in practice, they would not rely solely on the sensors to read signs:





I suppose another way to trick these cars would be to put a cardboard cut-out of a kid crossing the street in the path of an automated vehicle. The sensors would see it as a pedestrian and stop the car, and it would remain stopped until someone determined it wasn't a pedestrian or until someone moved it.

GPS databases are full of errors.
(It's something I watch as I drive around
the country. I like to avoid speeding tickets.)
So reading signs is useful.

 

[Revoltingest]

I am envisioning a movie about a driverless smart car colliding with a Roomba.....who gets sued?

Whoever has deep pockets.....duh!

 

[Twilight Hue]

The software doesn't have to be perfect, it just has to be better than the average human - or at least cheaper.

As long as a human being can still turn it on and off.

My wife had a new Subaru with automated lane control and it forces her back into the lane if she happens to drift a bit as with everybody from time to time. It's disconcerting to say at the least. Fortunately she can turn it off.

 

[Twilight Hue]

I can imagine going through a speed trap at a school zone and your car suddenly bumps up to 80 miles an hour.

Conspiracy theory anybody?

 

[Audie]

A piece of tape lad to Nixon's downfall.

 

[Heyo]

As long as a human being can still turn it on and off.

My wife had a new Subaru with automated lane control and it forces her back into the lane if she happens to drift a bit as with everybody from time to time. It's disconcerting to say at the least. Fortunately she can turn it off.

Personally, I'm also not a big fan of automatics that can't be turned of. But I see driverless cars, and especially trucks, as inevitable. Once they come close to the reliability of a human, they will be everywhere. It's economics 101.

 

[Stevicus]

GPS databases are full of errors.
(It's something I watch as I drive around
the country. I like to avoid speeding tickets.)
So reading signs is useful.

True, although another basic driving rule is "reasonable and prudent." Even if there is no posted speed limit, most drivers would govern their speed appropriate to whatever conditions there are, whether they're in a residential neighborhood, a busy city street, or a superhighway. They might slow down if it's nighttime, or snowing or raining.

Humans would adjust to these things, but would a computer driving a car be able to do it? Would it have to rely on signs to tell it what is going on?

 

[Twilight Hue]

Personally, I'm also not a big fan of automatics that can't be turned of. But I see driverless cars, and especially trucks, as inevitable. Once they come close to the reliability of a human, they will be everywhere. It's economics 101.

It's still going to be quite awhile though. But I do see it as being a real possibility down the road*.

*Pun not intended, but it works all the same. Lol

 

[Quetzal]

And this caused the sensor to misread it as "85" instead of "35."

I have a little bit of background in machine learning, which is what we are seeing here. The software is attempting to match the design to an integer. Since the middle of the three extended out, it increased the pixel density close to the top ends. Thus, it is, technically, closer to an 8 than a 3.

My knee jerk reaction is to enforce than an 8 is only present if the circles are completely closed. But then someone could cover only one part of the 8 and it would register as a 3, 6, or 9.

This is a good case study because it demonstrates the glaring weakness of this particular model: malicious people.I am not sure if an automated process can fix this problem.

 

[sun rise]

Can you imagine these things glitching out?

Or the sensors getting clogged with debris or snow?

Whoever thought automated cars was a smart idea wasn't very bright to begin with.

These horseless carriages are not a smart idea and clearly a dumdum came up with the idea. Get a horse.

And these flying contraptions, they'll never amount to anything.

And who needs a com-pu-ter anyway. They are gigantic and don't do much more than a person can do but cost a lot of money and are difficult to maintain.

 

[HonestJoe]

To be fair, most human drivers seem to "misread" speed limit signs, even when they're perfectly clear and unmodified.

 

[Revoltingest]

True, although another basic driving rule is "reasonable and prudent." Even if there is no posted speed limit, most drivers would govern their speed appropriate to whatever conditions there are, whether they're in a residential neighborhood, a busy city street, or a superhighway. They might slow down if it's nighttime, or snowing or raining.

Humans would adjust to these things, but would a computer driving a car be able to do it? Would it have to rely on signs to tell it what is going on?

A computer could recognize conditions which should limit speed,
eg, snow, rain, fog, residential streets.

 

[ChristineM]

The transport authority recently imposed a 50kph limit through a nearby hamlet. The previous speed limit was 80kph. Every morning someone draws a diagonal through the 5 making it an 8. Every midday a cleaner from the prefecture turns up to clean the signs.

This has been going on since they imposed the speed limit before Christmas, maybe one day they will think it worthwhile to send someone out early to apprehend the artist

 

[ChristineM]

A computer could recognize conditions which should limit speed,
eg, snow, rain, fog, residential streets.

Speed limit sign positions and restrictions are recorded and transmitted with with satnav data. Mine always displays the current limit and beeps like crazy if i exceed the limit by more than 5%

I though this was what computer guided cars worked off

 

[Revoltingest]

Speed limit sign positions and restrictions are recorded and transmitted with with satnav data. Mine always displays the current limit and beeps like crazy if i exceed the limit by more than 5%

I though this was what computer guided cars worked off

Who knows what goes on in the minds of those computers.
My Garmin Nuvi has a female voice...& you know the problem with women.