Most of us can drive a car, and many of us think we’re good at it. We probably learned when we were teenagers, and have taken the skill for granted since then — except, of course, when we’re silently cursing out another driver for cutting us off. As a result, we may not think about how complex the task is. Not just steering a vehicle along a path, but understanding and reacting to the hundreds of different objects in our environment.
This makes designing automated safety systems for cars that can act reliably in all conditions and not trigger at the wrong time a daunting technical challenge. It’s only been possible because the assumption has been that the driver is still in charge at all times, and the safety system is only there to try and assist as a last resort. But as we rush towards self-driving cars, that assumption is in question. We now have a variety of driver assist systems, of which Tesla’s Autopilot is the best known, that can take full control of your vehicle in many situations. However, as smart as they are, those automated systems can do some pretty dumb things if the driver isn’t paying enough attention — like smash into the back of parked emergency vehicles. We take a look at how some of the pieces of those systems work, and some of the reasons for accidents might be happening.
To Brake Or Not To Brake
In the aftermath of an accident where an Automated Emergency Braking (AEB) system failed to act, we typically only worry about how to make the system more sensitive and responsive. But the designers of those systems need to worry just as much about braking for the wrong reason — called a false positive. Having a car mistakenly slam on the brakes in the middle of a crowded freeway can be just as dangerous as failing to brake.
Toyota, for example, has had to recall several models of its AEB-equipped cars when some of them braked to a stop for harmless objects, such as metal plates on the roadway. In addition to having limitations when it comes to recognizing potential issues, AEB systems also can’t read your mind. Perhaps you’re speeding up a bit in your lane to make it easier to pass the car in front of you. If you stayed in a straight line, you’d hit it, but you know you’re going to steer out of the way. Your AEB system doesn’t know that, so it has to be quite conservative about when it decides a collision is really unavoidable, or it needs to guess. Volvo’s City Safety system, for example, can be over-ridden if the driver is applying acceleration. It assumes that the maneuver is deliberate and deactivates. That was cited as one of the possible causes of the much-publicized accident when a self-parking demo went wrong and caused injuries in 2015.
Drivers Don’t Understand Their Driver Assist Systems
Top-of-the-line cars have several different automated safety systems. Each one comes with its own set of capabilities and limitations. Most only work within a specific speed range, and will only be effective in particular situations or with particular objects. Plus, descriptions of the systems tend to be swathed in legalese, so it can be hard to figure out exactly what you can expect them to do, and what their practical limitations are.
The result, as has been verified by researchers surveying car owners, is that most people don’t have a detailed understanding of what their vehicle safety systems can and can’t do. That’s okay as long as the systems are really just an emergency backup, with the driver assuming personal and continuous responsibility for their car’s behavior. If they get into a situation that requires automated intervention, then that’s a “bonus” and is indeed why AEB systems are shown to save lives.
Sort-of-Self-Driving Cars: The Autopilot Problem
Most of us would never dream of simply putting our foot down on the accelerator and relying on our car’s AEB system to keep us from hitting anything. But when using advanced automated systems like Tesla’s Autopilot, many drivers are putting that much faith into their car’s hardware and software. Sure, the fine print in the Tesla manual says that you’re always supposed to be fully ready to take over, with hands on the wheel. But realistically people aren’t paying $12K extra to have to be just as alert as they were without Autopilot. It’s natural to relax, and hard to keep focused when you’re not actively driving. Tesla’s very lax initial policies for driver behavior didn’t help. The company tightened them up after a Florida man was killed while apparently watching a DVD and driving his model S on Autopilot, but there is still a mismatch between how many drivers rely on the system and the limitations specified in the fine print.
In fairness, we don’t read headlines every time AEB or Autopilot saves a life, although there are certainly plenty of cases where they do. For example, here’s a dashcam video showing Tesla’s Autopilot actively steering to avoid a collision when it’s cut off by a utility truck:
What’s With Hitting Parked Emergency Vehicles?
At least three times this year, Tesla cars have plowed into parked emergency vehicles. In each case drivers have stated they were using Autopilot. Two of the emergency vehicles were fire trucks, and one was a police SUV. All were very visible. That doesn’t count the fatal accident when a Tesla in Autopilot hit a highway center divider at high speed. Tesla’s response has been pretty consistent, basically saying the driver is still in charge, is supposed to be paying attention, and so on. Okay, that aside, intuitively these would seem to be the easiest kinds of accidents to avoid. So what’s lacking?
Tesla has been pretty close-lipped about the internals of its Autopilot system in these incidents. But we can get some clues from how it has updated its warnings, and the specific situations where the crashes occurred. First, emergency vehicles are often parked in odd locations. The fire truck that was rear-ended was partially in the carpool lane and partially in the emergency lane, as it was responding to a previous collision, so it seems likely that it confused Tesla’s Autopilot. Similarly, from the police photos, it appears that the Laguna Beach cruiser that was hit this week was parked partially on the curb.
Second, in some cases the lane markings were hard to see or confusing. That appears to have been one element in the crash into a highway barrier. Tesla specifies that Autopilot should only be used when lane markings are clearly visible. Finally, stationary objects turn out to be a real issue for automated systems. There are so many of them alongside and above the roadway that need to be ignored that it is harder than you’d think to know which ones to pay attention to. It’s possible Tesla’s reliance on radar, instead of the more-expensive lidar used by most other self-driving cars, also makes it more difficult for its cars to distinguish between large vehicles and overhead road signs, for example.
What About Uber and the Pedestrian?
Of course, the worst possible situation is when it isn’t a thing that gets hit, but a person, possibly fatally. That’s what happened in the much-publicized fatal crash of an Uber self-driving test vehicle into a bicycle-walking pedestrian, which illustrates some of the issues faced by companies developing self-driving vehicles.
The reasons for the Uber crash include an interlinked set of policy decisions and their tragic consequences. Uber disabled the Volvo’s vaunted safety systems so they wouldn’t interfere with its autonomous mode software. That isn’t uncommon, but typically it is done when the company doing the testing believes it has something better. In this case, though, Uber’s answer was that the safety driver was to provide that function. However, Uber also fielded its cars with only a single safety driver, and then assigned that driver other tasks that distracted from paying attention to driving. The combination was definitely a recipe for trouble. So when Uber’s software got confused, variously identifying the woman pushing the bicycle as a pedestrian and a cyclist, not only did it not act in time, the safety driver didn’t become aware of her until it was too late for her to retake control of the vehicle and stop or steer out of the way.
None of this means that it was safe for the pedestrian to be wandering across the lanes of traffic where there was no crosswalk. But for autonomous vehicles to be accepted, they are going to need to be good at responding to situations like that.
Experts Disagree Over How Much Automation Is Good Enough
For this reason, some companies, like Alphabet’s Waymo and GM’s Cruise, don’t believe in partially autonomous vehicles, and are working on full Level 5 autonomy for their initial commercial offerings. That takes the human completely out of the loop. Of course, it is also a much harder problem to solve.
Realistically, the industry isn’t going to wait for full autonomy. Nearly every car company in the world is working hard on adding various automated assists to its cars as a way to stay competitive. On balance, most of them are likely to save lives, but we are also going to have to learn the best ways to ensure they are used wisely. Stay tuned for a boom in active driver monitoring systems, for example.
[Feature photo courtesy of Laguna Beach Police Department]