“Level 3 autonomous driving is even more controversial. The main difference is that, while the driver must still remain vigilant and ready to intervene in an emergency, responsibility for all the critical safety functions is shifted from the driver to the car. This has a lot of engineers worried. Experience has not been good with control systems that relegate the operator to a managerial role outside the feedback loop, with the sole function of interceding in the case of an emergency.

It was this sort of thinking that allowed an accident at a nuclear power plant at Three Mile Island, in 1979, to escalate into a full-blown meltdown. Plant operators failed to react correctly when a valve stuck open and caused the reactor to lose cooling water. They then made matters worse by overriding the automatic emergency cooling system, thinking there was too much water in the reactor rather than too little. The accident report blamed inadequate operator training and a poorly designed computer interface.

Similar human failings have led to countless airline accidents—most recently, the Asiana Airlines crash at San Francisco in 2013. Over-reliance on automation and lack of systems understanding by the pilots when they needed to interevene were cited as major factors contributing to the Asiana crash. Some carmakers fear that—even more than reactor operators or professional pilots—untrained motorists may only compound the problem when suddenly required to take control of an otherwise fully automated system. Ford believes it is better to skip Level 3 altogether, and go straight to Level 4, even if it takes longer.”

The long, winding road for driverless cars
https://www.economist.com/news/science-and-technology/21722628-forget-hype-about-autonomous-vehicles-being-around-cornerreal-driverless-cars-will
via Instapaper