This is part one of a two-part series examining the ethical dilemma that arises when self-driving cars are forced to take evasive action which may result in serious injury or death. Read Part Two here.

First and foremost, a self-driving car (SDC) must obey the law and drive defensively. There are no unresolved ethical questions if it fails to do so.

This paper looks in detail at the ethics involved where the SDC is not at fault but must consider evasive action due to another party or thing illegally intersecting its trajectory. From the perspective of the SDC, this is ‘the party at fault’ (recognising the ultimate fault may rest with another party, including an ‘act of God’).

Often this problem is likened to ‘the trolley problem’ where several people are tied to a rail line, and you have to decide to let them be killed by an approaching train, or pull a switch to send the train down a sideline killing only one person.

Variants include knowledge of the character and/or age of the people involved.

However, the situation where an SDC is not at fault is very different.

In the case of the people on the train track, they have been forcibly tied to the track. In the great majority of real-life cases, those in the path of the SDC will be there due to their own fault.

Practically, the only facts that can be known for certain upfront by society (for the purposes of setting any ethical rules), and by the SDC in the moments of the accident (in order to follow the rules), are the roles of the people involved:

  1. The ‘party at fault’
  2. The ‘occupants of the SDC’
  3. The ‘occupants in other vehicles’
  4. ‘Unprotected bystanders’ (on foot and on mobility devices such as bikes, motorcycles etc).

With these parties in mind, the paper explores an ethical hierarchy that SDC should be programmed to follow (where possible) in order to decide: who to save when it is not possible to save everyone, in all manner of circumstances? It also considers the use of sidewalks and bike paths to avoid potential harm, as well as the practicality to causing minor injury to one party to avoid major injury and death to others.

First, we have to know who or what has responsibility for making the decision. This comes down to ‘who or what has operating control of the vehicle’.

Who or what is in control?

A previous paper argued that the road rules should permit only two operating modes:

  1. Person in legal and actual control (with and without various Driver Assist technologies), and
  2. System in legal and actual control (within specified areas and conditions)

It was suggested that the best way to ensure a person remains in actual control is to have them retain control of steering at all times. Under this rule, ‘hands-free’ driving would not be permitted until the system could take full control.

It was also argued that once full control is handed to the system there should be no requirement for a person to supervise it – simply because people make very poor ‘passive monitors’. They quickly become distracted and cannot orient themselves and react in time to avert a sudden emergency.

The only permitted handover from the system back to the person would be through a ‘managed’ process that ensured the person had effective control before the system was released from responsibility. This could include the car stopping safely on its own if there was any doubt about the person’s capacity to drive.

Under these rules, any emergency would have to be handled by ‘the system’ while it remains in control.

Responsibility of controller to obey the law and drive defensively

As a matter of principle, whoever or whatever is in legal operational control must obey the law.

There is also a moral/ethical (and in some jurisdictions legal) imperative to drive ‘defensively’. For example, to drive slower with a ‘watchful eye’ when passing a school, or where children are playing, or while overtaking or passing a stopped bus, or when driving through a shopping strip, or in a car park; and to keep a safe distance from the car in front, depending on the speed; etc.

If a Self-Driving Car (SDC) under the control of ‘the system’ breaks the law and/or is found to be driving without due care, in the absence of mitigating circumstances ‘the system’ will be wholly responsible for the consequences. Of course, ‘the system’ will not be liable for any fines and/or damages. It will be one or more of the owner/operator/manufacturer/parts/software supplier(s)/maintenance provider, depending on who contributed to the failure, and to what extent.

Mitigating factors could include failure of infrastructure, such as a collapse in the road due to a burst water main. Hacking is unlikely to be regarded as a ‘mitigating factor’, as the manufacturer ought to ensure cyber-risks are counter-measured.

It is possible for a designer, manufacturer, parts supplier or maintenance provider to be prosecuted for criminal negligence, but unlikely. The likelihood is that any system failure will be managed in accord with recall regulations, with activity focussed on what went wrong and how to fix it.

It is up to each manufacturer to ensure its SDC system obeys the law and drives defensively. This includes taking appropriate evasive action when others are at fault.

SDC must be able to take evasive action when others are ‘at fault’

Perhaps hundreds of thousands (millions?) of times every day in the US alone people take evasive action: braking hard to avoid hitting a stopped vehicle; or braking or accelerating hard to avoid a car that fails to give way at an intersection; or swerving into a parallel lane or an opposing lane to go around something or someone, or even onto the footpath. In most cases, they avoid any collision or incur only minor damage that goes unreported.

Such actions also help to mitigate potentially more serious accidents that are reported, so only property is damaged, or people sustain only minor injuries, instead of major injury or death.

Unfortunately, there are no reliable statistics for these saved incidents or mitigated accidents, but they are likely to be at least an order of magnitude greater than reported accidents.

Clearly, any SDC will have to be able to manage these incidents and accidents at least as well as most people – if they are not going to add to the toll.

In fact, they may have to do better than people to avoid liability.

In many jurisdictions, there is a recognition that the duty of care to others is reduced in an emergency. It is called the ‘sudden emergency doctrine’. It is based on the recognition that, in emergencies, people cannot be expected to always react in the most appropriate way – due to the fact that they have little or no experience in dealing with such situations and are prone to panic or other human responses that limit rational thought.

In the case of an SDC, it is likely that, as the response has to be ‘pre-programmed’, it will be expected to adhere to agreed ethical standards.

The next sections discuss what those standards ought to entail.

Ethical questions only arise in very limited situations

Ethics is an issue in only a tiny fraction of ‘incidents’, that are nonetheless significant from a moral perspective.

They only arise when:

  1. the SDC is not at fault (driving defensively within the law)
  2. another party causes it to take evasive action
  3. there is sufficient time to react
  4. there are alternative paths, but
  5. every path leads to someone being injured or killed.

The cause may be, for example, something has fallen off another vehicle into the path of the SDC, or another party is likely to illegally intersect the SDC’s path (as when a person or animal runs out, or another vehicle fails to give way when it should, or an oncoming vehicle overtakes toward the SDC in an unsafe manner, or loses control, etc).

The principal question is: who to save when it is not possible to save everyone?

Practical limitations

In practice, who to save comes down to which path to take: straight ahead, swerve left, or swerve right; and to brake or accelerate?

As it is impossible to compute the outcome of multiple collisions, the ethical decision must be limited to the path the SDC first chooses, without regard for what happens after any subsequent collision. This is no different to the choices people make, except an SDC may have more time and be better able to compute the best alternative due to the extended range of its sensors, access to a detailed 3D model of the immediate environment, its processing speed, and better ‘awareness’ of its own dynamic capacity to respond in the circumstances, unaffected by emotional factors.

In implementing any strategy, just like people, an SDC will also be constrained by the physics of the situation. In general, the faster it is going and the less traction it has, the narrower will be its ‘arc of response’ (the range within which it can veer left or right).

Before we can decide on the most appropriate path, it is first necessary to answer a subsidiary ethical question: should the footpath and/or bike paths be used as escape routes?

Use of the footpath and bike paths as potential escape routes

While driving onto any footpath or bike path is inherently unsafe, every day around the world, people do it to avert a collision or mitigate the outcome of an accident, even if the law does not specifically allow it.

In the case of SDC, a specific decision needs to be made to either prohibit or permit the use of the footpath and/or bike path to avert a collision in an emergency that is not of the SDC’s making, as this will need to be pre-programmed into the car.

Ethically, there is no rationale for treating an SDC any different to a human driver. It means that whatever decision is made for the SDC, it should apply to human drivers also.

Unfounded concerns relating to the use of the footpath and bike paths to avert an accident

If the law specifically authorises the use of the footpath and/or bike path as an emergency escape route, it is recognised that some people may become unnecessarily fearful that they are being exposed to a new risk, namely: collision with a car on the path.

However, the fear is unfounded for several reasons:

First, people on foot, bikes and other open mobility devices are already exposed to the risk, legally or not.

Secondly, though it happens, few people will encounter the risk in the whole of their lives due to the rarity of the event. You only have to consider how often it has happened to you and people you know. This is a consequence of two facts: 1) the need to drive onto a footpath or bike path to avoid collision is itself a relatively rare event; and 2) apart from city centres and shopping strips during certain hours, any point on a footpath is rarely occupied, and then only very briefly. As a result, (outside crowded centres), the chances that a pedestrian is in the path of a car in any incident is quite remote. And even when they are in the path, they often jump out of the way. Bike paths will be crowded in different places at different times. Even so, most of the path will be unoccupied most of the time.

Thirdly, and most importantly, the authority to use the footpath and/or bike path would be subject to the overriding requirement that such action be ‘safe’. In practice, this means that no unprotected person (eg people on foot, or bikes or other open mobility devices, etc), suburban fence or shopfront should be within the stopping range of the vehicle when it swerves onto the bike path and/or mounts the kerb. (The reason for excluding collision with fences and shop fronts is that it is impossible to know if anyone is on the other side).

Other countermeasures

In places and at times where people are in significant numbers but cars still need access to the road; emergency use of the footpaths and bike paths may be outlawed, but the speed limit can be reduced to mitigate any accident. Or, people can be given the right of way, forcing cars to slow and be on alert. In this case, the whole road is turned into a footpath/bike path, with cars permitted to use it with due care. This ‘traffic calming’ measure has been trialled with some success in a number of cities, Copenhagen amongst them. Melbourne is another city that has a long-term plan to develop a hierarchy of roads where pedestrians take precedence.

Consequences of any ban against using footpaths or bike paths as an escape route

The decision to ban or not to ban will likely be a ‘local’ (city x city) decision.

If any regulator acquiesces to fear and bans emergency use of the footpath and/or bike paths by any vehicle, it will doubtless lead to:

  • more traffic violations as people are prosecuted for crossing into a bike path and/or mounting the kerb to avoid an accident that is not their fault (which seems senseless if no one is hurt), and, of much greater concern
  • unnecessary property damage, injury and death as drivers and SDC obey the law and crash into people, animals, cars and other things they could have avoided by using the vacant footpath or bike path (which seems unethical). We don’t know to what extent this may happen, as we don’t know how many accidents are presently avoided or mitigated using the footpath and/or bike paths.

The ethical questions that will have to be answered by each city are:

  1. Do we hold back the use of SDC until they are proven safe to use the footpath and/or bike paths in an emergency (foregoing their overall benefits such as more spare time, greater mobility for all, and better safety generally)? or
  2. Do we allow wide use, but prohibit emergency access to the foot and bike paths? In this case, it would mean accepting a (possibly minor) increase in property damage, injury and death (due to the SDC inability to take evasive action), in order to gain the vastly greater overall benefits of using SDC? or
  3. Do we allow both wide use and access to the footpath and bike paths for emergencies, with the risk that (most likely rarely) someone may be severely injured or killed on a path as a result of an SDC taking evasive action that was not safe? and/or
  4. Do we institute other traffic control measures to mitigate the risks?

The answer to these questions may change if the data shows more than a very minor an increase in accidents relative to the overall reduction in other accidents due to the use of SDC.

Technical challenge of using footpaths and bike paths as an escape route

Unfortunately, approving the use of the footpath and bike paths to avoid an accident is only the easiest part of the problem solved.

Separately, each manufacturer must determine if its SDC can safely mount the kerb at the specific location of any incident (likely based on detailed 3D models of the area, including fences, shopfronts and the profile of the kerb and gutter) and stop before hitting a person, fence or shop front, given the car’s velocity and dynamics. If safe stopping cannot be determined with a high degree of assurance as the accident unfolds, it would simply mean that the SDC would have to be programmed to not use the paths to avoid the specific collision.

This raises a number of ethical issues.

If a manufacturer restricts its SDC from using the footpath and/or bike paths as an escape route (because it cannot do so safely) and as a result, has an accident that could have been avoided, does society indemnify them if, overall, their safety record is much better than for human drivers?

If one manufacturer solves the problem effectively, should this solution be proprietary, or should it be made mandatory and licenced to all the other SDC producers? Can this even be done if the solution is embedded in the whole SDC system?

Overall, property damage, injury and death rates ought to drop through the use of SDC that obey the law and drive defensively. This may suggest to some that, if it will be a rare event, ‘why bother developing detailed 3D models and algorithms to use the vacant footpath or bike paths in an emergency’.

The simple answer is that ethically if we can, we should… as long as the costs of doing so are not out of proportion.

Fortunately, the technology to model and recognise the natural and built form of the city and the behaviour of moving objects (including people) is improving at a rapid rate, so the challenge of utilising the footpath and bike paths to avert or mitigate an accident is becoming easier to meet. By the time SDC are ready for general use, it may no longer be a technical problem.

Assuming each city makes its own decision to allow or ban the use of the footpath and/or bike paths to avoid an accident, we can now address the principle ethical question.

This is part one of a two-part series examining the ethical dilemma that arises when self-driving cars are forced to take evasive action which may result in serious injury or death. Read Part Two here.

Save

Save

Previous post

Human touch returning to customer communications, says Twilio product VP

Next post

The shop still rules - 2.6 billion digital natives can’t be wrong

Join the digital transformation discussion and sign up for the Which-50 Irregular Insights newsletter.