Developing moral AI to support decision-making about antimicrobial use

The question of how to build moral AI decision-making systems is very important. Building a decision system based on an underlying moral paradigm, called the top-down approach in AI, is one of the most significant ways of doing this¹⁶. The main moral paradigms employed in AI ethics are utilitarianism (a type of consequentialism), deontology and virtue ethics. Below, we compare the applicability of these different ethical theories to the moral use of antimicrobials to determine which approach is most appropriate to apply to AI decision-making systems in this scenario.

First, by taking the perspective of a utilitarian ethicist who believes that an action is only ‘good’ if it creates utility (frequently measured as happiness)¹⁷, a moral balance may be achieved and applied to the development of AI-driven CDSSs. Utilitarianism in healthcare can be evaluated using several techniques. These include total, average, minimum and total-average utility¹⁷. Maximizing total-average utility is likely to be of greatest importance in the context of AMR and healthcare, given that it aims to optimise the average happiness for those people who are currently alive¹⁷. This aligns with the UK’s General Medical Council (GMC) ‘duties of a doctor’ formulation whereby the objective is to maximise health and extend life for the patient being treated, while also considering wider society and providing equality¹⁸. Frameworks such as Bentham’s felicific calculus are commonly used within utilitarianism and can be applied to complex healthcare questions to quantify the utility of an action¹⁹. Figure 1 provides an illustration of this calculus and its application to the decision to start antimicrobial treatment. When taken as a whole, this framework suggests that for prescribing antimicrobials to be justified, the intensity and duration of the utility gained for the individual patient must outweigh the negative effect on everyone else. The only utilitarian scenario in which this could occur is with what is known as a ‘utility monster’, who gains significantly greater utility from actions than others do, combined. This seems improbable to exist in practice, and thus, to maximize total average happiness, we must look to maintain, and promote, all life. Importantly, however, certainty and propinquity cannot be quantified without more information. AI models can therefore contribute toward utility evaluations by helping to estimate the effect of a particular agent on the development of AMR versus the likelihood of clinical efficacy. By combining Bentham’s felicific calculus and AI-based CDSSs, we can quantify the utility of an antimicrobial prescription and understand the potential resulting individual and societal implications.

The seven variables of Bentham’s felicific calculus and their associated description are shown on the left. An example of using this algorithm to estimate the utility of the decision to initiate antimicrobial treatment is provided on the right.

Deontological, or duty-based, ethics evaluates whether an action is morally good or bad based on whether one acts in accordance with one’s duty²⁰. Duty-based ethics is very amenable to rule-based decision-making; therefore, one could attempt to understand which perfect and imperfect duties could apply in this situation, based on, for instance, Kant’s categorical imperative²¹. The UK’s GMC ‘duties of a doctor’ and the Hippocratic oath can be considered relatively deontology-focused approaches, given that they take care of the patient as of primary concern¹⁸, which is a deeply intentional and duty-focused value. However, these principles were designed to be universal and are not tailored for the significant ethical dilemma posed by antimicrobial prescribing.

Virtue ethics, in contrast, focuses on the moral character of the agent carrying out actions, an assessment of which can be made based on comparisons with a virtuous person who possesses and embodies the virtues²². In this context, virtues need to be defined, embedded in the moral agent, as previously described¹⁴, and the question of ‘What would a virtuous person do?’ considered. One could argue that virtuous clinicians may act as moral exemplars for complex AI-supported decision-making²³, but common moral dilemmas arise in the context of decision-making to address AMR. For example, one might need to weigh the potential number of lives lost and the value of taking action to reduce the number of deaths versus taking personal responsibility for an individual’s outcome, or consider whether acting in a high-pressure situation brings equal moral responsibility as not acting.

Finally, applied ethical theories can also be explored: for example, the four principles of medical ethics (autonomy, beneficence, non-maleficence and justice), which are commonly used as a platform upon which moral agents in healthcare should act²⁴. When applying these principles at a societal level, one can consider that they should translate across time²⁵. In this case ‘justice’, defined as the obligation of fairness in the distribution of benefits and risk, means that there is a responsibility to provide equal and fair care to everyone, no matter whether they are alive yet or not. Furthermore, as modern medicine only began in the nineteenth century with breakthrough discoveries from Louis Pasteur and Florence Nightingale, one can argue that the vast majority of individuals who will need care won’t yet have been born. Therefore, for antimicrobials, which can be considered a finite and limited resource given the development of AMR, we must aim to optimise prescribing and reduce inappropriate use so that their associated benefits and risks are fairly distributed.

By comparing different ethical theories, we can infer that they may have contrasting viewpoints on what is considered morally right with regard to prescribing antimicrobials. We suggest that a utilitarian approach is most appropriate for antimicrobial decision-making given the number of individuals potentially affected by AMR, and alignment between current best practice, Bentham’s felicific calculus, and the principles of medical ethics indicating that antimicrobial resources should be fairly distributed.