The Global Governance of Artificial Intelligence: Some Normative Concerns

2.1 How Democratic Values and Principles are Knit Together

Let us start with the first aspect of the holistic approach, namely, an analysis of the relationship between values (and principles) from a democratic standpoint. As noted above, transparency, fairness, accountability and responsibility are all essential properties of democracy. In the ethics documents – as well as in the debate about the shape of AI governance – these tend to be articulated as separate values specified in principles, which are then operationalized mathematically and implemented through technical solutions (Hagendorff 2020, p. 103). Were we to implement such policies, however, we would run the risk of committing ourselves to the underlying assumption that the more each of these values is realized, the more democratization takes place. Let us call this the ‘additive view’ of democracy. The term ‘additive’ refers to the idea that strengthening a number of core ‘democratic’ values (i.e. values that we associate with democracy) leads to more democracy (Erman 2010).

In order to appropriately theorize democratic AI governance, we contend that this additive view should be rejected, since it does not appreciate the extent to which these values are tied together when regarded as parts of democracy. Above all, the additive view cannot account for the fact that democracy, ‘the rule by the people’, includes two sides that are intimately connected. First, there is an access side of political power, where those affected (e.g. by being subjected or by being the relevant stakeholder) should have a say in the decision-making according to some normative standard. Second, there is an exercise side of political power, where those very decisions should apply in appropriate ways according to some normative standard (Erman 2019, p. 129). These two aspects are typically analogous to the terms ‘input legitimacy’ and ‘output legitimacy’ often used in discussions about democracy in the general political science literature (see Scharpf 1999). Whereas the access side of political power concerns the participation, inclusiveness, deliberation, procedural fairness and political equality of the decision-making, generating input legitimacy of political authority, the exercise side concerns the effectiveness and performance of that authority, generating output legitimacy.

Crucially, however, input and output legitimacy are knit together such that we cannot compensate the lack of input legitimacy with more output legitimacy,^[6] and thus draw conclusions about increased democracy by only looking at strengthened output values, such as effectiveness and performance, as would be allowed by the additive view. Rather, the normative ideal of democracy incorporates an essential connection between input and output legitimacy, which constitutes a two-way relation between those who make decisions and those to whom these decisions apply. In other words, principles of democratic legitimacy regulate the relationship between rule-makers and rule-takers, responding to the question of who exercises power over whom. Hence, in the present context, it is not the case that any agents are supposed to be accountable and responsible (e.g. AI designers or multinational corporations) any more than it is the case that any agents are supposed to have the opportunity to participate in fair and impartial decision procedures as equals. Strengthening each of these values does not necessarily entail more democratic AI governance; it depends on how principles promoting fairness, equality, accountability, responsibility and transparency connect to each other. Moreover, as will be discussed next, it also depends on which entities (agents and institutions) are involved in the governance and what kind of decisions they make.

Needless to say, the answer to the question of what a democratic global governance of AI might look like depends heavily on what we mean by democracy. Since our contribution is metatheoretical, in the sense that we do not aim to develop any substantive theory but focus on desiderata for a good account, it is important to adopt a broad definition to make it consistent with all main conceptions in democratic theory, ranging from models based on voting and electoral representation to models based on civic engagement and deliberation. On this definition, ‘a political system is democratic if, and only if, those to whom its decisions apply have an opportunity to participate in their making as equals’ (Valentini 2014, p. 791).

2.2 The Normative Status of Entities and Different Kinds of Decision-Making

As mentioned earlier, a holistic approach focuses not only on how democratic values and principles are tied together, but also on the normative status of the entities (agents or institutions) that formulate and fortify these values and principles in the form of documented standards and regulations, as well as on the different kinds of decision-making through which this is done. We unfold these two aspects employing two distinctions: authorized and mandated entities, and coercive and non-coercive decision-making.

In global governance, there is a myriad of guidelines, standards, policy documents and laws regulating the AI space, pronouncing principles for the protection and promotion of fairness – understood as inclusion and equality and as impartial procedures – as well as of transparency, accountability and responsibility. But the democratic quality of this regulatory structure will very much depend on what kinds of authority lend it support and on what grounds. According to the broad understanding of democracy presumed here, the access side of political power requires that those who are affected by the regulatory structure (e.g. by being subjected or by being the relevant stakeholder) have influence in the decision-making about its basic form and content. What we mean by authorized entities (agents and institutions) are precisely those entities that are approved through such a democratic procedure. Typical examples of authorized entities are parliaments, like nation-state parliaments or the EU parliament. The use of the term ‘authorized entities’, however, is meant to capture in more abstract terms the normative status of those entities without any necessary ties to the current statist framework, since this opens up more space for future institutional arrangements in global governance with other properties and forms.

Indeed, having influence in decision-making means different things depending on which substantive democratic model is defended, but typically it means at least having a formalized ‘say’ in the decision process, in the form of equal voting rights.^[7] For sure, most theorists – not least proponents of participatory democracy and deliberative democracy – would add a number of conditions to fulfill the requirement of ‘influence’, such as having access to deliberative fora that feed into the decision process (Habermas 1996). But for our metatheoretical purposes, we may leave this an open question. Apart from authorized entities, what we here call mandated entities are entities that have been delegated political power by authorized entities, typical examples being executive bodies and administrative agents and institutions.

Intimately connected to the distinction between authorized and mandated entities is the distinction between coercive and non-coercive decision-making, which in practice typically alludes to law and policy-making.^[8] The latter distinction is often overlooked in the theoretical literature on democracy, which is unfortunate since there are differences between law and policy that are of normative significance from a democratic standpoint. In general terms, laws are both more formal and more fundamental than policies, constituting a system of rules that sets out procedures, principles and standards that mandate, proscribe or permit certain relationships between people and institutions. Policies typically consist of statements setting out certain procedural or substantive goals of what should be achieved in the near or remote future. Importantly, though, they comply with laws and are formulated within a legal framework, even if they may aim to fundamentally change an existing law or identify a new law that is needed. In sum, law-making is specific in the sense that laws generally coerce persons by mandating or proscribing certain relationships between people and institutions, backed up by force or the threat of sanctions (Erman 2020).

On the basic normative level, these theoretical building blocks are tied together in two ways. First, there is a link between authorized and mandated entities that is essential for democracy, since it establishes a justificatory hierarchy, where the former has supremacy over the latter, lending the latter legitimacy through delegation. Second, authorized entities are law-making entities, whereas mandated entities are policy-making entities.

2.3 A Holistic Approach to Global AI Governance

With the three aspects of the holistic approach on the table, let us apply this analytical framework to the AI context. It gives us the tools needed to assess the democratic credentials of different agents and institutions, different guidelines, regulations and documents, and different principles articulating democratic values in the AI space.^[9]

To begin with, there is a multitude of legal regulations already in place today, which are general but apply also to AI technology, ranging from human rights and liability law to antidiscrimination laws and road traffic regulations. The EU’s GDPR law, for instance, has a large effect on AI since it regulates the sizable volumes of digital data on which AI depends and articulates several rights of importance, such as the right against automated decision-making, the right to explanation and the right to erasure and to data portability. Adopting the holistic approach to assess the democratic credentials of GDPR, we see that since it was introduced by the European Union (in 2018) it was legislated by an authorized entity, as those affected (citizens of EU member states by proxy) had influence in the decision-making, thus securing values such as inclusion, equal participation and procedural fairness.

Another example is anti-discrimination laws. Discrimination is prohibited in many constitutions and treaties, including the United Nations Declaration of Human Rights and the European Convention on Human Rights. The latter contains EU law against direct discrimination (i.e. discrimination on the basis of race, sex, religion, color, political or other opinions) as well as indirect discrimination (i.e. discrimination occurring when a seemingly neutral practice places persons of, for example, ethnic or racial origin at a particular disadvantage). Both are used to combat discriminatory AI decisions, in particular protecting against indirect (and thus unintentional) discrimination that is more likely to occur in this context (Zuiderveen Borgesius 2020, p. 19).

These are examples of general laws that also apply to AI. But several challenges remain to strengthening the democratic quality of AI space, so specific laws might be needed. For instance, U.S. senators recently introduced a bill specifically designed to require law enforcement agencies to seek a court order before trying to access personal data from third-parties like private AI-centred companies (Robertson 2021). Moreover, some experts have claimed that the GDPR is too ambiguous to establish a strong protection for individuals and does not in fact provide a ‘right to explanation’ of automated decision-making (Wachter, Mittelstadt, and Floridi 2017). Moreover, since the GDPR only applies to decisions that are based exclusively on automated processing (unless one has given explicit consent), many kinds of automated decisions are out of reach for the GDPR’s rules.^[10] Also anti-discrimination laws face challenges. Existing laws protect against discrimination on particular grounds, which means that AI-assisted decision-making could potentially reinforce existing biases by sorting people into newly invented classes that merely correlate with the protected grounds or by accepting too many false positives or negatives in its classifications. This suggests, at the very least, that the introduction of AI-assisted decision-making warrants an oversight of existing non-discrimination law or an effort to ensure that intellectual property rights do not make oversight impossible (Zuiderveen Borgesius 2020).

If we move beyond authorized entities and law-making and take a look at mandated entities, their democratic credentials varies considerably. In fact, many policy documents on AI are instituted by entities without delegated power from authorized entities. Compare, for example, the processes by which two particularly influential ethics codes were developed. The so-called Asilomar AI principles were reached through a deliberative process among participants in the Beneficial AI conference in Asilomar, California in 2017 and have since been signed by almost 6000 AI researchers and others (Future of Life Institute 2017). The EU Commission’s Ethics Guidelines for Trustworthy AI, in turn, were developed by the independent High-Level Expert Group on AI and finalized, after a round of open consultation, in 2020. Both documents were developed in a transparent way by experts in AI technology, including researchers, representatives from different organizations, and business leaders, and both were driven by the same overall normative aim and without profit. Yet, unlike the authors of the Asilomar principles, the EU expert group were appointed by institutions that are, ultimately, sanctioned by the EU. On the suggested holistic view, this difference is normatively significant. In the EU case, the justificatory link between authorized and mandated entities is established, but in the case of the Asilomar principles the authors are, as it were, self-appointed. Furthermore, the access side (input legitimacy) and exercise side (output legitimacy) of political power are knit together in the EU case, generating both chains of democratic legitimacy and of accountability. Worth noting is that this difference cannot be accounted for by the additive view, which does not have the resources to distinguish the two cases in this respect, since it focuses on the promotion of separate values. So even if the additive view may be used to draw conclusions about the transparency of the process of reaching the Asilomar principles – for example, arguing that more transparency means improved AI governance in that respect – it cannot be used to draw conclusions about, for example, accountability without coupling these output aspects to the democratic values of the input side. In addition, this shows that the governance of AI development and deployment cannot be made more democratically legitimate simply by appealing to values and principles central to democracy, but that the procedures used to establish governance mechanisms also matter.

The global governance of AI is currently characterized both by non-binding ethics guidelines and by binding laws. The former can often complement the latter, not least since rapidly changing AI technology is difficult to pin down and legislation always risks lagging behind. Yet, the illustration above shows how a holistic analysis suggests that ethics guidelines can only be part of a governance structure that purports to be democratic. To get deeper knowledge of how to best democratize global AI governance, more research on the whole regulatory structure is called for. We might, for instance, need both new general laws that apply to AI as well as more authoritative policy instruments (e.g. ethics guidelines) from mandated entities. Regardless of which democratic model is favoured, we have argued here that upholding the justificatory link between authorized and mandated entities as well as the tie between input and output legitimacy is important to be able to assess the democratic credentials of decisions, agents and institutions in the AI space.

3.1 Procedural and Distributive Justice and Fairness

One indicative observation of how the concepts of justice are understood in the ethics guidelines is the attention given to the problem of algorithmic bias, that is, the way in which decision-making algorithms can reflect and exacerbate existing injustices. Essentially, machine-learning algorithms acquire their skills by studying earlier decisions made by humans, and many systems used for decision-making are simply mathematical models based on how we have acted. It is now widely recognized that any biases that have shaped human decision-making therefore risk being magnified and reinforced by the move to automated decision-making: if historical employment decisions have been biased in favour of men, for instance, an AI algorithm trained on these decisions will likely suggest that a company continue hiring mostly men (O’Neil 2016, ch. 6). The tendency to address this problem in terms of fairness suggests that the solution would require the same kind of impartial and non-biased decision-making procedures that we discussed in the analysis of democracy above. This fits well with the view that principles of justice establish when institutions give their subjects what they are entitled to. For instance, one of the things that seems to be unjust about biased AI-assisted decision-making is that it frustrates people’s legitimate expectation that like cases should be treated alike and attaches too much weight to statistical correlations drawn from individuals with similar characteristics. A second reason, however, is that equal treatment might also reinforce existing inequalities, unless institutions instead treat individuals differently in order to achieve equality of outcome. In both cases, the focus is on fairness as a procedural ideal specifying how institutions ought to treat individuals.^[12]

We believe it is crucial, however, to recall that justice and fairness can also be spelled out as distributive ideals, in the sense that they help determine what constitutes a fair distribution of a particular set of benefits and burdens. This is essential, since the general adoption of AI technology will have both beneficial and potentially harmful indirect effects on different individuals. Fairness spelled out as an ideal only pertaining to procedures cannot address this and neither can the principles of beneficence and non-maleficence, since it is uninformative to simply state – as the ethics documents often do – that bad effects should be minimized and that AI should be ‘socially beneficial’. The more pressing question is rather how to think about cases where the benefits and burdens of AI are unequally distributed between different groups and when the realization of some benefits are conditional on creating burdens. Pursuing this question requires engaging with the rich literature of distributive justice that already exists in political philosophy. Two examples illustrate this point.

One of the most important ways in which the current governance of AI – or lack or lack thereof – has distributive upshots is the fact that the technology is in high demand and can yield profits to those who develop and own it. At the same time, the potential harmful effects that AI can have and the mitigation of that risk is largely shared by society as a whole. One way to conceptualize this is to say that currently the profits of AI technology are privatized, while the negative externalities of the technology are socialized (see Korinek and Stiglitz 2017).

For instance, the profits generated by AI technology are to a large extent made possible by free access to the personal data that is constantly created by people’s online presence, and made useful to algorithms by precariously employed so-called ‘click workers’ who manually sort through and arrange the raw data, often in the Global South (see Hagendorff 2020, 105f.). Specifically, this data is crucial when training machine learning algorithms, such as the text-producing GPT-3 technology mentioned above, and the availability of this data is generally held as one of the preconditions for the current AI boom. Given this, it is not implausible to turn to theories of distributive justice to assess whether such data should be seen as a shared resource, much like other kinds of public commons, and whether profits that were created using the data should be privatized to the extent they currently are. Spiekermann et al. (2020), for instance, have recently defended a progressive data tax on companies that use the data to make a profit, by reference to the way it would counteract the negative externality of increasing societal economic inequality that free access to data leads to.

Similarly, many worry about the fact that AI technology soon will be able to replace many of the tasks that so far only humans have been able to perform and that this is likely to cause many jobs to disappear. So far, however, the most influential academic analyses of the phenomenon and possible political responses have been penned by economists (Acemoglu and Restrepo 2018; Frey 2019) or technologists (Brynjolfsson and McAfee 2014), and the issue has yet to be discussed extensively by political philosophers. It is ultimately an empirical question whether this will bring about mass unemployment or if, as in previous technological shifts, innovations will have an offsetting effect of not only destroying but also creating new jobs. On the other hand, even if new jobs are created, they might be much less satisfying. Alternatively, those who have been displaced by machines might not be qualified for them. While other disciplines have focused on the macroeconomic and political instability technology might bring about, there are many prior normative questions left for political philosophers, such as whether people have a right to a meaningful job, and what kinds of compensation claims, if any, are justified (see Danaher 2019). In sum, there is much to suggest that unless regulated, the benefits of AI-driven automation – such as new and cheaper goods and services – will fall on certain groups, while the negative externalities created – such as unemployment or reduced job satisfaction – will be carried by other groups who also gain less from technological shift itself.

Since these pressing problems fall outside of the purview of fairness understood procedurally, this illustrates the desideratum of allowing justice and fairness to be interpreted in non-procedural terms. Without questioning the tendency among scholars to research the ills of and solutions to algorithmic bias, our point is simply that AI technology also raises fairness issues concerning the more indirect benefits and burdens it creates. As the next section will argue, this in turn explains the desideratum of adopting a comprehensive view of what institutional reform might be necessary to fairly distribute the benefits and burdens of AI technology.

3.2 The Impact of AI on Existing Institutions

The lesson from the examples above is not that the existence of negative externalities requires AI development to be stopped. Rather, it shows that AI governance should be attentive to how the profits and externalities are distributed among those whose interests are significantly affected. Yet, since AI technology is likely to have extensive effects throughout society, we will also need to think through how the technology will change and possibly undermine the way many other existing institutions work, and consequently how they should be reformed or reinvented in order to counter these effects. Two examples illustrate this point.

As we noted above, the quality of training data is crucial for the success of developing machine learning innovations, and some of the best available data is currently held by government bodies. Allowing private AI companies to use health care data, for instance, can potentially enable the development of entirely new ways of diagnosing and treating many illnesses. But as we argued before, regulatory frameworks ultimately decide who owns this data and, by extension, who gets to reap the profits from the potentially lucrative end product it enables. Currently, the answer often seems to be that the private company may sell access to the product back to those whose data enabled it.^[13] Whether this is considered just will depend on whether we think of data as a collectively owned resource, but also whether we believe that the premiums entrepreneurs are awarded for refining such resources should be considered just (see Nozick 1974). Examples like this will only become increasingly common in the future, and our point is that they cannot be addressed merely by regulating what AI developers may or may not do. Rather, an analysis of justice in AI governance needs to include other legal and political institutions, such as those regulating intellectual property and public data use. Relating back to the analysis of democracy above, one possible implication of this is that, at the very least, there needs to be institutions that allow democratic deliberation and decision-making regarding the continuation of this practice.

Second, if there is a massive shift toward AI-based automation, this will arguably generate stress on central welfare state institutions. Not only could the share of citizens needing economic assistance grow larger than the institutions were designed to handle, labour-replacing technology will also tend to reinforce the current trend of profits going to capital rather than labour (Karabarbounis and Neiman 2014). Currently, low capital taxes are often motivated by claiming that they make not only owners of capital but everyone better off by helping to attract investments and stimulating growth. Assuming that this is true, the widely discussed difference principle defended by Rawls (1999) could, for instance, be interpreted as saying that the tax cuts are compatible with justice, even though they benefit some more than others. Yet, since the promise of AI-driven automation creates additional incentives for companies to replace human workers with machines (i.e. capital), this eventually risks becoming an existential threat to the income tax base that funds much of the welfare state. Simply put, if there are too few employed people that can be taxed and capital taxes are too low, it is difficult to raise the resources needed for welfare state institutions. This has led to a revived interest in the idea of a universal basic income as a way of both distributing resources fairly and maintaining consumer demand (Brynjolfsson and McAfee 2014). Yet, such proposals must not overlook the issue of how to collect the necessary resources to begin with, in light of how global AI-driven automation can undermine the relevant institutions. This illustrates some of the ways in which we need to consider how to transform the way we currently fund institutions, and what would be acceptable replacements if they turned out not to be viable in the long run.

One possible institutional reform to ensure access to the benefits of AI innovations and thereby enable redistributive policies is a radical idea defended in the debate on the development of new drugs. Some have suggested that the fundamental problem with how drugs are currently developed is the reliance on a system of patents. Patents essentially grant the right to a temporary monopoly on the use of a new technical development to the patent holder, and leave decisions about research and development to be dictated by the financial incentives created by consumer demand. This leads pharmaceutical companies to care less about certain diseases, and it restricts access to potentially life-saving medicines to those with the means to pay for them. Activists and scholars (e.g. Stiglitz 2006) have suggested that an attractive way of complementing the patent system would be to create a fund that awards prizes to those who succeed in making certain developments. These prizes would cover the costs involved in developing new drugs, thus replacing one of the main justificatory mechanisms of the patent system. Importing this idea into the AI technology case makes sense, since both drugs and AI systems are characterized by high development costs, while being comparatively cheap to run or replicate. Awarding prizes instead of patents would ideally make sure that advancements in AI technology are widely shared, and it could prevent companies from building up large patent portfolios in order to charge excessive prices or capture market shares. This would not only help to address the profit-sharing issues related to data use mentioned above, but also enable a degree of democratic control over what kinds of AI technology to research and develop, and what jobs we would like to see automated or not.^[14]

In sum, the way in which reforms like this one promise to address some of the central issues regarding fairness and justice illustrates the second desideratum relating to these ideals. It is plausible to assume that the profits of AI innovation may be differently distributed if people who have generated the necessary data have a say in how it is being used. Similarly, a shift towards a prize system could potentially both counteract unequal wealth accumulation and promote democratic control over the development of AI. The conclusion at the end of Section 2 – that our focus should be on the whole regulatory structure and not only laws and guidelines applying to AI directly – follows from the analysis of this section as well: we should indeed resist thinking of the global governance of AI as merely about properly regulating what AI developers may or may not do. The social and ethical impact of AI is so wide that a successful account of the fair global governance of AI must instead adopt a comprehensive approach to reforming and reinventing a wide range of political, economic and legal institutions.