Military artificial intelligence (AI) capabilities and services are increasingly being used across a range of military operations to support targeting decisions, among other functions. Because these capabilities are most often provided, enabled and sustained by industry actors, the use of AI in targeting has reignited old debates about how international humanitarian law (IHL) applies to industry—for example, under what circumstances industry actors can lose the protection of IHL and when, if ever, an industry actor can be held accountable for IHL violations.
This backgrounder aims to shed some light on three of the most commonly asked questions. In doing so, it draws on a vast literature on how IHL applies to private sector actors, while also identifying areas that merit further examination. If states wish to integrate AI into their military capabilities, a deep understanding of how IHL applies to the industry supporting these capabilities is essential.
The military AI industry’s involvement in armed conflict
Industry involvement in armed conflict is far from new. States have relied on private companies for war-fighting capabilities, not just for producing weapons but also for ongoing maintenance and training, for centuries. As a result, the IHL implications of industry involvement in armed conflict have also been examined for years, notably by the International Committee of the Red Cross (ICRC).
The rapid advance of AI for military applications has, however, given new urgency to this topic, not least following reports about how AI capabilities, developed and supported by industry actors, have been used in attacks across Gaza, Iran, Ukraine and Venezuela. While the number of publicly reported cases remains small, they suggest a future trend of increasingly complex industry involvement. For example, many AI-enabled products and services will need constant monitoring, updates and refinements from industry suppliers during use. Industry involvement is also likely to become particularly complex in cases where militaries do not have the technical literacy needed to fully understand and independently manage the systems procured for them and thus rely on suppliers’ expertise and advice during use.
This prospect has prompted questions about how IHL applies to the military AI industry. This relates not only to their responsibilities and potential legal accountability, but also to when their personnel or assets may become legitimate targets in warfare. It is crucial that states are equipped to answer these questions. Likewise, it is essential that military AI industry actors themselves understand the legal implications of their involvement, especially the many new entrants to the market that are becoming exposed to IHL for the first time.
When do companies providing or supporting military AI become lawful targets under IHL?
Earlier this year, Iran carried out multiple strikes on data centres in Bahrain and the United Arab Emirates operated by the United States-based company Amazon Web Services. Although the data centres provided a wide range of civilian services, Iran reportedly claimed its attacks were intended ‘to identify the role of these centres in supporting the enemy’s military and intelligence activities’. These attacks brought renewed attention to the question of when industry actors can become legitimate military targets.
As a legal baseline, private companies—specifically their employees and assets—have civilian status and are protected under IHL. However, under certain circumstances, they may lose their protections and become legitimate military objectives (i.e. targets) from the perspective of IHL.
Starting with employees, they can lose protection if they become members of the armed forces, if the company itself belongs to a party to the conflict, or if their activities amount to ‘direct participation in hostilities’ (DPH; see Article 51 of Additional Protocol I to the Geneva Conventions). DPH is particularly relevant in the context of military AI industry personnel, considering, for example, military forces’ reliance on ‘forward-deployed engineers’ who work alongside military personnel to adapt and support AI-enabled systems during use. Under the ICRC’s three-part test, to qualify as DPH, three criteria must all be fulfilled:
1) the act must reach a threshold of harm (such as inflicting death, injury or destruction on protected persons and/or objects);
2) there must be a direct causal link between the act and the harm, and;
3) the act must be specifically designed to cause harm.
These criteria have traditionally been applied to civilians, for example, when they pick up arms to join one side of the conflict. It is unclear, albeit increasingly relevant, what activities carried out by tech workers, including those from military AI companies, would satisfy the DPH criteria.
When it comes to a military AI company’s assets, they may lose their protections under IHL if they ‘make an effective contribution to military action and [their] total or partial destruction, capture or neutralization, in the circumstances ruling at the time, offers a definite military advantage’ (Article 52 of Additional Protocol I to the Geneva Conventions).
However, determining when a military AI company asset meets these criteria can be difficult, and the bar is high. In the case of data centres, which may provide critical infrastructure to enable AI-enabled military capabilities, it is, for example, debated whether destroying one data centre can be said to offer a ‘definite military advantage’ if the data is stored across other centres. Important to the assessment is also whether the asset provides both civilian and military services. If so, the attacker is obliged to take into account the expected effects on civilians who rely on the services.
Whether military AI industry assets such as data centres qualify as military objectives also matters for states’ obligation to take precautions to protect civilians and civilian objects under their control from the effects of attacks. As pointed out by legal scholars, insofar as data centres are considered legitimate military targets, relying on them for civilian uses will be increasingly hard for states to justify. Thus, states should be mindful about segregating military from civilian data centres, for example by adopting measures to physically separate military servers.
Can military AI companies be held accountable for IHL violations?
The complexity of industry involvement in armed conflict raises questions about when, if ever, suppliers of military AI capabilities can be held accountable for violations of IHL. Can an employee or company entity be held accountable, for instance, where flawed advice from on-site engineers, or the use of biased or outdated AI training data, contributes to an unlawful attack?
The answer is yes: military AI companies and employees can, under certain conditions, be held accountable for violations of IHL under civil or criminal law. It depends, among other factors, on the applicable legal framework and the company’s or individual’s degree of involvement and knowledge about the potential risks of harm. Focusing here on criminal responsibility, individuals, including employees and executives, can incur criminal responsibility for war crimes under either international or domestic criminal law; companies can only incur corporate criminal liability, in certain jurisdictions, under domestic law.
Starting with individual employees, they can be held criminally accountable if their conduct can be sufficiently connected to the armed conflict in which the IHL violation occurred (a so-called nexus requirement). This can either happen via ‘primary’ or ‘secondary’ responsibility regimes.
Primary responsibility, the gravest form of criminal responsibility, may arise where a company employee or executive intentionally commits a war crime (i.e. a serious violation of IHL norms such as of the rules of distinction or proportionality). For instance, an employee deliberately designs an AI-supported targeting system with features that make it prone to targeting civilians, such as giving a high rate of false positives. There could also be scenarios involving multiple actors, such as military AI developers, manufacturers, commanders and/or end users, who work together to develop and use an AI system to commit a war crime. That is, where a range of actors each make essential contributions to a common plan to commit a crime. In such a scenario, they could be held accountable via the doctrine of co-perpetration, as provided under the Rome Statute, as well as some comparable doctrines under national legal systems.
Employee accountability can also arise through ‘secondary’ or ‘accessory’ forms of responsibility where, for example, their conduct assiststhe commission of a war crime committed by military forces (under Article 25 of the Rome Statute). Here, the doctrine of ‘aiding and abetting’ is a particularly relevant framework in light of the kind of assistive and facilitative conduct AI industry actors engage in, not least because the assistive conduct does not necessarily need to take place at the same time or location as the harm inflicted. Establishing this form of responsibility generally requires that individuals’ conduct had a ‘substantial’ effect on the crime committed and that they acted with the knowledge that their conduct assisted in a crime. What amounts to a ‘substantial contribution’ is, however, subject to debate.
Meanwhile, company entities can be held accountable for war crimes via corporate criminal responsibility. Under the majority of national legal systems, criminal responsibility can apply to a company entity itself, rather than exclusively to corporate executives or employees. In the context of military AI, this may be especially significant where unlawful outcomes result from organizational decisions or failures distributed across the company, including in the selection of training data and in system design, testing, deployment and operational support.
While there are indeed avenues to hold military AI companies to account for IHL violations, establishing individual and corporate criminal responsibility faces considerable legal and evidentiary hurdles. These include the difficulty of establishing intentand prior knowledge of the potential for harm resulting from the use of a military AI system.
What responsibilities does the military AI industry have to promote IHL?
States and experts alike have stressed the positive role that industry can play in facilitating IHL compliance in the context of military AI, such as taking IHL into consideration during system design. However, IHL mainly applies to the states and other parties to a conflict. Therefore, any ‘responsibilities’ that industry actors have for promoting IHL compliance are imposed by the state, not by IHL itself.
Some of the obligations IHL places on states may require certain behaviour or specific action from industry. Under Common Article 1 of the Geneva Conventions, states have an obligation to ‘ensure respect’ for IHL, including among private individuals and business actors operating within their jurisdiction. According to the ICRC, the obligation to ensure respect imposes a positive due diligence obligation on a state—meaning it must take all reasonable measures to prevent actors within its jurisdiction from committing or contributing to an IHL violation—whether deliberately or negligently. In the context of military AI companies, this could entail a state adopting measures to regulate their conduct to ensure that capabilities are designed in a manner that minimizes the IHL risks associated with their use.
Besides Common Article 1, states also have an obligation to conduct legal reviews of any new weapons, means and methods of warfare that are developed on the state’s territory, or that the state acquires or uses (as required by Article 36 of the Geneva Conventions). To comply with this obligation, states may require certain behaviour from their industry suppliers, such as transparency regarding the provenance of data used to train AI models or regarding training and programming practices.
Regardless of legal requirements, it should be mentioned that the military AI industry will likely also have commercial and reputational incentives to develop products that can be used in compliance with IHL. Commercial incentives arise where states make IHL compliance a requirement for procurement. Association with a military AI system that contributes to violations of IHL could damage a company’s reputation, with a range of potential negative consequences.
The overarching responsibility of states
Ultimately, however, the burden of responsibility for ensuring compliance with—and preventing violations of—IHL falls on states. Once a state decides to acquire and use a military AI system it is responsible for its lawful use and any effects resulting from that use. Similarly, states’ obligation to ensure respect for IHL means that if a military AI company operating within the state’s jurisdiction violates IHL, the state can be held responsible for failing to take reasonable measures to prevent or respond to the violation.
While the state is not automatically accountable for an IHL violation by a company operating within its jurisdiction, it may become so in cases where either the company acted on the instructions of, or under the direction or control of, the state; or the state acknowledges and adopts the conduct as its own. In the context of military AI, it becomes particularly important to clarify under what circumstances an industry’s conduct becomes attributable to the state, not least in light of the many joint state–industry innovation projects that are emerging.
An urgent need for legal clarity regarding IHL and the military AI industry
For states and military AI companies to uphold respect for IHL in the context of military AI, a more detailed understanding of the IHL implications of the military AI industry’s involvement in armed conflict is urgently needed. Besides familiarizing themselves with existing studies of how IHL applies to private actors, states—individually and potentially at the multilateral level—also need to make and communicate clear determinations around some key issues, starting with the following:
First, states should articulate what they understand IHL to require of industry actors involved in the development and provision of military AI capabilities. Here, existing initiatives on the intersection of human rights and private business offer useful lessons. A good example is the ‘Guiding Principles on Business and Human Rights’ issued by the UN Office of the High Commissioner for Human Rights, which articulates expectations for arms companies to avoid, prevent and mitigate adverse impacts on civilians directly linked to their products. If states do so, military AI companies could then usefully make public commitments to IHL compliance in line with states’ expectations, something the ICRC has already recommended. Clarifying states’ expectations could also be an essential step towards establishing what the obligation to ‘ensure respect’ for IHL entails, as it relates to the conduct of military AI companies.
Second, national and international judicial bodies should seek to clarify under what conditions military AI companies and employees can be held accountable for IHL violations. In doing so, they should explore how existing modes of criminal responsibility apply to military AI industry actors, including the types of technical and operational contributions and levels of knowledge that might give rise to individual and corporate criminal responsibility. As a first step, this should happen at the national level.
Addressing these issues is critical to ensure respect for IHL in the context of military AI and to promote accountability for IHL violations involving the use of AI in armed conflict. And while it, of course, is critical that companies involved in providing military AI capabilities are clear-eyed about their protections and responsibilities under IHL, clarifying their roles and responsibilities will be essential to ensuring that states comply with their obligations to respect and ensure respect for IHL.
ABOUT THE AUTHOR(S)
Laura Bruun is a Senior Researcher in the SIPRI Governance of Artificial Intelligence Programme.
Dr Marta Bo is an Associate Senior Researcher within SIPRI’s Governance of Artificial Intelligence Programme.
Military artificial intelligence (AI) capabilities and services are increasingly being used across a range of military operations to support targeting decisions, among other functions. Because these capabilities are most often provided, enabled and sustained by industry actors, the use of AI in targeting has reignited old debates about how international humanitarian law (IHL) applies to industry—for example, under what circumstances industry actors can lose the protection of IHL and when, if ever, an industry actor can be held accountable for IHL violations.
This backgrounder aims to shed some light on three of the most commonly asked questions. In doing so, it draws on a vast literature on how IHL applies to private sector actors, while also identifying areas that merit further examination. If states wish to integrate AI into their military capabilities, a deep understanding of how IHL applies to the industry supporting these capabilities is essential.
The military AI industry’s involvement in armed conflict
Industry involvement in armed conflict is far from new. States have relied on private companies for war-fighting capabilities, not just for producing weapons but also for ongoing maintenance and training, for centuries. As a result, the IHL implications of industry involvement in armed conflict have also been examined for years, notably by the International Committee of the Red Cross (ICRC).
The rapid advance of AI for military applications has, however, given new urgency to this topic, not least following reports about how AI capabilities, developed and supported by industry actors, have been used in attacks across Gaza, Iran, Ukraine and Venezuela. While the number of publicly reported cases remains small, they suggest a future trend of increasingly complex industry involvement. For example, many AI-enabled products and services will need constant monitoring, updates and refinements from industry suppliers during use. Industry involvement is also likely to become particularly complex in cases where militaries do not have the technical literacy needed to fully understand and independently manage the systems procured for them and thus rely on suppliers’ expertise and advice during use.
This prospect has prompted questions about how IHL applies to the military AI industry. This relates not only to their responsibilities and potential legal accountability, but also to when their personnel or assets may become legitimate targets in warfare. It is crucial that states are equipped to answer these questions. Likewise, it is essential that military AI industry actors themselves understand the legal implications of their involvement, especially the many new entrants to the market that are becoming exposed to IHL for the first time.
When do companies providing or supporting military AI become lawful targets under IHL?
Earlier this year, Iran carried out multiple strikes on data centres in Bahrain and the United Arab Emirates operated by the United States-based company Amazon Web Services. Although the data centres provided a wide range of civilian services, Iran reportedly claimed its attacks were intended ‘to identify the role of these centres in supporting the enemy’s military and intelligence activities’. These attacks brought renewed attention to the question of when industry actors can become legitimate military targets.
As a legal baseline, private companies—specifically their employees and assets—have civilian status and are protected under IHL. However, under certain circumstances, they may lose their protections and become legitimate military objectives (i.e. targets) from the perspective of IHL.
Starting with employees, they can lose protection if they become members of the armed forces, if the company itself belongs to a party to the conflict, or if their activities amount to ‘direct participation in hostilities’ (DPH; see Article 51 of Additional Protocol I to the Geneva Conventions). DPH is particularly relevant in the context of military AI industry personnel, considering, for example, military forces’ reliance on ‘forward-deployed engineers’ who work alongside military personnel to adapt and support AI-enabled systems during use. Under the ICRC’s three-part test, to qualify as DPH, three criteria must all be fulfilled:
1) the act must reach a threshold of harm (such as inflicting death, injury or destruction on protected persons and/or objects);
2) there must be a direct causal link between the act and the harm, and;
3) the act must be specifically designed to cause harm.
These criteria have traditionally been applied to civilians, for example, when they pick up arms to join one side of the conflict. It is unclear, albeit increasingly relevant, what activities carried out by tech workers, including those from military AI companies, would satisfy the DPH criteria.
When it comes to a military AI company’s assets, they may lose their protections under IHL if they ‘make an effective contribution to military action and [their] total or partial destruction, capture or neutralization, in the circumstances ruling at the time, offers a definite military advantage’ (Article 52 of Additional Protocol I to the Geneva Conventions).
However, determining when a military AI company asset meets these criteria can be difficult, and the bar is high. In the case of data centres, which may provide critical infrastructure to enable AI-enabled military capabilities, it is, for example, debated whether destroying one data centre can be said to offer a ‘definite military advantage’ if the data is stored across other centres. Important to the assessment is also whether the asset provides both civilian and military services. If so, the attacker is obliged to take into account the expected effects on civilians who rely on the services.
Whether military AI industry assets such as data centres qualify as military objectives also matters for states’ obligation to take precautions to protect civilians and civilian objects under their control from the effects of attacks. As pointed out by legal scholars, insofar as data centres are considered legitimate military targets, relying on them for civilian uses will be increasingly hard for states to justify. Thus, states should be mindful about segregating military from civilian data centres, for example by adopting measures to physically separate military servers.
Can military AI companies be held accountable for IHL violations?
The complexity of industry involvement in armed conflict raises questions about when, if ever, suppliers of military AI capabilities can be held accountable for violations of IHL. Can an employee or company entity be held accountable, for instance, where flawed advice from on-site engineers, or the use of biased or outdated AI training data, contributes to an unlawful attack?
The answer is yes: military AI companies and employees can, under certain conditions, be held accountable for violations of IHL under civil or criminal law. It depends, among other factors, on the applicable legal framework and the company’s or individual’s degree of involvement and knowledge about the potential risks of harm. Focusing here on criminal responsibility, individuals, including employees and executives, can incur criminal responsibility for war crimes under either international or domestic criminal law; companies can only incur corporate criminal liability, in certain jurisdictions, under domestic law.
Starting with individual employees, they can be held criminally accountable if their conduct can be sufficiently connected to the armed conflict in which the IHL violation occurred (a so-called nexus requirement). This can either happen via ‘primary’ or ‘secondary’ responsibility regimes.
Primary responsibility, the gravest form of criminal responsibility, may arise where a company employee or executive intentionally commits a war crime (i.e. a serious violation of IHL norms such as of the rules of distinction or proportionality). For instance, an employee deliberately designs an AI-supported targeting system with features that make it prone to targeting civilians, such as giving a high rate of false positives. There could also be scenarios involving multiple actors, such as military AI developers, manufacturers, commanders and/or end users, who work together to develop and use an AI system to commit a war crime. That is, where a range of actors each make essential contributions to a common plan to commit a crime. In such a scenario, they could be held accountable via the doctrine of co-perpetration, as provided under the Rome Statute, as well as some comparable doctrines under national legal systems.
Employee accountability can also arise through ‘secondary’ or ‘accessory’ forms of responsibility where, for example, their conduct assists the commission of a war crime committed by military forces (under Article 25 of the Rome Statute). Here, the doctrine of ‘aiding and abetting’ is a particularly relevant framework in light of the kind of assistive and facilitative conduct AI industry actors engage in, not least because the assistive conduct does not necessarily need to take place at the same time or location as the harm inflicted. Establishing this form of responsibility generally requires that individuals’ conduct had a ‘substantial’ effect on the crime committed and that they acted with the knowledge that their conduct assisted in a crime. What amounts to a ‘substantial contribution’ is, however, subject to debate.
Meanwhile, company entities can be held accountable for war crimes via corporate criminal responsibility. Under the majority of national legal systems, criminal responsibility can apply to a company entity itself, rather than exclusively to corporate executives or employees. In the context of military AI, this may be especially significant where unlawful outcomes result from organizational decisions or failures distributed across the company, including in the selection of training data and in system design, testing, deployment and operational support.
While there are indeed avenues to hold military AI companies to account for IHL violations, establishing individual and corporate criminal responsibility faces considerable legal and evidentiary hurdles. These include the difficulty of establishing intent and prior knowledge of the potential for harm resulting from the use of a military AI system.
What responsibilities does the military AI industry have to promote IHL?
States and experts alike have stressed the positive role that industry can play in facilitating IHL compliance in the context of military AI, such as taking IHL into consideration during system design. However, IHL mainly applies to the states and other parties to a conflict. Therefore, any ‘responsibilities’ that industry actors have for promoting IHL compliance are imposed by the state, not by IHL itself.
Some of the obligations IHL places on states may require certain behaviour or specific action from industry. Under Common Article 1 of the Geneva Conventions, states have an obligation to ‘ensure respect’ for IHL, including among private individuals and business actors operating within their jurisdiction. According to the ICRC, the obligation to ensure respect imposes a positive due diligence obligation on a state—meaning it must take all reasonable measures to prevent actors within its jurisdiction from committing or contributing to an IHL violation—whether deliberately or negligently. In the context of military AI companies, this could entail a state adopting measures to regulate their conduct to ensure that capabilities are designed in a manner that minimizes the IHL risks associated with their use.
Besides Common Article 1, states also have an obligation to conduct legal reviews of any new weapons, means and methods of warfare that are developed on the state’s territory, or that the state acquires or uses (as required by Article 36 of the Geneva Conventions). To comply with this obligation, states may require certain behaviour from their industry suppliers, such as transparency regarding the provenance of data used to train AI models or regarding training and programming practices.
Regardless of legal requirements, it should be mentioned that the military AI industry will likely also have commercial and reputational incentives to develop products that can be used in compliance with IHL. Commercial incentives arise where states make IHL compliance a requirement for procurement. Association with a military AI system that contributes to violations of IHL could damage a company’s reputation, with a range of potential negative consequences.
The overarching responsibility of states
Ultimately, however, the burden of responsibility for ensuring compliance with—and preventing violations of—IHL falls on states. Once a state decides to acquire and use a military AI system it is responsible for its lawful use and any effects resulting from that use. Similarly, states’ obligation to ensure respect for IHL means that if a military AI company operating within the state’s jurisdiction violates IHL, the state can be held responsible for failing to take reasonable measures to prevent or respond to the violation.
While the state is not automatically accountable for an IHL violation by a company operating within its jurisdiction, it may become so in cases where either the company acted on the instructions of, or under the direction or control of, the state; or the state acknowledges and adopts the conduct as its own. In the context of military AI, it becomes particularly important to clarify under what circumstances an industry’s conduct becomes attributable to the state, not least in light of the many joint state–industry innovation projects that are emerging.
An urgent need for legal clarity regarding IHL and the military AI industry
For states and military AI companies to uphold respect for IHL in the context of military AI, a more detailed understanding of the IHL implications of the military AI industry’s involvement in armed conflict is urgently needed. Besides familiarizing themselves with existing studies of how IHL applies to private actors, states—individually and potentially at the multilateral level—also need to make and communicate clear determinations around some key issues, starting with the following:
First, states should articulate what they understand IHL to require of industry actors involved in the development and provision of military AI capabilities. Here, existing initiatives on the intersection of human rights and private business offer useful lessons. A good example is the ‘Guiding Principles on Business and Human Rights’ issued by the UN Office of the High Commissioner for Human Rights, which articulates expectations for arms companies to avoid, prevent and mitigate adverse impacts on civilians directly linked to their products. If states do so, military AI companies could then usefully make public commitments to IHL compliance in line with states’ expectations, something the ICRC has already recommended. Clarifying states’ expectations could also be an essential step towards establishing what the obligation to ‘ensure respect’ for IHL entails, as it relates to the conduct of military AI companies.
Second, national and international judicial bodies should seek to clarify under what conditions military AI companies and employees can be held accountable for IHL violations. In doing so, they should explore how existing modes of criminal responsibility apply to military AI industry actors, including the types of technical and operational contributions and levels of knowledge that might give rise to individual and corporate criminal responsibility. As a first step, this should happen at the national level.
Addressing these issues is critical to ensure respect for IHL in the context of military AI and to promote accountability for IHL violations involving the use of AI in armed conflict. And while it, of course, is critical that companies involved in providing military AI capabilities are clear-eyed about their protections and responsibilities under IHL, clarifying their roles and responsibilities will be essential to ensuring that states comply with their obligations to respect and ensure respect for IHL.
ABOUT THE AUTHOR(S)