Military AI Is a Democracy Test Humans Can’t Afford to Fail
Trust Power Accountability
By Sarra HannachiOctober 2026 · 14 min read
Military AI has no democratic mandate of its own, yet it is already reshaping wartime decision-making, whether through defence and AI policy or in active operations, including those on the battlefield. It has taken time, loss, and strength for today’s democracies to reach their goals and make durable change. The cost has been so high that humans cannot afford to lose what has been won, or to surrender the legitimacy that comes with it to the world’s most powerful and opaque technology.
Democratic legitimacy rests primarily on trust, power, and accountability. These elements can lose their purpose when defence institutions allow military AI systems to analyse intelligence, plan military action, and make critical decisions. The loss is much greater when AI executes decisions without agreed-upon intentions and acts faster than oversight can keep pace, all for the sake of urgency, performance, and security. Military AI is a test, and democracies can fail it terribly, even for the right reasons.
I
Trust
Can defence mirror democracy through trust?
Failure looks likeFailing to build trust and maintain transparency with citizens and the institutions involved
II
Power
Who or what is truly in command?
Failure looks likeLacking meaningful human control over machine power, especially in real-time applications
III
Accountability
The rule of law or the rule of prompt?
Failure looks likeFailing to enforce clear accountability measures for the grave consequences of failure
Part ITrust
Can defence mirror democracy through trust?
While defence is a core function and responsibility of government, it is also one of the ways in which a democracy is embodied. Defence strategies and military actions are just as much a reflection of democracy as electoral processes.
When the UK government deployed military troops to Iraq with the US, it made the decision despite failing to secure a second United Nations Security Council (UNSC) resolution after Resolution 1441 (November 2002) had set out Iraq’s disarmament obligations. If passed, that second resolution would have authorised military force. The UK’s decision relied on intelligence emphasising non-cooperation with inspectors and the threat of weapons of mass destruction (WMDs) and ballistic missiles. After seven years of investigation, the Iraq Inquiry (the Chilcot Report) concluded that the threats were ‘unjustified’ and based on ‘insufficient knowledge’. It is no longer 2003. Today, that same gap between decision and disclosure exists, but it takes a different shape. AI-equipped governments and intelligence agencies now make decisions in seconds, not years. Those decisions often rest on unverified processes or results and insufficient information.
In 2026, Anthropic’s CEO Dario Amodei said in a statement that the AI approaches of the Department of War (DoW) did not align with US democratic values. This signalled clear distrust of the DoW’s understanding of AI. The statement argued that, given AI’s current nature and capabilities, it should not be deployed for mass surveillance or autonomous weapons systems (AWS). Facial recognition systems used in mass surveillance risk algorithmic bias, because historical data infected with racially discriminatory labels can be reflected in these systems’ outputs, creating a dangerous loop of AI-dependent profiling that undermines algorithmic fairness. The UK Information Commissioner’s Office has highlighted this discriminatory practice, along with the use of imbalanced training data. Mass surveillance also raises serious privacy concerns, as shown by the surveillance operations flagged by Anthropic, including one that led Anthropic to ban an intelligence vendor from using Claude to build a social media surveillance platform targeting Iranian users.
On another note, autonomous weapons systems pose a distinct and potentially more immediate risk. They can take over critical judgements about the use of force that have traditionally been entrusted to the military. They can take control of the entire execution and force-engagement action without oversight or override by human operators. Because of these features, autonomous weapons systems pose numerous challenges, including safety concerns for both combatants and civilians, ethical concerns about delegating life-or-death decisions, and legal concerns about compliance with International Humanitarian Law (IHL).
Anthropic released the statement following the dispute with the Pentagon over safety guardrails. The Pentagon regarded Anthropic’s safety guardrails as an obstacle to national security. This case shows that distrust can come from the AI industry, not only from citizens. The Pentagon responded to this distrust by labelling Anthropic a national security threat and a supply-chain risk. This challenges democratic practice within the defence sector, both internally and externally.
Technology is transferable, whether through its infrastructure, the data or hardware that operate it, its applications or its intellectual property. If an AI technology strips away democratic principles in one military application, can that erosion be transferred to another military application? The UK’s strategic defence review of 2026 should address this question, given that Prime Minister Burnham announced a new UK-US defence and AI partnership during the 81st UN General Assembly.
In other contexts, democratic fragility manifests in ways beyond refusing safeguards. These include a lack of transparency about which technologies are deployed, which channels they pass through at each stage, and why. China is a particular case, as it does not share Western-style democracy and oversight. China emphasised its military-civil fusion (MCF) strategy, first introduced as civil-military integration (CMI), in a white paper on military strategy in 2015. Over the last decade, China’s military intelligentization process has increasingly relied on semiconductors and AI systems built by civilian enterprises. Despite the government’s disclosure of requests for proposals (RFPs), the line between civilian and military use remains blurred. In public contract awards, non-traditional commercial vendors without state ownership accounted for nearly 75% of procured companies between 2023 and 2024. China has pursued this multi-layered approach to AI since it released the “New Generation Artificial Intelligence Development Plan” in 2017. From an early stage, the plan emphasised internalising the process and deepening the military-civil (con)fusion in AI. This strategy leaves the public unaware of, or unsure about, whether a purchased AI solution is integrated into military procurement and use.
These examples underscore the need for democratic safeguards in military AI strategy. Democracy existed long before the emergence of AI. Today, the difficulty arises when the public questions the appropriateness of military AI, because governments can treat AI as a crucial competitive advantage against adversaries. Yet public opinion tells a different story. For instance, in the US, a 2026 survey shows that three-quarters find AI not reliable enough to trust with life-or-death military decisions without human oversight. The survey suggests that people are highly unlikely to trust what they do not understand. When it comes to AWS, the survey finds that 49% agree that the US should research AWS but not deploy them. Only 21% support further developing and deploying AI-controlled weapons.
Trust in AI is fragmented and depends on application context, technological capacity and public understanding. But can the public express doubts clearly to democratically elected officials? Can citizens no longer ask questions about military technology because it is presented as a tool of protection? Or do governments withhold answers because they anticipate greater security vulnerabilities? Both possibilities lead to a democratic dead end. If a government limits access to this information, other measures may be necessary to rebuild trust. To preserve democratic values, would it be enough to place military AI under meaningful command and oversight?
Part IIPower
Who or what is truly in command?
The redistribution of power from institutions to algorithms signals a shift in the roles of humans and machines. AI can perform many tasks in a military context, whether operational or strategic, depending on the deployment mode and military objective. In both cases, AI-enabled tasks can help analyse and execute a military course of action (CoA) in real time. When AI is deployed in these settings, cognitive power and decision-making authority tend to shift to the system.
Long before AI entered defence, NATO’s US-led Operation Allied Force (OAF) against the Federal Republic of Yugoslavia, launched in March 1999, exemplified a growing reliance on the military industry. The OAF deployed the B-2 Spirit, a stealth strategic bomber equipped with radar and sensors designed for low observability. When it was unveiled in the late 1980s, the B-2 shocked the public because of ten years of government secrecy and the bypassing of congressional approval. This points to the fragility of democratic integration within the US military-industrial complex. In 1961, Eisenhower warned about the complex’s influence on military procurement, ‘whether sought or unsought’, which could lead to ‘misplaced power’.
The B-2 incident in Belgrade during the OAF echoes Eisenhower’s concerns. On 7 May 1999, the US Air Force used the B-2 Spirit for a strike on a target in Belgrade, mistakenly bombing the Chinese Embassy and leading to 27 casualties, including 3 killed. It was a grave humanitarian and diplomatic incident. Following the investigation, in June 1999 the then Under Secretary of State, Thomas Pickering, briefed the Chinese government. He set out the main reasons for the mistake. First, there was ‘mistargeting’: a flawed land navigation technique was used to identify the target’s coordinates. Second, there were ‘flawed databases’ of civilian and diplomatic facilities, including embassies. Lastly, there were ‘faulty checks’: the review prioritised military value and failed to detect the error. The B-2 did not autonomously decide to engage the target; humans authorised the strike. But its capabilities at the time may have exerted enough power over its operators to dispel doubts about precision. Its development was withheld from congressional and public scrutiny. Its use on the battlefield rested on flawed information, and its review process favoured overconfidence over the detection of targeting errors.
01
Mistargeting
A flawed land navigation technique was used to identify the target’s coordinates.
02
Flawed databases
Databases of civilian and diplomatic facilities, including embassies, were flawed.
03
Faulty checks
The review prioritised military value and failed to detect the error.
7 May 1999
Chinese Embassy struck
A B-2 strike in Belgrade leads to 27 casualties, including 3 killed.
Today, AI intensifies these risks of loss of control by increasing the speed, scale, and opacity with which military systems can influence or make decisions. In the military domain, it is a question of both meaningful control and meaningful command, because both shape military action. There are two distinct types of power that AI can now seek in order to take command:
There is intended power, and there is rogue power.
Intended power is power that a human commander intentionally delegates to the machine. Nonetheless, even the intended delegation of authority in a military operation or task can lead to unintended consequences. In other words, unintended consequences are not limited to extreme cases such as lethal targeting. For instance, US senators called for an investigation in September 2026 after the US military reportedly relied on an AI-generated intelligence report falsely identifying nuclear weapons on board a Chinese cargo ship in the Middle East. The geopolitical escalation that could have followed this incident would have been profoundly destabilising, both diplomatically, through false accusations, and democratically, by handing power to flawed intelligence. This demonstrates that AI systems deployed in military operations indirectly claim power in proportion to the responsibility and resources allocated to them.
Then, there is rogue power, self-claimed and not delegated. Several incidents reported in summer 2026 illustrate rogue AI activity, especially within government institutions and bodies. The UK government’s AI Security Institute (AISI) reported unsanctioned multi-agent behaviour during a cyber security test, in which the agents used deceptive tactics to solve the cyber challenge. During this testing and evaluation exercise, the agents attempted to send malicious code to real organisations and to plant unauthorised prompts for other agents in order to manipulate cyber-attacks. Later, Australian Prime Minister Anthony Albanese announced in September 2026 that an OpenAI agent had ‘infiltrated’ an Australian government website and accessed public and non-public files from the Medicare Statistics Reporting Portal. PM Albanese described the incident as unacceptable, especially given that OpenAI disclosed the hacking only three months after it took place. While OpenAI claims the action taken by its models during internal evaluation was not intended or authorised, the incident still raises concerns about digital sovereignty, integrity and notification obligations, particularly given that OpenAI has been a key contractor and a supplier of AI solutions to the US DoW’s classified networks since the February 2026 deal.
Delegated
Intended power
Power that a human commander intentionally delegates to the machine.
- Risk
- Unintended consequences, not limited to extreme cases such as lethal targeting.
- Example
- An AI-generated intelligence report falsely identifying nuclear weapons on board a Chinese cargo ship (September 2026).
Self-claimed
Rogue power
Power that is self-claimed and not delegated.
- Risk
- Concerns about digital sovereignty, integrity and notification obligations.
- Examples
- Unsanctioned multi-agent behaviour in an AISI cyber security test; an OpenAI agent accessing files on an Australian government portal.
From intelligence gathering to command and control (C2) and military decision-making processes, AI has been shown to exploit all the power it can access and, in some cases, even to compress necessary tasks and redirect power between them. Among these tasks and processes, AI-driven targeting differs from intelligence and cyber security operations because it depends more on real-time action. The high-risk autonomous transition from decision to action compresses the interval between assessment and action, potentially leaving too little time for meaningful human review.
Autonomous systems used for these military functions depend on data that is abundant, precise, and cross-verifiable. Today, data fed into autonomous AI is unlikely to meet all three conditions at once. That gap exists because operational data can often be incomplete, biased or difficult to verify. Data is power, and AI is a technology that can use the power and authority of data to amplify outputs and claim success, with or without human oversight. Given that the fair distribution of power is a core democratic value, an autonomous AI-powered military CoA cannot by itself satisfy democratic principles.
Whether operating within an authorised role or acting independently, AI exceeds human capacity to collect, manage, and process large volumes of data, including images, sensor feeds, documents, and structured and unstructured data. AI therefore displaces at least a portion of a human operator’s role in command, along with the leverage that comes from possessing the modern world’s key commodity: data. Big data. Citizens’ data? Possibly. However, when a population helps to place power in a government responsible for national security, it expects authorised military commanders and officials, not AI, to lead military planning and action. It expects them to establish the pillars and authorise priorities for defence and security.
As AI systems advance, their growing capacities tend to persuade military leaders that their democratic loopholes are simply a small price to pay for efficiency and responsiveness. In doing so, they conceal the erosion of meaningful human command and accountable military authority. In an environment that prioritises optimisation and speed, this erosion challenges democratic oversight. The limited readiness of governance and institutional accountability mechanisms does not help.
Part IIIAccountability
The rule of law or the rule of prompt?
Accountability is a foundational democratic value, alongside transparency and integrity. However, it is particularly easy to undermine when defence institutions fail to protect it. The black-box nature of AI is a major threat to accountability, yet it is presented as an inevitability.
Accountability for military failures has traditionally relied on formal investigative procedures. An investigation typically establishes a case-specific commission of inquiry that coordinates expert opinion and evidence. The goal is to uncover the true flow of events, the individuals or units responsible for them, and the resulting consequences. Historically, military commissions of inquiry aimed to limit the ‘war power’ of military commanders and officers in charge of a particular operation, strategy or course of action.
In recent years, multilateral efforts to govern military AI have intensified. The UN-led Convention on Certain Conventional Weapons (CCW) Group of Governmental Experts (GGE) on Lethal Autonomous Weapons Systems (LAWS) concluded a critical session on 4 September 2026. The group reached consensus on a final report at the end of its three-year mandate. The report sets out elements on the characteristics of LAWS, IHL principles, human control, and accountability measures. Ahead of the UK Prime Minister’s first appearance at the 81st UN General Assembly in September 2026, the Joint Committee on Human Rights called for a new AI Bill to address the serious threats that AI systems pose to human rights. The committee advised that the Bill should apply across the entire AI lifecycle.
Regional and coalition-driven efforts to establish military AI standards have also emerged. One example is NATO’s announcement of the NATO-Industry Cooperation Strategy at the 2026 NATO Summit in Ankara. The strategy outlines NATO’s vision for AI integration through Task Force X. NATO is therefore considering developing norms, with its member states, on military AI development and use. However, the rapid pace of AI development makes detailed norms less likely in the short run.
AI governance efforts such as the EU AI Act exclude military applications of AI. In response, states have contributed independently to the discussion on military AI governance. This is how the Responsible AI in the Military Domain (REAIM) summit convened the Global Commission on Responsible Artificial Intelligence in the Military Domain (GC REAIM). The GC REAIM examines the implications of military AI, provides strategic recommendations and contributes to military AI diplomacy. The REAIM summit has been supported by the Netherlands, Spain, South Korea, Singapore and, most recently, Kenya. In 2027, Kenya will become the first African country to host the REAIM Summit. These initiatives offer new and thought-provoking insights on AI in the military domain and on multilateral agreements that move beyond competitive framing.
However, the proposed and reviewed frameworks have not yet demonstrated concrete impact. In fact, recent cases have shown how difficult it is to distinguish human from AI responsibility. Two major military failures linked to AI systems have demonstrated the ‘collapse of accountability’. First, Israeli operations in Gaza and Lebanon generated 850,000 real-time targets. Second, the US strikes on Iran in early 2026 resulted in the wrongful targeting of a school, killing 175 to 180 civilians, mostly children. To date, neither case has led to public legal accountability. This reflects the limited readiness and effectiveness of military AI governance frameworks for pursuing accountability after failure or negligence. In part, accountability may be limited deliberately, by invoking technological complexity to deflect responsibility, escape IHL obligations and attribute errors to the algorithms alone.
Democracy wasn’t built in a day. Clearly, any progress on complex issues such as AI and the military requires a process. Every such process involves milestones and setbacks. What poses a real hazard for democracy is the exploitation of those setbacks to evade accountability rather than addressing them with solutions. If democracies today judge an AI-powered military failure only by its intended outcome and not by the means that produced it, then the rule of the AI prompt has quietly begun to replace the rule of law. In the absence of real accountability today, AI proliferation and misuse are only likely to intensify in the future.
Closing
Conclusion
In the end, military AI is a democracy test with no cheat code.
The scalability of military AI integration makes it difficult for democracies to assess a deployment or use case before failure ripples through an entire chain of assessments, decisions, and actions. In an age of fast-paced technological development, military strategy and operational decision-making have become as opaque as the AI systems in use.
If nations fail to use AI safely and responsibly, they fail to uphold their own democratic values. In the context of military AI, failing democracy means failing to build trust and maintain transparency with citizens and the institutions involved. It also means lacking meaningful human control over machine power, especially in real-time applications. Failing democracy also signifies failing to enforce clear accountability measures for the grave consequences of failure. AI remains poorly understood in some civilian settings, let alone in military use, making it harder for the public and relevant AI governance institutions to identify risks and respond before the harmful consequences become irreversible.
So, are democracies willing to apply the same urgency to military AI oversight as they do to electoral oversight? Or are they more concerned with global standing and strategic advantage than with maintaining public trust and democratic legitimacy? The answer will depend on how seriously democracies treat trust, power, and accountability when it comes to military AI.
About the author
Cite this articleHannachi, S. (2026, October). Military AI is a democracy test humans can’t afford to fail. GCRD Insights. Global Centre for Rehumanising Democracy.
Published by the Global Centre for Rehumanising Democracy (GCRD) in the GCRD Insights series.
The views expressed in this article are those of the author and do not necessarily reflect the position of the Global Centre for Rehumanising Democracy.
Global Centre for Rehumanising Democracy
gcrd.org.uk