AI technologies Mythos and Maven have been changing the world in just the past three months.
In Iran, Maven has sped up how America wages war. "AI is in the kill chain," wrote Major Matthew Jefferies on the Australian army's professional development platform Cove, warning his country had to act now to keep up with the US.
From Silicon Valley, Mythos has sped up cyber attacks. "The latest models are beginning to show signs they could escape human control," AFP reported on Friday as Anthropic suggested a global pause.
Anthropic makes both Mythos and Claude, an AI model that has underpinned Maven.
The big picture is changing rapidly for everyone, including for the rulemakers of war.
"It's very much a transitional moment," said arms researcher Jeremy Moses, associate professor in international relations at Canterbury University.
"We've got some way to run before some of the issues around autonomous war and drone warfighting become clearer."
Doctrine on drones
A newly released document from the New Zealand Defence Force (NZDF) shows it has no doctrine on robotic autonomous systems (RAS) and was looking to the government to guide their ethical use.
The NZDF should consider new doctrine, tactics and operational guidance, it said.
Released under the Official Information Act 1982
Enclosure One Ethics
Ethics.
RAS have ethical implications. Some are practical, such as the technical capability for RAS making decisions aligned with authorities set for using them. Others are moral, particularly when systems’ decision-making includes using lethal force, which can be weighed against humans doing tasks in high-risk situations that RAS could otherwise do. Whatever the case might be, Government policy will set the NZDF ethical expectations. The potential for RAS to reduce NZDF casualties could create legal and ethical obligations for deploying RAS rather than Service people. RAS could more strictly adhere to International Humanitarian Law (IHL) than humans when planning and executing attacks, and eventually States might be legally obliged to use RAS in all such cases.1 The debate on this topic is not settled, and the situation it imagines is not expected in the foreseeable future. Science fiction has influenced wide-ranging debate about the risks of RAS. The NZDF must inform the public about how RAS and humans interact. Open and transparent dialogue is needed between Government, Defence, and the public to improve understanding and build trust in the benefits and lawful uses of RAS.
The central idea’s defining feature is the NZDF using RAS in human-commanded teams. Depending on the operational scenario and objectives, human commanders determine the extent to which personnel control RAS in the human-machine team. The NZDF can gain advantages by focussing on using RAS for increasing efficiency, generating mass, achieving decision superiority, and decreasing risk to personnel. Human-commanded teams should also help the NZDF mitigate technological and ethical challenges.
Doctrine
Concept development identified topics needing further study and analysis, including implications of RAS for operating concepts, policy, doctrine, workforce, and the capability life cycle. Without such development, the NZDF might not gain as many RAS advantages and its risks could increase, both that of falling behind potential competitors, and of losing opportunities for interoperability with partners.
Meaningful human control will be maintained for NZDF RAS that can use lethal force. NZDF C2 will need to be viable in degraded and contested environments, joint force deployable, and interoperable while balancing need-to-know and need-to-share. Doctrine. The NZDF should consider developing RAS doctrine. The Services must evaluate current tactical level doctrine and tactics, techniques, and procedures (TTPs), and generate new operational guidance as required.
1 Balme J. H., The Interpretation and Application of LOAC in Relation to Autonomous Weapons Systems, 2020.
"The NZDF should consider developing RAS doctrine," it said.
"The services must evaluate current tactical level doctrine and tactics, techniques, and procedures ... and generate new operational guidance as required."
It added that government policy will "set the NZDF ethical expectations".
It released a single page in response to RNZ's request for its latest half dozen documents about drone doctrine and ethics.
It said the page came from a single "concept" document that it withheld in part for national security reasons.
For the government policy on drone warfare ethics, Defence Minister Chris Penk referred RNZ to the Ministry of Foreign Affairs and Trade. MFAT has various documents on its website, mostly about New Zealand's position on autonomous weapons for international forums rather than specific to the NZDF.
"It sets out our preference for legally binding rules and limits on autonomous weapon systems," the ministry told RNZ.
More lethal
Moses was not surprised by the NZDF's single page.
"They haven't engaged deeply," he said.
"I'm not sure that if they put increased resources into thinking about the ethics of autonomous warfare right now that that would necessarily end up being a useful exercise.
"The United States does have developed policies and doctrines around these things, but it's ultimately meaningless in how they fight wars."
America's push to be more lethal was more important, and how that was influencing Australia and New Zealand spending and stance, Moses said.
"The more that you immerse your political leadership with that system, the more likely you're going to adopt the same kind of thinking, which is that you want to be as destructive as possible."
New Zealand's plan for building defence capability up to 2029 stresses enhanced lethality.
Previous Defence Minister Judith Collins - in a speech earlier this year - said: "We are focusing on more than doubling our defence spend and investing in a defence force that is combat capable with enhanced lethality and deterrence; a force multiplier with Australia and increasingly interoperable with partners."
Powerful influences
The single page on ethics and doctrine released by the NZDF focuses on keeping "the human in the loop".
"Human commanders determine the extent to which personnel control" the machine, it said.
"Meaningful human control will be maintained for NZDF RAS that can use lethal force."
The page also notes that:
- Robot systems might be able to stick to humanitarian law better than people in planning and executing attacks, so that eventually all states might be obliged to use them.
- They could cut down casualties among personnel.
- Science fiction was influencing the debate about the risks.
The NZDF document did not account for the more powerful influence of the Ukraine, Gaza and Iran wars on the debate and on military decision-making.
Drones, links to satellites and new US kamikaze drones used in Iran that were linked to satellites linked to Maven have raised new questions about speed and autonomy, while also fuelling the pursuit of them.
The South China Morning Post reported last month that Beijing had developed an algorithm that "promises to allow a fleet of fixed-wing drones to autonomously search a vast battlefield and eliminate every single enemy".
In Iran, the Pentagon says its central Maven AI system co-ordinated 13,000 strikes in 38 days - one every four minutes on average, often by missiles. The Department of War maintained humans always made the final decision to strike.
"Maven not only identifies targets - it tells commanders how to attack them, too," wrote The Economist.
'Closer to supervision'
The debate has included a neuroscientist in the MIT Technology Review suggesting this was an "illusion of human oversight" because when agentic AI did targeting, it would consider "hidden" factors the human does not know about
Katrina Manson's new book Project Maven charts how the system's creator always wanted it for targeting.
A lot of military commanders used the "human in the loop" phrase, Manson said in an interview, but it was not technically the policy of the Defence Department.
That 2023 policy said systems would be subject to "appropriate levels of human judgement over the use of force".
"So it implies something a little closer to supervision," she said.
Commentators have dubbed this "human on the loop" instead of "in" it.
From experiment to essential
Professor Moses said old questions about lethality and autonomy of the past decade had fallen to the wayside in the face of the AI wars.
"That's part of a global trend. We can now start to see what drone warfare looks like, but it's not really closely related to those issues of autonomy that people have been focusing on... that's partly why that debate hasn't really progressed over the last 15 years."
Iran was Maven's first outing on the battlefield, but it has been experimented with in exercises run by the US and involving Australia and New Zealand - and the UK and others - for at least several years.
The NZDF will send 50 personnel shortly to one such army exercise - Project Convergence Capstone - in the western US.
The Australian Defence Force (ADF) said last week it was testing Maven, with its AI function turned off.
The US Navy's Project Overmatch uses Maven, and the NZDF is helping plan a big exercise for that.
The US Navy said the work would enable "resilient communication and network connectivity amongst the Five Eyes (FVEY) in a distributed environment to close kill-chains and enable long -range fires".
NZDF agreed, but did not use the words "close kill-chains".
The 2027 Pentagon budget aims to put US$4 billion into Maven and a 'Joint Fires Network' to deliver the combined allies command-and-control system called CJADC2.
The NZDF is adding new drones to its small stable, with Budget 2026 having unspecified funding for new surveillance drones.
It also has "loitering" drones that are armed and act like bombs, has not ruled out acquiring other types of lethal drones and will be training in the US with aerial drones that have "autonomous terminal guidance" - a system that helps hit targets.
Moses said the systems kept advancing. but not the thinking about them.
"There is a lot of wind around this ethics stuff... Australia has published a heap of this stuff, but the substance of it is still pretty thin, pretty sort of boilerplate language."
'Debate about whether it should be is already over'
But for operations, the message was cutting edge and crystal clear, wrote Major Jefferies in April.
It had been spelt out by the US in the Iran war and snatching Venezuela's president in January in an AI-enabled raid.
"Operational speed and decision advantage will not be sacrificed for civilian-mandated safety frameworks. AI is in the kill chain," said Jefferies.
"The debate about whether it should be is already over.
"How it operates in the kill chain is likely to differ between coalition partners.
"If the ADF cannot operate at allied tempo, it loses meaningful sovereignty, even if it retains legal authority."
Think more, talk more
What might this mean for New Zealand?
The single page on drone doctrine and ethics from the NZDF is clear on two things: There has to be more thinking, and the public must be kept in the loop.
"Without such development, the NZDF might not gain as many RAS advantages and its risks could increase, both that of falling behind potential competitors, and of losing opportunities for interoperability with partners," it said.
"The NZDF must inform the public about how RAS and humans interact," it added.
"Open and transparent dialogue is needed between government, Defence, and the public to improve understanding and build trust in the benefits and lawful uses of RAS."
But intersections for public debate remain limited. An autonomous weapons advisory group to government was shut down in 2024.
Disarmament education grants of $100,000 a year were dropped entirely from Budget 2026 after 22 years.
Defence Minister Penk, asked what if anything he has asked officials to do in this space, referred RNZ's questions to the Ministry of Foreign Affairs and Trade (MFAT).
MFAT said in a statement on Monday the country's policy position on autonomous weapon systems was determined by Cabinet in 2021 and reaffirmed in 2024. It covered the country's international position and engagements.
In a short section on ethics in New Zealand's submission to the UN about autonomous weapons systems, it said, "A variety of overlapping and sometimes competing concepts, including 'sufficient' and 'meaningful' human control, and 'appropriate human judgement' or 'involvement' have been debated.
"While general awareness of the issues has increased, New Zealand would observe that the international community appears some way from a definitive answer, despite the stakes."
