Somewhere over the Black Sea in 2024, a Ukrainian drone identified a Russian military vehicle, calculated the optimal attack angle, adjusted for wind speed, and struck its target — all without a human pulling the trigger. The operator had designated the target, but the final attack decision was made by software. In military parlance, this is a "human on the loop" system: a person supervises, but the machine acts.
The line between "human-controlled weapon" and "autonomous weapon" is getting blurrier every month. And the international community has run out of time to draw it clearly.
What's Already Deployed
The public discourse around autonomous weapons often imagines futuristic killer robots. The reality is both more mundane and more unsettling:
Loitering munitions (also called "kamikaze drones") like the Switchblade 600 and the Lancet can identify and track targets using onboard AI. Some variants can autonomously select targets from a predefined category — "any vehicle matching this profile" — without human confirmation for each strike.
Defensive systems like Israel's Iron Dome and the U.S. Navy's Aegis system have operated with autonomous engagement capability for years. When an incoming missile gives you seconds to respond, there's no time for human decision-making. These systems detect, classify, and intercept threats autonomously.
Drone swarms are being developed by the U.S., China, and several other countries. Hundreds or thousands of small drones coordinating autonomously to overwhelm defenses. The coordination algorithms require AI — no human can manage hundreds of simultaneous flight paths and tactical decisions.
AI-powered targeting systems are used to identify targets from surveillance data. Israel's "Lavender" system, reported by independent journalists, used AI to generate lists of suspected militants from data analysis — a process that previously took intelligence analysts days but was reduced to seconds.
The Accountability Problem
International humanitarian law requires that attacks distinguish between combatants and civilians, that the expected military advantage is proportional to potential civilian harm, and that someone is responsible for each attack decision. Autonomous weapons challenge all three principles.
If an AI system strikes a target that turns out to be civilian, who is accountable? The commander who authorized deployment? The engineer who wrote the algorithm? The company that built the system? The military that set the rules of engagement? The legal frameworks that have governed warfare since the Geneva Conventions have no clear answer.
The Failed Diplomacy
The United Nations has been discussing autonomous weapons through the Convention on Certain Conventional Weapons (CCW) since 2014. A decade of meetings has produced no binding agreement. The reason is straightforward: the countries with the most advanced AI — the U.S., China, Russia, Israel — have no incentive to limit their own advantages. Proposed bans have been blocked repeatedly.
A coalition of over 100 countries supports a ban on fully autonomous weapons. But "fully autonomous" is doing a lot of work in that sentence. Every deploying nation insists their systems have "meaningful human control." The definition of "meaningful" is where the conversation collapses.
The Developer's Ethical Dilemma
Major AI companies are entangled in defense work whether they like it or not. Google's Project Maven controversy in 2018 — where employees protested the company's work on AI for military drone analysis — was an early flashpoint. Google dropped the contract. But Microsoft, Amazon, and Palantir have expanded their defense AI work substantially.
For individual developers, the question is personal: would you work on AI systems used in weapons? The answer isn't as simple as it seems when the same computer vision system that identifies military targets also identifies survivors in disaster zones. Dual-use technology defies clean ethical boundaries.
What Happens Next
The trajectory is clear and sobering. Autonomous weapons will become more capable, more widespread, and more autonomous. The international community will likely fail to regulate them effectively before they're entrenched. The best realistic hope is not a ban but a framework — clear rules about human oversight, accountability mechanisms, and restrictions on specific categories of autonomous targeting.
The worst realistic outcome is an AI arms race with no rules at all. We're closer to that than most people realize.