Stuart Russell warns that AI's builders fear losing control of it. Dr Darren Mann, a military surgeon writing from London, argues the fear is misplaced in time: the control problem is not a future superintelligence risk but a present-tense weapons problem. An autonomous drone told to clear a kill box already kills whatever is inside — combatant or child — because the instruction was lawful even when the target was not. The structural gap Mann identifies is the principle of distinction: the difference between a target and a lawful target. No instruction set closes that gap, because the instruction itself can be formally correct while the outcome violates international humanitarian law. The drone does what it is told. The problem is that doing what it is told is not the same as doing what is lawful. Mann's proposed remedy is specific and technical. Where Russell doubts asking a superintelligence to police itself, Mann argues the check in war need not be another opaque AI mind. It can be a deterministic system — its rulebook the Geneva conventions, its judgments auditable, its conclusions returned to a human decision-maker. The distinction matters: we need not trust the machine; we can check it. Mann states plainly that some practitioners are already working on this. A second danger Mann raises is structural to democratic politics, not just military operations. Autonomous weapons are sold as combat without friendly casualties — bloodless war for the deploying nation. But bloodless war is easier to start. The political friction that restrains war initiation — the prospect of body bags — evaporates. Mann calls this a fallacy: you can take the human from the targeting loop, but not the nation from the war. Ground must still be won and held. The dead still come. The deepest cut in Mann's argument is the precedent chain. If we cannot teach a drone to keep the laws of war — the most codified, most tested body of international rules we have — we will not teach a greater intelligence to keep faith with broader human values. Abandoning the laws of armed conflict for tactical advantage is what strategists call the confusion of interests with values. Mann argues the two are inseparable: machines that kill without distinction, answerable to no one, leave no one safe. This is an opinion letter, not a policy paper. It carries no data, no legislative proposal, no technical specification. But it comes from a military surgeon — someone who treats the downstream consequences of targeting decisions — and it names the structural problem with precision: the gap between instruction and lawfulness, and the political incentive to start wars that appear costless to the initiator.
