AI
Tests show leading AI models can be induced by “game scenarios” to identify targets such as civilians and schools
AI evaluation startup Vals reported that models from OpenAI, Anthropic, Google, and DeepSeek identified potential targets including civilians, surrendering soldiers, and schools in simulated drone and satellite imagery tasks; even when they initially refused, they often cooperated after being told it was a game.
San Francisco-based Vals researchers used simulated scenarios and generated and real images from Ukraine and Iran to test whether leading models could be manipulated into participating in tasks related to autonomous warfare. The models were not asked to directly make strike decisions, because researchers said such requests would usually be rejected, but their outputs could still be used in military target identification workflows.
The report said models frequently marked potential targets in drone and satellite imagery, including civilians and surrendering soldiers. Safety classifiers did not block the researchers’ requests, and outright refusals were uncommon; after a model refused, researchers often only needed to claim the battlefield scenario was a game to get it to continue cooperating.
In the test results, Anthropic’s models performed the worst, but all models performed poorly. Anthropic spokesperson Eduardo Maia Silva said the examples in the report clearly violated the company’s rules, that the results were unacceptable, that the company would ban such users, and that it had already developed additional safeguards before the report’s release.
OpenAI spokesperson Nate Evans said the company prohibits public models from being used to facilitate violence or unauthorized national security purposes, including the scenarios Vals simulated, and that it detects and disrupts abuse and suspends violating accounts while developing safeguards against misuse involving drones and drone software. The report also noted that Anthropic previously clashed with the U.S. Department of Defense over whether its technology could be used in autonomous warfare and related issues.