< formatted article >

How Playing Battleship Revealed a Flaw in AI—and How to Fix It

When Brute Force Isn’t Enough

Artificial intelligence today can outperform humans in answering questions, writing essays, and even diagnosing diseases—but ask it to ask the right questions, and many models stumble. Researchers at MIT and Harvard discovered this unexpected weakness by turning to an unlikely training ground: Battleship.

In a modified version of the classic game, AI agents had to deduce hidden ship positions by questioning each other in real time—a task that demands strategic inquiry, not just raw computation. The results were revealing.

Big Models Struggle with Strategy

While large language models like GPT-5 could hold their own, they often lacked the logical finesse needed for consistent success. Smaller models, such as Llama 4 Scout, frequently made irrational moves—unless researchers intervened.

The breakthrough came when the team introduced a verification step: before answering, the AI estimated the probability of its response being correct. This single adjustment skyrocketed Llama 4 Scout’s success rate from a dismal 8% to 82%, proving that efficiency doesn’t depend on sheer size—just the right approach.

The Missing Link: A "World Model"

The experiments exposed a critical gap in AI development. Pure size doesn’t equal smarter questioning. The key lies in what researchers call a "world model"—a framework for predicting outcomes. By shifting AI’s focus from answering to exploring possibilities, the team unveiled a path forward.

Instead of merely providing solutions, AI might soon think like humans—hypothesizing, testing, and refining strategies. The future of AI isn’t just in answers—it’s in the questions we teach it to ask.