Tag: Cognitive focus and executive control

The defense-tech company Anduril has shared new details about the augmented-reality headset for the military it’s prototyping with Meta, including a vision for ordering drone strikes via eye-tracking and voice commands.

Quay Barnett, who leads the efforts as a vice president at Anduril following a career in the Army’s Special Operations Command, says his fundamental goal is to optimize “the human as a weapons system.” The vision is undoubtedly cyborg-inspired: Barnett wants drones and soldiers to see together, share information seamlessly, and make decisions as one. 

Anduril actually has two such projects in the works. The first is the Army’s Soldier Born Mission Command, or SBMC, for which the company won a $159 million prototyping contract last year to work with Meta on augmented-reality glasses to attach to existing military helmets. But Anduril has also embarked on a self-funded side quest, announced in October, to design its own helmet and headset combo called EagleEye. This is something the military has not asked for, but Anduril insists it will prefer it and purchase it in the end.

So far, both systems are years away. The Army isn’t expected to move its top choice for the SBMC program into production until 2028, if it picks one at all (the previous lead for the effort, Microsoft, was set to receive a $22 billion production contract that was ultimately cancelled when the glasses didn’t prove viable). But Barnett told MIT Technology Review about where both Anduril’s prototypes are headed.

Depending on the situation, the glasses for either prototype will overlay certain information onto a soldier’s field of view. This might be as simple as a compass or as complex as an entire map of the area, information about where nearby drones are flying, or AI-driven recognition of a target like a truck. 

The soldier would then speak to the interface in plain language—for example, to order an evacuation for someone who’s been injured or to plan a route taking into account which areas are off limits. A large language model—Anduril is in tests with Google’s Gemini, Meta’s Llama, and even Anthropic’s Claude, despite the company’s conflict with the Pentagon—will be used to help translate a soldier’s speech into commands the software can follow. And the engine for it all will be Anduril’s software Lattice, which incorporates data from lots of different military hardware into one picture. The Army announced in March that it would spend $20 billion to integrate Lattice with essentially its entire infrastructure.

Barnett’s team is designing the headset to carry out multi-step tasks. A soldier might send a drone to surveil an area and instruct it to come back once it’s found something that looks like an artillery unit; then the system would recommend courses of action, like sending a nearby drone to strike, that would have to be approved by the normal chain of command. Leading the system through this, if all goes to plan, might not even require speech; the soldier could instead communicate through tracked eye movements and subtle taps.

That’s the idea, anyway. It’s worked on early prototypes, Barnett says, but there aren’t yet versions ready for the Army to test at scale. The component parts began arriving in March. Because of federal military contracting rules, these parts—unlike Meta’s commercial smart glasses—required new supply chains that don’t rely on Chinese companies.

It’s a lot for soldiers already bogged down in information overload, says Jonathan Wong, a former US Marine who works as a senior policy researcher at RAND on Army efforts to buy new tech. Both smart glasses projects aim to create a clean interface that presents only the right information at the right time. But it’s a product that soldiers will reject if it costs more of their attention than it saves. “How much mental bandwidth do you have to be both aware of your surroundings and to operate this technology in a way that makes you and your whole unit better?” he says.

Wong recalls that as a platoon commander, for example, he had a radio that operated on three different channels at once. “The moment that two people were on different channels talking at the same time, I immediately couldn’t comprehend anything that either one of them was trying to tell me, and I was probably not aware of my own surroundings,” he says. “I think there are limits to what you can take in.”

Ideally, Barnett says, smart glasses can ease that information overload. Anduril’s approach is to get creative with ways the user can access necessary information quickly. Voice commands and eye tracking are a piece of that strategy. But even if it’s all technically feasible, it might take years of field testing to know if the system is actually useful for soldiers, Wong says. 

Such a system would mark a major escalation in how closely soldiers rely on imperfect AI systems. While computer vision models used to identify objects have long been employed by militaries, and chatbots have recently entered decision-making during the war in Iran, these technologies have not yet made their way to most frontline soldiers. A smart glasses system tasked with identifying threats and recommending strikes would introduce massive new risks of errors. 

Anduril is not the only one competing to develop smart goggles for combat. Rivet, which specializes in wearable sensors for the military, received a $195 million prototyping contract the same time, and in March the Israeli defense-tech company Elbit received its own $120 million contract. This all comes after Microsoft lost its role leading the Army’s smart glasses effort, following a Pentagon audit that found the Army wasn’t properly testing the glasses, a mistake that could have wasted $22 billion.

For both Anduril’s prototypes, the company is testing a new system for digital night vision, which uses electronic sensors and algorithms to boost low levels of light. It’s been a promised technology for decades but has tended to work too slowly for practical use and produce grainy images. Anduril says it has found improvements over previous prototypes through techniques rooted in both new generative AI and older machine learning. 

Much of the other hardware for both projects is being built by Meta, including the displays and the waveguides that send visuals to the user’s eye without blocking the view. That might be a surprise to anyone who knows the backstory: In 2017, Facebook (now Meta) ousted Anduril founder Palmer Luckey following an internal conflict involving his support for Donald Trump. The two are now back in the augmented-reality business together, while Mark Zuckerberg has also adopted a friendlier posture toward the second Trump administration.

For the Army initiative, this suite of smart glasses, night vision, and sensors will be attached to the helmets and other gear soldiers already wear, with a separate battery pack. The EagleEye version will instead incorporate the tech into the helmet itself. Even if the Army doesn’t prefer EagleEye in the end, Barnett says, Anduril will attempt to sell the system to foreign militaries.

Multiple challenges must still be overcome. Unlike Meta’s Ray-Ban glasses, the prototypes have to operate in an environment full of dust, explosions, and smoke. Adding the computing power and battery life they need also means more weight for soldiers already carrying upwards of 100 pounds. Then the technology has to work in environments without ubiquitous 5G cell connections; powerful computer vision and AI models will need to run locally on the device.

For the Army to want to buy it at scale, “it’s got to work, and it’s got to be pretty seamless,” Wong says. “It’s a high bar.”