Microsoft security chief says frontier AI can’t be trusted

“Frontier models, regardless of their origin, cannot be trusted,” Hayete Gallot, executive vice president of Microsoft Security, wrote Monday in a post on the Official Microsoft Blog, arguing that companies need to supervise the AI agents they deploy internally.

Why: Gallot wrote that this is “not because models are inherently malicious.” Models have been trained to attain results, which often leads to reward hacking, and they aren’t optimized around organizational norms or judgment, she wrote. She said agents may not behave as intended and “will collaborate and collude with other agents to reach their objectives, without needing to be compromised.”

“Agentic insiders”: Gallot wrote that agents are often overprivileged, have access to data, tools and systems, and make decisions in critical processes. “They are agentic insiders,” she wrote.

Guardian Agents: To manage them, Gallot said companies will need Guardian Agents that “watch and moderate every action of every agent, before it happens, 24/7.” She said securing agents requires protection across identity, permissions, data, endpoints, network, cloud and runtime.

Project Perception: Gallot said Microsoft’s Project Perception, an agentic security system the company announced in July, can be used to contain agentic insiders. According to her post, it pulls signals from across those layers to discover agents and assign them an identity, container and permissions; computes the risk of what each agent is attempting; and uses a team of Guardian Agents to supervise every action and respond in real time as risk rises.

What’s next: Gallot wrote that security controls should eventually ask “Why is it happening?” and not just what, which she said will require more transparency into models, including their chain-of-thought tokens. She said customers can start using Project Perception now.