This post is also available in:
עברית (Hebrew)
As organizations begin deploying teams of AI agents to handle complex tasks, cybersecurity is becoming a growing concern. In these multi-agent systems, several large language models can exchange information, divide work and use external tools with limited human supervision. That collaboration makes them useful, but it also creates a weakness: compromising one agent could allow malicious information or behavior to spread through the entire system.
Researchers led by Worcester Polytechnic Institute have developed a security framework called AcMAS that takes a different approach to detecting such attacks. Instead of examining only what an AI agent says, it monitors numerical activity occurring inside the model while it processes information.
Existing defenses often inspect prompts, responses or communication patterns between agents. The problem is that a compromised model can still produce text that appears completely reasonable. The framework instead analyzes neural activations—internal numerical signals generated as information moves through an LLM’s neural network. An attack that subtly changes the model’s reasoning may alter these patterns even when its output looks normal.
According to TechXplore, the framework first learns a reference representation of how an agent behaves during benign operation. While the multi-agent system is running, the framework examines each active agent individually and measures how far its activation pattern deviates from that baseline. If the difference becomes large enough, the agent is flagged as potentially compromised.
This local approach also means it does not need to reconstruct every communication between agents. That is particularly useful for asynchronous systems, where AI agents perform tasks whenever information becomes available rather than operating in coordinated rounds.
Detection is only half of the system. Instead of immediately disconnecting a suspicious agent, which could remove an important capability from the team, it attempts to correct it. The framework uses activation-level intervention to steer abnormal internal states back toward patterns associated with normal behavior, adjusting the strength of the correction according to the detected deviation.
Researchers tested the approach across five benchmark settings, three attack families and systems containing between 8 and 80 agents. Compared with graph-based baselines, the framework improved the F1 detection score by 0.22 in synchronous systems and 0.55 in asynchronous ones. Its mitigation approach achieved a 0.97 task-completion rate while reducing measured attack success to 0.03.
The technology could eventually have important defense and homeland-security applications. Future military, intelligence and critical-infrastructure systems may use teams of AI agents to analyze sensor feeds, manage cyber operations or coordinate autonomous platforms. Detecting a compromised agent before it influences the wider network could provide an important additional security layer.
The framework remains an early research prototype and currently requires access to a model’s internal activations, making it most applicable to open-weight models or systems operated on controlled infrastructure. Future work will examine larger, mixed-model agent networks and attackers that deliberately attempt to disguise their internal activity.
The research was published here.
