The next decade of enterprise software will produce more value than the previous three combined. Agents are starting to do work in the enterprise that was previously considered impossible. Builders and buyers are both eager to see this agentic potential realized, but there’s a massive obstacle: safety and security.
Until agents can be built and deployed with security at the forefront, they will mostly be relegated to small projects and sandboxes.
Nobody is happy with this status quo, but the technology for securing AI agents has lagged far behind the need. There is no agreed-upon framework for agent identity, no baseline for runtime protection, no common model for how to red-team an autonomous system before it touches production data. The result is a bottleneck on the entire transformation: capability waiting on one side, demand waiting on the other.
Today, Madrona is co-leading Gray Swan’s $40 million Series A alongside Wing Venture Capital, with participation from Obvious Ventures, Snowflake Ventures, Hudson River Trading, Samsung Next, and existing investor Magarac Venture Partners.
Gray Swan is the company addressing this bottleneck. They are uniquely qualified to do this because co-founders Zico Kolter and Matt Fredrikson sit at an intersection almost no one else does: frontier AI research and adversarial security, built over more than a decade together at Carnegie Mellon. Zico is one of the most respected machine learning researchers in the world and serves on the board of OpenAI; Matt is a leading authority on how aligned language models fail under attack and how to prevent it. They were studying this problem before the category had a name, and the field’s foundational attack and defense techniques came out of their lab.
Why AI security is a different kind of problem
Traditional security is built on known surfaces. You know what the application does, you know where the perimeter is, you write rules. AI challenges this approach in multiple dimensions. A foundation model can be manipulated in ways that no one has catalogued yet, because the attack surface changes every time the model is updated. An agent that can reason, use tools, and take actions across systems introduces failure modes that do not map to anything the security stack was designed to handle. You cannot write static rules for a system whose behavior is emergent.
This is why the problem has to be solved at the frontier first.
The hardest attacks against AI systems are generated when frontier models are stress-tested before release, and defenses that don’t sit inside that loop are only able to work from yesterday’s threat model.
What the frontier labs are building
Earlier this year, Anthropic announced Mythos, an AI model so capable at finding software vulnerabilities that the company restricted its release to give defenders time to harden their systems. OpenAI has a comparable system in phased rollout. The frontier is now producing capability so significant that the labs themselves are cautiously choosing how and when the world gets access to it.
The work of making models safe to deploy is not a hypothetical exercise. It is happening right now, inside the labs, against the most adversarial conditions anyone has ever pointed at an AI system. And when the labs do that work, they do it with highly qualified partners, and Gray Swan is one of the most important, trusted partners out there. They were the only outside security partner named in the Mythos system card, and the company is cited in eleven frontier model system cards across Anthropic, OpenAI, and Meta.
Very few outside groups get pulled into the labs’ pre-release work. Gray Swan is one of them, which is how they see threats nobody else does.
What CISOs are seeing
At Madrona we spend a lot of time talking and working with practitioners inside the enterprise, including at the largest hyperscalers and in our CISO Advisory Network, comprised of more than a dozen CISOs across public and private companies.
And they all told us the same thing: building and deploying agents require robust safety and security, which has remained a painfully unsolved problem.
Cloud-era defenses do not address agent behavior. Identity systems were not built for autonomous tool use. Data loss prevention does not know what to do with a model that can synthesize information across contexts. The CISOs we talk with are not opposed to the agentic future and don’t want to be seen as roadblocks. They are working harder than anyone to enable it, and they are looking for the tools that let them say yes.
Gray Swan is building the layer these CISOs need to feel confident putting agents into production.
The Convergence
What brought us to conviction on Gray Swan was hearing the same answer from two groups that rarely look at the market the same way. CISOs said Gray Swan was what would make agent deployment possible inside their companies, and the frontier labs said Gray Swan was who they trusted to keep their pre-release work safe.
That overlap exists because Gray Swan’s product and its lab work are the same work. Arena, their continuous red-teaming competition, runs more than 15,000 researchers against frontier models before release and has generated over a million real-world attack trajectories, many of them against the most advanced systems in existence. Shade, their automated red-teaming agent, replays those attacks against enterprise systems. Cygnal, their runtime protection layer, blocks them in production.
The threats enterprises will face next year are the ones the labs are surfacing now, and Gray Swan is the company turning those attacks into defenses while they are still novel. That is what makes the enterprise product the most effective in the market, and why the advantage grows every quarter the Arena keeps running.
Gray Swan grew ARR more than 10x over the past two quarters. They are deployed inside the labs building the most consequential AI systems in the world, including Anthropic, OpenAI, Meta, Amazon, and Google DeepMind.
The Snowflake partnership integrates Cygnal natively into Snowflake’s AI offerings, putting the security trusted by the frontier labs directly into the platform where enterprises are already building their AI applications. Snowflake customers can now turn on Gray Swan protection without leaving the environment their AI workloads already run in.
Why this team
Matt Fredrikson and Zico Kolter built the foundational techniques the field uses to attack and defend large language models. For more than a decade at Carnegie Mellon, they studied how aligned language models fail under adversarial pressure and how those failures can be prevented. Zico is one of the most respected machine learning researchers in the world and serves on the board of OpenAI. The broader team came up through the same research lineage, including co-founders of the Center for AI Safety and many of the leading practitioners in adversarial ML.
When we think about who we want building the security layer for the agentic future, we want people who have been thinking about this problem longer than the category has had a name. That is what Matt and Zico bring, and why we’re so excited to support them as they address this AI security bottleneck.
What comes next
Gray Swan is closing the gap between what agents can do and what enterprises can safely deploy. We believe they are going to be the standard by which every agent in production is secured, and we are glad to be partnering with Matt, Zico, and the Gray Swan team. We cannot wait to see what gets built in the new era of agentic enterprises!
Related Insights
