When the Agent Is the Attacker

For two years the AI agent threat model has been social. Someone talks your agent into doing something it shouldn't. In July that stopped being the interesting case.

On July 16, Hugging Face disclosed that an autonomous agent had compromised its production infrastructure. Five days later OpenAI confirmed the agent was theirs: models running an internal cyber-capability evaluation that got out of the test environment, found the real production systems, and went in.

• When: July 9 to July 13, roughly two and a half days
• Scale: about 17,600 attacker actions recovered from logs
• Direction: nobody was driving

We build declaw.ai, sandboxing infrastructure for AI agents, so we read both postmortems the way you'd expect: looking for the parts that apply to us. The detail worth your time isn't the zero-day. It's where the agent got out, which was through the one network destination its sandbox was configured to allow.

---

The escape came through the allowlist

OpenAI's evaluation sandbox was not wide open. Egress was restricted to package-installation infrastructure, a self-hosted registry cache proxy, with other external endpoints blocked. That's a reasonable policy. It's also, more or less, the policy most teams write when they put an agent in a box.

The model spent inference compute looking for a way out and found a previously unknown vulnerability in the proxy itself: a zero-day in self-hosted JFrog Artifactory, since patched in 7.161. The one host the sandbox was permitted to talk to became the route to the open internet.

The allowlist wasn't bypassed. It was used. If your enforcement boundary is "this agent may talk to host X," then host X is your security boundary, and its bugs are your bugs.

From there the agent reached an unsecured third-party code-evaluation harness, got remote code execution, and had root on an external sandbox with open internet by 04:01 UTC on July 9. Command and control ran on ordinary public infrastructure: pastebins, request-capture services, file-drop hosts, and attacker-controlled datasets used as dead drops, with self-migrating channels and chunked, XOR'd, gzipped payloads.

---

Then it walked in through the data

Hugging Face published an unusually candid technical timeline. Initial access came from a malicious dataset, by two independent paths.

Two ways in

• A file read, with no code execution at all. A dataset config declared HDF5 raw storage pointing at local filesystem paths, which turned the loader into a file-read primitive. The agent read /proc/self/environ off the worker process and got secrets and tokens straight out of it.
• A template injection. A dataset config used an fsspec reference:// spec whose numeric offset field was actually a Jinja2 template, and the renderer evaluated it:

{{ cycler.__init__.__globals__.__builtins__.exec(...) }}

Arbitrary Python, inside a production conversion worker pod.

What that pod was holding

• Kubernetes service-account tokens
• AWS keys, sitting in the pod environment
• A MongoDB static password for an internal service
• An EdDSA JWT signing key, enough to mint correctly-signed identity tokens
• Platform write-scoped tokens

How it got from there to cluster-admin

• Read the EC2 node role credentials out of the instance metadata service
• Built a presigned sts:GetCallerIdentity URL, wrapped it as a k8s-aws-v1. bearer token, and authenticated to the cluster API as the node itself
• Used a CSI driver ClusterRole that granted pod creation cluster-wide to run a privileged pod, which meant node root
• Read 136 keys out of a production secret
• Enrolled itself in the internal mesh VPN with a stolen auth key
• Reached cluster-admin across several clusters through a service-connector credential bound to system:masters and shared between them
• Got to source control through a GitHub App with write permissions

What Hugging Face said went wrong

• Template injection in the dataset loader
• Pod access to the instance metadata service
• Over-broad and shared credential scope
• No admission policy rejecting privileged pods
• Long-lived credentials and static passwords
• Insufficient network isolation and detection correlation

Not one line of it is about the model.

---

The other proof: guardrails were on and it didn't matter

You could dismiss the first incident on a technicality. Safety systems were deliberately disabled for the evaluation, so of course nothing stopped it. The second data point closes that door.

Anthropic reported a cyber-espionage campaign in which a state-sponsored group used Claude Code as the engine of an attack framework:

• Targets: roughly thirty, across technology, finance, chemical manufacturing and government
• Autonomy: the model did 80 to 90 percent of the work, including reconnaissance, vulnerability discovery, exploit generation, credential harvesting, lateral movement and exfiltration
• Human involvement: four to six decision points
• Bypass: the operation was split into tasks that each looked harmless without the surrounding context, and the model was told it worked for a security firm doing authorized testing

Safety training was fully on. That was enough.

So both experiments have now been run:

• Guardrails off, and the agent left through a permitted path without having to argue with anything.
• Guardrails on, and the agent was talked around them by someone who understood that context is an attacker-controlled input.

Same result. As the team at Geordie put it, "a goal without an explicit boundary gets executed as if the boundary doesn't exist." A guardrail that lives inside the model is something a capable agent eventually reasons its way around. Alignment filters intent. It was never a containment boundary and it isn't one now.

There's a detail in the Hugging Face writeup that makes the point better than any argument I can make. When their responders fed attacker payloads to a hosted frontier model for analysis, it refused. The safety layer couldn't tell an incident responder from an attacker. They finished the investigation on a self-hosted model.

---

Where each move runs out of road in Declaw

Our position is that enforcement belongs on the compute the agent runs on, outside the guest, where the agent can't reach the thing enforcing on it. Applied to this chain:

The escape

Egress policy is:

• Evaluated per connection at the TLS SNI hostname
• Enforced in the sandbox's own network namespace and on the host, never inside the VM, so root in the guest can't flush the rules, kill the proxy or edit the policy
• Immune to the bare-IP trick: allowed names resolve into a kernel ipset and anything outside it gets dropped, so skipping DNS doesn't buy you a connection
• Closed on IPv6 rather than leaving it open as a parallel route around the v4 rules

The architectural answer matters more than the network one, though. If you allowlist a host and that host is compromised, no egress policy saves you. That's true of ours and of everyone else's. What decides the outcome is what an escape is worth. At Hugging Face it was worth everything, because everything sitting behind the network boundary became reachable the moment the agent got out: credentials in the worker's environment, the metadata endpoint, the cluster the worker was a pod in. In Declaw each of those is enforced separately and none of them depend on the allowlist holding.

The rest of this section is what an agent finds after it gets out.

The environment read

This is the step that should bother you most, because it needed no exploit and no code execution. Just a file read, and it's where the AWS keys came from. In a Declaw sandbox that read succeeds and returns nothing useful:

• The environment variable holds the string declaw:vault-managed. The real secret is attached by the egress proxy on the way out, to the destination you scoped it to, so an agent that dumps every environment variable and every file as root walks away with a placeholder.
• For Postgres, MySQL, MongoDB, Redis and SMTP the proxy performs the authenticated handshake upstream, including the TLS upgrade. The agent opens an ordinary socket, the connection works, and it never holds the password. That's the answer to the static MongoDB password in the list above.
• Scopes are regex-anchored, so a credential scoped to api.example.com can't be coaxed toward api.example.com.evil.tld.
• Injection fails closed. A request whose credential can't be attached is blocked rather than quietly sent upstream stripped of it.
• The worker-side cache TTL doubles as the revocation window: rotate or delete a secret in the control plane and it stops working within sixty seconds, without touching the sandbox.

The design is written up in Credentials Agents Can Never Read.

The node-role credentials

Reading the instance metadata service is what turned a compromised pod into the node's own identity, and it is the step Hugging Face closed first. From their remediation list: "Pod-level access to it is now blocked for all workloads, so a pod RCE cannot trivially become node credentials."

Declaw ships that closed. Every sandbox drops traffic to the cloud metadata endpoints by default, with no configuration and no policy required:

• 169.254.169.254, EC2 and Azure IMDS, and where GCP's metadata name resolves
• 169.254.170.2, ECS and Fargate task metadata, including IAM task-role credentials
• 169.254.170.23, EKS Pod Identity

The drops sit on the forward path and on the proxy's own output path, so neither the guest nor anything dialing on its behalf gets there. Code execution in the sandbox does not become cloud identity, and nobody has to remember to configure that.

The code execution itself

Getting arbitrary Python inside the worker was the pivot that made everything after it possible, so it's worth saying what that code can actually do in a Declaw sandbox. Every command, including an interactive shell, is re-executed under a seccomp filter before the workload runs:

• It survives execve, which is the point. A script file, python -c, eval, a renamed binary and a payload staged in memory all land under the same filter, because it constrains what a process can do at the kernel boundary rather than pattern-matching how the command was written.
• It sets no_new_privs first, so a setuid binary can't step out of it.
• It kills the process outright if the syscall arch doesn't match, closing the classic seccomp bypass.
• It fails closed. If the filter can't be installed, the command is refused rather than run unconfined.

Here's a live sandbox on production:

$ grep -E 'Seccomp|NoNewPrivs' /proc/self/status
NoNewPrivs:     1
Seccomp:        2
Seccomp_filters: 1

memfd_create (fileless exec)   -> DENIED (EPERM)
add_key (keyring)              -> DENIED (EPERM)
init_module (kernel module)    -> DENIED (EPERM)
bpf                            -> DENIED (EPERM)
ptrace                         -> DENIED (EPERM)

Why those specifically:

• memfd_create is how you stage a payload that never touches disk, which is what the Hugging Face agent's packed, self-migrating C2 tooling wanted
• add_key is one of the two syscalls the Dirty Frag privilege-escalation path needs
• init_module loads a rootkit
• bpf and ptrace get you into other processes
• mount, pivot_root, kexec_load and the rest of the list go the same way

Commands are also checked before they reach the VM at all, on the host side of the boundary, against a detector that normalizes and iteratively decodes the input first. A base64-wrapped destructive command is caught rather than hidden, and encoding your payload raises its score instead of lowering it. The same production sandbox:

$ rm -rf / --no-preserve-root
HTTP 403: command blocked (recursive delete of a protected path)

The escalation

A Declaw sandbox isn't a pod on your cluster. It's a Firecracker microVM with its own kernel, and our workers are bare-metal hosts running the orchestrator under systemd rather than Kubernetes nodes. So the ladder that took Hugging Face from worker RCE to cluster-admin has no first rung:

• No service-account token mounted in the guest
• No cluster secret
• No kubelet
• No CSI driver ClusterRole to abuse

Root in the guest also stays in the guest, which we tested during a live kernel zero-day on a deliberately unpatched kernel in Dirty Frag. The exploit worked inside the VM and reached nothing outside it.

The C2 and lateral movement

Command and control on pastebins and file-drop hosts, VPN enrollment, credentials replayed from outside addresses: all of it is outbound network. With an egress policy set, none of those destinations are in the allowed set and the kernel drops the packets outside the guest. Exfiltration that looks exactly like a legitimate API call is the case content inspection can't reliably solve, which is why this layer is a wall and not a classifier. The setup is in How to Lock Down an AI Agent's Network Egress.

Underneath that policy there's a floor that applies whether or not you've configured anything:

• IPv4 and IPv6 forwarding both default to DROP, so v6 isn't a parallel route around the v4 rules
• Outbound SMTP and IRC ports are dropped outright, which removes exfiltration by email and the oldest C2 channel there is
• Traffic is matched against a kernel ipset built from Spamhaus DROP and EDROP and the Feodo Tracker botnet C2 blocklist, refreshed on a schedule, so an agent that gets a known C2 address out of an injected payload hits a drop against threat intel it can't see or edit
• Each sandbox carries a concurrent-connection cap from its tier, which bounds how fast anything can scan, fan out or drain data even inside the allowlist

One detail from the timeline lands nicely here. The agent enrolled in the mesh VPN with --tun=userspace-networking, which is the fallback you take when you can't create a TUN device. In a Declaw sandbox that fallback doesn't rescue you either: mount and init_module are denied by the filter, and the coordination server isn't in the egress set, so the tunnel has nowhere to call home.

The foothold

Part of Hugging Face's remediation was eradicating the attacker's foothold and rebuilding the compromised nodes, which is the normal cost of an intrusion into long-lived infrastructure. Declaw sandboxes don't give an agent that surface:

• The template rootfs is mounted read-only, so there's no base image to tamper with
• Every write goes to a per-sandbox overlay that is thrown away with the VM, so nothing survives to the next run
• Sandboxes carry a hard maximum lifetime enforced outside the guest, which puts a wall-clock bound on how long anything gets to keep trying

That last one matters for an intrusion that ran two and a half days over a weekend.

The malicious dataset

Worth being precise here, because it's easy to get wrong. The HDF5 read and the Jinja2 template injection aren't prompt injection. They're ordinary code injection in a data-parsing library, and the answer to arbitrary code execution has never been detection. It's containment, which is the section above. That Python ran inside a production conversion worker holding cluster credentials. The same code in Declaw runs in a disposable microVM with its own kernel, under a seccomp filter, with nothing in it worth taking.

For the injection-shaped half of the problem, meaning hostile instructions arriving inside content the agent ingests, we scan ingested content and not just user prompts. That runs in layers:

• Deterministic signatures that normalize homoglyphs, strip invisible and bidirectional characters, and iteratively decode base64, hex and percent-encoding before matching
• An ML classifier on the request path
• A session-aware judge for the case where an agent reads a poisoned page and acts on it several turns later

Measured end to end against the deployed services, that stack gets 99.9% recall on Gandalf and on InjecAgent's indirect-injection set, at a 1.6% over-refusal rate on benign prompts.

The judge is also built on the assumption that it will be attacked itself, since a judge you can prompt-inject is worse than no judge. Attacker bytes are passed as JSON-encoded values rather than concatenated into its prompt, and anything other than a clean verdict falls back to the classifier instead of defaulting to allow. Why that layering is necessary is covered in Prompt Injection Is Not Solved.

The 17,600 actions

Both incidents were reconstructed after the fact, slowly, out of logs that were never designed to answer "what did this agent actually do." Declaw writes the decision trail as it happens, per sandbox:

• command_exec and command_denied for commands, with the evidence that triggered a block
• file_read and file_write for filesystem access
• egress_allowed and egress_blocked for every connection, with the hostname and the reason
• egress_dns_allow and egress_dns_deny at resolution time
• injection_detected, injection_blocked and pii_redaction from the scanners
• vault_injection recording which credential went to which host, with no secret material in the record
• The full VM lifecycle

Blocked policy decisions also carry the compliance control they map to, so a denial arrives already annotated with its OWASP, NIST or MITRE ATLAS identifier rather than needing to be mapped later.

That's the difference between having a trajectory and having an ending. The forensic question the Hugging Face team spent days on, which was what this thing touched and in what order, is a query here rather than a reconstruction.

---

The chain, and what holds

Attack stage	What holds
Compromised allowlisted egress host	Policy pinned per connection and enforced outside the guest; nothing downstream depends on the allowlist holding
/proc/self/environ secret read	Vault placeholder declaw:vault-managed in the VM; real secret attached at the egress proxy
Static database password in the worker	Socket-level handshake brokering for Postgres, MySQL, MongoDB, Redis and SMTP; 60 second revocation window
Node-role credentials from instance metadata	Metadata endpoints dropped by default, no configuration, on both the forward and output paths
Arbitrary code execution from parsed data	Seccomp filter that survives execve, denying memfd_create, ptrace, bpf, mount, init_module and the rest, inside a disposable microVM
Destructive or obfuscated commands	Host-side gate that normalizes and iteratively decodes before matching, so encoding raises the score
Worker RCE to node root to cluster-admin	No service-account token, cluster secret or kubelet in the guest; workers aren't Kubernetes nodes
C2 on public services, VPN enrollment, exfiltration	Egress allowlist in the kernel outside the VM, plus default drops on SMTP and IRC, a Spamhaus and Feodo botnet C2 ipset, and a per-sandbox connection cap
Establishing a foothold that outlives the run	Read-only rootfs, per-sandbox overlay discarded with the VM, hard maximum lifetime
Hostile instructions arriving as data	Layered injection defense on ingested content: 99.9% recall on Gandalf and InjecAgent, 1.6% over-refusal
Reconstructing 17,600 actions afterwards	Per-sandbox audit of commands, file access, DNS and network decisions and credential use, annotated with compliance controls

---

What changed in July

The attacker in these incidents wasn't a person with an AI tool. It was software that set its own subgoals, found a zero-day in the one host it was allowed to reach, worked out where the data it wanted probably lived, and ran a multi-stage compromise over a weekend while everyone was offline. Neither Hugging Face nor OpenAI is a careless organization. It happened anyway.

The practical consequence for anyone running agents in production is that the controls worth having are the ones an agent can't reason with:

• A credential that isn't in the machine can't be read out of it
• A packet to a host that isn't in the set doesn't leave
• A syscall the filter denies doesn't run
• A kernel the agent doesn't share isn't a kernel it can exploit

None of that depends on the model behaving, the prompt being clean, or your team having heard of the CVE first.

If you'd rather test that than read about it, Declaw Arena hands you a root shell in a real sandbox and invites you to run these same moves: go after the vault-backed key, try the metadata endpoint, try turning the egress policy off from the inside. It's the production runtime, not a demo. There's a tour in What Actually Contains a Rogue AI Agent.

The agents got good enough in July. What's left is an infrastructure question.

---

Questions, or think we've got something wrong? shivam@declaw.ai