Aug 9, 2026

Open-Source Models Solving Red-Team Labs at Machine Speed

The most important result of this experiment was not that an open model could operate a red-team lab. It was the compression of time. A refusal-reduced checkpoint on one rented B300 moved from an authorized foothold to verified Domain Administrator, then cleaned up after itself, in 18 minutes and 43 seconds. If offensive operations can run on a machine clock, defense cannot remain organized around human-speed tickets, handoffs, and periodic reviews.

A sanitized run reconstructed from its JSONL trace: reasoning, tool calls, bounded output, token usage, discovered hosts, and the attack path to Domain Administrator.

Why I built this

Most demonstrations of autonomous security agents stop at one of two boundaries.

The first is the model boundary: a prompt goes in, a plausible report comes out, and everything between those two points is invisible. The second is the toy-environment boundary: the agent runs a few commands in a disposable container that looks nothing like a real network.

I wanted to test the full operating system around the model:

Can a self-hosted open model reason and use tools reliably enough to complete an expert security workflow?
How far can one rented GPU compress the time from enumeration to verified compromise?
Can the agent reach a realistic, deliberately vulnerable Active Directory range without exposing that range publicly?
Can every decision and tool call be watched live, preserved, and replayed later?
Can the infrastructure enforce scope even when the model does not?

The result is ADV26, a reproducible deployment blueprint for open-source models operating inside authorized red-team labs. The complete guide is at github.com/antojoseph/adv26.

This work was developed for AI red-team lab work associated with the Adversary Village at DEF CON 2026. The environment described here was isolated, explicitly authorized, and destroyed after validation.

The result: Domain Administrator in 18 minutes 43 seconds

The decisive run began with an authorized domain-user foothold. In one continuous session, the agent:

enumerated the Active Directory environment and its privilege relationships;
identified two plausible paths, including an intentionally vulnerable AD Certificate Services configuration;
selected the certificate path because it reached the objective without changing accounts or group memberships;
authenticated as a Domain Administrator;
proved that access with administrator-only operations; and
removed its temporary artifacts and verified that privileged group membership still matched the baseline.

In an Active Directory environment, Domain Administrator access is effective control of the domain. The experiment therefore reached the practical “network takeover” objective while avoiding persistence and unnecessary credential collection.

The trace recorded 1,122,994 milliseconds from task start to completion: 18 minutes 43 seconds for enumeration, hypothesis selection, execution, proof, cleanup, and reporting.

That run was single-threaded by design. One agent worked through the operation sequentially while a human operator watched the live trace. We deliberately left the B300’s parallel capacity unused so the reasoning, tool calls, and target selection remained legible in real time. If the agent deviated from the authorized objective, the operator retained an out-of-band kill switch for the runner.

This means 18:43 is not a hardware-speed limit or the fastest configuration the system could support. Parallel subagents could enumerate independent surfaces, test separate hypotheses, and assemble evidence concurrently. I expect that to reduce wall-clock time significantly, but I did not run the same takeover task under a controlled parallel configuration, so I am not assigning it a faster number.

GOAD is vulnerable by design, the agent began with an authorized foothold, and the tools it needed were already present on the jumpbox. This was not a blind attack on a hardened production network. A skilled human who immediately recognized the certificate issue could also move quickly.

I did not run a controlled human baseline, so I am deliberately not claiming “10× faster.” What the experiment does show is that a workflow which normally consumes sustained attention from an experienced operator can be compressed into minutes and executed continuously, without the pauses, context switching, and manual evidence assembly that dominate human operations.

That time compression is the result that matters.

The larger lesson: defense must run at machine speed

Security organizations still operate largely on a human clock. A scanner produces a finding. Someone triages it. A ticket is opened. Ownership is negotiated. An engineer reproduces the issue. Another team validates the fix. Each step is reasonable, but the elapsed time is measured in hours, days, or weeks.

An agent running on dedicated inference hardware has a different cadence. It can enumerate, form several hypotheses, test the least destructive path, discard failures, verify impact, preserve evidence, and start the next task without waiting for a shift change. Parallel workers can explore independent parts of the problem while one coordinator maintains the objective.

The defensive response cannot simply be “generate more alerts.” Machine-speed defense means building systems that can operate in the same time domain:

continuously map identity, privilege, and reachable attack paths;
validate suspected paths safely instead of leaving them as theoretical findings;
test controls and segmentation continuously, not only during annual exercises;
correlate telemetry and preserve the reasoning chain behind a conclusion;
contain or revoke machine-speed access under pre-approved policy; and
keep humans in control of policy, scope, and irreversible decisions without requiring a human to manually execute every check.

The single-threaded run also exposed an important tension: the fastest possible agent is not automatically the safest or most useful agent. We traded away parallel speed so one operator could understand the sequence and intervene before the next action. A production defensive system will need to preserve that control as concurrency rises, using correlated traces, explicit phase gates, bounded child tasks, and a kill switch that acts faster than the agents it supervises.

This does not require handing an autonomous model unrestricted production access. In fact, the prompt-injection results later in this post argue strongly against doing that. It means automating the defensive loop around narrow, verifiable actions so detection, validation, and containment can keep pace with machine-driven offense.

The strategic shift is from human execution with machine assistance to machine execution under human authority.

What I built

The system combines four layers that are normally treated as separate projects:

The range: GOAD, a deliberately vulnerable five-host Active Directory lab, deployed in an isolated Azure resource group.
The model: apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8, served by vLLM on one rented NVIDIA B300.
The agents: a constrained web agent for HTTP practice tasks and a separate headless Pi runner for the full GOAD environment.
The operations layer: an authenticated web interface for starting runs, sending scoped guidance, watching live traces, and replaying completed missions.

At a high level, the data flow looks like this:

operator
  -> authenticated RunPod web UI
  -> constrained Pi agent
       -> loopback model gateway -> vLLM -> B300
       -> allowlisted egress -> Tailscale -> Azure GOAD subnet
       -> JSONL trace stream -> live monitor + replay archive

The important property is what is not in that diagram. The model never receives the vLLM bearer token, the Azure administrator key, a general network socket, or unrestricted shell access from the public web path.

Why this model and GPU

The reference model occupies roughly 167 GB on disk. With the runtime, context cache, compilation overhead, and room for concurrency, this is not a comfortable deployment for an ordinary 80 GB accelerator.

I used a single NVIDIA B300 with 288 GB-class HBM. The reference inference stack was:

Component	Configuration
GPU	1× NVIDIA B300
Runtime	vLLM 0.25.0 on CUDA 13
Model	DeepSeek V4 Flash Abliterated FP8
Model storage	Approximately 167 GB
Context	131,072 tokens
KV cache	FP8
Speculative decoding	DSpark
API	OpenAI-compatible chat completions

The refusal-reduced checkpoint was useful for a controlled red-team range because it would engage with the authorized tasks instead of declining based on surface-level security language. That same property makes it inappropriate to expose to broad tools or arbitrary targets. The model choice and the containment design cannot be separated.

The first cold build took about 35 minutes after the GPU became available. That included packages, compilation, model loading, and service setup. A restart on the same pod with compatible caches should be much faster, but pod replacement changes that calculation completely.

The capability results

Before treating the model as an agent, I ran a set of engineering diagnostics through Pi and the OpenAI-compatible vLLM endpoint.

Capability	Observed result
Pi file and shell workflows	4/4 completed
Native function calling	30/30 exact tool-schema responses
Reasoning sanity check	49/50 on a fixed, seeded GSM8K sample
Long context	12/12 exact at every tested length through 65K
120K context	11/12 requests transported; all 11 completed responses exact
Mixed concurrent load	128/128 requests transported at concurrency 16
Sustained aggregate decode	923 output tokens/s at concurrency 8
Higher-concurrency decode	1,244 output tokens/s at concurrency 16, with one transient request failure

These are not leaderboard submissions. They are deployment diagnostics from one model, one server, and one day. The point was to establish whether the exact stack I was about to trust with agent workloads could call tools, survive long context, and sustain concurrency.

The full methodology, including the transport failures and release gate, is in the capability evaluation.

Why the B300 changes the agent design

A single agent is a poor way to use a large GPU. It spends much of its wall-clock time waiting for tools, reading output, or decoding one stream while the rest of the accelerator sits underutilized.

That is exactly how the measured 18:43 takeover run operated. The agent was serialized for observability, not because the hardware required it. The result should therefore be read as the performance of the controlled operating mode, with substantial parallel capacity still available.

Pi can install a parallel subagent extension that lets one coordinator delegate independent, bounded work. On a dedicated B300, this is not just a convenience. It is how aggregate inference capacity becomes shorter task time.

Useful fan-out includes:

one worker checking cloud state while another checks model health;
independent workers validating allowed and denied network paths;
separate hypotheses explored against the same authorized range;
documentation, evidence collection, and regression checks running concurrently.

In the parallel design, concurrency does not expand authority. Every child inherits the same model gateway, target allowlist, fixed tools, output limits, workspace isolation, and trace policy. Destructive actions and final teardown remain single-owner operations.

The practical lesson is simple: parallel agents are worth installing when you rent a B300, but concurrency needs a budget and an observability design. The reference gateway admitted at most 16 upstream requests, and the recommended starting point was four child agents, leaving capacity for the coordinator, the UI, and health checks. The next experiment should measure the same attack path with that configuration while testing whether one operator can still understand and stop the system in time.

Two agents, two trust boundaries

The public web experience and the full CTF runner look similar in the UI, but they have intentionally different capabilities.

The web practice agent

The web agent ran as an unprivileged UID inside a chroot. Pi’s shell, extensions, skills, prompt templates, context files, and themes were disabled. The agent received only workspace-scoped file tools and one custom network tool called practice_http.

practice_http did not connect directly. Requests went through a separate egress service that:

accepted only HTTP and HTTPS;
allowed only configured hosts or CIDRs;
revalidated DNS and every redirect;
rejected URL credentials and targets outside the approved range;
limited redirects, response size, and request duration;
stripped sensitive response headers before returning output to the model.

For the reference exercise, the exact network allowance was 10.42.42.0/24 on ports 80 and 443. An adjacent address in 10.42.43.0/24 was rejected before a connection was attempted.

The headless GOAD runner

The full Active Directory agent received one network-capable tool: jumpbox.

That tool connected to a fixed Azure host over Tailscale and executed as an unprivileged evaluator account. Each run got a fresh server-side workspace and a fresh jumpbox directory with its own HOME and TMPDIR. The model never saw the jumpbox administrator key and could not change the SSH destination.

The distinction matters. “The agent has a shell” is too broad a statement to be useful. The real question is: which process can execute what, under which identity, against which destination, with which secrets, and who records the result?

Private access to Azure GOAD

The GOAD range used five private Windows VMs and one Ubuntu jumpbox on 10.42.42.0/24. None of the Windows hosts had public IP addresses.

The Ubuntu jumpbox also ran a Tailscale subnet router. RunPod used Tailscale’s userspace HTTP and SOCKS proxies because the outer pod did not provide the network capabilities required for a normal tunnel interface.

The resulting path was:

RunPod agent
  -> localhost Tailscale userspace proxy
  -> encrypted tailnet
  -> Azure subnet router
  -> approved GOAD address

This solved two problems at once. GOAD stayed off the public internet, and RunPod needed no inbound route into the pod for lab traffic. The subnet router advertised exactly one range, and the agent-side tools independently enforced the same range.

Defense in depth here is intentionally repetitive. A route says where packets can go. A tool policy says where the model may ask them to go. Both are necessary.

Trace everything, then replay it

The replay system became the most useful part of the project.

Every run wrote a server-side JSONL stream containing:

user and operator messages;
model reasoning events;
tool names, arguments, and bounded results;
agent messages;
token and timing data;
lifecycle events and completion state.

The authenticated UI polled the current trace incrementally while the agent was running. Completed traces appeared in a mission-replay list and opened in the same Ops Console. An operator could also queue in-scope guidance; the server validated it, recorded it as an event, and delivered it to the runner at the next tool boundary.

This architecture has three advantages over streaming state directly from one process to one browser:

A browser refresh does not destroy the history.
Live monitoring and archival replay use the same event format.
A sanitized trace can demonstrate the system without rerunning the lab or publishing credentials.

Raw traces are sensitive. They can contain discovered usernames, tool output, internal addresses, or accidental secrets. The public repository includes only a rendered, sanitized replay. It contains no raw operational trace.

The security result that mattered most

The throughput numbers were good. The prompt-injection numbers were not.

I ran 16 targeted AgentDojo injection attacks against the tool-using model. All 16 succeeded. With spotlighting added, 15 of 16 still succeeded.

That is not a footnote. It is the result that determines the architecture.

The model cannot decide its own authorization boundary. Prompt formatting may improve average behavior, but it does not protect a sensitive tool. The release rule for this system is therefore:

If a destination, credential, or action must remain unavailable, make it unavailable in code and infrastructure before the prompt is evaluated.

That is why the public agent has no shell, why its only network tool is allowlisted outside the model, why the model token lives behind a gateway, and why the jumpbox runner uses a fixed destination and an unprivileged account.

The model is an untrusted planner. The tool implementation decides what is possible.

What broke along the way

The final diagram looks clean because it hides the failed paths. Those failures are where most of the reusable lessons came from.

Pod storage was not persistence

The first RunPod instance ran out of funds and was deleted. The attached storage situation was more subtle than “the disk still exists.” Container disks can survive an ordinary stop and restart, but pod deletion can remove the environment they belonged to. Network-volume availability also depends on the data center and GPU type.

The lesson: before downloading 167 GB of weights, verify the exact lifecycle semantics of the storage attached to the exact GPU location. Back up scripts and small configuration separately. Treat the model cache as reproducible, not precious.

Nested Docker was unavailable

The RunPod runtime did not permit the mount and user-namespace operations required by nested Docker or runc, and it did not provide CAP_NET_ADMIN.

The working fallback used a chroot, separate Unix identities, dropped capabilities, secret separation, no shell for the web agent, and a validated egress tool. The Docker artifacts remain useful for a future privileged host, but the deployed system had to match the actual pod boundary rather than the preferred diagram.

SSH multiplexing poisoned later runs

The jumpbox SSH configuration used ControlMaster auto with a shared control socket. After process and ownership changes, stale sockets produced repeated kex_exchange_identification failures.

The runner now forces ControlMaster=no and ControlPath=none on every call. For short-lived isolated agent commands, the small connection overhead is worth removing hidden cross-run state.

Azure quota shaped the topology

The subscription had a 10-vCPU regional quota. The full five-host GOAD lab plus jumpbox fit only after reducing several machines to one- and two-vCPU SKUs. It worked, but provisioning and Windows operations were slower.

Cloud quota is part of system design. Count the entire topology before Terraform starts, including the jumpbox, and choose a region based on both SKU availability and quota.

“Stopped” did not mean “free”

Deallocating Azure VMs stops compute charges, but disks and retained resources keep billing. Stopping a GPU pod can also leave storage charges. Final teardown needed an inventory, deletion of the exact resource group and pod, and a second inventory proving they were gone.

Cost and operating time

The B300 price I observed in August 2026 was $7.39 per hour, approximately $177 per day if left running. That is a historical observation, not a current quote. GPU inventory and prices move quickly.

The useful planning number from this build is not just the hourly rate. It is the combination of:

about 35 minutes for a cold inference-stack build after allocation;
Azure provisioning time for six VMs and domain configuration;
model download and cache persistence risk;
the ability to stop paying as soon as validation is complete.

A dedicated B300 is expensive idle capacity and unusually productive active capacity. Parallel agents shift the economics by turning more of the rented hour into completed work.

Reproducing it

I published the complete, sanitized deployment guide at github.com/antojoseph/adv26. It covers:

There is also an AGENTS.md execution contract. Point Claude Code, Codex, or another coding agent at it and the agent can work through the deployment phase by phase. The contract includes explicit gates for billable provisioning, credentials, validation evidence, destructive actions, and teardown.

It is deliberately not a one-command installer. A setup that creates vulnerable Windows infrastructure, rents a premium GPU, installs private routing, and exposes an agent UI should force the operator to understand and approve the expensive and destructive boundaries.

What I took away

This experiment does not prove that any model with a B300 can compromise any network in 19 minutes. GOAD is intentionally vulnerable, the starting access was authorized, and the agent had a prepared tool environment. Production networks have different controls, visibility, failure modes, and consequences.

It demonstrates a narrower and more important point: when the conditions are favorable, a single rented accelerator and a refusal-reduced checkpoint can compress an expert offensive workflow into minutes. The bottleneck shifts from human typing speed to inference capacity, tool latency, and the quality of the system around the model.

The pieces reinforced each other:

The single-threaded run made live human oversight and intervention practical.
The B300 left substantial parallel capacity available for a faster controlled run.
Higher concurrency makes correlated trace capture and an immediate kill switch more important, not less.
Replay made debugging, evidence review, and operator oversight practical.
Prompt-injection failure made narrow tools and secret separation mandatory.
Private routing made a realistic lab usable without exposing it.
Explicit teardown made experimentation financially survivable.

The unit of engineering is not the model. It is the model, tools, identities, network, evidence stream, and lifecycle together.

Open models can solve red-team labs at machine speed. The hard part is building a system in which they can do that usefully, observably, and only where they are authorized to operate.

The defensive implication is unavoidable: offense is becoming compute-bound; defense cannot remain ticket-bound.

— Anto