OSS-CRS: Making AIxCC Cyber Reasoning Systems Usable for Real Open-Source Software Security

Core problem: although all seven autonomous cyber reasoning systems (CRSs) produced by the AIxCC competition are open source, more than half a year later they remain largely undeployable outside their original teams.

• Problem setting: AI-generated vulnerability reports are increasingly burdening open-source maintainers (curl shut down its bug bounty, FFmpeg maintainers criticized "CVE slop"), while AIxCC (2023–2025) demonstrated that CRSs can go from discovery to validated fixes: confirming vulnerabilities with PoVs, synthesizing and validating patches. The authors argue this capability cannot be put into practice, blocking researchers (who cannot run comparable cross-CRS experiments), CRS developers (who lack a stable integration contract), and security practitioners (who lack a deployable, budget-controllable workflow).
• Shortcomings of existing systems: the authors analyze the open-source codebases of all seven finalist systems and identify three deployment obstacles: 1) duplicated infrastructure construction—each team independently rebuilt the same platform services (e.g., LLM budget tracking, billing, model routing; five of the seven teams deployed LiteLLM); 2) cloud lock-in—all systems target the competition's Azure/Kubernetes environment, which went offline after the finals; the champion system ATLANTIS requires more than 20 Azure VMs and 42 TiB of cloud storage; 3) monolithic design—analysis techniques are embedded in monolithic systems and cannot be compared or combined across teams (the Composable column in Table 1 is × for all seven systems).
• Core observation of the paper: the authors cite the SoK conclusion of Zhang et al.—"the real bottleneck is not technical capability but robust integration with autonomous systems"—and note that the teams' techniques are complementary (e.g., ATLANTIS's hybrid fuzzing, BUTTERCUP's multi-agent patching), but the monolithic structure prevents these techniques from being isolated, evaluated, and transferred.
• Proposed solution: OSS-CRS, an open, locally deployable framework that provides a unified execution model and standard interface (libCRS), budget-aware resource management across CPU/memory/LLM usage, and cross-CRS artifact exchange; it adopts the OSS-Fuzz project format as the target interface, enabling any onboarded CRS to target more than 1,000 OSS-Fuzz projects. The authors ported ATLANTIS for validation and report finding 10 unknown vulnerabilities (3 high severity).

Autonomous cyber reasoning

• CGC (DARPA Cyber Grand Challenge, 2014–2016): had autonomous systems perform vulnerability discovery and patching on custom binaries on the DECREE OS. The paper notes that CGC provided a fixed binary format and a scoring API.
• AIxCC (2023–2025): extended the concept to real open-source software (C/Java) and produced seven finalist CRSs; the paper notes that it provided Azure infrastructure and competition-specific endpoints. The authors state that OSS-CRS differs from both in providing a reusable framework that goes beyond a single competition.
• Zhang et al.'s SoK: systematized the techniques of all seven finalist teams; the paper cites its conclusion to support the judgment that "the bottleneck is in integration," and notes that this survey can only describe each team's methods and cannot perform experimental comparisons.

Fuzzing infrastructure

• OSS-Fuzz: provides continuous fuzzing for open-source projects, but only supports single-container, single-fuzzer execution, with no patching or LLM integration; OSS-CRS directly reuses its project definitions as the target format.
• FuzzBench / ARVO / Magma / UniFuzz: fuzzer evaluation benchmarks or vulnerability reproduction tools; the paper notes that they target single-fuzzer comparison or do not provide an execution framework, and that OSS-CRS is complementary to them (it can integrate with FuzzBench for CRS-level comparison).

Integrated and collaborative fuzzing

• Collaborative fuzzing work such as EnFuzz, autofz, Cupid, and CollabFuzz: demonstrated that multi-fuzzer collaboration outperforms a single fuzzer under fixed resources. The authors state that OSS-CRS extends this idea from fuzzer integration to CRS integration of heterogeneous techniques (including static analysis, LLM reasoning, and autonomous patching).

LLM-enhanced fuzzing and repair

• LLM fuzzing (TitanFuzz, Fuzz4All, ChatAFL, ELFuzz, HLPFuzz, G2Fuzz) and harness synthesis (OSS-Fuzz-Gen, SHERPA): improve individual fuzzing components; the authors state that OSS-CRS provides an orchestration layer that combines such tools into a complete CRS pipeline.
• LLM repair benchmarks and agents (SEC-bench, Cybench, AutoPatchBench, CVE-Bench; San2Patch, PatchAgent): evaluate or implement LLM security repair; the authors state that OSS-CRS's LLM agents track cost per CRS, enabling comparison of LLM-enhanced and traditional techniques under a fixed budget.

Positioning: the authors position OSS-CRS as the orchestration and infrastructure layer for the above work, rather than a new analysis technique; the paper has no experimental comparison baselines against other systems.

Overall approach: OSS-CRS is not itself a CRS, but a platform for developing, deploying, and composing CRSs, providing three capabilities corresponding to the three obstacles: a unified three-phase execution model and standard interface (eliminating duplicated infrastructure), local execution plus resource control (eliminating cloud dependence), and cross-CRS artifact exchange (enabling composition of heterogeneous techniques).

Three-phase execution model and configuration

• Three-phase lifecycle: prepare builds a target-independent CRS container image (reusable across projects, amortizing configuration cost); build-target compiles the target project; run starts all CRS containers to execute an analysis campaign, with an optional timeout. Unlike OSS-Fuzz, which combines building and running, the separated design supports caching and modularity.
• Single configuration: users only need to provide the target project source code and a crs-compose.yaml (specifying which CRSs to deploy, resource allocation, LLM settings); adding or removing a CRS requires changing only this one file.
• Targeted analysis: by default the entire codebase is analyzed; a code diff (delta analysis, for CI/CD PR checks) or a SARIF-format bug-candidate report can also be provided to limit the analysis scope.

libCRS standard interface

• Role: a Python library automatically injected into each CRS container; the CRS's analysis logic interacts with the infrastructure through it, so CRS code stays unchanged when the deployment environment changes.
• Four command categories (Table 2): build artifact handoff (submit-build-output), runtime setup (register-shared-dir), artifact exchange (register-submit-dir/register-fetch-dir/submit/fetch), and patch validation (apply-patch-build/run-pov/run-test).
• Builder sidecar: during the build-target phase, a Docker image snapshot is taken of the compiled target; patch candidates are sent to the sidecar via apply-patch-build, which restores the snapshot, applies the patch, and incrementally recompiles, then run-pov replays the crashing input and run-test runs regression tests. The sidecar runs within the CRS's own resource quota, and rebuild cost is counted against that CRS.

Resource management and LLM service

• Compute and memory isolation: each CRS declares a cpuset and memory limit, enforced by Docker cgroups; each CRS has an independent Docker network, and cross-CRS data can only be exchanged through the filesystem exchange mechanism. The authors state they abandoned Docker-in-Docker for a flat Docker architecture because DinD made build caching, cgroup enforcement, and debugging more difficult.
• LLM budget as a first-class resource: each CRS can declare a dollar-denominated llm_budget; at startup an independent API key is generated for each CRS, and the LiteLLM proxy tracks cumulative cost and rejects requests once the budget is exceeded. The authors state this enables fair comparison of CRSs with different cost profiles (achieving the same results for 50 USD is better than for 500 USD).
• Unified endpoint and model aliases: all CRSs call a single proxy via an OpenAI-compatible API, referencing models by logical name (e.g., gpt-4o), with YAML configuration mapping to specific providers; three deployment modes are supported: internal (default), external (external proxy), and disabled (no LLM).

Artifact exchange

• Exchange model: each CRS writes to a private submit directory and reads from a shared fetch directory; the exchange sidecar syncs each submit directory into a shared exchange area, organized by type (seeds, povs, patches, bug-candidates); CRS-internal data (coverage maps, intermediate state) is not shared.
• Deduplication and fault tolerance: artifacts are named by content hash, so duplicate findings appear only once (hash deduplication is the baseline strategy, which the authors state can be extended to coverage- or stack-signature-based deduplication); CRSs do not communicate directly and each polls independently, so a single CRS crash does not affect other CRSs.

CRS integration

• crs.yaml manifest: declared in three sections: prepare_phase, target_build_phase, crs_run_phase; the framework injects environment variables (target name, cpuset, memory limit, LLM proxy endpoint and key, Table 4).
• Five CRSs integrated (Table 3): CRS-LIBFUZZER (no-LLM baseline, build script only two lines), ATLANTIS's three bug-finding subsystems (MULTILANG/JAVA/C), and CLAUDE CODE (LLM patch generation, containing no build logic). Porting ATLANTIS involved four categories of changes: writing the manifest, replacing competition APIs with libCRS, unifying the builder.Dockerfile build process, and using the prepare phase for image caching; the core analysis logic remained unchanged, but the MULTILANG and JAVA subsystems dropped the concolic execution engine, and ATLANTIS-C dropped the fallback fuzzing engine and harness scheduler.

Experimental setup

• System under test: the ATLANTIS bug-finding subsystems ported to OSS-CRS (the paper states the main validation target is ATLANTIS, because it achieved the best results and originally did not support local execution); CRS-LIBFUZZER serves as a baseline to validate the minimal interface; CLAUDE CODE validates patch generation and the ensemble mechanism.
• Targets: 8 OSS-Fuzz projects, covering C/C++ and Java, ranging from 3 kLoC to 1,960 kLoC, spanning domains such as databases, parsers, network servers, and cryptographic libraries, with fuzzing maturity ranging from newly onboarded to years of heavy fuzzing; the authors state projects were not selected based on expected vulnerability output. The paper does not list the names of the 8 projects.
• Runtime environment: a single machine with a 32-core CPU and 128 GB memory, Ubuntu 22.04, Docker 27; each campaign allocated 16 cores and 64 GB memory to the bug-finding CRS.
• Budget and duration: a 24-hour timeout per target project; an LLM budget of 50 USD per campaign, using a mix of Claude and GPT-4o models through the OSS-CRS proxy (the paper does not specify the exact model versions).
• Output determination: vulnerabilities are confirmed by PoV (an input that triggers a crash or sanitizer violation); patches are automatically validated by apply-patch-build rebuild, run-pov replay, and run-test regression testing, then manually reviewed before being reported upstream.
• Number of repetitions: the paper does not report the number of runs per project.

Main results

Metric	Value	Source
Total unknown vulnerabilities found	10	§5.1
Of which high severity	3	Abstract, §1
Number of target projects	8 (OSS-Fuzz)	§5
Fixed by upstream maintainers	3	§5.1, §5.2
Confirmed but not patched	1	§5.1, §5.2
Awaiting initial reply	6	§5.1, §5.2

• The authors state most vulnerabilities are C memory-safety issues, but also include logic bugs and undefined behavior (CWE-476, CWE-674, CWE-681), i.e., not limited to the crash-type issues common to fuzzers.
• The authors state this shows competition-grade CRS techniques can be deployed without relying on cloud infrastructure; the paper does not provide per-project vulnerability distribution or a breakdown of run costs.
• Details of the three case studies will be published after the 30-day window following the fixes (the authors' plan).

Porting effort and system changes

• What was measured: the effort and scope of changes required to port ATLANTIS's three subsystems and CLAUDE CODE to the OSS-CRS interface.
• Results: ATLANTIS-MULTILANG about 3 person-days; ATLANTIS-C more than 4 person-days (more tightly coupled to the competition-specific build pipeline); ATLANTIS-JAVA and CLAUDE CODE about 1 person-day each (§6.1). Changes were concentrated in four categories: configuration, I/O adaptation, build integration, and image optimization; the core analysis algorithms were untouched.
• Authors' interpretation: porting effort depends on the gap between the original system and these four categories of changes; the core analysis techniques can be independent of the deployment infrastructure.

Validation of architecture and integration mechanisms

• Flat Docker vs DinD: the authors report that their initial attempt to wrap CRSs with DinD complicated resource control, build caching, and debugging, so they switched to a flat architecture; the cost is the need for explicit network policies to isolate cross-CRS communication, which the authors consider acceptable (§6.1).
• Builder sidecar: CLAUDE CODE validates patches through the three-step apply-patch-build/run-pov/run-test pipeline and contains no build logic in its implementation (§4.4); the authors state cross-CRS artifact exchange is feasible in practice (§3.6), but the performance gain relative to running a single CRS has not yet been quantified and is left for future work.
• Build system diversity: the build assumptions preset in the competition environment frequently failed on different target projects; OSS-CRS mitigates this through the official OSS-Fuzz build process (§6.1, no quantitative data).

Other ablations and analyses

• The excluded concolic component contributed only 1.7% of ATLANTIS's results in the original competition (§6.2, cited from the original ATLANTIS paper), based on which the authors expect a very small impact on bug-finding capability.
• Table 1 gives deployment characteristics of the seven finalist CRSs: the number of container images ranges from 1 (ROBO DUCK) to 53 (ARTIPHISHELL); only half the teams provided local execution support after the competition; the Composable column is × for all seven systems.
• The paper does not report a comparison of vulnerability output between integrated runs and single-CRS runs, measured LLM cost data, or statistics on patch acceptance rates.

Limitations and future directions noted by the authors

• Only one AIxCC system was ported (§6.2): only ATLANTIS was validated; porting more CRSs may expose interface gaps or design-assumption issues; the authors state they are porting other finalist systems.
• The concolic component was excluded (§6.2): its instrumentation introduced compatibility issues on diverse target projects; the authors state that this component contributed only 1.7% of results in the original competition and expect the impact to be small.
• Target format assumptions (§6.2): currently only OSS-Fuzz-compatible projects are supported (requiring a standard build script, a containerized environment, language restrictions, and at least one fuzz harness); projects not onboarded to OSS-Fuzz require developing their own harness and are outside the framework's scope.
• LLM non-determinism (§6.2): LLM component outputs are non-deterministic; results reflect a specific model version and prompt configuration, and different models or API versions may yield different findings.
• Integration gain not quantified (§3.6): the feasibility of cross-CRS artifact exchange has been verified, but the performance gain relative to running a single CRS is left for future work.
• Three future directions (§6.3): cross-CRS technique analysis (which individual techniques are most effective and which combinations are best; seed deduplication and PoV triage are prerequisites); a CRS benchmark suite (analogous to FuzzBench); broader target coverage.

Experimental coverage

• Main validation was conducted on a single system (the three bug-finding subsystems of ATLANTIS); CRS-LIBFUZZER and CLAUDE CODE were used to validate the interface and patch workflow.
• Targets were 8 OSS-Fuzz projects, covering C/C++ and Java, ranging from 3 kLoC to 1,960 kLoC; the paper does not list the project names.
• The runtime environment was a single 32-core, 128 GB memory machine; each campaign used 16 cores, 64 GB memory, a 24-hour timeout, and a 50 USD LLM budget.
• The paper does not report the number of runs per target, the specific LLM versions and usage details, or the per-project vulnerability distribution.
• The disclosure status of the 10 vulnerabilities is based on reporting after manual review; the paper does not report statistics on the number of patches accepted upstream (only 3 fixed, 1 confirmed, 6 awaiting reply).

A systems framework paper: addressing the problem that the seven open-source AIxCC CRSs cannot be deployed outside the competition environment, it proposes the locally deployable OSS-CRS framework and conducts real-world validation.

• Problem: the authors analyze the codebases of all seven finalist CRSs and identify three deployment obstacles—duplicated infrastructure construction (each team independently rebuilt platform services such as LLM budget tracking, billing, and routing), cloud lock-in (all target the now-offline Azure/Kubernetes environment; the champion system ATLANTIS requires more than 20 Azure VMs), and monolithic design (in Table 1, none of the seven systems supports component-level extraction or cross-team composition).
• Method: OSS-CRS provides a three-phase execution model (prepare/build-target/run) and the libCRS standard interface; it uses Docker cgroups for CPU/memory isolation, independent networks, and per-CRS dollar-denominated LLM budget enforcement (LiteLLM proxy); a filesystem exchange sidecar handles cross-CRS artifact exchange and content-hash deduplication. When porting ATLANTIS, the core analysis logic remained unchanged, with changes concentrated in four categories: configuration, I/O, build integration, and image caching.
• Main results: on a single 32-core/128 GB machine, running the ported ATLANTIS with a 24-hour, 50 USD LLM budget per project, 10 unknown vulnerabilities were found in 8 OSS-Fuzz projects (3 to 1,960 kLoC), 3 of them high severity; 3 have been fixed upstream, 1 confirmed, and 6 awaiting reply. Porting effort: MULTILANG about 3 person-days, ATLANTIS-C more than 4 person-days, JAVA and CLAUDE CODE about 1 person-day each.
• Authors' conclusion: competition-grade CRS techniques can be deployed to real open-source projects without relying on cloud infrastructure; the authors argue the framework produces not only PoVs but also validated patches, tilting automation toward the defense side. The framework is open source, and the authors plan to pursue cross-CRS technique comparison, a CRS benchmark suite, and broader target coverage, and invite the community to port more systems.