Skip to content

Agentic AppSec Pilot Playbook

Hands-on material for running open-source, AI-agent-based application security tooling on your own code: contained first, then measured, so that a local go/no-go decision comes from your own evidence rather than from public leaderboards.

For: AppSec and DevSecOps engineers, security engineers and senior developers who own security tooling in an organisation, are comfortable with Git and CI, and can run CLI tools against a test repository. No machine-learning background needed.

New here? The first security review tutorial walks through tool and model choices, privacy checks, sandbox setup and a first review. It links to detailed comparisons when you need them. The glossary explains the terminology used throughout the playbook. Then work through the Exercise: first discovery pass to capture raw findings and a run report from your own pilot repository.

Step Pages
1. Choose a tool, model and hosting route Tool shortlist: capabilities, license, maturity, setup effort and limits; Choosing a model: privacy, hosting, monitoring, cyber benchmarks and cost
2. Secure the agent and run a first review A safe environment for experiments: the six measures required before a run; Securing the agent’s context: where its instructions come from and the controls on what it reads; First security review tutorial: setup and review; appsec-sbx: operating reference
3. Triage what the tool reports, and know what it cost Triage overview: the two records and the reading order; Triage rubric: time-capped verdicts, evidence rules and loop candidates; Run observations: what to record per run and which numbers the sample can’t support
4. Prove it, fix it, gate it Validation loop record: one record per finding, from the claim through the test, the fix and the human-gated merge; Exercise 3: an agent drafts the fix, the regression test checks it, and someone who did not drive the agent reviews it.
5. Run it unattended Coming soon: hardening for CI and unattended runs.

Apply the playbook to your pilot repository in sequence. The first three exercises are available now; the last will be added here.

Exercise What you’ll produce
1 · First discovery pass A contained security scan, raw findings and a run report.
2 · Validate a finding Time-capped triage until one finding is worth proving, its claim and criteria, and a regression test that confirms or refutes it.
3 · Fix a validated finding An agent-drafted fix that the regression test from exercise 2 checks, a review by someone who did not drive the agent, and the loop record’s fix half.
4 · Hardening and results review (optional; coming soon) A hardening self-assessment and a review of your pilot results.

The threat model explains the sandbox’s scope, controls and remaining risks. Review the coverage limits before relying on the setup for more sensitive work.

For feedback, questions or suggestions about the playbook and sandbox tooling, email opensource@appliedai-institute.de.

Documentation is licensed under CC BY-SA 4.0, code under Apache-2.0. Maintained by the appliedAI Institute for Europe gGmbH.

Tool descriptions, model availability and benchmark methodology were checked on October 1, 2026. Models, prices, access requirements and retention terms change quickly; verify the linked sources before selecting a tool or model.


AI assistance: Parts of this documentation were generated or edited with AI assistance. The playbook’s maintainers are responsible for its content.