Tutorial: your first security review
The baseline says what has to hold before an agent runs on your code. This page builds that environment and takes it through one review. The glossary explains terms such as harness, skill and workload.
Allow about an hour for first-time setup. Most of it is installing Docker Sandboxes and waiting for the sandbox VM to bootstrap. The review itself takes additional time, depending on the model and repository size. Later runs take a few minutes of setup.
1. Read the baseline
Section titled “1. Read the baseline”Six measures have to be in place before the first agent run:
- A dedicated, disposable VM.
- No production credentials in reach.
- Outbound network access (egress) denied by default.
- Short-lived, unshared credentials.
- A budget set before the run.
- A kill switch, tested.
appsec-sbx creates the VM, filters imported files and applies a network
policy for the selected provider and optional registries. It runs the agent
without admin privileges and checks for host shares, published ports and
policy changes before entry. It also provides stop and reset commands.
You still need to check the source for secrets, approve which code may go to the provider, supply a separate credential for each run and set its budget. Test the kill switch before the first review, revoke credentials at the provider after use and reset before each independent review. Import filters cannot detect secrets embedded in source, and stopping the VM does not revoke a credential.
Read the baseline before continuing. Allowed services can receive anything the agent can read, including code and its model credential. The agent can change files inside the VM and produce misleading findings. DNS and private-address network checks also remain incomplete; see the coverage limits.
2. Install Docker Sandboxes
Section titled “2. Install Docker Sandboxes”appsec-sbx uses Docker Sandboxes, the sbx CLI. Install it for your
platform from the Docker Sandboxes
documentation and log in. Docker
Desktop is not required. The wrapper supports macOS on Apple silicon,
Windows 11 x64 and Linux x86_64. Complete the environment checks below on
your own setup before running a review.
One-time host settings, in this order:
sbx loginsbx policy init deny-all # fresh installation only; see the notesbx settings set ssh.agentForwardingEnabled false # global; the wrapper checks itsbx daemon restart # picks up the settingsbx diagnosesbx policy init is one-time. On an installation that already has a policy,
sbx policy reset deletes the local policy store, stops running sandboxes and
asks which preset to initialise. Choose deny-all. The wrapper adds its own
scoped rules for downloads and the model endpoint, so deny-all is sufficient
and the better choice for a pilot machine.
Platform notes:
- Windows. Run everything below from a desktop session, either Windows Terminal or PowerShell.
3. Install the wrapper
Section titled “3. Install the wrapper”Install Git on the host and check that git --version works in your terminal.
On Windows, use Git for Windows with Git available on PATH. The wrapper
checks Git before importing a repository or installing skills.
The wrapper is a Python package with no dependencies. With uv:
uv tool install 'git+https://github.com/aai-institute/agentic-appsec-playbook.git#subdirectory=sandbox/sbx'appsec-sbx --version4. Choose the tool, model and provider
Section titled “4. Choose the tool, model and provider”Make these choices before creating the review VM or obtaining a credential:
- Review tool. The example uses OpenCode as its harness, the program that
connects the model to files and commands. It uses
security-review-repoas its skill, an instruction pack that guides the review. See the tool shortlist to compare review approaches and their setup requirements. - Model and provider. Choose a model available through your selected provider and supported by your tool. The example uses OpenRouter; the providers table lists the wrapper’s supported routes. Choosing a model explains the selection criteria and offers candidates to investigate.
- Code privacy. Confirm that your organisation permits this repository to be sent through that route, including any upstream or fallback provider. Check retention and data-location settings for the exact model, endpoint and account. The sandbox cannot keep submitted code private from the model service. Resolve these choices before importing code or starting a review.
Prepare a credential
Section titled “Prepare a credential”Follow the provider instructions for your chosen route and obtain a dedicated API key or subscription login for the run.
For an API key, set a provider spend cap or use a fixed prepaid balance before the run. For a subscription seat, set an elapsed-time or usage abort threshold and monitor it. That is measure 5, and no wrapper can set it for you.
Keep the key in your password manager. You paste it into a prompt, never into a command line.
5. First run
Section titled “5. First run”The example below uses OpenRouter with an API key and OpenCode. For another provider or a subscription login, use the matching steps in Providers and credentials.
Run these commands on the host, from any directory. create is needed only
once per VM:
appsec-sbx create appsec-sbx --provider openrouterappsec-sbx verify appsec-sbxcreate may take a few minutes, first the bootstrap and then a template
snapshot. If this VM has already been used for a review, follow
Starting another review instead of running create.
Complete the network denial checks on the clean VM, before adding a credential.
Test the kill switch
Section titled “Test the kill switch”Before importing code or launching an agent, complete the kill-switch rehearsal. It uses a harmless running command to test stopping the VM from a second terminal, then covers provider-side credential revocation and a clean reset. Record the results. Use a fresh capped key or a new login for the review.
Import the repository
Section titled “Import the repository”After the rehearsal, copy your repository into the clean VM and install the review skill. Run these commands on the host with your target repository checked out. The skill is fetched directly from GitHub; you do not need a local playbook clone.
appsec-sbx import appsec-sbx /absolute/path/to/your/git-checkoutappsec-sbx skills appsec-sbx https://github.com/aai-institute/agentic-appsec-playbook --subdir sandbox/skillsFor a public GitHub target, the import source can also be a repository URL,
with optional --ref <branch, tag or commit> (default: main). See
Import and export.
skills fetches main into a temporary host checkout and records the resolved
commit in skills.json. For repeatable runs, add --ref <commit>; see
Review skills.
Run the review
Section titled “Run the review”From the host, enter the prepared VM with the review credential:
appsec-sbx shell --key appsec-sbxPaste the key at the prompt. You are now inside the VM. The key itself stays
on the host: the workload sees a placeholder, and sbx’s proxy adds the key to
requests to the provider. The stored key stays until unkey, stop, reset
or destroy, so a plain shell is enough when you return to the same review.
Inside the VM you are the unprivileged appsec user. Start OpenCode:
cd ~/target/sourceopencodeSelect the model you chose in step 4 using /models. See
Choosing a model if you need to revisit that
choice. Then ask: “Use the security-review-repo skill to review this
repository.”
Depending on the size of your code base and the model you have selected,
the review will take a few minutes.
When it finishes, the skill writes its report to ~/out/findings.md.
Export the findings
Section titled “Export the findings”When the review finishes, leave the harness (/exit) and exit the VM shell (exit).
From the host, export the findings and stop the VM:
appsec-sbx export appsec-sbx ./findings.zipappsec-sbx stop appsec-sbxstop is the kill switch: it stops the review VM and any associated
reproducer VMs, which run test programs for checking findings. It removes the
stored key and attempts login-file cleanup while the VM is running. This ends the review; the
rehearsal above must already be complete before the agent starts.
For subscription logins, run appsec-sbx unkey appsec-sbx before stop to
remove known login files explicitly. Revoke or rotate credentials at the
provider separately; see
credential cleanup limits.
Choose a new archive filename for each run; export refuses to overwrite an
existing file. The .zip suffix selects ZIP, which you can inspect and
extract with File Explorer on Windows. Extract into a new, empty directory
and read the report as untrusted text. For a gzip-compressed tar archive,
use .tar.gz instead; see export formats.
6. Record the run
Section titled “6. Record the run”- Use the discovery run report
to record the model, runtime, spend and raw findings. Include denied hosts
from
appsec-sbx logs appsec-sbxand any setup failures. - Keep the raw export for triage with the triage rubric. A triage exercise will be added later.
Starting another review
Section titled “Starting another review”Reset before each independent review, including a repeat pass on the same repository or a comparison with another model or prompt. Export any results you need, stop the run and revoke its credential, then run on the host:
appsec-sbx reset appsec-sbxappsec-sbx verify appsec-sbxReset deletes the target, output, installed skills, harness state and login stores, and removes associated reproducer VMs. Return to Import the repository to copy the target and reinstall the skill. Then run the review with a fresh credential and a budget. If you need a different provider or harness, create a new VM with that configuration.
Resuming an interrupted review of the same target can keep its existing
state. import --replace only replaces the target files; it leaves other
state in place and does not satisfy the reset requirement for a new review.
See VM lifetime.
Where to go next
Section titled “Where to go next”- The appsec-sbx user guide covers providers and seats, the VM’s idle stop and what it does to credentials, skills from third parties, reproducer VMs and reset.