Vulnerability scanner vs penetration test: scanner data
Run of 11 November 2026 on a Mac, loopback only. Published by our lab under the MIT licence (LICENSE).
Post: https://excessionlabs.ai/blog/vulnerability-scanner-vs-penetration-test/
Fixtures: https://github.com/Excession-Labs/web-security-fixtures (commit 2c4a833)

Files
  results.csv   One row per fixture (27 rows). Columns: grade for each setup
                (yes, partial, no, na), alert and finding counts, the known flaw,
                and the names of the ZAP alerts and Nuclei templates that fired
                on the vulnerable and on the fixed server. Column names that end
                in _cold are the root-URL-only runs. The ones that end in
                _baseline_only are the ZAP report written before the active scan.
  counts.json   Count of yes, partial, no and na for each of the six setups.
                The row-level security fixture has no HTTP surface. It shows as na
                or null here, and the post counts it as a miss.
  run.sh        Reference copy of the reproduction script.

Tools: OWASP ZAP 2.17.0 (spider 1 minute, passive scan, active scan with the default
policy, 6 minute cap, 1 minute per rule) and Nuclei v3.11.1 with templates v10.5.1.
All jobs ran under sandbox-exec with a profile that denies non-loopback network access.

Grades come from regular expressions on alert and template names. We adjusted the
rubric after seeing the output. The names in results.csv let you regrade.

Also here: the grading rubric (fixtures.py, analyze.py), the launcher (launch.py),
the job runner (run.py) and the sandbox profile (nonet.sb). Local paths are replaced
with <local path>. The raw scanner output is not published.