Dry Run Testing
Preview and calibrate custom Drixy rules against historical pull requests before enforcing them across your engineering team.
When introducing a new security rule or style guideline, the biggest risk is false positive noise that frustrates developers and slows down release velocity.
ScanDrix includes a native Dry Run simulation engine that allows architects and security leads to test new rules against past pull requests before enabling them in production.
Why use Dry Run?
- Zero Developer Friction: Test rules completely in the background without posting comments to live pull requests.
- Accurate Hit Rates: Measure exactly how many historical PRs would have triggered the rule over the last 30, 60, or 90 days.
- Noise Calibration: Inspect AST matches and adjust query selectors or threshold limits until false positives reach 0%.
Running a Dry Run from the Dashboard
Navigate to Rules Editor
In the ScanDrix dashboard, go to Rules → Create Rule or edit an existing custom rule.
Enter your AST Pattern or Policy
Define your rule's YAML definition and AST pattern matcher:
match:
ast_call:
- "jwt.Parse($TOKEN)"
where:
$TOKEN: "!calls(verifyKey)"
Select Simulation Scope
Click Run Simulation (Dry Run) and select the evaluation scope:
- Select target repositories or choose all organization repos.
- Set the lookback window (e.g. Last 50 Pull Requests).
Review the Impact Report
ScanDrix parses historical diffs against your AST matcher and produces an interactive impact matrix showing:
- Total triggers across sample PRs.
- Diff line snippets showing where the comment would have landed.
- Estimated review delay and developer interaction metrics.
Promote to Production
Once satisfied that the rule only catches genuine violations, toggle the rule status from Draft / Dry Run to Active.
Running Dry Runs via the CLI
You can also run dry-run simulations locally in your terminal or inside CI pre-flight checks:
# Test a new local rule file against the current branch diff
scandrix dryrun --rule ./rules/sec-auth.yaml --branch main
# Simulate rule against a specific remote pull request
scandrix dryrun --rule ./rules/sec-auth.yaml --pr 142