VulnHunterX

SAST (CodeQL, Semgrep, OpenGrep) + LLM vulnerability hunting and verification

A Python framework that pairs static analysis with multi-turn LLM verification to suppress false positives in SAST findings, implementing the Vulnhalla methodology of guided-question, evidence-anchored triage.

• Overview
• Quick Start
• Pipeline Stages
• CLI Reference
• Python API
• Rules & Coverage
• Configuration
• Project Structure
• Development
• Benchmark
• References

SAST engines deliberately over-approximate — they flag every program point that might be vulnerable. The dominant cost of running them in production is not analysis time but human triage. VulnHunterX automates triage by feeding each SARIF finding to an LLM that must answer a rule-specific checklist of evidence-anchored questions, request additional code context when needed (multi-turn), and emit a structured verdict with a confidence score.

Source  ──>  Static Analysis  ──>  SARIF Findings  ──>  LLM Verification  ──>  Verdicts
(prepare)    (CodeQL/Semgrep/      (rule, file,          (guided questions,      (TP/FP/NMD +
              OpenGrep)             line, severity)       multi-turn context)     confidence)

The Vulnhalla methodology forces the LLM to:

• answer rule-specific guided questions before committing to a verdict,
• request callers, structs, globals, free-sites, etc. as needed via a fixed context vocabulary,
• reason across multiple turns rather than pattern-match a single snippet.

• Python 3.12+
• CodeQL CLI 2.15+
• Semgrep and/or OpenGrep — optional
• An LLM provider: OpenAI, Anthropic, Gemini (AI Studio), DeepSeek, or local Ollama

git clone https://github.com/vinsoc-cyber/VulnHunterX.git && cd VulnHunterX
uv venv --python python3.12 .venv && source .venv/bin/activate
uv pip install -e ".[dev]"

cp env.example .env       # add OPENAI_API_KEY / ANTHROPIC_API_KEY / GEMINI_API_KEY / OLLAMA_API_BASE
vuln-hunter-x check-env   # verify toolchain

Requires uv (curl -LsSf https://astral.sh/uv/install.sh | sh). Plain python3.12 -m venv .venv + pip install -e ".[dev]" also works if you'd rather not install uv.

The fastest path is one of the per-language example scripts under examples/, which clone a real-world target, a vulnerable target, and run the full pipeline against both so the FP-vs-TP contrast is visible:

python examples/pipeline_python.py
# Common flags: --dry-run, --skip-clone, --api

# Run the whole pipeline (prepare → analyze → verify → report) in one command
vuln-hunter-x scan --url https://github.com/org/app.git --lang csharp --profile full --limit 10

# Or be prompted for everything interactively
vuln-hunter-x interactive

scan accepts the union of the per-stage flags (--url/--local-path/--repo, --lang, --profile, --tool, --provider/--model, --limit) plus --skip-verify / --skip-report to stop early.

vuln-hunter-x prepare --repo pyyaml
vuln-hunter-x analyze --repo pyyaml --profile extended
vuln-hunter-x verify  --repo pyyaml --limit 5
vuln-hunter-x report  --repo pyyaml --lang python

C# is compiled but uses CodeQL buildless extraction by default (no dotnet build required); pass --build-command "dotnet build" for full fidelity:

vuln-hunter-x scan --url https://github.com/org/app.git --lang csharp

# Single-shot from URL
vuln-hunter-x prepare --url https://github.com/org/app.git --lang python

# Existing checkout
vuln-hunter-x prepare --local-path /path/to/app --lang python --name app

# Compiled languages need a build command
vuln-hunter-x prepare --url https://github.com/org/lib.git --lang c --build-command "make"

Or list it under repos: in config/repos.yaml and run vuln-hunter-x prepare --repo <name>.

Stages 2–3 accept --local-path to operate directly on an arbitrary directory.

vuln-hunter-x check-env

Verifies CodeQL CLI, Semgrep/OpenGrep, and LLM provider keys.

Clone a repository, create a CodeQL database, and extract context CSVs (functions, callers, structs, globals, macros, classes).

vuln-hunter-x prepare                                                 # batch from repos.yaml
vuln-hunter-x prepare --url https://github.com/org/repo.git --lang go
vuln-hunter-x prepare --local-path /path/to/repo --lang python
vuln-hunter-x prepare --skip-clone --skip-db --force --repo libucl    # re-extract context only

Context CSVs (functions, callers, structs/classes, globals, macros) feed multi-turn LLM verification. Two backends produce them:

Tree-sitter grammars (tree-sitter-c/cpp/python/javascript/java/php/go/c-sharp, pinned to >=0.23,<1.0) ship with the standard pip install -e . — no extra setup. Both backends emit the same CSV layout, so downstream stages (analyze, verify, report) work identically regardless of backend. vuln-hunter-x check-env verifies tree-sitter availability alongside CodeQL.

Run CodeQL, Semgrep, and/or OpenGrep against a prepared repo or a local path.

vuln-hunter-x analyze --repo libucl
vuln-hunter-x analyze --tool all --repo c-ares --profile full
vuln-hunter-x analyze --tool semgrep --local-path /path/to/project --lang python --profile extended

Verify SARIF findings using multi-turn LLM reasoning.

vuln-hunter-x verify --repo libucl
vuln-hunter-x verify --repo c-ares --lang c --category memory-safety
vuln-hunter-x verify --provider ollama --model ollama/llama3.2 --repo libucl
vuln-hunter-x verify --provider gemini --model gemini-2.5-flash --repo libucl
vuln-hunter-x verify --local-path /path/to/project --lang python --limit 20

Generate a markdown report from verification results. verify already writes report.md and report_vi.md; use this to regenerate or change the output path.

vuln-hunter-x report --repo c-ares --lang c
vuln-hunter-x report --results-dir output/c/c-ares/verification_results
vuln-hunter-x report --repo libucl -o my-report.md --lang-report en

Report sections: executive summary · findings overview (before/after verdicting) · severity breakdown · CWE distribution · per-finding detail (verdict, confidence, reasoning, dataflow).

Run any subcommand with --help for full options.

vuln-hunter-x info

Prints the resolved configuration — provider, model, paths.

Runs the full pipeline (prepare → analyze → verify → report) in one command by composing the individual stages. Accepts the union of their flags.

vuln-hunter-x scan --url https://github.com/org/app.git --lang csharp --profile full --limit 10
vuln-hunter-x scan --repo pyyaml --tool both             # config-mode, from repos.yaml
vuln-hunter-x scan --local-path ./app --lang go --skip-report

If the CodeQL database build fails (e.g. a compiled project that needs a specific build command), scan continues with source-only analysis when the analyzer can run without a database (--tool semgrep/opengrep/both/all) — CodeQL findings are skipped but Semgrep still runs. With --tool codeql the run aborts, since CodeQL needs the database.

A guided wizard that prompts for the inputs above, then dispatches a scan. The friendliest entry point — no flags to remember.

vuln-hunter-x interactive

It runs an environment check first and exits if no analyzer (CodeQL/Semgrep/OpenGrep) is installed. Each answer is validated as you enter it: a local path must exist, the chosen analyzer must be installed (unavailable ones are marked and rejected), and the LLM provider is live-tested — if it fails you can re-enter it, continue with verification skipped, or abort, instead of failing partway through the run.

Install the cli extra and enable argcomplete:

pip install "vuln-hunter-x[cli]"
# bash/zsh, for this shell:
eval "$(register-python-argcomplete vuln-hunter-x)"
# or globally (bash): activate-global-python-argcomplete

from pathlib import Path
from vuln_hunter_x import VerificationEngine
from vuln_hunter_x.reporting.markdown import MarkdownReportGenerator

engine = VerificationEngine.from_config("config/confirm_findings.yaml")
result = engine.verify_sarif("output/c/libucl/libucl.sarif", lang="c", repo_name="libucl")

for verdict in result.verdicts:
    print(f"{verdict.finding.rule_id}: {verdict.verdict} ({verdict.confidence})")

engine.save_results(result)
MarkdownReportGenerator().generate(
    result, Path("output/c/libucl/verification_results/report.md")
)

config/RULES.md is the authoritative inventory — every profile, every custom CodeQL query, every custom Semgrep rule, with CWE tags and severities. Quick summary:

Coverage growth from --profile standard to --profile full is roughly 5×–10× more rules per scan. Per-language registry packs (p/django, p/gosec, …) are only applied to matching repos so cross-language scans aren't polluted.

To add a custom rule:

• CodeQL — drop <name>.ql into config/codeql-custom/<lang>/src/ with @id <lang>/<name> matching a guided-question key.
• Semgrep — append a rule to config/semgrep-custom/<lang>.yaml with metadata.cwe: ["CWE-NNN"] so CWE-based routing works.

Both are activated by --profile full. Run python scripts/audit_rule_coverage.py --fail-on-gaps to verify the wiring.

Priority: CLI args > environment variables > config file > defaults.

provider: openai          # openai | anthropic | ollama | deepseek | gemini
model: gpt-4o
temperature: 0.0          # greedy by default; verification is seeded for reproducibility
seed: 42                  # fixed sampling seed (null disables)
max_tokens: 1500
max_iterations: 3         # conversation rounds per finding

repos:
  - name: pyyaml
    url: https://github.com/yaml/pyyaml.git
    language: python

  - name: libucl
    url: https://github.com/vstakhov/libucl.git
    language: c
    build_command: "mkdir -p build && cd build && cmake .. && make"

Rule-specific question banks that force the LLM to reason step-by-step.

The verifier matches each SARIF ruleId in three tiers — exact match → prefix/normalized → CWE map — falling back to default_questions.yaml only when none hits. See config/RULES.md § 7.

VulnHunterX/
├── src/vuln_hunter_x/
│   ├── cli/           # CLI commands
│   ├── codeql/        # Database creation, analysis, context extraction
│   ├── context/       # Heuristic + tree-sitter context extraction
│   ├── core/          # Types, config, constants
│   ├── llm/           # LLM client (LiteLLM) and prompt construction
│   ├── opengrep/      # OpenGrep integration
│   ├── questions/     # Guided-question loader
│   ├── reporting/     # Markdown report generation
│   ├── sarif/         # SARIF parsing
│   ├── semgrep/       # Semgrep integration
│   └── verification/  # Multi-turn verification engine
├── config/
│   ├── RULES.md                # Authoritative rule inventory
│   ├── rule_categories.yaml    # Profiles, categories, CWE map
│   ├── codeql-custom/          # Custom CodeQL queries (full profile)
│   ├── semgrep-custom/         # Custom Semgrep rules (full profile)
│   ├── prompts/                # Guided questions + system prompt
│   ├── queries/                # CodeQL context queries (per language)
│   ├── confirm_findings.yaml   # LLM and verification settings
│   └── repos.yaml              # Repository definitions
├── benchmarks/        # Evaluation framework — see benchmarks/README.md
├── examples/          # Per-language pipeline scripts
├── scripts/           # audit_rule_coverage.py, etc.
├── tests/
└── output/
    └── <lang>/<repo>/
        ├── database/              # CodeQL database
        ├── *.sarif                # SARIF results
        ├── context/               # Extracted CSVs
        └── verification_results/  # Verdict JSON + report.md

pip install -e ".[dev]"
pytest tests/
ruff check src/
ruff format src/
mypy src/

See CONTRIBUTING.md and CHANGELOG.md.

VulnHunterX ships a standalone benchmark framework comparing the full pipeline against raw-sast and ablation baselines on six ground-truth datasets (SecLLMHolmes, Juliet C/C++, DiverseVul, OWASP BenchmarkJava, OWASP BenchmarkPython, RealVuln).

python benchmarks/scripts/run_benchmark.py \
    --dataset secllmholmes --approach all \
    --model gpt-4o-mini --limit 50

Full documentation — datasets, per-dataset playbooks, metrics, resume semantics — lives in benchmarks/README.md. The literature review and design-decision rationale are in benchmarks/RESEARCH.md.

MIT — see LICENSE.

• Vulnhalla — CyberArk — original methodology
• CodeQL Documentation
• Semgrep Documentation
• SARIF Specification