claims-anomaly-radar

Explainable anomaly detection for healthcare claims: a hybrid rule engine plus isolation forest that flagged 100% of 4,643 seeded anomalies in a 252,750-claim population, with a plain-English reason on every flag.

• Duplicate, upcoded, and outlier claims hide in millions of rows and surface months late in audits; this pipeline flags them in one 47-second batch pass over 252,750 claims.
• ML-only detectors produce flags auditors cannot act on; here every flag carries a reason ("billed 950.00 vs median 32.01 for 85025, z=55.3"), and severity is assigned by provenance, not score.
• Scoring an incoming claim against a 250k population took 16 seconds naively; a precomputed scoring context brings it to 21 ms (p50 170 ms under 10-user load).

Payers process claims at a volume where manual review sees a fraction of a percent. Duplicate submissions, upcoding (billing a routine visit at the highest complexity code), and price outliers are individually small and collectively expensive. In the synthetic population used here, the seeded anomalies that the high-severity tier correctly caught carried $1.59M in billed amounts, 1.2% of total billed volume, which is consistent with the low-single-digit percentages commonly cited for claims waste and abuse. Real claims data is protected under HIPAA and UK GDPR, so the repo ships a generator that produces a realistic population with five seeded, labeled anomaly patterns instead of a dataset.

The solution is a two-tier detector. Deterministic rules catch what can be enumerated: exact duplicates, near-duplicate resubmissions inside a configurable window, providers whose high-complexity billing share is a statistical outlier against their peers, and amounts far outside the procedure's price band. An IsolationForest over five population features catches what rules cannot name, and everything lands in one flag table where severity encodes provenance: a rule hit is "high" (deterministic, auditable, actionable), an ML-only flag is "review" (a lead, not an accusation). A FastAPI service scores new claims online against a precomputed population context, and a Streamlit queue lets a reviewer confirm or dismiss flags backed by transactional Postgres updates.

Measured on the full 252,750-claim run (see benchmark/results/detection_quality.json, regenerate with three commands from the Quickstart): recall 1.00 on all five seeded anomaly types, precision 0.84 on the high-severity tier, 0.36 on the review tier, 0.69 combined. The per-tier gap is the design working as intended: reviewers drain the high-precision tier first.

Reviewer queue against live Postgres (252,750 claims scored, top 500 open flags shown):

Detection quality, measured against seeded ground truth the pipeline never reads:

flowchart LR
    G[Claims generator\nseeded + labeled] -->|claims.csv| P[Batch pipeline]
    subgraph P [Batch pipeline]
        V[Input validation] --> R[Rule engine\nR1-R4]
        V --> F[Feature builder]
        F --> M[IsolationForest]
        R --> X[Explain layer\nreasons + severity]
        M --> X
    end
    X -->|flags| PG[(PostgreSQL\nclaims + flags)]
    G -.->|context CSV| SC[ScoringContext\nprecomputed at startup]
    SC --> API[FastAPI /score]
    PG --> DB2[Streamlit reviewer queue]
    DB2 -->|confirm / dismiss\ntransactional| PG
    API -. failure boundary: 503 when\nflag store is down .- PG

Failure handling sits at two boundaries: input validation drops malformed rows before they poison population statistics (logged with counts), and the API degrades to scoring-only when Postgres is unavailable (the /flags endpoints return 503; /score keeps working because the scoring context lives in memory).

Prerequisites: Python 3.11+, Docker (for Postgres), ~2 GB free RAM.

git clone https://github.com/MohansaiSundarasetty/claims-anomaly-radar.git
cd claims-anomaly-radar
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

docker compose up -d                                   # Postgres 16 on :5432
python -m claims_radar.data.generator --claims 250000 --out data
python -m claims_radar.pipeline --claims data/claims.csv --persist   # ~50s for 250k
python -m claims_radar.evaluate --truth data/ground_truth.csv --flags data/flags.csv

# API startup fits the forest on the context, allow ~60s before first request
RADAR_CONTEXT_CLAIMS=data/claims.csv uvicorn claims_radar.api.main:app --port 8000
streamlit run dashboard/app.py                         # reviewer queue on :8501
pytest --cov=claims_radar                              # 31 tests; RADAR_DB_TESTS=1 adds Postgres tests

All configuration is environment-driven with the RADAR_ prefix (see src/claims_radar/config.py); nothing is hardcoded.

Methodology: Locust, 60 seconds per step at 10/25/50 concurrent users, traffic mix of 80% single-claim requests and 20% 50-claim batches, against one uvicorn worker on a 2-core container with the 252,750-claim scoring context in memory. Raw CSVs in benchmark/results/.

The knee is at roughly 10 users: total throughput saturates near 23 RPS and stays flat while latency grows, because a single CPU-bound worker is doing per-row Python loops for batch feature building. Batch requests degrade first and drag single-claim p50 up with them through head-of-line blocking. The fixes are known and deliberately not done yet (see the pivot section): vectorize the batch path and add workers.

Two ADRs in docs/adr/:

• ADR-001: hybrid rules + IsolationForest over a pure ML model. Severity-by-provenance is the load-bearing idea.
• ADR-002: PostgreSQL over a document store, the boring choice, chosen for transactional reviewer state and referential integrity.

• Streaming ingestion. Scoring is batch plus on-demand API. Trigger to add a queue consumer: a flag-to-review latency requirement under one hour.
• Batch-path vectorization and multi-worker serving. The 50-claim path does per-row Python work; the benchmark shows exactly where that bites. Trigger: sustained batch traffic beyond ~5 batch RPS.
• Model retraining schedule. The forest fits at startup from the context CSV. Trigger for scheduled retrains with drift monitoring: population statistics shifting enough that the frozen normalization bounds clip more than ~1% of scores.

• All data in this repo is synthetic, generated by claims_radar.data.generator. No real member, provider, or claim data exists anywhere in the history.
• Secrets come from environment variables (RADAR_DB_PASSWORD etc.); the .env file is gitignored. The production path would be a managed secret store (Azure Key Vault or AWS Secrets Manager) injected at deploy time.
• Logs are structured JSON and never contain raw member identifiers in production configuration; the log schema carries counts and hashed IDs. Reasons shown to reviewers contain claim context by design; the dashboard is the access-controlled surface for that.
• In a real deployment this system touches PHI: the claims store would need encryption at rest, TLS in transit, role-based access on the dashboard, and an audit log of reviewer decisions (the flags.updated_at column is the seed of that trail).

Upcoding recall came back 0.0 on the first full evaluation, while every other anomaly type scored 0.8 or better. The upcoding rule looked correct, the seeded data looked correct, and unit tests on hand-built frames passed.

The diagnostic path went backwards from the evaluation join: dump the provider high-code shares the rule computed, find the provider eligibility table empty, then find the E/M family filter matching zero rows. The root cause was two layers down: pandas.read_csv inferred the all-digit CPT codes as int64, so every string-set membership check (procedure_code.isin({"99214", "99215"})) silently matched nothing. In-memory tests passed because the generator produces string codes; only the CSV round-trip broke it. The fix (commit b281923) treats codes as identifiers with an explicit dtype at every load boundary, plus a regression test that writes to CSV and reads back.

The same evaluation run exposed a second, subtler bug: duplicate recall was 0.54 because duplicated(keep="first") flags whichever copy appears first in row order, which after a shuffle is the original claim as often as the resubmission. Sorting by claim_id (submission order) before deduplication fixed it (commit 92eeaed). Both bugs were found by measuring against ground truth rather than by reading code, which is the argument for shipping the evaluation harness in the repo at all.

• Vectorize the batch scoring path (the benchmark names it as the bottleneck) and run 2+ uvicorn workers behind a socket; first metric to watch after deploying: x50 p95 under the same Locust mix.
• Add SHAP-style per-feature attribution to review-tier reasons so ML flags explain themselves as concretely as rule flags.
• Reviewer feedback loop: confirmed/dismissed decisions become weak labels for a supervised second-stage ranker.
• Drift monitor comparing live population statistics to the scoring context, gating a scheduled rebuild.
• Add procedure-diagnosis consistency rules once a diagnosis code column exists in the upstream extract.