AutoScan AI — Car Damage Analysis System

car-damage-and-parts-segmentation/
├── app.py                              ← Flask entry point; registers blueprint, cleans uploads on start
├── requirements.txt
├── README.md                           ← Main README File
├── config/
│   └── config.py                       ← ALL tunable settings (paths, thresholds, resolutions, etc.)
├── uploads/                            ← Uploaded images saved here; auto-cleaned on startup
│   └── processed/
├── base/
│   ├── templates/
│   │   └── car_analysis/
│   │       └── index.html              ← Single-page frontend (dark industrial UI)
│   ├── static/
│   │   └── UI_Image/
│   │       └── UI_Screenshot.png       ← App UI screenshot (used in README)
│   └── com/
│       ├── controller/
│       │   └── car_analysis_controller.py   ← /analysis/upload and /analysis/render endpoints
│       └── service/
│           ├── image_analysis.py           ← Stage 1: YOLO car bounding-box detection
│           ├── car_parts_segmentation.py   ← Stage 2: RF-DETR 29-class parts segmentation
│           ├── car_damage_segmentation.py  ← Stage 3: RF-DETR 6-class damage + tile inference
│           ├── damage_validation.py        ← Stage 4: Spatial false-positive filter
│           ├── damage_analytics.py         ← Stage 5: Health score + per-part severity report
│           ├── output_handler.py           ← Pipeline orchestrator (calls stages 1–5)
│           └── coco_annotations.py         ← Optional COCO JSON export
└── evaluation/
    ├── README.md                               ← Top-level evaluation overview
    ├── Car Damage/
    │   ├── README.md                           ← CarDD dataset & damage model evaluation
    │   ├── Testing/
    │   │   └── evaluate_rfdetr_cardd_full.py   ← Full test-set evaluation script
    │   └── Training and Validation/
    │       └── plot_training_metrics.py        ← Training chart generator
    └── Car Parts/
        ├── README.md                           ← Car parts dataset & model evaluation
        ├── Testing/
        │   └── evaluate_rfdetr_carparts.py     ← Full test-set evaluation script
        └── Training and Validation/
            └── plot_parts_metrics.py           ← Training chart generator

User Uploads Image
        │
        ▼
[Auto-Resize] ──── if > 1920px longest side → downscale (LANCZOS4)
        │
        ▼
[Stage 1] YOLOv8 Car Detection
  → Finds the largest car bounding box
  → Adds 2% padding around it
  → Output: coords (x1, y1, x2, y2)
        │
        ▼
[Stage 2] RF-DETR Parts Segmentation (RFDETRSegNano, 960px)
  → Pre-NMS: collapse overlapping duplicate raw-class masks (IoU > 0.30)
  → Converts 19 raw classes → 29 Left/Right named classes using car side
  → Singleton enforcement: drop duplicate detections for classes
    that can only appear once (e.g. Left_Headlight)
  → Output: det_parts (masks + class IDs + labels)
        │
        ▼
[Stage 3] RF-DETR Damage Segmentation (RFDETRSegMedium, 960px)
  → Full-ROI pass: detects damage on the entire car crop
  → Tile pass: 2×2 grid with 35% overlap → finds small/fine damage
    (only crack, dent, scratch on tiles)
  → Smart merge NMS: merges overlapping/touching tile + full detections
  → Output: det_damage (masks + class IDs + confidence)
        │
        ▼
[Stage 4] Damage Validation (Spatial Filter)
  → Cross-references damage masks against part masks
  → First, a rule that applies to EVERY damage type:
      • if more than 60% of the damage mask lies off every named part
        → dropped as "mostly unknown region" (crack included)
  → Then type-specific rules, over the parts covering >= 20% of the mask:
      • dent          → must touch a part that is not glass / tyre / lamp
      • glass shatter → must touch a glass part
      • lamp broken   → must touch a lamp AND nothing else
      • tire flat     → must touch the tyre
      • scratch       → must touch a part, and must touch NO glass
      • crack         → no part rule beyond the unknown-region check
  → Removes spatially impossible detections
  → Output: filtered det_damage
        │
        ▼
[Stage 5] Analytics Engine
  → For each damage, finds which part(s) it overlaps by ≥15% AND
    are permitted for that damage type
  → Computes damage_percent = overlap_pixels / part_pixels × 100
  → Applies severity rules per damage type and percentage
  → Sums severity penalties → Health Score = max(0, 100 − Σ penalties)
  → Output: JSON report (health score, per-part severity rows)
        │
        ▼
[Drawing Pass]
  → Draws part masks on canvas (color-coded, with smooth contours)
  → Draws damage masks on canvas (semi-transparent overlay)
  → Adds labels at mask centroids
        │
        ▼
[Response]
  → original_image (base64 JPEG)
  → annotated_image (base64 JPEG)
  → parts_detections + damage_detections (metadata for legend)
  → analytics (health score + damage table)
  → Detections stored in server session for re-render
        │
        ▼
[Re-Render (no re-inference)]
  User toggles visibility / opacity →
  Frontend calls /analysis/render →
  Server re-draws from stored session detections (no model called again)

Image filename checked for prefix
        │
  Has prefix?  ──── YES ──→ Use prefix (e.g. "front_left_car.jpg" → "front_left")
        │
       NO
        │
  User selected from dropdown? ──── YES ──→ Use dropdown value as fallback
        │
       NO
        │
  Block analysis — return 422 error
  "Car side could not be determined"

Raw model output (19 classes)
        │
        ▼
Pre-NMS: For each raw class group
  → Sort detections by confidence (highest first)
  → Compute mask IoU between detections of same class
  → Suppress if IoU > 0.30 (keep highest confidence)
        │
        ▼
Side-aware class conversion (19 → 29 classes)
  → Fixed classes (Grill, Hood, Bumpers, etc.) pass through as-is
  → Side-dependent classes (Fender, Door, Headlight, etc.)
    get Left_ / Right_ prefix based on car side + box center X
  → Special: if rear view and 2 taillights detected →
    left-most = Left_Taillight, right-most = Right_Taillight
        │
        ▼
Singleton enforcement (post-conversion)
  → For each class that can appear once per image:
    if count > 1 → keep only highest-confidence detection

Car ROI (cropped from original image)
        │
        ├──→ Full-ROI pass → detect all 6 damage types
        │
        └──→ Tile pass (2×2 grid, 35% overlap):
               For each tile:
                 → Crop tile from ROI
                 → Run RF-DETR (higher threshold = 0.55)
                 → Only keep: crack, dent, scratch
                 → Offset detections back to full image coords
        │
        ▼
Smart Merge NMS across all detections:
  → Sort by confidence descending
  → For each detection, merge with others of same class if:
      box IoU > 0.35  OR
      mask IoU > 0.25 OR
      mask coverage > 0.80 OR
      masks touch within 3 px gap
  → Merged box = union of boxes, merged mask = union of masks

Fixed (side-independent):
  Diggi_Back_Door, Diggi_Back_Door_Glass, Front_Bumper,
  Front_Windshield_Glass, Grill, Hood_Bonnet, Rear_Bumper, Roof, tyre

Left side:
  Left_Fender, Left_Front_Door, Left_Front_Door_Glass,
  Left_Headlight, Left_Quarter_Panel, Left_Rear_Door,
  Left_Rear_Door_Glass, Left_Running_Board, Left_Side_Mirror, Left_Taillight

Right side:
  Right_Fender, Right_Front_Door, Right_Front_Door_Glass,
  Right_Headlight, Right_Quarter_Panel, Right_Rear_Door,
  Right_Rear_Door_Glass, Right_Running_Board, Right_Side_Mirror, Right_Taillight

Health Score = max(0,  100  −  Σ(penalty per damage row))

Where each damage row's penalty depends on its severity:
  Minor    → −3 points
  Moderate → −6 points
  Major    → −10 points
  Critical → −15 points

Severity is determined by:
  damage_percent = (overlap_pixels / part_mask_pixels) × 100
  → looked up in per-damage-type severity threshold table

Unmatched external damage — damage for which no part was credited, meaning no
part both covered at least MIN_ATTRIBUTION_OVERLAP (15%) of the damage mask *and*
was permitted for that damage type — is handled as follows:

  below MIN_EXTERNAL_PIXELS (150 px)  → discarded; it does not appear at all
  at or above 150 px                  → listed as an "Unknown_External" row
                                        with severity "—" and penalty 0

Either way it never affects the health score: with no reference part area there
is no meaningful percentage, so no severity is assigned and no penalty applies.