Autoencoder-Guided Trust Adaptation for Robust Kalman Filtering Under Sensor Assumption Violation
Ashwin R — SASTRA Deemed University (2026)
This repository implements an AE-guided trust-adaptive Kalman Filter that detects KF-blind sensor degradations — corruptions that preserve first- and second-order innovation statistics, making them invisible to standard diagnostic monitoring.
A convolutional autoencoder (CAE), trained exclusively on nominal IMU data, produces a reconstruction-error health signal. This signal is mapped through a bounded softplus function to scale the measurement covariance R adaptively, reducing sensor trust smoothly without resetting filter states.
Validated on:
- Synthetic KF-blind degradation (30-seed Monte Carlo)
- TUM-VI magistrale1 (real robot IMU)
- EuRoC MH_01_easy (MAV IMU)
- HHAR (human activity recognition IMU)
A live hardware demo streams from an Arduino Nano 33 BLE Sense Rev2 to a PC-side AE-KF pipeline via USB Serial.
| Degradation | KF-Blind? | Δα post-onset | Detection Latency |
|---|---|---|---|
| Nominal drift | — | +1% ± 5% | — |
| SlowEnvelope | ✓ (full) | +140% ± 8% | 5.3 ± 0.6 s |
| CorrShift | ✓ (full) | +130% ± 13% | 14.8 ± 3.3 s |
| QuantDrift | partial | +362% ± 6% | 6.7 ± 0.8 s |
EuRoC cross-dataset: ᾱ = 1.027, FAR = 1.1% — near-nominal ✓
python -m venv venv && source venv/bin/activate
pip install torch numpy scipy matplotlib pyserialpython synthetic_dataset_gen/generate_kf_blind_dataset.pypython autoencoder/train_synthetic_ae.pypython ae_guided_kf/run_kf_blind_monte_carlo.pymake mc # 30-seed Monte Carlo suite
make test # unit test suite (26 tests)
make demo # launch live Arduino dashboardFlash firmware/arduino_nano_ble/aekf_stream/aekf_stream.ino → then:
python demo/live_visualization.py
# Auto-detects Arduino serial port, opens 4-panel live dashboardSee firmware/arduino_nano_ble/README.md for flash instructions.
sr-mcu/
├── ae_guided_kf/ # Core AE-guided KF module
│ ├── adaptive_kf.py # Trust-adaptive Kalman Filter
│ ├── trust_mapping.py # α(h_t) softplus mapping
│ ├── health_signal.py # EMA + z-score health signal
│ ├── metrics.py # RMSE, innovation norm, divergence
│ ├── diagnostics.py # Kalman gain dominance analysis
│ ├── innovation_diagnostics.py # NIS, Ljung-Box whiteness test
│ ├── run_kf_blind_monte_carlo.py # 30-seed Monte Carlo runner
│ ├── run_experiments.py # Full experiment suite
│ ├── run_baseline_comparison.py # Rolling F-test baseline benchmark
│ ├── run_ablation_experiment.py # α_max / β sensitivity ablation
│ ├── run_divergence_experiment.py # Time-to-divergence characterization
│ ├── run_benchmarks.py # Latency and throughput benchmarks
│ └── README.md
│
├── autoencoder/ # 1D CNN Autoencoder
│ ├── model.py # Architecture (~60K parameters)
│ ├── train.py # General training script
│ ├── train_synthetic_ae.py # KF-blind AE training (primary)
│ ├── evaluate.py # Reconstruction error stats
│ ├── evaluate_enhanced.py # Extended stats + pipeline validation
│ └── README.md
│
├── synthetic_dataset_gen/ # Synthetic IMU data generator
│ ├── generate_kf_blind_dataset.py # Main dataset generation script
│ ├── degradation_injector.py # SlowEnvelope / CorrShift / QuantDrift
│ ├── state_model.py # 6-state kinematic model (F, H, Q, R)
│ ├── trajectory_generator.py # Ground-truth trajectory generation
│ ├── imu_model.py # MEMS accelerometer + bias model
│ ├── kalman_filter.py # Standard discrete Kalman Filter
│ ├── noise_models.py # White, pink, colored noise
│ ├── config.yaml # Configuration parameters
│ └── README.md
│
├── datasets/ # Preprocessing scripts
│ ├── preprocess_tum_vi.py # TUM-VI magistrale1
│ ├── preprocess_euroc_mh01.py # EuRoC MH_01_easy
│ ├── preprocess_synthetic.py # Synthetic windowing
│ └── README.md
│
├── firmware/ # Arduino Nano 33 BLE firmware
│ └── arduino_nano_ble/
│ ├── aekf_stream/
│ │ └── aekf_stream.ino # LSM9DS1 @ 100 Hz → USB Serial
│ └── README.md
│
├── hardware/ # Host-side hardware interface
│ ├── firmware/ # Additional firmware variants
│ ├── host/ # PC-side serial utilities
│ └── README.md
│
├── demo/ # Live dashboard
│ ├── live_visualization.py # 4-panel real-time dashboard
│ ├── serial_reader.py # Non-blocking serial reader
│ └── requirements.txt
│
├── tests/ # Unit tests (26 tests)
│ ├── test_ae_guided_kf.py
│ ├── test_autoencoder.py
│ └── test_preprocessing.py
│
├── Makefile # Convenience targets
└── pytest.ini # Test configuration
The core adaptive equation:
R_k = α(h_t) · R₀
α(h_t) = 1 + (α_max - 1) · softplus(k·h_t) / softplus(k·h_sat)
h_t = zscore(EMA(reconstruction_error_t))
Properties: monotonic, bounded (α ∈ [1, α_max]), continuous, no hard thresholds.
| Component | Role |
|---|---|
| Arduino Nano 33 BLE Sense Rev2 (LSM9DS1) | Streams raw 100 Hz IMU over USB Serial |
| PC (Python) | Runs AE + KF, sends α feedback |
| Arduino LED | Green (α<1.5) / Red (α>3.0) / Blue (intermediate) |
The AE model (~60K params, PyTorch) is too large for the Arduino's 256 KB RAM; the split PC-side strategy is the permanent deployment architecture.
Raw datasets are not included. Download from official sources:
After downloading, run the corresponding datasets/preprocess_*.py script to produce windowed .npy files.
Ashwin R — SASTRA Deemed University, Thanjavur
ORCID: 0009-0000-5120-4554
The autoencoder observes. The Kalman Filter decides. Trust is reduced smoothly, never revoked abruptly.