Closed-loop fault localization and checkpoint recovery for LLM Pre-training
-
Updated
Oct 26, 2026 - Python
#
silent-data-corruption
Here are 13 public repositories matching this topic...
Open-source framework for detecting Silent Data Corruption (SDC) in production GPU/accelerator clusters
machine-learning gpu reliability cuda sdc gpu-monitoring fault-detection ai-infrastructure silent-data-corruption
-
Updated
Mar 30, 2026 - Python
Offline operator-level SDC screening for GPU clusters via deterministic training-fragment replay + golden-tensor validation
-
Updated
Sep 28, 2026 - Python
Empirically calibrated false-positive bound for one-sided checksum ABFT on bfloat16 GEMM, where the classical τ = Ku threshold is undefined. Catches 76% of exponent-bit flips against a pre-registered 90% bar.
fault-tolerance reproducible-research gpu reliability cuda pytorch gemm error-detection abft pre-registration bfloat16 numerical-computing extreme-value-theory silent-data-corruption
-
Updated
Oct 6, 2026 - Python
A PyTorch-based implementation of DrDNA, a post-hoc framework for detecting and mitigating soft errors (SDCs) in deep neural networks. The project profiles layer-wise activation statistics and computes abnormality scores to enable early, high-accuracy SDC detection, validated on ResNet-18 with CIFAR-10 using fault injection.
-
Updated
Oct 9, 2026 - Jupyter Notebook
Detects silent data corruption on individual CPU cores via one narrowly-scoped instruction test. Verifies computed results against an independent path, per core, and names the failing byte. Finds faults that all-core stress tests miss.
x86-64 intel inline-assembly sdc cpu-diagnostic raptor-lake silent-data-corruption 14th-gen cpu-degradation mercurial-cores hardware-fault-detection 13th-gen
-
Updated
Oct 4, 2026 - Assembly
SDCShield detects silent data corruption (SDC) on live systems: stress-tests CPU compute units (FMA/SIMD/compression/SVD/crypto) and byte-compares against golden values. ARM64 port of OpenDCDiag.
stress-testing arm64 aarch64 sdc data-integrity hardware-testing silent-data-corruption cpu-diagnostics
-
Updated
Oct 26, 2026 - C++
Offline, fail-closed verifier for JSONL telemetry event logs. Emits deterministic audit certificates + human summaries with explicit claims/non-claims for bottleneck and integrity review.
ai telemetry audit performance-monitoring bottleneck tail-latency distributed-training jsonl event-logs mlops checkpointing silent-data-corruption offline-verification
-
Updated
Dec 26, 2025 - Python
SDC excitation engine for arm64 servers — a stress-ng fork that spends every cycle maximizing silent-data-corruption excitation (di/dt, SMT contention, cache/TLB/interconnect pressure, SDC-directed data shaping). Detection is delegated to SDCShield.
neon stress-testing arm64 aarch64 ras stress-ng kunpeng sve2 silent-data-corruption hardware-reliability silicon-validation cpu-diagnostics
-
Updated
Oct 15, 2026 - C
[IPDPS'24] Approximately Upper Bounding GPU Kernel SDC Probability across Multiple Inputs
fault-tolerance parallel-computing high-performance-computing fuzzing silent-data-corruption gpu-applications soft-errors sdc-upper-bounding
-
Updated
Jul 22, 2024 - C++
What one flipped bit does to a quantized language model, and where it has to land to matter.
-
Updated
Oct 14, 2026 - Python
Catch GPUs that compute wrong answers, inside your PyTorch training and vLLM serving
-
Updated
Oct 22, 2026 - Python
Open silent data corruption (SDC) tester for GPUs and AI chips, with every result in OCP Test and Validation format
-
Updated
Oct 24, 2026 - Python
Add this topic to your repo
To associate your repository with the silent-data-corruption topic, visit your repo's landing page and select "manage topics."