SparkGap

(Formerly OpenSkimmer. Not affiliated with SparkSDR by M0NNB.)

Open-source, Linux-native, multi-mode amateur radio skimmer. CW + FT8 + RTTY on multiple bands simultaneously, feeding RBN-style spot output via telnet.

Status: alpha. Actively developed. Running in production on a Red Pitaya STEMlab 125-14 (8 bands: 80m / 40m / 30m / 20m / 17m / 15m / 12m / 10m) feeding the Reverse Beacon Network as WF8Z-# via Aggregator, and PSKReporter directly via the built-in pskr_feeder.py.

Reference comparisons (single 15-min recording: B1_seg2, 40m, 2026-03-19 03:15–03:30 UTC, scored two ways for the local-vs-global distinction operators care about):

• recall_local — vs the same-antenna decoder ceiling: against the SkimSrv-curated 56-call CQ key (what CW Skimmer's offline mode caught on the same audio, hand-filtered to runner CQs): 92.9% (52 of 56, slash-tolerant). This is the fair decoder-quality metric — it controls for "what our antenna could even theoretically hear at this site that night."

• recall_global — vs RBN worldwide consensus: against the RBN historical CSV (≥2 distinct skimmers confirming, 153 unique calls after slash-normalization): 49.7% (76 of 153). Lower bounded by propagation/antenna — RBN's distributed nodes hear stations our single 40m loop physically could not.

• Vs CW Skimmer's own RBN recall: CW Skimmer offline mode on the same audio caught 22.9% of the RBN-153 set (35 calls). On recall_global, SparkGap is roughly 2× CW Skimmer on this recording. 47 RBN-validated calls SparkGap caught that CW Skimmer missed; 6 CW Skimmer caught that SparkGap missed.

• Precision side: 274 unique SparkGap spots, 96 (35%) independently validated by RBN or CW Skimmer; 178 are unverified — mix of real local stations (RBN missed) and false positives. Live mode disambiguates these via SDC tee; the file replay does not.

• The "local but not global" bucket exists: of those 96 validated spots, 20 are calls where SparkGap and CW Skimmer independently agreed but RBN missed entirely — two different decoders, different algorithms, both saw the signal that no worldwide skimmer ≥2-consensus-confirmed. That's empirical evidence for the "RBN-only scoring underweights local coverage" argument: an antenna-co-located decoder catches stations RBN's distributed network doesn't.

• Live deployment alongside UT4LW's SDC-Connectors (actively- maintained closed-source alternative) on a shared antenna split: matches 67% of SDC-Connectors' spots in the first 17 minutes of operation. Each catches some calls the other doesn't — they hear different pieces of the same band.

SDC-Connectors has years of contest tuning we don't yet match. We have features it doesn't (open source, multi-mode in one process, packet-loss telemetry). One recording is one recording — these numbers are calibration, not a contest result. Methodology in tools/eval/b1_benchmark.py and eval_rbn_validated.py.

A Linux-native RBN feeder is in development. In the meantime, sparkgap's :7300 telnet is SkimSrv-format-compatible and bridges cleanly through Aggregator on a Windows box — see Feeding the RBN below.

CW/multi-mode skimming has had no good open-source option. CW Skimmer, Skimmer Server, and Aggregator (the RBN's spot forwarder) are Windows-only and closed-source. UT4LW's SDC-Connectors runs on Linux, macOS, and Windows and is excellent, but it's also closed- source and isn't designed as an RBN feeder.

So radio operators who want auditable software in their spotting path, or who want to run an RBN-feeding node from any platform, haven't had a native option. This project is that option.

For benchmarking we compare two ways:

• Offline file replay against historical CW contest recordings with answer keys generated by SkimSrv. Tells us how we'd have performed against the most established skimmer during past contests.
• Live operation against SDC-Connectors running on a Flex 6700 sharing an antenna split with the Pitaya feeding sparkgap. Same RF, same QTH, same time — apples-to-apples against the actively- maintained closed-source tool we benchmark against.

• Multi-band CW skimming via Red Pitaya (HPSDR Protocol 1). 8 bands simultaneously on a stock Pitaya 125-14, sustained (80 / 40 / 30 / 20 / 17 / 15 / 12 / 10 m).
• FT8 decoding on each band's WSPR/FT8 sub-band, minute-aligned.
• PSKReporter feeder (pskr_feeder.py) — pushes FT8 decodes to PSKReporter via IPFIX/UDP. Visible as WF8Z-1 on the global map.
• RTTY skimming (currently runs alongside FT8; auto-disabled outside RTTY contests).
• Bayesian, soft-decision-keystate CW decoder ("ITILA" in the source — full pipeline below in How the decoder works). 2-state HMM forward-backward with EM parameter estimation and 16-bin WPM marginalization; lazy bin-spawn filters noise-driven candidates before burning decoder cycles on garbage.
• Adaptive consensus gates controlled by config flags — every precision/recall trade-off is a knob the operator can turn for their use case. Default ("ship-it") mode is permissive emit; cluster filters (VE7CC's 2+ skimmer rule) handle the noise downstream.
• Telnet spot output in standard DX cluster format, locally on port 7300. This is what Aggregator consumes when bridging from sparkgap to RBN.
• Diagnostic infrastructure for validating against SDC and RBN reference streams (sdc_tee.py, rbn_tee.py, score_loop.py, score_diff.py).

The CW decoder ("ITILA" in the source — named after MacKay's Information Theory, Inference, and Learning Algorithms textbook that motivated the approach, not a single named algorithm) is a Bayesian, soft-decision keystate decoder. It maintains continuous P(key=down | t) posteriors all the way through to character matching rather than thresholding the envelope into bits at any intermediate stage.

The pipeline, per channel:

1. Channelization. Per-bin IIR LPF on the FFT-decimated baseband, producing a real-valued envelope at 200 Hz.
2. EM parameter estimation. A two-state Gaussian-mixture EM iterates the noise mean (nm), signal amplitude (A), observation variance (σ²), and dit unit length (WPM) for this channel. Warm-starts from the previous window's converged parameters when available.
3. HMM forward-backward. Given the EM-fit observation model and a two-state Markov transition matrix derived from the estimated WPM, fb_core (C extension) computes the per-sample posteriors γ[t,0] = P(space | observation) and γ[t,1] = P(mark | observation). These are continuous, never thresholded inside the decoder.
4. Speed marginalization. Sixteen WPM bins span 8-60 WPM; the forward-backward log-evidence is computed at each bin and used to marginalize γ across speed hypotheses (gamma_marg). Multi-station density triggers GMM WPM detection (M8): two candidate WPMs decoded in parallel so a slow runner doesn't lose to a faster caller's clock.
5. Late quantization. γ_marg[:,1] > 0.5 produces the binary mark/space stream — the only threshold in the whole decoder, and it happens once the posterior has integrated evidence across the full window.
6. Beam-search run-length decode. A 64-wide beam search over the mark/space run-lengths consumes Morse-element timing log-likelihoods (canonical dit / dah / inter-element / inter-character / inter-word gaps) plus an ambiguity-aware emission model. Returns the top-K decoded text hypotheses.
7. Callsign extraction. Three-pass regex+SCP scan of the decoded text: literal SCP match, length-bounded substring scan, and a Morse-distance-aware edit-1 correction (e.g. T1LZ → K1LZ).
8. Adaptive consensus gates. Outside the decoder, in SpotTracker, per-frequency leader / CQ-runner / harmonic / blacklist / patt3ch gates filter what becomes a spot. Every gate is a config flag so the operator can tune precision/recall.

What this means relative to other open-source CW decoders:

• Not threshold-based. Threshold decoders (cwdaemon, fldigi's default CW) make a hard mark/space decision per sample and lose posterior evidence. Skimmer's reputation rests on not doing that; SparkGap doesn't either.
• Speed is marginalized, not assumed. Many decoders need the operator to set WPM (or auto-detect on training data once). SparkGap re-fits per window, every window, with a 60 s context.
• Soft keystate, but post-decode SCP fusion is shallow. Right now SCP / patt3ch corrections are applied after the beam decode picks a winner. Combining word-level priors during beam selection (BALUN vs BALAN style) is a known open area — tested on a 15-min CWT recording (research/fusion_per_window.py); +0.1 pp recall in that dataset, but the test conditions weren't dense enough to exercise it. Open question.

Source: itila_core.c (full implementation, ~1400 lines C), fb_core.c (forward-backward kernel), itila_scanner.c (band-wide channelizer + lazy bin-spawn), and _ItilaScanner + SpotTracker in sparkgap.py for the Python orchestration.

• Hardware abstraction beyond Red Pitaya HPSDR. RX-888, SDRPlay, Hermes Lite 2 backends would each require a plugin author who owns that hardware. See the IqSource discussion in the project docs.
• Hardened multi-host setup. rbn_feeder.py works (see docs/rbn-feeding.md) but the production deployment we're running is a single-host install. Operators who want to split skimmer and feeder onto different boxes should treat their setup as bespoke until we publish a tested multi-host recipe.
• Documentation polish. This README is the start; expect rough edges in install, config, and operations until the project matures.
• Validated multi-recording test suite. We have one rigorous comparison run; need many more across band conditions and contest formats before publishing reliability claims.

Dependencies (Debian/Ubuntu):

apt install build-essential gcc python3 python3-numpy python3-scipy \
            libfftw3-dev pkg-config

Build the C libraries:

make libpfb_scanner.so libcw_dispatcher.so
gcc -O3 -march=native -shared -fPIC -pthread -o libhpsdr_fast.so hpsdr_fast.c -lm
gcc -O3 -march=native -ffast-math -shared -fPIC -o libitila_scanner.so itila_scanner.c -lm

(See Makefile for the full list. ITILA decoder library libitila.so build is currently in bayes-skimmer3.cpp / Makefile.)

Pitaya: install Pavel Demin's sdr_receiver_hpsdr firmware and ensure it's reachable on your LAN. Update sdr_ip in your config.

Configure: copy sk_5band.json and edit:

{
  "callsign": "YOUR_CALL",
  "grid": "YOUR_GRID",
  "sdr_ip": "192.168.X.X",
  "bands": [3590000, 7090000, 10118000, 14090000,
            18083000, 21090000, 24902000, 28090000],
  "enable_ft8": true,
  ...
}

Run:

python3 sparkgap.py --config sk_5band.json

Connect to the spot stream:

telnet localhost 7300

Should work with light Makefile tweaks ... -shared produces .dylib not .so on macOS, and ctypes needs to know to look for the dylib extension. Apple Silicon also handles -march=native differently (-mcpu=apple-m1 or just omit the flag and let clang pick).

Dependencies via Homebrew:

brew install python@3 numpy scipy fftw pkg-config

Then either build with gmake (substituting .so → .dylib in the build commands above) or run from source after a one-time port pass. PRs welcome ... we test on Linux primarily, so Mac users get to be the porting authors.

WSL2 with Ubuntu, then follow the Linux instructions verbatim. The HPSDR firmware on the Pitaya is reachable from WSL2 the same way it is from native Linux.

SparkGap emits standard DX-cluster spots locally on telnet :7300 in SkimSrv-compatible wire format.

Recommended path today: bridge through Aggregator on a Windows box pointed at sparkgap's :7300 telnet. That setup still works for operators who already have Aggregator licensed and don't want to change.

For an all-Linux native path, rbn_feeder.py reads sparkgap's local :7300 telnet and posts spots directly to RBN — no Windows / no Aggregator required. See docs/rbn-feeding.md for setup.

The decoder behavior is fully configurable. Defaults are "ship-it" mode (permissive emit, trust cluster filtering). Each gate can be re-enabled individually for stricter local behavior:

Plus the standard knobs:

• signal_min_snr — bin spawn threshold (default 12 dB; lower = more weak DX, more CPU)
• itila_max_bins — concurrent bin ceiling per band (sk_5band.json ships at 400; 200 is conservative)
• scp_bypass_threshold — sightings required for non-SCP-validated calls
• enable_ft8 — runs FT8 + RTTY pipelines (RTTY currently piggy-backs)

Useful for offline analysis and gate-flag tuning:

python3 sparkgap.py --file recording.wav --center-khz 7090 \
                       --start-min 0 --end-min 15 --config sk_5band.json

Accepts 16-bit and 24-bit stereo IQ WAV files at 192 kHz.

• sdc_tee.py — connects to a local SDC/SkimSrv telnet server and appends timestamped spots to /tmp/sdc_stream.log.
• rbn_tee.py — same for the worldwide RBN at telnet.reversebeacon.net:7000.
• SIGUSR1 to a running sparkgap.py instance dumps the current per-band IQ buffer to /tmp/diag_<band>_<HHMMSS>.wav for offline replay (requires enable_ft8: true — taps the FT8 capture buffer).

SparkGap is developed using Claude Code as the primary development tool. Contributors are encouraged to use Claude Code for consistency.

GPLv3. See LICENSE.

Fred Krause, WF8Z (EM79SM). Contact info is on QRZ.

This is a working README. Expect it to evolve as the project matures. Bug reports and pull requests welcome.

This project stands on a lot of prior work — both software and theory. Where source files are derived from another project, the original copyright headers and license remain in those files; the list below is the high-level acknowledgment.

The current production pipeline uses the ITILA decoder (original to this project, named after MacKay's textbook). Several other decoders ship in this tree as alternative paths and as research baselines — they are not currently in the live spotting pipeline, but they are included in the source distribution and their authors deserve credit regardless.

• Pavel Demin — Red Pitaya HPSDR firmware (sdr_receiver_hpsdr) and the ft8d standalone FT8 decoder.
• Rudy Bakalov (N2WQ) — GoCluster, BSD-licensed cluster server. Cross-source validation techniques (recent-on-band support floor, harmonic suppression) in this project's spot pipeline are adapted from GoCluster's design.

These are present in the tree but not in the active pipeline. Each file retains its original headers and license.

• AG1LE (Mauri Niininen) — bmorse, a Bayesian Morse decoder (bmorse-skimmer.cpp, bmorse_lib.cxx, bmorse_src/, libbmorse.so). Original copyright 2012–2014, GPL-3.0. Used during decoder research.
• TF3LJ (Loftur E. Jonasson) and the UHSDR Firmware Project — the in-process UHSDR CW decoder (uhsdr_cw_lib.cpp, uhsdr_cw.cpp, libuhsdr_cw.so). Original copyright 2016, GPL-3.0.
• W1HKJ (Dave Freese), AG1LE (Mauri Niininen), OH2BNS (Tomi Manninen), VE7IT (Lawrence Glaister) — the fldigi CW decoder. fldigi_cw.cpp is a standalone port of fldigi's cw.cxx, GPL-3.0.
• HA7ILM (Andras Retzler) and Marat Fayzullin (luarvique) — libcsdr / csdr-cwskimmer. libcsdr_cw.cpp derives from this work, BSD-licensed.
• Erik de Castro Lopo — sndfile-spectrogram, code referenced by bmorse.

• David MacKay — Information Theory, Inference, and Learning Algorithms (Cambridge University Press, 2003). The Bayesian framework behind the ITILA decoder comes from there; the decoder itself is original to this project.

• Everyone who's run a CW skimmer for the last 15 years and kept the RBN ecosystem alive.