Skip to content

Latest commit

 

History

History
294 lines (256 loc) · 14.4 KB

CHANGELOG.md

File metadata and controls

294 lines (256 loc) · 14.4 KB

Changelog

All notable changes to pyModeS are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[3.6.1] - 2026-10-19

Fixed

  • Keep rejected CPR positions out of trusted position history so repeated messages cannot validate a previously rejected position (#204).

Changed

  • Clarify that decoder timestamps use seconds from a consistent clock (#203).
  • Refresh decoder guides and network examples. The filtered streaming example now defaults to localhost:10006.

[3.6.0] — 2026-07-16

Added

  • decode(), Message.decode(), and PipeDecoder accept include_meteo=True to opt into heuristic Comm-B BDS 4,4 routine meteorological reports and BDS 4,5 meteorological hazard reports. The default remains False because these payload patterns can overlap other Comm-B registers.
  • The modes decode, modes live, and scripts/stream_filtered.py interfaces expose the same behavior through --include-meteo.

[3.5.0] — 2026-07-15

Added

  • PipeDecoder now uses the last validated airborne position as a local CPR reference for subsequent BDS 0,5 frames. This makes a position available on the current streaming decode() call when no fresh global even/odd pair is available, instead of relying on retroactive mutation of an earlier result.
  • local_ref_window configures the maximum reference age (30 seconds by default); set it to zero to disable local airborne decoding. PipeDecoder.stats["local_positions"] reports positions produced by this path.
  • Position bootstrap now locks as soon as three candidates are pairwise motion-consistent instead of always waiting for all five candidate slots. The five-candidate limit still allows two unrelated outliers to be ignored.
  • The mixed-traffic benchmark now reports immediate streaming position yield and frame-by-frame coordinate agreement for v2.21.1, released v3.4.0, and v3.5.0 in addition to throughput.

Changed

  • crc_valid is now None when an address-parity message cannot be checked independently. It remains True or False for frames whose parity can be validated without assuming the address derived from that same parity.
  • PipeDecoder uses each message's source timestamp for state validity while retaining a monotonic high-water mark for eviction. Reordered capture data therefore cannot revive expired observations or prematurely evict newer state.
  • Plausibility and position validation are isolated in a dedicated internal module, keeping the streaming pipeline and its decision comments easier to follow without changing the public API.
  • Release publishing now runs only after the full test matrix succeeds, checks that the Git tag matches the package version, and smoke-tests the exact wheel artifact that will be uploaded.

Fixed

  • Surface references remain optional and are used only for BDS 0,6 surface positions. Positions derived from surface_ref can never seed local airborne CPR decoding.
  • Rejected global position candidates cannot become local CPR references; a separate last-accepted airborne reference is maintained and TTL-evicted.
  • Bootstrap promotion, including flush(), initializes the latest accepted airborne reference while keeping surface-derived positions separate.
  • Globally decoded surface CPR positions now select the correct longitude zone across the antimeridian and other CPR zone boundaries.
  • Beast stream input resynchronizes after malformed or truncated frames instead of losing subsequent valid messages.
  • Beast receiver timestamps are back-projected from the current wall clock with rollover-safe receiver-tick deltas, avoiding dates near the Unix epoch and preserving correct timing across the 12 MHz counter rollover.
  • BDS inference compares headings using circular angular distance, so values around 0/360 degrees are scored correctly.
  • Trusted ICAO observations expire according to the configured state TTL and cannot be refreshed by older, reordered messages.
  • Integer messages outside the valid 56-bit and 112-bit Mode S frame ranges are rejected instead of being silently truncated or misclassified.
  • Network endpoint and surface-reference CLI arguments are validated before a stream starts, and network sources are closed deterministically on normal exit, errors, and interruption.
  • The live terminal UI synchronizes access to shared aircraft state, preventing concurrent decoder and renderer updates from racing.

[3.4.0] — 2026-07-13

Fixed

  • PipeDecoder no longer scans every per-ICAO cache on every timestamped message. Full TTL eviction is throttled to a configurable one-second interval by default, capped at the TTL itself. A bounded per-ICAO check still expires state exactly before it is used, so sweep throttling cannot change BDS inference at the TTL boundary. Set eviction_interval=0 for the former sweep-per-message memory-reclamation behavior.
  • Timestamped messages that contain no fields used for stateful disambiguation no longer create empty per-ICAO state entries. Existing useful state is still refreshed when such a message arrives.
  • State updates skip field extraction for downlink formats that cannot contribute to stateful inference.

Added

  • scripts/stream_filtered.py, a configurable live/replay Beast example demonstrating high-volume DF/typecode prefiltering before PipeDecoder.
  • Reproducible version-isolated benchmarks for mixed live traffic and high-cardinality streaming, including checked-in reports and the public Beast capture used by the mixed-traffic benchmark.

[3.1.0] — 2026-04-14

Streaming-robustness release. Adds defences against phantom positions at stream start and against DF20 CRC-collisions that attribute a Comm-B reply to the wrong aircraft.

Added

  • PipeDecoder bootstrap cluster analysis for the first position per ICAO: lat/lon emission is held until five candidate positions agree under a motion-consistency check. Prevents an initial phantom frame from anchoring the rolling position history. On each decode the held result dicts are retro-filled once the cluster locks. Scattered buffers reset and accumulate a fresh K.
  • PipeDecoder altitude cross-check on DF20 Comm-B replies: if the message's 13-bit AC-code disagrees with the ICAO's most recent ADS-B-derived altitude by more than _altitude_tolerance(dt) (a linearly growing, floored-and-capped window), the BDS payload fields are stripped from the result and altitude_mismatch=True is set. The AC-code altitude itself is preserved so callers see why the frame was flagged.
  • PipeDecoder.flush() method that finalizes any still-bootstrapping ICAOs, running cluster analysis on whatever candidates are buffered (even if fewer than K) and retro-filling the held result dicts in place. Automatically called by the batch decode(list, timestamps=list) code path so batch callers get positions on every resolvable frame without having to reach into PipeDecoder.
  • Four new per-stream stat counters exposed via PipeDecoder.stats: altitude_mismatch, position_rejected, bootstrap_held, bootstrap_reset.

Fixed

  • airborne_position_pair now rejects CPR pairs that resolve to physically impossible latitudes (|lat| > 90). Root cause: real DF17 pairs that straddle a cprNL zone can slip past the pre-pair NL(lat_even) == NL(lat_odd) check and produce latitudes in [90, 270) — e.g. an aircraft climbing out of Amsterdam resolving to lat ≈ 113° over the Bering Sea. Observed on ICAO 485A33 (2025-04-13 19:34:57 UTC, OpenSky feed) — now rejected at the pair resolver rather than poisoning the receiver's position history.

[3.0.0] — 2026-04-13

First release of the v3 ground-up rewrite. Not backwards-compatible with pyModeS 2.x. See docs/migration.md for the migration guide.

Added

  • Unified pyModeS.decode(msg) returning a Decoded dict with every decodable field populated in one call
  • Batch mode: pyModeS.decode(list_of_msgs, timestamps=...) preserves list length (errors become error-dicts, no exceptions)
  • PipeDecoder stateful streaming decoder — per-ICAO state, CPR pair matching, TTL eviction, DF20/21 icao_verified promotion via a trusted-ICAO set populated from clean DF17/18 plain-text addresses
  • full_dict=True mode populates every key in the canonical _FULL_SCHEMA (123 fields) for pandas / parquet workflows
  • Phase 3 BDS 5,0/6,0 disambiguation via a known= aircraft-state kwarg — scores candidate registers against recent observations of groundspeed / track / heading / mach
  • Airport database (ICAO 4-letter codes only) for surface CPR resolution: surface_ref="EHAM" or surface_ref=(lat, lon)
  • Golden-file oracle test asserting v3 decode output matches pyModeS 2.21.1 on the deduplicated tests/data/ corpus
  • MkDocs + Material documentation site (docs added in a later commit in this release)
  • modes command-line tool with two subcommands:
    • modes decode MESSAGE for one-shot decoding (pretty or compact JSON)
    • modes decode HEX1,HEX2,HEX3 inline-batch mode with shared PipeDecoder state so CPR pairs resolve across the batch
    • modes decode --file PATH for batch decoding a file of hex messages
    • modes live --network HOST:PORT for streaming TCP decode from a Mode-S Beast binary feed
    • modes live --dump-to FILE to tee JSON lines to a file. Every record carries both raw_msg (source hex) and timestamp so captures are self-contained for offline re-decode
    • modes live --tui for an interactive textual-based aircraft table: keyboard navigation (j/k/g/G), live incremental search (/) that filters as you type, sort cycling (s/r), responsive column set (7/10/18 cols by terminal width). Requires pyModeS[tui] extra
    • Graceful SIGINT/SIGTERM shutdown with final stats line
  • pyModeS[tui] optional dependency: pulls in textual>=0.50 for the modes live --tui interactive display
  • pyModeS.util public helpers for low-level message inspection without going through decode(): hex2bin, bin2int, hex2int, bin2hex, crc, df, icao, typecode, altcode, idcode, cprNL. Thin wrappers over _bits.py, _altcode.py, _idcode.py and position/_cpr.py with no logic duplication.
  • scripts/smoke_test_wheel.sh installs the freshly built wheel into a clean Python 3.12 venv and verifies the public API end to end (single-message decode, PipeDecoder baseline, batch CPR pair resolution, error-dict on malformed input).

Fixed

  • BDS 5,0 / 6,0 Phase 3 disambiguation now works cold-start (before any ADS-B frames have been cached). Root cause: the BDS 5,0 validator accepted roll angles up to ±50°, the full wire-format range, which let garbage payloads pass as track- and-turn reports with physically implausible banks (observed -44.5° on the TU Delft live feed). Tightened the gate to ±35° to match the commercial-airliner envelope (30° bank ≈ 2 G in a level turn).
  • BDS 6,0 scorer now has reference fields to compare against in streaming mode. PipeDecoder derives approximate IAS and mach from cached groundspeed + altitude using a minimal ISA calculator in pyModeS._aero (zero-wind assumption, low-Mach approximation). Previously the scorer's ias/mach slots were always empty because ADS-B airborne-velocity BDS 0,9 subtypes 1/2 only emit groundspeed; the scorer then returned +inf for every BDS 6,0 candidate and BDS 5,0 won every tie by default.
  • modes live --tui no longer freezes after the "detected beast format, resyncing" line. Two root causes: the sink context manager wrap was lost during a prior refactor, and the on-detect stderr print corrupted rich's alt-screen buffer. Restored the context manager and silenced stderr writes when --tui is active.

Changed

  • Python 3.11+ minimum (was 3.9+)
  • Internal message representation: Python int + bit-shift primitives (replaces numpy arrays and the Cython c_common extension)
  • Every decoder API → single decode() returning a dict
  • CPR resolution API: reference=(lat, lon) for airborne (180 NM tolerance), surface_ref="EHAM" or surface_ref=(lat, lon) for surface (45 NM tolerance)
  • Header fields (df, icao, crc, crc_valid) are computed eagerly in Message.__init__ as plain attributes instead of lazy cached_property descriptors (measurably faster on streaming decode loads, and the decode path always needs all four)
  • modes live stream timestamps are now derived from the Beast 48-bit MLAT counter, with tick rate auto-calibrated per connection from the delta between consecutive recv() bursts. Works on both dump1090 12 MHz counters and radarcape / GNS AirSquitter 1 GHz nanosecond counters without configuration. Gives sub-microsecond per-frame precision across a TCP burst, replacing the previous coarse one-time.time()-per-batch behaviour.
  • modes live --tui rewritten from rich.live.Live to a full textual.App with a DataTable widget, interactive row cursor, and per-tick diff-update of cells (only cells whose values actually change are redrawn). The result: no more whole-table flashing on each message, cursor and scroll position survive refreshes, and the default sort is now by ICAO address (stable) instead of last-seen (reshuffles on every message). Dropped the rich>=13.0 dependency in favour of textual>=0.50.

Removed

  • Cython extension (c_common) and its build dependency
  • Legacy function-per-field API (pms.adsb.callsign, pms.commb.bds, pms.common.hex2bin, etc.). The former submodule paths (pyModeS.adsb, pyModeS.commb, pyModeS.ehs, pyModeS.els, pyModeS.common, pyModeS.bds, pyModeS.streamer, pyModeS.extra) are intercepted by a meta-path finder in pyModeS._v2_removed which raises V2APIRemovedError (an ImportError subclass) with a migration hint pointing at pyModeS.decode() and — for pyModeS.common specifically — at the restored helpers in pyModeS.util. Both from pyModeS.X import Y and bare attribute access (pms.X) hit the same error message.
  • numpy hard dependency.
  • modeslive CLI entry point: replaced by modes live / modes decode. The v3 package still registers modeslive as a console script — running it prints a migration hint to stderr and exits with code 2, rather than vanishing into a bare command not found after a pip install -U pyModeS.