nest: request periodic keyframes (PLI+FIR) and expose SPS/PPS in the RTSP SDP - #2494
Open
bhamiltoncx wants to merge 3 commits into
Open
bhamiltoncx wants to merge 3 commits into
bhamiltoncx wants to merge 3 commits into
Conversation
pkg/webrtc/conn.go sends an RTCP PictureLossIndication every 2s to keep the source emitting keyframes, but only for ModePassiveProducer (WHIP/browser push). Nest is an active-pull WebRTC source (ModeActiveProducer), so it was excluded -- its keyframe interval drifts long when idle, and any consumer that joins mid-GOP (RTSP live view, snapshot grab) waits up to a full keyframe interval to start. Enable the keyframe request for the Nest source, gated on FormatName "nest/webrtc" rather than the mode so other ModeActiveProducer WebRTC sources (ring/tuya battery cams etc.) aren't forced into 2s IDRs, which would be battery- and bandwidth-hostile. PLI is media-plane RTCP -- no SDM API quota impact. Also add defer ticker.Stop() (the upstream pattern relies on GC of an unreferenced ticker, fine on go>=1.23 but explicit is safer). Measured on a Nest doorbell: keyframe interval ~3.3s -> ~1.8s; on-demand frame.jpeg grabs drop to ~1s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 0b7ac7e)
AlexxIT#2365 measured Google ignoring bare PLI on nest/webrtc sources while honoring FIR (RFC 5104, with an incrementing sequence number), whereas AlexxIT#2368 saw PLI work. Both are negotiated in the offer, so send both for the Nest source only. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NscdWb3J5aR21vzEGqs6at (cherry picked from commit b2f8268)
…IBE) The Nest WebRTC source's H264 codec FmtpLine comes from Google's SDP answer, which carries profile-level-id but no sprop-parameter-sets (WebRTC sends SPS/PPS in-band). go2rtc's RTSP server copies FmtpLine verbatim into the DESCRIBE SDP, so an RTSP consumer (e.g. ffmpeg for HomeKit live view) gets no SPS/PPS in the SDP and can't learn video dimensions until an in-band keyframe arrives -- forcing a large -probesize and slow starts (or failing outright at low probesize). Capture SPS (NAL 7) / PPS (NAL 8) from the incoming H264 RTP in the OnTrack receive loop (handling STAP-A bundling, which is how libwebrtc packs SPS+PPS+IDR) and append sprop-parameter-sets to the codec FmtpLine once. Every consumer that clones the codec (RTSP) or reads GetParameterSet (MSE/MP4) then benefits. Gated on FormatName "nest/webrtc" like the keyframe-request patch. Upstream does the equivalent for H265 in pkg/dvrip. With preload + the 2s keyframe request, SPS/PPS are captured well before any DESCRIBE. This lets RTSP consumers use a small -probesize safely (dimensions come from the SDP). Note it does not by itself make stream-copy opens instant: ffmpeg still aligns copy output to the next keyframe, so the ~2s keyframe interval (bounded by the PLI patch) remains the floor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 27743b9)
bhamiltoncx
force-pushed
the
pr-nest-keyframes
branch
from
September 13, 2026 21:38
6a768d6 to
1d7dfaa
Compare
bhamiltoncx
marked this pull request as ready for review
September 13, 2026 21:39
AlexxIT
added
enhancement
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2365. Supersedes #2368.
Problem
pkg/webrtc/conn.gosends an RTCP PLI every 2 s to keep the source emitting keyframes, but only forModePassiveProducer(WHIP / browser push). Nest is an active-pull source, so its keyframe interval drifts long when idle and any consumer joining mid-GOP waits up to a whole interval to start: RTSP live view, snapshot grabs. #2365 also measured Google ignoring bare PLI on some cameras while honouring FIR; #2368 saw PLI work.Separately, Google's SDP answer carries
profile-level-idbut nosprop-parameter-sets(WebRTC sends SPS/PPS in-band), and go2rtc copies that line verbatim into its RTSP DESCRIBE. RTSP clients such as ffmpeg then can't learn the video dimensions until an in-band keyframe arrives, which forces a large-probesizeand slow opens.Change
FormatName == "nest/webrtc"rather than the mode so other active-pull WebRTC sources (battery cameras) aren't forced into 2 s IDRs. Send a FIR (RFC 5104, incrementing sequence number) alongside the PLI, since both are negotiated. PLI and FIR are media-plane RTCP, so this doesn't touch the SDM API quota.sprop-parameter-setsto the codec's fmtp line once, so RTSP, MSE and MP4 consumers all see them. Upstream does the equivalent for H265 inpkg/dvrip.Validation
Measured on a Nest doorbell: keyframe interval ~3.3 s → ~1.8 s; on-demand
frame.jpeggrabs ~1 s. RTSP consumers open with a small-probesize.