Skip to content

Fix A/V desync when resuming HLS with video transcode + audio copy - #16580

Merged
Bond-009 merged 12 commits into
jellyfin:masterfrom
LTe:audio-desync-when-seek
May 28, 2026
Merged

Fix A/V desync when resuming HLS with video transcode + audio copy#16580
Bond-009 merged 12 commits into
jellyfin:masterfrom
LTe:audio-desync-when-seek

Conversation

@LTe

@LTe LTe commented Apr 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Changes

When Jellyfin restarts ffmpeg mid-video with -ss (e.g., user resumes playback or seeks beyond the transcoded buffer), the demuxer seeks to the nearest source keyframe, which may be seconds before the target. For transcoded video, ffmpeg's accurate_seek (the default) discards decoded frames before the target — so video starts at the right time. But stream-copied audio bypasses the decoder entirely, so it starts from the keyframe position — potentially seconds before the video.

This creates segments where audio and video start at different times, causing A/V desync, subtitle offset, and SyncPlay drift. The issue affects both mpegts and fMP4 containers.

This PR replaces the previous -noaccurate_seek workaround with ffmpeg's noise bitstream filter:

-bsf:a noise=drop='lt(pts*tb\,<seek_seconds>)'

https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/22765#issuecomment-36410

This drops copied audio packets before the seek target at the demuxer level. Combined with accurate_seek trimming the transcoded video, both streams now start at the exact target time. No patched ffmpeg needed — the noise bsf with drop expressions has been available since ffmpeg 5.0 (https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/23da5caf09).

ffmpeg command before (fMP4 only):

ffmpeg -ss 00:01:03.063 -noaccurate_seek -i input.mkv \
    -c:v libx264 ... -c:a copy -copyts ...

Video AND audio both start at keyframe (60s) — aligned but 3s too early.

ffmpeg command before (mpegts — no workaround existed):

ffmpeg -ss 00:01:03.063 -i input.mkv \
    -c:v libx264 ... -c:a copy -copyts ...

Video starts at 63s (accurate_seek trims it), audio starts at 60s — 3s A/V desync.

ffmpeg command after (both containers):

ffmpeg -ss 00:01:03.063 -bsf:a noise=drop='lt(pts*tb\,63.063)' -i input.mkv \
    -c:v libx264 ... -c:a copy -copyts ...

Both video and audio start at 63s — aligned and accurate.

Reproduction and measurements:

Step 1: Generate test source (HEVC + AAC, keyframes every 10s) 
ffmpeg -y \
    -f lavfi -i "testsrc2=size=640x360:rate=24000/1001:duration=120" \
    -f lavfi -i "aevalsrc='if(lt(mod(t,1),0.03),0.8*sin(2*PI*1000*t),0)':s=48000:d=120" \
    -map 0:v -map 1:a \
    -c:v libx265 -preset fast -crf 28 \
    -x265-params "keyint=240:min-keyint=240" \
    -c:a aac -b:a 128k \
    -t 120 \
    desync_test.mkv

This produces HEVC video (which Jellyfin must transcode to H.264 for browsers) with AAC audio (which Jellyfin can copy). Keyframes every 10s to create a measurable gap when -ss lands between them.

Step 2: Reproduce on stock ffmpeg (no patches needed) 
mkdir before after

# Before: accurate_seek trims video but not copied audio
ffmpeg -y -ss 00:01:03.063 \
    -i desync_test.mkv \
    -map 0:0 -map 0:1 \
    -c:v libx264 -preset veryfast -crf 23 \
    -force_key_frames "expr:gte(t,n_forced*3)" \
    -c:a copy \
    -copyts -avoid_negative_ts disabled \
    -f hls -hls_time 3 -hls_segment_type fmp4 \
    -hls_fmp4_init_filename "init.mp4" -start_number 21 \
    -hls_segment_filename "before/seg%d.mp4" \
    -hls_playlist_type vod -hls_list_size 0 \
    -hls_segment_options movflags=+frag_discont \
    before/playlist.m3u8

# After: noise bsf trims copied audio to match
ffmpeg -y -ss 00:01:03.063 \
    -bsf:a "noise=drop=lt(pts*tb\,63.063)" \
    -i desync_test.mkv \
    -map 0:0 -map 0:1 \
    -c:v libx264 -preset veryfast -crf 23 \
    -force_key_frames "expr:gte(t,n_forced*3)" \
    -c:a copy \
    -copyts -avoid_negative_ts disabled \
    -f hls -hls_time 3 -hls_segment_type fmp4 \
    -hls_fmp4_init_filename "init.mp4" -start_number 21 \
    -hls_segment_filename "after/seg%d.mp4" \
    -hls_playlist_type vod -hls_list_size 0 \
    -hls_segment_options movflags=+frag_discont \
    after/playlist.m3u8
Step 3: External audio file (separate .mka) 
# Split source into video-only and audio-only files
ffmpeg -y -i desync_test.mkv -map 0:v -c:v copy -an video_only.mkv
ffmpeg -y -i desync_test.mkv -map 0:a -c:a copy audio_only.mka

mkdir ext_before ext_after

# Before: external audio starts at keyframe
ffmpeg -y -ss 00:01:03.063 -i video_only.mkv \
    -ss 00:01:03.063 -i audio_only.mka \
    -map 0:v -map 1:a \
    -c:v libx264 -preset veryfast -crf 23 \
    -force_key_frames "expr:gte(t,n_forced*3)" \
    -c:a copy \
    -copyts -avoid_negative_ts disabled \
    -f hls -hls_time 3 -hls_segment_type fmp4 \
    -hls_fmp4_init_filename "init.mp4" -start_number 21 \
    -hls_segment_filename "ext_before/seg%d.mp4" \
    -hls_playlist_type vod -hls_list_size 0 \
    -hls_segment_options movflags=+frag_discont \
    ext_before/playlist.m3u8

# After: bsf on both inputs (harmless on video-only input)
ffmpeg -y -ss 00:01:03.063 \
    -bsf:a "noise=drop=lt(pts*tb\,63.063)" -i video_only.mkv \
    -ss 00:01:03.063 \
    -bsf:a "noise=drop=lt(pts*tb\,63.063)" -i audio_only.mka \
    -map 0:v -map 1:a \
    -c:v libx264 -preset veryfast -crf 23 \
    -force_key_frames "expr:gte(t,n_forced*3)" \
    -c:a copy \
    -copyts -avoid_negative_ts disabled \
    -f hls -hls_time 3 -hls_segment_type fmp4 \
    -hls_fmp4_init_filename "init.mp4" -start_number 21 \
    -hls_segment_filename "ext_after/seg%d.mp4" \
    -hls_playlist_type vod -hls_list_size 0 \
    -hls_segment_options movflags=+frag_discont \
    ext_after/playlist.m3u8
Step 4: Video-only (no audio stream)

The bsf is harmless when no audio stream exists — ffmpeg ignores -bsf:a on inputs without audio.

mkdir noaudio
ffmpeg -y -ss 00:01:03.063 \
    -bsf:a "noise=drop=lt(pts*tb\,63.063)" \
    -i video_only.mkv \
    -map 0:v \
    -c:v libx264 -preset veryfast -crf 23 \
    -force_key_frames "expr:gte(t,n_forced*3)" \
    -copyts -avoid_negative_ts disabled \
    -f hls -hls_time 3 -hls_segment_type fmp4 \
    -hls_fmp4_init_filename "init.mp4" -start_number 21 \
    -hls_segment_filename "noaudio/seg%d.mp4" \
    -hls_playlist_type vod -hls_list_size 0 \
    -hls_segment_options movflags=+frag_discont \
    noaudio/playlist.m3u8
Step 5: Measure segment timestamps with ffprobe 
for dir in before after ext_before ext_after noaudio; do
  [ -f $dir/seg21.mp4 ] || continue
  echo "=== $dir ==="
  cat $dir/init.mp4 $dir/seg21.mp4 > /tmp/probe.mp4
  echo "video: $(ffprobe -v error -select_streams v -show_entries packet=pts_time -of csv=p=0 /tmp/probe.mp4 | head -1)"
  echo "audio: $(ffprobe -v error -select_streams a -show_entries packet=pts_time -of csv=p=0 /tmp/probe.mp4 | head -1)"
done

Results (stock ffmpeg 8.0, source keyframes at 60.06s, 70.07s, seek to 63.06s):

Single input (muxed video+audio):

Before (no bsf) After (with bsf)
seg21 video=63.06s audio=59.82s ❌ video=63.06s audio=63.08s ✓
seg22 video=66.07s audio=65.98s video=66.07s audio=65.98s
seg23 video=69.07s audio=68.99s video=69.07s audio=68.99s

External audio (separate .mkv + .mka):

Before (no bsf) After (with bsf)
seg21 video=63.10s audio=59.93s ❌ video=63.10s audio=63.08s ✓
seg22 video=66.11s audio=66.03s video=66.11s audio=66.03s
seg23 video=69.11s audio=69.03s video=69.11s audio=69.03s

Video-only (no audio stream):

With bsf
seg21 video=63.10s audio=NONE ✓ (bsf ignored, no crash)

The bsf fixes audio alignment for both single-input and external-audio cases, and is harmless when no audio stream is present.

Performance (3-hour HEVC+AAC source, seek to 1:30:00, transcode video + copy audio, 1800 HLS segments):

Time (avg of 3) Segments Output size
Without bsf 43.72s 1800 503M
With bsf 44.08s 1800 503M

Overhead: ~0.8% (~0.36s on a 44s transcode). The bsf evaluates one float comparison per audio packet — negligible compared to video encoding. Output size and segment count are identical.

Issues

Fixes #14194

This issue was extensively discussed in #15845 and PR #15926. The existing HlsAudioSeekStrategy enum previously offered two options:

  • DisableAccurateSeek (default): used -noaccurate_seek on fMP4 to align streams at the keyframe position, but caused subtitle desync and inaccurate player timestamps. Did nothing for mpegts.
  • TranscodeAudio: forces audio re-encoding when video is transcoded, which fixes sync but wastes CPU.

As @abitofevrything noted in #15845:

I don't think there's much we can do to get both accurate seeking AND minimal transcoding.

The noise bsf approach achieves exactly that — accurate timestamps with audio copy preserved, on both mpegts and fMP4, with no ffmpeg patches and no extra transcoding overhead. The approach was suggested by @GyanD (FFmpeg maintainer) in https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/22765.

What changed:

  • EncodingHelper.GetFastSeekCommandLineParameter(): replaced -noaccurate_seek with -bsf:a noise=drop='lt(pts*tb\,<seconds>)'
  • HlsAudioSeekStrategy.DisableAccurateSeek renamed to TrimCopiedAudio to reflect the new behavior
  • Removed unused OutputSeek strategy from earlier iteration
  • Added 9 unit tests covering all codec/container/seek combinations

Relevant discussions:

@Shadowghost
Shadowghost requested review from gnattu and nyanmisaka April 6, 2026 13:06
@abitofevrything

abitofevrything commented Apr 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

I'd considered this when writing my own PR, but came to the conclusion that this actually disables audio stream copy. From the ffmpeg docs for -ss:

When used as an output option (before an output url), decodes but discards input until the timestamps reach position.

The important word here is decodes - AFAIU this means that ffmpeg is doing a decode-encode round trip on the audio stream, which is exactly what we wanted to avoid. Is my understanding incorrect?

If my understanding is correct, then this PR is only adding another way of doing what HlsAudioSeekStrategy.TranscodeAudio already does, except this time it's not configurable.

@LTe

LTe commented Apr 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@abitofevrything

ffmpeg -y \
      -ss 00:01:03.063 -noaccurate_seek -fflags +genpts \
      -i /tmp/pr-verify/desync_test.mkv \
      -ss 00:01:03.063 -output_ts_offset 00:01:03.063 \
      -map 0:0 -map 0:1 \
      -c:v libx264 -preset ultrafast -crf 28 \
      -c:a copy \
      -copyts -avoid_negative_ts disabled \
      -t 3 /tmp/test.mp4

ffmpeg version 7.1.3-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with Apple clang version 17.0.0 (clang-1700.0.13.5)
  configuration: --prefix=/ffbuild/prefix --disable-shared --enable-static --pkg-config-flags=--static --enable-pthreads --cc=clang --host-cflags='-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/opt/ffbuild/prefix/include' --host-ldflags='-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/ffbuild/prefix/lib' --extra-version=Jellyfin --extra-cflags=-I/opt/ffbuild/prefix/include --extra-cxxflags= --extra-ldflags=-L/opt/ffbuild/prefix/lib --extra-ldexeflags= --extra-libs= --enable-gpl --enable-version3 --disable-ffplay --disable-debug --disable-doc --disable-sdl2 --enable-lto=thin --disable-libjack --disable-indev=jack --disable-libxcb --disable-xlib --enable-neon --enable-runtime-cpudetect --enable-audiotoolbox --enable-videotoolbox --enable-iconv --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-libxml2 --enable-openssl --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --enable-chromaprint --enable-libdav1d --disable-libfdk-aac --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libopenmpt --enable-libsrt --enable-libsvtav1 --enable-libx264 --enable-libx265 --enable-libzimg --enable-libzvbi
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, matroska,webm, from '/tmp/pr-verify/desync_test.mkv':
  Metadata:
    ENCODER         : Lavf61.7.100
  Duration: 00:02:00.02, start: -0.021000, bitrate: 530 kb/s
  Stream #0:0: Video: hevc (Main), yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc61.19.101 libx265
        DURATION        : 00:01:59.995000000
  Stream #0:1: Audio: aac (LC), 48000 Hz, mono, fltp
      Metadata:
        ENCODER         : Lavc61.19.101 aac
        DURATION        : 00:02:00.021000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[libx264 @ 0x130e08450] using SAR=1/1
[libx264 @ 0x130e08450] using cpu capabilities: ARMv8 NEON DotProd I8MM
[libx264 @ 0x130e08450] profile Constrained Baseline, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x130e08450] 264 - core 165 - H.264/MPEG-4 AVC codec - Copyleft 2003-2025 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=11 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=23 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, mp4, to '/tmp/test.mp4':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 24k tbn
      Metadata:
        DURATION        : 00:01:59.995000000
        encoder         : Lavc61.19.101 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp
      Metadata:
        ENCODER         : Lavc61.19.101 aac
        DURATION        : 00:02:00.021000000
[out#0/mp4 @ 0x600001a80240] video:435KiB audio:4KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.660801%
frame=   72 fps=0.0 q=-1.0 Lsize=     442KiB time=00:00:00.00 bitrate=N/A speed=   0x
[libx264 @ 0x130e08450] frame I:1     Avg QP:25.00  size: 13442
[libx264 @ 0x130e08450] frame P:71    Avg QP:25.24  size:  6075
[libx264 @ 0x130e08450] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x130e08450] mb P  I16..4:  5.8%  0.0%  0.0%  P16..4: 15.3%  0.0%  0.0%  0.0%  0.0%    skip:78.9%
[libx264 @ 0x130e08450] coded y,uvDC,uvAC intra: 9.4% 23.0% 14.9% inter: 7.0% 13.3% 11.4%
[libx264 @ 0x130e08450] i16 v,h,dc,p: 83% 12%  4%  1%
[libx264 @ 0x130e08450] i8c dc,h,v,p: 47% 18% 34%  1%
[libx264 @ 0x130e08450] kb/s:1184.89

take a look at the part

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)

I think in that case ffmpeg is doing copy, no decode-encode round trip.

When we force it

ffmpeg -y \
      -ss 00:01:03.063 -noaccurate_seek -fflags +genpts \
      -i /tmp/pr-verify/desync_test.mkv \
      -ss 00:01:03.063 -output_ts_offset 00:01:03.063 \
      -map 0:0 -map 0:1 \
      -c:v libx264 -preset ultrafast -crf 28 \
      -c:a aac \
      -copyts -avoid_negative_ts disabled \
      -t 3 /tmp/test_encode.mp4

you will get

ffmpeg version 7.1.3-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with Apple clang version 17.0.0 (clang-1700.0.13.5)
  configuration: --prefix=/ffbuild/prefix --disable-shared --enable-static --pkg-config-flags=--static --enable-pthreads --cc=clang --host-cflags='-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/opt/ffbuild/prefix/include' --host-ldflags='-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/ffbuild/prefix/lib' --extra-version=Jellyfin --extra-cflags=-I/opt/ffbuild/prefix/include --extra-cxxflags= --extra-ldflags=-L/opt/ffbuild/prefix/lib --extra-ldexeflags= --extra-libs= --enable-gpl --enable-version3 --disable-ffplay --disable-debug --disable-doc --disable-sdl2 --enable-lto=thin --disable-libjack --disable-indev=jack --disable-libxcb --disable-xlib --enable-neon --enable-runtime-cpudetect --enable-audiotoolbox --enable-videotoolbox --enable-iconv --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-libxml2 --enable-openssl --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --enable-chromaprint --enable-libdav1d --disable-libfdk-aac --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libopenmpt --enable-libsrt --enable-libsvtav1 --enable-libx264 --enable-libx265 --enable-libzimg --enable-libzvbi
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, matroska,webm, from '/tmp/pr-verify/desync_test.mkv':
  Metadata:
    ENCODER         : Lavf61.7.100
  Duration: 00:02:00.02, start: -0.021000, bitrate: 530 kb/s
  Stream #0:0: Video: hevc (Main), yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc61.19.101 libx265
        DURATION        : 00:01:59.995000000
  Stream #0:1: Audio: aac (LC), 48000 Hz, mono, fltp
      Metadata:
        ENCODER         : Lavc61.19.101 aac
        DURATION        : 00:02:00.021000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x11df07e60] using SAR=1/1
[libx264 @ 0x11df07e60] using cpu capabilities: ARMv8 NEON DotProd I8MM
[libx264 @ 0x11df07e60] profile Constrained Baseline, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x11df07e60] 264 - core 165 - H.264/MPEG-4 AVC codec - Copyleft 2003-2025 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=11 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=23 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, mp4, to '/tmp/test_encode.mp4':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 24k tbn
      Metadata:
        DURATION        : 00:01:59.995000000
        encoder         : Lavc61.19.101 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s
      Metadata:
        DURATION        : 00:02:00.021000000
        encoder         : Lavc61.19.101 aac
[out#0/mp4 @ 0x600001884000] video:435KiB audio:5KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.656654%
frame=   72 fps=0.0 q=-1.0 Lsize=     442KiB time=00:00:00.00 bitrate=N/A speed=   0x
[libx264 @ 0x11df07e60] frame I:1     Avg QP:25.00  size: 13442
[libx264 @ 0x11df07e60] frame P:71    Avg QP:25.24  size:  6075
[libx264 @ 0x11df07e60] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x11df07e60] mb P  I16..4:  5.8%  0.0%  0.0%  P16..4: 15.3%  0.0%  0.0%  0.0%  0.0%    skip:78.9%
[libx264 @ 0x11df07e60] coded y,uvDC,uvAC intra: 9.4% 23.0% 14.9% inter: 7.0% 13.3% 11.4%
[libx264 @ 0x11df07e60] i16 v,h,dc,p: 83% 12%  4%  1%
[libx264 @ 0x11df07e60] i8c dc,h,v,p: 47% 18% 34%  1%
[libx264 @ 0x11df07e60] kb/s:1184.89
[aac @ 0x11df77f60] Qavg: 62583.527

and here we can see round-trip

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))

and there is

[aac @ 0x11df77f60] Qavg: 62583.527

with copy mode there is no mention of aac @ 0x.....

@LTe
LTe force-pushed the audio-desync-when-seek branch from 9e77023 to fb66760 Compare April 6, 2026 14:34
@abitofevrything

abitofevrything commented Apr 6, 2026

Copy link
Copy Markdown
Contributor

Is there any reason to not revert #15926 if this is merged? This PR just appears like a better fix for the problems that PR fixes.

A full revert probably wouldn't be desirable as #15926 also removed some outdated parameters and fixed up some cases in the HLS controller, but at the very least this PR would make HlsAudioSeekStrategy redundant.

@jellyfin-bot

jellyfin-bot commented Apr 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Jellyfin API (v 10.12.0)

What's Changed

POST /SyncPlay/Queue
Request:

Changed content type : application/json

Updated QueueRequestDto :

  • Changed property Mode (string)

    Enum GroupQueueMode.

Changed content type : text/json

Updated QueueRequestDto :

  • Changed property Mode (string)

    Enum GroupQueueMode.

Changed content type : application/*+json

Updated QueueRequestDto :

  • Changed property Mode (string)

    Enum GroupQueueMode.

POST /SyncPlay/SetRepeatMode
Request:

Changed content type : application/json

Updated SetRepeatModeRequestDto :

  • Changed property Mode (string)

    Enum GroupRepeatMode.

Changed content type : text/json

Updated SetRepeatModeRequestDto :

  • Changed property Mode (string)

    Enum GroupRepeatMode.

Changed content type : application/*+json

Updated SetRepeatModeRequestDto :

  • Changed property Mode (string)

    Enum GroupRepeatMode.

POST /SyncPlay/SetShuffleMode
Request:

Changed content type : application/json

Updated SetShuffleModeRequestDto :

  • Changed property Mode (string)

    Enum GroupShuffleMode.

Changed content type : text/json

Updated SetShuffleModeRequestDto :

  • Changed property Mode (string)

    Enum GroupShuffleMode.

Changed content type : application/*+json

Updated SetShuffleModeRequestDto :

  • Changed property Mode (string)

    Enum GroupShuffleMode.

POST /Items/{itemId}/PlaybackInfo
Request:

Changed content type : application/json

Updated PlaybackInfoDto :

  • Changed property DeviceProfile (object)

    A MediaBrowser.Model.Dlna.DeviceProfile represents a set of metadata which determines which content a certain device is able to play.

    Specifically, it defines the supported containers and
    codecs (video and/or audio, including codec profiles and levels)
    the device is able to direct play (without transcoding or remuxing),
    as well as which containers/codecs to transcode to in case it isn't.

Changed content type : text/json

Updated PlaybackInfoDto :

  • Changed property DeviceProfile (object)

    A MediaBrowser.Model.Dlna.DeviceProfile represents a set of metadata which determines which content a certain device is able to play.

    Specifically, it defines the supported containers and
    codecs (video and/or audio, including codec profiles and levels)
    the device is able to direct play (without transcoding or remuxing),
    as well as which containers/codecs to transcode to in case it isn't.

Changed content type : application/*+json

Updated PlaybackInfoDto :

  • Changed property DeviceProfile (object)

    A MediaBrowser.Model.Dlna.DeviceProfile represents a set of metadata which determines which content a certain device is able to play.

    Specifically, it defines the supported containers and
    codecs (video and/or audio, including codec profiles and levels)
    the device is able to direct play (without transcoding or remuxing),
    as well as which containers/codecs to transcode to in case it isn't.

POST /LiveStreams/Open
Request:

Changed content type : application/json

Updated OpenLiveStreamDto :

  • Changed property DeviceProfile (object)

    A MediaBrowser.Model.Dlna.DeviceProfile represents a set of metadata which determines which content a certain device is able to play.

    Specifically, it defines the supported containers and
    codecs (video and/or audio, including codec profiles and levels)
    the device is able to direct play (without transcoding or remuxing),
    as well as which containers/codecs to transcode to in case it isn't.

Changed content type : text/json

Updated OpenLiveStreamDto :

  • Changed property DeviceProfile (object)

    A MediaBrowser.Model.Dlna.DeviceProfile represents a set of metadata which determines which content a certain device is able to play.

    Specifically, it defines the supported containers and
    codecs (video and/or audio, including codec profiles and levels)
    the device is able to direct play (without transcoding or remuxing),
    as well as which containers/codecs to transcode to in case it isn't.

Changed content type : application/*+json

Updated OpenLiveStreamDto :

  • Changed property DeviceProfile (object)

    A MediaBrowser.Model.Dlna.DeviceProfile represents a set of metadata which determines which content a certain device is able to play.

    Specifically, it defines the supported containers and
    codecs (video and/or audio, including codec profiles and levels)
    the device is able to direct play (without transcoding or remuxing),
    as well as which containers/codecs to transcode to in case it isn't.

POST /Users/AuthenticateByName
Return Type:

Changed response : 200 OK

User authenticated.

  • Changed content type : application/json

    • Changed property User (object)

      Class UserDto.

    • Changed property SessionInfo (object)

      Session info DTO.

  • Changed content type : application/json; profile="CamelCase"

    • Changed property User (object)

      Class UserDto.

    • Changed property SessionInfo (object)

      Session info DTO.

  • Changed content type : application/json; profile="PascalCase"

    • Changed property User (object)

      Class UserDto.

    • Changed property SessionInfo (object)

      Session info DTO.

POST /Users/AuthenticateWithQuickConnect
Return Type:

Changed response : 200 OK

User authenticated.

  • Changed content type : application/json

    • Changed property User (object)

      Class UserDto.

    • Changed property SessionInfo (object)

      Session info DTO.

  • Changed content type : application/json; profile="CamelCase"

    • Changed property User (object)

      Class UserDto.

    • Changed property SessionInfo (object)

      Session info DTO.

  • Changed content type : application/json; profile="PascalCase"

    • Changed property User (object)

      Class UserDto.

    • Changed property SessionInfo (object)

      Session info DTO.

Result

API changes are backward compatible

gnattu
gnattu approved these changes Apr 8, 2026
View reviewed changes

@gnattu gnattu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works in principle but I do noticed a slight regression in seek performance for very large files, but it was already slow enough which makes this regression likely irrelevant.

We can still keep the transcode strategy for some very edgy cases, where output seeking breaks, which means we can make current filter as another audio seek strategy.

I do not have any sample that the strategy in this PR breaks though, but for changes like this I do prefer to let the user decide for now because our users have far more video sample than myself and can usually hit the edge case.

Comment thread Jellyfin.Api/Controllers/DynamicHlsController.cs Outdated
return string.Format(
CultureInfo.InvariantCulture,
"{0} {1} -map_metadata -1 -map_chapters -1 -threads {2} {3} {4} {5} -copyts -avoid_negative_ts disabled -max_muxing_queue_size {6} -f hls -max_delay 5000000 -hls_time {7} -hls_segment_type {8} -start_number {9}{10} -hls_segment_filename \"{11}\" {12} -y \"{13}\"",
"{0} {1}{2} -map_metadata -1 -map_chapters -1 -threads {3} {4} {5} {6} -copyts -avoid_negative_ts disabled -max_muxing_queue_size {7} -f hls -max_delay 5000000 -hls_time {8} -hls_segment_type {9} -start_number {10}{11} -hls_segment_filename \"{12}\" {13} -y \"{14}\"",

@gnattu gnattu Apr 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should refactor this in the future, this is currently uh, too many numbers

@gnattu gnattu Apr 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to do in this PR though.

@LTe LTe Apr 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was my first impression, but I want to be consistent with other parts of the code

nyanmisaka
nyanmisaka requested changes Apr 8, 2026
View reviewed changes

@nyanmisaka nyanmisaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change may result in corrupted frames being output by the decoder, which will also annoy users.

Decoder errors (software and hardware are affected) 
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_nvenc))
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[h264 @ 0000020acb482dc0] cabac_init_idc 4 overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
Output #0, mp4, to 'R:\0.mp4':
Metadata:
  encoder         : Lavf62.12.100
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
Stream #0:0: Video: hevc (Main) (hev1 / 0x31766568), cuda(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 8000 kb/s, 60 fps, 15360 tbn
  Metadata:
    encoder         : Lavc62.28.100 hevc_nvenc
    BPS             : 4444527
    DURATION        : 00:01:00.034000000
    NUMBER_OF_FRAMES: 3601
    NUMBER_OF_BYTES : 33352848
    _STATISTICS_WRITING_APP: mkvmerge v94.0 ('Initiate') 64-bit
    _STATISTICS_WRITING_DATE_UTC: 2025-07-28 12:13:52
    _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Side data:
    CPB properties: bitrate max/min/avg: 8000000/0/8000000 buffer size: 16000000 vbv_delay: N/A
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp (default)
  Metadata:
    BPS             : 125667
    DURATION        : 00:01:00.023000000
    NUMBER_OF_FRAMES: 2585
    NUMBER_OF_BYTES : 942869
    _STATISTICS_WRITING_APP: mkvmerge v94.0 ('Initiate') 64-bit
    _STATISTICS_WRITING_DATE_UTC: 2025-07-28 12:13:52
    _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
  Last message repeated 1 times
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[h264 @ 0000020acb482dc0] reference count overflow
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb480cc0] reference count overflow
[h264 @ 0000020acb480cc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Error submitting packet to decoder: Invalid data found when processing input
[h264 @ 0000020acb482dc0] deblocking_filter_idc 15 out of range
[h264 @ 0000020acb482dc0] no frame!
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Decoding error: Invalid data found when processing input
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Decode error rate 0.946903 exceeds maximum 0.666667
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Task finished with error code: -1145393733 (Error number -1145393733 occurred)
[vist#0:0/h264 @ 0000020ac86fedc0] [dec:h264 @ 0000020aca300880] Terminating thread with return code -1145393733 (Error number -1145393733 occurred)
Image

nyanmisaka
nyanmisaka approved these changes Apr 8, 2026
View reviewed changes

@nyanmisaka nyanmisaka left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out that the corruption frame was caused by an open gop file, and it would fail to output first few frames after seeking to some point no matter what, regardless of the change.

However, I think this should have undergone more testing before being merged.

@sjorge

sjorge commented Apr 8, 2026

Copy link
Copy Markdown

We can still keep the transcode strategy for some very edgy cases, where output seeking breaks, which means we can make current filter as another audio seek strategy.

However, I think this should have undergone more testing before being merged.

Wouldn't the risk here be low if like gnattu suggest this gets implemented as a 3rd option for the in master configurable option on strategy? (I forgot the 2 options in master, I think old behavior, and transcode audio) so this would be a 3rd.

Depending on what is the default there is a fallback or this can be offered as an opt-in before it becomes the default.

nyanmisaka
nyanmisaka requested changes Apr 8, 2026
View reviewed changes

@nyanmisaka nyanmisaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately, I encountered a playback issue in HLS.js due to using an external audio stream.

The video was in an MKV container, while the FLAC audio was in an external MKA container. HLS.js kept requesting the two segments after a seek, even though they had already been downloaded. The issue disappeared after I reverted the change.

Image

@gnattu

gnattu commented Apr 9, 2026

Copy link
Copy Markdown
Member

External audio might not be the only edge case that users might face, I do think it is necessary to make it one of the configurable option so that users won't just got broken playback without workarounds, if they faced compatibility issues of this behavior.

By making it configurable we don't have to identify and fix all edge cases before merging.

@nyanmisaka

nyanmisaka commented Apr 9, 2026

Copy link
Copy Markdown
Member

External audio might not be the only edge case that users might face, I do think it is necessary to make it one of the configurable option so that users won't just got broken playback without workarounds, if they faced compatibility issues of this behavior.

By making it configurable we don't have to identify and fix all edge cases before merging.

It can be added as a new configurable option and should be disabled at least when remuxing external audio.

@LTe

LTe commented Apr 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

I investigated the external audio issue @nyanmisaka reported and found the root cause. When using output-level -ss with -output_ts_offset and -copyts, ffmpeg's interleaving queue sees a large DTS gap between encoded video and stream-copied audio. This exceeds the default max_interleave_delta threshold (10s), causing ffmpeg to force-flush video packets before audio arrives — producing HLS segments without audio. This affects both fMP4 and TS segments.

Adding -max_interleave_delta 0 alongside -ss and -output_ts_offset prevents the forced flush. I verified this fixes the external audio case (MKV + external MKA/FLAC) and tested 7 edge cases:

  • Internal audio (single input)
  • Very small seek (1s)
  • Seek near end of file
  • Long GOP (10s keyframe interval)
  • Multiple audio streams
  • External audio (two inputs)
  • External subtitles + audio copy

I'm not sure about 3rd option. I feel like this should be maybe separate option. Because I want ie. use DisableAccurateSeek and do not use output seek (or vice versa). Workaround is always TranscodeAudio.

Monosnap.screencast.2026-04-09.13-53-04.mp4

@LTe
LTe requested a review from nyanmisaka April 9, 2026 11:56
@gnattu

gnattu commented Apr 9, 2026

Copy link
Copy Markdown
Member

I'm not sure about 3rd option. I feel like this should be maybe separate option. Because I want ie. use DisableAccurateSeek and do not use output seek (or vice versa). Workaround is always TranscodeAudio.

What is the use case to use both transcode audio and this PR? Could you elaborate?

@LTe

LTe commented Apr 9, 2026

Copy link
Copy Markdown
Contributor Author

I'm not sure about 3rd option. I feel like this should be maybe separate option. Because I want ie. use DisableAccurateSeek and do not use output seek (or vice versa). Workaround is always TranscodeAudio.

What is the use case to use both transcode audio and this PR? Could you elaborate?

I was thinking about option when I want to use DisableAccurateSeek and output seek. I was thinking that having DisableAccurateSeek set we will add -noaccurate_seek to the command. But we are never checking that option directly. So using 3rd option — lets call it OutputSeek will disable accurate seek by default. Names are somewhat missleading.

So I guess we can have 3 options:

  • DisableAccurateSeek (default): audio copy, -noaccurate_seek, may desync
  • TranscodeAudio: re-encodes audio, accurate but costs CPU
  • OutputSeek: audio copy, output-level -ss trims precisely, little CPU overhead

@nyanmisaka

nyanmisaka commented Apr 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

I investigated the external audio issue @nyanmisaka reported and found the root cause. When using output-level -ss with -output_ts_offset and -copyts, ffmpeg's interleaving queue sees a large DTS gap between encoded video and stream-copied audio. This exceeds the default max_interleave_delta threshold (10s), causing ffmpeg to force-flush video packets before audio arrives — producing HLS segments without audio. This affects both fMP4 and TS segments.

Adding -max_interleave_delta 0 alongside -ss and -output_ts_offset prevents the forced flush. I verified this fixes the external audio case (MKV + external MKA/FLAC) and tested 7 edge cases:

  • Internal audio (single input)
  • Very small seek (1s)
  • Seek near end of file
  • Long GOP (10s keyframe interval)
  • Multiple audio streams
  • External audio (two inputs)
  • External subtitles + audio copy

Thank you for looking into it, but in my affected files, adding -max_interleave_delta 0 causes the ffmpeg hlsenc (fmp4) muxer to not write the fragments to the transcoding directory until the [q] key is pressed to exit.

Here's the ffmpeg log: FFmpeg.Transcode.log.txt

@LTe

LTe commented Apr 9, 2026

Copy link
Copy Markdown
Contributor Author

When I tested this on longer files I have the same problem. I think -max_interleave_delta 0 is not a good idea.

I think the strange thing is that I need to set -output_ts_offset {0} to match timestamps even when -copyts is set. Maybe there is a problem with the ffmpeg.

@LTe LTe mentioned this pull request Apr 10, 2026
Closed
@LTe

LTe commented Apr 10, 2026

Copy link
Copy Markdown
Contributor Author

@nyanmisaka it was strange to me that when we are using -copyts ffmpeg resets timestamps for some reason. I was playing around with ffmpeg code and created a patch both on upstream https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22779 and for jellyfin jellyfin/jellyfin-ffmpeg#685.

With this patched version we need only to use -ss with correct timestamp and that it — -copyts should be respected. All FATE tests passed from ffmpeg and works on my local machine.

Can you play around with your content if this fix things?

@abitofevrything

abitofevrything commented Apr 11, 2026

Copy link
Copy Markdown
Contributor

The added HlsAudioSeekStrategy would also require jellyfin/jellyfin-web#7473 to be updated (and merged).

LTe added 7 commits April 14, 2026 13:32
@LTe
Fix A/V desync when resuming HLS with video transcode + audio copy
14dd73e 
When Jellyfin restarts ffmpeg with -ss for a mid-video resume, the
demuxer seeks to the nearest source keyframe which may be seconds
before the target time. The video encoder discards frames before
the target and starts precisely, but copied audio starts from the
keyframe position, creating audible A/V desync (300ms+, depending
on source keyframe interval).

Add an output-level -ss after -i to trim the copied audio to match
the transcoded video start point. This is the standard ffmpeg
technique for combining fast seeking (input -ss) with precise
output alignment (output -ss).

The fix only applies when video is transcoded and audio is copied,
which is the scenario described in jellyfin#14194.

Fixes jellyfin#14194
@LTe
Fix missing audio in HLS segments when using output seek
3eebd97 
Add -max_interleave_delta 0 to the output seek parameters to prevent
ffmpeg from force-flushing video-only HLS segments when audio is
stream-copied.

The output -ss causes ffmpeg's of_streamcopy to subtract start_time
from stream-copied audio timestamps (e.g. 60s becomes 0s) while
encoded video retains its original timestamps from the trim filter
(stays at 60s). This DTS gap exceeds the default max_interleave_delta
threshold (10s), causing the muxer to flush video packets before audio
arrives, producing HLS segments without audio. This affects both fMP4
and TS segment formats.
@LTe
Add OutputSeek as a configurable HlsAudioSeekStrategy option
6041b50 
Instead of always applying the output seek, gate it behind a new
OutputSeek strategy so users can choose between:

- DisableAccurateSeek (default): audio copy, may desync
- TranscodeAudio: re-encodes audio, accurate but costs CPU
- OutputSeek: audio copy with output-level -ss, accurate, low overhead
@LTe
Remove -output_ts_offset and -max_interleave_delta from OutputSeek
efb59aa 
With the ffmpeg fix for stream-copied audio timestamp handling
(of_streamcopy), -copyts alone preserves correct timestamps without
needing -output_ts_offset to shift them back. The -max_interleave_delta 0
workaround is also no longer needed since audio packets are no longer
prematurely discarded by the trim logic.

The output -ss is kept to trim both streams to the exact seek position
after the keyframe-imprecise input seek.
@LTe
Fix A/V desync in HLS by trimming copied audio with noise bsf
c5d6d8d 
When Jellyfin seeks mid-video, -ss lands on the previous keyframe.
For transcoded video, accurate_seek (the default) discards decoded
frames before the target. For stream-copied audio, no decoder runs,
so audio starts from the keyframe — potentially seconds before the
video.

Previously, -noaccurate_seek was used on fMP4 to work around this by
making video also start at the keyframe. This fixed A/V sync but made
segments start at the wrong time (3-10s too early), causing subtitle
desync and SyncPlay drift. For mpegts (majority of clients), no
workaround existed at all.

Replace -noaccurate_seek with the noise bitstream filter:
  -bsf:a noise=drop='lt(pts*tb,<seek_seconds>)'

This drops copied audio packets before the seek target at the demuxer
level. Combined with accurate_seek trimming the transcoded video,
both streams now start at the exact target time.

The fix applies to both mpegts and fMP4 containers, resolving the
desync for all HLS clients. No patched ffmpeg needed — the noise bsf
with drop expressions has been available since ffmpeg 5.0.

Fixes jellyfin#14194
@LTe
Add unit tests for noise bsf audio trim in GetFastSeekCommandLinePara…
bc41eb8 
…meter

Tests verify that the noise bitstream filter is correctly applied when:
- HLS + transcode video + copy audio → bsf added with seek time
- Works for both mpegts and fMP4 segment containers

And NOT applied when:
- Both streams transcoded (accurate_seek handles both)
- Both streams copied (no trim needed)
- Video copied + audio transcoded (accurate_seek trims audio)
- Non-HLS jobs (Progressive)
- WTV input (excluded per existing workaround)
- Zero or null seek time
@LTe
Rename DisableAccurateSeek to TrimCopiedAudio
c0d7cdf 
The noise bsf approach no longer disables accurate seeking — it uses
accurate seek for video and trims copied audio with a bitstream
filter. Rename the enum value to reflect what it actually does.

Existing user configs with DisableAccurateSeek in encoding.xml will
fall back to the default (0) which is the same value, so behavior
is preserved.
@LTe
LTe force-pushed the audio-desync-when-seek branch from 4ecf800 to 4a9c515 Compare April 14, 2026 11:40
@sonarqubecloud

sonarqubecloud Bot commented Apr 14, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@LTe

LTe commented Apr 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

The previous iteration of this PR used an output-level -ss to trim both streams, which required a patched ffmpeg to handle the scheduler's cross-stream pacing correctly. Based on feedback from @GyanD in https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/22765#issuecomment-36410, switched to using ffmpeg's noise bitstream filter (-bsf:a noise=drop='lt(pts*tb\,<seconds>)') which achieves the same audio trim at the demuxer level. This works on stock ffmpeg 5.0+ with no patches needed.

nyanmisaka
nyanmisaka requested changes May 11, 2026
View reviewed changes
Comment thread MediaBrowser.Controller/MediaEncoding/EncodingHelper.cs Outdated
LTe added 3 commits May 19, 2026 13:40
@LTe
Gate noise bsf audio trim to ffmpeg >= 7.0
3174e9e 
The A/V desync only affects ffmpeg's threaded CLI rewrite introduced
in 7.0; earlier versions trim copied audio correctly during output seek.
@LTe
Chain noise audio trim with aac_adtstoasc in single -bsf:a
618bc4b 
Move the seek-side noise filter into GetAudioBitStreamArguments so it
co-exists with aac_adtstoasc as a comma-separated chain instead of two
conflicting -bsf:a flags (the second would have overridden the first
for HLS fMP4 segments built from MPEG-TS/AAC/HLS sources).
@LTe
Apply noise audio trim bsf unconditionally
361bb86 
Drop the >= 7.0 ffmpeg version gate. Cross-version tests show the
copied-audio leak under input -ss exists identically on ffmpeg 6.1.4,
7.1.3, and 8.0.1, so gating the workaround was a false hypothesis.
@GyanD

GyanD commented May 21, 2026 via email

Copy link
Copy Markdown

@LTe
Gate noise audio trim to ffmpeg >= 5.0
9d5f203 
The noise bsf's drop= parameter was added in ffmpeg 5.0. On 4.x the
filter exists but rejects the option ('Option drop not found'), which
would abort the HLS transcode. Skip the noise trim on < 5.0 and fall
back to the un-trimmed copied-audio behavior matching master.

Verified end-to-end against ffmpeg 4.4.6 / 6.1.4 / 7.1.3 / 8.0.1: on
4.4.6 only aac_adtstoasc is emitted (audio leak unchanged from master,
no regression); on >= 5.0 the chained noise+aac_adtstoasc produces
A/V alignment within one AAC frame.
@sonarqubecloud

sonarqubecloud Bot commented May 21, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@LTe
LTe requested a review from nyanmisaka May 27, 2026 04:58
@nyanmisaka
nyanmisaka requested a review from a team May 27, 2026 16:53
Bond-009
Bond-009 reviewed May 27, 2026
View reviewed changes
Comment thread MediaBrowser.Model/Configuration/HlsAudioSeekStrategy.cs
/// remains stream-copied (no re-encoding overhead).
/// </summary>
DisableAccurateSeek = 0,
TrimCopiedAudio = 0,

@Bond-009 Bond-009 May 27, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without a migration, won't this break existing configs?

@nyanmisaka nyanmisaka May 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In encoding.xml, this is stored as the value of the enum instead of the name of the enum, so I guess it should be safe? Moreover, these enums were introduced in unstable.

@LTe LTe May 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we store only value. But after change DisableAccurateSeek does not make sense as the name.

@abitofevrything abitofevrything May 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plus the change that added DisableAccurateSeek hasn't been released to any stable versions of Jellyfin yet, and it hasn't been exposed in the WebUI either (can't find the PR adding it right now but it has not been merged).

@nyanmisaka nyanmisaka May 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's here, but it needs some updates: jellyfin/jellyfin-web#7473

@PeachesMLG PeachesMLG May 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just updated the web PR 👍

@Bond-009
Bond-009 merged commit 8d544e4 into jellyfin:master May 28, 2026
17 of 18 checks passed
@PeachesMLG PeachesMLG mentioned this pull request May 30, 2026
Merged
@robert-w-gries robert-w-gries mentioned this pull request Aug 9, 2026
Open
@arsfeld arsfeld mentioned this pull request Sep 10, 2026
Merged
@gnattu gnattu mentioned this pull request Sep 12, 2026
Merged
@Roboatlas21 Roboatlas21 mentioned this pull request Sep 18, 2026
Closed
2 of 8 tasks
@ThomasvanEgmond ThomasvanEgmond mentioned this pull request Sep 20, 2026
Closed
@Crazy-Wx-Man Crazy-Wx-Man mentioned this pull request Sep 21, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nyanmisaka nyanmisaka nyanmisaka approved these changes
@IDisposable IDisposable IDisposable approved these changes
@gnattu gnattu gnattu approved these changes
@Bond-009 Bond-009 Bond-009 approved these changes
+2 more reviewers
@PeachesMLG PeachesMLG PeachesMLG left review comments
@abitofevrything abitofevrything abitofevrything left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Audio is out of sync when going back / resuming an episode

10 participants

@LTe @abitofevrything @jellyfin-bot @sjorge @gnattu @nyanmisaka @GyanD @IDisposable @Bond-009 @PeachesMLG