Seeking: diagnosis, specification and measurements

Status: M1 implemented for the Windows desktop app (2026-10-18). The engine measurements below come from headless libmpv (vo=null, ao=null) on the development machine. The UI fixes are covered by unit tests and a Chromium reproduction. They have not yet been checked by hand in the running app, because GUI automation was not run while the machine was in use. For mobile, see mobile.md.

1. Diagnosis: what was actually wrong

The measurements come from crates/nova-playback-mpv/examples/seek_probe.rs, run over the synthetic corpus from scripts/generate-seek-fixtures.sh. The corpus has 15 files:

• frame rates from 23.976 to 120 fps
• 2 s and 10 s GOPs, and an all-intra file
• HEVC with B-frames, AV1 and ProRes
• variable frame rate (24, 60 and 10 fps segments)
• a 2-hour file

Ground truth is the pts of every frame, read with ffprobe. Raw tables are in docs/testing/seek/.

These were checked and are not causes:

• ms↔s conversion. secs_to_ms rounds once, and commands use {:.3} seconds.
• Stale session events. The reducer drops events from other sessions. In the rapid-seek test, no snapshot disagreed with the engine by more than 50 ms.
• Duration clamping.

One engine behaviour is correct but needs to be understood. libmpv’s exact seek shows the first frame whose pts is at or after target − 5 ms. This held for 100 % of samples on every file. When the target falls in the middle of a frame’s display time, that frame is the next one. So the frame shown can be up to one frame later than the frame “on screen at T”. FFmpeg’s accurate -ss seeking uses the same convention. NOVA keeps this behaviour and documents it in §2.

2. Specification

Fast seek, Seek { exact: false }, lands on a nearby keyframe, which can be a whole GOP away. NOVA uses it only while dragging, in two cases: in adaptive mode when the engine is too slow for exact seeks, or when the user picks “Keyframes”. The UI never presents a frame from a fast seek as the exact target.

Accurate seek, Seek { exact: true } and every SeekRelative, lands on the first frame whose pts is at or after target − 5 ms. NOVA uses it on release and for keys, the wheel, chapters, Go to time and resume.

Timeline interaction (desktop)

1. One mapping. pointerToMs(x, left, width, duration) maps edge to edge and serves hover, press, drag and release. There is no thumb inset and no step rounding. Times are float ms internally and are rounded once per command.

2. Dragging is latest-wins. At most one seek is in flight, and a newer target replaces any queued one.

   • A seek has settled when the engine reported seeking after it and then left that state. Position updates while playing do not count.
   • A seek that is never reported as started is abandoned after 250 ms. One that started is abandoned after 1.5 s.

3. Scrub mode is set by the scrub preference:

   • auto: accurate seeks while the last accurate seek settled within 120 ms, keyframe seeks otherwise.
   • accurate: always accurate.
   • fast: always keyframes.

   Release is always one accurate seek.

4. No snap-back. The playhead holds the release target until that seek settles.

5. Keys. ← → go to the configurable global actions: ±5 s, Shift ±30 s, Ctrl ±60 s, Alt ±1 s. The slider itself handles PageUp and PageDown (±10 % of the media, at least 10 s) and End.

6. Previews are requested for the exact hovered time, in 100 ms steps. A cached nearby frame can stand in meanwhile. It is dimmed and labelled with its own time.

7. Frame stepping (frame-step / frame-back-step) stays separate from time seeking.

Acceptance criteria. Engine criteria apply per corpus file, paused, with n ≥ 40.

• Accurate absolute seek: 100 % of seeks land on the frame the engine rule predicts, and the worst error is at most one frame duration + 1 ms of rounding.
• Relative ±5 s: the worst error is at most one frame duration (before the fix it was up to one GOP).
• Rapid supersession (12 seeks sent without waiting): playback settles on the last target, per the engine rule, and the snapshot agrees with the engine within 50 ms.
• UI: the hover time equals the landing time at every pixel (unit tests), and the playhead does not snap back (unit-tested state machine). This still needs to be confirmed visually in the app.

3. Results (2026-10-18)

“Before” is main at 3731b03; “after” is this change. Each error is |presented frame pts − target|. “On-screen frame” counts seeks that landed on the frame displayed at T. “Engine rule” counts seeks that landed on the first frame at or after T − 5 ms.

What the table shows:

• Relative seeks are fixed. The median error is now 0 ms on every file (before: 1.0–5.8 s). The worst case is now one frame instead of up to 9.7 s. That worst case is 36.7 ms at 23.976 fps, caused by rounding the target to 1 ms on NTSC frame rates.

• Absolute exact seeks are unchanged and correct by the engine rule (40/40 on every file). The worst error stays under one frame duration on every file. That includes VFR, whose 88 ms worst case falls in the 10 fps segment.

• Exact relative seeks are slower than the old keyframe seeks (about 2 ms). They now take:

  • 5–16 ms on H.264
  • 38 ms on long-GOP HEVC
  • 64 ms on long-GOP AV1 (p95 146 ms)

  That is acceptable for single key presses.

4. Limitations and not yet measured

• Hardware decoding was not measured. This run was headless with software decoding; hardware decoding (hwdec=auto-safe) and real display output were not part of it. libmpv applies the same rule either way, but the run should be repeated with NOVA_HWDEC.
• Seeking while playing was not measured for accuracy. The presented frame cannot be read reliably while it advances. Only the state transitions are tested (tests/engine.rs).
• No real camera or streaming media yet. The corpus is synthetic. Lawful real samples should be added.
• No chaptered fixture. Chapter seeks are accurate absolute seeks, but the seek corpus has no file with chapters yet.
• The very end of a file is not covered. Seeking past the last frame lands on the last frame (keep-open). The probe samples nothing later than duration − 1 s.

Reproduce

bash scripts/fetch-libmpv.sh
bash scripts/generate-seek-fixtures.sh          # ~540 MB in tests/fixtures/seek (git-ignored)
cargo run -p nova-playback-mpv --example seek_probe --release
cargo test -p nova-playback-mpv --test engine   # includes relative_seeks_land_on_requested_time
pnpm --filter @nova/desktop test                # seek.test.ts: mapping, coalescing, previews