Seek, Buffering, and Error Recovery: Engineering a Player That Survives the User
The delta between a player that "technically decodes video" and a player that is "production-ready" is defined entirely by its interaction model. When a user scrubs the timeline, spams play/pause, or rapidly backgrounds the application, they are violently disrupting the established demuxing, decoding, and rendering cadences.
This chapter resolves three architectural imperatives:
1. Why executing a Seek often induces black screens or catastrophic visual tearing.
2. Why Buffer management cannot rely on a naive "is empty / is full" binary check.
3. Why Error Recovery must explicitly stratify into Recoverable and Fatal pathways.
Seeking is Not Arbitrary Frame Jumping
To execute a seek, you must understand inter-frame video compression. Video codecs do not store every frame as a complete bitmap.
I-Frame (Key Frame): A complete, self-contained image. Functions like a standalone JPEG.
P-Frame: Predictive frame. Stores only the visual delta (differences) from previous frames.
B-Frame: Bi-directional frame. Stores deltas dependent on both previous and future frames.
If you command the extractor to jump directly to a P-Frame, the decoder lacks the requisite historical context to reconstruct the image. The mathematical result is decoding failure, macroblock tearing (glitching), or a persistent black screen.
Therefore, seeking must almost exclusively target Key Frames.
AMediaExtractor_seekTo(
extractor,
targetUs,
AMEDIAEXTRACTOR_SEEK_PREVIOUS_SYNC
);
The Architectural Sequence of a Seek
A stable seek is not a function call; it is a localized teardown and reconstruction of the pipeline timeline.
1. Receive Seek Command.
2. Increment the monotonic Seek Serial.
3. Halt Demux execution.
4. Flush (Annihilate) the Packet Queue and Frame Queue.
5. Seek the AMediaExtractor to the nearest preceding Key Frame.
6. Flush the AMediaCodec queues.
7. Reset the Master Clock baseline.
8. Resume Queue filling.
9. Render the new I-Frame.
10. Transition state back to Playing or Paused.
The C++ skeleton:
class SeekController {
public:
int64_t nextSerial() {
return ++serial_;
}
bool isCurrent(int64_t serial) const {
return serial == serial_.load();
}
private:
std::atomic<int64_t> serial_{0};
};
void performSeek(int64_t targetUs, int64_t serial) {
packetQueue_.flush();
frameQueue_.flush();
AMediaExtractor_seekTo(
extractor_,
targetUs,
AMEDIAEXTRACTOR_SEEK_PREVIOUS_SYNC
);
// CRITICAL: Reset decoder state
AMediaCodec_flush(videoCodec_);
// CRITICAL: Reset AVSync baseline
clock_.resetTo(targetUs);
// Reject stale commands
if (!seekController_.isCurrent(serial)) {
return;
}
state_ = PlayerState::Ready;
}
The serial counter prevents older, delayed seek operations from overwriting newer seeks when the user rapidly scrubs the UI timeline.
The Imperative of Codec Flushing
AMediaCodec maintains internal input and output hardware queues. When you seek the extractor, the hardware decoder remains completely unaware that the timeline has been fractured. It still possesses queued output buffers from the old timeline.
If you seek the extractor but fail to flush the codec:
1. Extractor jumps to 60.000s.
2. Codec still holds decoded frames for 10.000s.
3. Render Thread displays the 10.000s frames.
4. User experiences a severe visual "flashback" to an old scene.
AMediaCodec_flush forcefully purges the codec's internal buffers, priming it to accept a fresh, discontinuous input stream. Official Note: If you are utilizing asynchronous callbacks, calling flush requires explicitly invoking start again to re-arm the callbacks. Synchronous polling is generally safer for initiates.
Buffer Hysteresis: The Dual-Watermark Policy
Buffering operates exactly like an industrial water tank. If it runs dry, playback starves. If it overflows, RAM is exhausted and Seek latency spikes (because you must flush a massive queue).
A single threshold is an anti-pattern. If you pause at < 5 packets and resume at >= 5 packets, the engine will violently oscillate between Buffering and Playing states every microsecond, creating catastrophic UI stutter.
The engineering standard is Dual-Watermark Hysteresis.
Low Watermark: If capacity drops below this, force Buffering.
High Watermark: Only resume Playing once capacity exceeds this.
struct BufferPolicy {
size_t lowWaterPackets = 4;
size_t highWaterPackets = 24;
};
PlayerState evaluateBuffer(size_t packetCount, const BufferPolicy& policy) {
if (packetCount < policy.lowWaterPackets) {
return PlayerState::Buffering;
}
if (packetCount >= policy.highWaterPackets) {
return PlayerState::Playing;
}
// If between marks, maintain current state.
return PlayerState::Buffering;
}
Stratifying Error Taxonomy
Not all exceptions warrant annihilating the player instance.
Transient I/O Read Timeout: Recoverable (Network hiccup).
Occasional Corrupted Packet: Recoverable (Drop and proceed).
Surface Destroyed: Wait (Suspend render until new Surface attaches).
Codec Instantiation Failure: Fatal (Hardware limitation).
Unsupported Container/MIME: Fatal.
Define an explicit Error Class matrix.
enum class ErrorClass {
Recoverable,
Fatal,
};
struct PlayerError {
ErrorClass klass;
int code;
const char* source;
const char* message;
};
When the Control Plane intercepts a Recoverable error, it can trigger a Buffering state or surgically restart a specific worker thread. When it intercepts a Fatal error, it locks into the Error state and delegates UI messaging to Kotlin.
Bounded Retry Budgets
Retry loops cannot be infinite. Infinite loops mask architectural defects and permanently hang UI threads.
class RetryBudget {
public:
explicit RetryBudget(int maxAttempts) : maxAttempts_(maxAttempts) {}
bool consume() {
if (attempts_ >= maxAttempts_) return false;
++attempts_;
return true;
}
void reset() {
attempts_ = 0;
}
private:
int maxAttempts_ = 0;
int attempts_ = 0;
};
Allocate localized budgets per component:
Demux Transient Error: 3 attempts.
Decode Dequeue Timeout: 5 attempts.
Surface Unavailable: Infinite wait (gated by UI lifecycle events).
Unsupported Format: 0 attempts (Immediate Fail).
Laboratory Verification
Experiment 1: Rapid Scrubbing
Execute 20 Seek commands within 5 seconds.
Verification: The final visual output strictly matches the final Seek coordinate. Stale intermediate seeks must not execute or cause visual flashbacks.
Experiment 2: Queue Annihilation
Pre-Seek: Packets sitting in queue are tagged with PTS ~10.000s.
Post-Seek: Packet queue must exclusively contain packets with PTS near the target Key Frame.
Verification: The engine must never intertwine 10s and 60s packets.
Experiment 3: Buffer Hysteresis
Artificially throttle the Demux thread's extraction speed.
Verification: The state must cleanly transition to Buffering, hold until the High Watermark is breached, and then transition to Playing. Rapid state oscillation proves the hysteresis is broken.
Conclusion
Seeking is fundamentally timeline reconstruction; Buffering is watermark engineering; Error Recovery is classification and routing. By architecting these three pillars with rigid, defensive logic, the player graduates from a fragile script into a robust engine capable of surviving real-world user hostility.