The problem
H.264’s data partitioning splits each slice into up to three NAL units:
- A carries the headers and motion vectors.
- B carries the residual for intra-coded macroblocks.
- C carries the residual for inter-coded macroblocks.
A decoder that receives A but loses C can still reconstruct something useful, which is the point: it’s an error-resilience feature for lossy transports. The Extended profile allows it, and some CCTV cameras use it.
FFmpeg had partial support once. It was removed in 2015 with the note “does not
work correctly and apparently never did,” along with the claim that nothing but
the reference software used the feature. The defect was that partitions B and C
were read from the start of their payload, skipping their slice_id,
colour_plane_id and redundant_pic_cnt headers. Every residual read therefore
began at the wrong bit offset. Since then, FFmpeg has rejected these streams
outright.
What I changed
- Parsed the partition headers that the old code skipped, and matched B and C to
their A by
slice_id. - Routed each macroblock’s residual in the CAVLC decoder to the partition that matches its type: intra from B, inter from C. A partition that never arrived is reported rather than silently read from A.
- Attached partitions while the slice is queued, before any slice executes, so none can arrive too late to be used.
- Refused hardware decoding for partitioned pictures, along with the chunked-input mode. Both assume one self-contained slice per NAL unit.
- Added FATE regression tests built from reference-encoder streams: all three partitions present, C legitimately absent, and constrained intra prediction. The last case is decoded both ways, as described below.
Judgement calls
The spec and the reference software disagree. Clause 9.2.1 of the standard
says that, under constrained intra prediction, an inter-coded neighbour is
unavailable when deriving nC for an intra macroblock’s coefficient token. JM,
the ITU reference encoder and decoder, instead treats that neighbour as available
with a count of zero, so it still counts in the average. On a JM-encoded stream the
literal reading puts the residual out of sync. Every partitioned stream in the wild
comes from JM or something derived from it, so JM’s behaviour has to be the one that
works by default.
I first implemented JM’s behaviour and flagged the divergence in the PR. The
maintainer who reviewed and merged it asked for the decoder to support both,
following the existing workaround_bugs convention. The result is a new flag, FF_BUG_H264_DP_NNZ. It is
switched on automatically when the first partition A is seen, and -bug 0 gives the
standard’s reading. Streams without data partitioning decode identically either way.
Arbitrary slice order is out of scope. The standard allows B and C to be separated from their A when slices arrive out of order. That case is reported as unsupported rather than decoded wrongly, and the maintainer agreed to leave it for later.
What review changed
FFmpeg’s automated reviewer went five rounds on the series. It found real problems: partitions from one picture could be matched to the next, a scan was quadratic on many-slice pictures, and hardware decoding could slip through on the first partition. It also found intermediate commits that introduced a problem a later commit fixed. FFmpeg expects every commit to stand on its own, so I rewrote the series into eight commits that each build and are correct by themselves.
On the mailing list, one developer repeated the 2015 argument: that data partitioning is theoretical and not worth the code. The answer was a real stream from a CCTV camera that uses it, published in FFmpeg’s sample archive.
Outcome
Merged on 31 August 2026, six days after it was opened. FFmpeg’s Changelog lists “H.264 data partitioning support” for the next release after 9.0.