This is mathematically equivalent to what we were doing before, but
gives subtly different results due to rounding (rows first vs columns
first). Doing it this way makes our film grain database generation match
reference implementation and now produces bit-exact outputs in my
testing.
Rename the transposed variables to be a bit less confusing.
The spec specified indices in the order [x][y], but our code follows the
traditional C convention of [y][x]. This was not correctly account for
when calculating the base index of the grain database access.
The spec specifies x^31 + x^3 + 1 as the polynomial, but the diagram in
Figure 1-1 omits the +1 offset. The initial implementation was based on
the diagram, but this is wrong (produces subtly incorrect results).
Fixes: left shift of negative value -3
Fixes: 37788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-6024714540154880
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This bug flew under the radar because, in practice, these values are
0-initialized for the first invocation. But for subsequent invocations
(with different h/v values), reading from the uninitialized parts of
`out` is undefined behavior.
Avoid this by simply adjusting the iteration range of the following
loops. Has the added benefit of being a minor speedup.
Signed-off-by: James Almer <jamrial@gmail.com>
This results in warnings on compilers which don't support it,
objections were raised during the review process about it but went unnoticed,
and the speed benefit is highly compiler and version specific, and
also not very critical.
We generally hand-write assembly to optimize loops like that, rather
than use compiler magic, and for 40% best case scenario, it's simply
not worth it.
Plus, tree vectorization is still problematic with GCC and disabled by default
for a good reason, so enabling it locally is sketchy.
This could arguably also be a vf, but I decided to put it here since
decoders are technically required to apply film grain during the output
step, and I would rather want to avoid requiring users insert the
correct film grain synthesis filter on their own.
The code, while in C, is written in a way that unrolls/vectorizes fairly
well under -O3, and is reasonably cache friendly. On my CPU, a single
thread pushes about 400 FPS at 1080p.
Apart from hand-written assembly, one possible avenue of improvement
would be to change the access order to compute the grain row-by-row
rather than in 8x8 blocks. This requires some redundant PRNG calls, but
would make the algorithm more cache-oblivious.
The implementation has been written to the wording of SMPTE RDD 5-2006
as faithfully as I can manage. However, apart from passing a visual
inspection, no guarantee of correctness can be made due to the lack of
any publicly available reference implementation against which to
compare it.
Signed-off-by: Niklas Haas <git@haasn.dev>
Signed-off-by: James Almer <jamrial@gmail.com>
