Fixes: timeout in 730/clusterfuzz-testcase-5265113739165696 (part 1 of 2)
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 55d7371fe0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is intended to workaround bug "665 Integer Divide Instruction May
Cause Unpredictable Behavior" on some early AMD CPUs, which causes a
div-by-zero in this codepath, such as reported in Mozilla bug #1293996.
Note that this isn't guaranteed to fix the bug, since a compiler is free
to reorder instructions that don't depend on each other. However, it
appears to fix the bug in Firefox, and a similar patch was applied to
libvpx also (see Chrome bug #599899).
(cherry picked from commit be885da342)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8b4f6d6be)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
the max value of the lookup in expanded form is:
(((1 << 7) - 1) << 1) - 65 + 1 + 64 = 254
add one entry of padding to inv_map_table[] to prevent out of bounds
access with non-conforming / fuzzed bitstreams
Signed-off-by: James Zern <jzern@google.com>
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91f860ea7)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise the check 'tile_size < size' treats a negative size as
unsigned, causing the check to pass. This subsequently leads to
segmentation faults.
This was originally fixed as part of Libav commit 72ca83, so the
original author is one of the following developers:
Anton Khirnov <anton@khirnov.net>
Diego Biurrun <diego@biurrun.de>
Luca Barbato <lu_zero@gentoo.org>
Martin Storsjö <martin@martin.st>
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b18eac7ff2)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '6d69f9f37689c999815a65a2d99999fad3a41705':
vp9: write uveob as 16-bit value for 16x16/32x32 transforms.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This is needed for future AVX2 implementations
Signed-off-by: James Almer <jamrial@gmail.com>
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The advantage of this is that the is32x32 division branch in
decode_coeffs_b is removed from the inner loop to outside the block
coef decoding loop in decode_coeffs. Also, it allows us to merge the
txsz branches from the block coef decoding loop, the context merge
and the context split.
The directional intra predictors either don't care about order (dc, h,
dc_left, tm), or they prefer inverted order (vr, dr, hd). This allows
more efficient SIMD implementations.
The spec doesn't describe how it should be decoded so this is probably
the safest thing to do. Fixes valgrind errors on fuzzed11.ivf and fixes
valgrind errors on fuzzed10.ivf differently.
Fixes invalid reads and crashes in vp90-2-05-resize.webm and fuzzed6.ivf.
The output is still not identical to what libvpx does (because we don't
actually scale in MC).
Reviewed-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Prevents some invalid memory accesses after resolution change in
vp90-2-05-resize.webm, and libvpx does this too.
Reviewed-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'ca96e337169093979d7c763064ad9dae12b3108c':
vp9: drop support for real (non-emulated) edges
Conflicts:
libavcodec/vp9block.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
The reference map is never used in such cases, but we accidently copied
it anyway. This could cause crashes if this map has not yet been
allocated. Fixes trac ticket 3188.
Reviewed-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It was previously int and would return error if decode_coeffs_b()
returns an error; however, that can never happen, so refactor all
that code to make all dependent functions return void also (all the
way up to decode_coeffs_sb()).
For a random 1080p sample, decoding time went from 9.7sec (1 threads)
to 6.0sec (2 threads) and 5.2sec (4 threads) in 2-pass decoding mode.
I don't have any samples that use the parallelmode feature, but the
gains should be higher.
We need more information from last/cur_frame than from reference
buffers, so we can use a simplified structure for reference buffers,
and then store mvs and segmentation map information in last/cur.
w and h are both read as uint16 + 1 so this can not happen. A similar
change was introduced in 97962b2 / 72ca830, with the
av_log()+AVERROR_INVALIDDATA form, suggesting it could be triggerable
somehow.
Change suggested by Ronald S. Bultje.
Those should not be necessary.
Original change by one of these developers:
Anton Khirnov <anton@khirnov.net>
Diego Biurrun <diego@biurrun.de>
Luca Barbato <lu_zero@gentoo.org>
Martin Storsjö <martin@martin.st>
See 97962b2 / 72ca830
vp8_rac_get_tree() is called with a tree of size 3, so the returned
value can not be outside [0;3]. All of the [0;3] cases are handled in
the switch, so the assert should not be triggerable by any means. A
similar change was introduced in 97962b2 / 72ca830, with the
av_log()+AVERROR_INVALIDDATA form, suggesting it could be triggerable
somehow. This assert might help static analyzer, or simply the reader.
The operands of an addition can be evaluated in any order, since
the addition isn't a sequence point. The only operators that
have a defined evaluation order are &&, ||, ?: and the sequence
operator ','.
This fixes fate-vp9 on ARM RVCT.
