This ensures the hwaccel privdata does not leak when a frame buffer could
not be allocated (and toggle the assert when the frame is re-used).
Having no frame buffer available is quite common when using the DXVA2
hwaccel in situations where the DXVA2 renderer is being re-allocated, for
example when moving between displays.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This ensures the hwaccel privdata does not leak when a frame buffer could
not be allocated (and toggle the assert when the frame is re-used).
Having no frame buffer available is quite common when using the DXVA2
hwaccel in situations where the DXVA2 renderer is being re-allocated, for
example when moving between displays.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Number of planes is not always equal to the number of components even
for formats marked with PIX_FMT_PLANAR -- e.g. NV12 has three components
in two planes.
The total frame size is a combination of the 12 bits in the sequence
header and 2 more bits in the the sequence extension. While the
specification explicitly forbids the dimensions from the sequence header
from being 0 (thus ruling out multiples of 4096), such videos
apparrently exist in the wild so we should attempt to decode them.
Based on a patch by Michael Niedermayer <michaelni@gmx.at>
Fixes Bug 416.
To avoid allocating ridiculous amounts of memory for corrupted files,
read the input in chunks limited to filesize or an arbitrary large
amount when that is not known (chosen to be 50M).
The check `src > dst' in the form `&c->out[-back] > c->out' invokes
pointer overflow, which is undefined behavior in C.
Remove the check. Also replace `&c->out[-back] < c->out_start' with
a safe form `c->out - c->out_start < back' to avoid overflow.
CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Simplify `p->buf > p->buf + p->buf_size - 4' as `p->buf_size < 4'.
Avoid a possible out-of-bounds pointer, which is undefined behavior
in C.
CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.
(0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))
Avoid oversized shifting.
CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This will help in supporting old versions, e.g. version 3.93 uses the same
range coder but different predictor and version 3.82 uses different range
coder and predictor. Also this should not make decoding newer versions slower
by introducing additional checks on versions.
The tool nowadays supports more than one track per file,
this makes reading the code slightly less confusing.
Signed-off-by: Martin Storsjö <martin@martin.st>
