avcodec/vp3: Add missing check for av_malloc

Since the av_malloc() may fail and return NULL pointer,
it is needed that the 's->edge_emu_buffer' should be checked
whether the new allocation is success.

Fixes: d14723861b ("VP3: fix decoding of videos with stride > 2048")
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Jiasheng Jiang <jiasheng@iscas.ac.cn>
(cherry picked from commit 656cb0450a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Jiasheng Jiang 2022-02-15 17:58:08 +08:00 committed by Michael Niedermayer
parent 9097001f80
commit 2cdddcd6ec
No known key found for this signature in database
GPG Key ID: B18E8928B3948D64

View File

@ -2677,8 +2677,13 @@ static int vp3_decode_frame(AVCodecContext *avctx,
if ((ret = ff_thread_get_buffer(avctx, &s->current_frame, AV_GET_BUFFER_FLAG_REF)) < 0)
goto error;
if (!s->edge_emu_buffer)
if (!s->edge_emu_buffer) {
s->edge_emu_buffer = av_malloc(9 * FFABS(s->current_frame.f->linesize[0]));
if (!s->edge_emu_buffer) {
ret = AVERROR(ENOMEM);
goto error;
}
}
if (s->keyframe) {
if (!s->theora) {