avcodec/pnmdec: Use 64bit for input size check

Fixes: out of array read
Fixes: poc3

Reported-by: VulDB CNA Team
Found-by: CookedMelon
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3faadbe2a2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2024-07-18 21:12:54 +02:00
parent 4430a979bc
commit da2240ccbe
No known key found for this signature in database
GPG Key ID: B18E8928B3948D64

View File

@ -256,7 +256,7 @@ static int pnm_decode_frame(AVCodecContext *avctx, void *data,
}
break;
case AV_PIX_FMT_GBRPF32:
if (avctx->width * avctx->height * 12 > s->bytestream_end - s->bytestream)
if (avctx->width * avctx->height * 12LL > s->bytestream_end - s->bytestream)
return AVERROR_INVALIDDATA;
scale = 1.f / s->scale;
if (s->endian) {