avformat/cdxl: Fix integer overflow in intermediate

Fixes: signed integer overflow: 65535 * 65312 cannot be represented in type 'int'
Fixes: 16704/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-6294115603447808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5c5575c8dc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2019-08-31 00:20:39 +02:00
parent 0394262515
commit 44b6cd884c

View File

@ -131,7 +131,8 @@ static int cdxl_read_packet(AVFormatContext *s, AVPacket *pkt)
height = AV_RB16(&cdxl->header[16]);
palette_size = AV_RB16(&cdxl->header[20]);
audio_size = AV_RB16(&cdxl->header[22]);
if (FFALIGN(width, 16) * (uint64_t)height * cdxl->header[19] > INT_MAX)
if (cdxl->header[19] == 0 ||
FFALIGN(width, 16) * (uint64_t)height * cdxl->header[19] > INT_MAX)
return AVERROR_INVALIDDATA;
if (format == 0x20)
image_size = width * height * cdxl->header[19] / 8;