avcodec/qdm2: Check frame size

Fixes: index 2304 out of bounds for type 'float [2304]'
Fixes: 16332/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5679142481166336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 12b909ba31)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2019-08-29 21:26:43 +02:00
parent c01055a1ff
commit 0e704512b5

View File

@ -1750,6 +1750,11 @@ static av_cold int qdm2_decode_init(AVCodecContext *avctx)
s->sub_sampling = s->fft_order - 7;
s->frequency_range = 255 / (1 << (2 - s->sub_sampling));
if (s->frame_size * 4 >> s->sub_sampling > MPA_FRAME_SIZE) {
avpriv_request_sample(avctx, "large frames");
return AVERROR_PATCHWELCOME;
}
switch ((s->sub_sampling * 2 + s->channels - 1)) {
case 0: tmp = 40; break;
case 1: tmp = 48; break;