ffmpeg/FFmpeg
1
0
Fork 0
mirror of https://git.ffmpeg.org/ffmpeg.git synced 2024-09-19 21:06:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

avcodec/motion_est_template: Fix undefined behavior in small_diamond_search()

Browse Source 
Fixes: asan_heap-oob_394322e_138_cov_4265020547_CVPCMNL1_SVA_C.264

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2015-07-10 13:21:18 +02:00
parent ae55fc82a8
commit e71ca21f30
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
libavcodec/motion_est_template.c
View File

@ -426,8 +426,8 @@ static av_always_inline int small_diamond_search(MpegEncContext * s, int *best,
chroma_cmpf = s->mecc.me_cmp[size + 1];
{ /* ensure that the best point is in the MAP as h/qpel refinement needs it */
const unsigned key = (best[1]<<ME_MAP_MV_BITS) + best[0] + map_generation;
const int index= ((best[1]<<ME_MAP_SHIFT) + best[0])&(ME_MAP_SIZE-1);
const unsigned key = ((unsigned)best[1]<<ME_MAP_MV_BITS) + best[0] + map_generation;
const int index= (((unsigned)best[1]<<ME_MAP_SHIFT) + best[0])&(ME_MAP_SIZE-1);
if(map[index]!=key){ //this will be executed only very rarey
score_map[index]= cmp(s, best[0], best[1], 0, 0, size, h, ref_index, src_index, cmpf, chroma_cmpf, flags);
map[index]= key;