FFmpeg/libavcodec/x86
yang 6a2bad2c4f dsputil_mmx: fix incorrect assembly code
In file libavcodec/x86/dsputil_mmx.c, function ff_put_pixels_clamped_mmx(), there are two assembly code blocks. In the first block (in the unrolled loop), the instructions "movq 8%3, %%mm1 \n\t" etc have problem.
For above instruction, it is clear what the programmer wants: a load from p + 8. But this assembly code doesn’t guarantee that. It only works if the compiler puts p in a register to produce an instruction like this: “movq 8(%edi), %mm1”. During compiler optimization, it is possible that the compiler will be able to constant propagate into p. Suppose p = &x[10000]. Then operand 3 can become 10000(%edi), where %edi holds &x. And the instruction becomes “movq 810000(%edx)”. That is, it will stride by 810000 instead of 8.
This will cause the segmentation fault.
This error was fixed in the second block of the assembly code, but not in the unrolled loop.

How to reproduce:
This error is exposed when we build the ffmpeg using Intel C++ Compiler, IPO+PGO optimization. The ffmpeg was crashed when decoding a mjpeg video.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-24 00:55:05 +02:00
..
ac3dsp_mmx.c Merge remote-tracking branch 'qatar/master' 2012-05-16 22:32:05 +02:00
ac3dsp.asm Merge remote-tracking branch 'qatar/master' 2012-05-16 22:32:05 +02:00
cabac.h Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
cavsdsp_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
dct32_sse.asm
deinterlace.asm
diracdsp_mmx.c
diracdsp_mmx.h
diracdsp_yasm.asm diracdsp: try to fix segfault 2012-06-16 23:16:54 +02:00
dnxhd_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
dsputil_mmx_avg_template.c
dsputil_mmx_qns_template.c
dsputil_mmx_rnd_template.c
dsputil_mmx.c dsputil_mmx: fix incorrect assembly code 2012-07-24 00:55:05 +02:00
dsputil_mmx.h Merge remote-tracking branch 'qatar/master' 2012-06-24 02:09:53 +02:00
dsputil_yasm.asm Merge remote-tracking branch 'qatar/master' 2012-07-23 21:25:09 +02:00
dsputilenc_mmx.c Merge remote-tracking branch 'qatar/master' 2012-07-04 21:03:28 +02:00
dsputilenc_yasm.asm
dwt_yasm.asm
dwt.c libavcodec/x86/dwt.c: move some missed things under HAVE_YASM 2012-06-10 03:20:27 +02:00
dwt.h
fdct_mmx.c
fft_3dn2.c x86: use new schema for ASM macros 2012-05-29 14:49:45 +02:00
fft_3dn.c
fft_mmx.asm Merge remote-tracking branch 'qatar/master' 2012-06-28 01:08:52 +02:00
fft.c x86: use new schema for ASM macros 2012-05-29 14:49:45 +02:00
fft.h x86: use new schema for ASM macros 2012-05-29 14:49:45 +02:00
fmtconvert_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-30 22:44:18 +02:00
fmtconvert.asm Merge remote-tracking branch 'qatar/master' 2012-06-30 22:44:18 +02:00
h264_chromamc_10bit.asm
h264_chromamc.asm Merge remote-tracking branch 'qatar/master' 2012-04-13 00:39:19 +02:00
h264_deblock_10bit.asm
h264_deblock.asm Merge remote-tracking branch 'qatar/master' 2012-04-13 00:39:19 +02:00
h264_i386.h Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
h264_idct_10bit.asm Merge remote-tracking branch 'qatar/master' 2012-04-13 00:39:19 +02:00
h264_idct.asm Merge remote-tracking branch 'qatar/master' 2012-04-13 00:39:19 +02:00
h264_intrapred_10bit.asm
h264_intrapred_init.c Merge remote-tracking branch 'qatar/master' 2012-07-05 21:55:31 +02:00
h264_intrapred.asm Merge remote-tracking branch 'qatar/master' 2012-07-07 20:29:43 +02:00
h264_qpel_10bit.asm x86inc improvements for 64-bit 2012-04-11 15:47:00 -04:00
h264_qpel_mmx.c Fix compilation with --disable-everything. 2012-05-24 08:08:31 +02:00
h264_weight_10bit.asm h264: use proper PROLOGUE statement for a function using 8 registers. 2012-04-16 08:07:21 -07:00
h264_weight.asm Merge remote-tracking branch 'qatar/master' 2012-04-13 00:39:19 +02:00
h264dsp_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-24 02:09:53 +02:00
idct_mmx_xvid.c
idct_mmx.c
idct_sse2_xvid.c
idct_xvid.h
imdct36_sse.asm
lpc_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
Makefile Merge remote-tracking branch 'qatar/master' 2012-07-18 22:34:48 +02:00
mathops.h Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
mlpdsp.c
motion_est_mmx.c
mpegaudiodec_mmx.c
mpegvideo_mmx_template.c
mpegvideo_mmx.c
pngdsp-init.c
pngdsp.asm
proresdsp-init.c
proresdsp.asm
rv34dsp_init.c rv34dsp x86: implement MMX2 inverse transform 2012-04-28 10:58:47 -07:00
rv34dsp.asm Merge remote-tracking branch 'qatar/master' 2012-04-29 21:45:54 +02:00
rv40dsp_init.c Merge remote-tracking branch 'qatar/master' 2012-06-22 22:34:02 +02:00
rv40dsp.asm Merge remote-tracking branch 'qatar/master' 2012-05-16 22:32:05 +02:00
sbrdsp_init.c
sbrdsp.asm dsputil x86: use SSE float instruction instead of SSE2 integer equivalent 2012-04-04 11:24:27 -07:00
simple_idct_mmx.c
snowdsp_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-25 16:55:31 +02:00
v210-init.c
v210.asm
vc1dsp_mmx.c Merge remote-tracking branch 'qatar/master' 2012-06-30 22:44:18 +02:00
vc1dsp_yasm.asm Merge remote-tracking branch 'qatar/master' 2012-06-30 22:44:18 +02:00
vp3dsp_init.c Merge remote-tracking branch 'qatar/master' 2012-07-18 22:34:48 +02:00
vp3dsp.asm Merge remote-tracking branch 'qatar/master' 2012-07-18 22:34:48 +02:00
vp8dsp-init.c Merge remote-tracking branch 'qatar/master' 2012-07-05 21:55:31 +02:00
vp8dsp.asm Merge remote-tracking branch 'qatar/master' 2012-07-05 21:55:31 +02:00
vp56_arith.h
vp56dsp_init.c
vp56dsp.asm
w64xmmtest.c