Commit Graph

1937 Commits

Author SHA1 Message Date
Michael Niedermayer
751e525757 swscale/utils: Fix chrSrcHSubSample for GBRAP16
Fixes part of Ticket5264

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67e5bd0c50)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-28 03:47:12 +02:00
Michael Niedermayer
19257891d1 swscale/input: Fix GBRAP16 input
Fixes part of Ticket5264

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit df36257a53)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-28 03:47:12 +02:00
Michael Niedermayer
13b75ba30a swscale/x86/output: Fix yuv2planeX_16* with unaligned destination
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f6492a2ea8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-28 03:47:12 +02:00
Michael Niedermayer
34e76d75a0 swscale/x86/output: Move code into yuv2planeX_mainloop
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d07f6e5f1c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-28 03:47:12 +02:00
Michael Niedermayer
1b6390dd5b swscale/swscale: Add some sanity checks for srcSlice* parameters
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 321e85e176)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:23:45 +01:00
Michael Niedermayer
edc7ef376b swscale/x86/rgb2rgb_template: Fix planar2x() for short width
Fixes: 451b3e0cf956c0bd2f27ed753ac24050/asan_heap-oob_2873c01_3231_7ed10a9464d15f0d57277f5917c566a8.AVI

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8a9aaab26)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:23:45 +01:00
Michael Niedermayer
1a88eef93b swscale/swscale_unscaled: Fix odd height inputs for bayer_to_yv12_wrapper()
Fixes: 372d2df1f04b49e25f109f07f90b1505/asan_heap-oob_2835d2e_8501_99e0114d7ba3a6db885d0b4684d200c1.cine
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 757248ea3c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:23:45 +01:00
Michael Niedermayer
d3cefc271d swscale/swscale_unscaled: Fix odd height inputs for bayer_to_rgb24_wrapper()
Fixes: 372d2df1f04b49e25f109f07f90b1505/asan_heap-oob_2835d2e_8501_99e0114d7ba3a6db885d0b4684d200c1.cine
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ad3b6fa7d8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:23:45 +01:00
Michael Niedermayer
4b17205c77 swscale/swscale-test: Fix slice height in random reference data creation.
Found-by: Pedro Arthur <bygrandao@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:23:45 +01:00
Michael Niedermayer
119659b759 swscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls
This avoids running various table inits unnecessarily

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc538e9dbd)

Conflicts:

	libswscale/utils.c
2016-01-15 16:00:32 +01:00
Michael Niedermayer
f0cc6f74fc swscale/yuv2rgb: Increase YUV2RGB table headroom
This makes SWS more robust
Fixes: 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264
Fixes: out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8f3a9a8c27)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 16:00:31 +01:00
Michael Niedermayer
f5060bce31 swscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e5f82a287)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 16:00:31 +01:00
Michael Niedermayer
d7a13ff23b swscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions
Fixes Ticket4960

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edf129cbc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 16:00:31 +01:00
Michael Niedermayer
d41b91ba33 swscale/utils: Fix intermediate format for cascaded alpha downscaling
Fixes Ticket4926

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b83d8be6bf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 16:00:31 +01:00
Michael Niedermayer
3ffc224435 swscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment is insufficient for SSE*
This also as a sideeffect fixes the non aligned case

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bc)
2015-12-15 11:17:44 +01:00
Michael Niedermayer
e7e4c65571 swscale/x86/rgb2rgb_template: Do not crash on misaligend stride
Fixes Ticket5013

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80bfce35cc)
2015-12-15 11:17:38 +01:00
Michael Niedermayer
dc0bc71471 swscale/utils: Fix for runtime error: left shift of negative value -1
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 325b59368d)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-14 16:51:01 +01:00
Michael Niedermayer
1cbd7b08f6 swscale/utils: Clear pix buffers
Fixes use of uninitialized memory
Fixes: a96874b9466b6edc660a519c7ad47977_signal_sigsegv_7ffff713351a_744_nc_sample.avi with memlimit 2147483648

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5d44d5c22)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-18 20:23:46 +02:00
Michael Niedermayer
d1f8eaf3d2 swscale/swscale_unscaled: Fix rounding difference with RGBA output between little and big endian
Fixes fate/dds-rgb16 on big endian

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f6ab967eae)

Conflicts:

	tests/ref/fate/dds-rgb16
2015-07-18 20:23:45 +02:00
James Almer
3f06023bd2 swscale/x86/rgb2rgb_template: fix signedness of v in shuffle_bytes_2103_{mmx,mmxext}
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit e22edbfd41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
f5c880cecb swscale/rgb2rgb_template: Fix signedness of v in shuffle_bytes_2103_c()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7604358018)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
2af2c7ecff swscale/rgb2rgb_template: Implement shuffle_bytes_0321_c and fix shuffle_bytes_2103_c on BE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit abb833c568)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
12aa4220dd swscale/rgb2rgb_template: Disable shuffle_bytes_2103_c on big endian
The function is specific to little endian

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4df3cf90bf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
James Almer
7f2ab5e50f swscale/x86/rgb2rgb_template: don't call emms on sse2/avx functions
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 0c15f2f158)
2015-06-26 16:30:23 -03:00
James Almer
aebb9410c5 swscale/x86/rgb2rgb_template: add missing xmm clobbers
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 910eeab480)
2015-06-26 16:22:56 -03:00
Nick Lewycky
891df2a1ae libswscale/x86/hscale_fast_bilinear_simd.c: There's no need to save BX if it's in the clobber list.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-13 11:59:30 +02:00
Nick Lewycky
48e9f68384 libswscale/x86/hscale_fast_bilinear_simd.c: Include BX in the clobber list on x86_64, because it isn't implicitly included when PIC is on.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-13 11:59:12 +02:00
Michael Niedermayer
54e64eaf68 swscale/x86/swscale: Fix warning about loosing significant bits in cast
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-10 15:09:04 +02:00
Michael Niedermayer
3b12335395 swscale/x86/hscale_fast_bilinear_simd: Remove ancient out-commented debug code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-07 23:58:08 +02:00
Michael Niedermayer
b28d5c49f7 Merge remote-tracking branch 'rbultje/vp9-profile23-wip'
* rbultje/vp9-profile23-wip:
  libvpxdec: add 440 pixfmts.
  vp9: add profile 2/3 fate tests.
  vp9: add inter-frame profile 2/3 suport.
  vp9: add keyframe profile 2/3 support.
  vp9: parse profile 2/3 bitdepth in frame header.
  vp9: add profile 2/3 to exported profiles.
  fate: add/update reference files for 440 addition.
  swscale: add yuv440p10/12 pixfmts.
  lavc: add yuv440p10/12 formats to aligned pixfmt list.
  lavu: add yuv440p10/12 pixfmts.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-07 13:22:04 +02:00
Michael Niedermayer
21b30947f2 swscale/x86/hscale_fast_bilinear_simd: Fix variable names in comments
Found-by: Nick Lewycky <nlewycky@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-07 03:57:21 +02:00
Ronald S. Bultje
711d8812ad swscale: add yuv440p10/12 pixfmts. 2015-05-06 12:04:31 -04:00
Rong Yan
603c839398 swscale/ppc/swscale_altivec.c: POWER LE support in yuv2planeX_8() delete macro GET_VF() it was wrong
GCC tool had a bug of PPC intrinsic interpret, which has been fixed in GCC 4.9.1. This bug lead to
errors in two of our previous patches. We found this when we update our GCC tools to 4.9.1 and by
reading the related info on GCC website. We fix our previous error in two separate commits

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-27 12:59:05 +02:00
Michael Niedermayer
b57f9f5724 swscale/swscale: Get rid of the SWS_GAMMA_CORRECT flag
This avoids using up a bit of the public flags

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-23 20:34:51 +02:00
Pedro Arthur
2a7128f4ed Add gamma encodign/decoding before/after scaling in libswscale
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-23 20:34:51 +02:00
Michael Niedermayer
bdcd36a4c8 swscale/output: Fix undefined shifts
Fixes: da14e86d8462be6493eab16bc2d40f88/asan_heap-oob_204cfd2_528_cov_340150052_COMPRESS.BMP

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-25 18:31:25 +01:00
Michael Niedermayer
324067d18b Merge commit '7ebb3022297aa00afda6800105684b8303f2608e'
* commit '7ebb3022297aa00afda6800105684b8303f2608e':
  swscale: Check memory allocation

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-22 03:40:37 +01:00
Federico Tomassetti
7ebb302229 swscale: Check memory allocation
Bug-Id: CID 1267890
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-03-21 23:18:11 +01:00
Michael Niedermayer
3e48616226 Merge commit '9a60b1fad02cb783b895b2145c3dafc01f7b337c'
* commit '9a60b1fad02cb783b895b2145c3dafc01f7b337c':
  libswscale: fix compiler warnings enumerated type mixed with another type

Conflicts:
	libswscale/swscale_unscaled.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-16 00:15:16 +01:00
Himangi Saraogi
9a60b1fad0 libswscale: fix compiler warnings enumerated type mixed with another type
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-03-15 18:29:22 +01:00
Christophe Gisquet
5d38c628b0 ppc: libswscale: use LOCAL_ALIGNED instead of DECLARE_ALIGNED
The later may yield incorrect code for on-stack variables.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-14 18:39:28 +01:00
Michael Niedermayer
fa74058dd3 swscale/utils: Fix undefined shift in initFilter()
Found-by: Clang -fsanitize=shift
Reported-by: Thierry Foucu <tfoucu@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-11 15:40:28 +01:00
Michael Niedermayer
ae20682f6b swscale: Add prefix to updateMMXDitherTables()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-27 19:23:46 +01:00
Michael Niedermayer
007498fc1a swscale/utils: clear formatConvBuffer on allocation
Fixes use of uninitialized memory
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-27 03:13:14 +01:00
Michael Niedermayer
fb8e5044b4 swscale/utils: add asserts to check filterpos
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-24 01:00:24 +01:00
Michael Niedermayer
1895d414aa swscale/utils: More carefully merge and clear coefficients outside the input
Fixes out of array read
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-24 01:00:24 +01:00
Michael Niedermayer
03bffb68f6 swscale: Use av_clip_uintp2()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-21 14:49:17 +01:00
Michael Niedermayer
b20426398c swscale/yuv2rgb: Use av_clip_uint8()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-21 14:44:47 +01:00
Michael Niedermayer
d501b986a9 swscale/bayer_template: Add () to protect the argument of BAYER_READ()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-18 01:55:15 +01:00
Michael Niedermayer
2a1b79d7e6 Merge commit 'e51f22122d23589e93ac4f0b3e570bb925755915'
* commit 'e51f22122d23589e93ac4f0b3e570bb925755915':
  swscale: Check memory allocations

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-17 22:22:17 +01:00