Commit Graph

105 Commits

Author SHA1 Message Date
Michael Niedermayer
2510e820c4 Merge commit '9ecabd7892ff073ae60ded3fc0a1290f5914ed5c'
* commit '9ecabd7892ff073ae60ded3fc0a1290f5914ed5c':
  h264: reset num_reorder_frames if it is invalid

Note, num_reorder_frames is not used in the failure case

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 22:59:46 +01:00
Anton Khirnov
9ecabd7892 h264: reset num_reorder_frames if it is invalid
An invalid VUI is not considered a fatal error, so the SPS containing it
may still be used. Leaving an invalid value of num_reorder_frames there
can result in writing over the bounds of H264Context.delayed_pic.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:53:06 +01:00
Michael Niedermayer
a60abb1ee0 Merge commit 'bfd26b7ce6efea594f2b99441d900419df3af638'
* commit 'bfd26b7ce6efea594f2b99441d900419df3af638':
  h264: reject mismatching luma/chroma bit depths during sps parsing

Conflicts:
	libavcodec/h264_ps.c

See: bdeb61ccc6
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:28:55 +01:00
Anton Khirnov
bfd26b7ce6 h264: reject mismatching luma/chroma bit depths during sps parsing
There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:23:45 +01:00
Michael Niedermayer
ac3fa95e73 Merge commit '94603feb1b3ad01a821a1a1cef1570b13f471821'
* commit '94603feb1b3ad01a821a1a1cef1570b13f471821':
  h264_ps: when parsing a VUI fails, only abort when explode is set

Merge only for metadata, ignoring invalid data can put the context
into an invalid state and can possibly be used for remote code
execution exploits.
Also we support all non standard VUIs that have been reported so
ignoring parsing failure makes no sense for us unless theres
some issue that has not been reported

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-28 11:08:58 +01:00
Anton Khirnov
94603feb1b h264_ps: when parsing a VUI fails, only abort when explode is set
A VUI doesn't contain anything strictly necessary for decoding.
Apparently there are many samples with truncated VUIs in the wild, this
commit should allow decoding them.
2013-10-27 21:39:01 +01:00
Michael Niedermayer
70a73213b7 Merge commit 'c18838f5eb7d7001a9dc653f5162868c04c1b2a1'
* commit 'c18838f5eb7d7001a9dc653f5162868c04c1b2a1':
  h264_ps: Use more meaningful error values

Conflicts:
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 13:03:40 +02:00
Michael Niedermayer
e853cf5325 Merge commit '330ad1f6a53a37dec228cb424ca57e1268fafc64'
* commit '330ad1f6a53a37dec228cb424ca57e1268fafc64':
  h264_ps: K&R formatting cosmetics

Conflicts:
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 12:56:37 +02:00
Michael Niedermayer
8299ed261a Merge commit '8fed466b0a7d636ae5035f9c6074fba9a621539b'
* commit '8fed466b0a7d636ae5035f9c6074fba9a621539b':
  h264_ps: Drop commented-out cruft

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 12:32:33 +02:00
Michael Niedermayer
58e12732db Merge commit '2a61592573d725956a4377641344afe263382648'
* commit '2a61592573d725956a4377641344afe263382648':
  avcodec: Remove some commented-out debug cruft

Conflicts:
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mvpred.h
	libavcodec/mjpegdec.c
	libavcodec/mjpegenc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 11:18:43 +02:00
Diego Biurrun
c18838f5eb h264_ps: Use more meaningful error values 2013-08-20 21:24:42 +02:00
Diego Biurrun
330ad1f6a5 h264_ps: K&R formatting cosmetics 2013-08-20 21:23:24 +02:00
Diego Biurrun
8fed466b0a h264_ps: Drop commented-out cruft 2013-08-20 21:09:05 +02:00
Diego Biurrun
2a61592573 avcodec: Remove some commented-out debug cruft 2013-08-20 19:59:50 +02:00
Michael Niedermayer
00b1401df6 avcodec/h264_ps: try to support Ambarella AVC
Fixes Ticket2763

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-09 05:21:26 +02:00
Michael Niedermayer
6952e2f82a h264_ps: fix memleak in ff_h264_decode_picture_parameter_set()
Fixes: CID1026763
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 14:09:12 +02:00
Michael Niedermayer
edabbfbaf2 ff_h264_decode_seq_parameter_set: dont set h->sps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:37:47 +02:00
Michael Niedermayer
ccdaa31d86 ff_h264_decode_picture_parameter_set: fix return code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:01:47 +02:00
Michael Niedermayer
dc35b77b28 ff_h264_decode_picture_parameter_set: use the correct sps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:01:05 +02:00
Michael Niedermayer
c3bd306e78 h264_ps: check croping values
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-30 23:48:53 +02:00
Michael Niedermayer
f4b05cd841 Merge commit '5e83d9aced2fc2b2e1360452794c58aba55d497c'
* commit '5e83d9aced2fc2b2e1360452794c58aba55d497c':
  h264: fully support cropping.

Conflicts:
	doc/APIchanges
	libavcodec/h264.c
	libavcodec/h264_ps.c
	libavcodec/options_table.h
	libavcodec/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-19 22:46:51 +02:00
Anton Khirnov
5e83d9aced h264: fully support cropping.
Based on a patch by Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes Bug 378.
2013-04-19 09:28:08 +02:00
Michael Niedermayer
a984efd104 Merge commit 'c242bbd8b6939507a1a6fb64101b0553d92d303f'
* commit 'c242bbd8b6939507a1a6fb64101b0553d92d303f':
  Remove unnecessary dsputil.h #includes

Conflicts:
	libavcodec/ffv1.c
	libavcodec/h261dec.c
	libavcodec/h261enc.c
	libavcodec/h264pred.c
	libavcodec/lpc.h
	libavcodec/mjpegdec.c
	libavcodec/rectangle.h
	libavcodec/x86/idct_sse2_xvid.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-26 13:05:10 +01:00
Diego Biurrun
c242bbd8b6 Remove unnecessary dsputil.h #includes 2013-02-26 00:51:34 +01:00
Michael Niedermayer
bdeb61ccc6 h264: check that luma and chroma depth match
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-21 15:02:09 +01:00
Michael Niedermayer
b7fe35c9e5 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: deMpegEncContextize

Conflicts:
	libavcodec/dxva2_h264.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mb_template.c
	libavcodec/h264_parser.c
	libavcodec/h264_ps.c
	libavcodec/h264_refs.c
	libavcodec/h264_sei.c
	libavcodec/svq3.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-16 23:30:09 +01:00
Anton Khirnov
2c54155407 h264: deMpegEncContextize
Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.

The nontrivial parts are:
1) extracting a simplified version of the frame management code from
   mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
   its own more complex system already and those were set only to appease
   the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
   for dxva, the draw_horiz_band() call is moved from
   ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
   because it's now different for h264 and MpegEncContext-based
   decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
   added some very simplistic frame management instead and dropped the
   use of ff_h264_frame_start(). Because of this I also had to move some
   initialization code to svq3.

Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-02-15 16:35:16 +01:00
Michael Niedermayer
25be63005f Merge commit 'b85a5e87af4254b80913fe33591d96361f30832b'
* commit 'b85a5e87af4254b80913fe33591d96361f30832b':
  lavu: Add av_strnstr()
  h264: Allow discarding the cropping information from SPS

Conflicts:
	Changelog
	doc/APIchanges
	libavcodec/avcodec.h
	libavcodec/version.h
	libavutil/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-25 14:36:45 +01:00
Vittorio Giovara
a84fb6e06f h264: Allow discarding the cropping information from SPS
Some 3D systems overload the meaning of the field for other purposes.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-25 07:50:35 +01:00
Michael Niedermayer
40ea343866 Merge commit 'c661cb6672af5ebcb900ec8766b24761bd2ab011'
* commit 'c661cb6672af5ebcb900ec8766b24761bd2ab011':
  cmdutils: pass number of groups to split_commandline().
  mov: handle h263 and flv1 for codec_tag 'H','2','6','3'
  h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles

Conflicts:
	libavcodec/h264_ps.c
	libavformat/isom.c
	libavformat/mov.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-20 12:06:15 +01:00
Victor Lopez
1c8bf3bfed h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles
Fixes bug 396.

CC: libav-stable@libav.org
2012-12-19 13:40:48 +01:00
Janne Grunau
a421bbfe83 h264: fix memleak on error during SPS parsing
Introduced in d7d6efe42b.
2012-12-15 19:16:39 +01:00
Michael Niedermayer
b9d887c225 Merge commit '072be3e8969f24113d599444be4d6a0ed04a6602'
* commit '072be3e8969f24113d599444be4d6a0ed04a6602':
  h264: set parameters from SPS whenever it changes
  asyncts: cosmetics: reindent

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:17:51 +01:00
Janne Grunau
072be3e896 h264: set parameters from SPS whenever it changes
Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.
2012-12-13 21:02:42 +01:00
Michael Niedermayer
7c425e4f2d Merge commit 'd7d6efe42b0d2057e67999b96b9a391f533d2333'
* commit 'd7d6efe42b0d2057e67999b96b9a391f533d2333':
  h264: check sps.log2_max_frame_num for validity
  mov: validate number of DataReferenceBox entries against box size
  mov: compute avg_frame_rate only if duration is known
  flac: change minimum and default of lpc_passes option to 1

Conflicts:
	libavcodec/h264_ps.c
	libavformat/mov.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-07 15:52:43 +01:00
Janne Grunau
d7d6efe42b h264: check sps.log2_max_frame_num for validity
Fixes infinite or long taking loop in frame num gap code in
the fuzzed sample bipbop234.ts_s223302.

CC: libav-stable@libav.org
2012-12-07 11:43:28 +01:00
Michael Niedermayer
039f4ff4c2 h264: 2nd try at getting sm2v working
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-11-11 12:50:03 +01:00
Michael Niedermayer
991f6bf93d h264: Support SMV2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-11-11 02:51:23 +01:00
Hendrik Leppkes
572781b25f h264: fix parsing of old lossless profile (profile_idc == 144)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-10-14 16:04:10 +02:00
Michael Niedermayer
ae0312ab2e h264: print num_reorder_frames in debug output
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-08-04 15:42:44 +02:00
Michael Niedermayer
b5da7d4c1a Merge remote-tracking branch 'qatar/master'
* qatar/master:
  avformat: Drop pointless "format" from container long names
  swscale: bury one more piece of inline asm under HAVE_INLINE_ASM.
  wv: K&R formatting cosmetics
  configure: Add missing descriptions to help output
  h264_ps: declare array of colorspace strings on its own line.
  fate: amix: specify f32 sample format for comparison
  tiny_psnr: support 32-bit float samples
  eamad/eatgq/eatqi: call special EA IDCT directly
  eamad: remove use of MpegEncContext
  mpegvideo: remove unnecessary inclusions of faandct.h
  af_asyncts: avoid overflow in out_size with large delta values
  af_asyncts: add first_pts option

Conflicts:
	configure
	libavcodec/eamad.c
	libavcodec/h264_ps.c
	libavformat/crcenc.c
	libavformat/ffmdec.c
	libavformat/ffmenc.c
	libavformat/framecrcenc.c
	libavformat/md5enc.c
	libavformat/nutdec.c
	libavformat/rawenc.c
	libavformat/yuv4mpeg.c
	tests/tiny_psnr.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-30 23:28:31 +02:00
Ronald S. Bultje
be391fb6df h264_ps: declare array of colorspace strings on its own line. 2012-07-29 14:53:42 -07:00
Michael Niedermayer
1d890ec849 h264/decode_scaling_matrices: fix coding order of 8x8 matrixes.
This fixes decoding of PPCV444I3_Thomson_A.bits

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-10 02:17:53 +02:00
Michael Niedermayer
e162733dcd h264: 12 and 14 bit support
This works with I, P and B frames
CAVLC and CABAC

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-10 02:17:53 +02:00
Michael Niedermayer
acb52d3228 h264: print error on unsupported seperate color planes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-05 19:18:10 +02:00
Michael Niedermayer
4330b8bba2 h264: sps: handle more profiles
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-03 00:40:26 +02:00
Michael Niedermayer
9091ba9dfa h264: add support for constraint_set4_flag / constraint_set5_flag
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-03 00:36:26 +02:00
Michael Niedermayer
0c9283a03f h264: check log2_max_poc_lsb for validity
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-01 20:49:31 +02:00
Michael Niedermayer
27744fe439 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  mpeg12: fixed parsing in some mpeg2 streams
  Add SMPTE240M transfer characteristics flag.
  mpegts: Some additional HDMV types and reg descriptors for mpegts
  motionpixels: Clip YUV values after applying a gradient.
  jpeg: handle progressive in second field of interlaced.
  ituh263dec: Implement enough of Annex O (scalability) to fix a FPE.
  h263: more strictly forbid frame size changes with frame-mt.
  h264: additional protection against unsupported size/bitdepth changes.
  tta: prevents overflows for 32bit integers in header.
  configure: remove malloc_aligned.
  vp8: update frame size changes on thread context switches.
  snowdsp: explicitily state instruction size.
  wmall: fix reconstructing audio with uncoded channels
  WMAL cosmetics: fix indentation
  gitignore: add Win32 library suffixes

Conflicts:
	configure
	libavcodec/h263dec.c
	libavcodec/h264.c
	libavcodec/ituh263dec.c
	libavcodec/mjpegdec.c
	libavcodec/wmalosslessdec.c
	libavcodec/x86/snowdsp_mmx.c
	libavformat/mpegts.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 21:02:32 +02:00
Ronald S. Bultje
732f9fcfe5 h264: additional protection against unsupported size/bitdepth changes.
Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-05-02 10:19:37 -07:00