MPEG - Motion Pictures Expert Group · Francesc Pinyol Margalef

		quincunx_sampling_flag
frame_packing_arrangement_type (*)		0	1
0	checkerboard	D-1
1	column based interleaving	D-2
2	row based interleaving	D-3
3	side-by-side	D-4	D-6
4	top-and-bottom	D-5
5	temporal interleaving of alternating frames	D-7

Free viewpoint television

Free viewpoint television (wp)
Implementations

iview (BBC)
Eye Vision (Carnegie Melon / Super Bowl)

Standards

MPEG-4 part 10 "Multiview coding (MVC)" (3DAV group)

MPEG at LTS
Re Technology Technical Library

ITU-T		(joint)	ISO-IEC (standards)						part	name		layer	free public standard		doc		implementation		application
			JTC	SC	WG	number	title	name					ISO/IEC	ITU	MPEG technologies	Tutorials and seminars MPEG Visions /	programari	maquinari
			1 (Information Technology)	24 (Computer graphics and image processing)			GKS, PHIGS, CGM, VRML, X3D
				29 (Coding of audio, picture, multimedia and hypermedia)	1 (Coding of still pictures)	03 (11544)	JBIG-1
						04 (10918)	JPEG
						10 (14492)	JBIG-2
						12 (14495)	JPEG-LS
						14 (15444)	JPEG 2000
						16 (16485)	MRC
						20 (24800)	JPSearch
					11 (Coding of moving pictures and audio) (MPEG)	11172	MPEG-1	Coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s	1	Systems - Program streams
									2	Video for CD-I
									3	Audio		I
												II
												III (mp3)
									4	Conformance			x
									5	Technical report - Software simulation			x
						13818	MPEG-2	Generic coding of moving pictures and associated audio information	1	Systems (TS, PES) (amdt. 6 "Transport of scalable video over ITU-T Rec H.220/ ISO/IEC 13818-1": AVC_video_descriptor)			x				ts-muxer
H.262 (doc)	VCEG								2	Video (profiles/levels)				07/95			ffmpeg
									3	Audio
									4	Conformance			x
									5	Technical report			x
									6	DSM-CC							programari / software
									7	AAC
									8
									9	RTI
									10	Conformance Extensions for DSMCC
									11	IPMP on MPEG2
						14496 (c)	MPEG-4	Coding of audio-visual objects	1	Sytems						MPEG-4 Sytems FAQ (Chiariglione)
									2	Visual (profiles/levels)	SP					MPEG-4 Video FAQ (Chiariglione)
H.263 (doc)									2	Visual (profiles/levels)	ASP			01/05					Flash Video DivX >=4.0 Xvid
									3	Audio (AAC)						MPEG-4 Audio FAQ (Chiariglione)	faac, faad (RareWares) libfdk
									4	Conformance			x
									5	Reference software			x
									6	DMIF						MPEG-4 DMIF FAQ (Chiariglione)
									7	Optimized software			x
									8	MPEG-4 on IP
									9	Reference hardware
H.264 (1,2) (doc)	VCEG								10	AVC (profiles/levels)				11/07 03/10 04/17 corrigendum		JVT (ITU-T) (ftp) JVT-G050r1 JVT-T202 JVT-U202 (oct 2006) JVT-X201 (jun 2007) Overview Wikipedia MPEG-4 & co MP4 FAQ White papers (Envivio): MPEG4 over RTP/TS MPEG-4 RTP transport	AVC programari / software Comparison with previous codecs (ITU H.264 flyer) comparison (doom9) MPEG-4 AVC/H.264 video codecs list (doom9) Wikipedia (products and implementations) coreAVC TML with KorFMD DGAVCDec	Wikipedia ATI Nvidia PureVideo (*) iPod MPEG-4 AVC/H.264 video codecs list (doom9)	Wikipedia HD DVD (HiP) Blue Ray (HiP)
H.264 (1,2) (doc)	VCEG	JVT								SVC (Annex G)						JVT JVT-T202 Annex G Scalable Extension of H.264/AVC (HHI) EBU Technical 2008 2008 Q2 Test sequences Transport MPEG-2 TS RTP streaming HTTP streaming NAL unit type Multimedia Communications with Scalable Video Coding Overview of the real-time SVC en/decoder (pdf) Scalable Video Coding: The Future of Video Delivery? (Streamingmedia) Other docs and publications	SVC software	Texas Instruments ms320c6474 Freescale-DSP ST Microelectronics Annonce SPEAr1340	GE Security Global IP Solutions Vidyo (Google gmail) Vidiyocast Radvision Projects: Scalim@ages SVC4QOE Scalnet OCEAN
										MVC (Annex H)					1-pager	Advances on Coding and Transmission of Scalable Video and Multiview Video 1-pager Multimedia Communications with Scalable Video Coding Test sequences	Fraunhofer HHI Main Concept Net Blender Nokia MVC software		DIOMEDES Nem Summit MUSCADE Nem Summit
									11	Scene Description and Application Engine (BIFS MPEG-J)
									12	ISO Base Media File Format			x			Formats de fichiers MPEG			(MP4, 3GP, MJP2, MP21, AVC)
										AMD 1: General Improvements including hint tracks, metadata support and sample groups
										AMD 3: DASH (sidx, ssix), RTP reception hint
									13	IPMP Extensions
									14	MP4 File format						(extension of part 12)	GPAC bento4
									15	Advanced Video Coding File Format (amd. 2008: includes SVC) amdt 2: avc3						(extension of part 12)
									16	AFX (3D)						MPEG-4 3D Graphics FAQ (Chiariglione)
									17	Streaming Text Format
									18	Font Compression and Streaming
									19	Synthesized Streams
									20	LASeR			x
									20	SAF			x
									21	MPEG-J Graphical Framework eXtension (GFX) (3D)						MPEG-4 3D Graphics FAQ (Chiariglione)
									22	Open Font Format
									23	Symbolic Music Representation (SMR)
									24	Audio and systems interaction
									25	3D Graphics Compression Model						MPEG-4 3D Graphics FAQ (Chiariglione)
									26	Audio conformance
									27	3D Graphics conformance
									28	Composite Font
									29	Web Video Coding
									30	Timed Text and Other Visual Overlays in ISO Base Media File Format
						15938 (c)	MPEG-7	Multimedia content description interface	1	Systems						MPEG-7 FAQ (Chiariglione)	programari / software
									2	DDL
			3						Visual
			4	Audio
			5	Multimedia description schemes
			6	Reference software							x
			7	Conformance testing							x
			8	Extraction and use of description
			9	MPEG-7 Profiles
			10	Schema Definition							x
			11	Profile schemas						x
			21000 (c)	MPEG-21	Multimedia framework	1	Technical Report			x			MPEG-21 FAQ (chiariglione) Klagenfurt University	programari / software
						2	DID			x
						3	DII
						4	IPMP
						5	Rights Expression Language
						6	Rights Data Dictionary
						7	DIA
						8	Reference Software			x
						9	File Format									MP21Box
						10	DIP
						11	Evaluation Methods for Persistent Association
						12	Test Bed for MPEG-21 Resource Delivery
						13	SVC
						14	Conformance Testing
						15	Event Reporting
						16	Binary Format
						17	Fragment Identification of MPEG Resources
						18	DIS
			23000	MPEG-A	Multimedia Application Format - MAF	1	Purpose of MAFs
						2	Music player AF
						3	Photo player AF
						4	Musical slide show AF
						5	Media streaming AF
						6	Professional archiver AF
						7	Open access AF
						8	Portable video AF
						9	Digital Multimedia Broadcasting - DMB AF
						10	Video surveillance AF
						11	Stereoscopic video AF
			23001	MPEG-B	MPEG systems technologies	1	Binary MPEG format for XML
						2	Fragment Request Units
						3	XML IPMP messages
						4	Codec configuration representation
						5	Bitstream Syntax Description Language (BSDL)
						6	Dynamic Adaptive Streaming over HTTP (DASH) (moved to 23009)			DIS
						7	Common encryption in ISO base media file format						Xifratge / Encryption
						8	Coding-independent code-points
						9	Common Encryption for MPEG-2 Transport Streams
			23002	MPEG-C	MPEG video technologies	1	Accuracy requirements for implementation of integer-output 8x8 inverse discrete cosine transform
						2	Fixed-point 8x8 inverse discrete cosine transform and discrete cosine transform
						3	Representation of auxiliary video streams and supplemental information (2D+Depth)
						4	Video tool library
			23003	MPEG-D	MPEG audio technologies	1	MPEG Surround
						2	Spatial Audio Object Coding (SAOC)
						3	Unified Speech and Audio Coding (USAC)
			23004	MPEG-E	Multimedia Middleware	1	Architecture
						2	Multimedia application programming interface (API)
						3	Component model
						4	Resource and quality management
						5	Component download
						6	Fault management
						7	System integrity management
						8	Reference software
			23008	MPEG-H	High Efficiency Coding and Media Delivery in Heterogeneous Environments	1	MMT MPEG Media Transport						Kyoto workshop 2010 London workshop 2009
H.265 / H.NGVC / ...		JCT				2	HEVC (High Efficiency Video Coding)						H265.net DVB Scence #36 (codec timeline) WTF is... H.265 aka HEVC? (3D CineCast) High Efficiency Video Coding (wp) Comparing HEVC and H.264 quality: see for yourself Sequences (BBC)	svn_HEVCSoftware encode: x265 (compilation) decode: OpenHEVC GPAC with OpenHEVC package (mp4) MPEG-HEVC Support in GPAC
						3	Audio
				(MPEG-J)
			23006	MPEG-M	MPEG extensible middleware (MXM) Multimedia Service Platform Technologies	1	MXM architecture and technologies						MPEG eXtensible Middleware Vision
						2	MXM API
						3	MXM Reference software
						4	MXM Protocols
H.721	Q.13 / SG16						ITU-T: The IPTV Terminal (Basic Model)				03/09
						?	Advanced IPTV Terminal						AIT Call for Proposals AIT Context & Objectives AIT Use cases AIT Requirements
							(Mobile IPTV Terminal)
			23007	MPEG-U	Rich media user interfaces (widgets)	1	Widgets						Presentation of MPEG-U (C. Concolato)	MPEG-U with GPAC (GPAC)
						2	Advanced User Interaction Interface
						3	Conformance and reference software
			23005	MPEG-V	Information exchange with virtual worlds (Metaverse) Media Context and Control	1	Architecture						Interfacing with virtual worlds (Media Context and control) (NEM)
						2	Control information
						3	Sensory information
						4	Virtual world object characteristics
						5	Data formats for interaction devices
						6	Common types and tools
						7	Conformance and reference software

			23009	DASH	Dynamic adaptive streaming over HTTP (formerly 23001-6)	1	Media presentation description and segment formats			c057623_ISO_IEC_23009-1_2012.zip 23009-1:2012 electronic attachments schema files (DASH-MPD.xsd) w12166			HTTP streaming HTTP streaming technologies MPD DASH Promoters Group	HTTP streaming software HTTP streaming with GPAC GPAC: `MP4Box -dash`
						2	Conformance and reference software for DASH			w12167
						3	Implementation Guidelines of DASH			w12168
			12	13522	MHEG		5	MHEG-5
			JTC	SC	WG	number	title	name					ISO/IEC	ITU			programari	maquinari
ITU-T		(joint)	ISO-IEC (standards)						part	name		layer	free public standard			doc	implementation		application

MPEG technologies (summary, 1-pager, white paper, presentation)

			standards
1. Media Coding	2D video coding	MPEG-1 video	MPEG-1/2
		MPEG-2 Video	MPEG-2/2
		MPEG-4 Visual (rect)	MPEG-4/2
		Shape coding
		Advanced video coding	MPEG-4/10
		Scalable video coding	MPEG-4/10
		Multi-view coding	MPEG-4/10
		High performance video coding
	Decoder representation	Reconfigurable video coding
	Decoder representation	Coding tool repository
	3D video coding	Auxiliary video data	MPEG-C/3 ?
	3D video coding	3D video coding
	Audio coding	...
	2D graphic coding	Texture coding
	2D graphic coding	2D mesh coding
	3D graphic coding	Face and body animation
		3D mesh coding
		AFX
	Synthetic audio coding	...
	Text coding	...
	Font coding	Font compression and streaming
	Font coding	Open Font Format
	Music coding	...
	Media Context and control	Control information
		Sensory information
		Virtual object characteristics
	Media value chains
2. Composition coding	Composition coding	Binary format for scenes
		Audio BIFS
		BIFS for digital audio
		Lightweigth scene representation	MPEG-4/20
		Presentation and modification od structured information
3. Description coding	Description technologies	Description definition language
	Description technologies	MPEG-7 schemas
	Video description	Low level descriptions
		High level descriptions
		Overview
		Visual description tools
		Image and video signature
	Audio description	Low level descriptions
	Audio description	High level descriptions
	Multimedia description	Multimedia description schemes
4. Systems support	Multiplexing and synchronization	MPEG-1
		MPEG-2
		MPEG-4
	Signaling	DSM-CC user to user
		DSM-CC user to network
		DMIF
5. IPMP	General	MPEG technologies for DRM
	Identification technologies	MPEG-2 copyright identifier
		Object content information
		Digital Item identification	MPEG-21/3
	Rights expression technologies	...
	Persitent association technologies	...
	Access technologies
6. Digital Item	Digital item technologies	Digital item declaration	MPEG-21/2
		Digital item processing
		C++ bindings
		Session mobility
		Event reporting
		Schema files
		Digital item representation
	Resources in Digital Items	Digital item adaptation
	Resources in Digital Items	Fragment identification for MPEG resources
7. Transport and file format	Transport of media streams	Program stream	MPEG-1/1
		Transport stream	MPEG-2/1
		M4Mux
		MPEG Media Transport
	Media file formats	ISO base media file formats	MPEG-4/12
		MPEG-4 file format	MPEG-4/14
		AVC file format	MPEG-4/15
		SVC file format	MPEG-4/15
		MVC file format
		Digital item file format	MPEG-21/9
	Transport of digital items	Digital item streaming
8. User interaction	User interaction	Widgets	MPEG-U
8. User interaction	User interaction	Advanced user interaction
9. Multimedia architecture	Terminal architecture	...
	Application programming interfaces	MPEG-J
		MPEG-J GFX	MPEG-4/21
		M3W multimedia API
		MXM API	MPEG-M
	Terminals	...
10. Application formats	Application formats	Music player	MPEG-A/2
		Photo player	MPEG-A/3
		Musical slide show	MPEG-A/4
		Media streaming	MPEG-A/5
		Professional archiving	MPEG-A/6
		Open access	MPEG-A/7
		Portable video	MPEG-A/8
		Digital multimedia broadcasting	MPEG-A/9
		Video surveillance	MPEG-A/10
		Stereoscopic video	MPEG-A/11
		Interactive music
11. Generic media technologies	XML technologies	Binary MPEG format for XML	MPEG-B/1
	Signal processing technologies	Generic inverse DCT specification	MPEG-C/1
	Signal processing technologies	Fixed point implementation of DCT/IDCT	MPEG-C/2
	Bitstream technologies	...
12. Protocols	Protocols	MXM protocols	MPEG-M
13. Reference implementations	Reference software
13. Reference implementations	Reference hardware description
14. Conformance	MPEG-1		MPEG-1/4
	MPEG-2		MPEG-2/4
	MPEG-4		MPEG-4/4
	MPEG-7		MPEG-7/7
	MPEG-21		MPEG-21/14
	MPEG-A
	MPEG-B
	MPEG-C
	MPEG-D
	MPEG-E
	MPEG-M
	MPEG-U
	MPEG-V
15. Maintenance	MPEG-1
	MPEG-2
	MPEG-4
	MPEG-7
	MPEG-21
	MPEG-A
	MPEG-B
	MPEG-C
	MPEG-D
	MPEG-E
	MPEG-M
	MPEG-U
	MPEG-V

Estàndards / Standards

ISO/IEC freely available standards

WD	Working Draft
CD	Comitee Draft
FCD	Final Comitee Draft
FDIS	Final Draft International Standard
IS	International Standard
PDAM	Proposed Amendment
FPDAM	Final Proposed Draft Amendment
FDAM	Final Draft Amendment
AMD	Amendment
DCOR	Draft Corrigenda
COR	Corrigenda
PDTR	Proposed Draft Technical Report
DTR	Draft Technical Report
TR	Technical Report

		3m		4m		2m
Specification	CD	->	FCD	->	FDIS	->	IS
Amendment	PDAM	->	FPDAM	->	FDAM	->	AMD
Corrigenda	-	->	DCOR	->	-	->	COR
Technical report	PDTR	->	-	->	DTR	->	TR

m: input documents; w:output documents

Maquinari
- MPEG hardware acceleration
Programari / Software
- Utilitats (analitzadors)
Taules SI
- PAT
- PMT
- NIT

03MPEG-2 ( ISO/IEC 13818 )#

MPEG-2

MPEG-2: Overview of the systems layer (pdf) (BBC Research)
MPEG headers

Appendix A: MPEG header tables (Investigation into digital video streams, Peter Daniel)

part 1: Systems

programme #1	ES (0xa6)	PES (PES packets)	SP-TS (single programme TS) (transport packet: 188 bytes) PCR#1 (every 0.1s)	TS
	ES (0x65)	PES
programme #2	ES (0xa7)	PES	SP-TS PCR#2 (every 0.1s)
	ES (0x66)	PES

For a given ES:

access units (compressed data)			AU#1		AU#1		AU#1		AU#1			AU#1			AU#1		AU#2			AU#2	...
PES packets		H	P		P		P		P		H	P			P		P			P	...
transport packets (H+P: 188 bytes)	H	P		H	P	H	P	H	P	H	P		H	A	P	H	P	H	A	P	...

H: header (header of first PES packet for an access unit can carry its PTS)
P: payload
A: adaptation field (carries PCR)
Transport Stream

MPEG Transport Stream Management Workshop (pdf)

Program-specific information (PSI) (wp)

MPEG-2 and DVB poster (JDSU)
DVB-SI

2.4.3.6 PES (wp)

table 2.22 2-18: stream_id
table 2.34 2-29: stream_type
PES packet header:

0	0	0	0	0	0	0	0	PES-packet start code prefix	start code
0	0	0	0	0	0	0	0
0	0	0	0	0	0	0	1
x	x	x	x	x	x	x	x	stream_id
x	x	x	x	x	x	x	x	PES-packet length
x	x	x	x	x	x	x	x	PES-packet length
1	0	x	x	x	x	x	x	flags 1
P	D	x	x	x	x	x	x	flags 2
x	x	x	x	x	x	x	x	PES header data length
								presentation timestamp PTS (if present)
								decoding timestamp DTS (if present)
								further optional fields

Transport packet

Header:

0
1
0
0
0
1
1
1
sync byte (=0x47)

TEI
PUSI
TP
PID

PID

TSC
AFC
CC

Clock references and timestamps

0	1	0	0	0	1	1	1	sync byte (=0x47)
TEI	PUSI	TP	PID
PID
TSC	AFC	CC

	clock	sample (27MHz, 42-bit)	carried in (every 0.1s)
Program Stream	System clock	System clock reference (SCR)
Transport Stream	Programme clock	Programme clock reference (PCR)	adaptation fields of transport packets

Timestamps (90kHz, 33-bit) (carried in PES header, every 0.7s)

Presentation Time Stamp (PTS)
Decoding Time Stamp (DTS)

part 2: video

video sequence

sequence header: aspect ratio (AR), [AFD]
[additional sequence header]
group of pictures (GOP)

group_of_pictures_header: [AFD]
picture: I, B, P

picture header: [AFD]
slice

macroblock (16x16 pixels): 4 Y blocks + 1 Cr block + 1 Cb block

block (8x8 pixels)

end of sequence

A beginners guide for MPEG-2 standard
MPEG-2 encoder chip (Sony)

max br (bps)	max samples			maximum video bit rate (bps)
80M	1920x1080@30 1920x1152@25 1280x720@60	levels	high		80M	300M			100M
60M	1440x1080@30 1440x1152@25 1280x720@30		high-1440		60M	80M		60M	80M
15M	720x480@30 720x576@25 352x288@15 320x240@24		main	15M 384k	15M	50M	15M		20M
4M	352x240@30 352x288@25 352x288@30 176x144@15		low	96k	4M		4M
				simple	main	422	SNR	spatial	high
				profiles
frames			I	x	x	x	x	x	x
			P	x	x	x	x	x	x
			B		x	x	x	x	x
YCbCr			4:2:0	x	x		x	x	x
YCbCr			4:2:2			x			x
scalable			SNR				x	x	x
scalable			spatial					x	x

ISO_639_language_descriptor (used in PMT) (dvbsnoop source code: dvbstrAudio_TYPE) (ISO/IEC 13818-1 2.6.19)

lang: ISO 639-2 (Alpha-3) (added by DVB)

audio_type		used for
0x00	undefined	main audio
0x01	clean effects	no language
0x02	hearing impaired	Clean Audio
0x03	visual impaired commentary
0x04 - 0xff	reserved

stream_id (used by PES) (ISO/IEC 13818-1, 2.4.3.6, table 2-18) (completed by DVB: TS 101 154, 4.1.6)

binary	hex		TS 101 154, 4.1.6
1011 1100	0xbc	program_stream_map
1011 1101	0xbd	private_stream_1	AC-3, Enhanced AC-3, DTS
1011 1110	0xbe	padding_stream
1011 1111	0xbf	private_stream_2
110x xxxx	0xc0 - 0xdf	ISO/IEC 13818-3 or ISO/IEC 11172-3 or ISO/IEC 13818-7 or ISO/IEC 14496-3 audio stream number x xxxx	MPEG-4 AAC, MPEG-4 HE AAC, MPEG-4 HE AAC v2 a
1110 xxxx	0xe0 - 0xef	ITU-T Rec. H.262 \| ISO/IEC 13818-2 or ISO/IEC 11172-2 or ISO/IEC 14496-2 video stream number xxxx	0xea: VC-1
1111 0000	0xf0	ECM_stream
1111 0001	0xf1	EMM_stream
1111 0010	0xf2	ITU-T Rec. H.222.0 \| ISO/IEC 13818-1 Annex A or ISO/IEC 13818- 6_DSMCC_stream
1111 0011	0xf3	ISO/IEC_13522_stream
1111 0100	0xf4	ITU-T Rec. H.222.1 type A
1111 0101	0xf5	ITU-T Rec. H.222.1 type B
1111 0110	0xf6	ITU-T Rec. H.222.1 type C
1111 0111	0xf7	ITU-T Rec. H.222.1 type D
1111 1000	0xf8	ITU-T Rec. H.222.1 type E
1111 1001	0xf9	ancillary_stream
1111 1010	0xfa	ISO/IEC14496-1_SL-packetized_stream
1111 1011	0xfb	ISO/IEC14496-1_FlexMux_stream
1111 1100 ... 1111 1110		reserved data stream
1111 1111	0xff	program_stream_directory

stream_type (used in PMT) (used by PES) (ISO/IEC 13818-1, 2.4.3.6, table 2-29) (dvbsnoop source code: dvbstrStream_TYPE)

		TS 101 154, 4.1.6
0x00	ITU-T \| ISO/IEC Reserved
0x01	ISO/IEC 11172-2 Video (MPEG-1 video)
0x02	ITU-T Rec. H.262 \| ISO/IEC 13818-2 Video or ISO/IEC 11172-2 constrained parameter video stream
0x03	ISO/IEC 11172-3 Audio (MPEG-1 audio)
0x04	ISO/IEC 13818-3 Audio (MPEG-2 audio)
0x05	ITU-T Rec. H.222.0 \| ISO/IEC 13818-1 private_sections
0x06	ITU-T Rec. H.222.0 \| ISO/IEC 13818-1 PES packets containing private data	AC-3, Enhanced AC-3, DTS
0x07	ISO/IEC 13522 MHEG
0x08	ITU-T Rec. H.222.0 \| ISO/IEC 13818-1 Annex A DSM CC
0x09	ITU-T Rec. H.222.1
0x0a	ISO/IEC 13818-6 type A
0x0b	ISO/IEC 13818-6 type B
0x0c	ISO/IEC 13818-6 type C
0x0d	ISO/IEC 13818-6 type D
0x0e	ISO/IEC 13818-1 auxiliary
0x0f	ISO/IEC 13818-7 Audio with ADTS transport sytax
0x10	ISO/IEC 14496 -2 Visual
0x11	ISO/IEC 14496-3 Audio with LATM transport syntax as def. in ISO/IEC 14496-3/AMD1	MPEG-4 AAC MPEG-4 HE AAC MPEG-4 HE AAC v2 a
0x12	ISO/IEC 14496-1 SL-packetized stream or FlexMux stream carried in PES packets
0x13	ISO/IEC 14496-1 SL-packetized stream or FlexMux stream carried in ISO/IEC 14496 sections
0x14	ISO/IEC 13818-6 DSM-CC synchronized download protocol
0x15	Metadata carried in PES packets using the Metadata Access Unit Wrapper
0x16	Metadata carried in metadata_sections
0x17	Metadata carried in ISO/IEC 13818-6 (DSM-CC) Data Carousel
0x18	Metadata carried in ISO/IEC 13818-6 (DSM-CC) Object Carousel
0x19	Metadata carried in ISO/IEC 13818-6 Synchronized Download Protocol using the Metadata Access Unit Wrapper
0x1a	IPMP stream (defined in ISO/IEC 13818-11, MPEG-2 IPMP)
0x1b	AVC video stream as defined in ITU-T Rec. H.264 \| ISO/IEC 14496-10 Video
0x1c-0x7e	ITU-T Rec. H.222.0 \| ISO/IEC 13818-1 Reserved
0xd1	Dirac video
0x7f	IPMP stream
0x08-0xff	User Private

04MPEG-4 ( ISO/IEC 14496 )#

MPEG-4

MPEG-4 demo applet
MPEG-4 (IEEE Spectrum article)
MPEG-4 Tutorial (GPAC)
MPEG-4 Part 3: AAC (Advanced Audio Coding)

Codec parameter (MPEG-4 audio):

AAC LC
mp4a.40.2

HE-AAC
mp4a.40.5

HE-AAC v2
mp4a.40.29

MPEG-4 Part 10: AVC-SVC-MVC / H.264

AAC LC	mp4a.40.2
HE-AAC	mp4a.40.5
HE-AAC v2	mp4a.40.29

ITU-T documents:

H.264 : Advanced video coding for generic audiovisual services (04/2017)
Final Draft of ITU-T Rec. H.264 & 14496-10 AVC (including SVC Amendment) (JVT-X201) (2007-06 Geneva)
ITU-T JVT-T202 Recommendation H264 (Joint draft 7 of SVC amendment) (2006-07 Klagenfurt)

AVC / H.264 (Wikipedia)

"Overview of the H.264/AVC Video Coding Standard" (Thomas Wiegand)
Programari / Software

Info

vegeu també / see also: SVC info

Anàlisi / Analysis

vegeu també / see also: SVC Anàlisi / Analysis
NAL-SIM

Encoder

x264

compilation
command line usage:

libav
ffmpeg
x264

help

x264 --help
x264 --longhelp
x264 --fullhelp

x264 --profile baseline -o output.mp4 input.mp4
x264 --profile baseline --sps-id 2 -o output.mp4 input.mp4

presets

x264/common/common.c, x264_param_apply_preset
ultrafast

superfast

veryfast

faster

fast

medium

slow

slower

veryslow

placebo

options


ultrafast
superfast
veryfast
faster
fast
medium
slow
slower
veryslow
placebo

X264 Settings (MeWiki)
x264 FFmpeg Options Guide (mapping between x264 and ffmpeg command line options)

x264 / avconv output (x264/common/common.c, x264_param2string)			AVCodecContext (libavcodec/avcodec.h)		avconv
	value			av_opt_set / av_opt_set_int	command line option	preset file (accessed with -pre:v)
cabac	%d	b_cabac				`x264-params=cabac=1`
ref	%d	i_frame_reference
deblock	%d:%d:%d	b_deblocking_filter i_deblocking_filter_alphac0 i_deblocking_filter_beta
analyse	%#x:%#x	analyse.intra analyse.inter
me	%s	analyse.i_me_method	me_method
subme	%d	analyse.i_subpel_refine
psy					-psy integer
psy_rd					-py-rd string
mixed_ref	%d	analyse.b_mixed_references
me_range
chroma_me
trellis	%d	analyse.i_trellis
8x8dct	%d	analyse.b_transform_8x8			-8x8dct integer
cqm
deadzone
fast_pskip
chroma_qp_offset
threads
sliced_threads
slices
slice_max_size
slice_max_mbs
nr
decimate
interlaced
bluray_compat
constrained_intra
bframes
b_pyramid					-b-pyramid integer
weightp
keyint				"key-int", 8
keyint_min
rc_lookahead					-rc-lookahead integer
rc
mbtree
crf					-crf float
bitrate
qcomp
qpmin
cplxblur
vbv_maxrate
crf_max					-crf_max float
qp					-qp integer
nal_hdr
crop_rect
frame-packing
ip_ratio
pb_ratio
aq	%d	rc.i_aq_mode			-aq-mode integer -aq-strength float
zones
				"preset", "ultrafast"	-preset ultrafast
					-pre:v toto (file: libx264-toto.avpreset)
				"tune", "zerolatency"	-tune
					-profile
	value			av_opt_set / av_opt_set_int	command line option	pre file
x264 / avconv output (x264/common/common.c, x264_param2string)			AVCodecContext (libavcodec/avcodec.h)		avconv

baixa latència / low latency

SVC

Documents

"Overview of the Scalable Video Coding Extension of the H.264/AVC Standard" (Heiko Schwarz)
Scalable Video Coding (pdf) – Jens-Rainer Ohm, Shenzhen Workshop October 2007
MPEG verification test for SVC
DIOMEDES D4.1 Specification of the encoder/decoder software for the MD-MVD and the spatial audio codec
Scalable Video Coding over RTP and MPEG-2 Transport Stream in Broadcast and IPTV Channels
Dr.-Ing. Thomas Schierl Publications

Priority-based Media Delivery using SVC with RTP and HTTP Streaming
Scalable Video Coding over RTP and MPEG-2 Transport Stream in Broadcast and IPTV Channels (pdf)

MPEG-21

An interoperable multimedia delivery framework for scalable video coding based on MPEG-21 Digital Item Adaptation (modified VLC) (P2P-Next) (ICME 2008)
An Interoperable Streaming Framework for Scalable Video Coding Based on MPEG-21 (P2P-Next)

OpenSVC conference papers

Open SVC decoder: a flexible SVC library

Layers

DQId = (dependency_id<<4) + quality_id (dependency_id * 16 + quality_id)
temporal_id

Programari / Software

	coder		package		segmentation		decode
		info	MP4	info	HTTP streaming		HTTP streaming		SVC	player
					HLS	MPEG-DASH	HLS	MPEG-DASH
JSVM (reference)	encoder: `toto.264`	`BitStreamExtractorStatic toto.264`
MPEG4IP			MP4Creator (modified by Scalnet)
OpenSVC Decoder			MP4Box.exe (modified by OSVCD)						opensvcdecoder	`mplayer -fps 25` `video_5.264` `[-setlayer DQId -settemporalid temporal_id`] `Libs/OpenSVCDecoder-h264 toto_svc.264 [-layer DQId -tempID temporal_id]` TCPMP
GPAC with OpenSVC			`MP4Box -add toto.264 toto.mp4`	`MP4Box -info toto.mp4` `MP4Box -diso toto.mp4`		MP4Box -dash		x	modules/opensvc_dec	`MP4Client toto.mp4`
VLC 2.0								DASH plugin (uni-kla)	SVC plugin (P2PNext)	vlc

SVC Reference software (coder/encoder) (JSVM) (HHI)

coder

Info

JSVM:

BitStreamExtractorStatic toto.264

Anàlisi / Analysis

package

MP4

"MP4 File Creator for SVC Adaptive Video Streaming" (modified MP4Creator) (Institute for Infocomm Research) IEEE International Conference on Internet Technology and Applications (iTAP 2010); 22-Apr-10
OpenSVCDecoder:

modified MP4Box (.exe) (GPAC)

Scalnet:

modified mp4creator (MPEG4IP)

MPEG-2 TS

...

decoder

OpenSVC decoder (sf)

players:

ffmpeg-devel

GoC 2008 and H264 SVC decoder

svcfilters (DirectShow)
modified VLC (P2P-Next)

Transport

MPEG-2 TS

13818-1 amdt.6

RTP streaming

RFC draft
Modified mp4creator, modified Darwin Streaming Server (Jordi Ortiz - U. Murcia - GSI) (Scalnet)
Scalable Video Transmission in Home PLC networks (Scalnet)

HTTP Streaming

23009-1 MPEG-DASH part 1

G.5 Example for SVC alternative streams

Priority-based Media Delivery using SVC with RTP and HTTP Streaming (pdf) (T. Schierl)
iDASH: improved Dynamic Adaptive Streaming over HTTP using Scalable Video Coding (Y. Sánchez, T. Schierl) (OCEAN / Cordis)
Scalable Video Coding based DASH for efficient usage of network resources (Y. Sánchez) (W3C Web TV)

search svc on lists

MVC

BD ROM AV white papers
Multiview Video Coding (pdf)– Jens-Rainer Ohm, Shenzhen Workshop October 2007

PAFF/MBAFF
bitrate CPU decoding example

PAFF high low Sky, TVC

MBAFF low high BBC
Interlaced encoding in x264

	bitrate	CPU decoding	example
PAFF	high	low	Sky, TVC
MBAFF	low	high	BBC

Structure (*) (number indicates nal_unit_type) in NAL units:

sequence#1

[7.SPS]

AccessUnit#1 (IDR picture)

9.AU delimiter	6.SEI	[8.PPS]	5.slice#1	5.slice#2	...

AU#2 (non-IDR picture)

9.AU delimiter	6.SEI	[8.PPS]	1.slice#1	1.slice#2	...

AU#3 (non-IDR picture)

9.AU delimiter	6.SEI	[8.PPS]	1.slice#1	1.slice#2	...

...

10.End of sequence

sequence#2

...

11.End of stream

VCL (Video Coding Layer) NAL unit; NON-VCL NAL unit
Picture: frame or field
AU: Access Unit (coded picture)
PU: Presentation Unit (uncompressed picture)
SPS: Sequence Parameter Set
PPS: Picture Parameter Set
SEI: Supplemental Enhancement Information

NAL bitstream (*)

	out-bond extra data	usage		programari / software
	(same as SPS/PPS?)		NALU delimiter	ffmpeg	gstreamer x264enc
Packet-Transport System (?AVCC format)	needed	streaming (RTP)	(?) implicitly defined by nal_size field		`byte-stream=false` (default)
Byte-Stream Format (Annex B format: JVT-X201/Annex B)	not needed	file ITU-T H.222.0 ISO/IEC 13818-1 systems (MPEG-2 TS) Rec. ITU-T H.320	0x00 0x00 0x00 0x01	`-bsf h264_mp4toannexb` `-bsf h264_mp4toannexb -f mpegts`	`byte-stream=true`

Annex B format

0x00 0x00 0x00 0x01

AVCC format

each NAL is preceded by a nal_size field (size of the field is usually: 4 bytes)

NAL header from ITU-T Recommendation H264 (AVC, SVC-AnnexG) JVT-T202-AnnexG (section 7.3) (RFC draft RFC draft-svc (section 3.3) for SVC):

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI nal_unit_type RR PRID TL DID QL R B U D G L O

F: forbidden_zero_bit
NRI: nal_ref_idc (ffmpeg debug: NAL=...):

00 (0): discardable
>00: required decoding
11 (3): picture key
Breif Description of nal_ref_idc Value in H.246 NALU
Introduction to H.264: (1) NAL Unit

RR: reserved_zero_two_bits
PRID: simple_priority_id
TL: temporal_level
DID: dependency_id
QL: quality_level
R: reserved_zero_bit
B: layer_base_flag
U: use_base_prediction_flag
D: discardable_flag
G: fragmented_flag
L: last_fragmented_flag
O: fragment_order

NAL unit types:

NAL unit type		name			NRI	RBSP syntax (Raw Byte Sequence Payload)	videolan x264 name videolan vlc name	GPAC (constants.h)	defined
Undefined	0	Unspecified					NAL_UNKNOWN		ITU-T JVT-G050r1 Recommendation H264: Table 7.5 ITU-T JVT-T202 Recommendation H264 (Joint draft 7 of SVC amendment): Section 7.4.1 / Table 7.5 ITU-T JVT-X201: Section 7.4.1NAL unit semantics / Table 7.5 draft-ietf-avt-rtp-svc (section 1.14.3: Table 1. NAL unit types introduced in SVC and this memo)
Single NAL unit	1	Coded slice of a non-IDR picture				slice_layer_without_partitioning_rbsp	NAL_SLICE	GF_AVC_NALU_NON_IDR_SLICE
	2	Coded slice data partition A				slice_data_partition_a_layer_rbsp	NAL_SLICE_DPA	GF_AVC_NALU_DP_A_SLICE
	3	Coded slice data partition B				slice_data_partition_b_layer_rbsp	NAL_SLICE_DPB	GF_AVC_NALU_DP_B_SLICE
	4	Coded slice data partition C				slice_data_partition_c_layer_rbsp	NAL_SLICE_DPC	GF_AVC_NALU_DP_C_SLICE
	5	Coded slice of an IDR picture			3	slice_layer_without_partitioning_rbsp	NAL_SLICE_IDR	GF_AVC_NALU_IDR_SLICE
	6	Supplemental enhancement information (SEI)				sei_rbsp	NAL_SEI	GF_AVC_NALU_SEI
	7	Sequence parameter set (SPS)				seq_parameter_set_rbsp	NAL_SPS	GF_AVC_NALU_SEQ_PARAM
	8	Picture parameter set (PPS)				pic_parameter_set_rbsp	NAL_PPS	GF_AVC_NALU_PIC_PARAM
	9	Access unit delimiter (AUD)				access_unit_delimiter_rbsp	NAL_AU_DELIMITER	GF_AVC_NALU_ACCESS_UNIT
	10	End of sequence				end_of_seq_rbsp		GF_AVC_NALU_END_OF_SEQ
	11	End of stream				end_of_stream_rbsp		GF_AVC_NALU_END_OF_STREAM
	12	Filler data				filler_data_rbsp		GF_AVC_NALU_FILLER_DATA
	13	Sequence parameter set extension				seq_parameter_set_extension_rbsp		GF_AVC_NALU_SEQ_PARAM_EXT
	14	Prefix NAL unit in scalable extension	(reserved)			prefix_nal_unit_rbsp
	15	Subset sequence parameter set				subset_seq_parameter_set_rbsp
	16	(reserved)
	17
	18
	19	Coded slice of an auxiliary coded picture without partitioning				slice_layer_without_partitioning_rbsp		GF_AVC_NALU_SLICE_AUX
	20	Coded slice in scalable extension				slice_layer_in_scalable_extension_rbsp		GF_AVC_NALU_SVC_SLICE
	21	(reserved)
	22
	23
Single-time aggregation packet	24	aggregation	STAP	A				GF_AVC_NALU_VDRD	IETF RFC 3984: RTP Payload for H264 Video: section 5.2 `packetization-mode` in SDP
Single-time aggregation packet	25		STAP	B
Multi-time aggregation packet	26		MTAP	16
Multi-time aggregation packet	27		MTAP	24
Fragmentation unit	28	fragmentation	FU-A
Fragmentation unit	29	fragmentation	FU-B
Undefined	30	Payload Content Scalability Information	PACSI						IETF RFC draft: RTP Payload Format for SVC Video: section 6.10 draft-ietf-avt-rtp-svc (section 4.9)
	31	subtype 1: Empty							draft-ietf-avt-rtp-svc (section 4.10)
	31	subtype 2: Non-Interleaved Multi-Time Aggregation Packets	NI-MTAP						draft-ietf-avt-rtp-svc (section 4.7)

SVC; VCL (Video Coding Layer) NAL unit; NON-VCL NAL unit

SEI (Supplemental Enhancement Information) (type 6) (ITU-T H.264 03/2010, Annex D)

SEI for 3DTV
stereo_video_info and frame_packing_arrangement: to support rendering decoded images from the DPB onto 3D display systems that support field/frame interleaving, checker board, line interleaving etc.
(Corrigendum AVC)

SPS (Sequence Parameter Set) (type 7) / PPS (Picture Parameter Set) (type 8)

Specified in:

NAL structure
RTP streaming

in-band: NAL structure

Reintroduce SPS and PPS when transmuxing with ffmpeg from RTMP to RTP
Force introduction of SPS and PPS (NALs 7, 8) with RTP streaming with GStreamer: rtph264pay config-interval=-1

out-band: RTP SDP: sprop-parameter-sets

MP4

SPS/PPS ->	in a specific avcC box (init file mp4)	inband (segment file m4s)
H.264 / AVC	avc1	avc3
H.265 / HEVC	hvc1	hev1

Development

x264

encoder/encoder.c

x264_sps_init
x264_pps_init

[Libav-user] Getting SPS and PPS-information from x264 encoding
How get sps struc from h264 video with libav ffmpeg
H.264 extradata (partially) explained - for dummies

H.264 NAL bitstream

How to get x264 SPS / PPS parameter out of AVCodecContext?
parser_h264.c (Feng)

Documents

ITU-T Rec. H.264 (04/2017): 7.3.2.1.1 Sequence parameter set data syntax

AUD (Access Unit Delimiter) (type 9)

Gstreamer: x264enc do not send AUD to RTP, to avoid problems with WebRTC in Chrome

VUI (Video Usability Information)

`aspect_ratio_idc` (14496-10, table E-1)	Sample aspect ratio	(informative) Examples of use
0	Unspecified
1	1:1 (“square”)	1280x720 16:9 frame without overscan 1920x1080 16:9 frame without overscan (cropped from 1920x1088) 640x480 4:3 frame without overscan
2	12:11	720x576 4:3 frame with horizontal overscan 352x288 4:3 frame without overscan
3	10:11	720x480 4:3 frame with horizontal overscan 352x240 4:3 frame without overscan
4	16:11	720x576 16:9 frame with horizontal overscan 540x576 4:3 frame with horizontal overscan
5	40:33	720x480 16:9 frame with horizontal overscan 540x480 4:3 frame with horizontal overscan
6	24:11	352x576 4:3 frame without overscan 540x576 16:9 frame with horizontal overscan
7	20:11	352x480 4:3 frame without overscan 480x480 16:9 frame with horizontal overscan
8	32:11	352x576 16:9 frame without overscan
9	80:33	352x480 16:9 frame without overscan
10	18:11	480x576 4:3 frame with horizontal overscan
11	15:11	480x480 4:3 frame with horizontal overscan
12	64:33	540x576 16:9 frame with horizontal overscan
13	160:99	540x480 16:9 frame with horizontal overscan
14	4:3	1440x1080 16:9 frame
15	3:2	1280x1080 16:9 frame
16	2:1	960x1080 16:9 frame
17..254	Reserved
255	Extended_SAR

...

B-frames

B-frames in ffmpeg

H.264 profiles / levels (ITU-T REC H.264-201704: A.2 Profiles, A.3 Levels) (profile-idc: Annex A) (SVC profiles and levels: G.10) (MVC profiles and levels: H.10) (italic text are parameters to JSVM reference software configuration files)

8192		7680	4096			3680	3616	2560	2048			1920	1600	1408		1280 (HD)			1024	800	720 (SD)		704		400	352				320		176	144	128
samples (max fps)																																					maximum video bit rate (bps)								`level_idc`
4320	4096	4320	2304	2160	2048	1536	1536	1920	1536	1088	1024	1088	1200	1152	960	1024	960	720	768	600	576	480	576	480	224	576	480	288	240	240	176	144	80	96
			16:9		4Kx2K		2.35:1	16VGA	4XGA	2Kx1080	2Kx1K	1080HD	4SVGA	16CIF	525 16SIF	SXGA	4VGA	720p HD	XGA	SVGA	625SD	525SD	4CIF	525 4SIF		625 HHR	525 HHR	CIF	525 SIF	QVGA		QCIF		SQCIF
120.9	127.5	128.9	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0	300.0		300.0	300.0	300.0		300.0	300.0	300.0	300.0	300.0		300.0		300.0	levels	6.2
60.4																																				6.1
30.2																																				6.0
																																				5.2
			26.7	28.4	30.0							120.5																								5.1 (4K)	240M			300M	720M	960M			33
						26.7		30.7		67.8	72.0	72.3																								5.0	135M			168,75M	405M	540M			32
										60.0		64.0 (60)																								4.2 (1080p)	50M			62,5M FF	150M	200M			2A
											30.0	30.1						68.3																		4.1 (1080i)	50M			62,5M FF	150M	200M			29
											30.0	30.1						68.3																		4.0 (HD)	20M			25M EE	60M	80M			28
																42.2		60.0																		3.2 (720p60)	20M			25M	60M	80M			20
																		30.0			66.7	80.0														3.1 (720p30)	14M			17,5M	42M	56M			1F
																					25.0	30.0				51.1	61.4									3.0 (SD)	10M DBDB	10M	10M D	12,5M D	30M	40M			1E
																					12.5	15.0				25.6	30.7									2.2	4M			5M	12M	16M			16
																										25.0	30.0									2.1	4M			5M	12M	16M			15
																												30.0		36.0 (30)						2.0	2M CC	2M	2M	2,5M	6M	8M			14
																												30.0		36.0 (30)						1.3	768k			960k	2304k	3072k			0D
																												15.2		20.0 (15)						1.2	384k BB	384k	384k	480k	1152k	1536k			0C
																												7.5		10.0		30.3				1.1	192k			240k	576k	768k			0B
																																15.0		30.9		1b	128k A	128k	128k	160k	384k	512k
																																15.0		30.9		1.0	64k			80k	192k	256k			0A

																																			AVC		42 (66)	58 (88)	4D (77)	64 (100)	6E (110)	7A (122)	F4 (244)	2C (44)	profile_idc
																																			SVC		53 (83)			56 (86)
																																			MVC	multiview				76 (118)
																																			MVC	stereo				80 (128)
																																					[scalable] BASELINE	EXTENDED	MAIN	[scalable[intra], stereo, multiview] HIGH	HIGH10 [intra]	HIGH 4:2:2 [intra]	HIGH 4:4:4 [intra]	CAVLC 4:4:4 intra
																																					[scalable] BASELINE	EXTENDED	MAIN	FRExt
											scalability	partition						prediction				MB				quantisation		slices			CA		transform				profiles
																												arbitrary slice order									x	x
																												redundant slices									x	x
																						flexible macroblock ordering															x	x
																																	4x4				x	x	x	x
																															CAVLC (lower complexity) (SymbolMode=0)						x	x	x	x
																												I slices									x	x	x	x
																												P slices									x	x	x	x
																												SP and SI slices										x
												data partioning																										x
																												B slices										x (constraint_set5_flag)	x (constraint_set5_flag)	x (constraint_set5_flag)
																		weighted prediction (WeightedPrediction, WeightedBiprediction)																				x	x	x
																															CABAC (high complexity) (SymbolMode=1)						(certain levels)		x	x
																										quantisation matrix														x
																																	8x8 (Enable8x8Transform=1)				(certain levels)			x
											layer spatial ratio																										1:1 1.5:1 2:1			unrestricted

profile-level-id (sdp), codecs (mpd)

	byte	bit	value	meaning	ffmpeg H.264
1st	profile_idc				`-profile:v ...`
2nd	`profile-iop`	`constraint_set0_flag`	0	(must be 0 when profile_idc is: 44, 100, 110, 122, or 244)	?
		`constraint_set0_flag`	1	all constraints in A.2.1 (baseline)
		`constraint_set1_flag`	0	(must be 0 when profile_idc is: 44, 100, 110, 122, or 244)
		`constraint_set1_flag`	1	all constraints in A.2.2 (main)
		`constraint_set2_flag`	0	(must be 0 when profile_idc is: 44, 100, 110, 122, or 244)
		`constraint_set2_flag`	1	all constraints in A.2.3 (extended)
		`constraint_set3_flag`	0
		`constraint_set3_flag`	1
		`constraint_set4_flag`	0
		`constraint_set4_flag`	1
		`constraint_set5_flag`	0
		`constraint_set5_flag`	1
		`reserved_zero_2bits`	0
		`reserved_zero_2bits`	0
3rd	level_idc				`-level:v ...`

1st byte: profile_idc (ITU-T REC H.264-200711: AVC: A.2; SVC: G.10; MVC: H.10):

profile			AVC	SVC	MVC
profile			AVC	scalable	multiview	stereo
CAVLC	4:4:4	intra	2C (44)
baseline			42 (66)	53 (83)
main			4D (77)
extended			58 (88)
high			64 (100)	56 (86)	76 (118)	80 (128)
		intra		56 (86)
	10		6E (110)
	10	intra	6E (110)
	4:2:2		7A (122)
	4:2:2	intra	7A (122)
	4:4:4		F4 (244)
	4:4:4	intra	F4 (244)

2nd byte: profile-iop

H.264 04/2017: 7.4.2.1.1 Sequence parameter set data semantics
Examples:

c0
e0

examples:

profile-level-id (sdp)	codecs (mpd)		meaning of profile-level-id
			profile_idc	`profile-iop`	level_idc
42c00d	avc1.42c00d		42: baseline	c0: 110- ----	0d: 1.3
42c01e	avc1.42c01e				1e: 3.0
42c01f	avc1.42c01f				1f: 3.1
42c028	avc1.42c028				28: 4.0
42e0..		constrained baseline		e0: 111- ----
4200..		baseline
4d00..		main	4d: main
640c..		constrained high	64: high
6400..		high			33: 5.1

ETSI TS 102 005 (IP):

Annex A.3.1.5 "Summary of key tools and parameter ranges for capability A to E IRDs"

constraint_set1_flag

Annex B: "TS 102 005 usage in DVB-IP datacast"

H264/AVC resolutions and capabilities (A,B,C,D,DB,E)

SVC

capability	H.264/AVC		SVC				VC-1		resolution	IPDC resolution
	profile	level	profile	level	scalability		profile	level
	profile	level	profile	level	SNR	spatial	profile	level
A	baseline	1b					simple	LL	176x144@15	176x144@15 128x96@30 144x80@30
B		1.2	scalable baseline	1.2	x	-	simple	ML	176x144@30 (VC-1) 320x240@24 (VC-1) 352x288@15	176x144@30 320x240@15 320x176@15
C		2.0		2.0		-	advanced	L0	352x288@30	320x240@30 320x176@30 400x224@30
DB		3.0		3.0		2-layer			720x576@25 720x576@30
D	main	3.0	scalable high	3.0		2-layer	advanced	L1	720x576@25 720x576@30	480x320@30 (SVC) 640x352@30 (SVC) 640x480@30 (SVC) 800x448@30 (SVC)
E	high	4.0		4.0		3-layer	advanced	L3	1280x720@50/60 1920x1080@25/30
F	high	4.2		4.2		3-layer			1920x1080@60

ISO Base Media File Format (MPEG-4 part 12)

Info

ISO base media file format (wp)

Extensions (wp)

	description	owner	brand	encryption
3GPP
3GPP2
AVC FF			iso6 (avc3)
CFF	Common File Format	UltraViolet / DECE	`ccff`
DVB-FF	DVB File Format	DVB	`dvbr1 dvbt1`
MP4		MPEG	`mp41 mp42`	[cenc]
OMA DCF		OMA	`odcf`	x
OMA PDCF		OMA	`opf2`	x
PIFF	Protected Interoperable File Format	Microsoft	`piff`	x

Stream Access Points (SAP) (ISO/IEC 14496-12: Annex I)

SAP types:

type		name	info
1	Tept = Tdec = Tsap =Tptf	Closed GOP random access point	accepted by DASH-AVC/264
2	Tept = Tdec = Tsap <Tptf	Closed GOP random access point	accepted by DASH-AVC/264
3	Tept < Tdec = Tsap <=Tptf	Open GOP random access point
4	Tept <= Tptf < Tdec = Tsap	Gradual Decoding Refresh (GDR)
5	Tept = Tdec < Tsap
6	Tept < Tdec < Tsap

MP4 (MPEG-4 part14) / AVC FF (MPEG-4 part 15) (extensions of ISOBMFF)

Formats de fichiers MPEG (C.Concolato)
AAC Audio and the MP4 Media Format
iso6 / avc3 (inband)

Video codecs
avc3 in GPAC

MP4Box
DashCast

"AVC3" format (BBC MPEG-DASH test streams)

SPS and PPS present in media segments (in every RAP), instead of in initialisation segment (thus an inisialitation segment can be common for several representations of the same video)

atoms

moov: metadata (timescale, duration, display...)

mvhd
trak

mdia
mdia
...

trak
...

mdat: data
...

Programari / Software

Bento4
MPEG4IP
GPAC
ffmpeg
gstreamer
...

Fragmentació / Fragmentation

Descàrrega progressiva / Progressive download
stream-m: Fragment sizes
Eines / Tools

mp4fragment
MP4Box
ffmpeg / libav

	description	creation			info	limitations
		MP4Box	ffmpeg	Bento4	MP4Box -info
regular file	mdat moov
progressive download file	moov mdat	(any)	-movflags faststart (fragmentation)		`File suitable for progressive download (moov before mdat)`
interleaved		-`inter ms`
fragmented	moov (can be forced to be empty) moof (frag#1) traf (track #1) traf (track #2) mdat (frag#1) moof (frag#2) traf (track #1) traf (track #2) mdat (frag#2) ...	-frag ms	-movflags empty_moov+frag_keyframe -frag_duration duration_µs -frag_size size_bytes	mp4fragment		empty_moov is not supported by QuickTime player fragmented files are only supported on Android >4.2
fragmented with separated tracks	?		-movflags separate_moof			HbbTV TS 102 796 B.3.1: "The movie fragment box ('moof') shall contain only one track fragment box ('traf')"
splitting	chunk#1.mp4 chunk#2.mp4 ...	-split seconds	segment	mp4split
dash	mp4 file (init) moov m4s file (frag#1) moof (frag#1) mdat (frag#1) m4s file (frag#2) moof (frag#2) mdat (frag#2) ...	-dash ms		mp4-dash

The MP4 Registration Authority (MP4RA)

Box: meta...
Brands: mp41, mp42, 3gp4, 3gp5...
Codecs: avc1, mp4a, mp4s, mp4v... (VLC, MP4Box -info) (see a superset at fourcc.org)
Track references
handler / group: sdsm, soun, vide, hint, mp21... (MP4Box -info)

objectTypeIndication / streamType:

			1	3	4	5
			streamType (sdp: a=fmtp:...streamType=) (<saf:...Header>)
			ObjectDescriptor	Scene Description	Visual	Audio
objectTypeIndication (*) (sdp: a=fmtp:...objectType) (<saf:...Header>)
1		MPEG4Systems1	Object Descriptor Stream	BIFS Stream version 1
2		MPEG4Systems2		BIFS Stream version 2
9	0x09	LASeR?		generated by Laser Java reference software: bin2xml (xml file) GPAC mp4_streamer: sdp Darwin: sdp
32	0x20	MPEG4Visual			MPEG-4 Visual Stream
64	0x40	MPEG4Audio				MPEG-4 Audio Stream
	0x60	MPEG2VisualSimple			MPEG-2 Visual Stream
97	0x61	MPEG2VisualMain
	0x62	MPEG2VisualSNR
	0x63	MPEG2VisualSpatial
	0x64	MPEG2VisualHigh
	0x65	MPEG2Visual422
	0x66	MPEG2AudioMain				MPEG-2 Audio Stream / Part 7 -AAC
	0x67	MPEG2AudioLowComplexity
	0x68	MPEG2AudioScaleableSamplingRate
	0x69	MPEG2AudioPart3				MPEG-2 Audio Stream Part 3
107	0x6b	MPEG1Audio
108	0x6c	JPEG			JPEG Visual stream
109	0x6d	PNG			PNG Visual stream
	0xde	Vorbis
	0xdf	Theora

Relacions d'aspecte / Aspect ratios

05MPEG-7 ( ISO/IEC 15938 )#

MPEG-7
- MPEG-4 and MPEG-7 (seminar in smd074)
- MPEG-7
- Multimedia Metadata Community
- Programari / Software

06MPEG-21 ( ISO/IEC 21000 )#

MPEG-21

MPEG21

Bitstream Syntax Description Language (BSDL)

"XML-based Exploitation of Region of Interest Scalability in Scalable Video Coding" (pdf)

"Using MPEG-21 DIDL to Represent Complex Digital Objects in the Los Alamos National Laboratory Digital Library" (UUID)
Programari / Software

GPAC

07MMT (MPEG Media Transport / MPEG Modern Transport / Modern Media Transport / Multi Media Transport )#

Kyoto Workshop

MMT Workshop in Kyoto 2009 (Kyoto, JP, 2010/01/20)

DVB Work on Multimedia Transport (estàndards DVB, esquemes de piles de protocols, GSE, MMT, HBB...)
Industry viewpoint of Internet TV over IP Networks: Trends and Technologies (OTT, P2P streaming, CDN-P2P...)
...

89th MPEG London Workshop 2009

Workshop on MMT (Modern Media Transport) – Call for Contributions (Multimedia Communication blog)
Presentations of MMT Workshop in London 2009 (Chiariglione)
Use cases and requirements of MMT

Fixed and mobile convergence (FMC)
Post HD
Adaptive progressive download
Open IPTV over unmanaged network
Context-aware streaming

Draft MPEG Media Transport (MMT) Context and Objectives

08MPEG-DASH ( ISO/IEC 23009 )#

Info

MPEG-DASH Timeline
[IBC 2013 Report] MPEG-DASH Ecosystem Status
Quick tutorial on MPEG-DASH

gràfic mp4: moof, traf

What is a standard and what does it cover? (featuring a MPEG-DASH case study) (GPAC licensing)
WebM Dash specification
MPEG-DASH in nginx-rtmp-module: live video in the browser without flesh
Multicam

MPEG DASH multiple video implementation over one single manifest

Profiles:

MPEG-DASH					HbbTV		DVB-DASH	GPAC
		urn	MPD@type		1.1	1.5 (Annex B) 2.0 (Annex E)	ETSI TS 103 285 - A168	--profile	HTTP streaming
	full	urn:mpeg:dash:profile:full:2011					urn:dvb:dash:profile:dvb-dash:2014	full
ISO base media FF	onDemand	urn:mpeg:dash:profile:isoff-on-demand:2011	static	each media presentation is a single segment				onDemand, dashavc264:onDemand	`MP4Box ... -single-segment ...`
	live	urn:mpeg:dash:profile:isoff-live:2011				urn:hbbtv:dash:profile:isoff-live:2012		live, dashavc264:live
	main	urn:mpeg:dash:profile:isoff-main:2011	dynamic					main
MPEG-2 TS	main	urn:mpeg:dash:profile:mp2t-main:2011
MPEG-2 TS	simple	urn:mpeg:dash:profile:mp2t-simple:2011						simple

GPAC

ondemand
live

MP4Box
DashCast

DASH-AVC/264

x

DVB-DASH

HbbTV

x
x

	GPAC
	ondemand	live
		MP4Box	DashCast
DASH-AVC/264		x
DVB-DASH
HbbTV		x	x

			DVB-DASH (HbbTV 2.0 Annex E)	HbbTV 1.5 Annex B
MPD
	Period		Subset: ignored Period.SegmentList: not present
		ondemand	Period.SegmentTemplate: not present
	AdaptationSet	live	@group: ignored absolute BaseURL: ignored AdaptationSet with ContentComponent: ignored AdaptationSet with more than one Representation: ignored, except: @segmentAlignment='1' @startWithSAP={'1','2'} @maxSegmentDuration or @type="static"
		ondemand	@group: ignored ...
	Representation	live	@mimeType="{video,audio,application,text}/mp4"	"Each Representation shall contain only one media component, i.e. a single audio or video track"
segment format			...
attributes and elements			AdaptationSet @contentType="video" @maxWidth @maxHeight @maxFrameRate @par Representation @contentType="video" @width @height @frameRate @scanType @sar
dimension constraints			MPD size < 256KB ... segment duration > 1s ...	1s < segment duration < 15s
video			...
audio			...

DVB-DASH
HbbTV 1.2.1

Generation of MPEG-DASH for HbbTV with GPAC

MPEG-DASH Industry Forum (DASH-IF)

Identifiers
DASH talks NAB 2014
Players

dash.js

Guidelines

Guidelines for Implementation: DASH-AVC/264 Interoperability Points (v2.0, 15th August 2013)

DASH

IDR-like SAPs (i.e., SAPs type 2 or below) at the start of each (Sub)Segment for simple switching
segment duration tolerance: +-50%
addressing schemes are restricted to:

templates with number-based addressing
templates with time-based addressing
subsegments with segment index

Video: H.264/AVC Progressive High Profile
Audio: HE-AAC v2 profile

multi audio

Subtitles: ISO/IEC 14496-30

W3C TTML
SMPTE profile on SMPTE Timed Text / EBU TT
not expected in DASH-AVC/264:

CEA-708 Digital Television (DTV) Closed Captioning
3GPP Timed Text
Web VTT

DRM: ISO/IEC 23001-7

MPEG-DASH Industry Forum releases implementation guidelines

HTTP streaming

HTTP streaming documents

MPEG-DASH Documents

MPEG-DASH & GPAC (pdf) (Jean Lefeuvre)
MPEG-DASH Dataset MMSys 2012 (slideshare) (22/02/2012)
A VLC Media Player Plugin enabling Dynamic Adaptive Streaming over HTTP (DASH) (DASH at the ACM Multimedia) (Christopher Mueller) (29/11/2011)

origin of libdash
slide 5: MPD / Period / Group / Representation
slide 6: DASH data model

Period: Separates the content for e.g. ad insertion or for camera angle changes
...

Poster from the ACM Multimedia 2011 (pdf)

Segment indexing

HTTP streaming of MPEG media (C.Timmerer, C.Müller) (Sep 2010) (slideshare)
Dynamic Adaptive Streaming over HTTP (C.Timmerer, C.Müller) (May 2011) (slideshare)
Dynamic Adaptive Streaming over HTTP (MPEG-DASH) Open source tools | Cloud services (Stefan Lederer) (2013) (slideshare)
Dynamic Adaptive Streaming over HTTP - Desing Principles and Standards (T. Stockhammer, Qualcomm) (pdf): MPEG-DASH

segment index: simple, hierarchical, daisy-chain

Usages of DASH for Rich Media Services (C. Concolato) (pdf)

HTTP streaming software

Servidor / Server

NGINX-based Media Streaming Server (nginx-rtmp-module)
HTTP streaming with GPAC

GPAC:

On demand

MP4Box -dash

Live

MP4Box-dash-live
DashCast

Dash.encrypt Segmenter (castLabs)
...

Validació / Validation

MPD

MPEG-DASH MPD Validator (ITEC)

Segments

DASH validator (DASH-IF)

Client

DASH-IF software
Native

libav / ffmpeg

avplay -i "concat:nom_segment_init.mp4|nom_segment_5.m4s|nom_segment_6.m4s"
or manually concatenate the init and the segments:

cat segment_init.mp4 segment_5.m4s segment_6.m4s > all.mp4
avplay all.mp4

GPAC

MP4Client

VLC

VLC 2.1.0

nightly build

VLC 2.0

basic-cm profile

VLC DASH plugin (*) (uni-klu) (YouTube)

compilació / compilation

libdash (ITEC AAU Klagenfurt)

Compilació / Compilation
slides
General unifying of the .mpd schemes (VLC and DashCast)

libdash will be put into VLC on summer 2013

player:

qtsampleplayer

JavaScript / Media Source Extensions (Chrome, Mozilla)

HTML5 video players
dash.js (implementation of DASH-AVC/264) (github)

DASH-264 JavaScript Reference Client (Chrome) (DASH-IF)

MPEG-DASH player (based on DASH.js) (Microsoft Azure)
Preparation of segments to be played with dash.js

MPEG-DASH / Media Source demo (requires Chrome v23+)
Shaka

Flash

dash.as (Castlabs)
MPEG-DASH Prototype Player (Edgesuit)
bitdash MPEG-DASH Player (mpd, mobilempd) (bitmovin)

multi-language, subtitles

DRM

5.8.4.1 ContentProtection

urn:mpeg:dash:mp4protection:2011

Data model (ISO/IEC 23009-1:2012(E), Figure 3)

Media Presentation Description

Period #1

Adaptation set (video 1)

Representation (bit rate 1)

Segment #1
(6 seconds: -dash 6000)

sidx
(-subsegs-per-sidx 4)

subsegment #1 (whole number of AU)
(whole number of complete movie fragments)

movie fragment #1
(500ms: -frag 500)

moof	mdat	...	mdat

movie fragment #2

movie fragment #3

subsegment #2

subsegment #3

subsegment #4

sidx

subsegment #5

subsegment #6

subsegment #7

subsegment #8

Segment #2 (6s)

Representation (bit rate 2)

Adaptation set (audio 1)

Adaptation set (audio 2)

Period #2

ISOBMF structure

Segment index (pdf)

GPAC MP4Box option

-

SS1	SS2	SS3	SS4	SS5	SS6

-subsegs-per-sidx -1

simple

SI1	SS1	SS2	SS3	SS4	SS5	SS6

-subsegs-per-sidx 0

hierarchical

SI1	SI2	SS1	SS2	SI3	SS3	SS4	SI4	SS5	SS6

-subsegs-per-sidx 2

daisy-chain

SI1	SS1	SS2	SI2	SS3	SS4	SI3	SS5	SS6

-subsegs-per-sidx 2
-daisy-chain

onDemand profile: each media presentation is a single segment (GPAC: -single-segment)
ISO/IEC 23009-1:2012(E)

p.10: "For example, in the very simplest cases each Segment or Subsegment begins with a random access point and the boundaries of Segments or Subsegments are aligned across the Representations of an Adaptation Set. In this case, switching Representation involves playing to the end of a (Sub)Segment of one Representation and then
playing from the beginning of the next (Sub)Segment of the new Representation."
p.11: "For live services, however, Segments become available with time as the content is produced. The Media Presentation Description may be updated regularly to reflect changes in the presentation over time, for example Segment URLs for new segments may be added to the MPD and those
for old, no longer available Segments may be removed. However, if Segment URLs are described using a template, this updating may not be necessary except for some redundancy/failover cases."
8.4 ISO Base media file format live profile: "Note that despite the profile is optimized for live services, the MPD@Type attribute may be set to 'static' to distribute non-live content, for example in case a live Media Presentation is terminated, but kept available as On-Demand service."

MPD

mime-type

application/dash+xml

MPEG-DASH MPD Validator (ITEC uni-kla)
Codec description (codecs=...)

AAC codec description
H.264 codec description (profile-level-id)

MPD
MP4Box option

SegmentBase
-single-segment

SegmentList
-segment-name

SegmentTemplate
-url-template

	urn:mpeg:DASH:schema:MPD:2011	urn:mpeg:DASH:schema:MPD:2011	urn:mpeg:mpegB:schema:DASH:MPD:DIS2011	urn:3GPP:ns:PSS:AdaptiveHTTPStreamingMPD:2009
MP4Client			-	x
VLC			x	-
	MPD xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011" profiles="urn:mpeg:dash:profile:isoff-main:2011" type="static" mediaPresentationDuration="PT1H37M38S" minBufferTime="PT10.0S" BaseURL http://www-itec.uni-klu.ac.at/ftp/datasets/mmsys12/RedBullPlayStreets/redbull_10s/ Period start="PT0S" AdaptationSet bitstreamSwitching="true" Representation id="0" codecs="avc1" mimeType="video/mp4" width="320" height="240" startWithSAP="1" bandwidth="100031" SegmentBase Initialization sourceURL="redbull_10sec_100kbit/redbull_100kbit_dash.mp4" SegmentList duration="10" SegmentURL media="redbull_10sec_100kbit/redbull_10sec1.m4s" SegmentURL media="redbull_10sec_100kbit/redbull_10sec2.m4s" ... Representation id="1" codecs="avc1" mimeType="video/mp4" width="320" height="240" startWithSAP="1" bandwidth="148267" SegmentBase Initialization sourceURL="...mp4" SegmentList duration="10" SegmentURL ... SegmentURL ... ... ... Representation id="16" codecs="avc1" mimeType="video/mp4" width="1920" height="1080" startWithSAP="1" bandwidth="5927049" SegmentBase Initialization sourceURL="redbull_10sec_6000kbit/redbull_6000kbit_dash.mp4" SegmentList duration="10" SegmentURL media="redbull_10sec_6000kbit/redbull_10sec1.m4s" SegmentURL media="redbull_10sec_6000kbit/redbull_10sec2.m4s" ...	MPD xmlns="urn:mpeg:DASH:schema:MPD:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011" xmlns:mas="urn:marlin:mas:1-0:services:schemas:mpd" type="static" mediaPresentationDuration="PT14M48S" minBufferTime="PT3.00S" Period AdaptationSet mimeType="audio/mp4" ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" ContentProtection schemeIdUri="urn:uuid:..." mas:MarlinContentIds mas:MarlinContentId urn:marlin:kid:... Representation bandwidth="137568" id="audio" SegmentList duration="3000" timescale="1000" Inititalisation sourceURL="audio/init.mp4" SegmentURL media="audio/seg-0000.m4f" SegmentURL media="audio/seg-0001.m4f" SegmentURL ... AdaptationSet mimeType="video/mp4" startWithSAP="1" ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" ContentProtection schemeIdUri="urn:uuid:..." mas:MarlinContentIds mas:MarlinContentId urn:marlin:kid:... Representation bandwidth="248784" id="video.1" SegmentList duration="3000" timescale="1000" Inititalisation sourceURL="video/1/init.mp4" SegmentURL media="video/1/seg-0000.m4f" SegmentURL media="video/1/seg-0001.m4f" ... Representation bandwidth="610512" id="video.2" SegmentList duration="3000" timescale="1000" Inititalisation sourceURL="video/2/init.mp4" SegmentURL media="video/2/seg-0000.m4f" SegmentURL media="video/2/seg-0001.m4f" ... Representation ... AdaptationSet...	MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema" xmlns="urn:mpeg:mpegB:schema:DASH:MPD:DIS2011" xsi:schemaLocation="urn:mpeg:mpegB:schema:DASH:MPD:DIS2011" profiles= "urn:mpeg:mpegB:profile:dash:isoff-basic-on-demand:cm" type="OnDemand" mediaPresentationDuration="PT5M0.08S" minBufferTime="PT10.00S" Period Group segmentAlignmentFlag="true" mimeType="video/3gpp" Representation bandwidth="249967" SegmentInfo duration="PT10.00S" InitialisationSegmentURL sourceURL="toto_v200.3gp" Url sourceURL="toto_v200_1.3gs" Url sourceURL="toto_v200_2.3gs" ... Representation bandwidth="430314" SegmentInfo duration="PT10.00S" InitialisationSegmentURL sourceURL="toto_v400.3gp" Url sourceURL="toto_v400_1.3gs" Url sourceURL="toto_v400_2.3gs" ... Representation...	MPD xmlns="urn:3GPP:ns:PSS:AdaptiveHTTPStreamingMPD:2009" type="OnDemand" ProgramInformation Title Period start="PT0S" duration="PT0H2M29.08S" Representation mimeType="video/mp4" width="480" height="272" sampleRate="22050" numChannels="2" lang="und" startWithRAP="false" bandwidth="1209572" SegmentInfo duration="PT10.00S" InitialisationSegmentURL sourceURL="toto_dash.mp4" Url sourceURL="seg1.m4s" Url sourceURL="seg2.m4s" ...
		MPD xmlns="urn:mpeg:DASH:schema:MPD:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" mediaPresentationDuration="PT14M48S" minBufferTime="PT3.00S" Period AdaptationSet mimeType="audio/mp4" Representation bandwidth="137568" id="audio"> SegmentTemplate duration="3000" initialisation="audio/init.mp4" media="audio/seg-$Number%04d$.m4f" startNumber="0" timescale="1000" AdaptationSet mimeType="video/mp4" startWithSAP="1" Representation bandwidth="248784" id="video.1" SegmentTemplate duration="3000" initialisation="video/1/init.mp4" media="video/1/seg-$Number%04d$.m4f" startNumber="0" timescale="1000" Representation bandwidth="610512" id="video.2" SegmentTemplate duration="3000" initialisation="video/2/init.mp4" media="video/2/seg-$Number%04d$.m4f" startNumber="0" timescale="1000" ... AdaptationSet...
example	RedBullPlayStreets_10s_isoffmain_DIS_23009_1_v_2_1c2_2011_08_30.mpd	it	mtbike.mpd	bbb_dash.mpd

09MPEG-H ( ISO/IEC 23008 )#

Part 2: HEVC / H.265

http://www.francescpinyol.cat/mpeg.html
Primera versió: 22 de febrer de 2008 / First version: 22nd February 2008
Darrera modificació: 25 de gener de 2024 / Last update: 25th January 2024

Cap a casa / Back home.

0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7
F	NRI		nal_unit_type					RR		PRID						TL			DID			QL		R	B	U	D	G	L	O

MPD	MP4Box option
SegmentBase	-single-segment
SegmentList	-segment-name
SegmentTemplate	-url-template