With increasing demand for higher resolution videos and exponential increase in video consumption, drives to demand for new video compression standard. HEVC is the latest video compression standard, to achieve 50% less bit-rate with same video quality compared to previous legacy video compression standard H.264. This article gives a quick glimpse to modules/blocks in HEVC.
Coding block size: Similar to H.264, HEVC is also a block-based video compression standard. H.264 splits the picture into block size of 16x16 whereas, HEVC splits the picture into maximum coding unit size and splits it further till minimum coding unit size in quad-tree manner. Coding unit sizes in HEVC can range from 64x64 to 8x8. Quad-tree coding unit size plays major role in compression gain achieved by HEVC which in-turns increases the complexity multi-fold.Below table, illustrates approximate complexity to estimate each depth.
Prediction block size: Coding unit is further split into prediction unit and transform unit. Prediction unit in H.264 ranges from 16x16 to 4x4 including rectangular blocks. Prediction unit in HEVC ranges from 64x64 to 4x8/8x4. In addition to rectangular prediction units, asymmetric partitioning of prediction units are allowed in HEVC to improve the prediction of inter coded block.
- a) 2Nx2N
- b) 2NxN
- c) Nx2N
- d) NxN
- e) 2NxnU
- f) 2NxnD
- g) nLx2N
- h) nRx2N
Transform Unit size: Transform plays a vital role in compressing the residue. H.264 allows transform sizes of 4x4 and 8x8, whereas HEVC allows transform sizes from 32x32 to 4x4. Additionally, HEVC enables transform units to be split in quad-tree manner resulting in optimal compression of residual values.
Better Prediction: Better the prediction, lesser is the bits needed to code the residual coefficients. HEVC focus on improving the prediction in both intra and inter modes. In intra-prediction, HEVC has introduced additional angles intended to improve the prediction. In case of inter-prediction, HEVC uses 8-tap filter for 1/4 pixel. Thus in both intra and inter prediction, complexity of estimation in-terms of encoder is multi-fold. In HEVC, motion vector predictor is estimated by constructing a list of neighboring motion vectors and selecting the optimal one. Thus compared to median-based motion vector predictor in H.264, HEVC results in lesser bits to code MVD. HEVC has additionally introduced merge mode which just signals the merge index to select the MVP and motion vector difference is implied as zero. In case of skip mode, merge index is the final motion vector for prediction in which residue is zero.
|Maximum number of entries in list||Motion Vector Difference||Residue|
|Motion Vector Predictor (MVP)||2||Present||Present|
|Skip mode||Same as merge mode||Zero||Zero|
Lossless coding: In addition to PCM mode, HEVC allows another lossless coding mode namely “trans-quant bypass” which disables transform-quantization & codes the coefficients directly. This result in lesser bits compared to PCM coding.Introduction of “trans-quant bypass” in HEVC helps in medical imaging, satellite imaging& other lossless coding applications in which compression gains by HEVC is better than other still-image formats. Technical page of HEIF indicates JPEG would need 139% higher bit-rate than HEVC with same quality . Also to save the bits, such as incase of noisy sequences, HEVC allows only transform to be skipped using “transform skip” feature.
Parallel Tools : In order to improve the parallelism, HEVC has introduced two new parallel tools namely Tiles and WPP. Enabling Tiles allows sub-division of picture into further rectangular regions which is encoded/decoded independently. WPP introduces parallelism in entropy coding in which multiple rows can be encoded/decoded simultaneously. WPP process the encoding module in wave-front fashion, which triggers the next row of picture once top and top-right blocks are processed by CABAC. In brief, dependency exists between current row and previous row. Though introduction of Tiles & WPP helps in parallelism, they have disadvantages of degrading the compression and picture quality. However WPP is more advantageous than Tiles in terms of visual artifacts. More details and implementation challenges are discussed in detailed in our white paper .
In-loop filters: Block-based prediction and different quantization parameter are the major source of blocking artifacts. HEVC uses de-blocking filter same as H.264 to reduce these blocking artifacts. However to simplify filtering process and to improve the parallelism, HEVC has increased edge length of filter to 8. Edges to be filtered are determined by prediction unit and transform unit edges coded in the particular picture. In addition to de-blocking filter, in-order to improve the picture further better HEVC has introduced Sample Adaptive Offset (SAO) filter. SAO helps to reduce the banding artifacts which is un-addressed by de-blocking filter. Additionally, SAO can be disabled/enabled at block level whereas flexibility to control de-blocking filter at block level is not available. Pros & Cons of de-block filter & SAO filter is discussed in our white paper .
|Macroblock/Coding Tree Unit Size||16x16||64x64, 32x32, 16x16|
|PU size||16x16, 8x16, 16x8, 8x8, 4x8, 8x4, 4x4||
64x64, 64x32, 32x64, 32x32, 32x16, 16x32, 16x16, 16x8, 8x16, 8x8, 8x4, 4x8.
AMP : 64x16/64x48, 16x64/48x64, 32x8/32x24, 8x32/24x32, 16x4/16x12, 4x16/12x16.
|TU size||4x4, 8x8||32x32, 16x16, 8x8, 4x4|
|Intra prediction||9 modes||35 modes|
|MV resolution||Integer, ½-pel, ¼-pel : Cascaded||Integer, ½-pel, ¼-pel : Non-cascaded|
|Intra mode prediction||Derive MPM from neighbors||Derived from list of MPM modes|
|MV prediction||Median of left, top and top-right neighbors||Derived from list with 2-entries|
|Entropy Coding||VLC, CAVLC, CABAC||CABAC|
|In-loop filters||De-block filter||De-block & SAO filter|
|De-blocking||4x4 Block Edge||8x8 Block Edge|
|Error resilience tools||FMO, ASO, Redundant slice||-|
|Parallel Tools||Slices||Slices, Tiles & WPP|
Achieving higher compression with HEVC is through introduction of these new coding tools which in-turn increases the complexity. Complexity of HEVC encoder is 6x-8x than of H.264 and HEVC decoder is 1.5x-2x complex than H.264. More detailed presentation on encoder solution is presented in our webinar .