The communications technology journal since 1924 2013 • 6Next generationvideo compressionApril 24, 2013
Next generationvideo compressionMPEG and ITU have recently approved a new video-compressionstandard known as High Efficien...
ConceptsthatcreateefficiencyOne of the primary target areas forHEVC compression is high-­resolutionvideo, like HD and UHD....
units of unequal width or height, asillustratedinFigure 3.The size of the prediction blocks inHEVCcanthereforevaryfrom4×4s...
in parallel. Tiles and wavefront paral-lel processing (WPP) are among severalHEVC tools that can provide high-levelparalle...
targets most consumer devices andservices.However,formorespecializedapplications, such as 3D, content pro-duction or heter...
Per Fröjdhis director of mediastandardization atEricsson and former headof visual technology atEricsson Research. He holds...
Telefonaktiebolaget LM EricssonSE-164 83 Stockholm, SwedenPhone: + 46 10 719 0000Fax: +46 8 522 915 99284 23-3198 | UenISS...
Upcoming SlideShare
Loading in...5

Next generation video compression


Published on

Requiring only half the bitrate of its predecessor, the new standard – HEVC or H.265 – will significantly reduce the need for bandwidth and expensive, limited spectrum. HEVC (H.265) will enable the launch of new video services and in particular ultra HD television (UHDTV).

State-of-the-art video compression techniques – HEVC/H.265 – can reduce the size of raw video by a factor of about 100 without any noticeable reduction in visual quality. With estimates indicating that compressed real-time video accounts for more than 50 percent of current network traffic, and this figure is set to rise to 90 percent within a few years, HEVC/H.265 will be a welcome relief for network operators.

New services, devices and changing viewing patterns are among the factors contributing to the growth in video traffic as people watch more and more traditional TV and video-streaming services on their mobile devices.

Ericsson has been heavily involved in the standardization of HEVC since it began in 2010, and this Ericsson Review article highlights some of the contributions that have led to the compression efficiency offered by HEVC.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Transcript of "Next generation video compression"

  1. 1. The communications technology journal since 1924 2013 • 6Next generationvideo compressionApril 24, 2013
  2. 2. Next generationvideo compressionMPEG and ITU have recently approved a new video-compressionstandard known as High Efficiency Video Coding (HEVC), or H.265, thatis set to provide double the capacity of today’s leading standards1.compression efficiency to date, and iscurrently the most widely used video-compression codec. It has been success-fully incorporated into most mobiledevices, and is the best way to reducethe size of video carried over the inter-net.ItisthepreferredformatforBlu-raydiscs, telepresence streams and, mostnotably,HDTV.Now imagine a codec that is twiceas efficient as H.264. This was the tar-get set by MPEG and ITU in 2010, whenthey embarked on a joint standardiza-tion effort that three years later deliv-eredHEVC/H.2654,5.The new codec offers a much moreefficient level of compression thanits predecessor H.264, and is particu-larly suited to higher-resolution videostreams,wherebandwidthsavingswithHEVC are around 50 percent. In simpleterms,HEVCenablesanetworktodeliv-er twice the number of TV channels.ComparedwithMPEG-2,HEVCcanpro-videuptofourtimesthecapacityonthesamenetwork.Like most standards, the MPEG andITU video codecs have been developedin a collaborative fashion involvingmany stakeholders – manufacturers,operators, broadcasters, vendors andacademics. Ericsson has been an activeparticipant in video standardizationfor more than 15 years, and was closelyinvolvedinHEVC.Throughout the development of thestandard, Ericsson has led several ofthe core experiments, chaired ad-hocworkinggroupsandcontributedsignif-icantly to the development of the tech-nology behind the codec. Our greatestexpertiseliesintheareasofthedeblock-ing filter6and in reference this growth, as is increasedviewing of traditional TV andvideo-streaming services, suchas Netflix, YouTube and Hulu, ona range of devices – from phonesand tablets to PCs and home-entertainment systems. As HDshifts from luxury to commodity,it will soon be challenged by UHD,which offers resolutions up to 16times greater.MakingstandardsMostvideoviewedbysubscriberstodayhas been digitized and reduced insize through the application of a com-pression standard. The more popularinclude the H.26x series from ITU andthe MPEG-x series from ISO/IEC. Firstpublished in 1994, the MPEG-2 stan-dard, also known as H.262, played acrucial role in the launch of digital-TVservices as it enabled the compressionof TV streams to fit the spectrum avail-able. This is also the standard used tocompressmoviesontoaDVD.TheH 264standard (also known as MPEG-4 AVC),publishedin2003,hasprovidedthebestPER FRÖJDH, ANDREY NORKIN AND RICKARD SJÖBERGBOX A Terms and abbreviationsAVC advanced video codingCABAC context-adaptive binary arithmetic coderCTU coding-tree unitCU coding unitfps frames per secondHD high definition; often refers to 1280 x 720 or 1920 x 1080 pixelsHEVC High Efficiency Video CodingIEC International Electrotechnical CommissionISO International Organization for StandardizationITU International Telecommunication UnionMPEG Moving Picture Experts GroupOTT over-the-topSAO sample adaptive offsetUHD ultra high definition: often refers to 3840 x 2160 (4K) or 7680 x 4320 (8K) pixelsWPP wavefront parallel processingRequiring only half the bitrateof its predecessor, the newstandard will significantly reducethe need for bandwidth andexpensive, limited spectrum.HEVC will enable new videoservices to be launched, and thefirst applications that are likely toappear will be for mobile devicesand OTT applications, followedby TV – and in particular ultra HDtelevision (UHDTV).State-of-the-art videocompression can reduce thesize of raw video by a factorof about 100 without anynoticeable reduction in visualquality. Estimates indicatethat compressed real-timevideo accounts for more than50 percent of current networktraffic2, and this figure is set torise to 90 percent within a fewyears3. New services, devices andchanging viewing patterns areamong the factors contributing2ERICSSON REVIEW • APRIL 24, 2013Same bandwidth, double the data
  3. 3. ConceptsthatcreateefficiencyOne of the primary target areas forHEVC compression is high-­resolutionvideo, like HD and UHD. The statisticalcharacteristics of such video streamstend to be different from lower-­resolution content: frame sizes arelarger, and frame rates and perceivedquality are higher – imposing toughrequirements on compression efficien-cy,aswellasonthecomputationalcom-plexity of the encoding and decodingprocesses.As smartphone and tablet architec-tures go multi-core, the ability to takeadvantage of parallel processing is keywhen it comes to the efficient compres-sion of high-resolution content. All ofthese points have been taken into con-sideration during the development ofthenewstandard.The hybrid block-based coding usedby the new codec is the same as theone used in earlier video-coding stan-dards. To encode content, video framesare divided into blocks that are codedindividually by applying prediction –based either on neighboring blocks inthe same picture (intra prediction) orfrompreviouslycodedpictures(motionestimation/compensation). The dif-ference between the predicted resultand original video data is subsequent-ly coded by applying block transformsand quantization. In this way, a blockcan be represented by just a few non-zero coefficients. Quantized transformcoefficients,motionvectors,predictiondirections,blockmodesandothertypesofinformationareencodedwithlosslessentropycoding.Hybridblock-basedcod-ingisillustratedinFigure 1.To ensure the highest level of com-pressionefficiency,andsupportforpar-allel processing, some parts of HEVChave been significantly modified com-pared with previous generations ofhybrid block-based codecs. For most ofthepreviousMPEG-xandH.26xcodecs,the largest entity that could be inde-pendently encoded was a macroblock(16 × 16 pixels). For HEVC, the pictureis split into coding-tree units (CTUs)with a maximum size of 64 × 64 pix-els. Every CTU is the root of a quadtree,which can be further divided into leaf-levelcodingunits(CUs),asillustratedinFigure 2. TheCTUsarecodedinrasterscan order, and each unit can itselfFIGURE 1 Simplified HEVC encoder diagramInput signalIntra-predictionIn-loopfiltersEntropycodingDecodedpicture bufferMotioncompensationMotionestimationTransform coefficientsMotion vectorsT QQ-1T-1FIGURE 2 Example of the coding-tree unit structure in HEVCCTU structureCTUCUCUCU3ERICSSON REVIEW • APRIL 24, 2013
  4. 4. units of unequal width or height, asillustratedinFigure 3.The size of the prediction blocks inHEVCcanthereforevaryfrom4×4sam-plesupto64×64,whiletransformsizesvaryfrom4×4to32×32samples.Largeprediction blocks and transform sizesare the most efficient way to encodelarge smooth areas, whereas smallerprediction blocks and transforms canbe used to achieve precision in areasthatcontainfinerdetail.The HEVC specification covers moreintra-prediction modes than H.264,including a planar mode to approx-imate a surface from neighboringpixels, a flat mode and 33 angular pre-diction modes. Motion-compensatedprediction for luma transform blocksis performed with up to quarter-pixelprecision, whereas motion compensa-tionforcolorcomponentsisperformedwith one-eighth-of-a-pixel precision.Interpolation for fractional pixel posi-tions uses 8-tap filters for luma blocksand4-tapfiltersforcolor.In HEVC there is a single entropycoder for low-level data. This is the con-text-adaptive binary arithmetic coder(CABAC), which is similar to the oneusedinH.264,butmodifiedtofacilitateparallel processing. Higher-level infor-mation, such as sequence parameters,isencodedwithvariable-lengthorfixed-lengthencoding.HEVC defines two in-loop filters: adeblockingfilterandasampleadaptiveoffset(SAO)filter.Thelatterisappliedtothe output of the deblocking filter, andincreases the quality of reference pic-tures by applying transmitted offsetsto samples that fulfill certain criteria.In-loop filters improve the subjectivequality of reconstructed video as wellas compression efficiency. Deblockingfiltering in HEVC is less complex thanthat of H.264, as it is constrained to an8×8blockgrid.Thisconstraint,togeth-er with filtering decisions and opera-tionsthatarenon-overlappingbetweentwo boundaries, simplifies multi-coreprocessing.ParallelprocessingTo make the most of the increasinglywidespread use of multi-core proces-sors, plus the ever-growing number ofcores used in consumer-class proces-sors, significant attention was paid tothe parallelization characteristics of­video encoding and decoding whendesigning HEVC. As it is computation-allymorecomplexthanitspredecessor,maximizing parallelization has beena key factor in making HEVC an effi-cient real-time encoding and decodingsolution.Several HEVC tools have beendesigned for easy parallelization. Thedeblocking filter can be applied to 8 × 8pixel blocks separately, and transform-coefficient-coding contexts for severalcoefficient positions can be processedcontain a quadtree structure. EachCUcontainsoneormorepredictionpar-titions that are predicted independent-ly of each other. A CU is also associatedwith a transform quadtree that com-presses the prediction residual and hasa structure similar to that of a CTU – asshowninFigure2.Partitions for motion prediction canform square or rectangular shapes,which is also the case with earlier stan-dards. HEVC also supports somethingcalledasymmetricmotionpartitioning,which can split the CU into predictionAsymmetric motion partitions are shown in the bottom row.Only square partitions are allowed for intra predictionFIGURE 3 Possible motion prediction partitions in HEVCCTUFIGURE 4 Multi-thread decoding with wavefronts. Gray areasindicate CTUs that have already been decoded4ERICSSON REVIEW • APRIL 24, 2013Same bandwidth, double the data
  5. 5. in parallel. Tiles and wavefront paral-lel processing (WPP) are among severalHEVC tools that can provide high-levelparallelism.The concept behind WPP is to re-ini-tialize CABAC at the beginning of eachline of CTUs. To facilitate CABAC adap-tationtothecontentofthevideoframe,the coder is initialized once the statis-tics from the decoding of the secondCTU in the previous row are available.Re-initializationofthecoderatthestartof each row makes it possible to begindecoding a row before the processingoftheprecedingrowhasbeencomplet-ed. Thus, as shown in the example inFigure 4, several rows can be decod-ed in parallel in several threads with adelayoftwoCTUsbetweentwoconsec-utiverows.The Tiles tool can be used for paral-lel encoding and decoding, and worksby dividing a picture into rectangularareas (tiles) – as shown in Figure  5 –where each tile consists of an integernumber of CTUs. The CTUs are pro-cessed in a raster scan order withineach tile, and the tiles themselves areprocessed in the same way. Predictionbased on neighboring tiles is disabled,andsotheprocessingofeachtileisinde-pendent. In-loop filters, however, canoperate over tile boundaries. And asdeblockingandSAOcanbeparallelized,filtering can be performed indepen-dently inside each tile, and tile bound-aries can be processed by in-loop filtersinafinalpass.TheHEVCstandardthereforeenablesbothhigh-andlow-levelparallelization,which can provide significant benefitsfor multi-thread encoding and decod-ingofvideosuchas4Kand8KthathasahigherresolutionthanHD.PerformanceandcomplexityThe improved coding efficiency ofHEVC does however come with a pricetag: increasedcomputationalcomplex-ity. Compared with its predecessor,HEVC is 50 -100 percent more complexfordecodingandupto400percentmorecomplex when it comes to encoding.While these comparisons are based onpreliminary tests, they do give an indi-cation of the new codec’s computation-alcomplexity.Real-time implementations of HEVCdemonstrate that decoding of full HD(1080p) at 50 or 60fps is possible on fastdesktopandlaptopcomputers,runningon a single core. Performance increas-es with multiple core implementations(hardware acceleration), so that a mod-ern smartphone is capable of 1080pdecodingat25or30fps8.ApplicationsThe new standard is a general one suit-able for the compression of all kindsof video. The focus for the first versionis consumer applications and for this,threeprofileshavebeendefined:Main,Main10andMainStillPicture.Main is an all-purpose profile witha depth of 8 bits per pixel, supporting4:2:0–themostcommonuncompressedvideoformatusedbyconsumerdevicesfrommobilephonestoHDTVs.Main10extendsthebitdepthto10bitsperpixel,which is well suited to consumer appli-cations, such as UHDTV, where veryhigh quality is critical. The increasedbit depth can compress wide ­dynamicrange video without creating band-ing artifacts, which sometimes occurswith 8 bits. The third profile, Main StillPicture,usedforstillimages,isasubsetofMainandcarriesasinglestillpictureatadepthof8bitsperpixel.The initial deployments of HEVCreleased in 2013 will be for mobilesand OTT applications. Softwareimplementations capable of decod-ing HEVC without hardware accel-eration can easily be downloaded tosmartphones,tabletsandPCs,enablingmobileTV,streaminganddownloadser-vices on existing devices. To this end,in August 2012, Ericsson announcedSVP 55009, the world’s first HEVC real-time video encoder for live-TV deliveryto mobile devices. However, as it is bet-ter to perform encoding on hardwareand as HEVC is computationally moredemanding than previous standards, itmaybesometimebeforevideotelepho-nybasedonthisstandardentersmobileplatforms, whereas encoding on PCs isalreadyfeasible.Set-topboxeswithnewdecoderswillbecome available soon, enabling con-tent broadcast via satellite, cable or ter-restrially to take advantage of HEVC.The new standard plays a key role inthe provision of UHDTV, and as pricesdrop and displays become affordable,the number of services utilizing suchhigh resolutions is expected to risewithin a few years. Flat-panel displaysfor HDTV have been on the market foralmost a decade, so this may be a goodtime for consumers to start upgradingtoUHDTV.What’scomingThe finalized version of HEVCFIGURE 5 Example of the way an image can be divided into tilesCTUCTUCTUColumn boundariesRow boundariestile 1 tile 2 tile 3tile 4 tile 5 tile 6tile 7 tile 8 tile 95ERICSSON REVIEW • APRIL 24, 2013
  6. 6. targets most consumer devices andservices.However,formorespecializedapplications, such as 3D, content pro-duction or heterogeneous devices andnetworks,someadditionstoHEVCmayprove useful. With this in mind, MPEGandITUareworkingtogetheronanum-berofideas,includingsupportforstereoand multi-view (glasses-free) 3D video,an extension that encodes multipleviews by rearranging picture buffersand reference picture lists. A first dropisexpectedinJanuary2014,withamoreadvancedversionthatwillsupportjointencodingoftextureanddepthinforma-tioncomingintheearlypartof2015.Scalability is a key attribute of anycodec, as it enables trimming of videostreams to suit different network con-ditions and receiver capabilities; scal-able extensions to HEVC are plannedfor July 2014. Range extensions, whichsupport several color formats as well asincreased bit depths, are another areacurrentlyunderdevelopment.In addition to these extensions, fur-therimprovementsareexpectedtotakeplace inside the current HEVC frame-work, such as more efficient encodingand decoding (both software and hard-ware). It is likely that the full potentialof HEVC will take some time to unfold,asencodingalgorithmsdevelopandthechallenge posed by the optimization ofencoders and decoders in multi-corearchitecturesisovercome.In short, HEVC or H.265 is twice asefficient as its 10-year-old predecessor,H.264.Theimprovedefficiencythatthiscodecbringswillhelptoeasetrafficloadin networks and enable the creation ofnewandadvancedvideo-basedservices.The codec supports parallel process-ingandeventhoughitismorecomplexfromadecodingperspective,testshaveshown that it is suitable for adoption inmobileservices.Compressionofmobilevideo streams and OTT content are themostlikelyinitialcandidatesforapplica-tionofthecodec,andwithinafewyearsit will undoubtedly bring UHDTV intoourhomes.6ERICSSON REVIEW • APRIL 24, 2013Same bandwidth, double the data
  7. 7. Per Fröjdhis director of mediastandardization atEricsson and former headof visual technology atEricsson Research. He holds an ­ engineering physics and a Ph.D. intheoretical physics from ChalmersUniversity in Gothenburg, Sweden.Part of his Ph.D. work was carried outon scholarship at Imperial CollegeLondon, UK. Following postdoctoralappointments in the US, and Denmark,he held the position of professor oftheoretical physics at StockholmUniversity, Sweden. He joined Ericssonin 2000 as manager of video researchand standardization. He hascontributed to MPEG and ITU work onH.264 and HEVC, served on theadvisory committee for the W3C, andhas been the editor of 15 standards onstreaming, file formats, andmultimedia telephony in MPEG, ITU,3GPP and IETF.Andrey Norkinis a senior researcher atEricsson Research, Kista,Sweden. He holds anM.Sc. in computerengineering from Ural State TechnicalUniversity, Yekaterinburg, Russia and aPh.D. in signal processing from theTampere University of Technology, inFinland. He has worked at EricssonResearch since 2008, contributing toHEVC standardization throughtechnical proposals and activities,including the coordination of a coreexperiment on deblocking filtering,chairing break-out groups andsubjective quality tests for the JointCollaborative Team on Video Coding(JCT-VC). He has also been active inthe 3D video standardization for JCT-3V. He is currently the project managerof the 3D VISION project at EricssonResearch, working on 3D videosystems, and algorithms, as well as onparts of the standardization.Rickard Sjöbergis a senior specialist invideo coding in theMultimedia Technologiesdepartment at EricssonResearch, Kista, Sweden. With an M.Sc. in computer science from the KTHRoyal Institute of Technology,Stockholm, Sweden, he has beenworking with Ericsson since 1997 andhas worked in various areas related tovideo coding, in both research andproduct development. In parallel, hehas been an active contributor in thevideo-coding standardizationcommunity, with more than 100proposals relating to the H.264 andHEVC video-coding standards. He iscurrently working as the technicalleader of Ericsson’s 2D video-codingresearch, including HEVC and itsscalable extensions. His researchinterests include video compressionand real-time multimedia processingand coding.1. ITU, January 2013, press release, New videocodec to ease pressure on global networks,available at: Ericsson, November 2012, Mobility Report,available at: Fierce Broadband Wireless, 2013,Ericsson CEO: 90% of network trafficwill be video, available at: ITU-T Recommendation H.265 | ISO/IEC23008-2: High Efficiency Video Coding,available at: IEEE, December 2012, Overview of the HighEfficiency Video Coding (HEVC) Standard,available at: IEEE, December 2012, HEVC DeblockingFilter, available at: IEEE, December 2012, Overview ofHEVC High-Level Syntax and ReferencePicture Management, available at: IEEE, DOCOMO Innovations, December2012, HEVC Complexity and ImplementationAnalysis, available at: Ericsson, 2012, Ericsson announcesworld’s first HEVC encoder for live TVdelivery to mobile devices, available at: REVIEW • APRIL 24, 2013
  8. 8. Telefonaktiebolaget LM EricssonSE-164 83 Stockholm, SwedenPhone: + 46 10 719 0000Fax: +46 8 522 915 99284 23-3198 | UenISSN 0014-0171© Ericsson AB 2013