Video has become the dominant driver of internet traffic globally, taking up the majority of bandwidth on most networks around the world. Network congestion during popular viewing times frequently manifests itself as a degradation in video streaming quality (stalling, shifts in resolution, pixelization, changes in frame rate and so on) which frustrates and annoys viewers. To overcome this challenge, mobile network operators and media service providers urgently require solutions that can provide an accurate estimation of video QoE. Full implementation of the P.1203 standard would be a significant step in the right direction.
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Ericsson Technology Review: Video QoE: leveraging standards to meet rising user expectations
1. ERICSSON
TECHNOLOGY
C H A R T I N G T H E F U T U R E O F I N N O V A T I O N | # 6 ∙ 2 0 1 7
LEVERAGING
STANDARDSFOR
VIDEO QoE
2. ✱ VIDEO QUALITY OF EXPERIENCE
2 ERICSSON TECHNOLOGY REVIEW ✱ JUNE 27, 2017
GUNNAR HEIKKILÄ,
JÖRGEN GUSTAFSSON
In 2016, Ericsson ConsumerLab found that
the average person watches 90 minutes
more TV and video every day than they did
in 2012 [1]. While traditionally broadcast
linear TV is still popular with many viewers,
internet-based TV and video delivery and
video on demand (VOD) services are all
growing rapidly. In fact, 20 percent of video
consumption occurred on handheld devices
in 2016 [1].
■Asusersbecomemoreaccustomedtovideo
streamingservices,theirqualityexpectations
rise,presentingabigchallengeformediaservice
providers(MSPs)andmobilenetworkoperators
(MNOs).Whiledeliveringhigh-qualityvideo
overafixedconnectioncanbedifficult,doingso
wirelesslyisamuchmoredemandingundertaking.
Yetby2022,75percentofallmobiledatatrafficis
expectedtocomefromvideo,accordingtothe2016
EricssonMobilityReport[2].
At the same time, TV screens are getting
larger, which requires higher video resolution.
High definition (HD) is now the new baseline,
and ultra high definition (UHD) is coming to
both fixed and mobile devices, demanding higher
bandwidth. To provide a consistently high QoE
– particularly in wireless cases with significant
fluctuations in available bandwidth – both MSPs
and MNOs must have a clear understanding of
which impairments their users are experiencing
and be able to accurately assess their perception
of video quality.
‘How happy are our users with their video experience?’ has become a vital
question for mobile network operators and media service providers alike.
New standards for QoE testing have the potential to help them ensure that
they are able to meet user expectations for the service that will account for
three-quarters of mobile network traffic in five years’ time.
VideoQoELEVERAGING STANDARDS
TO MEET RISING USER EXPECTATIONS
3. VIDEO QUALITY OF EXPERIENCE ✱
3JUNE 27, 2017 ✱ ERICSSON TECHNOLOGY REVIEW
AdaptiveHTTP-basedstreaming
AdaptiveHTTP-basedvideostreamingvariants
suchasDASHandHLSarethedominantvideo
deliverymethodusedtoday.Theirstreaming
serversprovideseveralversionsofthesamevideo,
eachseparatelyencodedtooffervaryinglevelsof
quality.Thestreamingclientintheuser’sphone,
tabletorPCdynamicallyswitchesbetweenthe
differentversionsduringplayoutdependingonhow
muchnetworkbitrateisavailable.
Iftheavailablenetworkbitrateishigh,theclient
willdownloadthebest-qualityversion.Ifthebitrate
suddenlydrops,theclientwillswitchtoalower-
qualityversionuntilconditionsimprove.The
purposeofthisistoavoidstalling(whentheclient
stopsplayoutintermittentlytofillitsvideobuffer)–
whichisknowntoannoyusers.
Whiletheabilitytoswitchintermittentlybetween
versionsofavideosignificantlydecreasestheriskof
stalling,thequalityvariationsthatthisleadstocan
alsobeannoyingfortheuser.Figure1providesan
exampleofaworst-casescenariowithbothquality
variationsandastallingevent,whichwouldresult
inanoveralllow-qualityexperiencefortheuser.
Subjectivequality
ITU-T P.910 [3] is the recognized standard for
performing subjective video quality tests. The
tests are carried out in a lab equipped with
mobile phones, tablets, PCs or TVs, where
a number of videos are shown to a group of
individuals. Each individual then grades each
video according to their subjective perception of
its quality, selecting one of the following scores:
5 (excellent), 4 (very good), 3 (good), 2 (fair) or 1
(poor). Finally, the average score for each video
is calculated. This number is known as the mean
opinion score (MOS).
The video sequences are typically produced
in a lab environment so that all types of
impairments can be included. High-quality
sports, nature and news videos are used as a
starting point. Impairments (codec settings,
rate and resolution changes, initial buffering,
stalling and so on) are then emulated by varying
the bitrate over a certain range, for example,
or placing stalling events of different lengths at
various points in the videos.
Thetestsaredesignedtomakeanalysisas
accurateandstraightforwardaspossible.Devising
subjectivetestsisatimeconsumingandexpensive
process,though,andlabtestscan’tassessexactly
whatanMNO’srealusersareexperiencing.The
bestwaytoovercomethesechallengesisbyusing
objectivequalityalgorithms.
THE TESTS ARE
DESIGNED TO MAKE
ANALYSIS AS ACCURATE
AND STRAIGHTFORWARD
AS POSSIBLE
Terms and abbreviations
APN–AccessPointName|AVC–advancedvideocoding|DASH–DynamicAdaptiveStreamingoverHTTP|
DM–devicemanagement|eNB–eNodeB(LTEbasestation)|ETSI–EuropeanTelecommunicationsStandards
Institute|HD–highdefinition|HEVC–HighEfficiencyVideoCoding|HLS–HTTPLiveStreaming|HTTP–Hypertext
TransferProtocol|ITU-T–InternationalTelecommunicationUnionTelecommunicationStandardizationSector|
MNO–mobilenetworkoperator|MOS–meanopinionscore|MPD–mediapresentationdescription|MSP–media
serviceprovider|OMA–OpenMobileAlliance|Pa–short-termaudiopredictor|Pq–long-termqualitypredictor|
Pv–short-termvideopredictor|QoE–qualityofexperience|RRC–RadioResourceControl|SMS–shortmessage
service|UHD–ultrahighdefinition|VOD–videoondemand|VP9–Anopen-sourcevideoformatandcodec
4. ✱ VIDEO QUALITY OF EXPERIENCE
4 ERICSSON TECHNOLOGY REVIEW ✱ JUNE 27, 2017
Figure 1 A 60-second video with quality variations (resolution) and a three-second stall in the middle
5. VIDEO QUALITY OF EXPERIENCE ✱
5JUNE 27, 2017 ✱ ERICSSON TECHNOLOGY REVIEW
Objectivequality
Asthewordingsuggests,objectivequality
algorithms(alsoknownasobjectivemodels)are
designedtomimicthebehaviorandperceptionof
humans.Thegoalistoproducethesamescoresas
theMOSvaluesthatwouldresultfromrunninga
subjectivetestonthesamevideos.
Manydifferenttypesofobjectivemodelscan
beadopted,dependingontheintendedusage
andthekindofinputdataemployed.Thoseusing
themostlimitedsetofinputparametersbasethe
objectivequalityestimationonencodingrates,
videoresolution,framerates,codecsandstalling
information,asthesefactorsprovidetheminimum
amountofinformationaboutthevideoplayout
thatisrequiredtoestimateaqualityscore.More
complexmodelsmightusethecompleteencoded
videobitstream,oreventhefullreceivedvideo
signal,tofurtherincreasetheestimationaccuracy.
Theobjectivemodelsdescribedaboveare
no-referencemodels,whereinputistakenonly
fromthereceivingendofthemediadistribution
chain.Full-referencemodelscanalsobeadopted,
wherethevideooriginallytransmittedis
comparedwiththeonethatisreceived.Another
variantisthereduced-referencemodel,where
theoriginalvideoisnotneededforreference,but
certaininformationaboutitismadeavailableto
themodel.
Traditionally,objectivemodelsareusedto
evaluatequalitybasedonrelativelyshortvideo
sequences:approximately10secondslong.
However,withadaptivevideostreaming,where
qualitycanvarysignificantlyduringagiven
session,themodelmustalsoassesshowthislong-
termvariationaffectsuserperception.Todothis,
modelevaluationofmuchlongervideosequences
(uptoseveralminutes)isrequired.
Ideally,differentlevelsofcomplexityshouldbe
usablebyasinglemodel–thatis,fromonlyafew
inputparametersuptothefullbitstream,depending
ondeployment.Thisisthescopethatapplieswith
thenewno-referenceITU-TP.1203standard.
StandardizationofITU-TP.1203
TheP.1203standard[4]wasdevelopedaspart
ofanITU-Tcompetitionbetweenparticipating
proponents(Ericssonandsixothercompanies),
whereeachonesentinitsproposedquality
models.Themodelsthatperformedbestwere
thenusedasabaselinetocreatethefinalstandard.
Aninternalmodelarchitecturewasalsodefinedto
facilitatethecreationofamodelthatwouldbeas
flexibleaspossible.
Architecture
Thestandardincludesmodulesforestimatingshort-
termaudioandvideoquality,andanintegration
moduleestimatingthefinalsessionqualitydueto
adaptationandstalling,asshowninFigure2.
Theshort-termvideoandaudiopredictor(Pv
andPa)modulescontinuouslyestimatetheshort-
termaudioandvideoqualityscoresforone-second
piecesofcontent.Thismeansthatfora60-second
video,therewillbe60audioscoresand60video
scores.ThesePvandPamodulesarespecificto
eachtypeofcodec.
ThePvandPamodulesoperateinuptofour
differentmodes,dependingonhowdetailedthe
inputisfromtheparameterextraction.Forthe
leastcomplexmode,themaininputsarerelatedto
resolution,bitrateandframerate,whilethemost
complexmodeperformsadvancedanalysisofthe
videopayload.
Theshort-termscoresfromthosemodules
arefedintothelong-termqualitypredictor(Pq)
module,togetherwithanystallinginformation,
andthefinalsessionqualityscoreforthetotal
videosessionisthenestimated.ThePqmodule
alsoproducesanumberofdiagnosticoutputs,
sothattheunderlyingcausesofthescorecanbe
analyzed.ThePqmoduleisnotmode-orcodec-
dependentandisthereforecommonforallcases.
IDEALLY, DIFFERENT
LEVELS OF COMPLEXITY
SHOULD BE USABLE BY A
SINGLE MODEL
6. ✱ VIDEO QUALITY OF EXPERIENCE
6 ERICSSON TECHNOLOGY REVIEW ✱ JUNE 27, 2017
Figure 2 ITU-T P.1203 architecture
8. ✱ VIDEO QUALITY OF EXPERIENCE
8 ERICSSON TECHNOLOGY REVIEW ✱ JUNE 27, 2017
Implementingaqualitymodel
Successfulimplementationofaqualitymodelis
dependentonaccesstotheinputdatarequiredby
themodelitself.Themostdemandingmodels,such
asfull-referencevariants,areusuallyimplemented
closetothevideostreamingclient,insidethedevice,
sothatthecompletereceivedvideocanbecompared
withtheonesent.Thisistypicallydoneformanual
testingscenarios,whereaspecialtestphoneisused
inwhichaqualitymodelhasbeenimplemented.
Thismethodisnotfeasibleforpassive
measurements,wherealloralargepartoflivevideo
trafficismonitored,somodelinputdataneedsto
becollectedinanotherway.Forexample,anMNO
thatwantstogainabetterunderstandingofits
overallperceivedvideostreamingqualitywould
needtocollectdatafromallstreamingsessions.
Onewayofdoingthiswouldbetointerceptthe
trafficatcertainnetworknodesandusethetraffic
contentandpatterntotrytoinferwhichserviceis
beingusedandthequalitylevelbeingdelivered.
Thiscanbedifficult,however,owingtothefact
thatmanyservicesarenowencrypted,which
significantlylimitsaccesstothedatarequiredtodo
adetailedqualityestimation.
Onewaytoovercomethischallengeiswiththe
helpofthestreamingclient,whichhasfullknowledge
ofwhatishappeningduringthevideosession.A
feedbacklinkfromthestreamingclientcanbeused
toreportselectedmetricstothenetwork(orthe
originalstreamingserver)wherethequalitycanbe
estimated.Thistechniqueisalreadyusedinternally
formanystreamingservices.Forexample,whena
userclicksonavideolinkontheinternet,theclient
typicallycontinuouslymeasuresdifferentmetrics
andsendsthemtotheserver.Unfortunatelyfor
MNOs,though,thesefeedbackchannelsareusually
encryptedandavailableonlytotheMSP.Evenif
anMSPweretomakethemetricsavailabletothe
MNO,theywouldstillbeproprietary,anditwould
bedifficultfortheMNOtocomparethemduetothe
factthatthecontentandlevelofdetailtypicallydiffer
betweenMSPs.
3GPPQoEreporting
Theusefulnessofhavingastandardizedclient
feedbackmechanismhasbeenrecognizedby3GPP,
anditstechnicalspecificationTS26.247describes
howthiscanbeimplementedforaDASHstreaming
client[5].The3GPPconceptiscalledQoEreporting,
asthemetricscollectedandreportedarerelated
specificallytothequalityofthesession.Sensitiveor
integrity-relateddatasuchastheuser’spositionand
thecontentviewedcannotbereported.
Thebasicconceptisthatthestreamingclient
canreceiveaQoEconfigurationthatspecifiesthe
metricstobecollected,howoftencollectionwill
occur,whenreportingwillbedoneandwhichentity
toreportto.TherearethreewaystosendaQoE
configurationtotheclienttofacilitatedifferent
deploymentcases:
1.Mediapresentationdescription
Inthiscase,theclientdownloadsamedia
presentationdescription(MPD)whenstreaming
starts.TheMPDspecifieshowthemediais
structuredandhowtheclientcanaccessand
downloadthemediachunks.TheMPDcanalso
containaQoEconfigurationthatmakesitpossible
togetfeedbackfromtheclient.SincetheMPD
isusuallycontrolledbythecontentorservice
provider,theQoEreportsfromtheclientare
typicallyconfiguredtogobacktotheirserversand
arenotalwaysvisibletotheMNO.
2.OpenMobileAllianceDeviceManagement
OpenMobileAllianceDeviceManagement
(OMA-DM)hasdefinedmethodsforhowan
MNOcanconfigurecertainaspectsofconnected
devicessuchasAPNs,SMSserversandsoon.
ThesemethodsalsoincludeanoptionalQoE
THE USEFULNESS OF
HAVING A STANDARDIZED
CLIENT FEEDBACK
MECHANISM HAS BEEN
RECOGNIZED BY 3GPP
9. VIDEO QUALITY OF EXPERIENCE ✱
9JUNE 27, 2017 ✱ ERICSSON TECHNOLOGY REVIEW
Figure 3 Example of a video QoE microservice
10. ✱ VIDEO QUALITY OF EXPERIENCE
10 ERICSSON TECHNOLOGY REVIEW ✱ JUNE 27, 2017
configurationthatactivatesQoEreportingfrom
theclient.However,notallMNOsdeployOMA-
DMintheirnetworks.
3.RadioResourceControl
TheRadioResourceControl(RRC)protocol[6]
isusedbetweentheeNBandthemobiledevice
tocontrolthecommunicationintheRAN.The
possibilityofincludingaQoEconfigurationwas
addedin3GPPrel-14,givingMNOstheabilityto
useRRCtoactivateQoEreporting.Asaresult,
theQoEconfigurationcanbehandledlikemany
othertypesofRAN-relatedconfigurationsand
measurements.
TheQoEmetricsreportedbytheclientineach
ofthesethreemethodsarewellalignedwiththe
inputrequirementsinP.1203.Thismeansthat,
inprinciple,anyofthemwouldenableanMNO
togainagoodoverviewofthevideostreaming
qualityexperiencedbyitsusers.Beforethiscan
happen,though,thenewstandardswillneedto
bedeployedbothinthenetworkandintheclients.
DeploymentofvideoQoEestimation
Thedeliveryofover-the-topvideotodaymost
commonlyusesacloud-basedmicroservices
platformwithinbuiltvideo-qualityestimation
features.TheP.1203algorithmforestimatingvideo
qualityismostsuitableforimplementationasa
microservicethatestimatesvideoquality(interms
ofMOS)forallvideostreamsandforallindividual
videostreamingsessions.Iftheestimatedvideo
qualitydistributionshowsthatthereisaquality
issue,arootcauseanalysiscanbecarriedoutandthe
necessarymeasurescanbetakentoimprovequality.
Whenrunningamediaservice,andproviding
thevideoclientaspartoftheservice,theinput
parameterstotheP.1203algorithmaretakendirectly
fromthevideoclientthathasallthedetailsabout
theplayoutofthestreamandreportedoverthe
networktoananalyticsbackend.Figure3outlines
anexampleofsuchanimplementation,inwhich
playereventsarereportedtoastreamprocessing
system(Kafka)wherethevideoQoEiscalculated
inavideoQoEmicroservice.Theoutputfromthe
videoQoEcalculationisthenpostedbackintothe
streamprocessingsystemtobeusedformonitoring,
visualization,rootcauseanalysisandotherpurposes.
ForanMNO,thestandardarchitectureisto
installaprobeinsidethenetwork–inthecore
network,forexample.Theprobemonitorsvideo
trafficusingshallowordeeppacketinspection
andgathersinformationthatcanbeusedasinput
toaQoEestimationalgorithm(P.1203).Ifavideo
serviceisnotencrypted,therelevantmetrics
andeventsfromthevideostreamscanoftenbe
measuredorestimated.Thetaskbecomesmore
challengingifthevideostreamsareencrypted,but
qualityestimationsofthesevideostreamscanstill
bedonetosomeextentbyusingacombinationof
probesandstandardizedandproprietaryalgorithms
andmodels.However,theabilitytoreportquality-
relatedmetricsdirectfromthevideoclientsenables
muchmoreaccurateestimationofquality.
Conclusion
MNOsandMSPsstandtogainagreatdealfrom
developingabetterunderstandingofhowusers
experience video quality, and the ITU-T P.1203
and 3GPP TS 26.247 standards provides the
framework that is necessary to help them do so.
Implementation of these standards by MNOs,
MSPs and device manufacturers will enable
the efficient and accurate estimation of video
QoE required to meet continuously rising user
expectations.Thestandard’ssmarthandlingof
theeffectsofstallingeventsandqualityadaptations
makesitwellsuitedtoovercomethechallenges
presentedbyVODandbyadaptivevideo
streaminginparticular.
THE INPUT PARA-
METERS TO THE P.1203
ALGORITHM ARE TAKEN
DIRECTLY FROM THE
VIDEO CLIENT
11. VIDEO QUALITY OF EXPERIENCE ✱
11JUNE 27, 2017 ✱ ERICSSON TECHNOLOGY REVIEW
Gunnar Heikkilä
◆ is a senior specialist in
machine intelligence at
Ericsson Research. Since
1996, he has focused
on user experience
quality assessment and
measurements, including
standardization in 3GPP,
ETSI and ITU-T. He joined
Ericsson in 1987 and has
previously worked with
control system software
for military defense radar
systems, and with software
design for synchronous
digital hierarchy optical
fiber transmission systems.
Heikkilä holds an M.Sc.
in computer science
from Luleå University of
Technology, Sweden.
Jörgen Gustafsson
◆ is a research manager at
Ericsson Research, heading
a research team in the areas
of machine learning and
QoE. The research is applied
to a number of areas, such as
media, operations support
systems/business support
systems, the Internet of
Things and more. He joined
Ericsson in 1993. He is
co-rapporteurofQuestion
14 in ITU-T Study Group
12, where leading and
global standards on
parametric models and
toolsformultimediaquality
assessment are being
developed, including the
latest standards on quality
assessment of adaptive
streaming.HeholdsanM.Sc.
in computer science from
LinköpingUniversity,Sweden.
theauthors
1. EricssonConsumerLabreport,TVandMedia2016,availableat:https://www.ericsson.com/networked-
society/trends-and-insights/consumerlab/consumer-insights/reports/tv-and-media-2016
2. EricssonMobilityReport,November2016,availableat:https://www.ericsson.com/en/mobility-report
3. ITU-TP.910,April2008,Subjectivevideoqualityassessmentmethodsformultimediaapplications,availableat:
http://www.itu.int/itu-t/recommendations/rec.aspx?rec=9317
4. ITU-TP.1203,November2016,Parametricbitstream-basedqualityassessmentofprogressive
downloadandadaptiveaudiovisualstreamingservicesoverreliabletransport,availableat:
http://www.itu.int/itu-t/recommendations/rec.aspx?rec=13158
5. 3GPP TS 26.247, January 2015, Transparent end-to-end Packet-switched Streaming Service (PSS);
Progressive Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH), available at:
http://www.3gpp.org/DynaReport/26247.htm
6. 3GPP TS 25.331, January 2016, Radio Resource Control (RRC); Protocol specification, available at:
http://www.3gpp.org/DynaReport/25331.htm
References: