Performance	  analysis	  of	  VP8	  image	  and	  video	  compression	  based	  on	  subjec8ve	  evalua8ons	              ...
Outline	  •  Introduc@on	  •  Codecs	  and	  configura@ons	  •  Subjec@ve	  quality	  evalua@on	     – MMSPG	  test	  envir...
INTRODUCTION	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	  Based	 ...
Introduc@on	  •  Trends	  clearly	  indicate	  that	  the	  video	  consump@on	  over	  the	  Web	  is	     on	   the	   r...
CODECS	  AND	  CONFIGURATIONS	   	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  ...
Image	  codecs	  and	  configura@ons	  •  WebP	     – Intra	  frame	  coding	  of	  VP8	  ini@ally	  developed	  by	  On2	 ...
Image	  codecs	  and	  configura@ons	  •  JPEG	  2000	     – Wavelet	  based	  image	  compression	  standard	  with	  impr...
Video	  codecs	  and	  configura@ons	  •  WebM	    – VP8	  video	  codec	  ini@ally	  developed	  by	  On2	  and	  recently...
Video	  codecs	  and	  configura@ons	  •  HEVC	     – Recently	  started	  ini@a@ve	  of	  JCT-­‐VC	  to	  develop	  more	 ...
SUBJECTIVE	  QUALITY	  EVALUATION	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  ...
MMSPG	  test	  environment	  •  3	   high	   quality	   LCD	   monitors	   (Eizo	     CG301W)	     –  na@ve	  resolu@on	  ...
Image	  dataset	  •  Use	  8	  images	  from	  the	  JPEG	  XR	  Evalua@on	  Dataset	  •  Resolu@on	  of	  1280x1600	  pix...
Video	  dataset	  •  Use	  5	  video	  from	  the	  VQEG	  HDTV	  SVT	  Dataset	  •  Various	  resolu@ons	  available	  (2...
Subjec@ve	  image	  quality	  evalua@on	  •  Adopted	   methodology	   which	   was	     recently	  developed	  within	  t...
Subjec@ve	  video	  quality	  evalua@on	  •  Double	  s@mulus	  impairment	  scale	  (DSIS)	     [ITU	  BT.500-­‐11]	  •  ...
Sessions	  and	  subjects	  •  Training	  session	  for	  naïve	  subjects	     – Oral	  instruc@ons	  to	  explain	  the	...
Score	  processing	  and	  analysis	  •  Adopted	  established	  procedure	  described	  in	  [DeSimone2011]	  •  Outlier	...
RESULTS	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	  Based	      ...
Image	  coding	  RDP	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	 ...
Image	  coding	  RDP	  (bike)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Comp...
Image	  coding	  RDP	  (p01)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compr...
Image	  coding	  RDP	  (p06)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compr...
Image	  coding	  RDP	  (p10)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compr...
Image	  coding	  RDP	  (p14)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compr...
Image	  coding	  RDP	  (woman)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Com...
Image	  coding	  MCA	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	 ...
Image	  coding	  MCA	  (0.125	  bpp)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Image	  coding	  MCA	  (0.250	  bpp)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Image	  coding	  MCA	  (0.500	  bpp)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Image	  coding	  MCA	  (0.750	  bpp)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Image	  coding	  MCA	  (1.000	  bpp)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Bpp	  0.25	                                                                                                          Origi...
Bpp	  0.5	                                                                                                          Origin...
Bpp	  0.25	                                                                                                          Origi...
Bpp	  0.5	                                                                                                          Origin...
Bpp	  0.25	                                                                                                          Origi...
Bpp	  0.5	                                                                                                          Origin...
Video	  coding	  RDP	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	 ...
Video	  coding	  RDP	  (CrowdRun)	  SPIE	  Op@cs+Photonics	      Performance	  Analysis	  of	  VP8	  Image	  and	  Video	 ...
Video	  coding	  RDP	  (DucksTakeOff)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video...
Video	  coding	  RDP	  (InToTree)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  ...
Video	  coding	  RDP	  (OldTownCross)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Vide...
Video	  coding	  MCA	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	 ...
Video	  coding	  MCA	  (250	  kbps)	  SPIE	  Op@cs+Photonics	      Performance	  Analysis	  of	  VP8	  Image	  and	  Video...
Video	  coding	  MCA	  (500	  kbps)	  SPIE	  Op@cs+Photonics	      Performance	  Analysis	  of	  VP8	  Image	  and	  Video...
Video	  coding	  MCA	  (750	  kbps)	  SPIE	  Op@cs+Photonics	      Performance	  Analysis	  of	  VP8	  Image	  and	  Video...
Video	  coding	  MCA	  (1250	  kbps)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Video	  coding	  MCA	  (2250	  kbps)	  SPIE	  Op@cs+Photonics	       Performance	  Analysis	  of	  VP8	  Image	  and	  Vid...
Video	  coding	  RS	  (CrowdRun	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  an...
Video	  coding	  RS	  (CrowdRun	  500	  kbps)	  NOTE:	  for	  this	  content	  webM	  did	  not	  reach	  the	  target	  b...
Video	  coding	  RS	  (CrowdRun	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  an...
Video	  coding	  RS	  (CrowdRun	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  an...
Video	  coding	  RS	  (CrowdRun	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  an...
Video	  coding	  RS	  (CrowdRun	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  a...
Video	  coding	  RS	  (CrowdRun	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  a...
Video	  coding	  RS	  (CrowdRun	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  a...
Video	  coding	  RS	  (CrowdRun	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  a...
Video	  coding	  RS	  (CrowdRun	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  a...
Video	  coding	  RS	  (CrowdRun	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  a...
Video	  coding	  RS	  (DucksTakeOff	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	 ...
Video	  coding	  RS	  (DucksTakeOff	  500	  kbps)	  NOTE:	  for	  this	  content	  webM	  did	  not	  reach	  the	  target	...
Video	  coding	  RS	  (DucksTakeOff	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	 ...
Video	  coding	  RS	  (DucksTakeOff	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	 ...
Video	  coding	  RS	  (DucksTakeOff	  500	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	 ...
Video	  coding	  RS	  (DucksTakeOff	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	...
Video	  coding	  RS	  (DucksTakeOff	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	...
Video	  coding	  RS	  (DucksTakeOff	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	...
Video	  coding	  RS	  (DucksTakeOff	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	...
Video	  coding	  RS	  (DucksTakeOff	  1250	  kbps)	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	...
CONCLUSION	  SPIE	  Op@cs+Photonics	     Performance	  Analysis	  of	  VP8	  Image	  and	  Video	  Compression	  Based	   ...
Conclusion	  •  VP8	  image	  compression	  showed	  performance	  comparable	  to	  JPEG	     XR	   and	   JPEG	   2000,	...
References	  •  De	   Simone,	   F.,	   Goldmann,	   Baroncini	   V.,	   and	   Ebrahimi,	   T.,	     “Subjec@ve	   evalua...
Thank	  you	  for	  your	  aeen@on!	     Visit	  us	  at:	  hep://mmspg.epfl.ch	                               	  
Upcoming SlideShare
Loading in...5
×

Rate distortion performance of VP8 (WebP and WebM) when compared to standard image and video compression techniques

5,743

Published on

These are the slides of my presentation at SPIE Optics and Photonics 2011, August 2011, San Diego comparing rate distortion performance of VP8 (WebP and WebM) to major image and video compression standards from subjective evaluation point of view.

1 Comment
8 Likes
Statistics
Notes
No Downloads
Views
Total Views
5,743
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
1
Likes
8
Embeds 0
No embeds

No notes for slide
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Performance analysis of WebP and VP8 based on subjective evaluations Paper 8135-20 Time: 11:20 AM - 11:40 AMAuthor(s): Francesca De Simone, Jong-Seok Lee, Lutz Goldmann, TouradjEbrahimi, Ecole Polytechnique Fédérale de Lausanne (Switzerland)
  • Transcript of "Rate distortion performance of VP8 (WebP and WebM) when compared to standard image and video compression techniques"

    1. 1. Performance  analysis  of  VP8  image  and  video  compression  based  on  subjec8ve  evalua8ons   Francesca  De  Simone,  Lutz  Goldmann,   Jong-­‐Seok  Lee,  Touradj  Ebrahimi   Mul@media  Signal  Processing  Group,   Ecole  Polythechnique  Fédérale  de  Lausanne,  Lausanne     SPIE  Op@cs+Photonics  2011   Applica@ons  of  Digital  Image  Processing  
    2. 2. Outline  •  Introduc@on  •  Codecs  and  configura@ons  •  Subjec@ve  quality  evalua@on   – MMSPG  test  environment   – Datasets   – Test  methodology   – Score  processing  and  analysis  •  Results   – Rate  distor@on  plots  (RDP)   – Mul@ple  comparison  analysis  (MCA)   – Visual  samples  •  Conclusions   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   2   2011   on  Subjec@ve  Evalua@ons  
    3. 3. INTRODUCTION  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   3  2011   on  Subjec@ve  Evalua@ons  
    4. 4. Introduc@on  •  Trends  clearly  indicate  that  the  video  consump@on  over  the  Web  is   on   the   rise.   At   the   same   @me,   users   demand   for   increased   resolu@on  and  higher  quality  is  growing.  •  Alterna@ves   for   compression   of   digital   pictures   and   video   sequences:   – interna@onally  recognized  standard  solu@ons   – VP8  open  access  image  and  video  compression  •  Our   study:   methodology   and   results   of   the   rate-­‐distor@on   performance   analysis   of   VP8   based   on   subjec@ve   quality   assessment  experiments   – comparison   of   VP8   to   JPEG,   JPEG   2000,   JPEG   XR   for   4:2:0   image   compression,   to  H.264/AVC  and  HEVC  for  4:2:0  video  compression   – first  study  in  the  state  of  the  art  on  formal  subjec@ve  quality  evalua@on  of  VP8     SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   4   2011   on  Subjec@ve  Evalua@ons  
    5. 5. CODECS  AND  CONFIGURATIONS    SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   5  2011   on  Subjec@ve  Evalua@ons  
    6. 6. Image  codecs  and  configura@ons  •  WebP   – Intra  frame  coding  of  VP8  ini@ally  developed  by  On2   – Use  of  Google  implementa@on  0.1.2  (hep://code.google.com/speed/webp)   – Two  configura@ons  were  considered:  default,  photo   – Varying  quality  factor  to  reach  different  target  bitrates  •  JPEG   – Block  based  image  compression  standard  developed  in  1992   – Use  of  the  IJG  implementa@on  8c  (hep://ijg.org/)   – Coded  using  baseline  profile  and  by  varying  the  quality  factor   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   6   2011   on  Subjec@ve  Evalua@ons  
    7. 7. Image  codecs  and  configura@ons  •  JPEG  2000   – Wavelet  based  image  compression  standard  with  improved  func@onality  and   coding  efficiency   – Use  of  the  Kakadu  implementa@on  6.4.1  (hep://www.kakadusojware.com/)   – Encoding  in  YUV  4:2:0  format  with  necessary  color  conversion  and  sampling   – Rate  control  op@on  to  obtain  target  bitrates  •  JPEG  XR   – Latest  image  compression  standard  published  in  2009   – Considered  two  implementa@ons   •  JPEG  XR  reference  sojware  1.20   •  Implementa@on  provided  by  Microsoj   – Varying  quality  factor  to  reach  target  bitrates   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   7   2011   on  Subjec@ve  Evalua@ons  
    8. 8. Video  codecs  and  configura@ons  •  WebM   – VP8  video  codec  ini@ally  developed  by  On2  and  recently  released  by  Google   – Use  of  libvpx  implementa@on  0.9.6  (hep://www.webmproject.org/)   – Use  of  best  configura@on  from  recent  MSU  codec  benchmark   – Contant  bitrate  (CBR)  mode  to  achieve  target  bitrates  •  H.264/AVC   – Very  popular  video  coding  standard  completed  by  JVT  in  2003   – Considered  two  implementa@ons   •  x264  implementa@on  r2019  (hep://www.videolan.org/developers/x264.html)   •  JM  reference  sojware  18.0  (hep://iphome.hhi.de/suehring/tml/)   – Use  of  one  configura@on  for  each  implementa@on   •  X264:  best  performing  configura@on  from  recent  MSU  codec  benchmark  and   constant  bitrate  (CBR)  mode  to  achieve  target  bitrates   •  JM:  alpha  anchor  configura@on  corresponding  to  random  access  constraint  and   varying  QP  to  achieve  target  bitrates   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   8   2011   on  Subjec@ve  Evalua@ons  
    9. 9. Video  codecs  and  configura@ons  •  HEVC   – Recently  started  ini@a@ve  of  JCT-­‐VC  to  develop  more  efficient  video  coding   standard  with  extended  and  new  coding  tools   – HM  sojware  (hep://hevc.hhi.fraunhofer.de/svn/svn_HEVCSojware/)  v3.2   – Use  of  the  high  efficiency  configura@on  sa@sfying  the  random  access  scenario   – Varying  QP  to  achieve  target  bitrates   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   9   2011   on  Subjec@ve  Evalua@ons  
    10. 10. SUBJECTIVE  QUALITY  EVALUATION  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   10  2011   on  Subjec@ve  Evalua@ons  
    11. 11. MMSPG  test  environment  •  3   high   quality   LCD   monitors   (Eizo   CG301W)   –  na@ve  resolu@on  of  2560x1600  pixels   –  gray-­‐to-­‐gray  response  @me  of  6  ms   –  black-­‐white-­‐black  response  @me  of  12  ms   –  calibrated  using  an  EyeOne  Display2  color   calibra@on  device  •  1   high   performance   video   server   with   SSD  •  controlled   ligh@ng   system   (neon   lamps  with  6500  K  color  temperature)  •  mid   gray   background   walls   and   curtains  •  S a m e   e n v i r o n m e n t   u s e d   f o r   evalua@on   and   selec@on   of   best   performing   proposals   submieed   for   H E V C   s t a n d a r d i z a @ o n   [DeSimone2011]   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   11   2011   on  Subjec@ve  Evalua@ons  
    12. 12. Image  dataset  •  Use  8  images  from  the  JPEG  XR  Evalua@on  Dataset  •  Resolu@on  of  1280x1600  pixels,  4:4:4  RGB  format  •  2  images  (p30,  cafe)  for  training  and  6  images  for  tes@ng  •  Coded  using  4  different  codecs  and  6  overall  codec  configura@ons  •  5  different  bitrates  (bpp):  0.125,  0.250,  0.500,  0.750,  1.000  •  A  few  codec  configura@ons  (JPEG,  VP8)  did  not  reach  the  lower  bitrates   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   12   2011   on  Subjec@ve  Evalua@ons  
    13. 13. Video  dataset  •  Use  5  video  from  the  VQEG  HDTV  SVT  Dataset  •  Various  resolu@ons  available  (2160p50,  1080p50,  1080i25,  etc.)  •  Spa@al  and  temporal  subsampling  of  1080p50  to  854x480@25  •  1  video  (ParkJoy)  for  training    and  4  videos  for  tes@ng  •  Coded  using  4  different  codecs  and  4  overall  codec  configura@ons  •  5  different  bitrates  (kbps):  250,  500,  750,  1250,  2250  •  A  few  codec  configura@ons  (VP8)  did  not  reach  the  lower  bitrates   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   13   2011   on  Subjec@ve  Evalua@ons  
    14. 14. Subjec@ve  image  quality  evalua@on  •  Adopted   methodology   which   was   recently  developed  within  the  context  of   the  JPEG  XR  evalua@on  [DeSimone2009]  •  Modified   double   s@mulus   con@nuous   quality  scale  (DSCQS)  [ITU  BT.500-­‐11]  •  Implicit   reference   (SRC)   and   compressed   i m a g e   ( H R C )   a r e   p r e s e n t e d   simultaneously  •  The  quality  of  both  s@muli  is  rated  on  a   con@nuous   quality   scale   from   0   to   100   with   5   quality   levels   (bad,   poor,   fair,   good,  excellent)   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   14   2011   on  Subjec@ve  Evalua@ons  
    15. 15. Subjec@ve  video  quality  evalua@on  •  Double  s@mulus  impairment  scale  (DSIS)   [ITU  BT.500-­‐11]  •  Explicit  reference  (SRC)  and  compressed   video  (HRC)  are  presented  sequen@ally  •  Amount   of   ar@facts   with   respect   to   the   reference  is  rated  on  a  con@nuous  scale   from   0   to   100   with   5   impairment   levels   (very   annoying,   annoying,   slightly   annoying,  percep@ble,  impercep@ble)   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   15   2011   on  Subjec@ve  Evalua@ons  
    16. 16. Sessions  and  subjects  •  Training  session  for  naïve  subjects   – Oral  instruc@ons  to  explain  the  subjec@ve  evalua@on  task   – Training  session  to  explain  methodologies  to  the  subjects   – Training  samples  for  each  quality/impairment  level  selected  by  expert  viewer  •  Number  of  image  and  videos  samples  to  large  for  a  single  session   – Image:  4  sessions  of  14  minutes  each   – Video:  3  sessions  of  13  minutes  each   – 10  minutes  break  for  a  subject  between  two  sessions  •  3  dummy  pairs  and  1  repe@@on  for  stability  and  reliability  check  •  18   different   subjects   with   an   average   age   of   24   years   for   both   image  (5  women,  11  men)  and  video  (9  women,  9  men)   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   16   2011   on  Subjec@ve  Evalua@ons  
    17. 17. Score  processing  and  analysis  •  Adopted  established  procedure  described  in  [DeSimone2011]  •  Outlier  detec@on  for  each  session  in  order  to  discard  inconsistent   subjects  (no  outliers  detected  for  both  image  and  video)  •  Computa@on  of  sta@s@cal  measures  for  each  combina@on  of   content,  coding  condi@on  and  bitrate   – Mean  opinion  scores  (MOS)  for  both  DSIS  and  DSCQS   – 95%  confidence  intervals  (CI)    using  Student’s  t-­‐distribu@on  •  Rate  distor@on  plots  (RDP)  for  each  content  across  all  bitrates  •  Mul@ple  comparison  analysis  (MCA)  to  iden@fy  sta@s@cal   significant  differences  among  different  codecs  for  the  same  bitrate   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   17   2011   on  Subjec@ve  Evalua@ons  
    18. 18. RESULTS  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   18  2011   on  Subjec@ve  Evalua@ons  
    19. 19. Image  coding  RDP  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   19  2011   on  Subjec@ve  Evalua@ons  
    20. 20. Image  coding  RDP  (bike)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   20  2011   on  Subjec@ve  Evalua@ons  
    21. 21. Image  coding  RDP  (p01)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   21  2011   on  Subjec@ve  Evalua@ons  
    22. 22. Image  coding  RDP  (p06)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   22  2011   on  Subjec@ve  Evalua@ons  
    23. 23. Image  coding  RDP  (p10)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   23  2011   on  Subjec@ve  Evalua@ons  
    24. 24. Image  coding  RDP  (p14)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   24  2011   on  Subjec@ve  Evalua@ons  
    25. 25. Image  coding  RDP  (woman)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   25  2011   on  Subjec@ve  Evalua@ons  
    26. 26. Image  coding  MCA  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   26  2011   on  Subjec@ve  Evalua@ons  
    27. 27. Image  coding  MCA  (0.125  bpp)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   27  2011   on  Subjec@ve  Evalua@ons  
    28. 28. Image  coding  MCA  (0.250  bpp)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   28  2011   on  Subjec@ve  Evalua@ons  
    29. 29. Image  coding  MCA  (0.500  bpp)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   29  2011   on  Subjec@ve  Evalua@ons  
    30. 30. Image  coding  MCA  (0.750  bpp)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   30  2011   on  Subjec@ve  Evalua@ons  
    31. 31. Image  coding  MCA  (1.000  bpp)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   31  2011   on  Subjec@ve  Evalua@ons  
    32. 32. Bpp  0.25   Original   JPEG   JPEG2k   JPEG-­‐XR   VP8   (MS)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   32  2011   on  Subjec@ve  Evalua@ons  
    33. 33. Bpp  0.5   Original   JPEG   JPEG2k   JPEG-­‐XR   VP8   (MS)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   33  2011   on  Subjec@ve  Evalua@ons  
    34. 34. Bpp  0.25   Original   JPEG   JPEG2k   JPEG-­‐XR   VP8   (MS)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   34  2011   on  Subjec@ve  Evalua@ons  
    35. 35. Bpp  0.5   Original   JPEG   JPEG2k   JPEG-­‐XR   VP8   (MS)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   35  2011   on  Subjec@ve  Evalua@ons  
    36. 36. Bpp  0.25   Original   JPEG   JPEG2k   JPEG-­‐XR   VP8   (MS)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   36  2011   on  Subjec@ve  Evalua@ons  
    37. 37. Bpp  0.5   Original   JPEG   JPEG2k   JPEG-­‐XR   VP8   (MS)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   37  2011   on  Subjec@ve  Evalua@ons  
    38. 38. Video  coding  RDP  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   38  2011   on  Subjec@ve  Evalua@ons  
    39. 39. Video  coding  RDP  (CrowdRun)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   39  2011   on  Subjec@ve  Evalua@ons  
    40. 40. Video  coding  RDP  (DucksTakeOff)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   40  2011   on  Subjec@ve  Evalua@ons  
    41. 41. Video  coding  RDP  (InToTree)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   41  2011   on  Subjec@ve  Evalua@ons  
    42. 42. Video  coding  RDP  (OldTownCross)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   42  2011   on  Subjec@ve  Evalua@ons  
    43. 43. Video  coding  MCA  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   43  2011   on  Subjec@ve  Evalua@ons  
    44. 44. Video  coding  MCA  (250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   44  2011   on  Subjec@ve  Evalua@ons  
    45. 45. Video  coding  MCA  (500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   45  2011   on  Subjec@ve  Evalua@ons  
    46. 46. Video  coding  MCA  (750  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   46  2011   on  Subjec@ve  Evalua@ons  
    47. 47. Video  coding  MCA  (1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   47  2011   on  Subjec@ve  Evalua@ons  
    48. 48. Video  coding  MCA  (2250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   48  2011   on  Subjec@ve  Evalua@ons  
    49. 49. Video  coding  RS  (CrowdRun  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   49  2011   on  Subjec@ve  Evalua@ons  
    50. 50. Video  coding  RS  (CrowdRun  500  kbps)  NOTE:  for  this  content  webM  did  not  reach  the  target  bitrate  of  500  kbps  so  this  frame  has  been  extracted  from  the  webM  sample  with  bitrate  closest  to  500  kbps  (see  slide  39)   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   50   2011   on  Subjec@ve  Evalua@ons  
    51. 51. Video  coding  RS  (CrowdRun  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   51  2011   on  Subjec@ve  Evalua@ons  
    52. 52. Video  coding  RS  (CrowdRun  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   52  2011   on  Subjec@ve  Evalua@ons  
    53. 53. Video  coding  RS  (CrowdRun  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   53  2011   on  Subjec@ve  Evalua@ons  
    54. 54. Video  coding  RS  (CrowdRun  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   54  2011   on  Subjec@ve  Evalua@ons  
    55. 55. Video  coding  RS  (CrowdRun  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   55  2011   on  Subjec@ve  Evalua@ons  
    56. 56. Video  coding  RS  (CrowdRun  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   56  2011   on  Subjec@ve  Evalua@ons  
    57. 57. Video  coding  RS  (CrowdRun  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   57  2011   on  Subjec@ve  Evalua@ons  
    58. 58. Video  coding  RS  (CrowdRun  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   58  2011   on  Subjec@ve  Evalua@ons  
    59. 59. Video  coding  RS  (CrowdRun  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   59  2011   on  Subjec@ve  Evalua@ons  
    60. 60. Video  coding  RS  (DucksTakeOff  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   60  2011   on  Subjec@ve  Evalua@ons  
    61. 61. Video  coding  RS  (DucksTakeOff  500  kbps)  NOTE:  for  this  content  webM  did  not  reach  the  target  bitrate  of  500  kbps  so  this  frame  has  been  extracted  from  the  webM  sample  with  bitrate  closest  to  500  kbps  (see  slide  40)   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   61   2011   on  Subjec@ve  Evalua@ons  
    62. 62. Video  coding  RS  (DucksTakeOff  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   62  2011   on  Subjec@ve  Evalua@ons  
    63. 63. Video  coding  RS  (DucksTakeOff  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   63  2011   on  Subjec@ve  Evalua@ons  
    64. 64. Video  coding  RS  (DucksTakeOff  500  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   64  2011   on  Subjec@ve  Evalua@ons  
    65. 65. Video  coding  RS  (DucksTakeOff  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   65  2011   on  Subjec@ve  Evalua@ons  
    66. 66. Video  coding  RS  (DucksTakeOff  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   66  2011   on  Subjec@ve  Evalua@ons  
    67. 67. Video  coding  RS  (DucksTakeOff  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   67  2011   on  Subjec@ve  Evalua@ons  
    68. 68. Video  coding  RS  (DucksTakeOff  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   68  2011   on  Subjec@ve  Evalua@ons  
    69. 69. Video  coding  RS  (DucksTakeOff  1250  kbps)  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   69  2011   on  Subjec@ve  Evalua@ons  
    70. 70. CONCLUSION  SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   70  2011   on  Subjec@ve  Evalua@ons  
    71. 71. Conclusion  •  VP8  image  compression  showed  performance  comparable  to  JPEG   XR   and   JPEG   2000,   all   significantly   outperforming   JPEG   compression.    •  VP8   video   compression   showed   performance   compe@@ve   with   x264,   while   the   new   HEVC   technology   under   defini@on   usually   showed  the  best  performance.  •  For  some  contents,  both  for  image  and  video  compression,  it  could   be  no@ced  that  the  current  implementa@on  of  VP8  seemed  to  have   difficul@es  reaching  low  bit  rates  condi@ons  that  most  other  codecs   reached.  •  Future   studies   will   consider   other   resolu@ons   of   the   same   test   material,  as  well  as  other  subjec@ve  evalua@on  methodologies.   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   71   2011   on  Subjec@ve  Evalua@ons  
    72. 72. References  •  De   Simone,   F.,   Goldmann,   Baroncini   V.,   and   Ebrahimi,   T.,   “Subjec@ve   evalua@on   of   JPEG   XR   image   compression,"   SPIE   7443   (2009).  •  De  Simone,  F.,  Goldmann,  L.,  Lee,  J.  S.,  and  Ebrahimi,  T.,  “Towards   High   Efficiency   Video   Coding:   Subjec@ve   Evalua@on   of   Poten@al   Coding  Technologies,"  Journal  of  Visual  Communica@on  and  Image   Representa@on  (2011).   SPIE  Op@cs+Photonics   Performance  Analysis  of  VP8  Image  and  Video  Compression  Based   72   2011   on  Subjec@ve  Evalua@ons  
    73. 73. Thank  you  for  your  aeen@on!   Visit  us  at:  hep://mmspg.epfl.ch    

    ×