Cloud Services for Improved User Experience in Sharing Mobile Videos


Published on

Despite the popularity of mobile video sharing, mobile user experience (UX) is not comparable with traditional TV or desktop video productions. The issue of poor UX in mobile video sharing can be associated with the high development cost, since the creation and utilization of a multimedia processing and distribution infrastructure is a non-trivial task for small groups of developers. In this paper, we present our solution comprised of mobile video processing services based on standard libraries which augment the raw video streams. Our services utilize the cloud computing paradigm for fast and intelligent processing in near-real time. Video streams are split in chunks and then fed to the "resource-unlimited" distributed/cloud infrastructure which accelerate the processing phase. Application developers have the possibility to apply arbitrary computer vision algorithms on the video stream thus improving the quality of user experience depending on the application requirements. We providing navigation cues and content-based zooming of raw video streams. We evaluated the proposed solution from two perspectives - distributed chunk-based processing in the cloud and a user study by means of mental workload. Running experiments in mobile video applications demonstrate that our proposed techniques improve mobile user experience significantly.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cloud Services for Improved User Experience in Sharing Mobile Videos

  1. 1. Cloud Services for Improved User Experience in Sharing Mobile Videos Dejan Kovachev, Yiwei Cao & Ralf Klamma RWTH Aachen University Advanced Community Information Systems (ACIS) kovachev@dbis.rwth-aachen.deLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-1 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
  2. 2. Advanced Community Information Systems (ACIS) Web Engineering Responsive Community Web Analytics Open Visualization Community and Information Simulation Systems Community Community Support AnalyticsLehrstuhl Informatik 5 Requirements(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-2 Engineering
  3. 3. Agenda  Motivation  Background and related work  Conceptual approach  System design and implementation  Evaluation  Conclusions and outlookLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-3
  4. 4. Motivation  Mobile video is rapidly increasing – Video accounts more than half of all global mobile data traffic (Cisco Visual Network Index, 2011) – Video-based sharing of life experiences in near real time anywhere anytime  Mobile user experience (MEX) is still poor – Significant challenges must be addressed in the areas of quality, user experience and cost of delivery (IEEE, 2012) – Technical issues - mobile video quality of service – Mobile networks constantly fluctuate in bandwidth, delay, jitter, and packet loss – Presentation issues - perceived experience from user – Mobile devices are not optimized for streaming of high-quality content (HD, Super Hi-Vision) – Amateur video content shot with smartphones lacks the aesthetics of professional videos – Difficult video navigation browsingLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-4
  5. 5. How can we improve the MEX?  Intelligent video processing - Content-aware video retargeting to overcome small screen size problem  Object recognition, feature extraction, segmentation, indexing, bitrate adaptation - Processing issues, specialized software tools,  Cloud ComputingLehrstuhl Informatik 5(Information Systems) - “Unlimited” processing power and storage Prof. Dr. M. Jarke I5-KCKl-0313-5 - Real-time video processing by parallel processing files (Perreira et al., 2010)
  6. 6. Related Work (2)  Cloud video processing – MapReduce-based solutions – Content mixing (pictures, video) [Sandholm 2011] – Video format and bitrate transcoding [Garcia et al. 2010, Pereira et al. 2010] – Feature detection [Chen and Schlosser 2008] – Pay-per-use cloud resources to improve stream quality [Trajkovska et al. 2009]  Adaptive streaming – Chunk streaming [Mazzola Paluska and Pham 2010] – MPEG-DASH standard [Vetro 2011]Lehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-6
  7. 7. Related Work  Mobile User Experience – Definition: “a person’s perceptions and responses that result from the use or anticipated use of a product, system or service” (ISO, 2010)  MEX enhancements – Optimal zoom ratio [Knoche et al. 2007, Song et al. 2010] – Region of interest enhancement [Knoche et al. 2007] – Segment-based video navigation [Bursuc et al. 2010] – Annotation-based navigation [Bentley and Groble 2009] – Stream personalization [Patrikakis 2011] – Mixing video streams into one view [Kaheel et al. 2009] – Video image stabilizationLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-7
  8. 8. MVCS Workflow  MVCS – Mobile video cloud servicesLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-8
  9. 9. Mobile Client UI Camera Activities login browse videos Main Activity Video Player Activity without login Login Activity Browse Activities Main ActivityLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke Preferences Activity I5-KCKl-0313-9
  10. 10. MEX EnhancementLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-10
  11. 11. MVCS Prototype – Mobile Client  Implemented for Android SDK 2.2+ MVCS Client  Libraries used: Smack, FFmpeg, Apache GUI Video Recorder Video Player mime, Apache HTTP Tags Metadata & Segments Overlay  Lazy load to save resources Video List Preferences  Technical issues Handlers – Mobile upstreaming a big trend but no off- Lazy Load Handler MP4 Handler the-shelf solutions Metadata Handler Sync Handler – Only basic RTP streaming support in Android – Sipdroid, FFmpeg (NDK, C++), RTP socket Communication Layer XMPP Connector (aSmack) – No official Smack (XMPP) library for Android RTP Stream Client – Video player and camera activity are File Transfer proprietaryLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-11
  12. 12. MVCS Prototype – Cloud Services  Implemented in Java using Smack, OpenCV, FFmpeg, Apache HTTP, x264 and shotdetect libraries  Object recognition using Haar classifiers (Messom et al., 2006) and JavaCV (OpenCV)  Scene detection using histogram differences, fixed threshold and FFmpeg  Zooming realized by cropping Intelligent Video Processing Services Segmentation Parallel processing of video chunks Transcoding Recognition  (OpenCV) (FFmpeg) (FFmpeg) (FFmpeg, Zooming Object x264) – Splitting and merging video files into chunks (keyframes have to be identified) Video Metadata Service ServiceLehrstuhl Informatik 5 XMPP Service (Smack)(Information Systems) Prof. Dr. M. Jarke MVCS Cloud I5-KCKl-0313-12
  13. 13. i5Cloud TestbedLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke [Kovachev et al. 2012] I5-KCKl-0313-13
  14. 14. Chunk Processing of Videos  Inspired by Split & Merge approach (Perreira et al., 2010)  Video is split into chunks at keyframe (I-frame)  Chunks are processed on multiple instances and should speed up processing of videosLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-14
  15. 15. Evaluation  Subjects – 3 representative types of videos: sport, talk, and documentary – 12 Users: CS background, between 23 and 54 years old, mixed experience – Questionnaire: NASA-TLX (Mental Demand, Frustration, Physical Demand, Effort, Performance, Temporal Demand) (NASA, 1986) – NASA-TLX is a subjective workload assessment tool  Zooming Evaluation – User has to understand the content – NASA-TLX has to be answered  Browsing Evaluation – User has to find a certain position – NASA-TLX has to be answeredLehrstuhl Informatik 5  Questionnaire filled out in the end(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-15
  16. 16. MEX Evaluation Results  Video zooming reduces workload by 26% to 49%  Video browsing reduces workload by 64% to 67%  Zooming and browsing good approaches to reduce workload and to improve MEXLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-16
  17. 17. Cloud Services Performance Evaluation - Setup  Subjects – Videos with different length (1:03 mins, 2:59 mins, 5:21 mins) – H.264 video codec, AAC audio codec, 30fps  Procedure – Each video as a single file processed by all intelligent video services – Each video as chunks processed by all intelligent video services – Segmentation service – Zooming service – Comparison of one file solution to chunk based solution – Number of instances is increased proportional to video length – Simple video transcoding – Processing time is recordedLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-17
  18. 18. Cloud Evaluation Results  Chunk approach enables faster video processing  Parallel processing of chunks results in fast intelligent video processing services 02:53 02:36 02:18 Processing Time in Minutes 02:01 Single Transcoding 01:44 Chunk Transcoding 01:26 Single Scene 01:09 Chunk Scene 00:52 Single Zooming Chunk Zooming 00:35 00:17 00:00 01:03 02:59 05:21 Dura on in MinutesLehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-18
  19. 19. Conclusions  Mobile Video Cloud Services (MVCS) combine different approaches and algorithms to deliver fast intelligent video processing services for a better MEX – Improved mobile browsing by segmentation and tags – Zoomed videos do overcome screen size problems  Cloud services – Better utilization of cloud environment by splitting videos into chunks – Complex tasks like object movement offload to the cloud  Future Work – More user studies in different environments – Automatic annotation of videos with metadata – Hardware acceleration using GPULehrstuhl Informatik 5 – Extension with robust and fast computer vision algorithms(Information Systems) Prof. Dr. M. Jarke – Adaptive HTTP live streaming I5-KCKl-0313-19
  20. 20. Thanks for your attention! Q&ALehrstuhl Informatik 5(Information Systems) Prof. Dr. M. Jarke I5-KCKl-0313-20
  21. 21. References  IEEE P2200. IEEE Standard Protocol for Stream Management in Media Client Devices  IEEE P1907.1. Standard for Network-Adaptive Quality of Experience (QoE) Management Scheme for Real-Time Mobile Video Communications  R. Pereira, M. Azambuja, K. Breitman, and M. Endler. An Architecture for Distributed High Performance Video Processing in the Cloud. In CloudCom, 2010  ISO FDIS 9241-210:2010. Ergonomics of human system interaction - Part 210, 2010  W. Song, D. W. Tjondronegoro, S.-H. Wang, and M. J. Docherty. Impact of Zooming and Enhancing Region of Interests for Optimizing User Experience on Mobile Sports Video. In ACM Multimedia, 2010  H. Knoche, M. Papaleo, M. A. Sasse, and A. Vanelli-Coralli. The Kindest Cut: Enhancing the User Experience of Mobile TV Through Adequate Zooming. In ACM Multimedia, 2007  A. Bursuc, T. Zaharia, and F. Prêteux. Mobile Video Browsing and Retrieval with the OVIDIUS Platform. In ACM Multimedia, 2010  A. Kaheel, M. El-Saban, M. Refaat, and M. Ezz. Mobicast: A System for Collaborative Event Casting Using Mobile Phones. In MUM, 2009  F. R. Bentley and M. Groble. TuVista: Meeting the Multimedia Needs of Mobile Sports Fans. In ACM Multimedia, 2009  P. Patrikakis, N. Papaoulakis, C. Stefanoudaki, A. Voulodimos, and E. Sardis. Handling Multiple Channel Video Data for Personalized Multimedia Services: A Case Study on Soccer Games Viewing. In PerCom, 2011  T. Sandholm. HP Labs Cloud-Computing Test Bed: VideoToon Demo, 2011  A. Garcia, H. Kalva, and B. Furht. A Study of Transcoding on Cloud Environments for Video Content Delivery. In ACM Multimedia, 2010Lehrstuhl Informatik 5  S. Chen and S. W. Schlosser, “Map-Reduce Meets Wider Varieties of Applications,” Intel Labs Pittsburgh Tech Report,(Information Systems) Prof. Dr. M. Jarke May 2008 I5-KCKl-0313-21