Despite the popularity of mobile video sharing, mobile user experience (UX) is not comparable with traditional TV or desktop video productions. The issue of poor UX in mobile video sharing can be associated with the high development cost, since the creation and utilization of a multimedia processing and distribution infrastructure is a non-trivial task for small groups of developers. In this paper, we present our solution comprised of mobile video processing services based on standard libraries which augment the raw video streams. Our services utilize the cloud computing paradigm for fast and intelligent processing in near-real time. Video streams are split in chunks and then fed to the "resource-unlimited" distributed/cloud infrastructure which accelerate the processing phase. Application developers have the possibility to apply arbitrary computer vision algorithms on the video stream thus improving the quality of user experience depending on the application requirements. We providing navigation cues and content-based zooming of raw video streams. We evaluated the proposed solution from two perspectives - distributed chunk-based processing in the cloud and a user study by means of mental workload. Running experiments in mobile video applications demonstrate that our proposed techniques improve mobile user experience significantly.

1. Cloud Services for Improved User
Experience in Sharing Mobile Videos
Dejan Kovachev, Yiwei Cao & Ralf Klamma
RWTH Aachen University
Advanced Community Information Systems (ACIS)
kovachev@dbis.rwth-aachen.de
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-1 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

2. Advanced Community
Information Systems (ACIS)
Web Engineering Responsive
Community
Web Analytics
Open
Visualization
Community
and
Information
Simulation
Systems
Community Community
Support Analytics
Lehrstuhl Informatik 5
Requirements
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-2
Engineering

3. Agenda
 Motivation
 Background and related work
 Conceptual approach
 System design and implementation
 Evaluation
 Conclusions and outlook
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-3

4. Motivation
 Mobile video is rapidly increasing
– Video accounts more than half of all global mobile data traffic (Cisco Visual
Network Index, 2011)
– Video-based sharing of life experiences in near real time anywhere anytime
 Mobile user experience (MEX) is still poor
– Significant challenges must be addressed in the areas of quality, user experience
and cost of delivery (IEEE, 2012)
– Technical issues - mobile video quality of service
– Mobile networks constantly fluctuate in bandwidth, delay, jitter, and packet loss
– Presentation issues - perceived experience from user
– Mobile devices are not optimized for streaming of high-quality content (HD, Super Hi-Vision)
– Amateur video content shot with smartphones lacks the aesthetics of professional videos
– Difficult video navigation browsing
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-4

5. How can we improve the MEX?
 Intelligent video processing
- Content-aware video retargeting to overcome small screen size problem
 Object recognition, feature extraction, segmentation, indexing, bitrate adaptation
- Processing issues, specialized software tools,
 Cloud Computing
Lehrstuhl Informatik 5
(Information Systems)
- “Unlimited” processing power and storage
Prof. Dr. M. Jarke
I5-KCKl-0313-5 - Real-time video processing by parallel processing files (Perreira et al., 2010)

6. Related Work (2)
 Cloud video processing
– MapReduce-based solutions
– Content mixing (pictures, video) [Sandholm 2011]
– Video format and bitrate transcoding [Garcia et al. 2010, Pereira et al. 2010]
– Feature detection [Chen and Schlosser 2008]
– Pay-per-use cloud resources to improve stream quality
[Trajkovska et al. 2009]
 Adaptive streaming
– Chunk streaming [Mazzola Paluska and Pham 2010]
– MPEG-DASH standard [Vetro 2011]
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-6

7. Related Work
 Mobile User Experience
– Definition: “a person’s perceptions and responses that result from
the use or anticipated use of a product, system or service” (ISO,
2010)
 MEX enhancements
– Optimal zoom ratio [Knoche et al. 2007, Song et al. 2010]
– Region of interest enhancement [Knoche et al. 2007]
– Segment-based video navigation [Bursuc et al. 2010]
– Annotation-based navigation [Bentley and Groble 2009]
– Stream personalization [Patrikakis 2011]
– Mixing video streams into one view [Kaheel et al. 2009]
– Video image stabilization
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-7

8. MVCS Workflow
 MVCS – Mobile video cloud services
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-8

9. Mobile Client UI
Camera Activities
login browse videos
Main Activity
Video Player Activity
without login
Login Activity Browse Activities
Main Activity
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke Preferences Activity
I5-KCKl-0313-9

10. MEX Enhancement
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-10

11. MVCS Prototype – Mobile Client
 Implemented for Android SDK 2.2+ MVCS Client
 Libraries used: Smack, FFmpeg, Apache GUI
Video Recorder Video Player
mime, Apache HTTP Tags
Metadata & Segments
Overlay
 Lazy load to save resources Video List Preferences
 Technical issues
Handlers
– Mobile upstreaming a big trend but no off- Lazy Load Handler
MP4 Handler
the-shelf solutions
Metadata Handler Sync Handler
– Only basic RTP streaming support in Android
– Sipdroid, FFmpeg (NDK, C++), RTP socket Communication Layer
XMPP Connector (aSmack)
– No official Smack (XMPP) library for Android
RTP Stream Client
– Video player and camera activity are
File Transfer
proprietary
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-11

12. MVCS Prototype – Cloud Services
 Implemented in Java using Smack, OpenCV, FFmpeg, Apache HTTP,
x264 and shotdetect libraries
 Object recognition using Haar classifiers (Messom et al., 2006) and
JavaCV (OpenCV)
 Scene detection using histogram differences, fixed threshold and
FFmpeg
 Zooming realized by cropping Intelligent Video Processing Services
Segmentation
Parallel processing of video chunks
Transcoding
Recognition

(OpenCV)
(FFmpeg)
(FFmpeg)
(FFmpeg,
Zooming
Object
x264)
– Splitting and merging video files into
chunks (keyframes have to be identified)
Video
Metadata Service
Service
Lehrstuhl Informatik 5 XMPP Service (Smack)
(Information Systems)
Prof. Dr. M. Jarke MVCS Cloud
I5-KCKl-0313-12

13. i5Cloud Testbed
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
[Kovachev et al. 2012]
I5-KCKl-0313-13

14. Chunk Processing of Videos
 Inspired by Split & Merge approach (Perreira et al., 2010)
 Video is split into chunks at keyframe (I-frame)
 Chunks are processed on multiple instances and should speed up
processing of videos
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-14

15. Evaluation
 Subjects
– 3 representative types of videos: sport, talk, and documentary
– 12 Users: CS background, between 23 and 54 years old, mixed experience
– Questionnaire: NASA-TLX (Mental Demand, Frustration, Physical Demand,
Effort, Performance, Temporal Demand) (NASA, 1986)
– NASA-TLX is a subjective workload assessment tool
 Zooming Evaluation
– User has to understand the content
– NASA-TLX has to be answered
 Browsing Evaluation
– User has to find a certain position
– NASA-TLX has to be answered
Lehrstuhl Informatik 5
 Questionnaire filled out in the end
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-15

16. MEX Evaluation Results
 Video zooming reduces workload by 26% to 49%
 Video browsing reduces workload by 64% to 67%
 Zooming and browsing good approaches to reduce workload and to
improve MEX
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-16

17. Cloud Services Performance
Evaluation - Setup
 Subjects
– Videos with different length (1:03 mins, 2:59 mins, 5:21 mins)
– H.264 video codec, AAC audio codec, 30fps
 Procedure
– Each video as a single file processed by all intelligent video services
– Each video as chunks processed by all intelligent video services
– Segmentation service
– Zooming service
– Comparison of one file solution to chunk based solution
– Number of instances is increased proportional to video length
– Simple video transcoding
– Processing time is recorded
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-17

18. Cloud Evaluation Results
 Chunk approach enables faster video processing
 Parallel processing of chunks results in fast intelligent video processing services
02:53
02:36
02:18
Processing Time in Minutes
02:01
Single Transcoding
01:44
Chunk Transcoding
01:26
Single Scene
01:09 Chunk Scene
00:52 Single Zooming
Chunk Zooming
00:35
00:17
00:00
01:03 02:59 05:21
Dura on in Minutes
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-18

19. Conclusions
 Mobile Video Cloud Services (MVCS) combine different approaches
and algorithms to deliver fast intelligent video processing services for a
better MEX
– Improved mobile browsing by segmentation and tags
– Zoomed videos do overcome screen size problems
 Cloud services
– Better utilization of cloud environment by splitting videos into chunks
– Complex tasks like object movement offload to the cloud
 Future Work
– More user studies in different environments
– Automatic annotation of videos with metadata
– Hardware acceleration using GPU
Lehrstuhl Informatik 5
– Extension with robust and fast computer vision algorithms
(Information Systems)
Prof. Dr. M. Jarke – Adaptive HTTP live streaming
I5-KCKl-0313-19

20. Thanks for your attention!
Q&A
Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-KCKl-0313-20

21. References
 IEEE P2200. IEEE Standard Protocol for Stream Management in Media Client Devices
 IEEE P1907.1. Standard for Network-Adaptive Quality of Experience (QoE) Management Scheme for Real-Time Mobile
Video Communications
 R. Pereira, M. Azambuja, K. Breitman, and M. Endler. An Architecture for Distributed High Performance Video Processing
in the Cloud. In CloudCom, 2010
 ISO FDIS 9241-210:2010. Ergonomics of human system interaction - Part 210, 2010
 W. Song, D. W. Tjondronegoro, S.-H. Wang, and M. J. Docherty. Impact of Zooming and Enhancing Region of Interests for
Optimizing User Experience on Mobile Sports Video. In ACM Multimedia, 2010
 H. Knoche, M. Papaleo, M. A. Sasse, and A. Vanelli-Coralli. The Kindest Cut: Enhancing the User Experience of Mobile TV
Through Adequate Zooming. In ACM Multimedia, 2007
 A. Bursuc, T. Zaharia, and F. Prêteux. Mobile Video Browsing and Retrieval with the OVIDIUS Platform. In ACM
Multimedia, 2010
 A. Kaheel, M. El-Saban, M. Refaat, and M. Ezz. Mobicast: A System for Collaborative Event Casting Using Mobile Phones.
In MUM, 2009
 F. R. Bentley and M. Groble. TuVista: Meeting the Multimedia Needs of Mobile Sports Fans. In ACM Multimedia, 2009
 P. Patrikakis, N. Papaoulakis, C. Stefanoudaki, A. Voulodimos, and E. Sardis. Handling Multiple Channel Video Data for
Personalized Multimedia Services: A Case Study on Soccer Games Viewing. In PerCom, 2011
 T. Sandholm. HP Labs Cloud-Computing Test Bed: VideoToon Demo, 2011
 A. Garcia, H. Kalva, and B. Furht. A Study of Transcoding on Cloud Environments for Video Content Delivery. In ACM
Multimedia, 2010
Lehrstuhl Informatik 5  S. Chen and S. W. Schlosser, “Map-Reduce Meets Wider Varieties of Applications,” Intel Labs Pittsburgh Tech Report,
(Information Systems)
Prof. Dr. M. Jarke
May 2008
I5-KCKl-0313-21