Skip to main content
CAdViSE: Cloud-based Adaptive Video Streaming
Evaluation Framework for the Automated
Testing of Media Players
Babak Taraghi, Anatoliy Zabrovskiy,
Christian Timmerer*, Hermann Hellwagner
Institute of Information Technology
Universität Klagenfurt, *Bitmovin
Austria 2020
Introduction and Motivation
CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players MMSys ’20, June 8–11, 2020, Istanbul, Turkey
• Evaluate how a media player can adopt the stream attributes to ups and
downs of a real network conditions
• Mimic the real-life internet connections quality (latency, packet drops,
available bandwidth and packet corruptions)
• Integratable with common CI/CD pipelines
• Visualize the results with diagrams and compare the statistical results for
multiple media players
• Using cloud infrastructure with automated deployment instead of running
on bare-metal machines
Proposed Solution
Three main components or modules:
1. Application;
i. Runner script
ii. Initializer script
iii. Starter script
2. Cloud infrastructure;
i. Player container (VNC and Selenium)
ii. Network emulator
iii. EC2, SSM, S3 and CloudWatch
3. Analytics;
i. Bitmovin Analytics plugin for players
ii. Bitmovin Analytics dashboard
CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players MMSys ’20, June 8–11, 2020, Istanbul, Turkey
Experiments and Results
• 5 Experiments, 9:00 minutes each
• t2.medium EC2 instances (4Gib RAM, 2 CPU cores)
• Using three players:
– Bitmovin Player
– Shaka Player
– Dash.js Player
• Emulated network:
a. 4 mbit/s
b. 800 kbit/s
c. 4 mbit/s
CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players MMSys ’20, June 8–11, 2020, Istanbul, Turkey
Experiments and Results
• Bitmovin Analytics
Dashboard
– Important metrics to be
compared easily for all
tested media players
– Showing clues of what
could go wrong in each
player for most users
– What are the improvable
points for players
CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players MMSys ’20, June 8–11, 2020, Istanbul, Turkey
Thanks
Reach out for Questions and Answers!
CAdViSE: Cloud-based Adaptive Video Streaming Evaluation
Framework for the Automated Testing of Media Players