Shoaib Haseeb presented on software quality engineering and testing of the VLC media player. VLC is a free open source cross-platform media player available in many languages. It plays most multimedia files and discs. The presentation outlined testing criteria like functionality, system design, implementation, and integration testing. Specific bugs in VLC were discussed along with suggested fixes and improving the test suite. The goal is to develop test protocols and improve VLC quality.

1. Software Quality Engineering

2. Presented By:
Shoaib Haseeb
Software Engineer
NUST H-12 Islamabad
Pakistan
+923465374294

3. VLC media player
VLC is a free and open source cross-platform multimedia player and
framework that plays most multimedia files as well as DVDs, Audio CDs,
VCDs, and various streaming protocols.

4. Background
• Initial release in Feb.2001.
• Written in C, C++,Objective–C Lua.
• Platform IA-32,Arm,MIPS.
• Available in 48 languages.
• VLC media player is downloaded at an approximate monthly rate of 5
millions from the main website and that's not including third-party
distributions

5. Application's:
VLC media player is a cross-platform media player, with versions for
Windows, OS X, iOS, Linux, Android, BSD, BeOS, OS/2.
.

6. Testing Criterion:
• Functionality & UI (UI & System Storyboards, Automated Acceptance Tests).
• System Design(Guided Inspections, Simulated Execution).
• Implementation(Unit Tests, Code Quality Analysis, Performance Tests).
• Configuration & Integration(Continuous Integration, Smoke Tests
End-to-end Tests).

7. Testing Criterion:
• VLC is not very easy to test, as it does not provide "parsable" results.
What we could test quite easily:
• Playlist operation
• Core operation
• Media control
• VLM
• Check that the streams appears to play (we can't assess the result)
• Stream output: check size of output
• to be completed.

8. Repo of Methodologies :
• Inspect and adapt.
• Black Box testing.
• Smoke Testing.
• White Box Testing.
• Unit testing.
• Automated Testing.
• Agile / Extreme Testing

9. Our focus:
• Bug # 1: We can forward video while converting it to some other
format.
• Suggested fix: Forward option should be disabled while converting.

10. Our focus:
• Bug # 2: When we navigate to video option and then to zoom option,
video does not zoom.
Suggested fix: video should zoom when we are in the video options.

11. Our focus:
• Bug # 3: After changing the skin, complete functionality is not visible.
• Suggested fix: all the functionality should remain in vlc even after
switching to any skin

12. Our focus:
• Bug # 4: After changing skin, skin is not changed instantly, if we play
video again only then we can know about the change in skin..... user
is not even given the confirmation message that skin has been
changed.
• suggested fix: skin should be changed immediately or user should be
told that he needs to play video again to see the changed skin

13. Our focus:
• Bug # 5: In some versions, If we are recording a video from web cam,
we cannot know that our video is being recorded, we can see only the
vlc home screen in front of us.
• Suggested Fix: Recording Screen should have its own working
interface.

14. Our Focus:
• Bug # 6 : In some versions which we have tested, on clicking the
record button it shows message that the webcam is not supported
but still the video gets recorded.
• Suggested Fix: Integration issue between the two functionalities
should be removed.

15. Our focus:
• Bug # 7: While in recording mode, either we have initiated recording
or not, time of video keeps on changing.
• Suggested fix: time should only go forward when we have clicked on
record button.

16. Tests to run
• You want to help by testing VLC ? There are a few tests that you can
handle:
• VLC playback tests (codecs and files related) Win32 and MacOS
focused.
• VLC functionality tests
• VLC transcode and streaming tests
• VLC fundamental tests in src/test (run make check)
• There is a small test suite for the Android port located at Android test
suite.

17. External Dependencies:
• Data feeds from Social network servers like facebook, Google, yahoo
mails etc.
• Data migration from any of the db servers for testing purpose.
• External interfaces document.
• There are no significant changes in priorities that require
redeployment of resources
• No direct or virtual interaction (Through Bugzilla).
• No approved requirement document available.
• No arch/design doc available

18. What we suggest
• The idea is to develop test protocols, regroup test files, and at the end
improve the whole quality of VLC, to avoid regressions and check
roadmaps.
• The bugs should be tracked down and killed using trac.
• The community of VideoLAN's numerous users and developers can
enforce the project by doing systematic tests and reporting bugs and
regressions. Some projects rely on a lot of external code that evolves
a lot. Being numerous can help to make it better.

19. Test Suite
• This project aims at making automatic tests to improve VLC quality.
• Write a series of tests for vlc-internal
• Integrate a framework for automated plugins testing with the
automake build system
• Automate the different codec playback/mux/etc tests
• Automate the subtitles tests
• Write tests for the different bindings: Mac OS X Framework/Python
Bindings... etc.

20. Goals:
• VideoLAN’s was originally designed to stream MPEG videos on high
bandwidth networks, but VideoLAN's main software, VLC media
player, has evolved to become a full-featured, cross-platform media
player.

21. Improvements:
• Improved audio quality, improvements to the material design interface,
including the black theme and switch to audio mode. Further, it is a major
update for Android TV adding support for media discovery via UPnP, with
improvements for recommendations and gamepads.
• VLC for Windows Phone accelerated decoding allowing playback of HD
contents of certain formats as well as further iterations on the user
interface.
• Focused on improved cloud integration adding support for iCloud Drive,
OneDrive and Box.com, a 10-band equalizer as well as sharing of the media
library on the local network alongside an improved playback experience.

22. Dark Testing :
• Its a structural testing method, i.e. we use the structure of the code
to guide the test process. In a small change in a program. Such small
changes are intended to model low level defects that arise in the
process of coding systems. Ideally itshould model low-level defect
creation.

24. References:
• http://nightlies.videolan.org/
• http://www.videolan.org/news.html
• https://wiki.videolan.org/Quality/