How we test tvideo at skype


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

How we test tvideo at skype

  1. 1. How we test Video at SkypeApril 2012
  2. 2. Oksana Dementsova• SDET, Video in Tallinn office• Microsoft Platforms Team oksana.dementsova Slide 2
  3. 3. Agenda1. Skype Video Team2. How we develop Video3. How we test Video 2011 © Skype. Commercially confidential. Slide 3
  4. 4. Video Team Slide 4
  5. 5. Video Team• Video Team develops Skype Video functionality (Video Library)• Structure: • Video Developers team: 18 Developers • Video QE team: 9 Quality Engineers (QEs)• Location: Tallinn and Stockholm Slide 5
  6. 6. Development process Slide 6
  7. 7. Video Library• Platform • Microsoft Platforms • iOS, OSX Platform • Embedded and Android• Streaming• Codec, processing Slide 7
  8. 8. Video Library release• RV – collection of functionality that provides an end user or third party a complete set of functionality that is valuable in the market.• RVs at Skype: Video, Audio, iPhone UI, Call Signalling• Release of Video Library each 2 months• Important fixes are backported to the release• Other RVs (release vehicles) consume last released Video Library during development Slide 8
  9. 9. Development processes• Scrum teams of 2-5 developers, 1-2 QEs• Sprint length: 2 weeks• QEs tasks in Scrum: • Adding Acceptance Criteria to Product Backlog Items (PBIs) • Creating Testing Product Backlog Items (PBIs) • Taking part in task estimation • Working on task during the Sprint • Reporting on quality status during Sprint review Slide 9
  10. 10. What else does Video QE do?• We own the product we develop• We are responsible for quality• We make the product quality status visible• We are a communication channel between user and developers• Technology scouting Slide 10
  11. 11. QE Team Communications• Why? To share knowledge, update the status outside Scrum team, to get the idea of Video Library status• How? Skype chats; Video calls; Face to face communication; Offsites; Presentations; workshops• When? Weekly PPP (Progress, Problems, Plans) update; Weekly Video QE meeting; Video QE summits, Video team offsites Slide 11
  12. 12. Testing process Slide 12
  13. 13. Tools used for test planning and reporting• Bug tracking system: Jira• Test repository: TMT• Documentation, reporting system: Confluence Slide 13
  14. 14. Testing metrics for real time video• Objective: • Frame rate • Resolution and aspect ratio • Color space • Bitrate • Delay• Subjective: • Smoothness, jerkiness, freezes • Sharpness, pixelation, artifacts, flickering Slide 14
  15. 15. Objective testing• Logs• Call technical info Slide 15
  16. 16. Subjective testing• Visually• Call quality feedback• Labs Slide 16
  17. 17. Types of testing• Manual and automatic• Unit, Component, system, integration• Functional and non-functional (NFR – non-functional requirements)• Performance• Integration: Exploratory, Smoke, Interoperability Slide 17
  18. 18. After integration Slide 18
  19. 19. Smoke testing Slide 19
  20. 20. Functional testing• Does it work? Slide 20
  21. 21. Non-functional testing• How does it work? Slide 21
  22. 22. Interoperability testing Slide 22
  23. 23. Performance testing Slide 23
  24. 24. Release testing• Build configurations• Release testing matrix for Windows desktop: Slide 24
  25. 25. Tools we use Slide 25
  26. 26. Automatic calling system (ACS)• Internally developed• Hundreds of computers in Tallinn and Stockholm offices, a lot of individual machines over the world Slide 26
  27. 27. Automatic calling system (ACS)• Runs thousands automatic 1:1 and conference calls every day on different platforms. Test reports provided• Testcases: number of calls, their duration, idle time, participants, build version, video input device (camera or screen sharing) Slide 27
  28. 28. Client logging• Should be enabled• Logs are encrypted• Internal tool which allows: • Decrypting logs • Parsing logs • Saving logs Slide 28
  29. 29. Parsing the log files• Example of Windows Phone log lines parsing Slide 29
  30. 30. Helpful tools• Network emulating tools: Dummynet• Tools to load the PC: CPU killer, CPU burn• Virtual cameras Slide 30
  31. 31. Helpful Windows Phone tools• Bugsense• Internal tool for saving call stack• zSystemInfo Slide 31
  32. 32. Thank you! Questions? Slide 32