B2G QA-automation lab presentation

997 views

Published on

This briefly covers the goals, challenges, progress, and future work remaining for the buildout of a custom Firefox OS/B2G (Boot2Gecko) test-automation lab, running our Flame reference devices.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
997
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

B2G QA-automation lab presentation

  1. 1. B2G Phone-Automation Lab Goals, Challenges, Accomplishments July 31, 2014 Stephen Donner sdonner@mozilla.com
  2. 2. Goals ● greater capacity: ○ keep up with b2g-inbound builds for UI + perf (esp. the latter), as much as possible ○ multi-node mapping/support ● high reliability ● easier device troubleshooting/administration
  3. 3. From this ...wait for it!
  4. 4. 1st Buildout
  5. 5. To this ...getting there; closer there
  6. 6. 2nd Buildout
  7. 7. And here we are Bam!
  8. 8. And still going...
  9. 9. Challenges (1) ● balancing buildout with current-rig’s maintenance ● near-constant task (re)prioritization ● real carriers/networks (AT&T/T-Mobile), real problems: ○ spam calls/voicemail ○ Amber alerts ○ low signal/data-connection resets ■ slow throughput ■ DNS-lookup failures
  10. 10. Challenges (2) ● multi-node ADB mapping ○ solution had to work with both UI and perf tests ○ also required: ■ Jenkins-config changes (job/system) ■ physical reconfiguration ● active SIMs vs. inactive SIMs ○ reduce interference/cost/maintenance cost ● SecOps/NetOps requirements ○ new office, new problems (new policies) ■ port-by-port, host-by-host firewall pass-through ○ VPN/Jenkins (CI) access granted on a user-by-user basis
  11. 11. Challenges (3) ● RF (Radio Frequency) ○ Wi-Fi ■ ateam vs. Mozilla Mobile ● capacity + speed + reach ○ FM tuner interference ■ requires cabling magic ● ...and perhaps a low-power, local FM transmitter? ○ more devices, more interference? ● physical space/configuration ○ shelving space ○ power-strip capacity
  12. 12. Challenges (4) ● remote teams requested live Air Mozilla streams of the phones ○ prototype works, but doesn’t immediately scale bandwidth (# streams, codecs/encoders) ■ informs # of encoder boxes/# of channels ○ live (remote) demo ● “keeping the light on” ○ camera test failures due to low light ■ ...but intermittent
  13. 13. Currently*, we have: ● ~23 active Flame nodes (phones) ○ 12 dedicated to UI / 11 perf ○ 26 potentially ● attached to 13 active Mac Minis * number of active nodes fluctuates due to investigational/maintenance work, etc.
  14. 14. What’s Left? ● remainder of phones (20) to come online in the next few weeks ● MozPool for managing devices, including: ○ taking devices offline ○ power measurement through ammeter ○ remote-reboot capability through power harness ● Puppetization for managing node configurations
  15. 15. Comments? Questions? ● Mailing list: mozwebqa@mozilla.org ● Team page: https://quality.mozilla. org/teams/web-qa/ ● IRC: #mozwebqa on irc.mozilla.org

×