This briefly covers the goals, challenges, progress, and future work remaining for the buildout of a custom Firefox OS/B2G (Boot2Gecko) test-automation lab, running our Flame reference devices.
2. Goals
● greater capacity:
○ keep up with b2g-inbound builds for UI + perf (esp.
the latter), as much as possible
○ multi-node mapping/support
● high reliability
● easier device
troubleshooting/administration
11. Challenges (2)
● multi-node ADB mapping
○ solution had to work with both UI and perf tests
○ also required:
■ Jenkins-config changes (job/system)
■ physical reconfiguration
● active SIMs vs. inactive SIMs
○ reduce interference/cost/maintenance cost
● SecOps/NetOps requirements
○ new office, new problems (new policies)
■ port-by-port, host-by-host firewall pass-through
○ VPN/Jenkins (CI) access granted on a user-by-user basis
12. Challenges (3)
● RF (Radio Frequency)
○ Wi-Fi
■ ateam vs. Mozilla Mobile
● capacity + speed + reach
○ FM tuner interference
■ requires cabling magic
● ...and perhaps a low-power, local FM transmitter?
○ more devices, more interference?
● physical space/configuration
○ shelving space
○ power-strip capacity
13. Challenges (4)
● remote teams requested live Air Mozilla
streams of the phones
○ prototype works, but doesn’t immediately scale
bandwidth (# streams, codecs/encoders)
■ informs # of encoder boxes/# of channels
○ live (remote) demo
● “keeping the light on”
○ camera test failures due to low light
■ ...but intermittent
14. Currently*, we have:
● ~23 active Flame nodes (phones)
○ 12 dedicated to UI / 11 perf
○ 26 potentially
● attached to 13 active Mac Minis
* number of active nodes fluctuates due to
investigational/maintenance work, etc.
15.
16.
17.
18. What’s Left?
● remainder of phones (20) to come online in
the next few weeks
● MozPool for managing devices, including:
○ taking devices offline
○ power measurement through ammeter
○ remote-reboot capability through power harness
● Puppetization for managing node
configurations
19. Comments? Questions?
● Mailing list: mozwebqa@mozilla.org
● Team page: https://quality.mozilla.
org/teams/web-qa/
● IRC: #mozwebqa on irc.mozilla.org