Mozilla Internship 2010 Presentation

454 views

Published on

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
454
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Mozilla Internship 2010 Presentation

    1. 1. Anamaria Stoica Release Engineering Intern Summer 2010
    2. 2. Dashboards for thebuild infrastructure
    3. 3. Project Goals• Better and more reliable insight on how the build system is performing • How long jobs wait in the queue before even starting? • How many pushes per month to different branches? • How long does a build run take to complete? • Which platforms/builders take the longest? • How long do individual builds/unittests/talos take? • What is the status of existing jobs? • What is the infrastructure load? • Do we need more machines? What platform? • When is the best time for a downtime?
    4. 4. Dashboards/Reports• Wait Times Report• Pushes Report• End to End Times Report • Build Run Report • TryChooser Report• Average Time per Builder Report • Builder Report
    5. 5. Buildbot Build Pool * 4 Build Masters * 300 Slaves Try Build Pool * 1 Build Master * 200 Slaves Test Build Pool * 7 Test Masters * 371 Slaves
    6. 6. Build Run Life Cycle 169 Build Requests: • Builds: 11 platforms • Unittests: 108 • Talos: 50
    7. 7. The Great Wall of Mac minis
    8. 8. Technologies• Python• SQLAlchemy• Pylons• Google Chart Tools (Visualization API) & Data Source Python Library
    9. 9. Reports Overview• Permissive parameters • startime, endtime: unixtimestamp • int_size: interval size (seconds) • mpb: minutes per block (minutes) • filters: platform, build_type, job_type, detail_level• JSON (format=json)• Visualization API (charts)
    10. 10. Wait Times Report• How much jobs wait before being assigned to free Slaves (wait to start) Perfect DayBuildpool / Sep 29
    11. 11. Wait Times Report Less ideal day: Testpool / Sep 29 Bad day: Trybuildpool / Aug 17
    12. 12. Monthly ReportSeptemberTestpoolQuarter ReportJune-OctTrybuildpool
    13. 13. Pushes Report • How many pushes per branch?Daily Report Sep 22
    14. 14. Weekly ReportSep 13-20m-c vs. try
    15. 15. Monthly ReportJulyAnnual ReportJune-Sep
    16. 16. End to End Report• How long does a build run take to complete?• e2e time averages: • ~6h 7m, m-c Sep • ~7h 44m, try Sep • ~9h 51m, m-c Aug • ~11 10m, try Aug• Problem: nightly builds have no revision, nightly tests have a wrong revision
    17. 17. End to End Report• Outliers: • 500h build run / failed, wait times 500h • 60h build run -> 1 talos 52h / successful • 19h build run -> 1 talos 17h / exception • 127h build run -> Cancelled jobs
    18. 18. Average Time / Builder• How much time is it spent per builder- type?How much time is itspent on each platform?
    19. 19. Average Time / BuilderHow much time per How much time pereach Linux builder? each Leopart optimized job type?
    20. 20. Demo!
    21. 21. Thanks!• Chris AtLee (catlee)• John O’Duinn (joduinn)• Lukas Blakk (lsblakk)• Release Engineering Team• Erica, Julie, Karen• Interns!

    ×