Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Build Infrastructure Powers the Node.js Foundation

321 views

Published on

Learn about what the Node Build Working Group does, all the cool stuff we have in the pipeline, and how you could get involved!

Published in: Software
  • Be the first to comment

  • Be the first to like this

How Build Infrastructure Powers the Node.js Foundation

  1. 1. How Build Infrastructure Powers the Node.js Foundation Gibson Fahnestock, IBM
  2. 2. Me • Gibson Fahnestock, @gibfahn • Node team at IBM Runtimes • Node.js core collaborator • @nodejs/{Build,Release,Moderation,CitGM…} • gibfahn @gibfahn IRC: gibfahn ✉️ gibfahn@gmail.com
  3. 3. The Plan 1. How do we do things today? 2. What are we working towards? 3. How you can get involved? 1
  4. 4. The Plan
  5. 5. Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvagg Gibson Fahnestock @gibfahn Refael Ackermann @refack Phillip Johnsen @phillipj Rich Trott @trott Myles Borins @thealphanerd Kunal Pathak @kunalspathak Michele Capra @piccoloaiutante Hans Kristian Flaatten @Starefossen Michael Dawson @mhdawson Wyatt Preul @geek George Adams @gdams
  6. 6. Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvagg Gibson Fahnestock @gibfahn Refael Ackermann @refack Phillip Johnsen @phillipj Rich Trott @trott Myles Borins @thealphanerd Kunal Pathak @kunalspathak Michele Capra @piccoloaiutante Hans Kristian Flaatten @Starefossen Michael Dawson @mhdawson Wyatt Preul @geek George Adams @gdams
  7. 7. Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvagg Gibson Fahnestock @gibfahn Refael Ackermann @refack Phillip Johnsen @phillipj Rich Trott @trott Myles Borins @thealphanerd Kunal Pathak @kunalspathak Michele Capra @piccoloaiutante Hans Kristian Flaatten @Starefossen Michael Dawson @mhdawson Wyatt Preul @geek George Adams @gdams
  8. 8. • Support for Node.js and other projects. • Which requires: • Wide platform coverage • High availability of build farms. • Automation and documentation to reduce bus factor • We have no 24/7 on-call staff! The Mission • Build • Test • Benchmark • Release • Host
  9. 9. A message to our sponsors
  10. 10. Community Contributions:
  11. 11. Community Contributions: Community Contributions:
  12. 12. Platform coverage • 166 machines • 99 x86_64 • 54 ARM (arm6, arm7, arm8) • 19 Power (BE and LE) and z • Breadth of OS coverage • Linux, Win, macOS • AIX, SmartOS, FreeBSD • node-test-commit runs on 44 platforms
  13. 13. Platform coverage • 166 machines • 99 x86_64 • 54 ARM (arm6, arm7, arm8) • 19 Power (BE and LE) and z • Breadth of OS coverage • Linux, Win, macOS • AIX, SmartOS, FreeBSD • node-test-commit runs on 44 platforms
  14. 14. Platform coverage • Easy for collaborators to get started.
  15. 15. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  16. 16. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  17. 17. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  18. 18. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  19. 19. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  20. 20. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible
  21. 21. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible
  22. 22. Jenkins • Public test CI: • https://ci.nodejs.org • Restricted release CI: • https://ci-release.nodejs.org • Material design theme!!!
  23. 23. Jenkins - teams • Access controlled by Github teams • Per-job accesses • Core, ChakraCore, Libuv, CitGM • Streams, Llnode, node-report
  24. 24. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  25. 25. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  26. 26. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  27. 27. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  28. 28. Ansible • One command to set up any new machine • Define our own scripts that others can rely on • Want to build node? Go to: • https://github.com/nodejs/build/tree/master/ansible
  29. 29. 2. The Quest •Make Build more accessible 2
  30. 30. Jenkins pipelines • Some of our jobs are a little…
  31. 31. Jenkins pipelines … complex • Problem: Few people have access, hard to review
  32. 32. Jenkins pipelines • Problem: Few people have access, hard to review • Solution: • Use pipelines!
  33. 33. Jenkins pipelines • Case study: what is wrong with this code?
  34. 34. Jenkins pipelines • Case study: what is wrong with this code? 6.0.0 -> 6 10.0.0 -> 1
  35. 35. Jenkins pipelines • Problem: we don’t have many pipeline experts • Solution: amazing people from the community show up to help out! • Great way to get involved (nodejs/build#838).
  36. 36. The mistake of the machine • Mysterious build failures
  37. 37. The mistake of the machine
  38. 38. The Dream • Problem: How do we give people the confidence to fix machines, in architectures they’re unfamiliar with? • Solution: One-click “destroy and reprovision machine” Fix Everything
  39. 39. Sponsors page • Currently acknowledged on the Build WG README. • Want to do something like https://adoptopenjdk.net/sponsors.html • Only basic html knowledge required! nodejs/nodejs.org#1257 • Get involved! 3
  40. 40. The Dream: sponsors page
  41. 41. Current challenges • macOS – automating the setup • Ansible – platform support and testing • Jenkins– moving to pipelines • Come get involved!
  42. 42. Questions?

×