Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
How Build Infrastructure Powers the
Node.js Foundation
Gibson Fahnestock, IBM
Me
• Gibson Fahnestock, @gibfahn
• Node team at IBM Runtimes
• Node.js core collaborator
• @nodejs/{Build,Release,Moderati...
The Plan
1. How do we do things today?
2. What are we working towards?
3. How you can get involved?
1
The Plan
Johan Bergström
@jbergstroem
João Reis
@joaocgreis
Rod Vagg
@rvagg
Gibson Fahnestock
@gibfahn
Refael Ackermann
@refack
Phi...
Johan Bergström
@jbergstroem
João Reis
@joaocgreis
Rod Vagg
@rvagg
Gibson Fahnestock
@gibfahn
Refael Ackermann
@refack
Phi...
Johan Bergström
@jbergstroem
João Reis
@joaocgreis
Rod Vagg
@rvagg
Gibson Fahnestock
@gibfahn
Refael Ackermann
@refack
Phi...
• Support for Node.js and other projects.
• Which requires:
• Wide platform coverage
• High availability of build farms.
•...
A message to our sponsors
Community
Contributions:
Community
Contributions:
Community Contributions:
Platform coverage
• 166 machines
• 99 x86_64
• 54 ARM (arm6, arm7, arm8)
• 19 Power (BE and LE) and z
• Breadth of OS cove...
Platform coverage
• 166 machines
• 99 x86_64
• 54 ARM (arm6, arm7, arm8)
• 19 Power (BE and LE) and z
• Breadth of OS cove...
Platform coverage
• Easy for collaborators to
get started.
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
jenkins.io
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
jenkins.io
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
jenkins.io
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
jenkins.io
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
jenkins.io
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
Tech we use
• Jenkins
• Cloud provisioning (e.g. OpenStack)
• Ansible
Jenkins
• Public test CI:
• https://ci.nodejs.org
• Restricted release CI:
• https://ci-release.nodejs.org
• Material desi...
Jenkins - teams
• Access controlled by Github teams
• Per-job accesses
• Core, ChakraCore, Libuv, CitGM
• Streams, Llnode,...
30 minute builds
• Build and test on all platforms in 30 minutes.
• Easier on a LinuxOne server than a Raspberry Pi!
• Pro...
30 minute builds
• Build and test on all platforms in 30 minutes.
• Easier on a LinuxOne server than a Raspberry Pi!
• Pro...
30 minute builds
• Build and test on all platforms in 30 minutes.
• Easier on a LinuxOne server than a Raspberry Pi!
• Pro...
30 minute builds
• Build and test on all platforms in 30 minutes.
• Easier on a LinuxOne server than a Raspberry Pi!
• Pro...
Ansible
• One command to set up any new machine
• Define our own scripts that others can rely on
• Want to build node? Go ...
2. The Quest
•Make Build more accessible
2
Jenkins pipelines
• Some of our jobs are a little…
Jenkins pipelines
… complex
• Problem: Few people have access,
hard to review
Jenkins pipelines
• Problem: Few people have access,
hard to review
• Solution:
• Use pipelines!
Jenkins pipelines
• Case study: what is wrong with this code?
Jenkins pipelines
• Case study: what is wrong with this code?
6.0.0 -> 6
10.0.0 -> 1
Jenkins pipelines
• Problem: we don’t have many pipeline experts
• Solution: amazing people from the community show up to ...
The mistake of the machine
• Mysterious build failures
The mistake of the machine
The Dream
• Problem: How do we give people the confidence to fix
machines, in architectures they’re unfamiliar with?
• Sol...
Sponsors page
• Currently acknowledged on the Build WG README.
• Want to do something like
https://adoptopenjdk.net/sponso...
The Dream: sponsors page
Current challenges
• macOS – automating the setup
• Ansible – platform support and testing
• Jenkins– moving to pipelines
...
Questions?
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

How Build Infrastructure Powers the Node.js Foundation

Download to read offline

Learn about what the Node Build Working Group does, all the cool stuff we have in the pipeline, and how you could get involved!

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

How Build Infrastructure Powers the Node.js Foundation

  1. 1. How Build Infrastructure Powers the Node.js Foundation Gibson Fahnestock, IBM
  2. 2. Me • Gibson Fahnestock, @gibfahn • Node team at IBM Runtimes • Node.js core collaborator • @nodejs/{Build,Release,Moderation,CitGM…} • gibfahn @gibfahn IRC: gibfahn ✉️ gibfahn@gmail.com
  3. 3. The Plan 1. How do we do things today? 2. What are we working towards? 3. How you can get involved? 1
  4. 4. The Plan
  5. 5. Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvagg Gibson Fahnestock @gibfahn Refael Ackermann @refack Phillip Johnsen @phillipj Rich Trott @trott Myles Borins @thealphanerd Kunal Pathak @kunalspathak Michele Capra @piccoloaiutante Hans Kristian Flaatten @Starefossen Michael Dawson @mhdawson Wyatt Preul @geek George Adams @gdams
  6. 6. Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvagg Gibson Fahnestock @gibfahn Refael Ackermann @refack Phillip Johnsen @phillipj Rich Trott @trott Myles Borins @thealphanerd Kunal Pathak @kunalspathak Michele Capra @piccoloaiutante Hans Kristian Flaatten @Starefossen Michael Dawson @mhdawson Wyatt Preul @geek George Adams @gdams
  7. 7. Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvagg Gibson Fahnestock @gibfahn Refael Ackermann @refack Phillip Johnsen @phillipj Rich Trott @trott Myles Borins @thealphanerd Kunal Pathak @kunalspathak Michele Capra @piccoloaiutante Hans Kristian Flaatten @Starefossen Michael Dawson @mhdawson Wyatt Preul @geek George Adams @gdams
  8. 8. • Support for Node.js and other projects. • Which requires: • Wide platform coverage • High availability of build farms. • Automation and documentation to reduce bus factor • We have no 24/7 on-call staff! The Mission • Build • Test • Benchmark • Release • Host
  9. 9. A message to our sponsors
  10. 10. Community Contributions:
  11. 11. Community Contributions: Community Contributions:
  12. 12. Platform coverage • 166 machines • 99 x86_64 • 54 ARM (arm6, arm7, arm8) • 19 Power (BE and LE) and z • Breadth of OS coverage • Linux, Win, macOS • AIX, SmartOS, FreeBSD • node-test-commit runs on 44 platforms
  13. 13. Platform coverage • 166 machines • 99 x86_64 • 54 ARM (arm6, arm7, arm8) • 19 Power (BE and LE) and z • Breadth of OS coverage • Linux, Win, macOS • AIX, SmartOS, FreeBSD • node-test-commit runs on 44 platforms
  14. 14. Platform coverage • Easy for collaborators to get started.
  15. 15. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  16. 16. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  17. 17. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  18. 18. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  19. 19. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible jenkins.io
  20. 20. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible
  21. 21. Tech we use • Jenkins • Cloud provisioning (e.g. OpenStack) • Ansible
  22. 22. Jenkins • Public test CI: • https://ci.nodejs.org • Restricted release CI: • https://ci-release.nodejs.org • Material design theme!!!
  23. 23. Jenkins - teams • Access controlled by Github teams • Per-job accesses • Core, ChakraCore, Libuv, CitGM • Streams, Llnode, node-report
  24. 24. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  25. 25. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  26. 26. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  27. 27. 30 minute builds • Build and test on all platforms in 30 minutes. • Easier on a LinuxOne server than a Raspberry Pi! • Problem: need fast builds • Solutions: • File caching • ccache • Fanning
  28. 28. Ansible • One command to set up any new machine • Define our own scripts that others can rely on • Want to build node? Go to: • https://github.com/nodejs/build/tree/master/ansible
  29. 29. 2. The Quest •Make Build more accessible 2
  30. 30. Jenkins pipelines • Some of our jobs are a little…
  31. 31. Jenkins pipelines … complex • Problem: Few people have access, hard to review
  32. 32. Jenkins pipelines • Problem: Few people have access, hard to review • Solution: • Use pipelines!
  33. 33. Jenkins pipelines • Case study: what is wrong with this code?
  34. 34. Jenkins pipelines • Case study: what is wrong with this code? 6.0.0 -> 6 10.0.0 -> 1
  35. 35. Jenkins pipelines • Problem: we don’t have many pipeline experts • Solution: amazing people from the community show up to help out! • Great way to get involved (nodejs/build#838).
  36. 36. The mistake of the machine • Mysterious build failures
  37. 37. The mistake of the machine
  38. 38. The Dream • Problem: How do we give people the confidence to fix machines, in architectures they’re unfamiliar with? • Solution: One-click “destroy and reprovision machine” Fix Everything
  39. 39. Sponsors page • Currently acknowledged on the Build WG README. • Want to do something like https://adoptopenjdk.net/sponsors.html • Only basic html knowledge required! nodejs/nodejs.org#1257 • Get involved! 3
  40. 40. The Dream: sponsors page
  41. 41. Current challenges • macOS – automating the setup • Ansible – platform support and testing • Jenkins– moving to pipelines • Come get involved!
  42. 42. Questions?

Learn about what the Node Build Working Group does, all the cool stuff we have in the pipeline, and how you could get involved!

Views

Total views

504

On Slideshare

0

From embeds

0

Number of embeds

329

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×