Keeping the Node.js
Community Infrastructure
Humming
Michael Dawson, IBM
About Michael Dawson
Loves the web and building software (with Node.js!)
Senior Software Developer @ IBM
IBM Runtime Technologies Node.js Technical Lead
Node.js collaborator and CTC member
Active in LTS, build, benchmarking , api
and post-mortem working groups
Contact me:
michael_dawson@ca.ibm.com
Twitter: @mhdawson1
https://www.linkedin.com/in/michael-dawson-6051282
3
About The Build Team
Loves hardware, operating systems, infrastructure and building Node.js
Johan Bergström
@jbergstroem
João Reis
@joaocgreis
Rod Vagg
@rvag
Alexis Campailla
@orangemocha
Michael Dawson
@mhdawson
Hans Kristian
Flaatten
@Starefossen
Julien Gilli
@misterdjules
Rich Trott
@trott
Myles Borins
@thealphanerd
Ryan Graham
@rmg
Imran Iqbal
@iWuzHere
Phillip Johnsen
@phillipj
4
https://github.com/nodejs/build/blob/master/README.md
 Providing computing infrastructure
 Streamline testing and releasing
 Operate and maintain
– Jenkins(ci and release)
– benchmarking
– nodejs.org
– and more
Purpose
 Preference for community donations
–Expands involvement
–In line with Node.js approach - contributions influence
direction
 Cloud based
 Spread across providers
5
Our Approach
6
Infrastructure Providers
 Tier 1
 Tier 2
Microsoft Joyent IBM(osusol/softlayer)
Voxer Scaleway Nodesource
CloudFlare Arm
DigitalOcean Rackspace
7
Community Donations
From -> https://github.com/nodejs/build/blob/master/README.md
Picture provided by Rod Vagg
 Jenkins Based
 Separate Build and Release Instances
– https://ci.nodejs.org
– https://ci-release.nodejs.org
 Build
– 116 machines on last count
– 8 architectures (x86, x64, arm6, arm7, arm8, PPC be 64bit , PPC le
64 bit, s390x)
– 6 OS’s (Linux®, SmartOS, Windows®, OSX, AIX, FreeBSD)
 Release
– 14 machines on last count
– 8 architectures (x86, x64, arm6, arm7, arm8, PPC be 64bit , PPC le
64 bit, s390x)
– 4 OS’s (Linux, SmartOS, Windows, OSX)
8
Infrastructure
9
Many Linux Variants
 Test
 Release
10
Builds
 Nothing too interesting here
 bash like syntax
 Most of logic built into:
11
Jobs
https://github.com/nodejs/node/blob/master/Makefile
 Mostly virtual machines except:
– Arm (Nodesource)
– Bare metal benchmarking machine (Softlayer)
 Virtual machines
– free accounts from Infrastructure Providers
 It’s a lot of Machines to manage ! (over 130)
 Ansible - https://www.ansible.com/
12
Machines
Picture provided by Rod Vagg
 Agentless
 Connects over ssh
 Key files
– ansible-inventory
– ansible-playbook.yaml
– ansible-vars.yaml
13
Ansible
https://github.com/nodejs/build/
tree/master/setup
14
Ansible
ansible-inventory
ansible-vars.yaml
15
ansible-
playbook
.yaml
 Multiple machines (at least 2 per platform)
 ccache
 Fanned jobs
 Parallel test execution
16
Staying Nimble
Build/test
across all
platforms
30 Mins
 Shared secrets
– Infra
– Test
– Release
17
Trust Delegation
 Website - https://nodejs.org
 Benchmarking - https://benchmarking.nodejs.org/
 Bots
 Machine availability notifications
 Backups
 rsync mirrors
 May support other projects
– Ex serialport module
18
Not just build/testing
 You may not have access to test on all platforms/os combinations
 Potential access to build infra to test/investigate issues
 https://github.com/nodejs/build/blob/master/doc/process/special_access_to_build_resources
.md
19
Machine Access
 OSX Support
 Jenkins
20
Current Challenges
Get Involved !
 Great chance to learn
– Hardware, OS’s, Jenkins, Ansible
– Cloud provisioning (openstack etc)
 Donate
 Where
–Most work done through GitHub
issues/Pull Requests
–https://github.com/nodejs/build/
–IRC: #node-build at Freenode
Copyrights and Trademarks
© IBM Corporation 2016. All Rights Reserved
IBM, the IBM logo, ibm.com are trademarks or registered
trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and
service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
Node.js is an official trademark of Joyent. IBM SDK for Node.js is not formally related to
or endorsed by the official Joyent Node.js open source or commercial project.
Java, JavaScript and all Java-based trademarks and logos are trademarks or registered
trademarks of Oracle and/or its affiliates.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or
both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft
Corporation in the United States, other countries, or both.

Update from-build-workgroup

  • 1.
    Keeping the Node.js CommunityInfrastructure Humming Michael Dawson, IBM
  • 2.
    About Michael Dawson Lovesthe web and building software (with Node.js!) Senior Software Developer @ IBM IBM Runtime Technologies Node.js Technical Lead Node.js collaborator and CTC member Active in LTS, build, benchmarking , api and post-mortem working groups Contact me: michael_dawson@ca.ibm.com Twitter: @mhdawson1 https://www.linkedin.com/in/michael-dawson-6051282
  • 3.
    3 About The BuildTeam Loves hardware, operating systems, infrastructure and building Node.js Johan Bergström @jbergstroem João Reis @joaocgreis Rod Vagg @rvag Alexis Campailla @orangemocha Michael Dawson @mhdawson Hans Kristian Flaatten @Starefossen Julien Gilli @misterdjules Rich Trott @trott Myles Borins @thealphanerd Ryan Graham @rmg Imran Iqbal @iWuzHere Phillip Johnsen @phillipj
  • 4.
    4 https://github.com/nodejs/build/blob/master/README.md  Providing computinginfrastructure  Streamline testing and releasing  Operate and maintain – Jenkins(ci and release) – benchmarking – nodejs.org – and more Purpose
  • 5.
     Preference forcommunity donations –Expands involvement –In line with Node.js approach - contributions influence direction  Cloud based  Spread across providers 5 Our Approach
  • 6.
    6 Infrastructure Providers  Tier1  Tier 2 Microsoft Joyent IBM(osusol/softlayer) Voxer Scaleway Nodesource CloudFlare Arm DigitalOcean Rackspace
  • 7.
    7 Community Donations From ->https://github.com/nodejs/build/blob/master/README.md Picture provided by Rod Vagg
  • 8.
     Jenkins Based Separate Build and Release Instances – https://ci.nodejs.org – https://ci-release.nodejs.org  Build – 116 machines on last count – 8 architectures (x86, x64, arm6, arm7, arm8, PPC be 64bit , PPC le 64 bit, s390x) – 6 OS’s (Linux®, SmartOS, Windows®, OSX, AIX, FreeBSD)  Release – 14 machines on last count – 8 architectures (x86, x64, arm6, arm7, arm8, PPC be 64bit , PPC le 64 bit, s390x) – 4 OS’s (Linux, SmartOS, Windows, OSX) 8 Infrastructure
  • 9.
    9 Many Linux Variants Test  Release
  • 10.
  • 11.
     Nothing toointeresting here  bash like syntax  Most of logic built into: 11 Jobs https://github.com/nodejs/node/blob/master/Makefile
  • 12.
     Mostly virtualmachines except: – Arm (Nodesource) – Bare metal benchmarking machine (Softlayer)  Virtual machines – free accounts from Infrastructure Providers  It’s a lot of Machines to manage ! (over 130)  Ansible - https://www.ansible.com/ 12 Machines Picture provided by Rod Vagg
  • 13.
     Agentless  Connectsover ssh  Key files – ansible-inventory – ansible-playbook.yaml – ansible-vars.yaml 13 Ansible https://github.com/nodejs/build/ tree/master/setup
  • 14.
  • 15.
  • 16.
     Multiple machines(at least 2 per platform)  ccache  Fanned jobs  Parallel test execution 16 Staying Nimble Build/test across all platforms 30 Mins
  • 17.
     Shared secrets –Infra – Test – Release 17 Trust Delegation
  • 18.
     Website -https://nodejs.org  Benchmarking - https://benchmarking.nodejs.org/  Bots  Machine availability notifications  Backups  rsync mirrors  May support other projects – Ex serialport module 18 Not just build/testing
  • 19.
     You maynot have access to test on all platforms/os combinations  Potential access to build infra to test/investigate issues  https://github.com/nodejs/build/blob/master/doc/process/special_access_to_build_resources .md 19 Machine Access
  • 20.
     OSX Support Jenkins 20 Current Challenges
  • 21.
    Get Involved ! Great chance to learn – Hardware, OS’s, Jenkins, Ansible – Cloud provisioning (openstack etc)  Donate  Where –Most work done through GitHub issues/Pull Requests –https://github.com/nodejs/build/ –IRC: #node-build at Freenode
  • 22.
    Copyrights and Trademarks ©IBM Corporation 2016. All Rights Reserved IBM, the IBM logo, ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml Node.js is an official trademark of Joyent. IBM SDK for Node.js is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. Java, JavaScript and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.