Chetan shares best practices and lessons learned on Intuit’s two year journey in introducing Node.js to their TurboTax technology stack. He will specifically be focusing on Intuit’s build and deploy principles in hosting Node.js services. Chetan will also share steps in how to build a Node.js service using enterprise best practices, including reliable deployment practices.

1. Chetan Desai, Intuit
DevOps Architect, TurboTax
@chetanddesai
Building Your DevOps
for Node.js

2. 2
About Me

3. 3
1. TurboTax’s Services Journey
2. Node.js in the TurboTax Stack
3. Enterprise Principles for Node Services
4. Best Practices for Publishing Node Modules
Topics

7. Beautiful Sunset
Tropical Paradise

8. 8
30+ million users filed with TurboTax
• ~5 million used desktop
• ~25 million used online
Mobile is the biggest growing segment
TurboTax is 25 years old
• Roots as a Desktop App

9. 9
Monolith

10. 10
Marketing
Account
Tax Engine
Help

11. 11
SERVICES

12. 12
Node.js in the TurboTax Stack
Application Services
Capability Services
Utility Services
User Experience Multi-device user experiences (cares about layout)
Application specific services (data interaction for UX)
Re-usable capabilities across applications (tax engine)
Data platform type services (login & identity)

13. 13
My First Reaction
I come from deploying in containers like JBoss & Tomcat
What do you mean the whole thing dies if there is an exception?!
Run 30 million+ customers through it.
Consult some experts: NodeSource
Our DevOps practices was integral to our success

14. Enterprise Principles for
Node Services

15. 15
Enterprise Principles for Services
1. Isolation: Shared build farms
2. Reproducibility: Build once / deploy multiple times
3. Reliability: Minimize failure points during deployment
4. Availability: Keep server running

16. 16
Build & Deploy Flow at Intuit
Build Farm
GitHub
Enterprise
npm
on-site
Artifact
Repository
Intuit Data
Center

17. 17
Isolation: Shared Build Farms
• Problem: global npm dependencies
npm WARN prefer global <pkgName>@<ver> should be installed with –g
• No root access or sudo privileges
• Other node services with different versions

18. 18
Isolation: Shared Build Farms
• Solution: package.json npm
scripts!
• Add global dependencies to
devDependencies section
• Add your CLI calls as npm scripts
• Execute Scripts
• npm run <script>

19. 19
The power of the package.json
Other Benefits
1. Local setup documentation:
- scripts for building, testing, running.
2. Obtain consistency between developer workstations
- at least compatible
3. Works in shared build environments!
- dependencies localized

20. 20
Reproducibility:
Build Once / Deploy Multiple
Problem: Transitive Dependencies in package.json
• semver major.minor.patch
• ~1.1.1 "Approximately equivalent to version”
• ^1.1.1 "Compatible with version”
• 1.1.1 "Specific version”
• Did you install version 2.0.0 or version 2.0.0?
• Transitive dependencies still had ~ or ^ notation.

21. 21
Solution: npm shrinkwrap –dev
• Full transitive dependency list and versions installed
Landed somewhere in the middle
• Don’t check in shrinkwrap.json
• Generate it once at build time
How to balance developer speed and compliance?
Reproducibility:
Build Once / Deploy Multiple
Developer Speed Reproducibility / Compliance

22. 22
Reliability: Fast and Reliable Deployments
Problem: when deploying, choose between
1. massive node_modules folder compared to the code size
2. dependency on an npm registry to download modules – potential failure point
Solution:
• Reduce size of node_modules with npm prune –production
• Zip up the remaining contents with service code
• Never run npm at deployment time
• Our build OS (RHEL) is same as runtime OS

23. 23
Availability: Keep Server Running
Problem: Process Management
• “Let it terminate and restart”
• Lots of options: pm2, forever, strongloop pm,
cluster
http://strong-pm.io/compare/
• Build & Deploy
• Clustering & Management
• Profiling
• Metrics

24. 24
Availability: Keep Server Running
Enterprises have heterogeneous set of languages
• Deploy, monitor, & existing features need to work across the stack
Who can manage processes better than the OS itself?
• RHEL6 upstart & RHEL7 systemd
Upstart Approach
• Multiple stateless processes, 1:1 with CPUs
• Load balanced & SSL termination with nginx
• If the process terminates, upstart restarts it
– Configurable respawn window
– Splunk for log monitoring

25. 25
Recap: Node Services
1. Isolation: Shared build farms
- localize dependencies through npm scripts
2. Reproducibility: Build once / deploy multiple times
- shrinkwrap for transitive dependencies
3. Reliability: Minimize failure points during deployment
- only run npm at build time
- prune out dev dependencies
4. Availability: Keep server running
- upstart (RHEL6), systemd (RHEL7)

26. Best Practices for Node
Modules

27. 27
Best Practices for Node Modules
1. Only publish what’s needed
2. Reproducibility: Tagging codebase
3. Traceability: Server side publishing

28. 28
Publish only what’s needed
We’re all familiar with .gitignore - used to exclude:
• generated files
• secrets
Use .npmignore to only publish what’s needed. Exclude:
• Same items in .gitignore
• Test data and files
Understand .npmignore precedence
There are default items ignored during a publication
.gitignore will be used if .npmignore is missing
.npmignore won’t combine with .gitignore

29. 29
Tagging Codebase
Reproducibility: version bumps and tagging
• maven-release-plugin for maven pom.xml
• bumpversion library for python setup.py
Solution: npm version [<newversion> | major | minor | patch …] for your
package.json
Add git repository type for tagging support
Push commits back to origin

30. 30
Server Side Publishing
Server builds for traceability and reproducibility
Local builds have too much risk
• Forget to check-in a file
• Out of sync with other changes
Same applies for publishing node modules
How to provide publishing credentials from the server?

31. 31
Server Side Publishing
.npmrc for npm configuration
Developers had to set a dummy value set for NPM_TOKEN
Error: Failed to replace env in config: ${NPM_TOKEN}
Solution: Split them up
• Project home .npmrc specifies internal registry (line 2)
• Build user home ~/.npmrc contains credentials (line 1)

32. 32
Recap: Best Practices for Node Modules
1. Only publish what’s needed
- Use of .npmignore file
2. Reproducibility: Tagging codebase
- npm versions to tag codebase
- postversion script to push
3. Traceability: Server side publishing
- Two .npmrc files for registry and credentials

33. 33
The solutions will evolve…
continue the conversation.

34. Chetan Desai, Intuit
DevOps Architect, TurboTax
@chetanddesai
Thank You!