SlideShare a Scribd company logo
DevOps and Node.js
Chetan Desai, Intuit
DevOps Architect, TurboTax
@chetanddesai 11/17/2015
Intuit’s SOA
Application Services
Capability Services
Utility Services
User Experience Multi-device user experiences (cares about layout)
Application specific services (data interaction for UX)
Re-usable capabilities across applications (tax engine)
Data platform type services (login & identity)
My First Reaction
•  I come from deploying in containers like JBoss &
Tomcat
•  What do you mean the whole thing dies if there is an
exception?!
–  Run 30 million+ customers through it.
•  Consult some experts: NodeSource
•  Our DevOps practices was integral to our success
Enterprise Considerations
1.  Shared build farms
2.  Build once / deploy multiple times
3.  Minimize failure points during deployment
4.  Availability
Shared Build Farms
Problem: global npm dependencies
•  npm WARN prefer global <pkgName>@<ver> should be
installed with –g
–  No root access or sudo privileges
–  Other node services with different versions
CLI Tools
•  Solution: package.json
npm scripts!
•  Add global
dependencies to
devDependencies
section
•  Add your CLI calls as
npm scripts
•  Execute Scripts
–  npm run <script>
The power of the package.json
Other Benefits
1.  Local setup documentation:
- scripts for building, testing, running.
2.  Obtain consistency between developer workstations
- at least compatible
3.  Works in shared build environments!
- dependencies localized
Build Once / Deploy Multiple
•  Reproducibility is a requirement
•  Problem: Transitive Dependencies in package.json
•  semver major.minor.patch
–  ~1.1.1 "Approximately equivalent to version”
–  ^1.1.1 "Compatible with version”
–  1.1.1 "Specific version”
•  Did you install version 2.0.0 or version 2.0.0?
npm shrinkwrap
•  Solution: npm shrinkwrap –dev
–  Full transitive dependency list and versions installed
•  Somewhere in the middle
–  Don’t check in shrinkwrap.json
–  Generate it once at build time
•  How to balance dev speed and compliance?
|-----------------------------------------|
Developer Speed Reproducibility / Compliance
Fast & Reliable Deployments
•  Problem:
–  massive node_modules folder compared to the code size
–  dependency on an npm registry
•  Solution:
–  Our build OS (RHEL) is same as runtime OS
–  Reduce size of node_modules with npm prune –production
–  Zip up the remaining contents with service code
•  Never run npm at deployment time
Availability
•  Problem: Process
Management
–  Let it die and restart
–  Lots of options: pm2,
forever, strongloop pm,
cluster
•  http://strong-pm.io/
compare/
Availability
•  Solution: Who can manage processes better than the OS
itself?
–  RHEL6 upstart & RHEL7 systemd
–  Enterprises have heterogeneous set of languages
•  Deploy & monitor features needs to work across stack
•  Upstart Approach
–  Multiple stateless processes, 1:1 with CPUs
–  Load balanced & SSL termination with nginx
–  If the process dies, upstart restarts it
•  Configurable respawn window
•  Splunk for log monitoring
Recap: Enterprise Considerations
1.  Shared build farms
- local dependencies and npm scripts
2.  Build once / deploy multiple times
- zip and version deployable with modules
- shrinkwrap for reproducibility
3.  Minimize failure points during deployment
- only run npm at build time
4.  Availability
- upstart (RHEL6), systemd (RHEL7)
The solutions will evolve…
continue the conversation.
Thank you!
Chetan Desai, Intuit
DevOps Architect
@chetanddesai

More Related Content

Viewers also liked

MySQL 5.7: Focus on Replication
MySQL 5.7: Focus on ReplicationMySQL 5.7: Focus on Replication
MySQL 5.7: Focus on Replication
Mario Beck
 
Self evaluation
Self evaluationSelf evaluation
Self evaluation
MulkacE147
 
UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...
UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...
UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...
Marco Terán
 
食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」
食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」
食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」YUMIKO HIRAI
 
10 Amazon Sponsored Ads Myths You Show Know About
10 Amazon Sponsored Ads Myths You Show Know About10 Amazon Sponsored Ads Myths You Show Know About
10 Amazon Sponsored Ads Myths You Show Know About
Danny Mac
 
Amazon VPC Best Practices 2016
Amazon VPC Best Practices 2016Amazon VPC Best Practices 2016
Amazon VPC Best Practices 2016
AWSBulgaria
 
Active teaching quick learning 8th Class Maths
Active teaching  quick learning 8th Class Maths Active teaching  quick learning 8th Class Maths
Active teaching quick learning 8th Class Maths
Karnataka OER
 
(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell
(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell
(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell
Amazon Web Services
 
Network Security and Access Control in AWS
Network Security and Access Control in AWSNetwork Security and Access Control in AWS
Network Security and Access Control in AWS
Amazon Web Services
 
Building Scalable Services with Amazon API Gateway - Technical 201
Building Scalable Services with Amazon API Gateway - Technical 201Building Scalable Services with Amazon API Gateway - Technical 201
Building Scalable Services with Amazon API Gateway - Technical 201
Amazon Web Services
 
Rol del ingeniero industrial del peru frente a
Rol del ingeniero industrial del peru  frente aRol del ingeniero industrial del peru  frente a
Rol del ingeniero industrial del peru frente a
luisc1001
 
Women's Access to Healthcare - Georgia CORE Presentation
Women's Access to Healthcare - Georgia CORE PresentationWomen's Access to Healthcare - Georgia CORE Presentation
Women's Access to Healthcare - Georgia CORE Presentation
Georgia Commission on Women
 

Viewers also liked (12)

MySQL 5.7: Focus on Replication
MySQL 5.7: Focus on ReplicationMySQL 5.7: Focus on Replication
MySQL 5.7: Focus on Replication
 
Self evaluation
Self evaluationSelf evaluation
Self evaluation
 
UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...
UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...
UTE_Marco Teran _Gonzalo Remache_Fortalecer las Capacidades y Potencialidades...
 
食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」
食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」
食品と科学Vol.54 no.10「迷路から抜け出す問題解決の第一歩」
 
10 Amazon Sponsored Ads Myths You Show Know About
10 Amazon Sponsored Ads Myths You Show Know About10 Amazon Sponsored Ads Myths You Show Know About
10 Amazon Sponsored Ads Myths You Show Know About
 
Amazon VPC Best Practices 2016
Amazon VPC Best Practices 2016Amazon VPC Best Practices 2016
Amazon VPC Best Practices 2016
 
Active teaching quick learning 8th Class Maths
Active teaching  quick learning 8th Class Maths Active teaching  quick learning 8th Class Maths
Active teaching quick learning 8th Class Maths
 
(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell
(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell
(DEV202) Under the Desk to the AWS Cloud with Windows PowerShell
 
Network Security and Access Control in AWS
Network Security and Access Control in AWSNetwork Security and Access Control in AWS
Network Security and Access Control in AWS
 
Building Scalable Services with Amazon API Gateway - Technical 201
Building Scalable Services with Amazon API Gateway - Technical 201Building Scalable Services with Amazon API Gateway - Technical 201
Building Scalable Services with Amazon API Gateway - Technical 201
 
Rol del ingeniero industrial del peru frente a
Rol del ingeniero industrial del peru  frente aRol del ingeniero industrial del peru  frente a
Rol del ingeniero industrial del peru frente a
 
Women's Access to Healthcare - Georgia CORE Presentation
Women's Access to Healthcare - Georgia CORE PresentationWomen's Access to Healthcare - Georgia CORE Presentation
Women's Access to Healthcare - Georgia CORE Presentation
 

Recently uploaded

How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
wottaspaceseo
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
Roshan Dwivedi
 
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
takuyayamamoto1800
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Natan Silnitsky
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
Globus
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
Globus
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
Adele Miller
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Donna Lenk
 
APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)
Boni García
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
Philip Schwarz
 
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptxText-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
ShamsuddeenMuhammadA
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
Ortus Solutions, Corp
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Globus
 
Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
NYGGS Automation Suite
 

Recently uploaded (20)

How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
 
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
 
APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
 
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptxText-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
 
Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
 

DevOps and Node.js

  • 1. DevOps and Node.js Chetan Desai, Intuit DevOps Architect, TurboTax @chetanddesai 11/17/2015
  • 2. Intuit’s SOA Application Services Capability Services Utility Services User Experience Multi-device user experiences (cares about layout) Application specific services (data interaction for UX) Re-usable capabilities across applications (tax engine) Data platform type services (login & identity)
  • 3. My First Reaction •  I come from deploying in containers like JBoss & Tomcat •  What do you mean the whole thing dies if there is an exception?! –  Run 30 million+ customers through it. •  Consult some experts: NodeSource •  Our DevOps practices was integral to our success
  • 4. Enterprise Considerations 1.  Shared build farms 2.  Build once / deploy multiple times 3.  Minimize failure points during deployment 4.  Availability
  • 5. Shared Build Farms Problem: global npm dependencies •  npm WARN prefer global <pkgName>@<ver> should be installed with –g –  No root access or sudo privileges –  Other node services with different versions
  • 6. CLI Tools •  Solution: package.json npm scripts! •  Add global dependencies to devDependencies section •  Add your CLI calls as npm scripts •  Execute Scripts –  npm run <script>
  • 7. The power of the package.json Other Benefits 1.  Local setup documentation: - scripts for building, testing, running. 2.  Obtain consistency between developer workstations - at least compatible 3.  Works in shared build environments! - dependencies localized
  • 8. Build Once / Deploy Multiple •  Reproducibility is a requirement •  Problem: Transitive Dependencies in package.json •  semver major.minor.patch –  ~1.1.1 "Approximately equivalent to version” –  ^1.1.1 "Compatible with version” –  1.1.1 "Specific version” •  Did you install version 2.0.0 or version 2.0.0?
  • 9. npm shrinkwrap •  Solution: npm shrinkwrap –dev –  Full transitive dependency list and versions installed •  Somewhere in the middle –  Don’t check in shrinkwrap.json –  Generate it once at build time •  How to balance dev speed and compliance? |-----------------------------------------| Developer Speed Reproducibility / Compliance
  • 10. Fast & Reliable Deployments •  Problem: –  massive node_modules folder compared to the code size –  dependency on an npm registry •  Solution: –  Our build OS (RHEL) is same as runtime OS –  Reduce size of node_modules with npm prune –production –  Zip up the remaining contents with service code •  Never run npm at deployment time
  • 11. Availability •  Problem: Process Management –  Let it die and restart –  Lots of options: pm2, forever, strongloop pm, cluster •  http://strong-pm.io/ compare/
  • 12. Availability •  Solution: Who can manage processes better than the OS itself? –  RHEL6 upstart & RHEL7 systemd –  Enterprises have heterogeneous set of languages •  Deploy & monitor features needs to work across stack •  Upstart Approach –  Multiple stateless processes, 1:1 with CPUs –  Load balanced & SSL termination with nginx –  If the process dies, upstart restarts it •  Configurable respawn window •  Splunk for log monitoring
  • 13. Recap: Enterprise Considerations 1.  Shared build farms - local dependencies and npm scripts 2.  Build once / deploy multiple times - zip and version deployable with modules - shrinkwrap for reproducibility 3.  Minimize failure points during deployment - only run npm at build time 4.  Availability - upstart (RHEL6), systemd (RHEL7)
  • 14. The solutions will evolve… continue the conversation.
  • 15. Thank you! Chetan Desai, Intuit DevOps Architect @chetanddesai