SlideShare a Scribd company logo
Node.js
In Production




                20.01.2011
About

             Contributor                                         Co-founder




                                          Felix Geisendörfer
             node.js driver                                    node-formidable




-   Joined the mailing list in June 26, 2009
-   Co-Transloadit
-   When I joined there where #24 people
-   First patch in September 2009
-   Now mailing list has > 3400 members
File uploading & processing as a
                   service for other web applications.


- The app we run in production
- Not a “typical” app, no html rendering, only JSON APIs
Are you running
node in production?
Our imagination
Our first attempt
~1
                                                              Ye

                         Transloadit v1
                                                                 a   r
                                                                         ag
                                                                           o




          • Complete failure

          • Node randomly crashed with:
              (evcom) recv() Success


          • A mess of code, hard to maintain
* http://transloadit.com/blog/2010/04/to-launch-or-not
Today
Well, almost : )
N

        Transloadit v2
                                           ow




• Processed > 2 TB of data

• Not seen a node bug in production yet

• Clean code base, 100% test coverage
How to get there?
Hosting
Managed / Heroku-style

• Joyent (no.de)
• Nodejitsu (nodejitsu.com)
• Duostack (duostack.com)
• Nodefu (nodefu.com)
• Heroku (heroku.com)
Self-managed

• Amazon Ec2
• Linode
• Rackspace Cloud Servers
• etc.
Deployment
Deployment

• Heroku service

• Custom deploy script
Custom “deploy script”


$ git clone <my-project> /var/www/<my-project>
$ nohup bin/server.js &




             Enough to get started
Staying alive
The problem
SyntaxError: Unexpected token ILLEGAL
    at Object.parse (native)
    at Server.<anonymous> (/path/to/my-script.js:4:8)
    at Server.emit (events.js:45:17)
    at HTTPParser.onIncoming (http.js:862:12)
    at HTTPParser.onHeadersComplete (http.js:85:31)
    at Socket.ondata (http.js:787:22)
    at Socket._onReadable (net.js:623:27)
    at IOWatcher.onReadable [as callback] (net.js:156:10)



    Your node process is killed by any runtime error
The wrong solution

process.on('uncaughtException', function(err) {
  console.log('Something bad happened: '+err);
  // continue on ...
});



     Continuing after an exception will leave your
            system in unpredictable state
The right solution

process.on('uncaughtException', function(err) {
  console.log('Something bad happened: '+err);
  // Every process has to die one day, it’s life
  process.exit(0);
});




    Exiting the program will prevent corrupted state
Even better
var Hoptoad = require('hoptoad-notifier').Hoptoad;
Hoptoad.key = 'YOUR_API_KEY';

process.on('uncaughtException', function(err) {
  console.log('Something bad happened: '+err);
  Hoptoad.notify(err, function() {
    process.exit(0);
  });
});

       Send your exception to a logging service.
Your process is dead.

     Now what?
Use a process monitor
          •    Monit *


          •    Upstart


          •    God (ruby)


          •    forever (node.js)

* http://kevin.vanzonneveld.net/techblog/article/run_nodejs_as_a_service_on_ubuntu_karmic/
Debugging
Debugging tools

• console.log

• node-inspector (use webkit inspector.)

• node debug <script.js> (new in 0.3.x)
Unit tests
 (Real ones, not integration tests)
Load balancing
Load Balancing

• Haproxy

• Amazon Load Balancer

• Nginx (supports only http 1.0)
Our stack
Our stack
                              Ec2 Load Balancer




                             :80            :443
  :80            :443                                  :80            :443

haproxy         stunnel    haproxy         stunnel   haproxy         stunnel

 node           nginx       node            nginx     node           nginx

          php                                                  php
                                     php
Workers

• Use workers a la Unicorn

• All workers listen on a shared socket

• Master process starts & kills workers
Workers

• spark

• fugue

• custom
master.js
var netBinding = process.binding('net');
var spawn = require('child_process').spawn;
var net = require('net');

var serverSocket = netBinding.socket('tcp4');
netBinding.bind(serverSocket, 8080);
netBinding.listen(serverSocket, 128);

for (var i = 0; i < 10; i++) {
  var fds = netBinding.socketpair();
  var child = spawn(process.argv[0], [__dirname+'/worker.js'], {
    customFds: [fds[0], 1, 2]
  });
  var socket = new net.Stream(fds[1], 'unix');
  socket.write('hi', 'utf8', serverSocket);
}
worker.js
var net = require('net');
var http = require('http');
var socket = new net.Socket(0, 'unix');

socket
  .on('fd', function(fd) {
     startServer(fd);
  })
  .resume();

function startServer(fd) {
  http.createServer(function(req, res) {
    res.writeHead(200);
    res.end('Hello from: '+process.pid+'n');
  }).listenFD(fd);
  console.log('Worker ready: '+process.pid);
}
Questions?




✎   @felixge / felix@debuggable.com

More Related Content

What's hot

Nodejs - A-quick-tour-v3
Nodejs - A-quick-tour-v3Nodejs - A-quick-tour-v3
Nodejs - A-quick-tour-v3
Felix Geisendörfer
 
Node.js - A Quick Tour
Node.js - A Quick TourNode.js - A Quick Tour
Node.js - A Quick Tour
Felix Geisendörfer
 
Dirty - How simple is your database?
Dirty - How simple is your database?Dirty - How simple is your database?
Dirty - How simple is your database?
Felix Geisendörfer
 
node.js: Javascript's in your backend
node.js: Javascript's in your backendnode.js: Javascript's in your backend
node.js: Javascript's in your backend
David Padbury
 
Nodejs Explained with Examples
Nodejs Explained with ExamplesNodejs Explained with Examples
Nodejs Explained with Examples
Gabriele Lana
 
introduction to node.js
introduction to node.jsintroduction to node.js
introduction to node.js
orkaplan
 
Nginx-lua
Nginx-luaNginx-lua
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
Tom Croucher
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
How to Test Asynchronous Code (v2)
How to Test Asynchronous Code (v2)How to Test Asynchronous Code (v2)
How to Test Asynchronous Code (v2)
Felix Geisendörfer
 
Writing robust Node.js applications
Writing robust Node.js applicationsWriting robust Node.js applications
Writing robust Node.js applications
Tom Croucher
 
Non-blocking I/O, Event loops and node.js
Non-blocking I/O, Event loops and node.jsNon-blocking I/O, Event loops and node.js
Non-blocking I/O, Event loops and node.js
Marcus Frödin
 
Node.js
Node.jsNode.js
Node.js
Jan Dillmann
 
Node.js and How JavaScript is Changing Server Programming
Node.js and How JavaScript is Changing Server Programming  Node.js and How JavaScript is Changing Server Programming
Node.js and How JavaScript is Changing Server Programming
Tom Croucher
 
Node.js - Best practices
Node.js  - Best practicesNode.js  - Best practices
Node.js - Best practices
Felix Geisendörfer
 
Introduction to Nodejs
Introduction to NodejsIntroduction to Nodejs
Introduction to Nodejs
Gabriele Lana
 
Building servers with Node.js
Building servers with Node.jsBuilding servers with Node.js
Building servers with Node.js
ConFoo
 
Streams are Awesome - (Node.js) TimesOpen Sep 2012
Streams are Awesome - (Node.js) TimesOpen Sep 2012 Streams are Awesome - (Node.js) TimesOpen Sep 2012
Streams are Awesome - (Node.js) TimesOpen Sep 2012
Tom Croucher
 
NodeJS
NodeJSNodeJS
NodeJS
.toster
 
Node js presentation
Node js presentationNode js presentation
Node js presentation
martincabrera
 

What's hot (20)

Nodejs - A-quick-tour-v3
Nodejs - A-quick-tour-v3Nodejs - A-quick-tour-v3
Nodejs - A-quick-tour-v3
 
Node.js - A Quick Tour
Node.js - A Quick TourNode.js - A Quick Tour
Node.js - A Quick Tour
 
Dirty - How simple is your database?
Dirty - How simple is your database?Dirty - How simple is your database?
Dirty - How simple is your database?
 
node.js: Javascript's in your backend
node.js: Javascript's in your backendnode.js: Javascript's in your backend
node.js: Javascript's in your backend
 
Nodejs Explained with Examples
Nodejs Explained with ExamplesNodejs Explained with Examples
Nodejs Explained with Examples
 
introduction to node.js
introduction to node.jsintroduction to node.js
introduction to node.js
 
Nginx-lua
Nginx-luaNginx-lua
Nginx-lua
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
How to Test Asynchronous Code (v2)
How to Test Asynchronous Code (v2)How to Test Asynchronous Code (v2)
How to Test Asynchronous Code (v2)
 
Writing robust Node.js applications
Writing robust Node.js applicationsWriting robust Node.js applications
Writing robust Node.js applications
 
Non-blocking I/O, Event loops and node.js
Non-blocking I/O, Event loops and node.jsNon-blocking I/O, Event loops and node.js
Non-blocking I/O, Event loops and node.js
 
Node.js
Node.jsNode.js
Node.js
 
Node.js and How JavaScript is Changing Server Programming
Node.js and How JavaScript is Changing Server Programming  Node.js and How JavaScript is Changing Server Programming
Node.js and How JavaScript is Changing Server Programming
 
Node.js - Best practices
Node.js  - Best practicesNode.js  - Best practices
Node.js - Best practices
 
Introduction to Nodejs
Introduction to NodejsIntroduction to Nodejs
Introduction to Nodejs
 
Building servers with Node.js
Building servers with Node.jsBuilding servers with Node.js
Building servers with Node.js
 
Streams are Awesome - (Node.js) TimesOpen Sep 2012
Streams are Awesome - (Node.js) TimesOpen Sep 2012 Streams are Awesome - (Node.js) TimesOpen Sep 2012
Streams are Awesome - (Node.js) TimesOpen Sep 2012
 
NodeJS
NodeJSNodeJS
NodeJS
 
Node js presentation
Node js presentationNode js presentation
Node js presentation
 

Viewers also liked

Be Your Own Technology Brand Ambassador
Be Your Own Technology Brand AmbassadorBe Your Own Technology Brand Ambassador
Be Your Own Technology Brand Ambassador
Pragati Rai
 
DesignOps Skunk Works - SXSW 2015
DesignOps Skunk Works - SXSW 2015DesignOps Skunk Works - SXSW 2015
DesignOps Skunk Works - SXSW 2015
Russ U
 
The changing nature of things: the present and future of connected products.
The changing nature of things: the present and future of connected products.The changing nature of things: the present and future of connected products.
The changing nature of things: the present and future of connected products.
Alexandra Deschamps-Sonsino
 
Cosas conectadas, vidas conectadas, negocios que conectan
Cosas conectadas, vidas conectadas, negocios que conectanCosas conectadas, vidas conectadas, negocios que conectan
Cosas conectadas, vidas conectadas, negocios que conectan
Carat UK
 
Transformaciones lineales
Transformaciones linealesTransformaciones lineales
Transformaciones lineales
jesus alberto abreu mogollon
 
Defeitos da visão humana
Defeitos da visão humanaDefeitos da visão humana
Defeitos da visão humana
Aryleudo De Oliveira
 
4 Key Challenges in the Shift to Digital Recurring Revenue
4 Key Challenges in the Shift to Digital Recurring Revenue4 Key Challenges in the Shift to Digital Recurring Revenue
4 Key Challenges in the Shift to Digital Recurring Revenue
Zuora, Inc.
 
Professional involvement
Professional involvementProfessional involvement
Professional involvement
Bethan Ruddock
 
The Seismic Shift to Recurring Revenue Models & What It Means For Your Company
The Seismic Shift to Recurring Revenue Models & What It Means For Your CompanyThe Seismic Shift to Recurring Revenue Models & What It Means For Your Company
The Seismic Shift to Recurring Revenue Models & What It Means For Your Company
Zuora, Inc.
 
Owning Your Recruiting
Owning Your RecruitingOwning Your Recruiting
Owning Your Recruiting
Daniel Portillo
 
Predictable Revenue. Predictable Risk? Sales Tax & Recurring Revenue
Predictable Revenue. Predictable Risk? Sales Tax & Recurring RevenuePredictable Revenue. Predictable Risk? Sales Tax & Recurring Revenue
Predictable Revenue. Predictable Risk? Sales Tax & Recurring Revenue
Zuora, Inc.
 
Reunió del segon trimestre
Reunió del segon trimestre Reunió del segon trimestre
Reunió del segon trimestre
Jose María Díaz-Crespo Ramírez
 
Презентация к проекту "Создание мультфильма"
Презентация к проекту "Создание мультфильма"Презентация к проекту "Создание мультфильма"
Презентация к проекту "Создание мультфильма"
kargapoltseva
 
Reprodução humana
Reprodução humanaReprodução humana
Reprodução humana
Elisa Margarita Orlandi
 
Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...
Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...
Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...
Millennium Intézet
 
Horváth Ágnes: Válaszút előtt: A magyar energiamix jövője
Horváth Ágnes: Válaszút előtt: A magyar energiamix jövőjeHorváth Ágnes: Válaszút előtt: A magyar energiamix jövője
Horváth Ágnes: Válaszút előtt: A magyar energiamix jövője
Millennium Intézet
 
Dr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívája
Dr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívájaDr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívája
Dr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívája
Millennium Intézet
 
Design for the Network - IA Summit, March 2014 - No Notes Version
Design for the Network - IA Summit, March 2014 - No Notes VersionDesign for the Network - IA Summit, March 2014 - No Notes Version
Design for the Network - IA Summit, March 2014 - No Notes Version
Matthew Milan
 
Anti-Patterns that Stifle Lean UX Teams
Anti-Patterns that Stifle Lean UX TeamsAnti-Patterns that Stifle Lean UX Teams
Anti-Patterns that Stifle Lean UX Teams
Bill Scott
 
Sociedad de la informacion y el conocimiento
Sociedad de la informacion y el conocimientoSociedad de la informacion y el conocimiento
Sociedad de la informacion y el conocimiento
camilo2799
 

Viewers also liked (20)

Be Your Own Technology Brand Ambassador
Be Your Own Technology Brand AmbassadorBe Your Own Technology Brand Ambassador
Be Your Own Technology Brand Ambassador
 
DesignOps Skunk Works - SXSW 2015
DesignOps Skunk Works - SXSW 2015DesignOps Skunk Works - SXSW 2015
DesignOps Skunk Works - SXSW 2015
 
The changing nature of things: the present and future of connected products.
The changing nature of things: the present and future of connected products.The changing nature of things: the present and future of connected products.
The changing nature of things: the present and future of connected products.
 
Cosas conectadas, vidas conectadas, negocios que conectan
Cosas conectadas, vidas conectadas, negocios que conectanCosas conectadas, vidas conectadas, negocios que conectan
Cosas conectadas, vidas conectadas, negocios que conectan
 
Transformaciones lineales
Transformaciones linealesTransformaciones lineales
Transformaciones lineales
 
Defeitos da visão humana
Defeitos da visão humanaDefeitos da visão humana
Defeitos da visão humana
 
4 Key Challenges in the Shift to Digital Recurring Revenue
4 Key Challenges in the Shift to Digital Recurring Revenue4 Key Challenges in the Shift to Digital Recurring Revenue
4 Key Challenges in the Shift to Digital Recurring Revenue
 
Professional involvement
Professional involvementProfessional involvement
Professional involvement
 
The Seismic Shift to Recurring Revenue Models & What It Means For Your Company
The Seismic Shift to Recurring Revenue Models & What It Means For Your CompanyThe Seismic Shift to Recurring Revenue Models & What It Means For Your Company
The Seismic Shift to Recurring Revenue Models & What It Means For Your Company
 
Owning Your Recruiting
Owning Your RecruitingOwning Your Recruiting
Owning Your Recruiting
 
Predictable Revenue. Predictable Risk? Sales Tax & Recurring Revenue
Predictable Revenue. Predictable Risk? Sales Tax & Recurring RevenuePredictable Revenue. Predictable Risk? Sales Tax & Recurring Revenue
Predictable Revenue. Predictable Risk? Sales Tax & Recurring Revenue
 
Reunió del segon trimestre
Reunió del segon trimestre Reunió del segon trimestre
Reunió del segon trimestre
 
Презентация к проекту "Создание мультфильма"
Презентация к проекту "Создание мультфильма"Презентация к проекту "Создание мультфильма"
Презентация к проекту "Создание мультфильма"
 
Reprodução humana
Reprodução humanaReprodução humana
Reprodução humana
 
Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...
Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...
Tóth Bertalan: Időzített bombák és eltékozolt lehetőségek az Orbán-kormány en...
 
Horváth Ágnes: Válaszút előtt: A magyar energiamix jövője
Horváth Ágnes: Válaszút előtt: A magyar energiamix jövőjeHorváth Ágnes: Válaszút előtt: A magyar energiamix jövője
Horváth Ágnes: Válaszút előtt: A magyar energiamix jövője
 
Dr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívája
Dr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívájaDr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívája
Dr. Munkácsy Béla: Az energiaforradalomnak nincs alternatívája
 
Design for the Network - IA Summit, March 2014 - No Notes Version
Design for the Network - IA Summit, March 2014 - No Notes VersionDesign for the Network - IA Summit, March 2014 - No Notes Version
Design for the Network - IA Summit, March 2014 - No Notes Version
 
Anti-Patterns that Stifle Lean UX Teams
Anti-Patterns that Stifle Lean UX TeamsAnti-Patterns that Stifle Lean UX Teams
Anti-Patterns that Stifle Lean UX Teams
 
Sociedad de la informacion y el conocimiento
Sociedad de la informacion y el conocimientoSociedad de la informacion y el conocimiento
Sociedad de la informacion y el conocimiento
 

Similar to Node.js in production

OSCON 2011 - Node.js Tutorial
OSCON 2011 - Node.js TutorialOSCON 2011 - Node.js Tutorial
OSCON 2011 - Node.js Tutorial
Tom Croucher
 
Node js quick-tour_v2
Node js quick-tour_v2Node js quick-tour_v2
Node js quick-tour_v2
tianyi5212222
 
Node js quick-tour_v2
Node js quick-tour_v2Node js quick-tour_v2
Node js quick-tour_v2
http403
 
Node js quick tour v2
Node js quick tour v2Node js quick tour v2
Node js quick tour v2
Wyatt Fang
 
Node.js - The New, New Hotness
Node.js - The New, New HotnessNode.js - The New, New Hotness
Node.js - The New, New Hotness
Daniel Shaw
 
Node.js - async for the rest of us.
Node.js - async for the rest of us.Node.js - async for the rest of us.
Node.js - async for the rest of us.
Mike Brevoort
 
Developing realtime apps with Drupal and NodeJS
Developing realtime apps with Drupal and NodeJS Developing realtime apps with Drupal and NodeJS
Developing realtime apps with Drupal and NodeJS
drupalcampest
 
Dcjq node.js presentation
Dcjq node.js presentationDcjq node.js presentation
Dcjq node.js presentation
async_io
 
JavaScript Event Loop
JavaScript Event LoopJavaScript Event Loop
JavaScript Event Loop
Thomas Hunter II
 
Beginner's Guide to the nmap Scripting Engine - Redspin Engineer, David Shaw
Beginner's Guide to the nmap Scripting Engine - Redspin Engineer, David ShawBeginner's Guide to the nmap Scripting Engine - Redspin Engineer, David Shaw
Beginner's Guide to the nmap Scripting Engine - Redspin Engineer, David Shaw
Redspin, Inc.
 
Ferrara Linux Day 2011
Ferrara Linux Day 2011Ferrara Linux Day 2011
Ferrara Linux Day 2011
Gianluca Padovani
 
Server-Side JavaScript Developement - Node.JS Quick Tour
Server-Side JavaScript Developement - Node.JS Quick TourServer-Side JavaScript Developement - Node.JS Quick Tour
Server-Side JavaScript Developement - Node.JS Quick Tour
q3boy
 
An introduction to node3
An introduction to node3An introduction to node3
An introduction to node3
Vivian S. Zhang
 
Ropython-windbg-python-extensions
Ropython-windbg-python-extensionsRopython-windbg-python-extensions
Ropython-windbg-python-extensions
Alin Gabriel Serdean
 
Practical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsPractical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.js
async_io
 
Node.js 1, 2, 3
Node.js 1, 2, 3Node.js 1, 2, 3
Node.js 1, 2, 3
Jian-Hong Pan
 
Dockerizing the Hard Services: Neutron and Nova
Dockerizing the Hard Services: Neutron and NovaDockerizing the Hard Services: Neutron and Nova
Dockerizing the Hard Services: Neutron and Nova
clayton_oneill
 
Nodejs Intro Part One
Nodejs Intro Part OneNodejs Intro Part One
Nodejs Intro Part One
Budh Ram Gurung
 
GeekCampSG - Nodejs , Websockets and Realtime Web
GeekCampSG - Nodejs , Websockets and Realtime WebGeekCampSG - Nodejs , Websockets and Realtime Web
GeekCampSG - Nodejs , Websockets and Realtime Web
Bhagaban Behera
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 Minutes
Hiroshi SHIBATA
 

Similar to Node.js in production (20)

OSCON 2011 - Node.js Tutorial
OSCON 2011 - Node.js TutorialOSCON 2011 - Node.js Tutorial
OSCON 2011 - Node.js Tutorial
 
Node js quick-tour_v2
Node js quick-tour_v2Node js quick-tour_v2
Node js quick-tour_v2
 
Node js quick-tour_v2
Node js quick-tour_v2Node js quick-tour_v2
Node js quick-tour_v2
 
Node js quick tour v2
Node js quick tour v2Node js quick tour v2
Node js quick tour v2
 
Node.js - The New, New Hotness
Node.js - The New, New HotnessNode.js - The New, New Hotness
Node.js - The New, New Hotness
 
Node.js - async for the rest of us.
Node.js - async for the rest of us.Node.js - async for the rest of us.
Node.js - async for the rest of us.
 
Developing realtime apps with Drupal and NodeJS
Developing realtime apps with Drupal and NodeJS Developing realtime apps with Drupal and NodeJS
Developing realtime apps with Drupal and NodeJS
 
Dcjq node.js presentation
Dcjq node.js presentationDcjq node.js presentation
Dcjq node.js presentation
 
JavaScript Event Loop
JavaScript Event LoopJavaScript Event Loop
JavaScript Event Loop
 
Beginner's Guide to the nmap Scripting Engine - Redspin Engineer, David Shaw
Beginner's Guide to the nmap Scripting Engine - Redspin Engineer, David ShawBeginner's Guide to the nmap Scripting Engine - Redspin Engineer, David Shaw
Beginner's Guide to the nmap Scripting Engine - Redspin Engineer, David Shaw
 
Ferrara Linux Day 2011
Ferrara Linux Day 2011Ferrara Linux Day 2011
Ferrara Linux Day 2011
 
Server-Side JavaScript Developement - Node.JS Quick Tour
Server-Side JavaScript Developement - Node.JS Quick TourServer-Side JavaScript Developement - Node.JS Quick Tour
Server-Side JavaScript Developement - Node.JS Quick Tour
 
An introduction to node3
An introduction to node3An introduction to node3
An introduction to node3
 
Ropython-windbg-python-extensions
Ropython-windbg-python-extensionsRopython-windbg-python-extensions
Ropython-windbg-python-extensions
 
Practical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsPractical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.js
 
Node.js 1, 2, 3
Node.js 1, 2, 3Node.js 1, 2, 3
Node.js 1, 2, 3
 
Dockerizing the Hard Services: Neutron and Nova
Dockerizing the Hard Services: Neutron and NovaDockerizing the Hard Services: Neutron and Nova
Dockerizing the Hard Services: Neutron and Nova
 
Nodejs Intro Part One
Nodejs Intro Part OneNodejs Intro Part One
Nodejs Intro Part One
 
GeekCampSG - Nodejs , Websockets and Realtime Web
GeekCampSG - Nodejs , Websockets and Realtime WebGeekCampSG - Nodejs , Websockets and Realtime Web
GeekCampSG - Nodejs , Websockets and Realtime Web
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 Minutes
 

Recently uploaded

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 

Recently uploaded (20)

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 

Node.js in production

  • 2. About Contributor Co-founder Felix Geisendörfer node.js driver node-formidable - Joined the mailing list in June 26, 2009 - Co-Transloadit - When I joined there where #24 people - First patch in September 2009 - Now mailing list has > 3400 members
  • 3. File uploading & processing as a service for other web applications. - The app we run in production - Not a “typical” app, no html rendering, only JSON APIs
  • 4. Are you running node in production?
  • 7. ~1 Ye Transloadit v1 a r ag o • Complete failure • Node randomly crashed with: (evcom) recv() Success • A mess of code, hard to maintain * http://transloadit.com/blog/2010/04/to-launch-or-not
  • 10. N Transloadit v2 ow • Processed > 2 TB of data • Not seen a node bug in production yet • Clean code base, 100% test coverage
  • 11. How to get there?
  • 13. Managed / Heroku-style • Joyent (no.de) • Nodejitsu (nodejitsu.com) • Duostack (duostack.com) • Nodefu (nodefu.com) • Heroku (heroku.com)
  • 14. Self-managed • Amazon Ec2 • Linode • Rackspace Cloud Servers • etc.
  • 16. Deployment • Heroku service • Custom deploy script
  • 17. Custom “deploy script” $ git clone <my-project> /var/www/<my-project> $ nohup bin/server.js & Enough to get started
  • 19. The problem SyntaxError: Unexpected token ILLEGAL at Object.parse (native) at Server.<anonymous> (/path/to/my-script.js:4:8) at Server.emit (events.js:45:17) at HTTPParser.onIncoming (http.js:862:12) at HTTPParser.onHeadersComplete (http.js:85:31) at Socket.ondata (http.js:787:22) at Socket._onReadable (net.js:623:27) at IOWatcher.onReadable [as callback] (net.js:156:10) Your node process is killed by any runtime error
  • 20. The wrong solution process.on('uncaughtException', function(err) { console.log('Something bad happened: '+err); // continue on ... }); Continuing after an exception will leave your system in unpredictable state
  • 21. The right solution process.on('uncaughtException', function(err) { console.log('Something bad happened: '+err); // Every process has to die one day, it’s life process.exit(0); }); Exiting the program will prevent corrupted state
  • 22. Even better var Hoptoad = require('hoptoad-notifier').Hoptoad; Hoptoad.key = 'YOUR_API_KEY'; process.on('uncaughtException', function(err) { console.log('Something bad happened: '+err); Hoptoad.notify(err, function() { process.exit(0); }); }); Send your exception to a logging service.
  • 23. Your process is dead. Now what?
  • 24. Use a process monitor • Monit * • Upstart • God (ruby) • forever (node.js) * http://kevin.vanzonneveld.net/techblog/article/run_nodejs_as_a_service_on_ubuntu_karmic/
  • 26. Debugging tools • console.log • node-inspector (use webkit inspector.) • node debug <script.js> (new in 0.3.x)
  • 27. Unit tests (Real ones, not integration tests)
  • 29. Load Balancing • Haproxy • Amazon Load Balancer • Nginx (supports only http 1.0)
  • 31. Our stack Ec2 Load Balancer :80 :443 :80 :443 :80 :443 haproxy stunnel haproxy stunnel haproxy stunnel node nginx node nginx node nginx php php php
  • 32. Workers • Use workers a la Unicorn • All workers listen on a shared socket • Master process starts & kills workers
  • 34. master.js var netBinding = process.binding('net'); var spawn = require('child_process').spawn; var net = require('net'); var serverSocket = netBinding.socket('tcp4'); netBinding.bind(serverSocket, 8080); netBinding.listen(serverSocket, 128); for (var i = 0; i < 10; i++) { var fds = netBinding.socketpair(); var child = spawn(process.argv[0], [__dirname+'/worker.js'], { customFds: [fds[0], 1, 2] }); var socket = new net.Stream(fds[1], 'unix'); socket.write('hi', 'utf8', serverSocket); }
  • 35. worker.js var net = require('net'); var http = require('http'); var socket = new net.Socket(0, 'unix'); socket .on('fd', function(fd) { startServer(fd); }) .resume(); function startServer(fd) { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello from: '+process.pid+'n'); }).listenFD(fd); console.log('Worker ready: '+process.pid); }
  • 36. Questions? ✎ @felixge / felix@debuggable.com