Node.js !

in a heterogeneous system

Alexey Migutsky : Senior Software Developer @ Lohika
What do we do?
What do we do?

Ubix.io
PaaS for BigData analytics in cloud
environments.
What do we use?
Scala
Python

What do we use?

Bash
Javascript
BDAS (Spark + Shark)
AWS

What do we use?

Black Voodoo Magic™
Velocipede™
Brains
What’s next?
Node & Maven

Jenkins & Node testing

What’s next?

Velocipede™
CI
?
Why Node.js?
Why Node.js?
Fast and easy
Easy websockets support
Plays nice with AWS
We use it as a web-tier orchestrator
Node & Maven
Node & Maven
We use maven (pom.xml)
We use npm (packege.json)
We have node+maven projects!
Project structure
Project structure
+ Subprojects are maven modules
+ Subprojects use common code base
+ Easy « build process »
+ Easy deplo...
Custom modules
Custom modules
Live in root /node_modules right now
Planning to move to separate git repos
Tested with subprojects
Tests
Tests
Mocha + Sinon + Chai
Sinon-chai + mocha-sinon
Chai-as-promised
Rewired
Jenkins TAP plugin
Test example
describe('##getConfiguration', function () {!
it('should return promise and resolve with data ', function (do...
Test Env How-to
Test Env How-to
Goals:
- 
- 
- 
- 
- 

run from root using mvn test
run from sub using npm test
human-readable result logs...
Test Env How-to
package.json
"devDependencies": {	
  
"mocha": "*",	
  
"sinon": "1.6.*",	
  
"chai": "*",	
  
"sinon-chai...
Test Env How-to
runtest.sh

#!/bin/bash!
mocha $(find test -name '*.js') -R tap | !
tee test/testresults.tap; (exit ${PIPE...
Test Env How-to
pom.xml (maven-antrun-plugin)

<execution>!
<id>test-nodejs</id>!
<phase>test</phase>!
<configuration>!
<t...
Build process
Build process
-  No npm install on build server (too slooow)
-  Fixed deps versions (update by-hand)
-  All deployable dep...
Deployment
Deployment
-  Builds are stored at S3
-  Copied to instances by orchestration tools
-  Managed by upstart
Deployment
upstart (server.conf)
	
  	
  

#this should be put in /etc/init/server.conf on Ubuntu !
description ”Node.js s...
Deployment hint
Have a look at http://pm2.io/
Continuous Integration
« hack some node.js »
npm test
de
h co
pus
de
h co
pus
de
h co
pus

build & test
de
h co
pus

build & test
store
de
h co
pus

build & test
store
de
h co
pus

build & test
store
de
h co
pus

build & test
store

deplo

y
That’s it!
That’s it!
Thanks

@mr_mig_by
bit.ly/mr_mig
Node.js in a heterogeneous system
Node.js in a heterogeneous system
Upcoming SlideShare
Loading in …5
×

Node.js in a heterogeneous system

1,331 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,331
On SlideShare
0
From Embeds
0
Number of Embeds
477
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Node.js in a heterogeneous system

  1. 1. Node.js ! in a heterogeneous system Alexey Migutsky : Senior Software Developer @ Lohika
  2. 2. What do we do?
  3. 3. What do we do? Ubix.io PaaS for BigData analytics in cloud environments.
  4. 4. What do we use?
  5. 5. Scala Python What do we use? Bash Javascript
  6. 6. BDAS (Spark + Shark) AWS What do we use? Black Voodoo Magic™ Velocipede™ Brains
  7. 7. What’s next?
  8. 8. Node & Maven Jenkins & Node testing What’s next? Velocipede™ CI
  9. 9. ?
  10. 10. Why Node.js?
  11. 11. Why Node.js? Fast and easy Easy websockets support Plays nice with AWS We use it as a web-tier orchestrator
  12. 12. Node & Maven
  13. 13. Node & Maven We use maven (pom.xml) We use npm (packege.json) We have node+maven projects!
  14. 14. Project structure
  15. 15. Project structure + Subprojects are maven modules + Subprojects use common code base + Easy « build process » + Easy deployments - Uber root package.json - Mangled dependencies
  16. 16. Custom modules
  17. 17. Custom modules Live in root /node_modules right now Planning to move to separate git repos Tested with subprojects
  18. 18. Tests
  19. 19. Tests Mocha + Sinon + Chai Sinon-chai + mocha-sinon Chai-as-promised Rewired Jenkins TAP plugin
  20. 20. Test example describe('##getConfiguration', function () {! it('should return promise and resolve with data ', function (done) {! var promise = sysStore.getConfiguration('somekey');! promise.should.eventually.become({key: 'somekey'})! " .then(function () {! mockDb.getItem.should.be.calledWithMatch(! ! mockTables.system, ! " " {key: sinon.match.string}! " ");! })! .should.notify(done);! });! });  
  21. 21. Test Env How-to
  22. 22. Test Env How-to Goals: -  -  -  -  -  run from root using mvn test run from sub using npm test human-readable result logs result >> in console and file Jenkins support http://www.mindmeister.com/346337860
  23. 23. Test Env How-to package.json "devDependencies": {   "mocha": "*",   "sinon": "1.6.*",   "chai": "*",   "sinon-chai": "*",   "mocha-sinon": "*",   "chai-as-promised" : "*",   "rewire" : "*"   },   "scripts" : {   "test" : "./runtest.sh"   }  
  24. 24. Test Env How-to runtest.sh #!/bin/bash! mocha $(find test -name '*.js') -R tap | ! tee test/testresults.tap; (exit ${PIPESTATUS[0]})   The whole command runs in a single line. Wrapped here for display purpose J
  25. 25. Test Env How-to pom.xml (maven-antrun-plugin) <execution>! <id>test-nodejs</id>! <phase>test</phase>! <configuration>! <tasks name="Run mocha tests">! <exec dir="${basedir}”! executable="npm”! failonerror="true">! <arg value="test"/>! </exec>! </tasks>! </configuration>! <goals>! <goal>run</goal>! </goals>! </execution>  
  26. 26. Build process
  27. 27. Build process -  No npm install on build server (too slooow) -  Fixed deps versions (update by-hand) -  All deployable deps are checked in! -  Build phase only copies deps in subprojects http://www.futurealoof.com/posts/nodemodules-in-git.html
  28. 28. Deployment
  29. 29. Deployment -  Builds are stored at S3 -  Copied to instances by orchestration tools -  Managed by upstart
  30. 30. Deployment upstart (server.conf)     #this should be put in /etc/init/server.conf on Ubuntu ! description ”Node.js server" ! author "Alexey Migutsky" ! ! # used to be: start on startup ! # until we found some mounts weren't ready yet while booting: ! start on started mountall ! stop on shutdown ! ! # Automatically Respawn: ! respawn ! respawn limit 20 5 ! ! # Max open files are @ 1024 by default. Bit few.! limit nofile 32768 32768 ! ! ! script ! "# Not sure why $HOME is needed, but we found that it is: "! "export HOME="/root”! "exec /usr/local/bin/node /usr/local/server/app.js >> /var/log/server-err.log 2>&1 ! end script ! post-start script end script ! http://kvz.io/blog/2009/12/15/run-nodejs-as-a-service-on-ubuntu-karmic/
  31. 31. Deployment hint Have a look at http://pm2.io/
  32. 32. Continuous Integration
  33. 33. « hack some node.js » npm test
  34. 34. de h co pus
  35. 35. de h co pus
  36. 36. de h co pus build & test
  37. 37. de h co pus build & test store
  38. 38. de h co pus build & test store
  39. 39. de h co pus build & test store
  40. 40. de h co pus build & test store deplo y
  41. 41. That’s it!
  42. 42. That’s it! Thanks @mr_mig_by bit.ly/mr_mig

×