Download as PDF, PPTX
![PROTOCOL
Request
json body
target
headers
body/post-data
loginfo
request = {
URL = "http://www.citigroup.net/",
method = "GET",
timeout = 19500,
clientInfo = {
identifier = “…E”,
model = "iPad Simulator",
systemName = "iPhone OS",
systemVersion = "7.1",
},
headers = {
Accept = "text/html,application/xhtml
+xml,application/xml;q=0.9,*/*;q=0.8",
Cookie = "CGPLNG=ENG; JSESSIONID_CGNR3=..”,
"User-Agent" = "Mozilla/5.0 (iPad; CPU OS 7_1 like
Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/
11D167"
},
logEntries = [
{
URL = “https://cinternal.site/target/fooa”,
downstreamDuration = 656,
httpMethod = "GET",
roundtripDuration = 3461,
statusCode = 200
} ]
}](https://image.slidesharecdn.com/akohli-nodeconfeu-publish-140916070619-phpapp01/85/The-Real-World-Plugging-the-Enterprise-Into-It-nodejs-18-320.jpg)
![RESPONSE
body = “<base64>",
code = 200,
duration = 31,
headers = {
"Accept-Ranges" = [
"bytes"
],
"Content-Length" = [
225
],
"Content-Type" = [
"text/html"
],
Date = [
"Thu, 29 May 2014 15:28:29 GMT"
],
Etag = [
""e1-4e50c74f""
],
"Last-Modified" = [
"Sun, 21 Aug 2011 08:52:31 GMT"
]
},
message = "OK"
}](https://image.slidesharecdn.com/akohli-nodeconfeu-publish-140916070619-phpapp01/85/The-Real-World-Plugging-the-Enterprise-Into-It-nodejs-19-320.jpg)
![FLOD OUTPUT
## 6k page results
ec2-user@ip-10-199-51-233 node-hapi]$ flod -n 2000 -t 1500 -c 100..1000 -v http://localhost/loremipsum-6k-ish.html
This is Flod, version 0.2.2
Copyright 2013 Walmart, http://github.com/spumko/flod
!
Benchmarking (hold on)...
!
Server Requests/sec Latency (ms)
--------------------------------------- ------------ ---------------
http://localhost/loremipsum-6k-ish.html 100 96.48 ± 18.54
http://localhost/loremipsum-6k-ish.html 200 164.24 ± 17.03
http://localhost/loremipsum-6k-ish.html 300 263.80 ± 62.44
http://localhost/loremipsum-6k-ish.html 400 359.61 ± 49.20
http://localhost/loremipsum-6k-ish.html 500 437.66 ± 58.69
http://localhost/loremipsum-6k-ish.html 600 481.29 ± 120.04
http://localhost/loremipsum-6k-ish.html 700 606.74 ± 114.45
http://localhost/loremipsum-6k-ish.html 800 555.08 ± 133.74
http://localhost/loremipsum-6k-ish.html 900 674.08 ± 190.91
http://localhost/loremipsum-6k-ish.html 1000 763.27 ± 69.25
## running with high timeout - doubling responses times vs nginx direct
[ec2-user@ip-10-199-51-233 node-hapi]$ ../node_modules/flod/bin/flod -n 2000 -t 4500 -c 100..1000 -v http://localhost:8000
This is Flod, version 0.2.2
Copyright 2013 Walmart, http://github.com/spumko/flod
!
Benchmarking (hold on)...
!
Server Requests/sec Latency (ms)
--------------------- ------------ ----------------
http://localhost:8000 100 200.55 ± 39.40
http://localhost:8000 200 389.54 ± 67.39
http://localhost:8000 300 558.14 ± 112.57
http://localhost:8000 400 777.09 ± 160.01
http://localhost:8000 500 970.61 ± 305.76
http://localhost:8000 600 1032.37 ± 274.44
http://localhost:8000 700 1216.49 ± 249.94
http://localhost:8000 800 1483.31 ± 690.64
http://localhost:8000 900 1559.54 ± 805.31
http://localhost:8000 1000 1909.23 ± 845.81](https://image.slidesharecdn.com/akohli-nodeconfeu-publish-140916070619-phpapp01/85/The-Real-World-Plugging-the-Enterprise-Into-It-nodejs-25-320.jpg)
Uploaded byAman Kohli
The Real World - Plugging the Enterprise Into It (nodejs)
This document discusses using Node.js as the foundation for building applications that connect the physical world to enterprise systems through mobile devices and sensors. It describes initial work done to build a proxy and protocol for handling requests and addresses challenges with authentication, scalability, and performance testing. The document shares results from benchmarking the system under different network conditions and outlines next steps to improve concurrency, security, and infrastructure elasticity.
![[Js hcm] Deploying node.js with Forever.js and nginx](https://cdn.slidesharecdn.com/ss_thumbnails/js-hcmdeployingnode-131016220832-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
The Real World - Plugging the Enterprise Into It (nodejs)
- 1. THE REAL WORLD Plugging The Enterprise Into It.
- 2. MOBILE ARCHITECT ‣LoveDistributed Systems ‣Entropy Reducer ‣Payment systems ‣R&D Work ‣B2E and Commercial Banking Apps Experience ‣ Front Office Trading Systems ‣ Messaging Middleware Integration ‣ Big Systems ‣ C/C++/C#/Java MORE ‣ @akohli https://slideshare.net/akohli series 2, episode 22, “Daddy Pig’s Office” http:// www.channel5.com/shows/peppa-pig/episodes/daddy-pigs-office
- 3. TODAY Why Node What we want to do Node as the underpinning of real world or electronic asset interaction Backing our interactions, eventing services Not so much about monolith deconstruction What we did Initial proxy and protocol Our performance and scalability testing
- 5.
- 7. WHY NODE? ✔Node • Asynchronous Eventing Model • We live in an async nonblocking world • Ideal for mobile and sensor applications • Everyone knows Javascript, right? • Community • Diverse protocol and lots of modules • Rapid development and Expediency
- 8. HOMOLOGATED or howwe can use it in a big company • Node is approved for internal usage • Less Yak Shaving than other solutions • different at least • good internal community beware of dog, staff only
- 9. “Walmart has hadgood success with HAPI and Node” - @adam_baldwin “Node is good. I’ve heard good things - @ eoinbrazil about HAPI”
- 10.
- 11. ENTERPRISE MOBILE APPLICATIONS • Plurality of systems, services • web resources • web sites • Connectivity challenges • direct • mediated • Security • AuthN • AuthZ • Data Encryption at rest
- 12. Security Pass SensorsEmployee Devices The Physical World THE REFLEKTOR Security Services AuthZ AuthN … Eventing Engine Bridge Payment Services Access Services Printing Services the Reflektor Bridge and New Services App Services and Resources
- 13.
- 14. IT AIN’T EASY but we gotta try
- 15.
- 16.
- 17. THE FLOW •The Protocol • Security - Gateway Access • Federated Identity, my foot • NTLM, has it’s own approach
- 18. PROTOCOL Request jsonbody target headers body/post-data loginfo request = { URL = "http://www.citigroup.net/", method = "GET", timeout = 19500, clientInfo = { identifier = “…E”, model = "iPad Simulator", systemName = "iPhone OS", systemVersion = "7.1", }, headers = { Accept = "text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8", Cookie = "CGPLNG=ENG; JSESSIONID_CGNR3=..”, "User-Agent" = "Mozilla/5.0 (iPad; CPU OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/ 11D167" }, logEntries = [ { URL = “https://cinternal.site/target/fooa”, downstreamDuration = 656, httpMethod = "GET", roundtripDuration = 3461, statusCode = 200 } ] }
- 19. RESPONSE body =“<base64>", code = 200, duration = 31, headers = { "Accept-Ranges" = [ "bytes" ], "Content-Length" = [ 225 ], "Content-Type" = [ "text/html" ], Date = [ "Thu, 29 May 2014 15:28:29 GMT" ], Etag = [ ""e1-4e50c74f"" ], "Last-Modified" = [ "Sun, 21 Aug 2011 08:52:31 GMT" ] }, message = "OK" }
- 20. NTLM AUTHENTICATION Enterprise authentication protocol (Microsoft). ! NTLM requires all phases to take place across a single HTTP connection. ! NTLM messages are sent and received as request headers. ! The server’s response from the NTLM type 3 message is the requested content. ! This authentication process must be completed for every requested resource, unless an open connection is maintained.
- 21. WORKING Implementation Challenges • Storage of password on mobile device is prohibited, but is required in the authentication process. • Persistent connection not available. • Latency issues – 3 requests for every web resource. Solution • Ported from Apache Java implementation to Node.js. • Hashed username / password pair stored on device, transmitted to server for authentication rather than raw password. • hmac_md5(username, md4(password)) • NTLM message calculation split between client app and proxy server. • Defaults used and optional parameters omitted – simplified messages. • Observed desktop browsers wait for a 401 before beginning the authentication process. Pre-emptively sending the username / password hash eliminates the initial 401 response. Process is reduced from 3 direct requests to a single client request, mapped to 2 proxy requests.
- 22. SCALABILITY AND PERFORMANCETESTING
- 23. GITHUB.COM/SPUMKO/FLOD $ flod-n 2000 -t 1500 -c 100..1000 -v http://target-place ! !
- 24. GITHUB.COM/SPUMKO/FLOD num reqper batch range of concurrent request per batch - $ flod -n 2000 -t 1500 -c 100..1000 -v http://target-place ! ! timeout “rate”
- 25. FLOD OUTPUT ##6k page results ec2-user@ip-10-199-51-233 node-hapi]$ flod -n 2000 -t 1500 -c 100..1000 -v http://localhost/loremipsum-6k-ish.html This is Flod, version 0.2.2 Copyright 2013 Walmart, http://github.com/spumko/flod ! Benchmarking (hold on)... ! Server Requests/sec Latency (ms) --------------------------------------- ------------ --------------- http://localhost/loremipsum-6k-ish.html 100 96.48 ± 18.54 http://localhost/loremipsum-6k-ish.html 200 164.24 ± 17.03 http://localhost/loremipsum-6k-ish.html 300 263.80 ± 62.44 http://localhost/loremipsum-6k-ish.html 400 359.61 ± 49.20 http://localhost/loremipsum-6k-ish.html 500 437.66 ± 58.69 http://localhost/loremipsum-6k-ish.html 600 481.29 ± 120.04 http://localhost/loremipsum-6k-ish.html 700 606.74 ± 114.45 http://localhost/loremipsum-6k-ish.html 800 555.08 ± 133.74 http://localhost/loremipsum-6k-ish.html 900 674.08 ± 190.91 http://localhost/loremipsum-6k-ish.html 1000 763.27 ± 69.25 ## running with high timeout - doubling responses times vs nginx direct [ec2-user@ip-10-199-51-233 node-hapi]$ ../node_modules/flod/bin/flod -n 2000 -t 4500 -c 100..1000 -v http://localhost:8000 This is Flod, version 0.2.2 Copyright 2013 Walmart, http://github.com/spumko/flod ! Benchmarking (hold on)... ! Server Requests/sec Latency (ms) --------------------- ------------ ---------------- http://localhost:8000 100 200.55 ± 39.40 http://localhost:8000 200 389.54 ± 67.39 http://localhost:8000 300 558.14 ± 112.57 http://localhost:8000 400 777.09 ± 160.01 http://localhost:8000 500 970.61 ± 305.76 http://localhost:8000 600 1032.37 ± 274.44 http://localhost:8000 700 1216.49 ± 249.94 http://localhost:8000 800 1483.31 ± 690.64 http://localhost:8000 900 1559.54 ± 805.31 http://localhost:8000 1000 1909.23 ± 845.81
- 26. MODIFYING FLOD •modified server to pull our decorated response timing information • modified reporting/logging to include this information • hope to contribute back to mainline
- 27. ENVIRONMENT Machine OSType Processor Cores Memory Int Server RHEL 6.4 VM Xeon 2.6GHz 2 4GB Prod Server Windows Server 2k8r2 VM Xeon 1.8 Ghz 4 6GB Dev Mac Mini Full i5 2.5 GHz 2 8 GB • HTTP 1.1 no Keep-Alive, request payload is json • Client iOS ObjectiveC;Server is Node + Hapijs (with Some Good Monitoring)
- 28. SCENARIOS • Closednetwork, direct connection, Mac to Mac • Client server on a redhat VM, loopback. Redhat VM • Redhat client to Windows Server via network, Redhat to Windows • via Mobile network/wifi could only support 100 transactions/s because of latency Req/s Response (ms) Mac to Mac 1000 2000 Redhat VM 1000 8500 RH to Windows 1000 30, 000 External 100 17, 000
- 29. RESULTS • Consistentproxied service response • ~20ms Mac ➔ Mac • ~250ms RHEL ➔ Windows Server • Gateway service < 50 ms • We need better concurrency, request servicing • Infrastructure adds significant overhead
- 30. RESULTS 100# 200#300# 400# 500# 600# 700# 800# 900# 1000# 70000# 60000# 50000# 40000# 30000# 20000# 10000# 0# Requests'per'second' Milliseconds(ms)' 99th'Percen7les' Mac#To#Mac# RedhatVM# Redhat#to#Windows#
- 31.
- 32.
- 33. TABULAR Request MacTo Mac RHEL to Windows 100/s 483 17,235 200/s 783 21,209 300/s 1127 17,479 400/s 1493 22,937 500/s 1859 28,872 600/s 2171 34,253 700/s 2487 39,878 800/s 2878 46,970 900/s 3285 51,083 1000/s 3555 57,189
- 34.
- 35. EXPERIENCE • Enterpriseand Legal approvals hard • We are ahead of Ops, so waiting for VMs and infrastructure to catch up - software, machines, and network • Some bits of node need tightening - especially around security and password storage • Still learning and it is fun!
- 36. SCALABILITY PACKETS •Pile of VMs to auto-scale • Need elastic environment with a smart load balancer and configuration management • Great Details on Best practice • https://gist.github.com/hueniverse/7686452
- 37.
- 38.
- 39. NOUN PROJECTS THANKS Smartphone designed by James Fenton from the Noun Project ! Creative Commons – Attribution (CC BY 3.0) Identification designed by Mark Shorter from the Noun Project Ibeacon designed by Stéphanie Rusch from the Nount Project ! Creative Commons – Attribution (CC BY 3.0) Arduino designed by uizin from the Noun Project !