EBAY’S

CUBEJS:

NODE.JS ADOPTION JOURNEY

@
Patrick Steele-Idem
Senior Platform Architect @

@psteeleidem
psteeleidem@ebay.com
github.com/patrick-steele-idem
EBAY HAS EMBRACED NODE.JS…
SO HOW DID WE GET HERE?
BEFORE NODE.JS, JAVA WAS KING
BUT THINGS EVOLVED…
XML + XSLT  JAVA
PROPRIETARY  OPEN SOURCE
ONE LANGUAGE  POLYGLOT
ONE ...
“A DECLARATIVE, DATA-RETRIEVAL AND
AGGREGATION GATEWAY FOR QUICKLY
CONSUMING HTTP APIS”
…BUILT USING NODE.JS

WHY NODE.JS?...
NODE.JS IS GREAT,
BUT CAN EBAY USE IT TO
BUILD FRONT-ENDS AS WELL?
WHY NODE.JS FOR THE FRONT-END?
DEVELOPER AGILITY, SCALABILITY AND
PERFORMANCE
―My advice: always bet on JavaScript‖
–Brend...
WHAT IS REQUIRED TO SUPPORT
NODE.JS AT EBAY?
Tracking

Monitoring
Security

Encryption/Decryption

I18n Testing

Deploymen...
POLYGLOT CHALLENGE:
HUGE INVESTMENT IN JAVA CODE
Possible Solutions

Pros

Cons

Port code to
JavaScript

• Works for Node...
1ST CHOICE: EXPOSE AS A SERVICE
MAXIMIZE CODE REUSE
CALLS CAN BE MADE IN PARALLEL

2ND CHOICE: PORT CODE TO JAVASCRIPT
PER...
RUNTIME ARCHITECTURE
Application Layer

Infrastructure Layer
Resource
Server
Distributed
Logging

Tracking
cluster2

Clien...
DEVELOPMENT AND DEPLOYMENT
INFRASTRUCTURE

Source Code

an

initiative
powered by:

npm-delegate
Private npm

Public npm
DESIGN GOAL:
MAXIMIZE DEVELOPER PRODUCTIVITY
DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY

Small, bite-sized and easy-to-digest
modules
Keep the documentation with the...
DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY

Avoid “magic”
If developers cannot reason about a system
then they will not ...
DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY

+ File Watching
+ Hot Reloading
+ Web Sockets
Live Coding
Instant Gratificat...
DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY

NO BUILD STEP FOR RESOURCE
OPTIMIZATION!

Resource
bundling, minification, c...
DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY

Command Line Tools
Powered by Rápido
Features
• Module and application lifec...
DESIGN GOAL:
HIGH PERFORMANCE
DESIGN GOAL: HIGH PERFORMANCE

PERFORMANCE IS GREAT*
• Node.js provides a very low-level programming platform

• V8 is a p...
DESIGN GOAL: HIGH PERFORMANCE

STREAMING, ASYNCHRONOUS
UI RENDERING
• Why?
•
•
•
•

Start rendering the template immediate...
DESIGN GOAL: HIGH PERFORMANCE

Traditional

ASYNCHRONOUS UI RENDERING
1. Make service calls
2. Wait for all data to come b...
DESIGN GOAL: HIGH PERFORMANCE

Raptor Templates
Extensible, high performance, asynchronous template
rendering engine for J...
DESIGN GOAL:
ROBUST
DESIGN GOAL: ROBUST

cluster2
•
•
•
•
•
•
•

Multi-process cluster management
Live production debugging
Monitoring
Idle ti...
DESIGN GOAL: ROBUST

Monitoring Infrastructure

Adapted to eBay Ops
Monitoring Tools
DESIGN GOAL: ROBUST

Automated Testing
• Netmorphic
• Simulate network failures and slowness
• Mocha
• Asynchronous tests
...
DESIGN GOAL: ROBUST

Configuration Management
• Per-module configuration
• Real-time configuration updates
• Configuration...
ASYNCHRONOUS PROGRAMMING
USING PROMISES
All CubeJS modules have standardized on promises for
asynchronous programming.
Exa...
ADOPTION:
PULL RATHER

THAN

PUSH

• 10+ customers before CubeJS goes GA
• Several live CubeJS applications running in
pro...
QUESTIONS?
psteeleidem@ebay.com
@psteeleidem
https://github.com/cubejs
http://raptorjs.org
Upcoming SlideShare
Loading in...5
×

CubeJS: eBay’s Node.js Adoption Journey

6,918

Published on

CubeJS is a Node.js based application development platform at eBay. In this presentation, we will share our journey to build and launch this new platform at the eBay Marketplaces. We will discuss CubeJS’ overall architecture, versatile functionalities, robust operationalization, and streamlined development lifecycle process, as well as its customer adoption momentum in the company. We will also share a number of high-profile projects, powered on CubeJS, to be debuted by eBay Inc. ranging from Web to Mobile.

Published in: Technology
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,918
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
36
Comments
0
Likes
18
Embeds 0
No embeds

No notes for slide

Transcript of "CubeJS: eBay’s Node.js Adoption Journey"

  1. 1. EBAY’S CUBEJS: NODE.JS ADOPTION JOURNEY @
  2. 2. Patrick Steele-Idem Senior Platform Architect @ @psteeleidem psteeleidem@ebay.com github.com/patrick-steele-idem
  3. 3. EBAY HAS EMBRACED NODE.JS… SO HOW DID WE GET HERE?
  4. 4. BEFORE NODE.JS, JAVA WAS KING BUT THINGS EVOLVED… XML + XSLT  JAVA PROPRIETARY  OPEN SOURCE ONE LANGUAGE  POLYGLOT ONE IDE  ANY EDITOR/IDE HEAVY FRONT-END  LIGHTWEIGHT FRONT-END CLEARCASE  GIT + GITHUB SLOW DEVELOPMENT  RAPID DEVELOPMENT JAVA APIS  SERVICE APIS
  5. 5. “A DECLARATIVE, DATA-RETRIEVAL AND AGGREGATION GATEWAY FOR QUICKLY CONSUMING HTTP APIS” …BUILT USING NODE.JS WHY NODE.JS? • Non-blocking I/O • Single-threaded
  6. 6. NODE.JS IS GREAT, BUT CAN EBAY USE IT TO BUILD FRONT-ENDS AS WELL?
  7. 7. WHY NODE.JS FOR THE FRONT-END? DEVELOPER AGILITY, SCALABILITY AND PERFORMANCE ―My advice: always bet on JavaScript‖ –Brendan Eich
  8. 8. WHAT IS REQUIRED TO SUPPORT NODE.JS AT EBAY? Tracking Monitoring Security Encryption/Decryption I18n Testing Deployment Personalization Resource Optimization Release Process Look & Feel CSRF Tools Scalability SOAP REST Logging Configuration Authentication App Metadata
  9. 9. POLYGLOT CHALLENGE: HUGE INVESTMENT IN JAVA CODE Possible Solutions Pros Cons Port code to JavaScript • Works for Node.js • Easy to debug • Tedious • Error prone • What about Python, Ruby, etc.? Expose as a service • Polyglot friendly • • • • Latency Monitoring Network Failures Infrastructure Port code to C/C++ • Fast • Bindings available • Difficult to debug • Difficult to maintain JVM ↔ JavaScript Bridge • Polyglot friendly • Difficult to debug • Slower startup • Monitoring multiple VMs
  10. 10. 1ST CHOICE: EXPOSE AS A SERVICE MAXIMIZE CODE REUSE CALLS CAN BE MADE IN PARALLEL 2ND CHOICE: PORT CODE TO JAVASCRIPT PERFORMANCE CRITICAL BEST HANDLED LOCALLY
  11. 11. RUNTIME ARCHITECTURE Application Layer Infrastructure Layer Resource Server Distributed Logging Tracking cluster2 Client Layer … App Service Layer Search …
  12. 12. DEVELOPMENT AND DEPLOYMENT INFRASTRUCTURE Source Code an initiative powered by: npm-delegate Private npm Public npm
  13. 13. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY
  14. 14. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY Small, bite-sized and easy-to-digest modules Keep the documentation with the code. README files for the win Encourage contributions
  15. 15. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY Avoid “magic” If developers cannot reason about a system then they will not be happy or productive. Do not be a restrictive framework Developers love options and being able to use the latest and greatest.
  16. 16. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY + File Watching + Hot Reloading + Web Sockets Live Coding Instant Gratification Web Development is Fun Again CHECKOUT: • hot-reload https://github.com/philidem/node-hot-reload • socket.io http://socket.io/
  17. 17. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY NO BUILD STEP FOR RESOURCE OPTIMIZATION! Resource bundling, minification, compilation, externalization— all at runtime. Compile Raptor Templates, LESS, Dust, CoffeeScript, etc. MORE INFO: • RaptorJS Optimizer: http://raptorjs.org/optimizer/
  18. 18. DESIGN GOAL: MAXIMIZE DEVELOPER PRODUCTIVITY Command Line Tools Powered by Rápido Features • Module and application lifecycle • Scaffolding • IDE/editor independence • PaaS integration • Jenkins CI integration • Command-line auto-completion • Colorized output Example: ecube create app ecube create page ecube register ecube create pool ecube publish ecube deploy CHECKOUT: Rápido: https://github.com/raptorjs/rapido
  19. 19. DESIGN GOAL: HIGH PERFORMANCE
  20. 20. DESIGN GOAL: HIGH PERFORMANCE PERFORMANCE IS GREAT* • Node.js provides a very low-level programming platform • V8 is a powerful and fast JavaScript engine • Node.js can be extended with C/C++ libraries • Clustering based on OS-level socket sharing • High performance evented I/O at its core (libuv) • Single-threaded so no synchronization issues or locks * The not-so-great: • Garbage collection is slow • Memory usage can easily get out of control
  21. 21. DESIGN GOAL: HIGH PERFORMANCE STREAMING, ASYNCHRONOUS UI RENDERING • Why? • • • • Start rendering the template immediately Give the browser a head start Let the template drive the data Minimize idle time on the server
  22. 22. DESIGN GOAL: HIGH PERFORMANCE Traditional ASYNCHRONOUS UI RENDERING 1. Make service calls 2. Wait for all data to come back 3. Prepare view model Input Controller View Model Template HTML Long Delay CubeJS 1. Setup data providers 2. Go straight to the template! Data Providers Input Controller View Model No Delay Template HTML
  23. 23. DESIGN GOAL: HIGH PERFORMANCE Raptor Templates Extensible, high performance, asynchronous template rendering engine for JavaScript that works on both the server and the client MORE INFO: Raptor Templates: http://raptorjs.org/raptor-templates/
  24. 24. DESIGN GOAL: ROBUST
  25. 25. DESIGN GOAL: ROBUST cluster2 • • • • • • • Multi-process cluster management Live production debugging Monitoring Idle timeouts Pause and resume (out-of-traffic GC!) Process warm-up (coming soon) Shared cache (coming soon) MORE INFO: https://github.com/cubejs/cluster2
  26. 26. DESIGN GOAL: ROBUST Monitoring Infrastructure Adapted to eBay Ops Monitoring Tools
  27. 27. DESIGN GOAL: ROBUST Automated Testing • Netmorphic • Simulate network failures and slowness • Mocha • Asynchronous tests • Code coverage reporting • JSHint • Flexible JavaScript static code analysis tool • Casper + PhantomJS (in-progress) • Headless browser testing • Jenkins CI
  28. 28. DESIGN GOAL: ROBUST Configuration Management • Per-module configuration • Real-time configuration updates • Configuration repository based on MongoDB
  29. 29. ASYNCHRONOUS PROGRAMMING USING PROMISES All CubeJS modules have standardized on promises for asynchronous programming. Example: Q() .then(loadUserPreferences) .then(performSearch) .then(renderSearchResults) .fail(function(error) { res.end(); console.error("Something went wrong!") })
  30. 30. ADOPTION: PULL RATHER THAN PUSH • 10+ customers before CubeJS goes GA • Several live CubeJS applications running in production • Overwhelming attendance of CubeJS Tech Talks • ―Word of Mouth‖ spreading within eBay application teams • No need to push
  31. 31. QUESTIONS? psteeleidem@ebay.com @psteeleidem https://github.com/cubejs http://raptorjs.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×