Jeremy Whitlock (@whitlockjc)
Gluecon 2014
Bringing Node.js to the JVM
Why bring Node.js to the JVM?
Use Case: Node.js API for Managing API Proxies
Use Case: Writing MapReduce Jobs With JavaScr...
Why do people love/hate Java?
Why do people love/hata JavaScript?
Which language is better?
(insert flame war starter here...
Use Cases
A while back, Apigee was looking for a simple, programmable approach for
configuring and managing API proxies. We wanted t...
On the data team, we were building a data processing system around
MapReduce. When we talked to some of the PMs, we decide...
The Problem
To summarize the problem: We want to use Node.js in a Java world
Like or not, many of us live in a Java world: Cassandra, ...
Node.js Is Bad Ass Rock Star Tech
The Solution
The best of both worlds is to figure out some way to provide interop
● This will allow code reuse between Java and Node.js...
Node.js Trireme
What is Trireme?
Single-threaded event engine
Non-blocking TCP I/O
Non-blocking UDP datagrams
Timers
Non-B...
Trireme Architecture
One thread per Node.js application
Async I/O handled via NIO within that thread
Additional thread poo...
Trireme lets you embed a Node.js runtime within the JVM. You have full
control over its environment.
Trireme allows you to...
Let’s Open Source Something
Lembos provides you with a MapReduce runtime and runner that allows you
to write your MapReduce jobs using Node.js.
Jobs r...
Lembos Demonstration
● Trireme: https://github.com/apigee/trireme
● Lembos: https://github.com/apigee/lembos
● Anatomy of a Node.js MapReduce J...
I Made It
Thank you
Gluecon 2014
Upcoming SlideShare
Loading in …5
×

Gluecon 2014 - Bringing Node.js to the JVM

2,070 views
1,844 views

Published on

Presentation about a few situations at Apigee where we wanted/needed interoperability between Node.js and Java. One example is Hadoop-based MapReduce. During this presentation I discuss some use cases, some of the problems and then finish talking about technology that allows you to write MapReduce jobs using Node.js and run them natively on Hadoop as if written in Java.

Published in: Software, Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,070
On SlideShare
0
From Embeds
0
Number of Embeds
86
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Gluecon 2014 - Bringing Node.js to the JVM

  1. 1. Jeremy Whitlock (@whitlockjc) Gluecon 2014 Bringing Node.js to the JVM
  2. 2. Why bring Node.js to the JVM? Use Case: Node.js API for Managing API Proxies Use Case: Writing MapReduce Jobs With JavaScript/Node.js Discuss why Java/Node.js interop makes sense An introduction to the project enabling Java/Node.js interop Let’s open source something What this talk is about
  3. 3. Why do people love/hate Java? Why do people love/hata JavaScript? Which language is better? (insert flame war starter here)... We know people love/hate Java… We know people love/hate JavaScript… We know both languages have their pros/cons… ... What this talk is not about
  4. 4. Use Cases
  5. 5. A while back, Apigee was looking for a simple, programmable approach for configuring and managing API proxies. We wanted to take an XML-based and provide an easier, programmable interface without losing what makes our API ecosystem what it is today. The timing around this decision was right when Walmart, LinkedIn, eBay/PayPal and others were developing high performance, scalable REST API libraries/tools on Node.js. We quickly realized how well-suited Node.js was for what we were trying to accomplish with these API proxies: highly concurrent, asynchronous scalable servers Where Java interop becomes important in this scenario is that our backend is Java-based and we need some way to deploy/manage these Node.js servers from our existing infrastructure. Node.js API for Managing API Proxies
  6. 6. On the data team, we were building a data processing system around MapReduce. When we talked to some of the PMs, we decided that we wanted a simpler way to do this...without Java. While we wanted a simpler approach than Java, we did not want to reinvent the wheel and we wanted to make it easy for those with MapReduce experience to just jump in and be productive. JavaScript was the language we decided on because it was built into the JVM from Java 6 onward and it was small/simple enough to learn. While JavaScript was the language we chose to use, without Node.js we faced problems with packaging problems, code reuse, dependencies, etc. We decided to align with the rest of the company and use Node.js for our data processing support. Writing MapReduce Jobs With JavaScript/Node. js
  7. 7. The Problem
  8. 8. To summarize the problem: We want to use Node.js in a Java world Like or not, many of us live in a Java world: Cassandra, Hadoop, Mule, Qpid, ... For some Java-based tools, there are Node.js clients/ports to make interoperability a non-issue. But for most of the aforementioned systems, like Hadoop, this is not an option. What do we do? ● Write a pure Node.js module? (Not always possible) ● Not use Node.js? (Not ideal) ● Not use the Java system (Hadoop, …)? (Not ideal) The Problem
  9. 9. Node.js Is Bad Ass Rock Star Tech
  10. 10. The Solution
  11. 11. The best of both worlds is to figure out some way to provide interop ● This will allow code reuse between Java and Node.js ● This will require the least amount of wheel reinventing ● The time it takes to use both is invaluable ● Much less time than porting one tool/library to another language ● Much less time than waiting on someone to port it for you How do we do we provide this interoperability between Java and Node.js? Thankfully, this is already solved for us and the solution we chose was a project called Trireme: https://github.com/apigee/trireme Trireme lets us embed Node.js in the JVM, something required for both of our use cases, and lets us share code between both languages. The Solution
  12. 12. Node.js Trireme What is Trireme? Single-threaded event engine Non-blocking TCP I/O Non-blocking UDP datagrams Timers Non-Blocking File I/O “Buffer” object Module loading system Utility modules Third-party components V8 JavaScript engine OpenSSL ZLib Single-threaded event engine Non-blocking TCP I/O Non-blocking UDP datagrams Timers Non-Blocking File I/O “Buffer” object Module loading system Utility modules Third-party components Rhino JavaScript engine Java SE Bouncy Castle (crypto, optional)
  13. 13. Trireme Architecture One thread per Node.js application Async I/O handled via NIO within that thread Additional thread pool for blocking operations File I/O (especially on Java 6) DNS lookups Replace native code from Node.js with Java alternatives Internal modules such as “tcp_wrap,” etc. Implement a few popular native modules with Java code “iconv,” “node_xslt”, eventually others
  14. 14. Trireme lets you embed a Node.js runtime within the JVM. You have full control over its environment. Trireme allows you to interact with Node.js modules and JavaScript objects from Java. Trireme allows you to expose Java objects to JavaScript code either by calling JavaScript directly or by exposing Java-based Node.js modules Trireme allows you to bundle/ship JavaScript Node.js modules (This seems trivial but imagine creating a vetted Node.js environment and shipping it with your runtime) Don’t believe me? Let’s look at a project that uses Trireme to allow you to write MapReduce jobs using JavaScript/Node.js What Does Trireme Bring to the Table
  15. 15. Let’s Open Source Something
  16. 16. Lembos provides you with a MapReduce runtime and runner that allows you to write your MapReduce jobs using Node.js. Jobs run in the JVM with Hadoop driving the process, as if written in Java. Lembos provides a runner, to be ran via hadoop jar, like all other MapReduce jobs, and a runtime that provides the bridge between the Java- based Hadoop world and the JavaScript-based Node.js world. The name Lembos is a play on Trireme. (A Trireme is an ancient warship with “three banks of oars”. Since Trireme made this project possible, it seemed fitting to have a similar name. A Lembos is a lightweight warship typically used for piracy...not that I’m planning the same thing.) Project Homepage: https://github.com/apigee/lembos Introducing Lembos
  17. 17. Lembos Demonstration
  18. 18. ● Trireme: https://github.com/apigee/trireme ● Lembos: https://github.com/apigee/lembos ● Anatomy of a Node.js MapReduce Job ● Lembos Architecture ● Lembos CLI Reference ● Lembos Node.js API Reference ● The Slides: http://www.slideshare.net/whitlockjc ● Contacting me: ● Twitter: @whitlockjc ● GitHub: whitlockjc Resources
  19. 19. I Made It
  20. 20. Thank you Gluecon 2014

×