Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Real World Enterprise Reactive Programming using Vert.x

1,767 views

Published on

Shows best practices and lessons learned

Published in: Technology
  • Be the first to comment

Real World Enterprise Reactive Programming using Vert.x

  1. 1. Mariam Hakobyan, Sascha Möllering, | zanox AG Björn Stahl Real World Enterprise Reactive Programming using Vert.x
  2. 2. TABLE OF CONTENTS 1.Introduction 2.The Beginning 3.How to start? 4.Best Practices 5.Architecture 6.Deployment 7.Vert.x module system 8.Integration with messaging system
  3. 3. INTRODUCTION ZANOX Europe‘s leading performance advertising network
  4. 4. THE BEGINNING Java Magazin 04.14: Vert.x im Unternehmenseinsatz Entwicklung und Betrieb von asynchronen Applikationen mit Vert.x in der Praxis
  5. 5. THE BEGINNING ● New core system for zanox ● Requirements are pretty high (not negotiable): ● Low latency ● High throughput ● Reactive ● Fast
  6. 6. THE BEGINNING https://github.com/Mr-Steel/vertx_fatjar
  7. 7. WHAT IS VERT.X? Vert.x “Vert.x is a lightweight, high performance application platform for the JVM that's designed for modern mobile, web, and enterprise applications.”
  8. 8. WHAT IS VERT.X? Polyglot
  9. 9. WHAT IS A VERTICLE? ● Classes with an own Classloader ● operates Single Threaded ● executed by an Event Loop
  10. 10. V V V W W W Event Loop Pool Event Bus TAKE A LOOK INSIDE
  11. 11. HOW TO START? ● mvn archetype:generate - Dfilter=io.vertx: (do not forget the colon!) ● generates structure for all languages (JS, Ruby, Groovy, Python) ● maven pom is already set with relevant data
  12. 12. HOW TO START?
  13. 13. Lessons learned - Profiling ● Verticles and WorkerVerticles appear in different subtrees ● Be careful with blaming, a lot of stuff is just waiting ● If you think a verticle is a bottleneck: start more instances of this verticle ● Measure every change & compare results (yeah yeah - but really - do it!)
  14. 14. BEST PRACTICES Starter Verticle
  15. 15. BEST PRACTICES Logging
  16. 16. BEST PRACTICES Logger Verticle ● Disadvantages: ● log entries are not in order ●loss of log levels ● Solutions: ●use IDs to follow log message flow ●use one LoggerVerticle per log level
  17. 17. BEST PRACTICES - On the fly deployment
  18. 18. BEST PRACTICES - On the fly deployment
  19. 19. BEST PRACTICES In General ● Do not block the loop ● put blocking code or extensive computation into worker verticles ● Keep the application responsive ● stress test as often as possible ● Encapsulate common code in modules (more on this later)
  20. 20. BEST PRACTICES Deployment fat-jar: one package to rule the deployment
  21. 21. BEST PRACTICES Deployment zip-File: ● have a separate folder for scripts ● bind packaging to verify phase to have it all with one command “mvn verify” ● complete example is on GitHub
  22. 22. Internet Availability Zone Auto Scaling group Auto Scaling group Availability Zone
  23. 23. Architecture
  24. 24. Architecture ● Main framework Vert.x ● mod-kinesis (Kinesis-Adapter for Vert.x) ● AWS Java SDK ● IAM roles for Amazon EC2 instances ● Coda Hale metrics and CloudWatch reporter ● Jedis (Redis client for Java) ● …
  25. 25. Deployment Workstation GitHub Jenkins Nexus Python/Fabric Datacenter Artefakt
  26. 26. Deployment
  27. 27. Deployment EC2 instance with Docker registry (port 5000) S3 bucket to store Docker images
  28. 28. Deployment Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry S3 bucket with Docker Images
  29. 29. Deployment
  30. 30. Deployment
  31. 31. Deployment
  32. 32. Deployment python start_docker_instance.py -r <myregistry>:5000 -i kinesis-producer -t 95 -q 2 -s quality -d '-d -p 8080:8080' https://github.com/SaschaMoellering/aws-docker-scripts
  33. 33. Deployment python start_elb.py -r <myregistry>:5000 -i kinesis-producer -t 95 -s quality -d '-d -p 8080:8080' https://github.com/SaschaMoellering/aws-docker-scripts
  34. 34. Deployment
  35. 35. Deployment
  36. 36. Vert.x Module System
  37. 37. Vert.x Module System Benefits: ● Re-usable components as a zip file ● Maven and Bintray integration ● Vert.x module registry ● Vert.x downloads and installs modules for you
  38. 38. Vert.x Module Registry
  39. 39. Vert.x Event Bus ● Address is a simple string ● Types - Pub/Sub & P2P Pub/Sub P2P
  40. 40. Sender Receiver
  41. 41. Integration with Messaging System KAFKA MESSAGING SYSTEM ● Apache Kafka is a distributed publish-subscribe messaging system ● Fast ● Scalable ● Durable ● Distributed
  42. 42. Integration with Messaging System Main Terminology: ● Topic ● Producer ● Consumer ● Broker
  43. 43. Kafka Module in Module Registry Open source Kafka module in Vert.x’ module registry ● http://modulereg.vertx.io/
  44. 44. Using Vert.x Module // programmatically // command line
  45. 45. Using Kafka Module Configuration is a JSON object:
  46. 46. StatsD Support Config
  47. 47. Metrics of Vert.x Project On 4 Cores virtual machine we had the following results: ●~28 K requests per second without Kafka, with lookup from Redis ●~18 K requests per second with Kafka and lookup from Redis
  48. 48. Metrics of Vert.x Project Test Traffic ~1000-3000 requests per minute: ● mean response time: 1-2ms ● 90% of responses 0.86 ms ●fully cached: 0.65 ms ● CPU Load: 1-3% ● Memory: 90MB (300MB reserved)
  49. 49. GET INVOLVED Kafka Module - https://github.com/zanox/mod-kafka Kinesis Module - https://github.com/zanox/mod-kinesis
  50. 50. https://github.com/Mr-Steel/vertx_fatjar

×