Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using RabbitMQ and Netty library to implement RPC protocol

2,989 views

Published on

Using RabbitMQ and Netty library to implement RPC protocol

Published in: Software
  • Be the first to comment

Using RabbitMQ and Netty library to implement RPC protocol

  1. 1. Tho Luong SILILAB Vietnam Hanoi, March 1, 2014 Rubichat
  2. 2. Introduction 2  Rubichat:  A Realtime Communication Service  Allow website owner to communicate with customer more easily
  3. 3. Rubichat: customer & operator 3
  4. 4. Introduction 4  Rubichat:  A Realtime Communication Service  Allow website owner to communicate with customer more easily  Easy to setup:
  5. 5. Architecture X user Facebook user Customer and operator X1 X2 crane crane RabbitMQ tractor tractor Java, php, spring, hibernate, amqp, nodejs, zend framework, netty sensor Apache ? MySQL, Redis and MongoDB
  6. 6. Issue 6  Develop rubiweb project in PHP using Zend Framework  Some services were implemented in Java (30%)  Should we implement other services in Java or PHP or both of them?  3 solutions
  7. 7. Solution #1 7  Reimplement all services in PHP  Some services will be implemented in both Java and PHP  not a big problem  PHP and Zend work well with MySQL, do they still work well with MongoDB and Redis?  Just some services use MongoDB and Redis  not a big problem
  8. 8. Solution #1 8  When Java wants to use a service that is implemented in PHP  port this implementation to Java?  Some services are hard to implement, ex: access control service Solution #1: complicated and not easy for maintainance.
  9. 9. Solution #2 9  Beauty and maintainability are #1 priority  All services will be implemented in Java  PHP calls Java service implementations remotely via RabbitMQ e2 PHP Java e1 req resp OK
  10. 10. Solution #2 10  Problem: how to receive response properly? PHP -01 PHP-02 Resp-01 Resp-02  Mission impossible
  11. 11. Solution #2 11  How about using one queue for each PHP request? PHP-01 PHP-02 ….. ….. e PHP-n  At least, one “dirty” queue is produced after each PHP request lifecycle No, RabbitMQ must manage a lot of queues.
  12. 12. Solution #3 12  RPC: PHP invokes Java API remotely PHP Java Server X protocol  Issues: 1. System architecture 2. Protocol selection Flappy 3. Technology selections for Java Server
  13. 13. System architecture 13  Option 1: flappy accesses database directly PHP Java Server X protocol Flappy MySQ L Mong oDB Redis  Option 2: flappy serves PHP connections, access service implentations via RabbitMQ PHP Java Server Flappy RabbitMQ MySQ L Mong oDB Redis Tractor Tractor ….. …..
  14. 14. Protocol selection 14  Common RPC protocols  XML RPC  JSON RPC  Protobuf (google)  Thrift (facebook)  SOAP  Flappy uses XMPP (RFC 3920,RFC 3921) message format  Easy to control  Familiar to Rubichat developers
  15. 15. XMPP in JSON format 15  Stanza: IQ  Stanza: M
  16. 16. Binary data format 16  Data type: 1 byte  RABBIT_TRACTOR_IN, RABBIT_TRACTOR_OUT, FLAPPY_IN, FLAPPY_OUT, NA  Length: int: 4 byte  Body type length body 1 4 length
  17. 17. Technologies selection 17  Socket library: Netty, a great library for network application  Twitter uses Netty to improve search engine performance. (3x faster)  Interceptor design in Tractor server is influenced by Netty
  18. 18. Netty advantages 18  Asynchronous I/O: decouple threads from Client Socket Connections  Rich Buffer data structure  Advanced Components for More Rapid Development  Event Model based on the Interceptor Chain Pattern
  19. 19. Test result 19  Create 100 users, find them and measure time: Service Query Tractor(ms) Client (ms) Δ (ms) Create 100 5196 5665 469 Find 1 187 203 16 Service Query Tractor(ms) Client (ms) Δ (ms) Create 100 4414 Find 1 34 (New FlappyClient implementation: use single connection(March 26, 2014))
  20. 20. 20 Demo: 1. Flappy Server 2. Tractor
  21. 21. Solution disadvantages 21 response PHP Java Server Flappy .phpModel ToJSON() request .jsonTo PhpModel() .jsonTo JavaModel() .javaModel ToJSON()  Write data-mappers manually
  22. 22. Future work 22  Tool for generating data mappers  Optimize PHP FlappyClient and develop clients for other languages
  23. 23. Thank you! 23

×