Apache Thrift : One Stop Solution for Cross Language Communication


Published on

Slides from my talk at Scaling PHP in the Cloud Event.

Published in: Technology, Education
  • Be the first to comment

Apache Thrift : One Stop Solution for Cross Language Communication

  1. 1. Apache ThriftOne Stop Solution for Cross Language Communication Piyush Goel Capillary Technologies piyush.goel@capillary.co.in
  2. 2. Problem Statement ?????????????????? SMS07/07/11
  3. 3. Solutions ?? SOAP : Do we really need to parse so much XML ? COM : Works only for windows Protocol Buffers: Does not support RPC out of the box Mainly focussed on serialization and deserialization MessagePack: Well, It wasn’t popular enough when the system was designed.07/07/11
  4. 4. Problem THRIFT07/07/11
  5. 5. Thrift Remote Procedural Call (RPC) Framework for developing scalable cross language services. Developed by Facebook, Open sourced in April 2007 Entered Apache Inbubation in May 2008 Current Version 0.6 Supported Languages : C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, Squeak, OCaml 07/07/11
  6. 6. Components Type System Generic Type System compatible with all languages. Transport Layer Actual data transfer over stream sockets/HTTP etc. Protocol Layer Encoding/Decoding of data types Processors Encapsulates logic of reading/writing to input/output streams Server Wrapper over all above layers for the server end07/07/11
  7. 7. Type System Define your data types in a Thrift Interface Description File Base Types: bool byte Integers: i16, i32, i64 double string Structs : C style structures Containers: List, Set, Map Exceptions: Basically structs Services: Interfaces defining the remote methods. Equivalent to an abstract class 07/07/11
  8. 8. Transport Layer Abstracts out the logic of reading/writing over the communication layer through Interfaces Decouples the remaining system from nitty-gritty of raw data transfer TTransport Interface : open, close, isOpen, read, write, flush TServerTransport Interface: open, listen, accept, close Available Implementation : TSocket, TFileTransport, TFramedTransport, TBufferedTransport, TServerSocket etc 07/07/11
  9. 9. Protocol Layer Abstracts out the serialization and deserialization of data types Exposes read/write methods for all data types Implementations: Binary Compact JSON Plain Text, XML07/07/11
  10. 10. Processor and Server TProcesser interface handles the reading and writing of data through TProtocol objects. Invokes the user implemented handlers for remote methodsServer:Encapsulates all the above layers: Creates transport Creates protocols for the transport Creates a processor based on the input/output protocols Listen for incoming connections and pass them to the processor TSimpleServer, TThreadedServer, TThreadPoolServer07/07/11
  11. 11. The Whole Flow07/07/11
  12. 12. Thrift at CapillaryDynamic Vouchers LAMP based server Java Based Business Rule Engine Over 1000 rules to be processed on every transaction Prioritization/Selection of the vouchers generated Thrift for PHP-JAVA communication. TSocket TBinary TThreadedServer 25-30 RPS during peak time 07/07/11
  13. 13. Thrift at CapillarySMS Engine:Transactional SMS: “Hello Mr. Sharma, thanks for shopping at our store today. You have been awarded 100 points. “ PHP web-service / Java based sms sending engine. Thrift for PHP-Java communication Promotional: Sends out personalized SMS to customers “Hello Mr Shahrukh Khan, its been 3 months since you visited our MG Road outlet. Hoping to see you soon !!” On average ~1.5-2 MM SMS per day Java based system for message generation/template replacement Java based system to send messages to gateway. Java – Java communication over thrift. 07/07/11
  14. 14. References1) Thrift Whitepaper: http://thrift.apache.org/static/thrift- 20070401.pdf2) http://en.wikipedia.org/wiki/Apache_Thrift3) Thrift: The Missing Guide: http://diwakergupta.github.com/thrift-missing-guide/4) http://chanian.com/2010/05/13/thrift-tutorial-a-php-client/5) http://mikecvet.wordpress.com/2010/05/13/apache-thrift- tutorial-the-sequel/ 07/07/11
  15. 15. Questions07/07/11
  16. 16. 07/07/11
  17. 17. Which of the following languages Thrift does not support ? 1) SmallTalk 2) Squeak 3) C 4) Objective-C07/07/11
  18. 18. Which interface implementation is responsible for managing incoming connections on the Server in Thrift ?1) TProtocol2) TProcesser3) TTransport4) TServer07/07/11
  19. 19. Which of the following projects is/are using Thrift internally?1) HBase2) HyperTable3) Voldemort4) Cassandra07/07/11
  20. 20. We are Hiring !!! career-tech@capillary.co.in piyush.goel@capillary.co.in http://capillary.co.in07/07/11