Your SlideShare is downloading. ×
0
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

JavaOne 2013 CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments

593

Published on

JavaOne 2013 presentation for CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments.

JavaOne 2013 presentation for CON7370: Java Interprocess Communication Challenges in Low-Latency Deployments.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
593
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 26 Sept 2013 CON 7370: Java Interprocess Communication Challenges in Low-Latency Deployments © 2013 IBM Corporation
  • 2. Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS 2 © 2013 IBM Corporation
  • 3. Introduction to the speakers Daryl Maier – 13 years experience developing and deploying Java SDKs at IBM Canada Lab – Recent work focus: • X86 Java just-in-time compiler development and performance • Java benchmarking – Contact: maier@ca.ibm.com Anil Kumar – 11 years experience in server Java performance ensuring best customer experience on all Intel Architecture based platforms – Contact: anil.kumar@intel.com 3 © 2013 IBM Corporation
  • 4. Why is IPC necessary?  IPC == intra- and extra-process communication between entities  Real applications are often partitioned into independent modules  Clean design, logical separation of entities  Improved QA  Component re-use  Communication with legacy or non-proprietary systems (web services)  But entities need to communicate  Communication latency  Bandwidth  Complex distributed scheduling and deadlocks © 2013 IBM Corporation
  • 5. What This Talk Is About  Help developers understand and solve IPC issues in the context of an application with low response time requirements (millisecond)  Share our experiences with developing an industry standard Java benchmark – Software design – Hardware deployment – How does it apply to your application or environment? © 2013 IBM Corporation
  • 6. IPC Requirements (Architecture)  How is your application structured? – Loosely or tightly coupled? – Java only?  Will simple signals do?  Are response/replies required?  Do you have special transport requirements? (e.g., encryption)  Is response time important?  Java SE only? © 2013 IBM Corporation
  • 7. Java Building Blocks for IPC (Implementation)  java.net  java.util.concurrent  NIO/NIO2 packages  CORBA  RMI © 2013 IBM Corporation
  • 8. Key Challenges With IPC  Maintain high throughput and low latency (eliminate bottlenecks)  Scalability of overall application  Avoid deadlocks and starvation  Work around limitations of communication technology All are even more challenging in low SLA environments! © 2013 IBM Corporation
  • 9. SPECjbb2013  Next generation Java business logic benchmark from SPEC  Business model is a supermarket supply chain: HQ, suppliers, supermarkets  Scalable, self-injecting workload with multiple supported configurations: – Composite (standalone) – Multi-VM (several JVMs on the same host) – Distributed (several JVMs on multiple hosts)  Throughput ops/sec metric and a response-time metric  Customer-relevant technologies: security, XML, JDK 7, etc.  Designed to share data between entities © 2013 IBM Corporation
  • 10. Response Time / Throughput Curve Increasing load © 2013 IBM Corporation
  • 11. SPECjbb2013 General Architecture Control Entity - benchmark infrastructure Company DB Inventory Business Entity - workload Config, Results Driver SM HQ SP Controller Agent Agent Controller TxInjector Storage - shared state Control traffic - start/stop requests - audit requests Business Data Traffic - business logic requests - fine-grained status Backend (SUT) © 2013 IBM Corporation
  • 12. SPECjbb2013 Communication Requirements  Separation of workload components – Intra- and inter-JVM traffic  Control, status, data, and notifications traffic – Request/Response and one-way Messages  Multiple benchmark deployments – Scalable architecture  Millisecond response times under high transactional rate  Simple developer API – Uniform API for communication between agents: intra- or inter-process © 2013 IBM Corporation
  • 13. SPECjbb2013 Communications Architecture Driver SM IC Conn Conn Clients Server SM SP HQ IC JVM1 JVM2 TxInjector Backend Active Entity - SM/HQ/SP - Driver Transport - Encapsulation - Compression - Encryption Intra-JVM data traffic Benchmark state: - Business-logic requests - Fine-grained status Inter-JVM data traffic Benchmark state: - Business-logic requests - Fine-grained status © 2013 IBM Corporation
  • 14. Interconnects Master Registry Client 1 Client 2 Client 3 Local Registry Connectivity IC JVM  Interconnect is the central communication fabric – Each JVM has its own IC  Registry handles routing – Global namespace for all clients – Local locations added when connected – Remote locations added lazily as they are resolved  Transports define the format of data transferred between clients  Connectivity defines the exchange format between two ICs © 2013 IBM Corporation
  • 15. Transports Serial Encapsulation Compression Plain Encryption XML none GZIP none JCE  Transports model business data marshallers / unmarshallers  Transport mappings are fixed – Most frequent transport is PLAIN, i.e., “pass by reference”  Transport overheads can be LARGE, consider allowing adjustment in your application © 2013 IBM Corporation
  • 16. Connectivities Server Server Tier 0 Server Tier 1 Server Tier n Inbound IC Clients Outbound  Connectivity is the client-server pair delivering data between ICs  Pluggable providers: Grizzly/NIO, Grizzly/HTTP, Jetty/HTTP  Consider pooling outbound client connections to remote servers  Connectivities are complex to design, implement, and test. Consider a pluggable solution for production environment. © 2013 IBM Corporation
  • 17. Connection Clients Connection Client Uplink for out-bound traffic Downlink for in-bound traffic  Clients (or entities) that can connect to an IC – Agents (communicate with Controller) – BatchExecutors (process communication in batches in parallel)  Communicate with IC via uplinks and downlinks  Simple communications API © 2013 IBM Corporation
  • 18. Messages and Requests  Communication delivery: – messages (no reply expected, non-blocking) – requests (reply expected, block for response)  Repeated, identical messages/requests to same destination can be marshalled once and cached by uplink to improve throughput – Important for scalability  Batching multiple messages/requests to same destination in same packet  improved throughput and overhead (payload size and marshalling costs)  need careful consideration of payload size and flush mechanism to not harm SLAs © 2013 IBM Corporation
  • 19. Communication Design FAQ  Why did you choose a message passing scheme? – Fits the business model – Allows flexible configurations without changing code  Why didn’t you use the Java Messaging Service (JMS)? – Java SE not Java EE – Not a JMS benchmark!  Why did you re-implement a communication scheme rather than re-use an existing one? – Wanted tight control over performance and bottlenecks – Re-use complex components, carefully choosing implementations to meet flexibility requirements © 2013 IBM Corporation
  • 20. Worker Thread Pools  WTPs execute transactions initiated by incoming requests and other transactions Worker Thread Pool (WTP) Tier 0 Tier 1  ForkJoin implementation for efficient batch decomposition and work stealing Tier n SM SP HQ  Each connection client within a JVM shares a single WTP with the other connection clients – Reduces artificial context switches  Be careful with FJ and threads with blocking I/O Conn Server IC JVM  Proper design of WTPs essential for scalability and throughput © 2013 IBM Corporation
  • 21. Deadlocks Request 1 Client IC1 WTP 2 IC2 Server WTP 1 Server Client Request 2 JVM 1 JVM 2 Potential deadlock if Request 2 is queued and all threads in WTP 1 are waiting for results from remote WTP 2.  Solutions? – Complicated distributed deadlock avoidance strategies – Rework transactions to eliminate cycles © 2013 IBM Corporation
  • 22. Communication Tiers  Introduce tiers to connectivity servers and thread pools  Architected to guarantee progress by ensuring there is always an available thread  Must be sized according to communication patterns  Communication requests: – within the same IC are sent on the same tier as requestor thread – to a remote IC are sent to the next highest tier – for infrastructure/control are sent on a reserved tier  Destination tiers are transparent to sender © 2013 IBM Corporation
  • 23. Communication With Tiers Request 1 from Tier1 Request 1 routed to Tier 2 WTP Request 1 arrives on Tier1 Client Server IC1 IC2 Server WTP 1 Client JVM 1 Request 2 Routed to Tier 3 WTP WTP 2 JVM 2 Request 2 arrives on Tier2 Request 2 from Tier 2 © 2013 IBM Corporation
  • 24. Debugging Advice  Keep the communication API simple  Avoid aggregating timeout failures – a timeout does not often reveal the underlying cause  Using I/O to debug may introduce latency that produces an artificial problem  Stress testing on large systems is critical for evaluating performance and overall design © 2013 IBM Corporation
  • 25. Factors That Affect Response Time © 2013 IBM Corporation
  • 26. Deployment Decisions  HW configuration: – Systems with multiple network cards, network latency could be improved by affinitizing network traffic to particular network cards and directing them to specific processors – System could be connected via infiniBand or fiber optic instead of regular Ethernet cable  Native OS: – Within Single OS image: could take advantage of loop back traffic optimizations – Across Multiple OS image: careful about traffic routing  Virtual OS images: – Good to affinitize VM to cores instead of free floating – Be careful about IO across virtual OS images  Process level – Process affinitized to a socket delivers more consistent response time © 2013 IBM Corporation
  • 27. Tunings: GC  If long latencies are tracking with GC pauses, tune your garbage collector – Run with verbose GC – GC policies, Heap size, Nursery size Match with GC log time stamps © 2013 IBM Corporation
  • 28. Tunings: Time outs  Ensure communication time outs are appropriately sized – Too large may not react quickly enough, affecting response time – Too small does not tolerate normal workload delays  Time outs must scale when moved to larger deployments  Architect an appropriate response when something times out – Retry mechanism? © 2013 IBM Corporation
  • 29. Tuning: Communications  Number of connection pools : Number of sockets  Number of grizzly pools : Number of threads in each Grizzly pool  Number of ForkJoin workers: – Mostly need to set > logical processors © 2013 IBM Corporation
  • 30. Tuning: Timing APIs  Watch for too much contention on timing APIs when HPET is being used 30 © 2013 IBM Corporation
  • 31. Top 10 Takeaways 10 Reduce communication as much as possible 9 Send big chunks of data 8 Combine messages in batches 7 If you care about throughput, use an asynchronous model 6 If you care about strict correctness, use a synchronous model 5 Re-use connection channels: do not create new ones for each message 4 Check errors: it is difficult to debug IPC, so log any error as much as possible 3 Use frameworks when appropriate 2 Profile and tune 1 Don’t believe it when someone says “It will never happen” © 2013 IBM Corporation
  • 32. Questions? http://ibm.co/JavaOne2013 Visit the IBM booth and meet other IBM developers at JavaOne 2013 © 2013 IBM Corporation
  • 33. References and Credits  Some benchmark slides and diagrams were adapted from SPECjbb2013 Design Committee documentation and correspondence.  Aleksey Shipilev (Oracle) for communication architecture diagrams © 2013 IBM Corporation
  • 34. Trademarks SPECjbb®2013 is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. © 2013 IBM Corporation

×