Real-Time Java
for Latency Critical
Banking Applications

Bertrand Delsart                  Real-Time
JavaRTS Technical Le...
Agenda
•   Background
•   Benefits of a Real-Time Java Virtual Machine
•   Benefits of Real-Time Garbage Collectors
•   Q&...
Agenda
•   Background
•   Benefits of a Real-Time Java Virtual Machine
•   Benefits of Real-Time Garbage Collectors
•   Q&...
Background

Banking Trend
• Avoiding latencies (unexpected delays) has always been
  important

Steve Rubinow (NYSE Group ...
Background

Banking Trend
• Physical co-location is removing the external sources of
  jitter (variation of execution time...
Background

Categories of Application Predictability

      Non        • No time-based deadlines
    real-time    > Standa...
Background

Traditional Design Choices to Improve Predictability

   Use mainstream                                     Us...
Background

Traditional Real-Time Design Choices: Examples
• Example 1: Global engineering conglomerate needs PLC
  contro...
Background

Real-Time Design Choices

    Use mainstream                                  Use customized lower-
  programm...
Background

Real-Time Specification for Java (RTSJ)


   1998                 2002                2005                    ...
Agenda
•   Background
•   Benefits of a Real-Time Java Virtual Machine
•   Benefits of Real-Time Garbage Collectors
•   Q&...
RTSJ

Use case 1: Send Market Data at a Fixed Rate

          Neither too late... nor too often !
• Mainstream Java soluti...
RTSJ

RTSJ Benefit 1 : Priority Semantic
  You must specify the relative importance of the tasks,
      including with res...
RTSJ

Use case 1 revisited
• Code executed by a RealtimeThread
  setPriority(my_RTPriority);
  while (true) do {
    compu...
RTSJ

RTSJ Benefit 2: Rich Real-Time APIs
• RTSJ provides what you will find in most RT Operating
  Systems
  > Absolute T...
RTSJ

Use case 1, RTSJ version
• Code executed by a RealtimeThread
  setPriority(my_RTPriority);
  setReleaseParameters(my...
RTSJ

Yes... if optimized for Determinism
• The fastest Garbage Collectors suspend all the threads
  while they recycle me...
RTSJ

NoHeapRealtimeThreads
• Code executed by a NoHeapRealtimeThread
  setPriority(my_RTPriority);
  setReleaseParameters...
RTSJ

Memory areas for NHRTs

 Don't bother if 200 microseconds latencies are OK !!!

• ImmortalMemory for non recycled ob...
RTSJ

Use case 1, ScopedMemory version
• Code executed by a NoHeapRealtimeThread
  while (true) do {
    scopedMemory1.ent...
Agenda
•   Background
•   Benefits of a Real-Time Java Virtual Machine
•   Benefits of Real-Time Garbage Collectors
•   Q&...
RTGCs in General

Making Java Predictable, RTGC
• Real-Time GC
  > Essentially a “garbage collector with knobs”
  > Large ...
RTSJ + RTGCs

Overall System Model
                                                          Real-time code and
          ...
Java RTS

 Sun's RTGC
 Apply concept of policy to Henriksson's approach
 • Goal: Hard RT latencies for high priority GC'd ...
Java RTS

Default RTGC Scheduling Policy
                                                 Could run concurrently
         ...
Java RTS

Use case 1 revisited
• Code executed by a hard RealtimeThread
  while (true) do {
    waitForNextPeriod();
    s...
Java RTS

Use case 1 Deadline Miss Handler
• Hard Real-Time AsynchronousEventHandler
  void handleAsyncEvent() {
    quick...
Java RTS

Use Case 2 : Events Driven Request with Deadlines

• A request is valid:
  > For a limited time
  > When a set o...
Java RTS

Use Case 3 : NASDAQ
“Reduce I/O Jitter and Context Switches”
• I/O must not hold other requests
• Implemented So...
Java RTS

Use Case 3 : Why Care about Context Switches ?

               This should not be your role !
• Risk : under-uti...
Java RTS

Sun Java RTS Unique Selling Proposition

  Sun's Java Real-Time System – absolute execution
  predictability wit...
Java RTS

  Customer Evaluations
   • Over 500 companies and universities from more than
     60 countries have evaluated ...
Product Overview

Key Benefits of RTSJ
          TECHNICAL                                     BUSINESS
•   Maximize syste...
Product Overview

Key Benefits of RTSJ + RTGC
           TECHNICAL                                    BUSINESS
•   Maximiz...
Product Overview

Key Benefits of Sun Java RTS
           TECHNICAL                                     BUSINESS
•   Maxim...
Questions?
Bertrand.Delsart@Sun.COM
Upcoming SlideShare
Loading in …5
×

Bertrand Delsart Java R T S

1,580
-1

Published on

Published in: Technology, Economy & Finance
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,580
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
35
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Bertrand Delsart Java R T S

  1. 1. Real-Time Java for Latency Critical Banking Applications Bertrand Delsart Real-Time JavaRTS Technical Leader System Author of Sun's RTGC technology
  2. 2. Agenda • Background • Benefits of a Real-Time Java Virtual Machine • Benefits of Real-Time Garbage Collectors • Q&A QCon 2008 - Bertrand.Delsart@Sun.COM 2
  3. 3. Agenda • Background • Benefits of a Real-Time Java Virtual Machine • Benefits of Real-Time Garbage Collectors • Q&A QCon 2008 - Bertrand.Delsart@Sun.COM 3
  4. 4. Background Banking Trend • Avoiding latencies (unexpected delays) has always been important Steve Rubinow (NYSE Group CTO): “ If you've got some trades going through at 10 milliseconds and some at 1 millisecond, that's a problem. Our customers don't like variance” QCon 2008 - Bertrand.Delsart@Sun.COM 4
  5. 5. Background Banking Trend • Physical co-location is removing the external sources of jitter (variation of execution time) • Application jitter is now much more visible Dave Cummings (BATS CEO) “Five years ago we were talking seconds, now we're into the milliseconds. Five years from now we'll probably be measuring latency in microseconds” Alistair Brown (LIME Brokerage CEO) “ Shortly, we'll be talking micro- versus milliseconds, and at that point speed will probably have less and less relevance” QCon 2008 - Bertrand.Delsart@Sun.COM 5
  6. 6. Background Categories of Application Predictability Non • No time-based deadlines real-time > Standard programming logic > Example: UI, web services, algorithmic calculations • Deadlines may be missed occasionally Soft under well-defined conditions real-time > Mainstream real-time needs > Example: Automated trading system, 3G telco router • Deadlines cannot be missed Hard > Typically highest priority on system real-time > Example: Robotic motion control and management, Data feed processing QCon 2008 - Bertrand.Delsart@Sun.COM 6
  7. 7. Background Traditional Design Choices to Improve Predictability Use mainstream Use customized lower- programming tools and OR level tools and proprietary standard platforms platforms > Everything is equally > Predictable solution, but... important; so try go faster to > Expensive – dedicated get everything done tools, training, headcount; > Guessing game: no longer development cycles, guarantee that deadlines will complicated maintenance be met – just postponed > Challenging integration of > Low infrastructure utilization real-time elements with the > Lack of predictable solution standard applications QCon 2008 - Bertrand.Delsart@Sun.COM 7
  8. 8. Background Traditional Real-Time Design Choices: Examples • Example 1: Global engineering conglomerate needs PLC control elements with hard real-time capabilities for industrial automation PC-oriented SBC boards cost Custom boards from a specialized ~20% of custom boards but have no integrated real-time OR provider are very costly and require more for software/tools solution • Example 2: Leading financial services institution needs to scale its market-facing system while meeting Service Level Agreements Standard Java applications offer Legacy system able to meet flexibility and scalability but GC current SLAs but no longer viable pauses threaten SLAs OR with regard to maintainability and scalability QCon 2008 - Bertrand.Delsart@Sun.COM 8
  9. 9. Background Real-Time Design Choices Use mainstream Use customized lower- programming tools and OR level tools and standard platforms proprietary platforms How about an open standards-based platform that formally >deals solution, but... > Everything is equally Predictable with real-time challenges? important; try go faster to > Expensive – dedicated get everything done tools, training, headcount, > Guessing game – no longer development cycles, guarantee that deadlines will effort-intensive maintenance be met > Challenging integration of > Low infrastructure utilization real-time elements with the > Lack of predictable solution standard applications QCon 2008 - Bertrand.Delsart@Sun.COM 9
  10. 10. Background Real-Time Specification for Java (RTSJ) 1998 2002 2005 2007 2008 Real-Time JSR-001 RTSJ update Specification for approved by the proposal RTGC added to Java (JSR-001) Java Community submitted Sun Java RTS proposal Process (JSR-282) submitted Others companies Joint Sun/IBM - Apogee New Sun/IBM JSR implementing Many others: TimeSys Aphelion RTSJ (not yet Ajile, Apogee, Reference - Sun Java certified) Motorola, Nortel, Implementation Real-Time QNX, Thales, System TimeSys, - IBM's WindRiver webSphere RealTime QCon 2008 - Bertrand.Delsart@Sun.COM 10
  11. 11. Agenda • Background • Benefits of a Real-Time Java Virtual Machine • Benefits of Real-Time Garbage Collectors • Q&A QCon 2008 - Bertrand.Delsart@Sun.COM 11
  12. 12. RTSJ Use case 1: Send Market Data at a Fixed Rate Neither too late... nor too often ! • Mainstream Java solution: while (true) do { compute_data(); now = System.currentTimeMillis() Thread.sleep(next_period – now); send_data(); next_period += period; } • Problem: > Not guaranteed to wake-up quickly after the sleep call ! QCon 2008 - Bertrand.Delsart@Sun.COM 12
  13. 13. RTSJ RTSJ Benefit 1 : Priority Semantic You must specify the relative importance of the tasks, including with respect to the other processes. Mainstream “setPriority(10);” is not sufficient ! • RTSJ offers in Java usual real-time scheduling semantics > RealtimeThreads preempt non real-time ones > Higher priority threads preempt lower priority ones > Locks are properly handled (a low priority thread owning a critical resource will be boosted by the thread that requires it) QCon 2008 - Bertrand.Delsart@Sun.COM 13
  14. 14. RTSJ Use case 1 revisited • Code executed by a RealtimeThread setPriority(my_RTPriority); while (true) do { compute_data(); now = System.currentTimeMillis() Thread.sleep(next_period – now); send_data(); next_period += period; } • Problem: > What if a more important RT threads preempts me just before calling sleep ? QCon 2008 - Bertrand.Delsart@Sun.COM 14
  15. 15. RTSJ RTSJ Benefit 2: Rich Real-Time APIs • RTSJ provides what you will find in most RT Operating Systems > Absolute Time Clock, Timers, Non Blocking Queues... • RTSJ provides an additional layer to make your life simpler > Periodic Threads, Asynchronous Event Handlers, RawMemoryAccess, Deadline Miss Monitoring and Management, Asynchronous Transfer of Control... • RTSJ optionally defines advanced APIs > Cost Overflow Monitoring and Management, Feasibility Analysis... QCon 2008 - Bertrand.Delsart@Sun.COM 15
  16. 16. RTSJ Use case 1, RTSJ version • Code executed by a RealtimeThread setPriority(my_RTPriority); setReleaseParameters(myPeriodParam); while (true) do { compute_data(); RealtimeThread.waitForNextPeriod(); send_data(); } (other variants with Timers, AbsoluteTime wait, ...) • Problem: > Is this sufficient ? QCon 2008 - Bertrand.Delsart@Sun.COM 16
  17. 17. RTSJ Yes... if optimized for Determinism • The fastest Garbage Collectors suspend all the threads while they recycle memory • The fastest JIT compilers make some assumptions to generate more efficient code and must 'deoptimize' threads if the assumption becomes invalid • Solution 1: > RTSJ defines additional threads optimized for determinism and isolated for GC jitter. > They are as deterministic as C/C++ code ! QCon 2008 - Bertrand.Delsart@Sun.COM 17
  18. 18. RTSJ NoHeapRealtimeThreads • Code executed by a NoHeapRealtimeThread setPriority(my_RTPriority); setReleaseParameters(myPeriodParam); while (true) do { compute_data(); RealtimeThread.waitForNextPeriod(); send_data(); } • Problem: > What about memory allocation since 'isolated' from the GC ? QCon 2008 - Bertrand.Delsart@Sun.COM 18
  19. 19. RTSJ Memory areas for NHRTs Don't bother if 200 microseconds latencies are OK !!! • ImmortalMemory for non recycled objects • ScopedMemory for recycling > Memory areas not subject to the Garbage Collector > Per area counters to know how many threads are using an area > Objects automatically deleted when the count of their area is 0 > Dynamic read/write checks to guarantee the safety of this recycling QCon 2008 - Bertrand.Delsart@Sun.COM 19
  20. 20. RTSJ Use case 1, ScopedMemory version • Code executed by a NoHeapRealtimeThread while (true) do { scopedMemory1.enter(runnable); // sm1 recycled for the next loop } void run() { compute_data(); waitForNextPeriod(); send_data(); } • Problem: > Is send_data() endorsing read/write constraints ? QCon 2008 - Bertrand.Delsart@Sun.COM 20
  21. 21. Agenda • Background • Benefits of a Real-Time Java Virtual Machine • Benefits of Real-Time Garbage Collectors • Q&A QCon 2008 - Bertrand.Delsart@Sun.COM 21
  22. 22. RTGCs in General Making Java Predictable, RTGC • Real-Time GC > Essentially a “garbage collector with knobs” > Large interrupts are avoided at the cost of small, regular, and predictable collections > The GC runs often enough to recycle memory on time, depending on allocation and collection rates • Several models/approaches exist > IBM, Aicas, Sun, ... • Way simpler from a coding point of view > Single heap > No read/write checks QCon 2008 - Bertrand.Delsart@Sun.COM 22
  23. 23. RTSJ + RTGCs Overall System Model Real-time code and existing, non real-time, Real-Time JVM code can communicate, share memory, state, and overall environment – Non • Standard Java Heap something not possible in current real-time > Regular Java threads solutions real-time • Standard Java Heap or Soft Scoped or Immortal memory real-time > Real-Time threads > Real-Time Garbage Collector • Scoped/Immortal/Heap Hard > NoHeapReal-Time threads real-time > Real-Time threads > Hard Real-Time Garbage Collector QCon 2008 - Bertrand.Delsart@Sun.COM 23
  24. 24. Java RTS Sun's RTGC Apply concept of policy to Henriksson's approach • Goal: Hard RT latencies for high priority GC'd threads • Solution: implement GC as real-time threads > Running at a priority lower than the hard RT threads > And possibly concurrently with the other threads • Advantages > Scalable (no issues with multi-processor support) > Flexible QCon 2008 - Bertrand.Delsart@Sun.COM 24
  25. 25. Java RTS Default RTGC Scheduling Policy Could run concurrently Priority on another CPU NHRTs Critical RT threads Boosted GC Work at GC Max prio Soft RT threads Run-to-block GC Start GC Work at low priority Time Thread doing work Thread pause – priority interruption Boosting (Remaining Memory Too Low) GC active/running QCon 2008 - Bertrand.Delsart@Sun.COM 25
  26. 26. Java RTS Use case 1 revisited • Code executed by a hard RealtimeThread while (true) do { waitForNextPeriod(); send_data(); } • Code executed by a soft RealtimeThread while (true) do { compute_data(); waitForNextPeriod(); } • softRT.setDeadlineMissHandler(dmh) QCon 2008 - Bertrand.Delsart@Sun.COM 26
  27. 27. Java RTS Use case 1 Deadline Miss Handler • Hard Real-Time AsynchronousEventHandler void handleAsyncEvent() { quickly_compute_simpler_data(); softRTThread.schedulePeriodic(); } or void handleAsyncEvent() { softRTTThread.setPriority(hardPrio); softRTThread.schedulePeriodic(); } QCon 2008 - Bertrand.Delsart@Sun.COM 27
  28. 28. Java RTS Use Case 2 : Events Driven Request with Deadlines • A request is valid: > For a limited time > When a set of asynchronous input feeds dependent conditions are true • Proposed design: > Use a high priority critical thread for the request > If necessary, rely on priority boosting to safely use locks to check the <condition,deadline> pair(s) > Use AsynchronousEventHandlers to process the input feeds > Potentially at different priorities (hard and/or soft) > Use Minimum Inter-arrival Time policies to handle overflows QCon 2008 - Bertrand.Delsart@Sun.COM 28
  29. 29. Java RTS Use Case 3 : NASDAQ “Reduce I/O Jitter and Context Switches” • I/O must not hold other requests • Implemented Solution: > One thread per-CPU > One front-end to receive requests and executed non I/O ones > I/O requests dispatched to another thread > I/O completion can be seen as a new request by the front-end • Benefits from Java RTS deterministic compilation and memory management but : > What if I/O requests can be processed at different rates ? > Could I handle more non I/O requests ? QCon 2008 - Bertrand.Delsart@Sun.COM 29
  30. 30. Java RTS Use Case 3 : Why Care about Context Switches ? This should not be your role ! • Risk : under-utilization of resources • Scalable Design: > Chosen number of high priority time-critical front-end threads > Could even be at different priorities depending on the input stream > AsynchronousEventHandler to process I/Os at a lower priority > New I/O processing threads will automatically be created by the VM server thread pool to maximize the number of parallel requests > Requests started later can complete earlier > The server thread automatically grabs a new one... without context switch QCon 2008 - Bertrand.Delsart@Sun.COM 30
  31. 31. Java RTS Sun Java RTS Unique Selling Proposition Sun's Java Real-Time System – absolute execution predictability with all the benefits of the Java platform • Truly predictable, sample latency numbers: > RTGC: 200 microseconds Standard Java RTS > NHRT: 20 microseconds Java SE libraries libraries Sun Java RTS VM • Open > Based on open, SOLARIS 10 RTLinux community-driven SPARC x86 standards QCon 2008 - Bertrand.Delsart@Sun.COM 31
  32. 32. Java RTS Customer Evaluations • Over 500 companies and universities from more than 60 countries have evaluated Sun Java Real-Time System • Some data points from them: Maximum Latency Required Evaluations by Industry < 10 micros Financial serv- 10-100 mi- ices cros Telecom 100-1000 Military/ Aer- micros ospace 1-10 millis Automotive 10-100 millis Other industry Educational use Maximum latency Source: Evaluation download survey for Sun Java RTS QCon 2008 - Bertrand.Delsart@Sun.COM 32
  33. 33. Product Overview Key Benefits of RTSJ TECHNICAL BUSINESS • Maximize system utilization • Eliminate latency outliers • Cross-platform portability > Save millions of $ lost due to missed market opportunities, • Leverage standard OS's productivity losses etc. • Real-time tools • Eliminate risks inherent in custom and proprietary solutions • Leverage investment in existing Java code, skills • Increase developer productivity using Java QCon 2008 - Bertrand.Delsart@Sun.COM 33
  34. 34. Product Overview Key Benefits of RTSJ + RTGC TECHNICAL BUSINESS • Maximize system utilization • Eliminate latency outliers • Cross-platform portability > Save millions of $ lost due to missed market opportunities, • Leverage standard OS's productivity losses etc. • Real-time tools • Eliminate risks inherent in custom and proprietary • Minimize solution complexity solutions • Better design / architecture • Leverage investment choices in existing Java code, skills • Increase developer productivity using Java QCon 2008 - Bertrand.Delsart@Sun.COM 34
  35. 35. Product Overview Key Benefits of Sun Java RTS TECHNICAL BUSINESS • Maximize system utilization • Eliminate latency outliers even more (hard + soft RT) > Save millions of $ lost due to missed market opportunities, • Cross-platform portability productivity losses etc. • Leverage standard OS's • Eliminate risks inherent in • Real-time tools (DTrace) custom and proprietary solutions • Minimize solution complexity • Leverage investment • Better design / architecture in existing Java code, skills choices • Increase developer • Easy integration of hard, soft, productivity using Java and non time-critical design elements QCon 2008 - Bertrand.Delsart@Sun.COM 35
  36. 36. Questions? Bertrand.Delsart@Sun.COM
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×