Sovereign: Migrating Java Threads to Improve Availability of Web Applications

Strategies Recording Modiﬁed VM Outlook Sovereign: Migrating Java Threads to Improve Availability of Web Applications Johannes Eickhold, Technical University Munich Markus Knauer, EclipseSource March 22, 2011 eclipsesource-lojeick@so.in.tum.de Sovereign March 22, 2011 1 / 36

Strategies Recording Modiﬁed VM OutlookWhat is high availablility for web applications? Load Balancer Cluster Servlet Container Every node can take over Only works if app is stateless eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 2 / 36

Strategies Recording Modiﬁed VM OutlookWhat is RAP? RAP supports Reusing RCP’s UIThread → singe sourcing Job framework eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 3 / 36

Strategies Recording Modiﬁed VM OutlookChallenge: Transfer Server Side State Load Balancer sticky sessions! no-fail-over = true Cluster S S S Servlet Container UIThread UIThread UIThread Goal Enable fail-save operation of all RAP features. eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 4 / 36

Strategies Recording Modiﬁed VM OutlookOverview 1 Strategies 2 Recording Strategy Explained Example Demo 3 Modiﬁed VM Strategy Explained Demo 4 Outlook eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 5 / 36

Strategies Recording Modiﬁed VM OutlookTwo Strategies for Enabling Session Failover 1 Recording Strategy Records the widget state to shared heap Operates on application level 2 Modiﬁed VM Strategy Enables thread migration on the JVM level eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 7 / 36

Strategies Recording Modiﬁed VM OutlookRecording Strategy eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 8 / 36

Strategies Recording Modiﬁed VM OutlookModiﬁed VM Strategy licate threads rep primary VM secondary VM freeze thread + replicate state and data + continue thread = thread migration eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 9 / 36

Strategies Recording Modified VM OutlookRecording Strategy vs. Modified VM Strategy UIThread UIThread record replay byte code weaving unmodified application unmodified Hot Spot VM enhanced Maxine VM Terracotta shared heap object and thread replication eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 10 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoOverview 1 Strategies 2 Recording Strategy Explained Example Demo 3 Modiﬁed VM Strategy Explained Demo 4 Outlook eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 11 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoRecording Strategy Modiﬁcations to existing RAP framework 1 Record widget creation order on primary 2 Replay recorded widgets during creation phase on secondary 3 Cluster the session including widget tree 4 Remove non-serializable UIThread from session Incrementally add failover support for additional RAP widgets eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 12 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoLifecycle of a RAP application UI Thread Shared Heap request Client Initial request creates the UIThread eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 13 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoLifecycle of a RAP application UI Thread Shared Heap request Client UIThread executes createUI() and creates widget graph eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 13 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoLifecycle of a RAP application UI Thread Shared Heap record request Client The widget graph is stored on the shared heap eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 13 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoLifecycle of a RAP application UI Thread Shared Heap response Client Servlet responds with rendered UI to client. eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 13 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoHot Standby Takes Over UI Thread Shared Heap FAILURE request Client Primary servlet container fails and secondary jumps in eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 14 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoHot Standby Takes Over UI Thread UI Thread Shared Heap FAILURE request Client Primary servlet container fails and secondary jumps in eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 14 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoHot Standby Takes Over UI Thread UI Thread Shared Heap FAILURE request Client Skip widget creation in new UIThread! eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 14 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoFailover Situation UI Thread UI Thread Shared Heap FAILURE replay request Client Widgets are restored from the shared heap eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 15 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoFailover Situation UI Thread UI Thread Shared Heap FAILURE response Client We achieved transparent failover! eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 15 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoImplementation You could implement this yourself or reuse existing technologies. AspectJ is used to: Record order in which widgets are created Track method calls to enable method skipping Terracotta provides a shared heap eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 16 / 36

Strategies Recording Modified VM Outlook Strategy Explained Example DemoRecap of Recording Strategy Share Record TIMs: Equinox, Tracing widget Jetty creation with Configure RAP AspectJ (tc-config.xml) Skip methods on secondary No silver bullet! RAP application has to be designed for failover eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 17 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoJump to the example Example RAP application: "swt layout" Aspects implementing the recording/replay eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 18 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoSWT Layout Example1 Label l = new Label (shell , SWT.NONE );2 l. setText (" relocate me!");3 ...4 Button b = new Button (shell , SWT.PUSH );5 b. setText ("left");6 b. addSelectionListener (new SelectionAdapter () {7 public void widgetSelected ( SelectionEvent e) {8 localCounter ++;9 globalCounter ++;10 l. setText (" relocated "11 + localCounter + "/" + globalCounter12 + " times ");13 }14 }); eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 19 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoTrace RAP Widget Creation and Method CallsUIThread Button shared heap UIThread Button shared heap new() new() store restore setText() setText() update On secondary On primary eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 20 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoTraceRAPWidgetCreation.aspectj1 Object around () : traceRAPWidgetCreation () {2 if ( primary ) {3 widget = proceed ();4 // record object5 recordObject ( widget );6 return widget ;7 } else {8 // replay object9 widget = replayObject ();10 return widget ;11 }12 } eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 21 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example DemoTraceRAPMethodCalls.aspectj1 Object around () : traceRAPMethodCalls () {2 if ( primary ) {3 // normal operation4 return proceed ();5 } else {6 // do nothing7 return null;8 }9 } eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 22 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Example Demo Recording Demo eclipsesource-lojeick@so.in.tum.de Sovereign March 22, 2011 23 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoOverview 1 Strategies 2 Recording Strategy Explained Example Demo 3 Modiﬁed VM Strategy Explained Demo 4 Outlook eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 24 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoModiﬁed VM Strategy Extensions to the Maxine research VM Object replication Thread replication All VMs in web cluster execute: MigrationAgent + target application eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 25 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoMaxine VM Internals Maxine VM is written in Java (metacircular) Threads Uses pthreads One-on-one mapping: Java ↔ pthread Safepoint operations inserted @ backward branches Freezing of threads only via VmOperation Compilers Baseline: template JIT Optimizing: port of C1X from HotSpot or CPS Details in wiki: http://wikis.sun.com/display/MaxineVM eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 26 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoMigration via Agents RAP clustering Agents run next to Equinox in same VM Replication only used for UIThreads of RAP sessions migrationAgent.migrate(uiThread); eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 27 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoContinuous Thread Replication On primary: Client RequestThread Replicator UIThread Secondary HTTP request handle(request) notify(request) handle(request) finished replicate(UIThread) frames & objects HTTP response eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 28 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoReplication of Stack Frames On secondary: stack of ThreadStub nativeThreadCreate() VmThread.run() ThreadStub.run() receiving prepare migration wait Thread agent receives replicated thread eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 29 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoReplication of Stack Frames On secondary: stack buffer receiving MigratedThread.run() migration create frame n ... frame 2 frame 1 agent & rewire method1() method2() receives debug information ... methodN() eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 29 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoReplication of Stack Frames On secondary: stack of ThreadStub nativeThreadCreate() VmThread.run() ThreadStub.run() receiving migration resize frame n ... frame 2 frame 1 agent receives debug information eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 29 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoReplication of Stack Frames On secondary: stack of ThreadStub nativeThreadCreate() VmThread.run() ThreadStub.run() receiving migration frame n ... frame 2 frame 1 agent resize receives debug information eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 29 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoReplication of Stack Frames On secondary: stack of ThreadStub nativeThreadCreate() VmThread.run() ThreadStub.run() receiving MigratedThread.run() migration insert frame n ... frame 2 frame 1 agent buffer method1() method2() receives debug information ... methodN() eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 29 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoReplication of Stack Frames On secondary: stack of ThreadStub nativeThreadCreate() VmThread.run() ThreadStub.run() receiving MigratedThread.run() migration frame n ... frame 2 frame 1 agent rewire method1() method2() receives debug information ... methodN() ThreadStub.patch() eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 29 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained DemoInspecting the Migration Algorithm RCP based tooling: Inspect migrated entities @ VM level eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 30 / 36

Strategies Recording Modiﬁed VM Outlook Strategy Explained Demo Migration Demo eclipsesource-lojeick@so.in.tum.de Sovereign March 22, 2011 31 / 36

Strategies Recording Modiﬁed VM OutlookFuture Work 1 Future directions for clustered RAP: Support more SWT widgets JFace and Workbench Support alternatives to Terracotta 2 Potential for Java VMs with thread migration: Distributed Equinox Dynamic scaling for the cloud Vertical scaling of running applications 3 Improvements for migration agents: Migrate locked objects Migrate interlocked threads Migratable Resources Sockets Files eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 33 / 36

Strategies Recording Modiﬁed VM OutlookConclusion Clustering RAP’s UIThreads is possible Achieves transparent failover eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 34 / 36

Strategies Recording Modiﬁed VM Outlook Thank you for listening! Questions? eclipsesource-lojeick@so.in.tum.de Sovereign March 22, 2011 35 / 36

Strategies Recording Modiﬁed VM OutlookReferences http://www.so.in.tum.de/sovereign http://www.terracotta.org http://www.eclipse.org/aspectj http://wikis.sun.com/display/MaxineVM Quitadamo, Mobile JikesRVM: A framework to support transparent Java thread migration, 2008 eclipsesource-lo jeick@so.in.tum.de Sovereign March 22, 2011 36 / 36

Sovereign: Migrating Java Threads to Improve Availability of Web Applications

by Markus Knauer

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 210

Statistics

Sovereign: Migrating Java Threads to Improve Availability of Web Applications Presentation Transcript

http://www.eclipsecon.org	206
http://eclipsecon.org	4