Logging Last Resource Optimization for Distributed Transactions in Oracle Weblogic Server


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Logging Last Resource Optimization for Distributed Transactions in Oracle Weblogic Server

  1. 1. Logging Last Resource Optimization for Distributed Transactions in Oracle WebLogic Server T. Barnes, A. Messinger, P. Parkinson, A. Ganesh, G. Shegalov, S. Narayan, S. Kareenhalli
  2. 2. OLTP: Online Transaction ProcessingTransaction is an ACID contract● Atomic – all or nothing● Consistent – from the application perspective● Isolated – masked concurrency through locking or snapshots● Durable – once committed changes survive subsequent failures begin c -= 1000 Checking = 2000 s += 1000 Checking = 1000 Savings = 8000 commit Savings = 9000 time
  3. 3. OLTP: Single Resource ● A and D are typically implemented using Write-Ahead Logging ● Transaction recovery is “simple”: REDO phase, UNDO phase. BEGIN TRANSACTION /* LSN = 1: log for undo and redo in MM buffer*/ UPDATE Accounts SET balance = balance – 1000 WHERE Number = 1 /* LSN = 2: log for undo and redo in MM buffer*/ UPDATE Accounts SET balance = balance + 1000 WHERE Number = 2 /* LSN = 3: log commit and force (5-6 orders slower)*/ COMMIT TRANSACTION Accounts LSN=0 Accounts LSN=1 Accounts LSN=2 1 2000 1 1000 1 1000 2 8000 2 8000 2 9000
  4. 4. OLTP: Distributed / Two-Phase CommitLike a wedding ceremonyCoordinator: Will you ...? (prepare)Resource: I will (OK)Coordinator: I pronounce you … (commit) Transaction Resource 1 Resource 1 Coordinator prepare --> force-log prepare force-log prepare <-- OK <-- OK commit --> force-log commit force-log commit <-- ACK <-- ACK
  5. 5. 2PC is A CI D ● 2PC is not about Concurrency Control. ● 2PC transaction is therefore ○ Globally Atomic ○ Locally Isolated ○ Locally Consistent ○ Globally Durable
  6. 6. OLTP: Queued Transactionsclient` app server databasebegin transactionreq_q.enqueue(req1)commit transaction begin transaction creq = req_q.dequeue() resp = creq.execute() res_q.enqueue(resp) commit transactionbegin transactionresp = res_q.dequeue()process(resp)commit transaction
  7. 7. WebLogic Server: Java EE++● App containers: Web (Servlets, WS), EJB, app clients● Services: Messaging (JMS), Transactions (JTA), Database (JDBC), …
  8. 8. Example: Queued Transactions (JEE) @MessageDriven( mappedName="jms/inboundQueue“ activationConfig = {@ActivationConfigProperty( propertyName = “connectionFactoryJndiName", propertyValue = “jms/inboundConnectionFactory" )})) @TransactionAttribute(TransactionAttributeType.REQUIRED) public class OrderMDB implements javax.jms.MessageListener { @Resource javax.jms.Queue outboundQueue; @Resource javax.jms.ConnectionFactory outboundCf; @Resource javax.sql.DataSource orderDataSource; public void onMessage(Message message) { java.sql.Connection jdbc = orderDataSource.getConnection(); javax.jms.Connection jms = outboundCf.createConnection(); // update SQL via JDBC, notify via JMS connections … } }
  9. 9. “School” Presumed Abort 2PC 2n+1 writes, 4n messages TM Resources prepare force-log preparedTimeline yes all-prepared: force-log commit commit force-log commit ack all-commit: log end
  10. 10. PA2PC (1): TM (Coordinator)
  11. 11. PA2PC (2): Resource (Participant)
  12. 12. “Real Life” XA 2PC 2n+1 writes, 8n messages TM Resources xa_start ack_started xa_end ack_endedTimeline xa_prepare force-log prepared ack_prepared all-prepared: force-log commit xa_commit force-log commit ack_committed all-commit: log end
  13. 13. Standard 2PC Optimizations● 1PC: if only one resource enlisted, prepare skipped● Read-Only: if voted read-only, commit skipped● XA ceremony of xa_(start|end) is always present
  14. 14. Nested 2PC: Coordinator Role Transfer [Gray’78] prepare p commit TC Res2 Res3 c commit commit c ● Last Resource is committed in one phase ● 2n messages/ 2n-1 forced writes ● Known topology: linked Databases
  15. 15. WebLogic Design Constraints and Goals ● No control over foreign XAResource, TM and topology ● Broadband: minimize blocking RPC, not messages ● Unneeded XA on Res3: save xa_start, xa_end
  16. 16. Typical WLS Deployment● JMS and TM share the same FileStore● Collocated JMS connection cost is negligible● JDBC Datasource is remote: blocking RPC● DB internal resources (locks, latches, etc.) are more expensive and JEE is not a single client● Outbound JMS notifies about a JDBC update● Ideally: JDBC updates visible before JMS updates
  17. 17. JDBC as Logging Last Resource● User enables a non-XA JDBC Datasource as LLR ○ LLR table WL_LLR_<server> in the DS schema ○ No XA overhead for the LLR● TM log is local log UNION LLR table log ○ WLS does not boot if any LLR table is unavailable● Restriction: 1 LLR datasource / Transaction● No coordinator transfers as in Nested 2PC
  18. 18. XA 2PC Commit with LL Resource1. Prepare concurrently all non-LLR XAResources2. Insert XID into the LLR table3. Commit the LLR-Resource4. If 3 is successful, commit non-LLR XAResources5. Lazy garbage-collection of 2PC records of completed transactions is piggybacked on future LLR transactions
  19. 19. LLR Failure Recovery ● Failure before LLR.commit() => global abort ● Failure during LLR.commit() => similar to media failure ○ Wait until LLR Datasource / table is available for read ○ Presence of the LLR commit log decides the global outcome ○ If unavailable for AbandonTimeoutSeconds log abandoned ● JVM/OS crash: TM scan local log UNION LLR ○ Usual transaction outcome resolution ● 2PC recovery guarantees are not compromised
  20. 20. LLR Savings Back-of-the-envelope for the single-threaded case with Jeff Dean’s numbers [Google key notes]: ● xa_start (RPC), ● xa_end (RPC), ● xa_prepare (RPC + force-log) ● Insert into LLR table + commit done via single RPC ------------------------------------------------ 4xRTT + 1xDiskSeek = 4x500,000ns + 10,000,000ns = 12 milliseconds
  21. 21. LLR in DS Wizard: Non-XA Driver
  22. 22. LLR in DS Wizard: Safe unlikeEmulate
  23. 23. Research Workload EAStress2004 [SPECjAppServer’04]
  24. 24. EAStress2004 Benchmark HW/SetupDriver 1 Driver 2 External Supplier(3x Dealer, (3x Dealer, Emulator 3x Manufacturing) 3x Manufacturing)2x Quad Core 3.7Ghz 2x Quad Core 2.7Ghz 2x Quad Core 2.7Ghzx86_64, 2MB L2, 8GB x86_64, 2MB L2, 16GB x86_64, 2MB L2, 16GBRAM RAM RAM System Under TestOracle WebLogic Server 11gR1 Oracle RDBMS 11gR1 EE(Middle Tier) (Database Tier)2x Quad Core 2.83Ghz x86_64, 4x Quad Core 2.7Ghz x86_64,6MB L2, 16 GB RAM 2MB L2, 64GB RAM
  25. 25. Performance Evaluation (Utilization) EAStess2004 v1.08, IR = 700 (not reviewed by SPEC)MDB scenarioMT = WLS on JrockItDB = Oracle Database
  26. 26. Performance Evaluation (Response Time) EAStess2004 v1.08, IR = 700 (not reviewed by SPEC) Purchase Manage Browse Manufacturing XA 4.20 2.40 5.40 3.75 LLR 1.50 1.20 1.90 3.00 improvement 2.8x 2x 2.8x 1.25x
  27. 27. Future Improvements (probably in 12c)● LLR does not detect Read-Only● Transaction GUID instead of LLR table for Oracle
  28. 28. Thank You. Questions? oracle.com/weblogic oracle.com/benchmarks
  29. 29. WebLogic FileStore● XA-capable KV store on local file system● Mime design: allocate under write-head ○ fast writes ○ slow recovery ○ works well up to a couple of GiB● Transactional use: for JMS messages and JTA logs● Non-transactional use: Diagnostics and Config