Chen Wang, profile picture
Uploaded byChen Wang
PDF, PPTX872 views

Learning Greenplum Distributed Transaction

The document discusses distributed transaction processing in Greenplum. It covers the transaction framework, call flows for normal and retry cases, state transfers at the high and low levels, and the roles of the transaction manager and resource manager in distributed transactions. Key aspects include transaction blocks, states like TBLOCK_DEFAULT and TRANS_DEFAULT, protocol commands between the transaction manager and resource managers, and transaction logging.

Embed presentation

Download as PDF, PPTX
Learning Greenplum Distributed Transaction Code Analysis & Call flow
Transaction Framework Begin Transaction Client Server End Transaction UPDATE Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Execute SQL PostgreSQL TransactionStateData -TBlockState ： high-level stat -TransState ： low-level state
QE Distributed Transaction Transaction Framework Begin Transaction 客户端 Master End Transaction UPDATE Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Greenplum Segment 0 Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Execute SQL QD DTM Execute SQL QE Distributed Transaction Segment 0 Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Execute SQL Dtm Command Dtm Command TMGXACT -DtxState TransactionStateData -TBlockState ： high-level stat -TransState ： low-level state DtxContext TransactionStateData -TBlockState ： high-level stat -TransState ： low-level state DtxContext
Main Call Flow (Normal) Master(GP_ROLE_DISPATCH) Seg-N exec_simple_query start_xact_command ProcessUtility finish_xact_command doQEDistributedExplicitBegin setupQEDtxContext performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘M’ only for “BEGIN” firstchar = ‘T' doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND Middle Layer function low level function Middle Layer function low level function
Main Call Flow (Retry) Master(GP_ROLE_DISPATCH) Seg-N doDtxPhase2Retry StartTransactionCommand doDispatchDtxProtocolCommand CommitTransactionCommand DTX_CONTEXT_QD_RETRY_PHASE_2 When DTX_STATE_RETRY_COMMIT_PREPARED DTX_STATE_RETRY_COMMIT_PREPARED DTX_STATE_RETRY_ABORT_PREPARED doInsertForgetCommitted performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND
Middle Layer function • Normal system's calls • StartTransactionCommand • CommitTransactionCommand • AbortCurrentTransaction • User change calls by issue “BEGIN, COMMIT, ROLLBACK, SAVEPOINT, ROLLBACK TO or RELEASE” • BeginTransactionBlock • EndTransactionBlock • UserAbortTransactionBlock • PrepareTransactionBlock • DefineSavepoint • RollbackToSavepoint • ReleaseSavepoint
Low level calls • StartTransaction • CommitTransaction • AbortTransaction • CleanupTransaction • PrepareTransaction • StartSubTransaction • CommitSubTransaction • AbortSubTransaction • CleanupSubTransaction
Postgres Normal middle level call flow / StartTransactionCommand; / StartTransaction; 1) < ProcessUtility; << BEGIN BeginTransactionBlock; CommitTransactionCommand; / StartTransactionCommand; 2) / ProcessQuery; << SELECT ... CommitTransactionCommand; CommandCounterIncrement; / StartTransactionCommand; 3) / ProcessQuery; << INSERT ... CommitTransactionCommand; CommandCounterIncrement; / StartTransactionCommand; / ProcessUtility; << COMMIT 4) < EndTransactionBlock; CommitTransactionCommand; CommitTransaction; 1) BEGIN 2) SELECT * FROM foo 3) INSERT INTO foo VALUES (...) 4) COMMIT
High Level State transfer TBLOCK_DEFAULT TBLOCK_START TBLOCK_ABORT_PENDING TBLOCK_BEGIN TBLOCK_INPROGRESS TBLOCK_ABORT TBLOCK_END TBLOCK_ABORT_END BeginTransactionBlock StartTransactionCommand AbortCurrentTransaction UserAbortTransactionBlock EndTransactionBlock CommitTransactionCommand AbortCurrentTransaction CommitTransactionCommand AbortCurrentTransaction CommitTransactionCommand AbortCurrentTransaction EndTransactionBlock UserAbortTransactionBlock AbortCurrentTransaction AbortCurrentTransactionCommitTransactionCommand PostgreSQL TBLOCK_PREPARE PrepareTransactionBlock CommitTransactionCommand failed xact failed xact AbortOutOfAnyTransaction AbortOutOfAnyTransaction CommitTransactionCommand
High Level State transfer TBLOCK_SUBEND TBLOCK_SUBBEGIN TBLOCK_SUBABORT_PENDING TBLOCK_SUBINPROGRESS TBLOCK_SUBABORT TBLOCK_SUBABORT_RESTART TBLOCK_SUBABORT_END TBLOCK_SUBRESTART PostgreSQL CommitTransactionCommand PushTransaction ReleaseSavepoint EndTransactionBlock AbortCurrentTransaction UserAbortTransactionBlock EndTransactionBlock EndTransactionBlock UserAbortTransactionBlock RollbackToSavepoint RollbackToSavepoint RollbackToSavepoint RollbackToSavepoint failed xact failed xact failed xact
Detailed transfer TBLOCK_DEFAULT TBLOCK_START TBLOCK_ABORT_PENDING StartTransactionCommand AbortCurrentTransaction UserAbortTransactionBlock CommitTransactionCommand AbortCurrentTransaction CommitTransactionCommand Single Query Transaction Error Happened Rollback issued
Detailed transfer TBLOCK_DEFAULT TBLOCK_START TBLOCK_BEGIN TBLOCK_INPROGRESS BeginTransactionBlock StartTransactionCommand CommitTransactionCommand Explicit Begin AbortCurrentTransaction Error Happened
Detailed transfer TBLOCK_DEFAULT TBLOCK_INPROGRESS TBLOCK_END EndTransactionBlock CommitTransactionCommand AbortCurrentTransaction TBLOCK_PREPARE PrepareTransactionBlock CommitTransactionCommand AbortOutOfAnyTransaction Commit/End Error Happened
Detailed transfer TBLOCK_INPROGRESS TBLOCK_ABORT AbortCurrentTransaction failed xact SQL Error Happened in Transaction
Detailed transfer TBLOCK_DEFAULT TBLOCK_INPROGRESS TBLOCK_ABORT TBLOCK_ABORT_END CommitTransactionCommand AbortCurrentTransaction EndTransactionBlock UserAbortTransactionBlock failed xact failed xact AbortOutOfAnyTransaction Rollback TBLOCK_ABORT_PENDING CommitTransactionCommand AbortCurrentTransaction UserAbortTransactionBlock
Low Level State transfer TRANS_DEFAULT TRANS_START TRANS_COMMITTRANS_INPROGRESS TRANS_ABORT TRANS_PREPARE StartTransactionStartTransaction CommitTransaction PrepareTransaction PrepareTransaction CommitTransaction AbortTransaction CleanupTransaction
TM Control Block - TMGXACT char gid[TMGIDSIZE]; DistributedTransactionId gxid; DtxState state; int sessionId; TransactionId localXid; LocalDistribXactRef localDistribXactRef; bool explicitBeginRemembered; DistributedTransactionId xminDistributedSnapshot; bool bumpedPhase1Count; bool badPrepareGangs; bool retryPhase2RecursionStop; int debugIndex; bool directTransaction; uint16 directTransactionContentId;
TM Actions to RM • Normal • doPrepareTransaction • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_PREPARE) • doInsertForgetCommitted • Insert FORGET COMMITTED into the xlog. • doNotifyingCommitPrepared • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_COMMIT_PREPARED) • doNotifyingAbort • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_ABORT_PREPARED) • Crash Recovery • doNotifyCommittedInDoubt • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_RECOVERY_COMMIT_PR EPARED) • doAbortInDoubt • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREP ARED)
TM Actions to RM • Others • sendDtxExplicitBegin • dispatchDtxCommand(“BEGIN ISOLATION LEVEL %s, READ %s”) • DefineDispatchSavepoint • dispatchDtxCommand(“savepoint”) • ReleaseSavepoint • dispatchDtxCommand(“release savepoint”) • DispatchRollbackToSavepoint • dispatchDtxCommand(“rollback to savepoint”)
TM dispatch command to RM • doDispatchDtxProtocolCommand • cdbdisp_dispatchDtxProtocolCommand • cdbdisp_dispatchToGang(GP_DISPATCH_COMMAND_TYPE_DTX_PROTOC OL) • CdbCheckDispatchResult(&ds, DISPATCH_WAIT_NONE); • ----------------wait for QE------------------ • cdbdisp_returnResults • cdbdisp_destroyDispatchThreads
DTM to RM protocol commands • DTX_PROTOCOL_COMMAND_NONE • DTX_PROTOCOL_COMMAND_STAY_AT_OR_BECOME_IMPLIED_WRITER = 1, • DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED, • DTX_PROTOCOL_COMMAND_PREPARE, • DTX_PROTOCOL_COMMAND_ABORT_SOME_PREPARED, • DTX_PROTOCOL_COMMAND_COMMIT_PREPARED, • DTX_PROTOCOL_COMMAND_ABORT_PREPARED, • DTX_PROTOCOL_COMMAND_RETRY_COMMIT_PREPARED, • DTX_PROTOCOL_COMMAND_RETRY_ABORT_PREPARED, • DTX_PROTOCOL_COMMAND_RECOVERY_COMMIT_PREPARED, • DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREPARED, • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_BEGIN_INTERNAL, • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_ROLLBACK_INTERNAL, • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_RELEASE_INTERNAL,
TM State transfer DTX_STATE_NONE DTX_STATE_ACTIVE_DISTRIBUTED DTX_STATE_PREPARING DTX_STATE_INSERTING_COMMITTED DTX_STATE_INSERTED_COMMITTED DTX_STATE_FORCED_COMMITTED DTX_STATE_INSERTING_FORGET_COMMITTED DTX_STATE_INSERTED_FORGET_COMMITTED DTX_STATE_NOTIFYING_COMMIT_PREPARED doPrepareTransaction DTX_STATE_PREPARED doPrepareTransaction DTX_STATE_NOTIFYING_ABORT_SOME_PREPARED DTX_STATE_NOTIFYING_ABORT_NO_PREPARED DTX_STATE_NOTIFYING_ABORT_PREPARED RecordTransactionCommit/ forcedDistributedCommitted doNotifyingCommitPrepared doInsertForgetCommitted RecordTransactionCommit/insertingDistributedCommitted rollbackDtxTransaction DTX_STATE_RETRY_ABORT_PREPARED rollbackDtxTransaction(badPrepareGangs) rollbackDtxTransaction(badPrepare) rollbackDtxTransaction(xa rollback??) DTX_STATE_RETRY_COMMIT_PREPARED doNotifyingCommitPrepared(commit failed) doDtxPhase2Retry/doInsertForgetCommitted ateDtx/initGxact startTransaction/createDtx DTX_STATE_ACTIVE_NOT_DISTRIBUTED dtmPreCmd
DTM State • Uninitialized transaction • DTX_STATE_NONE = 0 • The distributed transaction is active • DTX_STATE_ACTIVE_NOT_DISTRIBUTED //auto-commit on the QEs • DTX_STATE_ACTIVE_DISTRIBUTED //it is explicit or an implicit writer transaction
DTM State • For two-phase commit, the first phase is about to run • DTX_STATE_PREPARING • For two-phase commit, the first phase has completed • DTX_STATE_PREPARED • DTX_STATE_INSERTING_COMMITTED • DTX_STATE_INSERTED_COMMITTED • DTX_STATE_FORCED_COMMITTED • DTX_STATE_NOTIFYING_COMMIT_PREPARED • DTX_STATE_INSERTING_FORGET_COMMITTED • DTX_STATE_INSERTED_FORGET_COMMITTED
DTM State • Transaction rollback has been requested and QD is notifying all QD processes • no QEs have started work on the first phase of two-phase commit • DTX_STATE_NOTIFYING_ABORT_NO_PREPARED • at least one QE has done the first phase of two-phase commit • DTX_STATE_NOTIFYING_ABORT_SOME_PREPARED • the QE processes have done the first phase of two-phase commit • DTX_STATE_NOTIFYING_ABORT_PREPARED, • DTX_STATE_RETRY_COMMIT_PREPARED, • DTX_STATE_RETRY_ABORT_PREPARED, • DTX_STATE_CRASH_COMMITTED
DTX Context • DTX_CONTEXT_LOCAL_ONLY • There is no distributed transaction. This is the initial state and, for utility mode connections or master-only queries, the only state. It is also the state to which the QD and QE return to between transactions. • DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE, • On QD: the process is currently part of a distributed transaction. Whether or not the transaction has been started on a QE is not a part of the QD state -- that is tracked by assigning one of the DTX_CONTEXT_QE* values on the QE process, and by updating the state field of the currentGxact. • DTX_CONTEXT_QD_RETRY_PHASE_2, • On QD: indicates that the second-phase of the two-phase commit is being retried.
DTX Context • DTX_CONTEXT_QE_ENTRY_DB_SINGLETON, • TODO: how is something an Entry db? fix this documentation • DTX_CONTEXT_QE_AUTO_COMMIT_IMPLICIT, • On a QE that is the root of a query, this context means that the distributed transaction was opened implicitly by a non-writing query in an auto-commit transaction. • This is essentially the same as _TWO_PHASE_IMPLICIT_WRITER except we know the query won't dirty anything and so we don't need a two-phase commit. • DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER, • On a QE, this context means that the distributed transaction was opened explicitly by a BEGIN query statement Note that this state can happen even if there are no writer queries (because this state is entered when the BEGIN statement is processed). • DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER, • On a QE that is the root of a query, this context means that the distributed transaction was opened implicitly by a writing query in an auto-commit transaction, and this is the writing QE process Same as _AUTO_COMMIT_IMPLICIT except that we need a two-phase of the dirty data. • DTX_CONTEXT_QE_READER, • On a QE that is not the root of a query, this context is used. This can happen regardless of whether the overall transaction for the query is implicit or explicit, and auto-commit vs two-phase • DTX_CONTEXT_QE_PREPARED, • On the QE, the two-phase commit has been prepared (first phase complete). • DTX_CONTEXT_QE_FINISH_PREPARED • On the QE, the two-phase commit has been committed or aborted (second phase complete).
TM: ProcessUtility • case nodeTag(parsetree) = T_TransactionStmt • Case Stmt->kind: • -----normal----- • TRANS_STMT_BEGIN or TRANS_STMT_START • BeginTransactionBlock • sendDtxExplicitBegin • TRANS_STMT_COMMIT • EndTransactionBlock • TRANS_STMT_ROLLBACK • UserAbortTransactionBlock
TM/RM : ProcessUtility • case nodeTag(parsetree) = T_TransactionStmt • Case Stmt->kind: • -----savepoint----- • TRANS_STMT_SAVEPOINT • RequireTransactionChain • sendDtxExplicitBegin • DefineDispatchSavepoint • TRANS_STMT_RELEASE • RequireTransactionChain • ReleaseSavepoint • TRANS_STMT_ROLLBACK_TO • RequireTransactionChain • RollbackToSavepoint
RM: ProcessUtility ??? XA??? • case nodeTag(parsetree) = T_TransactionStmt • Case Stmt->kind: • -----I think it’s XA feature, not know when called in RM---- • TRANS_STMT_PREPARE • PrepareTransactionBlock • TRANS_STMT_COMMIT_PREPARED • PreventTransactionChain • FinishPreparedTransaction • TRANS_STMT_ROLLBACK_PREPARED • PreventTransactionChain • FinishPreparedTransaction
RM: doQEDistributedExplicitBegin • Handle ExplicitBegin • StartTransactionCommand(); • BeginTransactionBlock(); • CommitTransactionCommand();
RM: performDtxProtocolCommand • Case dtxProtocolCommand • DTX_PROTOCOL_COMMAND_STAY_AT_OR_BECOME_IMPLIED_WRITER • Case DistributedTransactionContext: • DTX_CONTEXT_LOCAL_ONLY • setupQEDtxContext • StartTransactionCommand • DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER • DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED • AbortOutOfAnyTransaction • DTX_PROTOCOL_COMMAND_PREPARE • Case DistributedTransactionContext: • DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER or DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER • performDtxProtocolPrepare • DTX_PROTOCOL_COMMAND_ABORT_SOME_PREPARED • Case DistributedTransactionContext: • DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER or DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER • AbortOutOfAnyTransaction • DTX_CONTEXT_QE_PREPARED • performDtxProtocolAbortPrepared • DTX_PROTOCOL_COMMAND_COMMIT_PREPARED • performDtxProtocolCommitPrepared • DTX_PROTOCOL_COMMAND_ABORT_PREPARED • performDtxProtocolAbortPrepared
RM: performDtxProtocolCommand • Case dtxProtocolCommand • DTX_PROTOCOL_COMMAND_RETRY_COMMIT_PREPARED • performDtxProtocolCommitPrepared • DTX_PROTOCOL_COMMAND_RETRY_ABORT_PREPARED • performDtxProtocolAbortPrepared • DTX_PROTOCOL_COMMAND_RECOVERY_COMMIT_PREPARED • performDtxProtocolCommitPrepared • DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREPARED • performDtxProtocolAbortPrepared • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_BEGIN_INTERNAL • Case DistributedTransactionContext: • DTX_CONTEXT_LOCAL_ONLY • setupQEDtxContext • StartTransactionCommand • BeginInternalSubTransaction • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_ROLLBACK_INTERNAL • RollbackAndReleaseCurrentSubTransaction • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_RELEASE_INTERNAL • ReleaseCurrentSubTransaction
Log Type • Nomal log: • #define XLOG_XACT_COMMIT 0x00 • #define XLOG_XACT_ABORT 0x20 • 2PC Log • RM Log • #define XLOG_XACT_PREPARE 0x10 • #define XLOG_XACT_COMMIT_PREPARED 0x30 • #define XLOG_XACT_ABORT_PREPARED 0x40 • TM log • #define XLOG_XACT_DISTRIBUTED_COMMIT 0x50 • #define XLOG_XACT_DISTRIBUTED_FORGET 0x60
TM Log write • RecordTransactionCommit if (isDtxPrepared) { XLogInsert(RM_XACT_ID, XLOG_XACT_DISTRIBUTED_COMMIT) } else { XLogInsert(RM_XACT_ID, XLOG_XACT_COMMIT); } XLogFlush() • RecordDistributedForgetCommitted XLogInsert(RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN)
TM Log write • XLOG_XACT_DISTRIBUTED_COMMIT • XLogRecData rdata[4]; 0 1 2 3 XLogRecData Data: xl_xact_commit Len: MinSizeOfXactCommit Data: persistentCommitBuffer Len: persistentCommitSerializeLen Data: children Len: nchildren * sizeof(TransactionId) Data: TMGXACT_LOG Len: sizeof(TMGXACT_LOG) persistentCommitObjectCount > 0 nchildren > 0 isDtxPrepared
TM Log write • XLOG_XACT_DISTRIBUTED_FORGET • XLogRecData rdata[4]; 0 XLogRecData Data: xl_xact_distributed_forget Len: sizeof(xl_xact_distributed_forget) TMGXACT_LOG
RM Log write • Prepare LOG • Log state format 1. TwoPhaseFileHeader 2. TransactionId[] (subtransactions) 3. RelFileNode[] (files to be deleted at commit) 4. RelFileNode[] (files to be deleted at abort) 5. TwoPhaseRecordOnDisk 6. ... 7. TwoPhaseRecordOnDisk (end sentinel, rmid == TWOPHASE_RM_END_ID) 8. CRC32 • XLog
RM Log write • Log state format
RM Log write • XLOG_XACT_PREPARE • XLogRecData rdata[n]; 0 1 2 XLogRecData Data: TwoPhaseFileHeader Len: TwoPhaseFileHeader or 512 Data: children Len: nchildren * sizeof(TransactionId) Data: persistentCommitBuffer Len: persistentCommitSerializeLen nchildren > 0 persistentCommitObjectCount > 0 3 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_NOTIFY_ID） Len: sizeof(TwoPhaseRecordOnDisk) 3 Data: relname Len: sizeof(relname) 4 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_FLATFILES_ID） Len: sizeof(TwoPhaseRecordOnDisk) info != 0 5 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_INVAL_ID） Len: sizeof(TwoPhaseRecordOnDisk) RelcacheInitFileInval != NULL 6 x n Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_LOCK_ID） Len: sizeof(TwoPhaseRecordOnDisk) Has locks 6 x n Data: TwoPhaseLockRecord Len: sizeof(TwoPhaseLockRecord) Has locks 7 x n Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_PGSTAT_ID） Len: sizeof(TwoPhaseRecordOnDisk) 7 x n Data: TwoPhasePgStatRecord Len: sizeof(TwoPhasePgStatRecord) xact_state != NULL 8 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_END_ID） Len: sizeof(TwoPhaseRecordOnDisk)
RM Log write • XLOG_XACT_DISTRIBUTED_COMMIT • XLogRecData rdata[4] 0 1 2 XLogRecData Data: xl_xact_commit_prepared Len: MinSizeOfXactCommitPrepared Data: persistentCommitBuffer Len: persistentCommitSerializeLen Data: children Len: nchildren * sizeof(TransactionId) persistentPrepareObjectCount > 0 nsubxacts > 0
RM Log write • XLOG_XACT_DISTRIBUTED_COMMIT • XLogRecData rdata[3]; 0 1 2 XLogRecData Data: xl_xact_abort_prepared Len: MinSizeOfXactAbortPrepared Data: persistentCommitBuffer Len: persistentCommitSerializeLen Data: children Len: nchildren * sizeof(TransactionId) persistentCommitObjectCount > 0 nchildren > 0
RM Log write • EndPrepare XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE) XLogFlush() • RecordTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) XLogFlush() only when we are deleting files • RecordTransactionCommitPrepared XLogInsert(RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_NO_TRAN) XLogFlush() • RecordTransactionAbortPrepared XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT_PREPARED | XLOG_NO_TRAN) XLogFlush()
Log Write Seq when Commit Master(GP_ROLE_DISPATCH) Seg-N XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_PREPARE) XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_COMMIT) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_NO_ XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_ABORT_PREPARED | XLOG_NO_T XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_NO_ Commit Rollback Retry Commit Retry Rollback
Test Details • BEGIN • INSERT • END • MANUNALLY ROLLBACK • INSERT FAILED WITH AUTO ROLLBACK • Kill RM-0 before master write log, RM-1 Prepared • Kill RM-0 when master send commit to RMs, RM all Prepared • Error happened when master send commit to RMs, no Gangs errors • Crash recovery
PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar = 'Q' exec_simple_query start_xact_command StartTransactionCommand StartTransaction Block state: TBLOCK_DEFAULT -> TBLOCK_STARTED State: TRANS_DEFAULT -> TRANS_START PortalRun ProcessUtility TRANS_STMT_START BeginTransactionBlock Block state: TBLOCK_STARTED -> TBLOCK_BEGIN sendDtxExplicitBegin DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE createDtx Gxact->state : DTX_STATE_NONE -> DTX_STATE_ACTIVE_NOT_DISTRIBUTED dispatchDtxCommand cdbdisp_dispatchRMCommand cdbdisp_dispatchCommand cdbdisp_dispatchToGang CdbCheckDispatchResult cdbdisp_returnResults cdbdisp_destroyDispatchThreads CommitTransactionCommand finish_xact_command Block state: TBLOCK_BEGIN -> TBLOCK_INPROGRESS doQEDistributedExplicitBegin StartTransactionCommand BeginTransactionBlock CommitTransactionCommand setupQEDtxContext needTwoPhase = false; explicitBegin = true Block state: TBLOCK_DEFAULT -> TBLOCK_STARTED State: TRANS_DEFAULT -> TRANS_START StartTransaction firstchar = ‘M' Block state: TBLOCK_STARTED -> TBLOCK_BEGIN Block state: TBLOCK_BEGIN -> TBLOCK_INPROGRESS State: TRANS_START -> TRANS_INPROGRESS State: TRANS_START -> TRANS_INPROGRESS pq_flush ReadyForQuery_QEWriter DTX_CONTEXT_LOCAL_ONLY DistributedTransactionContext: DTX_CONTEXT_LOCAL_ONLY -> DTX_CONTEX BEGIN; --------------------wait---------------------- dtmPreCommand Gxact->state: DTX_STATE_ACTIVE_NOT_DISTRIBUTED -> DTX_STATE_ACTIVE_DISTRIBUTED
PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar = 'Q' exec_simple_query start_xact_command StartTransactionCommand TBLOCK_INPROGRESS PortalRun PortalRunMulti ProcessQuery ExecutorStart start_xact_command exec_mpp_query firstchar = ‘M' dtmPreCommand StartTransactionCommand INSERT ; CommitTransactionCommand finish_xact_command --------------------wait---------------------- PortalRun PortalRunMulti ProcessQuery ExecutorRun ExecutePlan ExecInsert heap_insert heap_log_tuple_insert heap_log_tuple_insert XLogInsert (RM_HEAP_ID) CommitTransactionCommand finish_xact_command ExecutorEnd mppExecutorFinishup CdbCheckDispatchResult CommandId++
PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar = 'Q' exec_simple_query start_xact_command StartTransactionCommand TBLOCK_INPROGRESS PortalRun ProcessUtility TRANS_STMT_COMMIT EndTransactionBlock Block state: TBLOCK_INPROGRESS -> TBLOCK_END CommitTransactionCommand finish_xact_command BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command State: TRANS_BEGIN -> TRANS_PREPARE firstchar = ‘T' CommitTransaction prepareDtxTransaction dtxProtocolCommand = DTX_PROTOCOL_COMMAND_PREPARE DistributedTransactionContext == DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_ performDtxProtocolPrepare StartTransactionCommand PrepareTransactionBlock CommitTransactionCommand DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER -> DTX_CONTEXT_Q EndTransactionBlock Block state: TBLOCK_INPROGRESS -> TBLOCK_END Block state: TBLOCK_END -> TBLOCK_PREPARE PrepareTransaction XLogFlush Block state: TBLOCK_PREPARE -> TBLOCK_DEFAULT pq_flush ReadyForQuery_QEWriter END/COMMIT 1; doPrepareTransaction Gxactstate: DTX_STATE_ACTIVE_DISTRIBUTED ->DTX_STATE_PREPARING doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_PREPARE cdbdisp_dispatchDtxProtocolCommand Gxactstate: DTX_STATE_PREPARING -> DTX_STATE_PREPARED cdbdisp_returnResults EndPrepare --------------------wait---------------------- XLogInsert (RM_XACT_ID, XLOG_XACT_PR State: TRANS_INPROGRESS -> TRANS_COMMIT State: TRANS_PREPARE -> TRANS_DEFAULT StartPrepare gxact = MarkAsPreparing()
Master(GP_ROLE_DISPATCH) Seg-N PostgresMain BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' dtxProtocolCommand = DTX_PROTOCOL_COMMAND_COMMIT_PREPARED performDtxProtocolCommitPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_LOCAL_ONLY FinishPreparedTransaction Block state: TBLOCK_DEFAULT -> TBLOCK_STARTED Block state: TBLOCK_STARTED -> TBLOCK_DEFAULT State: TRANS_DEFAULT -> TRANS_START StartTransaction DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED State: TRANS_START -> TRANS_INPROGRESS State:TRANS_INPROGRESS ->TRANS_COMMIT CommitTransaction RecordTransactionCommit XLogFlush END/COMMIT 2; doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_COMMIT_PREPARED cdbdisp_dispatchDtxProtocolCommand cdbdisp_returnResults doNotifyingCommitPrepared notifyCommittedDtxTransaction Dtx state : DTX_STATE_FORCED_COMMITTED -> DTX_STATE_NOTIFYING_COMMIT_PREPARED doInsertForgetCommitted Dtx state : DTX_STATE_NOTIFYING_COMMIT_PREPARED -> DTX_STATE_INSERTING_FORGET_COMMITTED Dtx state : DTX_STATE_NOTIFYING_COMMIT_PREPARED -> DTX_STATE_INSERTED_FORGET_COMMITTED doInsertForgetCommitted RecordDistributedForgetCommitted RecordTransactionCommit forcedDistributedCommitted Dtx state : DTX_STATE_INSERTED_COMMITTED-> DTX_STATE_FORCED_COMMITTED --------------------wait---------------------- insertingDistributedCommitted Dtx state : DTX_STATE_PREPARED -> DTX_STATE_INSERTING_COMMITTED XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_COMMIT) insertedDistributedCommitted Dtx state : DTX_STATE_INSERTING_COMMITTED -> DTX_STATE_INSERTED_COMMITTED RecordTransactionCommitPrepared XLogFlush CommitTransactionCommand finish_xact_command CommitTransaction …… …… XLogFlush …… XLogInsert (RM_XACT_ID, XLOG_XACT_CO XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) Block state: TBLOCK_END -> TBLOCK_DEFAULT State:TRANS_COMMIT ->TRANS_DEFAULT State:TRANS_COMMIT ->TRANS_DEFAULT
PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar = 'Q' exec_simple_query start_xact_command StartTransactionCommand TBLOCK_INPROGRESS PortalRun ProcessUtility TRANS_STMT_ROLLBACK UserAbortTransactionBlock Block state: TBLOCK_INPROGRESS -> TBLOCK_ABORT_PENDING CommitTransactionCommand finish_xact_command BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' AbortTransaction rollbackDtxTransaction dtxProtocolCommand = DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED DistributedTransactionContext == DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_ AbortOutOfAnyTransaction pq_flush ReadyForQuery_QEWriter ROLLBACK; Gxactstate: DTX_STATE_ACTIVE_DISTRIBUTED ->DTX_STATE_NOTIFYING_ABORT_NO_PREPARED doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED cdbdisp_dispatchDtxProtocolCommand cdbdisp_returnResults --------------------wait---------------------- CleanupTransaction Block state: TBLOCK_ABORT_PENDING -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT doNotifyingAbort AbortTransaction CleanupTransaction Block state: TBLOCK_ABORT_PENDING -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT State: TRANS_ABORT -> TRANS_DEFAULT State: TRANS_ABORT -> TRANS_DEFAULT RecordTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT)
PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar = 'Q' AbortCurrentTransaction AbortTransaction BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' rollbackDtxTransaction dtxProtocolCommand = DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED DistributedTransactionContext == DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_ AbortOutOfAnyTransaction pq_flush ReadyForQuery_QEWriter INSERT FAILED; Gxactstate: DTX_STATE_ACTIVE_DISTRIBUTED ->DTX_STATE_NOTIFYING_ABORT_NO_PREPARED doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED cdbdisp_dispatchDtxProtocolCommand cdbdisp_returnResults --------------------wait---------------------- CleanupTransaction Block state: TBLOCK_ABORT_END -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT doNotifyingAbort AbortTransaction CleanupTransaction Block state: TBLOCK_ABORT_PENDING -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT State: TRANS_ABORT -> TRANS_DEFAULT State: TRANS_ABORT -> TRANS_DEFAULT RecordTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) Need to call rollback or end manually here!!! PostgresMain firstchar = 'Q' exec_simple_query start_xact_command StartTransactionCommand Block state: TBLOCK_INPROGRESS -> TBLOCK_ABORT PortalRun ProcessUtility TRANS_STMT_ROLLBACK UserAbortTransactionBlock Block state: TBLOCK_ABORT -> TBLOCK_ABORT_END CommitTransactionCommand finish_xact_command
Master(GP_ROLE_DISPATCH) Seg-N Kill RM-0 before master write log, RM-1 Prepared --------------------some RM Failed---------------------- PostgresMain CommitTransactionCommand finish_xact_command CommitTransaction prepareDtxTransaction doPrepareTransaction AbortCurrentTransaction TRANS_INPROGRESS TBLOCK_END DTX_STATE_PREPARING AbortTransaction rollbackDtxTransaction Gxactstate: DTX_STATE_PREPARING -> DTX_STATE_RETRY_ABORT_PREPARED State: TRANS_INPROGRESS -> TRANS_ABORT disconnectAndDestroyAllGangs GP Kill RM-1 doDtxPhase2Retry StartTransactionCommand DTX_STATE_RETRY_COMMIT_PREPARED StartTransaction TRANS_DEFAULT -> TRANS_START doDispatchDtxProtocolCommand GP Renew all RMs PostgresMain BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' dtxProtocolCommand = DTX_PROTOCOL_COMMAND_ABORT_PREPARED performDtxProtocolAbortPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand DTX_CONTEXT_LOCAL_ONLY -> DTX_CONTEXT_QD_RETRY_PHASE_2 DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE -> DTX_CONTEXT_LOCAL_ONLY CommitTransactionCommand DTX_CONTEXT_QD_RETRY_PHASE_2 -> DTX_CONTEXT_LOCAL_ONLY finishDistributedTransactionContext Do nothing on RM-0 Do Abort Prepared on RM-1 performDtxProtocolAbortPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand RecordTransactionAbortPrepared XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_ABORT_PRE GP Kill all RMs then DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED
Master(GP_ROLE_DISPATCH) Seg-N Kill RM-0 when master send commit to RMs --------------------some RM Failed---------------------- PostgresMain disconnectAndDestroyAllGangs GP Kill RM-1 doDtxPhase2Retry StartTransactionCommand DTX_STATE_RETRY_COMMIT_PREPARED StartTransaction TRANS_DEFAULT -> TRANS_START DTX_PROTOCOL_COMMAND_COMMIT_PREPARED doDispatchDtxProtocolCommand GP Renew all RMs DTX_CONTEXT_LOCAL_ONLY -> DTX_CONTEXT_QD_RETRY_PHASE_2 DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE -> DTX_CONTEXT_LOCAL_ONLY CommitTransactionCommand DTX_CONTEXT_QD_RETRY_PHASE_2 -> DTX_CONTEXT_LOCAL_ONLY Do nothing on RM-0 Do Commit Prepared on RM-1 performDtxProtocolCommitPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand RecordTransactionCommitPrepared XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPAR doInsertForgetCommitted Dtx state : DTX_STATE_RETRY_COMMIT_PREPARED -> DTX_STATE_INSERTING_FORGET_COMMITTED Dtx state : DTX_STATE_INSERTING_FORGET_COMMITTED -> DTX_STATE_INSERTED_FORGET_COMMITTED doInsertForgetCommitted RecordDistributedForgetCommitted XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) doDispatchDtxProtocolCommand DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED RecordTransactionCommit CommitTransaction XLogFlush Dtx state : DTX_STATE_NOTIFYING_COMMIT_PREPARED-> DTX_STATE_RETRY_COMMIT_PREPARED
Master(GP_ROLE_DISPATCH) Seg-N Kill all after RMs write prepare log ---------------------------- ---------------------------- ------------------- Kill All TM/RMs and restart ------------------- ---------------------------- -- PostgresMain DTX_PROTOCOL_COMMAND_PREPARE doDispatchDtxProtocolCommand PostgresMain exec_mpp_dtx_protocol_command firstchar = ‘T' XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_PREPARE) Gp_role == GP_ROLE_DISPATCH initTM initTM_recover_as_needed recoverTM recoverInDoubtTransactions gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gid from pg_prepared_xacts) First char: 'M'; Return indoubt transactions abortRMInDoubtTransactions doAbortInDoubt doDispatchDtxProtocolCommand performDtxProtocolAbortPrepared StartTransactionCommand FinishPreparedTransaction RecordTransactionAbortPrepared XLogInsert (RM_XACT_ID, XLOG_XACT_ABORT_PREPARED | XLO DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED segmentMaxDistributedTransactionId gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gp_max_distributed_xid()) First char: 'M'; Return gp_max_distributed_xid First char: ‘T'; DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREPARED XLogFlush shmNumGxacts = 0 ReplayRedoFromUtilityMode redoDistributedCommitRecord
Master(GP_ROLE_DISPATCH) Seg-N Kill all after TMs write commit log ---------------------------- ---------------------------- ------------------- Kill All TM/RMs and restart ------------------- ---------------------------- -- PostgresMain DTX_PROTOCOL_COMMAND_COMMIT_PREPARED doDispatchDtxProtocolCommand PostgresMain exec_mpp_dtx_protocol_command firstchar = ‘T' XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_PREPARE) Gp_role == GP_ROLE_DISPATCH initTM initTM_recover_as_needed recoverTM recoverInDoubtTransactions shmNumGxacts = 1 gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gid from pg_prepared_xacts) segmentMaxDistributedTransactionId gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gp_max_distributed_xid()) First char: 'M'; Return gp_max_distributed_xid ReplayRedoFromUtilityMode redoDistributedCommitRecord dispatchDtxProtocolCommand doNotifyCommittedInDoubt performDtxProtocolCommitPrepared StartTransactionCommand FinishPreparedTransaction RecordTransactionAbortPrepared XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_ DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED First char: ‘T'; XLogFlush doInsertForgetCommitted RecordDistributedForgetCommitted XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) First char: 'M'; Return 0 indoubt transactions
Master(GP_ROLE_DISPATCH) Seg-N savepoint s1; PostgresMain firstchar = ‘M' exec_simple_query start_xact_command PortalRun PortalRunMulti CommitTransactionCommand finish_xact_command state: TRANS_DEFAULT-> TRANS_START- StartSubTransaction Block state: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS state: TRANS_START-> TRANS_INPROGRESS dispatchDtxCommand (SAVEPOINT s1) CommitTransactionCommand finish_xact_command state: TRANS_DEFAULT-> TRANS_START- StartSubTransaction Block state: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS state: TRANS_START-> TRANS_INPROGRESS ProcessUtility PortalRunUtility DefineDispatchSavepoint DefineSavepoint PushTransaction TRANS_STMT_SAVEPOINT TRANS_DEFAULT TBLOCK_SUBBEGIN ProcessUtility DefineSavepoint sendDtxExplicitBegin();
Master(GP_ROLE_DISPATCH) Seg-N rollback to savepoint s2 PostgresMain firstchar = ‘M' exec_simple_query start_xact_command PortalRun PortalRunMulti CommitTransactionCommand finish_xact_command state: TRANS_DEFAULT-> TRANS_START- StartSubTransaction state: TRANS_START-> TRANS_INPROGRESS dispatchDtxCommand (rollback to savepoint s2) CommitTransactionCommand finish_xact_command AbortSubTransaction state: TRANS_INPROGRESS -> TRANS_ABORT ProcessUtility PortalRunUtility RollbackToSavepoint TRANS_STMT_ROLLBACK_TO ProcessUtility RollbackToSavepoint TRANS_STMT_ROLLBACK_TO Sub state: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBABORT_PENDING State: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBRESTART CleanupSubTransaction state: TRANS_ABORT -> TRANS_DEFAULT DispatchRollbackToSavepoint Sub state: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBABORT_PENDING State: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBRESTART AbortSubTransaction state: TRANS_INPROGRESS -> TRANS_ABORT CleanupSubTransaction state: TRANS_ABORT -> TRANS_DEFAULT RecordSubTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) DefineSavepoint PushTransaction TRANS_DEFAULT TBLOCK_SUBBEGIN StartSubTransaction State: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS DefineSavepoint PushTransaction TRANS_DEFAULT TBLOCK_SUBBEGIN StartSubTransaction State: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS
Master(GP_ROLE_DISPATCH) Seg-N release savepoint s2 PostgresMain firstchar = ‘M' exec_simple_query CommitTransactionCommand finish_xact_command state: TRANS_INPROGRESS -> TRANS_COMMIT CommitSubTransaction CommitTransactionCommand finish_xact_command CommitSubTransaction state: TRANS_INPROGRESS -> TRANS_COMMIT ProcessUtility TRANS_STMT_RELEASE ProcessUtility ReleaseSavepoint TRANS_STMT_RELEASE dispatchDtxCommand While Sub blockState: TBLOCK_SUBEND If blockState: TBLOCK_END CommitTransaction State: TBLOCK_END -> TBLOCK_DEFAULT If blockState: TBLOCK_PREPARE PopTransaction Commit with sp State: TRANS_BEGIN -> TRANS_PREPARE PrepareTransaction XLogFlush EndPrepare XLogInsert (RM_XACT_ID, XLOG_XACT_PREP State: TRANS_PREPARE -> TRANS_DEFAULT prepareDtxTransaction ......
Debug tips • regres01 55749 55656 0 08:50 ? 00:00:00 postgres: port 5432, regres01 postgres [local] con6 [local] cmd5 idle in transaction • regres01 56475 55619 0 09:28 ? 00:00:00 postgres: port 40000, regres01 postgres localhost(6655) con6 seg0 idle in transaction • regres01 56478 55620 0 09:28 ? 00:00:00 postgres: port 40001, regres01 postgres localhost(14862) con6 seg1 idle in transaction • gp_vmem_idle_resource_timeout is for idle mem time in case the resource will be released • gpconfig -c gp_vmem_idle_resource_timeout –v 300
Debug tips • b StartTransactionCommand • b CommitTransactionCommand • b AbortCurrentTransaction • b BeginTransactionBlock • b EndTransactionBlock • b UserAbortTransactionBlock • b DefineSavepoint • b RollbackToSavepoint • b ReleaseSavepoint • b StartTransaction • b CommitTransaction • b AbortTransaction • b CleanupTransaction • b StartSubTransaction • b CommitSubTransaction • b AbortSubTransaction • b CleanupSubTransaction • b performDtxProtocolCommand • b exec_mpp_dtx_protocol_command • b recoverTM • b recoverInDoubtTransactions • b gatherRMInDoubtTransactions • b abortRMInDoubtTransactions • b cleanRMResultSets • b resolveInDoubtDtx • b doDispatchDtxProtocolCommand • b doPrepareTransaction • b doInsertForgetCommitted • b doNotifyingCommitPrepared • b doNotifyingAbort • b doNotifyCommittedInDoubt • b doAbortInDoubt • b doQEDistributedExplicitBegin • b UtilityModeSaveRedo • b ReplayRedoFromUtilityMode • b RemoveRedoUtilityModeFile • b performDtxProtocolCommitPrepared • b performDtxProtocolAbortPrepared • b cdbdisp_debugDispatchResult • b ProcessUtility • b prepareDtxTransaction • b PrepareTransaction • b XLogFlush • b ReadyForQuery_QEWriter • b RecordDistributedForgetCommitted • b RecordTransactionCommit • b RecordCrashTransactionAbortRecord • b XLogInsert • b XLogFlush • b forcedDistributedCommitted • b rollbackDtxTransaction • b dtmPreCommand • b PrepareTransaction • b disconnectAndDestroyAllGangs
61

More Related Content

PPT
High Availability And Oracle Data Guard 11g R2
byMario Redón Luz
 
PPTX
Hydra
byTensor
 
PDF
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021
byStreamNative
 
PPTX
Oracle ACFS High Availability NFS Services (HANFS)
byAnju Garg
 
PDF
"Taming the Dragon": Zenoh's Genesis
byZettaScaleTechnology
 
PDF
Advanced Streaming Analytics with Apache Flink and Apache Kafka, Stephan Ewen
byconfluent
 
PDF
Troubleshooting Tips and Tricks for Database 19c - EMEA Tour Oct 2019
bySandesh Rao
 
PDF
ClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
byAltinity Ltd
 
High Availability And Oracle Data Guard 11g R2
byMario Redón Luz
 
Hydra
byTensor
 
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021
byStreamNative
 
Oracle ACFS High Availability NFS Services (HANFS)
byAnju Garg
 
"Taming the Dragon": Zenoh's Genesis
byZettaScaleTechnology
 
Advanced Streaming Analytics with Apache Flink and Apache Kafka, Stephan Ewen
byconfluent
 
Troubleshooting Tips and Tricks for Database 19c - EMEA Tour Oct 2019
bySandesh Rao
 
ClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
byAltinity Ltd
 

What's hot

PDF
Unified Big Data Processing with Apache Spark (QCON 2014)
byDatabricks
 
PPTX
Elk
byCaleb Wang
 
PPTX
Flink Streaming
byGyula Fóra
 
PPTX
Hadoop Hive Tutorial | Hive Fundamentals | Hive Architecture
bySkillspeed
 
PPTX
Top 10 tips for Oracle performance (Updated April 2015)
byGuy Harrison
 
PDF
RocksDB Performance and Reliability Practices
byYoshinori Matsunobu
 
PDF
Database management system chapter15
byMd. Mahedi Mahfuj
 
PDF
The Parquet Format and Performance Optimization Opportunities
byDatabricks
 
PDF
Oracle 索引介紹
byChien Chung Shen
 
PPTX
Data Guard Architecture & Setup
bySatishbabu Gunukula
 
PDF
My First 100 days with an Exadata (PPT)
byGustavo Rene Antunez
 
PPTX
ELK Elasticsearch Logstash and Kibana Stack for Log Management
byEl Mahdi Benzekri
 
PDF
Cluster con postgresql
byesmeraldaq2011
 
PPTX
Oracle Database: Checklist Connection Issues
byMarkus Flechtner
 
PDF
Build Agentic AI Applications with Oracle AI Database Private Agent Factory -...
bySandesh Rao
 
ODP
Low level java programming
byPeter Lawrey
 
PPTX
Elastic - ELK, Logstash & Kibana
bySpringPeople
 
PDF
MySQL InnoDB Cluster 소개
byrockplace
 
PDF
Efficient Data Storage for Analytics with Parquet 2.0 - Hadoop Summit 2014
byJulien Le Dem
 
PPTX
ELK Stack
byPhuc Nguyen
 
Unified Big Data Processing with Apache Spark (QCON 2014)
byDatabricks
 
Elk
byCaleb Wang
 
Flink Streaming
byGyula Fóra
 
Hadoop Hive Tutorial | Hive Fundamentals | Hive Architecture
bySkillspeed
 
Top 10 tips for Oracle performance (Updated April 2015)
byGuy Harrison
 
RocksDB Performance and Reliability Practices
byYoshinori Matsunobu
 
Database management system chapter15
byMd. Mahedi Mahfuj
 
The Parquet Format and Performance Optimization Opportunities
byDatabricks
 
Oracle 索引介紹
byChien Chung Shen
 
Data Guard Architecture & Setup
bySatishbabu Gunukula
 
My First 100 days with an Exadata (PPT)
byGustavo Rene Antunez
 
ELK Elasticsearch Logstash and Kibana Stack for Log Management
byEl Mahdi Benzekri
 
Cluster con postgresql
byesmeraldaq2011
 
Oracle Database: Checklist Connection Issues
byMarkus Flechtner
 
Build Agentic AI Applications with Oracle AI Database Private Agent Factory -...
bySandesh Rao
 
Low level java programming
byPeter Lawrey
 
Elastic - ELK, Logstash & Kibana
bySpringPeople
 
MySQL InnoDB Cluster 소개
byrockplace
 
Efficient Data Storage for Analytics with Parquet 2.0 - Hadoop Summit 2014
byJulien Le Dem
 
ELK Stack
byPhuc Nguyen
 

Recently uploaded

PDF
Incident Management Roles & Responsibilities.pdf
byTaskCall
 
PDF
DSD-INT 2025 Large scale unsaturated zone modelling - Sequential Steady State...
byDeltares
 
PDF
DSD-INT 2025 iMOD Coupler - coupling Ribasim – MetaSWAP – MODFLOW 6 - Leander
byDeltares
 
PPTX
Instrumentation.Instrumentation.Instrumentation.pptx
byDavid Kurtz
 
PDF
Cloud Automotive Software: Cost, Speed & Scalability
byShiv Technolabs Pvt. Ltd.
 
PPTX
Feasibility Analysis in System Development Using Data Flow Diagrams
byM Munim
 
PDF
828275126-INTUNE-Instructor-Handout-Part2.pdf
byamdusias67
 
PDF
C_ABAPD_2507 (83%).pdf antwoten certificate
byImaneKrioutate
 
PPTX
Free AI Paraphrasing Tool to Rewrite Sentences – SENTENCIFY
bySENTENCIFY
 
PPTX
1-introduction-to-databases notes start .pptx
byBeatrice1625
 
PDF
Healthcare Mobile App Development: Features & Benefits
byjoealwyn38
 
PDF
DSD-INT 2025 Groundwater table lowering due to surface water lowering - Reusen
byDeltares
 
PDF
Problem Solving_20241203_220045_0000.pdf
bythakurayush00991
 
PPTX
Performance Engineering: New and Conflicting Trends
byAlexander Podelko
 
PDF
Cybernetic Global Intelligence Securing Businesses in a Digital-First World.pdf
byCybernetic Global Intelligence
 
PDF
How Fireworks AI Achieves 1TB_s+ Throughput for Model Deployment Across Multi...
byAlluxio, Inc.
 
PPTX
ONLINE TRAVEL BUSINESS SOFTWARE | BUSINESS TRAVEL SOFTWARE
byphilipnathen82
 
PDF
DSD-INT 2025 iMOD Parallel coupler development plan - Verkaik
byDeltares
 
PDF
our environment ppt made by many people name rarang yadav
byrarang180
 
PPTX
Introduction to Software Development and Modern Practices
byM Munim
 
Incident Management Roles & Responsibilities.pdf
byTaskCall
 
DSD-INT 2025 Large scale unsaturated zone modelling - Sequential Steady State...
byDeltares
 
DSD-INT 2025 iMOD Coupler - coupling Ribasim – MetaSWAP – MODFLOW 6 - Leander
byDeltares
 
Instrumentation.Instrumentation.Instrumentation.pptx
byDavid Kurtz
 
Cloud Automotive Software: Cost, Speed & Scalability
byShiv Technolabs Pvt. Ltd.
 
Feasibility Analysis in System Development Using Data Flow Diagrams
byM Munim
 
828275126-INTUNE-Instructor-Handout-Part2.pdf
byamdusias67
 
C_ABAPD_2507 (83%).pdf antwoten certificate
byImaneKrioutate
 
Free AI Paraphrasing Tool to Rewrite Sentences – SENTENCIFY
bySENTENCIFY
 
1-introduction-to-databases notes start .pptx
byBeatrice1625
 
Healthcare Mobile App Development: Features & Benefits
byjoealwyn38
 
DSD-INT 2025 Groundwater table lowering due to surface water lowering - Reusen
byDeltares
 
Problem Solving_20241203_220045_0000.pdf
bythakurayush00991
 
Performance Engineering: New and Conflicting Trends
byAlexander Podelko
 
Cybernetic Global Intelligence Securing Businesses in a Digital-First World.pdf
byCybernetic Global Intelligence
 
How Fireworks AI Achieves 1TB_s+ Throughput for Model Deployment Across Multi...
byAlluxio, Inc.
 
ONLINE TRAVEL BUSINESS SOFTWARE | BUSINESS TRAVEL SOFTWARE
byphilipnathen82
 
DSD-INT 2025 iMOD Parallel coupler development plan - Verkaik
byDeltares
 
our environment ppt made by many people name rarang yadav
byrarang180
 
Introduction to Software Development and Modern Practices
byM Munim
 

Learning Greenplum Distributed Transaction

  • 1.
  • 2.
    Transaction Framework Begin Transaction Client Server End Transaction UPDATE TransactionBlock Begin Transaction Block End Transaction Block Low level Transaction Layer Execute SQL PostgreSQL TransactionStateData -TBlockState ： high-level stat -TransState ： low-level state
  • 3.
    QE Distributed Transaction TransactionFramework Begin Transaction 客户端 Master End Transaction UPDATE Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Greenplum Segment 0 Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Execute SQL QD DTM Execute SQL QE Distributed Transaction Segment 0 Transaction Block Begin Transaction Block End Transaction Block Low level Transaction Layer Execute SQL Dtm Command Dtm Command TMGXACT -DtxState TransactionStateData -TBlockState ： high-level stat -TransState ： low-level state DtxContext TransactionStateData -TBlockState ： high-level stat -TransState ： low-level state DtxContext
  • 4.
    Main Call Flow(Normal) Master(GP_ROLE_DISPATCH) Seg-N exec_simple_query start_xact_command ProcessUtility finish_xact_command doQEDistributedExplicitBegin setupQEDtxContext performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘M’ only for “BEGIN” firstchar = ‘T' doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND Middle Layer function low level function Middle Layer function low level function
  • 5.
    Main Call Flow(Retry) Master(GP_ROLE_DISPATCH) Seg-N doDtxPhase2Retry StartTransactionCommand doDispatchDtxProtocolCommand CommitTransactionCommand DTX_CONTEXT_QD_RETRY_PHASE_2 When DTX_STATE_RETRY_COMMIT_PREPARED DTX_STATE_RETRY_COMMIT_PREPARED DTX_STATE_RETRY_ABORT_PREPARED doInsertForgetCommitted performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND
  • 6.
    Middle Layer function •Normal system's calls • StartTransactionCommand • CommitTransactionCommand • AbortCurrentTransaction • User change calls by issue “BEGIN, COMMIT, ROLLBACK, SAVEPOINT, ROLLBACK TO or RELEASE” • BeginTransactionBlock • EndTransactionBlock • UserAbortTransactionBlock • PrepareTransactionBlock • DefineSavepoint • RollbackToSavepoint • ReleaseSavepoint
  • 7.
    Low level calls •StartTransaction • CommitTransaction • AbortTransaction • CleanupTransaction • PrepareTransaction • StartSubTransaction • CommitSubTransaction • AbortSubTransaction • CleanupSubTransaction
  • 8.
    Postgres Normal middlelevel call flow / StartTransactionCommand; / StartTransaction; 1) < ProcessUtility; << BEGIN BeginTransactionBlock; CommitTransactionCommand; / StartTransactionCommand; 2) / ProcessQuery; << SELECT ... CommitTransactionCommand; CommandCounterIncrement; / StartTransactionCommand; 3) / ProcessQuery; << INSERT ... CommitTransactionCommand; CommandCounterIncrement; / StartTransactionCommand; / ProcessUtility; << COMMIT 4) < EndTransactionBlock; CommitTransactionCommand; CommitTransaction; 1) BEGIN 2) SELECT * FROM foo 3) INSERT INTO foo VALUES (...) 4) COMMIT
  • 9.
    High Level Statetransfer TBLOCK_DEFAULT TBLOCK_START TBLOCK_ABORT_PENDING TBLOCK_BEGIN TBLOCK_INPROGRESS TBLOCK_ABORT TBLOCK_END TBLOCK_ABORT_END BeginTransactionBlock StartTransactionCommand AbortCurrentTransaction UserAbortTransactionBlock EndTransactionBlock CommitTransactionCommand AbortCurrentTransaction CommitTransactionCommand AbortCurrentTransaction CommitTransactionCommand AbortCurrentTransaction EndTransactionBlock UserAbortTransactionBlock AbortCurrentTransaction AbortCurrentTransactionCommitTransactionCommand PostgreSQL TBLOCK_PREPARE PrepareTransactionBlock CommitTransactionCommand failed xact failed xact AbortOutOfAnyTransaction AbortOutOfAnyTransaction CommitTransactionCommand
  • 10.
    High Level Statetransfer TBLOCK_SUBEND TBLOCK_SUBBEGIN TBLOCK_SUBABORT_PENDING TBLOCK_SUBINPROGRESS TBLOCK_SUBABORT TBLOCK_SUBABORT_RESTART TBLOCK_SUBABORT_END TBLOCK_SUBRESTART PostgreSQL CommitTransactionCommand PushTransaction ReleaseSavepoint EndTransactionBlock AbortCurrentTransaction UserAbortTransactionBlock EndTransactionBlock EndTransactionBlock UserAbortTransactionBlock RollbackToSavepoint RollbackToSavepoint RollbackToSavepoint RollbackToSavepoint failed xact failed xact failed xact
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
    Detailed transfer TBLOCK_DEFAULT TBLOCK_INPROGRESS TBLOCK_ABORT TBLOCK_ABORT_END CommitTransactionCommand AbortCurrentTransaction EndTransactionBlock UserAbortTransactionBlock failed xact failedxact AbortOutOfAnyTransaction Rollback TBLOCK_ABORT_PENDING CommitTransactionCommand AbortCurrentTransaction UserAbortTransactionBlock
  • 16.
    Low Level Statetransfer TRANS_DEFAULT TRANS_START TRANS_COMMITTRANS_INPROGRESS TRANS_ABORT TRANS_PREPARE StartTransactionStartTransaction CommitTransaction PrepareTransaction PrepareTransaction CommitTransaction AbortTransaction CleanupTransaction
  • 17.
    TM Control Block- TMGXACT char gid[TMGIDSIZE]; DistributedTransactionId gxid; DtxState state; int sessionId; TransactionId localXid; LocalDistribXactRef localDistribXactRef; bool explicitBeginRemembered; DistributedTransactionId xminDistributedSnapshot; bool bumpedPhase1Count; bool badPrepareGangs; bool retryPhase2RecursionStop; int debugIndex; bool directTransaction; uint16 directTransactionContentId;
  • 18.
    TM Actions toRM • Normal • doPrepareTransaction • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_PREPARE) • doInsertForgetCommitted • Insert FORGET COMMITTED into the xlog. • doNotifyingCommitPrepared • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_COMMIT_PREPARED) • doNotifyingAbort • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_ABORT_PREPARED) • Crash Recovery • doNotifyCommittedInDoubt • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_RECOVERY_COMMIT_PR EPARED) • doAbortInDoubt • doDispatchDtxProtocolCommand(DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREP ARED)
  • 19.
    TM Actions toRM • Others • sendDtxExplicitBegin • dispatchDtxCommand(“BEGIN ISOLATION LEVEL %s, READ %s”) • DefineDispatchSavepoint • dispatchDtxCommand(“savepoint”) • ReleaseSavepoint • dispatchDtxCommand(“release savepoint”) • DispatchRollbackToSavepoint • dispatchDtxCommand(“rollback to savepoint”)
  • 20.
    TM dispatch commandto RM • doDispatchDtxProtocolCommand • cdbdisp_dispatchDtxProtocolCommand • cdbdisp_dispatchToGang(GP_DISPATCH_COMMAND_TYPE_DTX_PROTOC OL) • CdbCheckDispatchResult(&ds, DISPATCH_WAIT_NONE); • ----------------wait for QE------------------ • cdbdisp_returnResults • cdbdisp_destroyDispatchThreads
  • 21.
    DTM to RMprotocol commands • DTX_PROTOCOL_COMMAND_NONE • DTX_PROTOCOL_COMMAND_STAY_AT_OR_BECOME_IMPLIED_WRITER = 1, • DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED, • DTX_PROTOCOL_COMMAND_PREPARE, • DTX_PROTOCOL_COMMAND_ABORT_SOME_PREPARED, • DTX_PROTOCOL_COMMAND_COMMIT_PREPARED, • DTX_PROTOCOL_COMMAND_ABORT_PREPARED, • DTX_PROTOCOL_COMMAND_RETRY_COMMIT_PREPARED, • DTX_PROTOCOL_COMMAND_RETRY_ABORT_PREPARED, • DTX_PROTOCOL_COMMAND_RECOVERY_COMMIT_PREPARED, • DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREPARED, • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_BEGIN_INTERNAL, • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_ROLLBACK_INTERNAL, • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_RELEASE_INTERNAL,
  • 22.
    TM State transfer DTX_STATE_NONE DTX_STATE_ACTIVE_DISTRIBUTED DTX_STATE_PREPARING DTX_STATE_INSERTING_COMMITTED DTX_STATE_INSERTED_COMMITTED DTX_STATE_FORCED_COMMITTEDDTX_STATE_INSERTING_FORGET_COMMITTED DTX_STATE_INSERTED_FORGET_COMMITTED DTX_STATE_NOTIFYING_COMMIT_PREPARED doPrepareTransaction DTX_STATE_PREPARED doPrepareTransaction DTX_STATE_NOTIFYING_ABORT_SOME_PREPARED DTX_STATE_NOTIFYING_ABORT_NO_PREPARED DTX_STATE_NOTIFYING_ABORT_PREPARED RecordTransactionCommit/ forcedDistributedCommitted doNotifyingCommitPrepared doInsertForgetCommitted RecordTransactionCommit/insertingDistributedCommitted rollbackDtxTransaction DTX_STATE_RETRY_ABORT_PREPARED rollbackDtxTransaction(badPrepareGangs) rollbackDtxTransaction(badPrepare) rollbackDtxTransaction(xa rollback??) DTX_STATE_RETRY_COMMIT_PREPARED doNotifyingCommitPrepared(commit failed) doDtxPhase2Retry/doInsertForgetCommitted ateDtx/initGxact startTransaction/createDtx DTX_STATE_ACTIVE_NOT_DISTRIBUTED dtmPreCmd
  • 23.
    DTM State • Uninitializedtransaction • DTX_STATE_NONE = 0 • The distributed transaction is active • DTX_STATE_ACTIVE_NOT_DISTRIBUTED //auto-commit on the QEs • DTX_STATE_ACTIVE_DISTRIBUTED //it is explicit or an implicit writer transaction
  • 24.
    DTM State • Fortwo-phase commit, the first phase is about to run • DTX_STATE_PREPARING • For two-phase commit, the first phase has completed • DTX_STATE_PREPARED • DTX_STATE_INSERTING_COMMITTED • DTX_STATE_INSERTED_COMMITTED • DTX_STATE_FORCED_COMMITTED • DTX_STATE_NOTIFYING_COMMIT_PREPARED • DTX_STATE_INSERTING_FORGET_COMMITTED • DTX_STATE_INSERTED_FORGET_COMMITTED
  • 25.
    DTM State • Transactionrollback has been requested and QD is notifying all QD processes • no QEs have started work on the first phase of two-phase commit • DTX_STATE_NOTIFYING_ABORT_NO_PREPARED • at least one QE has done the first phase of two-phase commit • DTX_STATE_NOTIFYING_ABORT_SOME_PREPARED • the QE processes have done the first phase of two-phase commit • DTX_STATE_NOTIFYING_ABORT_PREPARED, • DTX_STATE_RETRY_COMMIT_PREPARED, • DTX_STATE_RETRY_ABORT_PREPARED, • DTX_STATE_CRASH_COMMITTED
  • 26.
    DTX Context • DTX_CONTEXT_LOCAL_ONLY •There is no distributed transaction. This is the initial state and, for utility mode connections or master-only queries, the only state. It is also the state to which the QD and QE return to between transactions. • DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE, • On QD: the process is currently part of a distributed transaction. Whether or not the transaction has been started on a QE is not a part of the QD state -- that is tracked by assigning one of the DTX_CONTEXT_QE* values on the QE process, and by updating the state field of the currentGxact. • DTX_CONTEXT_QD_RETRY_PHASE_2, • On QD: indicates that the second-phase of the two-phase commit is being retried.
  • 27.
    DTX Context • DTX_CONTEXT_QE_ENTRY_DB_SINGLETON, •TODO: how is something an Entry db? fix this documentation • DTX_CONTEXT_QE_AUTO_COMMIT_IMPLICIT, • On a QE that is the root of a query, this context means that the distributed transaction was opened implicitly by a non-writing query in an auto-commit transaction. • This is essentially the same as _TWO_PHASE_IMPLICIT_WRITER except we know the query won't dirty anything and so we don't need a two-phase commit. • DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER, • On a QE, this context means that the distributed transaction was opened explicitly by a BEGIN query statement Note that this state can happen even if there are no writer queries (because this state is entered when the BEGIN statement is processed). • DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER, • On a QE that is the root of a query, this context means that the distributed transaction was opened implicitly by a writing query in an auto-commit transaction, and this is the writing QE process Same as _AUTO_COMMIT_IMPLICIT except that we need a two-phase of the dirty data. • DTX_CONTEXT_QE_READER, • On a QE that is not the root of a query, this context is used. This can happen regardless of whether the overall transaction for the query is implicit or explicit, and auto-commit vs two-phase • DTX_CONTEXT_QE_PREPARED, • On the QE, the two-phase commit has been prepared (first phase complete). • DTX_CONTEXT_QE_FINISH_PREPARED • On the QE, the two-phase commit has been committed or aborted (second phase complete).
  • 28.
    TM: ProcessUtility • casenodeTag(parsetree) = T_TransactionStmt • Case Stmt->kind: • -----normal----- • TRANS_STMT_BEGIN or TRANS_STMT_START • BeginTransactionBlock • sendDtxExplicitBegin • TRANS_STMT_COMMIT • EndTransactionBlock • TRANS_STMT_ROLLBACK • UserAbortTransactionBlock
  • 29.
    TM/RM : ProcessUtility •case nodeTag(parsetree) = T_TransactionStmt • Case Stmt->kind: • -----savepoint----- • TRANS_STMT_SAVEPOINT • RequireTransactionChain • sendDtxExplicitBegin • DefineDispatchSavepoint • TRANS_STMT_RELEASE • RequireTransactionChain • ReleaseSavepoint • TRANS_STMT_ROLLBACK_TO • RequireTransactionChain • RollbackToSavepoint
  • 30.
    RM: ProcessUtility ???XA??? • case nodeTag(parsetree) = T_TransactionStmt • Case Stmt->kind: • -----I think it’s XA feature, not know when called in RM---- • TRANS_STMT_PREPARE • PrepareTransactionBlock • TRANS_STMT_COMMIT_PREPARED • PreventTransactionChain • FinishPreparedTransaction • TRANS_STMT_ROLLBACK_PREPARED • PreventTransactionChain • FinishPreparedTransaction
  • 31.
    RM: doQEDistributedExplicitBegin • HandleExplicitBegin • StartTransactionCommand(); • BeginTransactionBlock(); • CommitTransactionCommand();
  • 32.
    RM: performDtxProtocolCommand • CasedtxProtocolCommand • DTX_PROTOCOL_COMMAND_STAY_AT_OR_BECOME_IMPLIED_WRITER • Case DistributedTransactionContext: • DTX_CONTEXT_LOCAL_ONLY • setupQEDtxContext • StartTransactionCommand • DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER • DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED • AbortOutOfAnyTransaction • DTX_PROTOCOL_COMMAND_PREPARE • Case DistributedTransactionContext: • DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER or DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER • performDtxProtocolPrepare • DTX_PROTOCOL_COMMAND_ABORT_SOME_PREPARED • Case DistributedTransactionContext: • DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER or DTX_CONTEXT_QE_TWO_PHASE_IMPLICIT_WRITER • AbortOutOfAnyTransaction • DTX_CONTEXT_QE_PREPARED • performDtxProtocolAbortPrepared • DTX_PROTOCOL_COMMAND_COMMIT_PREPARED • performDtxProtocolCommitPrepared • DTX_PROTOCOL_COMMAND_ABORT_PREPARED • performDtxProtocolAbortPrepared
  • 33.
    RM: performDtxProtocolCommand • CasedtxProtocolCommand • DTX_PROTOCOL_COMMAND_RETRY_COMMIT_PREPARED • performDtxProtocolCommitPrepared • DTX_PROTOCOL_COMMAND_RETRY_ABORT_PREPARED • performDtxProtocolAbortPrepared • DTX_PROTOCOL_COMMAND_RECOVERY_COMMIT_PREPARED • performDtxProtocolCommitPrepared • DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREPARED • performDtxProtocolAbortPrepared • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_BEGIN_INTERNAL • Case DistributedTransactionContext: • DTX_CONTEXT_LOCAL_ONLY • setupQEDtxContext • StartTransactionCommand • BeginInternalSubTransaction • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_ROLLBACK_INTERNAL • RollbackAndReleaseCurrentSubTransaction • DTX_PROTOCOL_COMMAND_SUBTRANSACTION_RELEASE_INTERNAL • ReleaseCurrentSubTransaction
  • 34.
    Log Type • Nomallog: • #define XLOG_XACT_COMMIT 0x00 • #define XLOG_XACT_ABORT 0x20 • 2PC Log • RM Log • #define XLOG_XACT_PREPARE 0x10 • #define XLOG_XACT_COMMIT_PREPARED 0x30 • #define XLOG_XACT_ABORT_PREPARED 0x40 • TM log • #define XLOG_XACT_DISTRIBUTED_COMMIT 0x50 • #define XLOG_XACT_DISTRIBUTED_FORGET 0x60
  • 35.
    TM Log write •RecordTransactionCommit if (isDtxPrepared) { XLogInsert(RM_XACT_ID, XLOG_XACT_DISTRIBUTED_COMMIT) } else { XLogInsert(RM_XACT_ID, XLOG_XACT_COMMIT); } XLogFlush() • RecordDistributedForgetCommitted XLogInsert(RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN)
  • 36.
    TM Log write •XLOG_XACT_DISTRIBUTED_COMMIT • XLogRecData rdata[4]; 0 1 2 3 XLogRecData Data: xl_xact_commit Len: MinSizeOfXactCommit Data: persistentCommitBuffer Len: persistentCommitSerializeLen Data: children Len: nchildren * sizeof(TransactionId) Data: TMGXACT_LOG Len: sizeof(TMGXACT_LOG) persistentCommitObjectCount > 0 nchildren > 0 isDtxPrepared
  • 37.
    TM Log write •XLOG_XACT_DISTRIBUTED_FORGET • XLogRecData rdata[4]; 0 XLogRecData Data: xl_xact_distributed_forget Len: sizeof(xl_xact_distributed_forget) TMGXACT_LOG
  • 38.
    RM Log write •Prepare LOG • Log state format 1. TwoPhaseFileHeader 2. TransactionId[] (subtransactions) 3. RelFileNode[] (files to be deleted at commit) 4. RelFileNode[] (files to be deleted at abort) 5. TwoPhaseRecordOnDisk 6. ... 7. TwoPhaseRecordOnDisk (end sentinel, rmid == TWOPHASE_RM_END_ID) 8. CRC32 • XLog
  • 39.
    RM Log write •Log state format
  • 40.
    RM Log write •XLOG_XACT_PREPARE • XLogRecData rdata[n]; 0 1 2 XLogRecData Data: TwoPhaseFileHeader Len: TwoPhaseFileHeader or 512 Data: children Len: nchildren * sizeof(TransactionId) Data: persistentCommitBuffer Len: persistentCommitSerializeLen nchildren > 0 persistentCommitObjectCount > 0 3 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_NOTIFY_ID） Len: sizeof(TwoPhaseRecordOnDisk) 3 Data: relname Len: sizeof(relname) 4 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_FLATFILES_ID） Len: sizeof(TwoPhaseRecordOnDisk) info != 0 5 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_INVAL_ID） Len: sizeof(TwoPhaseRecordOnDisk) RelcacheInitFileInval != NULL 6 x n Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_LOCK_ID） Len: sizeof(TwoPhaseRecordOnDisk) Has locks 6 x n Data: TwoPhaseLockRecord Len: sizeof(TwoPhaseLockRecord) Has locks 7 x n Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_PGSTAT_ID） Len: sizeof(TwoPhaseRecordOnDisk) 7 x n Data: TwoPhasePgStatRecord Len: sizeof(TwoPhasePgStatRecord) xact_state != NULL 8 Data: TwoPhaseRecordOnDisk （TWOPHASE_RM_END_ID） Len: sizeof(TwoPhaseRecordOnDisk)
  • 41.
    RM Log write •XLOG_XACT_DISTRIBUTED_COMMIT • XLogRecData rdata[4] 0 1 2 XLogRecData Data: xl_xact_commit_prepared Len: MinSizeOfXactCommitPrepared Data: persistentCommitBuffer Len: persistentCommitSerializeLen Data: children Len: nchildren * sizeof(TransactionId) persistentPrepareObjectCount > 0 nsubxacts > 0
  • 42.
    RM Log write •XLOG_XACT_DISTRIBUTED_COMMIT • XLogRecData rdata[3]; 0 1 2 XLogRecData Data: xl_xact_abort_prepared Len: MinSizeOfXactAbortPrepared Data: persistentCommitBuffer Len: persistentCommitSerializeLen Data: children Len: nchildren * sizeof(TransactionId) persistentCommitObjectCount > 0 nchildren > 0
  • 43.
    RM Log write •EndPrepare XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE) XLogFlush() • RecordTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) XLogFlush() only when we are deleting files • RecordTransactionCommitPrepared XLogInsert(RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_NO_TRAN) XLogFlush() • RecordTransactionAbortPrepared XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT_PREPARED | XLOG_NO_TRAN) XLogFlush()
  • 44.
    Log Write Seqwhen Commit Master(GP_ROLE_DISPATCH) Seg-N XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_PREPARE) XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_COMMIT) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_NO_ XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_ABORT_PREPARED | XLOG_NO_T XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_NO_ Commit Rollback Retry Commit Retry Rollback
  • 45.
    Test Details • BEGIN •INSERT • END • MANUNALLY ROLLBACK • INSERT FAILED WITH AUTO ROLLBACK • Kill RM-0 before master write log, RM-1 Prepared • Kill RM-0 when master send commit to RMs, RM all Prepared • Error happened when master send commit to RMs, no Gangs errors • Crash recovery
  • 46.
    PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar ='Q' exec_simple_query start_xact_command StartTransactionCommand StartTransaction Block state: TBLOCK_DEFAULT -> TBLOCK_STARTED State: TRANS_DEFAULT -> TRANS_START PortalRun ProcessUtility TRANS_STMT_START BeginTransactionBlock Block state: TBLOCK_STARTED -> TBLOCK_BEGIN sendDtxExplicitBegin DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE createDtx Gxact->state : DTX_STATE_NONE -> DTX_STATE_ACTIVE_NOT_DISTRIBUTED dispatchDtxCommand cdbdisp_dispatchRMCommand cdbdisp_dispatchCommand cdbdisp_dispatchToGang CdbCheckDispatchResult cdbdisp_returnResults cdbdisp_destroyDispatchThreads CommitTransactionCommand finish_xact_command Block state: TBLOCK_BEGIN -> TBLOCK_INPROGRESS doQEDistributedExplicitBegin StartTransactionCommand BeginTransactionBlock CommitTransactionCommand setupQEDtxContext needTwoPhase = false; explicitBegin = true Block state: TBLOCK_DEFAULT -> TBLOCK_STARTED State: TRANS_DEFAULT -> TRANS_START StartTransaction firstchar = ‘M' Block state: TBLOCK_STARTED -> TBLOCK_BEGIN Block state: TBLOCK_BEGIN -> TBLOCK_INPROGRESS State: TRANS_START -> TRANS_INPROGRESS State: TRANS_START -> TRANS_INPROGRESS pq_flush ReadyForQuery_QEWriter DTX_CONTEXT_LOCAL_ONLY DistributedTransactionContext: DTX_CONTEXT_LOCAL_ONLY -> DTX_CONTEX BEGIN; --------------------wait---------------------- dtmPreCommand Gxact->state: DTX_STATE_ACTIVE_NOT_DISTRIBUTED -> DTX_STATE_ACTIVE_DISTRIBUTED
  • 47.
    PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar ='Q' exec_simple_query start_xact_command StartTransactionCommand TBLOCK_INPROGRESS PortalRun PortalRunMulti ProcessQuery ExecutorStart start_xact_command exec_mpp_query firstchar = ‘M' dtmPreCommand StartTransactionCommand INSERT ; CommitTransactionCommand finish_xact_command --------------------wait---------------------- PortalRun PortalRunMulti ProcessQuery ExecutorRun ExecutePlan ExecInsert heap_insert heap_log_tuple_insert heap_log_tuple_insert XLogInsert (RM_HEAP_ID) CommitTransactionCommand finish_xact_command ExecutorEnd mppExecutorFinishup CdbCheckDispatchResult CommandId++
  • 48.
    PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar ='Q' exec_simple_query start_xact_command StartTransactionCommand TBLOCK_INPROGRESS PortalRun ProcessUtility TRANS_STMT_COMMIT EndTransactionBlock Block state: TBLOCK_INPROGRESS -> TBLOCK_END CommitTransactionCommand finish_xact_command BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command State: TRANS_BEGIN -> TRANS_PREPARE firstchar = ‘T' CommitTransaction prepareDtxTransaction dtxProtocolCommand = DTX_PROTOCOL_COMMAND_PREPARE DistributedTransactionContext == DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_ performDtxProtocolPrepare StartTransactionCommand PrepareTransactionBlock CommitTransactionCommand DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_WRITER -> DTX_CONTEXT_Q EndTransactionBlock Block state: TBLOCK_INPROGRESS -> TBLOCK_END Block state: TBLOCK_END -> TBLOCK_PREPARE PrepareTransaction XLogFlush Block state: TBLOCK_PREPARE -> TBLOCK_DEFAULT pq_flush ReadyForQuery_QEWriter END/COMMIT 1; doPrepareTransaction Gxactstate: DTX_STATE_ACTIVE_DISTRIBUTED ->DTX_STATE_PREPARING doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_PREPARE cdbdisp_dispatchDtxProtocolCommand Gxactstate: DTX_STATE_PREPARING -> DTX_STATE_PREPARED cdbdisp_returnResults EndPrepare --------------------wait---------------------- XLogInsert (RM_XACT_ID, XLOG_XACT_PR State: TRANS_INPROGRESS -> TRANS_COMMIT State: TRANS_PREPARE -> TRANS_DEFAULT StartPrepare gxact = MarkAsPreparing()
  • 49.
    Master(GP_ROLE_DISPATCH) Seg-N PostgresMain BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar =‘T' dtxProtocolCommand = DTX_PROTOCOL_COMMAND_COMMIT_PREPARED performDtxProtocolCommitPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_LOCAL_ONLY FinishPreparedTransaction Block state: TBLOCK_DEFAULT -> TBLOCK_STARTED Block state: TBLOCK_STARTED -> TBLOCK_DEFAULT State: TRANS_DEFAULT -> TRANS_START StartTransaction DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED State: TRANS_START -> TRANS_INPROGRESS State:TRANS_INPROGRESS ->TRANS_COMMIT CommitTransaction RecordTransactionCommit XLogFlush END/COMMIT 2; doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_COMMIT_PREPARED cdbdisp_dispatchDtxProtocolCommand cdbdisp_returnResults doNotifyingCommitPrepared notifyCommittedDtxTransaction Dtx state : DTX_STATE_FORCED_COMMITTED -> DTX_STATE_NOTIFYING_COMMIT_PREPARED doInsertForgetCommitted Dtx state : DTX_STATE_NOTIFYING_COMMIT_PREPARED -> DTX_STATE_INSERTING_FORGET_COMMITTED Dtx state : DTX_STATE_NOTIFYING_COMMIT_PREPARED -> DTX_STATE_INSERTED_FORGET_COMMITTED doInsertForgetCommitted RecordDistributedForgetCommitted RecordTransactionCommit forcedDistributedCommitted Dtx state : DTX_STATE_INSERTED_COMMITTED-> DTX_STATE_FORCED_COMMITTED --------------------wait---------------------- insertingDistributedCommitted Dtx state : DTX_STATE_PREPARED -> DTX_STATE_INSERTING_COMMITTED XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_COMMIT) insertedDistributedCommitted Dtx state : DTX_STATE_INSERTING_COMMITTED -> DTX_STATE_INSERTED_COMMITTED RecordTransactionCommitPrepared XLogFlush CommitTransactionCommand finish_xact_command CommitTransaction …… …… XLogFlush …… XLogInsert (RM_XACT_ID, XLOG_XACT_CO XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) Block state: TBLOCK_END -> TBLOCK_DEFAULT State:TRANS_COMMIT ->TRANS_DEFAULT State:TRANS_COMMIT ->TRANS_DEFAULT
  • 50.
    PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar ='Q' exec_simple_query start_xact_command StartTransactionCommand TBLOCK_INPROGRESS PortalRun ProcessUtility TRANS_STMT_ROLLBACK UserAbortTransactionBlock Block state: TBLOCK_INPROGRESS -> TBLOCK_ABORT_PENDING CommitTransactionCommand finish_xact_command BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' AbortTransaction rollbackDtxTransaction dtxProtocolCommand = DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED DistributedTransactionContext == DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_ AbortOutOfAnyTransaction pq_flush ReadyForQuery_QEWriter ROLLBACK; Gxactstate: DTX_STATE_ACTIVE_DISTRIBUTED ->DTX_STATE_NOTIFYING_ABORT_NO_PREPARED doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED cdbdisp_dispatchDtxProtocolCommand cdbdisp_returnResults --------------------wait---------------------- CleanupTransaction Block state: TBLOCK_ABORT_PENDING -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT doNotifyingAbort AbortTransaction CleanupTransaction Block state: TBLOCK_ABORT_PENDING -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT State: TRANS_ABORT -> TRANS_DEFAULT State: TRANS_ABORT -> TRANS_DEFAULT RecordTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT)
  • 51.
    PostgresMain Master(GP_ROLE_DISPATCH) Seg-N PostgresMain firstchar ='Q' AbortCurrentTransaction AbortTransaction BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' rollbackDtxTransaction dtxProtocolCommand = DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED DistributedTransactionContext == DTX_CONTEXT_QE_TWO_PHASE_EXPLICIT_ AbortOutOfAnyTransaction pq_flush ReadyForQuery_QEWriter INSERT FAILED; Gxactstate: DTX_STATE_ACTIVE_DISTRIBUTED ->DTX_STATE_NOTIFYING_ABORT_NO_PREPARED doDispatchDtxProtocolCommand DTX_PROTOCOL_COMMAND_ABORT_NO_PREPARED cdbdisp_dispatchDtxProtocolCommand cdbdisp_returnResults --------------------wait---------------------- CleanupTransaction Block state: TBLOCK_ABORT_END -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT doNotifyingAbort AbortTransaction CleanupTransaction Block state: TBLOCK_ABORT_PENDING -> TBLOCK_DEFAULT State: TRANS_INPROGRESS -> TRANS_ABORT State: TRANS_ABORT -> TRANS_DEFAULT State: TRANS_ABORT -> TRANS_DEFAULT RecordTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) Need to call rollback or end manually here!!! PostgresMain firstchar = 'Q' exec_simple_query start_xact_command StartTransactionCommand Block state: TBLOCK_INPROGRESS -> TBLOCK_ABORT PortalRun ProcessUtility TRANS_STMT_ROLLBACK UserAbortTransactionBlock Block state: TBLOCK_ABORT -> TBLOCK_ABORT_END CommitTransactionCommand finish_xact_command
  • 52.
    Master(GP_ROLE_DISPATCH) Seg-N Kill RM-0before master write log, RM-1 Prepared --------------------some RM Failed---------------------- PostgresMain CommitTransactionCommand finish_xact_command CommitTransaction prepareDtxTransaction doPrepareTransaction AbortCurrentTransaction TRANS_INPROGRESS TBLOCK_END DTX_STATE_PREPARING AbortTransaction rollbackDtxTransaction Gxactstate: DTX_STATE_PREPARING -> DTX_STATE_RETRY_ABORT_PREPARED State: TRANS_INPROGRESS -> TRANS_ABORT disconnectAndDestroyAllGangs GP Kill RM-1 doDtxPhase2Retry StartTransactionCommand DTX_STATE_RETRY_COMMIT_PREPARED StartTransaction TRANS_DEFAULT -> TRANS_START doDispatchDtxProtocolCommand GP Renew all RMs PostgresMain BeginCommand performDtxProtocolCommand exec_mpp_dtx_protocol_command firstchar = ‘T' dtxProtocolCommand = DTX_PROTOCOL_COMMAND_ABORT_PREPARED performDtxProtocolAbortPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand DTX_CONTEXT_LOCAL_ONLY -> DTX_CONTEXT_QD_RETRY_PHASE_2 DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE -> DTX_CONTEXT_LOCAL_ONLY CommitTransactionCommand DTX_CONTEXT_QD_RETRY_PHASE_2 -> DTX_CONTEXT_LOCAL_ONLY finishDistributedTransactionContext Do nothing on RM-0 Do Abort Prepared on RM-1 performDtxProtocolAbortPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand RecordTransactionAbortPrepared XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_ABORT_PRE GP Kill all RMs then DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED
  • 53.
    Master(GP_ROLE_DISPATCH) Seg-N Kill RM-0when master send commit to RMs --------------------some RM Failed---------------------- PostgresMain disconnectAndDestroyAllGangs GP Kill RM-1 doDtxPhase2Retry StartTransactionCommand DTX_STATE_RETRY_COMMIT_PREPARED StartTransaction TRANS_DEFAULT -> TRANS_START DTX_PROTOCOL_COMMAND_COMMIT_PREPARED doDispatchDtxProtocolCommand GP Renew all RMs DTX_CONTEXT_LOCAL_ONLY -> DTX_CONTEXT_QD_RETRY_PHASE_2 DTX_CONTEXT_QD_DISTRIBUTED_CAPABLE -> DTX_CONTEXT_LOCAL_ONLY CommitTransactionCommand DTX_CONTEXT_QD_RETRY_PHASE_2 -> DTX_CONTEXT_LOCAL_ONLY Do nothing on RM-0 Do Commit Prepared on RM-1 performDtxProtocolCommitPrepared StartTransactionCommand FinishPreparedTransaction CommitTransactionCommand RecordTransactionCommitPrepared XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPAR doInsertForgetCommitted Dtx state : DTX_STATE_RETRY_COMMIT_PREPARED -> DTX_STATE_INSERTING_FORGET_COMMITTED Dtx state : DTX_STATE_INSERTING_FORGET_COMMITTED -> DTX_STATE_INSERTED_FORGET_COMMITTED doInsertForgetCommitted RecordDistributedForgetCommitted XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) doDispatchDtxProtocolCommand DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED RecordTransactionCommit CommitTransaction XLogFlush Dtx state : DTX_STATE_NOTIFYING_COMMIT_PREPARED-> DTX_STATE_RETRY_COMMIT_PREPARED
  • 54.
    Master(GP_ROLE_DISPATCH) Seg-N Kill allafter RMs write prepare log ---------------------------- ---------------------------- ------------------- Kill All TM/RMs and restart ------------------- ---------------------------- -- PostgresMain DTX_PROTOCOL_COMMAND_PREPARE doDispatchDtxProtocolCommand PostgresMain exec_mpp_dtx_protocol_command firstchar = ‘T' XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_PREPARE) Gp_role == GP_ROLE_DISPATCH initTM initTM_recover_as_needed recoverTM recoverInDoubtTransactions gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gid from pg_prepared_xacts) First char: 'M'; Return indoubt transactions abortRMInDoubtTransactions doAbortInDoubt doDispatchDtxProtocolCommand performDtxProtocolAbortPrepared StartTransactionCommand FinishPreparedTransaction RecordTransactionAbortPrepared XLogInsert (RM_XACT_ID, XLOG_XACT_ABORT_PREPARED | XLO DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED segmentMaxDistributedTransactionId gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gp_max_distributed_xid()) First char: 'M'; Return gp_max_distributed_xid First char: ‘T'; DTX_PROTOCOL_COMMAND_RECOVERY_ABORT_PREPARED XLogFlush shmNumGxacts = 0 ReplayRedoFromUtilityMode redoDistributedCommitRecord
  • 55.
    Master(GP_ROLE_DISPATCH) Seg-N Kill allafter TMs write commit log ---------------------------- ---------------------------- ------------------- Kill All TM/RMs and restart ------------------- ---------------------------- -- PostgresMain DTX_PROTOCOL_COMMAND_COMMIT_PREPARED doDispatchDtxProtocolCommand PostgresMain exec_mpp_dtx_protocol_command firstchar = ‘T' XLogFlush XLogInsert (RM_XACT_ID, XLOG_XACT_PREPARE) Gp_role == GP_ROLE_DISPATCH initTM initTM_recover_as_needed recoverTM recoverInDoubtTransactions shmNumGxacts = 1 gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gid from pg_prepared_xacts) segmentMaxDistributedTransactionId gatherRMInDoubtTransactions cdbdisp_dispatchRMCommand(select gp_max_distributed_xid()) First char: 'M'; Return gp_max_distributed_xid ReplayRedoFromUtilityMode redoDistributedCommitRecord dispatchDtxProtocolCommand doNotifyCommittedInDoubt performDtxProtocolCommitPrepared StartTransactionCommand FinishPreparedTransaction RecordTransactionAbortPrepared XLogInsert (RM_XACT_ID, XLOG_XACT_COMMIT_PREPARED | XLOG_ DTX_CONTEXT_QE_PREPARED -> DTX_CONTEXT_QE_FINISH_PREPARED First char: ‘T'; XLogFlush doInsertForgetCommitted RecordDistributedForgetCommitted XLogInsert (RM_XACT_ID, XLOG_XACT_DISTRIBUTED_FORGET | XLOG_NO_TRAN) First char: 'M'; Return 0 indoubt transactions
  • 56.
    Master(GP_ROLE_DISPATCH) Seg-N savepoint s1;PostgresMain firstchar = ‘M' exec_simple_query start_xact_command PortalRun PortalRunMulti CommitTransactionCommand finish_xact_command state: TRANS_DEFAULT-> TRANS_START- StartSubTransaction Block state: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS state: TRANS_START-> TRANS_INPROGRESS dispatchDtxCommand (SAVEPOINT s1) CommitTransactionCommand finish_xact_command state: TRANS_DEFAULT-> TRANS_START- StartSubTransaction Block state: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS state: TRANS_START-> TRANS_INPROGRESS ProcessUtility PortalRunUtility DefineDispatchSavepoint DefineSavepoint PushTransaction TRANS_STMT_SAVEPOINT TRANS_DEFAULT TBLOCK_SUBBEGIN ProcessUtility DefineSavepoint sendDtxExplicitBegin();
  • 57.
    Master(GP_ROLE_DISPATCH) Seg-N rollback tosavepoint s2 PostgresMain firstchar = ‘M' exec_simple_query start_xact_command PortalRun PortalRunMulti CommitTransactionCommand finish_xact_command state: TRANS_DEFAULT-> TRANS_START- StartSubTransaction state: TRANS_START-> TRANS_INPROGRESS dispatchDtxCommand (rollback to savepoint s2) CommitTransactionCommand finish_xact_command AbortSubTransaction state: TRANS_INPROGRESS -> TRANS_ABORT ProcessUtility PortalRunUtility RollbackToSavepoint TRANS_STMT_ROLLBACK_TO ProcessUtility RollbackToSavepoint TRANS_STMT_ROLLBACK_TO Sub state: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBABORT_PENDING State: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBRESTART CleanupSubTransaction state: TRANS_ABORT -> TRANS_DEFAULT DispatchRollbackToSavepoint Sub state: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBABORT_PENDING State: TBLOCK_SUBINPROGRESS -> TBLOCK_SUBRESTART AbortSubTransaction state: TRANS_INPROGRESS -> TRANS_ABORT CleanupSubTransaction state: TRANS_ABORT -> TRANS_DEFAULT RecordSubTransactionAbort XLogInsert(RM_XACT_ID, XLOG_XACT_ABORT) DefineSavepoint PushTransaction TRANS_DEFAULT TBLOCK_SUBBEGIN StartSubTransaction State: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS DefineSavepoint PushTransaction TRANS_DEFAULT TBLOCK_SUBBEGIN StartSubTransaction State: TBLOCK_SUBBEGIN -> TBLOCK_SUBINPROGRESS
  • 58.
    Master(GP_ROLE_DISPATCH) Seg-N release savepoints2 PostgresMain firstchar = ‘M' exec_simple_query CommitTransactionCommand finish_xact_command state: TRANS_INPROGRESS -> TRANS_COMMIT CommitSubTransaction CommitTransactionCommand finish_xact_command CommitSubTransaction state: TRANS_INPROGRESS -> TRANS_COMMIT ProcessUtility TRANS_STMT_RELEASE ProcessUtility ReleaseSavepoint TRANS_STMT_RELEASE dispatchDtxCommand While Sub blockState: TBLOCK_SUBEND If blockState: TBLOCK_END CommitTransaction State: TBLOCK_END -> TBLOCK_DEFAULT If blockState: TBLOCK_PREPARE PopTransaction Commit with sp State: TRANS_BEGIN -> TRANS_PREPARE PrepareTransaction XLogFlush EndPrepare XLogInsert (RM_XACT_ID, XLOG_XACT_PREP State: TRANS_PREPARE -> TRANS_DEFAULT prepareDtxTransaction ......
  • 59.
    Debug tips • regres0155749 55656 0 08:50 ? 00:00:00 postgres: port 5432, regres01 postgres [local] con6 [local] cmd5 idle in transaction • regres01 56475 55619 0 09:28 ? 00:00:00 postgres: port 40000, regres01 postgres localhost(6655) con6 seg0 idle in transaction • regres01 56478 55620 0 09:28 ? 00:00:00 postgres: port 40001, regres01 postgres localhost(14862) con6 seg1 idle in transaction • gp_vmem_idle_resource_timeout is for idle mem time in case the resource will be released • gpconfig -c gp_vmem_idle_resource_timeout –v 300
  • 60.
    Debug tips • bStartTransactionCommand • b CommitTransactionCommand • b AbortCurrentTransaction • b BeginTransactionBlock • b EndTransactionBlock • b UserAbortTransactionBlock • b DefineSavepoint • b RollbackToSavepoint • b ReleaseSavepoint • b StartTransaction • b CommitTransaction • b AbortTransaction • b CleanupTransaction • b StartSubTransaction • b CommitSubTransaction • b AbortSubTransaction • b CleanupSubTransaction • b performDtxProtocolCommand • b exec_mpp_dtx_protocol_command • b recoverTM • b recoverInDoubtTransactions • b gatherRMInDoubtTransactions • b abortRMInDoubtTransactions • b cleanRMResultSets • b resolveInDoubtDtx • b doDispatchDtxProtocolCommand • b doPrepareTransaction • b doInsertForgetCommitted • b doNotifyingCommitPrepared • b doNotifyingAbort • b doNotifyCommittedInDoubt • b doAbortInDoubt • b doQEDistributedExplicitBegin • b UtilityModeSaveRedo • b ReplayRedoFromUtilityMode • b RemoveRedoUtilityModeFile • b performDtxProtocolCommitPrepared • b performDtxProtocolAbortPrepared • b cdbdisp_debugDispatchResult • b ProcessUtility • b prepareDtxTransaction • b PrepareTransaction • b XLogFlush • b ReadyForQuery_QEWriter • b RecordDistributedForgetCommitted • b RecordTransactionCommit • b RecordCrashTransactionAbortRecord • b XLogInsert • b XLogFlush • b forcedDistributedCommitted • b rollbackDtxTransaction • b dtmPreCommand • b PrepareTransaction • b disconnectAndDestroyAllGangs
  • 61.