Data Replication Implementation
Upcoming SlideShare
Loading in...5
×
 

Data Replication Implementation

on

  • 868 views

 

Statistics

Views

Total Views
868
Slideshare-icon Views on SlideShare
868
Embed Views
0

Actions

Likes
1
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Data Replication Implementation Data Replication Implementation Presentation Transcript

    • What is New in IBM Replication Solution Richard Sinn IBM Santa Teresa Lab AS/400 Tech Conference 2000
    • IBM Santa Teresa Lab
    • Presentation Goals
      • Learn introduction materials to replication
      • Learn IBM’s solution and where to order
      • Understand the concept of Capture/Apply
      • Pickup useful details of Capture/Apply
      • Quick Overview of Advanced Features and Future Enhancement
    • IBM DB2 Data Propagator
      • Product ID:
        • 5769DP2 for V4R1+
        • 5769DPX for future release (X is a number)
      • Product Home Page at: http://www.software.ibm.com/data/dpropr
      • You could order online OR try it free for 70 days in future release
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features
      • Future Enhancement
      • Conclusion
    • What is Replication
      • Movement of Data from a database source system to a target database
      • Copy only the "Changes" vs Copy "the whole thing"
      • Redefining Application Development Model
      • Build Data Warehouse
      • Making Data Available
    • Replication Requirements
    • Why Replication ?
      • Making Data Available
      • Use Information Intelligently and improve decision making
      • Stay competitive with both Data access and Data movement
      • Speedup application deployment
      • Reengineer business processes
      • Increase online throughput
      • Improve system availability
      • Support audit requirements
      • Support Data Warehousing
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features
      • Future Enhancement
      • Conclusion
    • IBM Data Propagator Overview
    • IBM Data Propagator Overview
      • Capture works on "Source System"
      • Apply works on "Target System"
      • To tell Capture what to work on: Registration
      • To tell Apply what to work on: Subscription
      • Registrations/Subscriptions are created by DJRA
    • IBM Data Propagator Overview
      • DPR runs on multiple platforms (AIX, NT, AS/400, 95, HP-UNIX, OS/2, etc)
      • All control info is SQL based
      • Published Architecture: works with NotesPump, etc
      • Well established technology
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features
      • Future Enhancement
      • Conclusion
    • Overview: Admin with DJRA
    • Overview: Administration
    • Administration: Registration
      • Select AS/400 database
      • Select table
      • Select necessary column only
      • Pick capture policy
      • Pick update capture policy
      • Pick conflict detection level
    • Overview: Capture
    • Overview: Capture
      • DJRA creates registration information in the control tables associated with Capture/400
      • "Changes" are actually put into "Change Data" table
      • When there is a commit, a row will be put into "Unit of Work" table
      • Captures base table changes from Journal
      • Timestamps changes
      • Maintains transaction consistency
      • Automatically maintains staging tables
      • Pruning of CD tables are automatically as well
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features
      • Future Enhancement
      • Conclusion
    • Overview: Subscription Set
    • Overview: Subscription Member
    • Overview: Subscription
      • Subscription is bounded by set
      • Set can have multiple members
      • Event based, or Time based
      • Blocking factor
      • Can select necessary columns only
    • Overview: Apply
    • Overview: Apply
      • DJRA creates control information associated with Apply/400
      • According to the control info, Apply
        • Runs at user-specified intervals or events
        • Refreshes, updates, and enhances copies
        • Do distribution optimizations
      • Apply picks up "changes" from the "Change Data" table and "Unit of Work" table
      • Apply communicates with Capture via Syn point
      • Apply is a big SQL application
      • Apply can be "Push" or "Pull"
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features
      • Future Enhancement
      • Conclusion
    • Useful Internal: Capture
    • Capture: Control Job
      • Control Job - QDPRCTL5
        • Analyze the user options and new registration
        • Check replication source validity
        • Sort out all selected replication sources by journals
        • Determine eligibility
        • Determine journal job starting time
        • Start up journal jobs
        • Prune - CD tables, UOW and AUTHTKN
        • Handle messages (from commands and journal jobs)
    • Capture: Journal Jobs
      • Journal Job(s) - Top Program QZSNCAP2
        • Issue RCVJRNE command
          • Use time to specify which journal entries to process
          • Give a list of up to 300 file/members
          • Give a list of 24+ specific journal entry types
          • Specify a delay of 60 seconds
        • Handles messages from exit program
    • Capture: Exit Program
      • Journal Job(s) - Exit Program QZSNCAP3
        • Receive and analyze changes from the journal
        • Advance SYNCHPOINT in PRUNCNTL table on a set basis when necessary
        • Maintain CD_OLD_SYNCHPOINT and CD_NEW_SYNCHPOINT in register table
        • Maintain the global row in register table
        • Support application commitment control
        • Handle special journal types (end journal, renamed, etc)
        • Cancel handling (Warm start table), etc
    • Capture: Performance Tuning
      • Latency is 4 to 5 minutes
      • CPU overhead is about 6 to 8%
      • Ways to tune performance:
        • Job priority
        • Num of replication sources per journal job
        • COMMIT_INTERVAL in CCPPARMS
    • Apply: Job Information
      • STRDPRAPY APYQUAL(ABC ) TRACE(*ALL)
      Work with Subsystem Jobs STL400F 01/24/00 15:30:03 Subsystem . . . . . . . . . . : QZSNDPR Opt Job User Type -----Status----- Function ABC SINN BATCH MSGW PGM-QZSNAPV2 KSCHIA KSCHIA BATCH ACTIVE PGM-QZSNAPV2 QDPRCTL5 KSCHIA BATCH MSGW PGM-QZSNCAP1 QSQJRN KSCHIA BATCH ACTIVE PGM-QZSNCV72
    • Apply: Trace Information
      • STRDPRAPY APYQUAL(ABC ) TRACE(*ALL)
      Work with Job Spooled Files Job: ABC User: SINN Number: 048177 Device or Total Current Opt File Queue User Data Status Pages Page Copies QPZSNATRC PRT01 ABC OPN 1818 1
    • Apply: Apply Trail Table
      • SQL format trail info is also available
      • STRSQL
      • select * from asn/ibmsnap_applytrail
      APPLY_QUAL SET_NAME WHOS_ON_FIRST ASNLOAD MASS_DELETE EFFECTIVE_MEMBERS STL400F STL400F S N Y 1 STL400F STL400F S N N 0 STL400F STL400F S N N 0 STL400F STL400F S N N 0 STL400F STL400F S N N 1 STL400F STL400F S N N 0 STL400F STL400F S N N 0 STL400F STL400F S N N 1
    • Apply: Apply Trail Table
      • Some of the available information are:
      • APPLY_QUAL, SET_NAME, ASNLOAD, MASS_DELETE, EFFECTIVE_MEMBERS, SET_INSERTED, SET_DELETED, SET_UPDATED, SET_REWORKED, STATUS, LASTRUN, LASTSUCCESS, SOURCE_SERVER, SOURCE_TABLE, SOURCE_VIEW_QUAL, TARGET_SERVER, TARGET_TABLE, SQLSTATE, SQLCODE, SQLERRP, SQLERRM
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features
      • Future Enhancement
      • Conclusion
    • Advance Replication Features
      • AS/400 Relative Record Number (RRN) Support
      • Remote Journal Replication Support
      • Automated Deletion of Journal Receivers
      • Performance Improvement to Apply/400
      • Programming with ASNLOAD and ASNDONE
      • Set Subscriptions
      • Update Anywhere
      • IBM Replication Solution Extendibility
    • RRN Replication Support
    • RRN Replication Support
      • Allow user to have a unique key for CCD and Point-in-time target tables when no combination of target columns is unique (no SQL key)
      • DJRA support administration of RRN replication on AS/400
    • Remote Journal Replication
      • Remote Journal can be setup using AS/400 commands (V4R3+)
      • Capture and Apply run on the same system
      • Free up resource on operational (source) system
      • Better performance (no network overhead, etc)
    • Remote Journal Replication
    • Automated Deletion of JR Receivers
    • Automated Deletion of JR Receivers
      • Exit program registered at installation time (QIBM_QJO_DLT_JRNRCV)
      • DLTJRNRCV calls the exit program before deleting JR receivers
      • If Capture needs the receiver, it votes no on delete
      • When receivers are managed by system, it can then be deleted automatically
    • Performance Improvements to Apply
      • Blocked reading and writing to work files
      • Use of OS/400 API QSQPRCED to perform blocked insert into local AS/400 target tables during full refresh
      • Use of CLRPFM (Clear Physical File Member) CL command to clear local AS/400 target tables
      • Use of Static SQL (instead of Dynamic SQL) when possible
      • "Direct insert" in remote journal replication
    • Programming with Data Propagator
      • ASNDONE exit routine is a program that Apply can optionally call after subscription processing completes, regardless of success or failure
      • Divided by Apply qualifier
      • Other done logic can be added
    • Programming with Data Propagator
      • ASNLOAD full-refresh exit routine is called by Apply
        • when a full refresh of a target table is necessary OR
        • if you specify the name of a full refresh program on the FULLREFPGM parameter when you start Apply
      • Can be used to do "offline, non-SQL" full refresh
      • Other pre-load logic can also be added
      • ASNDONE/ASNLOAD can be named anything, located in any library
      • ASNDONE/ASNLOAD can be coded in C, COBOL, RPG, etc.
    • Features: Set Subscriptions
    • Features: Set Subscriptions
      • Maintains referential constraints over replication
      • Replicates tables within a set in a single unit of work
      • Coordinates the components of a view subscription
      • Limits the boundary for cascade rejections due to RI violation or update collision
    • Features: Update Anywhere
    • Features: Update Anywhere
      • Hierarchical Architecture
      • Three levels of conflict detection
      • Automatic transaction compensation
      • Compensation of dependent transactions
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features: Extendibility
      • Future Enhancement
      • Conclusion
    • Extendibility: DataJoiner + Data Propagator
    • Extendibility: DataJoiner + Data Propagator
    • Extendibility: DataJoiner + Data Propagator
    • Extendibility: Lotus Notes, IMS, Flat Files
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features: Extendibility
      • Future Enhancement
      • Conclusion
    • Future Enhancement
      • Enhanced Remote Journal Replication
        • Administration
        • Performance
      • CD Row Changed Only Support
        • Generate CD Row when there is change in the captured column
        • DASD vs CPU
      • LOB replication Support
        • Easy management of LOB object across the network
    • Future Enhancement
      • Delivered as Keyed Stamped Media
        • Every new OS/400 customers will have a copy
        • Try and Buy
        • Manual available online
      • BIG INTEGER Replication Support
      • Support for greater than 18 characters object
      • DataLinks Replication
        • Integrated with your database and file system
        • Customer programming for replication
    • Agenda
      • What is Replication
      • IBM Data Propagator Overview
        • Capture Side
        • Apply Side
      • AS/400 Data Propagator Useful Internal
      • Advance Replication Features: Extendibility
      • Future Enhancement
      • Conclusion
    • Conclusion: IBM Data Propagator Solution
      • Versatile Infrastructure
        • Operational application
        • Informational application
        • Mobile & occasionally connected systems
        • Update anywhere with conflict detection and automatic compensation
      • Leverages Current Assets
        • Legacy Sources
        • Standard SQL
      • Efficient & Automated Operations
        • Journal/Log-based capture for DB2
        • Trigger-based capture for non-DB2 sources
        • Network optimizations
      • Scaleable Design
        • Servers
        • Networks
      • Flexible Distribution
        • Subsetting with join views
        • Fan-out distribution
      • Integrated Administration
        • DB2 UDB Control Center GUI for administration for V5
        • CL commands for pre-V5 function
        • Automated initialization
        • Tailorable
      • Robust Data Enhancements
        • Derivation, Summarization, Translation...
      • Multi-vendor Interoperability
        • SQL based architecture
        • Architected data staging area
        • Replication among DB2, Oracle, Sybase, SQL Server, Informix, IMS, IDMS and VSAM
    • Conclusion: IBM Data Propagator Solution
    • Reference
      • http://www.software.ibm.com/data/dpropr
      • http://www.software.ibm.com/data/datajoiner
      • http://www.software.ibm.com/data
      • http://as400bks.rochester.ibm.com
      • http://www.as400.ibm.com
      • THANKS MUCH FOR COMING !!!
    • Trademark Information
      • AIX, AS/400, DataHub, DataJoiner, DataPropagator, DataRefresher, DB2, DProp, DRDA, IBM, IMS, MVS, OS/2, OS/400, RS/6000, VM/ESA and VSE/ESA are trademarks or registered trademarks of the IBM Corpopration in the United States or other countries or both.
      • Microsoft, Windows, and the Windows 95, and the Windows 98, and the Windows NT logos are trademarks or registered trademarks of Microsoft Corporation.
      • UNIX is a registered trademark in the United States and other countries licensed exclusively through X/Open Company Limited.
      • Lotus Notes is a trademark of the Lotus Development Corporation.
      • Other company, product, and service names may be trademarks or service marks of others.