Oracle GoldenGate Architecture Performance

  • 255 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
255
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Copyright  ©  2014  Oracle  and/or  its  affiliates.  All  rights  reserved.    |   Understanding  Oracle   GoldenGate  Architecture   Bobby  CurCs   Senior  Technical  Consultant   Accenture  Enkitec  Group   July  9,  2014   1   Performance  Tuning  to  the  OS  Level  
  • 2. Speaker  Info   •  Douglasville,  Georgia   •  Senior  Technical  Consultant   •  Expert  OEM12c  (2013)/PracCcal  ODA   (2014)   •  TwiUer:  @dbasolved   •  Blog:  hUp://dbasolved.com   •  Email:  bcurCs@enkitec.com                                    curCsbl@gmail.com  
  • 3. Accenture  Enkitec  Group   Enkitec  has  joined  Accenture’s  Infrastructure  Services   as  Accenture  Enkitec  Group   •  17,000  Infrastructure  Services  professionals   •  52,000  Oracle  professionals   Focus  on  Oracle  Engineered  Systems  SoluCons   •  Database  MigraCons  &  TransformaCons   •  Database-­‐as-­‐a-­‐Service   •  Oracle  ApplicaCons  on  Engineered  Systems   •  Cloud-­‐based  soluCons  leveraging  Engineered   Systems  
  • 4. Agenda   •  General  InformaCon   •  What  is  GoldenGate   •  Benefits   •  Basic  Architecture   •  Use  Cases   •  Tuning  InformaCon   •  GoldenGate  Tuning   •  OperaCng  System  Tuning   •  Processes   •  Memory   •  Disk  
  • 5. General  InformaCon  
  • 6. What  is  Oracle  GoldenGate   •  Comprehensive  sobware  for  real-­‐Cme  data  integraCon   in  heterogeneous  environments   •  Benefits   •  High  Availability  SoluCons   •  Real-­‐Time  Data  IntegraCon   •  TransformaCon  of  Data   •  TransacConal  Change  Data  Capture   •  Data  ReplicaCon   •  VerificaCon  of  Data  (Veridata)   •   Primary  ReplicaCon  Tool  in  Oracle  Database  12c   (12.1.0.1)   •  Oracle  Upgrade  Guide,  SecCons  8.1.6  &  8.1.7   •  MulCple  ways  to  successfully  monitor  Oracle   GoldenGate  
  • 7. Benefits     •  Real-­‐Time  Data  Capture   •  Near-­‐Zero  DownCme  migraCons   •  Quickly  upgrade  databases   •  Open  Architecture  (Heterogeneous)   •  Disaster  Recovery   •  Data  DistribuCon   •  Distributed  Offices   •  Query  Off-­‐loading   •  ReporCng  Instances  
  • 8. Basic  Architecture   Capture:  commiUed  transacCons  are  captured  (and  can  be  filtered)  as  they  occur  by   reading  the  transacCon  logs.   Trail:  stages  and  queues  data  for  rouCng.   Pump:  distributes  data  for  rouCng  to  target(s).     Route:  data  is  compressed,  encrypted  for  rouCng  to  target(s).   Delivery:  applies  data  with  transacCon   integrity,  transforming  the  data  as  required.   Source Oracle / Non-Oracle Database Target Oracle / Non-Oracle Database Capture Delivery Trail Files Pump Trail Files Pump Delivery Capture Bi-directional LAN / WAN / Internet Over TCP/IP Trail Files Trail Files Modular  Architecture  
  • 9. Use  Cases     Unidirec8onal   Query  Offloading   Zero-­‐DownCme  MigraCon   Data  IntegraCon  Cloud  or  On-­‐Premise     Bi-­‐Direc8onal   AcCve-­‐AcCve  for  MulC-­‐Master/HA   Cloud  or  On-­‐Premises   Data  Distribu8on   via  Messaging   Cloud  Apps  Integra8on   Big  Data  Delivery   Real/Time  and  Batch  Delivery   Structured  Data  to  Data   Reservoir  
  • 10. Tuning  InformaCon  
  • 11. Performance  Tuning     •  Per  Oracle   •  Baselining   •  GoldenGate  Tuning   •  Lag  Times   •  AWR/ASH  Reports   •  CPU   •  Memory   •  I/O   •  OperaCng  System  Tuning  
  • 12. Per  Oracle   •  MulCple  Process  Groups   •  Data  Integrity   •  Number  of  Groups   •  Dependent  on  amount  of  memory   •  Use  Network  Efficiently   •  Benchmark  Extract/Replicat   (checkpoints)   •   Eliminate  disk  I/O   •  Writes  data  sequenCally,  RAID  0+1   •   Manage  VM  and  Paging   •  Cachemgr  
  • 13. Baselining   •  Should  be  first  thing  you  gather   •  What  should  be  gathered?   •  GoldenGate   •  Lag   •  Checkpoint  Info   •  General  AWR  InformaCon  (DB)   •  OperaCng  System   •  CPU  performance  (mpstat)   •  Memory  performance  (vmstat)   •  I/O  performance  (iostat)  
  • 14. GoldenGate  Lag  Info   •  ReporCng  Lag   •  Gives  an  idea  of  how  long  it  takes  data   to  transfer  on  network   •  GGSCI  commands   •  lag  extract  <group  name>   •  lag  replicat  <group  name>  
  • 15. GoldenGate  Checkpoint  Info   •  Checkpoint   •  Helps  detect  network  boUlenecks   •  GGSCI  commands   •  info  extract  <group  name>,  showch   <number>  
  • 16. Database  Tuning   •  AWR  Reports   •  Wait  Event:  Streams  Miscellaneous  Event  
  • 17. CPU  InformaCon   •  Use  mpstat   •  mpstat  –P  ALL  [count  [interval]]   •  sar  can  be  used  as  well  
  • 18. Memory  Performance   •  Use  vmstat   •  Reports  on  virtual  memory  staCsCcs   •  vmstat  –S  M  [count  [interval]]  
  • 19. Disk  I/O  Tuning   •  Use  iostat   •  Reports  staCsCcs  transfered  between  devices   •  iostat  –m  [count  [interval]]  
  • 20. Baseline  Completed   Metric   Baseline   Running   Lag   5  sec  (ext)/12  sec  (rep)   1  sec  (ext)/4  sec  (rep)   CPU   4.64%  (iowait  (all))   7.73%  (iowait(all))   Memory   2214M  (free)   2109M  (free)   Disk  IO   29.16  tps   29.06  tps   •  ObservaCon:   •  Serial  processing  (no  parallel)   •  Baseline  was  taken  with  higher  transacCon  count   •  Disk  I/O  was  slightly  lower  with  high  CPU  waits   •  More  memory  was  used  during  the  “running”  test  
  • 21. Tune  GoldenGate  
  • 22. Tune  GoldenGate   •  MulCple  Process  Groups  (Classic)   •  Parallel  Replicat  Groups   •  3  to  1  raCo  (replicat  to  trail  (classic))   •  Reduced  with  Integrated  Replicat   •  DB  (11.2.0.4)  and  OGG  12c  (12.1.2)   Source   Oracle  &  Non-­‐Oracle   Database(s)   Target   Oracle  &  Non-­‐Oracle   Database(s)   Capture Trail Files Pump DeliveryTrail Files Delivery Delivery LAN / WAN / Internet Over TCP/IP
  • 23. Tune  GoldenGate   •  MulCple  Process  Groups  (Classic)   •  Parallel  Extract  Groups   •  1  to  1  raCo  (extract  to  trail  to  replicat)   •  Reduced  with  Integrated  processes   Source   Oracle  &  Non-­‐Oracle   Database(s)   Target   Oracle  &  Non-­‐Oracle   Database(s)   Trail Files Pump Trail Files Delivery Delivery Delivery LAN / WAN / Internet Over TCP/IP Capture Capture Capture
  • 24. Tune  GoldenGate   •  Use  Network  Efficiently   •  Lag  Cmes  increase   •  Check  target  database  performance   •  Consult  Network  Team  for  any   performance  issue   •  Checkpoint  Extract/Replicat   •  send replicat rep, status •  Indicates  any  delay  with  replicat  
  • 25. Tune  GoldenGate   •  Eliminate  disk  I/O   •  Try  to  eliminate  this  first   •  System  ConfiguraCons   •  Most  shops,  disk  are  controlled  by   SAN  Admin  or  internal  devices   •  Choose  RAID  0+1  over  RAID  5   •  GoldenGate  ConfiguraCon   •  Increase  values  of  parameters:   •  CHECKPOINTSECS •  GROUPTRANSOPS •  EOFDELAY or EOFDELAYSECS  
  • 26. Tune  GoldenGate   •  Manage  VM  and  Paging   •  Default:  use  as  much  memory  allowed   •  Default:  ./dirtmp  and  as  much  space  as   allowed   •  Can  be  controlled  with  cachemgr parameter   •  Valid  for  Extract/Replicat   •  Current  seyngs  can  be  viewed  in   report  file  
  • 27. Tune  GoldenGate  (cachemgr  con’t)   •  EsCmate  swap  space  needed   •  Extract_Swap_Needed =(PROCESS_VM * num_extracts) + (swap_other_processes) •  Replicat_Swap_Needed =(PROCESS_VM * num_replicats) + (swap_other_processes)   •   Review  in  report  file   •  send [extract/replicat] <group name>, cachemanager    
  • 28. Profiling  GoldenGate  
  • 29. Profiling  the  GoldenGate  Processes   •  perf     •  Used  to  profile  OS  processes  and   hardware  performance   •  strace   •  Used  to  see  what  processes  are   calling   •  backtrace  file  
  • 30. Profiling  the  GoldenGate  Extract   •  backtrace  file  output   •  InteresCng  find  with  extract  process     (6  Threads  =  1  extract)  
  • 31. Profiling  the  GoldenGate  Extract   •  Threads  IdenCfied  for  extract   •  Number  of  threads  increase/add   extracts   •  1  extract  =  6  threads   •  2  extracts  =  12  threads,  etc..   •  Each  thread  takes/uses  memory   •  Average  Memory  22-­‐50  MB   •  3.6-­‐8.3  MB  per  thread  
  • 32. Conclusion   •  Reviewed  Oracle  GoldenGate   Architecture   •  Discussed  tuning  aspects  of  Oracle   GoldenGate  for  applicaCon  and   operaCng  system   •  Quickly  looked  at  trace  of  an  extract   process  
  • 33. Speaker  Info   •  Douglasville,  Georgia   •  Senior  Technical  Consultant   •  Expert  OEM12c  (2013)/PracCcal  ODA   (2014)   •  TwiUer:  @dbasolved   •  Blog:  hUp://dbasolved.com   •  Email:  bcurCs@enkitec.com                                    curCsbl@gmail.com  
  • 34. Copyright  ©  2014  Oracle  and/or  its  affiliates.  All  rights  reserved.    |   Understanding  Oracle   GoldenGate  Architecture   Bobby  CurCs   Senior  Technical  Consultant   Accenture  Enkitec  Group   July  9,  2014   Oracle  ConfidenCal  –  Internal/Restricted/ Highly  Restricted   35   Performance  Tuning  to  the  OS  Level