Your SlideShare is downloading. ×
0
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
01   coherence & golden gate - technical overview
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

01 coherence & golden gate - technical overview

1,425

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,425
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
72
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Gartner “Cool Vendor” award Internet Scale – what happens to infrastructure as you open your services to customers and partners What happens as you come to rely on partner services How do you manage the flood of data and still meet SLA’s? How does infrastructure provide competitive advantage as companies compete in real-time and globally?
  • Come join this large, vibrant community.We have a pretty significant engagement through various social media channels like twitter and facebook. ….
  • Come join this large, vibrant community.We have a pretty significant engagement through various social media channels like twitter and facebook. ….
  • Transcript

    • 1. Coherence GoldenGate Oracle Coherence & GoldenGate Technical Overview
    • 2. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remain at the sole discretion of Oracle.
    • 3. Agenda • Coherence Overview • GoldenGate Overview • Integration
    • 4. Agenda • Coherence Overview • GoldenGate Overview • Integration
    • 5. The Need for an In-Memory Data Grid Scalability, Performance and Reliability at Internet Scale Scalability Performance 5 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Less # hits Better cost
    • 6. Modern Caching Architecture Pioneered by Coherence First Generation Cache Solutions Coherence Cache Solutions (still commonly seen today) Application Application Application Application Data Fabric Collocated Collocated Collocated Collocated Local Cache Local Cache Local Cache Local Cache • Difficult to scale – must scale vertically only • Complex to keep cache synchronized • Often performance problems due to large memory usage, large data sizes 6 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. • Easy to scale – scales horizontally & vertically • Clustered caching avoids synchronization issues • Can dramatically improve performance with parallel query capabilities and compute
    • 7. The Data Grid at a Glance Major Benefits • Lower cost / Backend systems protection Less #hits to backend systems Match data supply with data demand by without scaling-up backends Extreme scale-out using commodity hardware • Enhanced performance / Better user-experience In-memory data access – information close in memory, not in backend systems • Enhanced availability / Better user-experience Reliable in-memory data grid providing high-availability to applications Reliable transaction processing at any load 7 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 8. Oracle Coherence Through 12c Pioneer & Market Leader Since 2001 Oracle acquires Tangosol 2000 2001 Tangosol founded to pioneer distributed caching 8 Coherence 1.0 ….. 2007 Cool Vendor Award Deep adoption in Fusion Middleware, Oracle Horizontal and Vertical and Business Applications and Exalogic 2008 2009 2010 2011 Alignment with Fusion Middleware Releases 2012 2013 Coherence Coherence Coherence Coherence Coherence 3.5 3.6 3.7 3.7.1.1 12.1.2 Increasing market share and feature capabilities for In Memory Data Grid. Market widened All Categories – Financial Services, E-Commerce, Travel, Insurance, Retail, Manufacturing, Telco … Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 9. Oracle Coherence Defined Distributed in-Memory Object-Oriented Data Grid Object – Oriented • • • • • Identified by a key (key-value pair) Any serializable object No proprietary classes to extend Not forced to use relational models, ORMs, SQL Fully native Java, .NET , C++ interoperability Distributed Grid / Clustered Caching • • • • 9 Data distribution > massive scale-out Backup replicas > High availability No single points of failure (SPoF) No single points of bottleneck (SPoB) Copyright © 2013 Oracle and/or its affiliates. All rights reserved. KEY No cost per object assembly
    • 10. Coherence Clustered Caching Explained Partitioned, Fault Tolerant, Self-Healing Cache • Cluster of nodes holding % of primary data locally • Back-up of primary data is distributed across all other nodes • Logical view of all data from any node ? • All nodes verify health of each other • In the event a node is unhealthy, other nodes diagnose state • Unhealthy node isolated from cluster • Remaining nodes redistribute primary and back-up responsibilities to healthy nodes 10 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. X
    • 11. Data Distribution Options Partitioned Caching > Scalability • • • • 11 Data is load balanced across the data grid Data and processing capacity scales linearly Ownership responsibilities also partitioned Access and update latency are constant Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 12. Data Distribution Options Partitioned Caching w/ Backup Replicas > High Availability • Automatically manages the fault tolerance of your data • Backups guaranteed to be on separate node / machine / rack / site from primary • Backup for data owned by one node is shared across the others Backup Storage Primary Storage 12 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 13. Data Distribution Options Cache Server vs. Cache Client > Application Protection • Storage enabled grid members > Cache Servers • Storage disabled grid members > Cache Clients Application Application Application Cache Clients Cache Servers 13 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 14. Data Distribution Options Near Caching > Enhanced Performance • 2-level cache • Most recently/frequently used data is stored locally Application Application Application Cache Clients Front Local Cache Back Distributed Cache Cache Servers 14 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 15. Querying The Data grid Parallel Queries & Aggregation • • • • 15 Programmatic & SQL-like query mechanism Queries performed in parallel across the grid Index support for better performance Application Aggregation support Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 16. Querying The Data grid Continuous Query Cache • Snapshot automatically updated as back entries change • Event – based Application 16 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 17. Event Notifications Real Time Event Driven Architectures • JavaBean Model (addListener), key and filter based events • Real-time notifications • Full-state & Lite events Application Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Application ! 17 Application ! !
    • 18. Processing on the grid In-place Data Processing and Map/Reduce Aggregation • Sends the processing to where the data is in the grid Once and only once guarantees / automatically fault-tolerant • Key(s) and filter based processing Application Processing Unit 18 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 19. Managing Persistence Caching Patterns • Cache Aside – Architecture manages cache ‒ Check the cache before reading from data source DAO ‒ Put data into cache after reading from data source ‒ Evict or update cache when updating data source Cache • Cache On Top – All data reads/writes occur through cache ‒ Read Through/Write Through • Cache miss causes load from data source automatically • Updates to cache written synchronously to the data source DAO Cache DAO Cache ‒ Write Behind • 19 Updates to cache written asynchronously to the data source Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 20. Use cases Caching Reduced latency / Backend (DB, Mainframe) downsizing Reliability / Scalability Data Grid acts as a transactional system of record, hosting data and business logic Parallel Computing & Events Parallel queries and processing / Automated processing based on event Interoperabiltiy Data exchange between heterogeneous apps (Java, .Net, C++) Shared contexts HTTP Session externalization / Data consolidation (profiles) 20 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 21. Join the Coherence Community http://coherence.oracle.com @OracleCoherence facebook.com/OracleCoherence blogs.oracle.com/OracleCoherence Oracle Coherence Users youtube.com/OracleCoherence coherence.oracle.com/display/CSIG Coherence Special Interest Group 21 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 22. Agenda • Coherence Overview • GoldenGate Overview • Integration
    • 23. Oracle GoldenGate Defined No-Impact, Real-Time Data Integration & Transactional Replication • Oracle GoldenGate is a real-time, log-based change data capture and replication software platform • The software provides: Capture Routing Transformation Delivery • of transactional data across heterogeneous data sources in real time 23 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 24. The Need for a Change Data Capture System Capturing Changes and Applying them Throughout the Enterprise Zero Downtime Upgrade & Migration New DB/HW/OS/APP Highly Available / Disaster Recovery Legacy HW Migration Mainframe modernization Mainframe downsizing Web specialization Fully Active Distributed DB Active - Active Query Offloading Mainframe Offloading Log-based, Reporting Database changed data Database Exadata Real-time BI, Operational Reporting, MDM Data Warehouse Data Synch Across the Enterprise Global Data Centers Message Bus 24 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Disaster Recovery Event Driven Architecture, SOA
    • 25. How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Source Oracle & Non-Oracle DBs 25 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Target Oracle & Non-Oracle DBs
    • 26. How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Source Oracle & Non-Oracle DBs 26 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Target Oracle & Non-Oracle DBs
    • 27. How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Source Oracle & Non-Oracle DBs 27 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Target Oracle & Non-Oracle DBs
    • 28. How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Source Oracle & Non-Oracle DBs 28 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Target Oracle & Non-Oracle DBs
    • 29. How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Delivery: applies data with transaction integrity, transforming the data as required. Source Oracle & Non-Oracle DBs 29 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Target Oracle & Non-Oracle DBs
    • 30. How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Delivery: applies data with transaction integrity, transforming the data as required. Target Oracle & Non-Oracle DBs Source Oracle & Non-Oracle DBs Bi-directional 30 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 31. Differentiator: Flexibility • One-to-One, One-to-Many, Many-to-One, Many-to-Many Unidirectional Bi-Directional Peer-to-Peer Query Offloading Zero-Downtime Migration Hot Standby or Active-Active for HA Load Balancing Multi-Master Broadcast Integration/Consolidation Data Distribution Data Distribution Data Warehouse via Messaging BPM BAM OEP Java 31 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 32. Oracle GoldenGate Differentiators Performance Flexible & Extendable Reliable Logical replication 32 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. • Zero-impact capture, routing, transformation, and delivery of transactional data • Sub-second latency for data movement • Open, modular architecture • Supports heterogeneous sources and targets • One-to-one, one-to-many, many-to-many, many-to-one • Maintains transactional integrity • Resilient against interruptions and failures • Fast Recovery after process interruptions. • • • • Conflict resolution in active-active configurations Filtering, mapping between different data models Simple transformations Encryption, compression,…
    • 33. Oracle GoldenGate Differentiators • Filtering & “Compression” Unnecessary change data is discarded immediately Updates and deletes are “compressed” by default Before images are discarded by default • Transformations and mappings Within either the Capture or Delivery modules - no middle tier server is needed Built-in functions, PL/SQL, custom functions (user-exits) • Flexible topology support Variety of topologies, one-to-one, one-to-many, many-to-one, many-tomany, and cascading and bidirectional configurations. 33 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 34. Oracle GoldenGate Differentiators • Bidirectional configuration support Oracle GoldenGate enables both active-passive and active-active solutions for maximum system availability. • Conflict detection and resolution Implemented globally, object by object, based on data values and complex filters, or through event-driven criteria. • Routing and compression Oracle GoldenGate uses TCP/IP for sending data, so no geographical distance constraints are imposed. • Security Data encryption (trail files, data sent) 34 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 35. Oracle Golden Gate Differentiators • Transaction integrity Transaction boundaries recorded in GoldenGate trail file Checkpointing and recovery are based on transaction boundaries Original commit sequence is maintained by the delivery process • Resiliency & fast recovery Individual processes can be restarted automatically Tolerance to network outages (configurable) Recovery ensures that no operations are skipped or duplicated after failure of any kind 35 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 36. Oracle GoldenGate Supported Platforms Databases Oracle GoldenGate Capture and Delivery:  Oracle  DB2  Microsoft SQL Server  Sybase ASE  Teradata  Enscribe  SQL/MP  SQL/MX  MySQL v 5.5  JMS message queues Oracle GoldenGate Delivery:  All listed above, plus: TimesTen, Postgres  Netezza, Greenplum, HP Neoview  ETL products 36 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. O/S and Platforms Linux Sun Solaris v11 Windows 2000, 2003, XP HP NonStop HP-UX IBM AIX v7.1 IBM z Series zLinux IBM i Series 36
    • 37. Use Cases Zero Downtime Upgrade & Migration New DB/HW/OS/APP Highly Available / Disaster Recovery Legacy HW Migration Mainframe modernization Mainframe downsizing Web specialization Fully Active Distributed DB Active - Active Query Offloading Mainframe Offloading Log-based, Reporting Database changed data Database Exadata Real-time BI, Operational Reporting, MDM Data Warehouse Data Synch Across the Enterprise Global Data Centers Message Bus 37 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Disaster Recovery Event Driven Architecture, SOA
    • 38. Join the GoldenGate Community http://www.oracle.com/technetwork/middleware/goldengate/overview/index.html twitter.com/ORCLGoldenGate facebook.com/OracleDataIntegration blogs.oracle.com/dataintegration Oracle Data Integration youtube.com/oraclegoldengate 38 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 39. Agenda • Coherence Overview • GoldenGate Overview • Integration
    • 40. Coherence HotCache Solve the Stale Cache Problem Coherence Application Read / Write Coherence Application HotCache 3rd Party Application 40 Copyright © 2013 Oracle and/or its affiliates. All rights reserved. Read / Write
    • 41. Coherence HotCache Solve the Stale Cache Problem • Propagate database changes to cache in real time • Leverage existing technologies TopLink Grid GoldenGate • Event-driven • Low latency • No application changes 41 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.
    • 42. GoldenGate as a Coherence Feed Turn the Database Into an Event Source Read / Write Batch Process 3rd Party Applications HotCache 42 Copyright © 2013 Oracle and/or its affiliates. All rights reserved.

    ×