Innovations in Data Grid Technology with Oracle Coherence

Innovations in Data Grid Technology with Oracle Coherence Randy Stafford | Architect At-Large Oracle Product Development | Cloud Application Foundation1 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

The following is intended to outline general product use and direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.2 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Presentation Objectives To Understand… 1) What is driving the need for Distributed Caching and In- Memory Data Grid products? 2) How Coherence improves performance, enables linear scalability, and provides availability for shared services 3) Coherence’s Strategic Direction and Latest Features3 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Internet Scale Problem: How to deal with the flood of customer demand for services that occurs when you expose your infrastructure to a large, potentially unbounded network. - Or, stated more positively - Opportunity: How to build infrastructure that allows you to scale business and improve margins by exposing your services to a large, potentially unbounded network.4 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

The Internet Scale Problem is Multi-Faceted• Virtually unlimited number of customers• Services exposed to partners’ overuse• Customers’ experience relies on partners’ services• Global and Real-time 5 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Scaling Infrastructure to Meet the Problem • Applications must act on data in real-time – Access, query, and aggregate the data – Trigger business actions based on changes to data • Must scale predictably and cost-effectively – Failure to do so results in loss of revenue and cost overruns – No time to re-architect in Internet time • Need a new paradigm in data management6 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

More. Faster. Cheaper.•Scale to more users and more data•Scale predictably and cost-effectively•Act on data in real-time•Reduce spending on backend systems Get on the Grid7 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Oracle Coherence • Coherence: in-memory data grid End Users – Distributed caching to applications – Scalable extreme transaction processing Application Servers – Real-time eventing, query, and map/reduce aggregations – Abstraction from back-end data Application sources Objects – High-availability to applications Mainframes, Databases, etc.8 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Oracle Fusion Middleware Complete, Open, Integrated, Best-in-Class Web Mobile Social User Engagement Content Management Identity Management Business Intelligence Development Business Process Tools Management Service Integration Enterprise Management Data Integration Cloud Application Foundation9 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Presentation Objectives To Understand… 1) What is driving the need for Distributed Caching and In- Memory Data Grid products? 2) How Coherence improves performance, enables linear scalability, and provides availability for shared services10 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Enterprise Application Scalability ChallengesEase of Scaling Service Clients • Grows and scales Browsers, supplier and partner clients, application naturally clients, mobile apps, and other service consumers. • Increasing load Service Implementation • Computationally Custom applications, BPM processes, service bus intensive work endpoints, UI services, and other service providers. • Highly dependent on shared services Shared Services • Complex and RDBMS, cloud services, supplier and partner services, expensive to scale mainframe applications, and so on. • High latency, SPOB, and SPOF 11 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Traditional Clustering • Dependent on shared service latency Service Clients • Increase in size of cluster leads to increase in load on shared serviceService Service ServiceNode 1 Node 2 Node N • Inconsistent view of data across cluster nodesCached Cached Cached • No possibility for in-memoryObjects Objects Objects analytics • Limited in-memory cache due to • Capacity implications • Java garbage collection Shared Service 12 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Enter the Coherence Data GridEase of Scaling • Clustered Caching Service Clients – Consistent view of data across cluster • Partitioned Caching Service Service Service Node 1 Node 2 Node N – Increase in size of cluster adds capacity but does not decrease performance In-memory Data Grid • Distributed Data Processing – Enables a scalable platform for high performance in- memory analytics Shared Services 13 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

How to Integrate with Coherence Two Options Custom integration through Coherence API - or – Through Existing ‘Switch-On’ OOTB Integrations Oracle Coherence*Web: HTTP session state replication with WebLogic, GlassFish, Others Oracle TopLink Grid: JPA object-relational data Other Oracle Products that integrate Coherence14 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

ActiveCache: Premier WebLogic integration • Distributed HTTP Session State Replication via WebLogic SPI • Configuration, Lifecycle and Monitoring of Coherence Clusters and Servers via WebLogic Admin Console & MBeans • Dependency Injection in JEE modules – Configure in web.xml, use annotations to inject named cache – Alternative to CacheFactory.getCache() lookup15 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Gartner, Magic Quadrant for Enterprise Application ServersThis Magic Quadrant graphic was published by Gartner, Inc. as partof a larger research note and should be evaluated in the context ofthe entire report. The Gartner report is available here:http://www.gartner.com/technology/reprints.do?id=1-17GUO5Z&ct=110928&st=sbSource: Gartner, Magic Quadrant for Enterprise Application Servers – MassimoPezzini, Yefim V. Natis, Kimihiko Iijima, Daniel Sholler, Raffaella Faveta –September 26, 2011 16 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Coherence Basic Inteface Custom Integration NamedCache neCache = CacheFactory• Application uses .getCache("NetworkElement"); Coherence API to … store/access its data NetworkElement ne = new• Application cache NetworkElement(“ID-321”, “OC- 196”, “Nortel Networks); properties are configured neCache.put(“ID-321”, ne); externally through XML … file NetworkElement ne = neCache.get(“ID-321”);• Basic cache access through NamedCache Map interface17 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Management Partitioned Caching• Extreme Scalability: Automatically, dynamically and transparently partitions the data set across the members of the grid.• Pros: – Linear scalability of data capacity – Processing power scales with data capacity. – Fixed cost per data access• Cons: – Cost Per Access: High percentage chance that each data access will go across the wire.• Primary Use: • Large in-memory storage environments • Parallel processing environments 18 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Management Partitioned Fault Tolerance• Automatically, dynamically and transparently manages the fault tolerance of your data.• Backups are guaranteed to be on a separate physical machine as the primary.• Backup responsibilities for one node’s data is shared amongst the other nodes in the grid. 19 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Management Near Caching• Extreme Scalability and Extreme Performance: The best of both worlds between the Replicated and Partitioned topologies. Most recently/frequently used data is stored locally.• Pros: – All of the same Pros as the Partitioned topology plus… – High percentage chance data is local to request.• Cons: – Cost Per Update: There is a cost associated with each update to a piece of data that is stored locally on other nodes.• Primary Use: – Large in-memory storage environments with likelihood of repetitive data access. 20 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Source Integration Read Through / Write Through • Read Through – All data reads occur through cache – If there is a cache miss, the cache will load the data from the data source automatically • Write Through – All data writes occur through cache – Updates to the cache are written synchronously to the data source21 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Source Integration Write Behind • All data writes occur through cache • Updates to the cache are written asynchronously to the data source22 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Processing Parallel Query • Programmatic query mechanism • Queries performed in parallel across the grid • Standard indexes provided out-of-the-box and supports implementing your own custom indexes23 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Processing Continuous Query Cache • Automatically, transparently and dynamically maintains a view locally based on a specific criteria (i.e. Filter) • Same API as all other Coherence caches24 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Processing Invocable Map • The inverse of caching • Sends the processing to where the data is in the grid – Once and only once guarantees – Processing is automatically fault-tolerant • Processing can be: • Targeted to a specific key or collection of keys • Targeted to any object that matches a specific criteria (i.e. Filter)25 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Data Processing Events • Build Complex Event-Driven Apps • Java Bean Model • Key-Based • Filter-Based26 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Presentation Objectives To Understand… 1) What is driving the need for Distributed Caching and In- Memory Data Grid products? 2) How Coherence improves performance, enables linear scalability, and provides availability for shared services 3) Coherence’s Strategic Direction and Latest Features27 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Release Overview • Oracle Coherence 3.7.1 – Released September 23 – Strategic investment in: • Exalogic Innovation • Ease of Use • Integration Points – Significant hardening28 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 © 2011 Oracle Corporation reserved.

Elastic Data Virtual Memory for Your Data • Revolutionizes the scale of data grids • Use block storage for cached data – Optimized for flash Cache Cache Cache – Works well with NAS, Disk, etc. Server 1 Server 2 Server 3 • Simplifies capacity planning and deployment Heap Heap Heap – Configure amount of on-heap data to store – Overflow data written to block storage – Data stored in buffers until flushed to journal – Reduces chances of Out-Of-Memory errors Flash Flash Flash29 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

REST Support Simplified Integration Scalable Distributed Cache to Scalable, Event-Driven Real-Time Applications Java / C++ / .NET REST (Python, PHP,…) Aggregation Query Filter Grid Real-Time Map API (Map-Reduce) API Processing API Client API API Coherence Replication Coherence Cluster Cluster Per Member Data CacheSource Store/Loader API30 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Coherence API Types Scalable Distributed Cache to Scalable, Event-Driven Real-Time Applications Java / C++ / .NET REST (Python, PHP,…) Aggregation Query Filter Grid Real-Time Map API (Map-Reduce) API Processing API Client API API Coherence WAN Coherence Cluster Replication Cluster Per Member Data CacheSource Store/Loader API31 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Query Explain Plan• Evaluate query cost and index effectiveness• Quicker time to market of optimized Coherence solutions• Explain Plan • Provides the estimated cost of evaluating a filter as part of a query operation• Trace • Performs the associated query • Provides the actual cost of evaluating a filter as part of a query operation. 32 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 © 2011 Oracle Corporation reserved.

Live Objects New Coherence Incubator Project• Building blocks for an event- driven finite-state machine – …that is distributed – …and scalable – …and resistant to machine failure• Architectural possibilities: – Stated Event Driven Processing – Ripple Effect 33 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Coherence on Exalogic Engineered System Optimized Scalability and Performance in a Box• Coherence optimized for Exabus 4x Throughput,• Pre-configured network/compute nodes 1/6th Latency!!!• Elastic Data: Expand Capacity with Flash• Easy deployment as demand spikes• Grow from ¼ to multi-rack 34 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Coherence Integrations Oracle IGBU Oracle Golden Gate WebCenter Oracle CGBU Meta-data Services (MDS) SOA BPEL Oracle GlassFish Oracle ADF SOA Human Workflow Oracle Access Manager Oracle PeopleSoft SOA Business Rules Oracle CEP Oracle Secure Token Service SOA Coherence Adapter Real-time Decision Oracle Identity Federation SOA Technology Adapters TopLink ATG BI Publisher WebLogic Portal Oracle Entitlement ServicesService Delivery Platform Oracle BEAM WebLogic Server Oracle Web Services Manager Oracle Service Bus Advanced Capabilities SOA Infrastructure Data Grid ATG Hosted Services Data CacheOracle Data Integration Simple Clustering 35 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

GoldenGate/Coherence Integration Custom Coherence TopLink Grid Update GoldenGate Packaged Enabled Coherence Cohere Cache Updater Java Client Application Application nce• Simplifies Coherence use in shared database environment <entity – Propagates DB updates to Coherence class="Address"> <table name="ADDR”/> <attributes> Coherence – GoldenGate captures changes to <id name="id"> <generated- value /> Database Log Stream Capture </id> database tables Direct Database Updates <version name="ver“/> </attributes> Update </entity> – TopLink maps database changes to Coherence cached objects JPA Metadata Read/Write Through – Availability—2012 – GoldenGate 11gR2 – Available as patch to WebLogic 12.1.1 and 11.1.1.6/10.3.6 – Oracle database only Database 36 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Oracle Enterprise Pack for Eclipse (OEPE) • Coherence Project Configuration – Facets – Library Management – Descriptor Generation • Runtime Configuration – Launch Config Editor – Run/Deploy/Debug from IDE • Configuration Support – tangosol-coherence-override.xml editor – Validation – Context Sensitive Help Integration – Cache Configuration Editor (OEPE 11.1.1.8)37 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 © 2011 Oracle Corporation reserved.

Oracle Enterprise Manager 12C Plan Optimize DEEP PERFORMANCE VISIBILITY & ALERTS-BASED MONITORING Meter & Setup Charge Applications and REAL-TIME JVM DIAGNOSTICS FOR Business Services COHERENCE NODES* Platform as a Service DBaaS MWaaS Build CENTRALIZED CACHE DATA Infrastructure MANAGEMENT Manage as a Service AUTOMATED PROVISIONING Test Monitor Deploy38 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Oracle Enterprise Manager 12C Monitoring & Diagnostics Deep Performance Visibility & Alerts Based Proactive Monitoring • Completely customizable performance views – save different views for different monitoring requirements of the enterprise • Proactive Alerts and notifications based on granular thresholds • Topology view to show associations and dependencies • Log monitoring – generate Alerts based on log patters • Advanced monitoring – push replication, reap sessions, transactional caches, etc Real-time JVM Diagnostics for Coherence Nodes • Real-time threads analysis – find call stack, locks, method local variables, etc • Real-time heap analysis – heap distribution in different spaces (eden, perm-gen, etc), garbage collection • Differential heap analysis – pin point heap leaks by comparing two snapshots39 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Oracle Enterprise Manager 12C Administration & Provisioning Centralized Cache Data Management • Query based data operations • Central UI for key cache operations – view, update, import, export, purge, add/remove indexes • Drastically saves time in cache operations and makes it easy for administrators • Save queries for future references Automated Provisioning • Maintain ‘gold image’ in the software library • Provisioning new cluster or add nodes to an existing cluster • Support Unicast Addrress as well as WKA based clusters40 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

For More Information • General Information: http://coherence.oracle.com • Coherence Training: http://education.oracle.com • Coherence Discussion Forum: http://forums.oracle.com • Coherence User Group on Linkedin • “Oracle Coherence 3.5” by Aleks Seovic • Email: craig.blitz@oracle.com41 Copyright © 2011, Oracle and/or its affiliates. All rights Insert Information Protection Policy Classification from Slide 8 reserved.

Innovations in Data Grid Technology with Oracle Coherence

by Bob Rhubart on Dec 13, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Innovations in Data Grid Technology with Oracle Coherence Presentation Transcript