Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
Big Data – New Reference Architectures for Informati...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
Big Data
New Reference Architectures for Information...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Where’s the New Information?
New Integrations
of Dat...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
What does “New Data” really means?
Any Data,
Any Sou...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
What does “All Data” really means?
Any Data,
Any Sou...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
What does “Any Data” really means?
Any Data,
Any Sou...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
From 3-Tier to N-Tier Architecture
Presentation
Tier...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
What about High Performance Computing?
Does everyone...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
The New Reference Architectures
Emerging Challenges ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Low Latency Systems
Real Time processing for the ma...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Pattern Recognition
Predictive systems
• Act on pat...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Data Science as a Service
Lend the power of science...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Oracle Big Data Reference Architecture
Security and...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Translated into Oracle Product Architecture
Securit...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Translated into Oracle Engineered Systems
Security ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Big Data Appliance
Hadoop Ecosystem for the Enterpr...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.17
Hadoop Oracle Database
Oracle’s Big Data Connectors...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.18
(1) Oracle Data Integrator Application
Adapters for...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.19
(2) Oracle SQL Connector for Hadoop
Accessing HDFS ...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.20
(3) Oracle R Connector for Hadoop
R Analytics lever...
What is ?
• Brings R’s statistical functionality to the Oracle Database
• Eliminates R’s memory constraints
• Allows R to ...
Oracle R Architecture
• Leverages SQL for data prep, analysis and enhanced statistics engine
• R engine runs on database n...
Oracle Data Mining (ODM)
• Data Mining: Insight from discovering relationships
• Knowledge about what happened in the past...
Data Mining – Some Definitions
Supervised Learning
Problem Classification Sample Problem
Classification Predict customer r...
Data Mining – Some Definitions
Unsupervised Learning
Problem Classification Sample Problem
Anomaly
Detection
Identify cust...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
Endeca Information Discovery
Sandbox and Production...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
What is the world doing today
Large Spanish Clothes...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
What is the world doing today
Second Largest Bank i...
© 2012 Oracle Corporation – Proprietary and Confidential
Telco Industry
Deep, Big and Fast
Deep
• SNA*, Find Influencers, ...
© 2012 Oracle Corporation – Proprietary and Confidential
Retail Industry
Marketing, Merchandising and Supply Chain
Marketi...
31 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Oil and Gas Use Cases
Hadoop and Seismic Data Proc...
© 2012 Oracle Corporation – Proprietary and Confidential
Life Sciences / Pharmaceutical
Life Sciences
• DNA Sequencing, Di...
© 2012 Oracle Corporation – Proprietary and Confidential
Wrap Up
 New Challenges and the New Information
 N-Tier, HPC
 ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Thank You
@luigicampos
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
Upcoming SlideShare
Loading in …5
×

Big Data – New Reference Architectures for Information Management Luis Campos Big Data Solutions Lead, Oracle EMEA

3,681 views

Published on

Big Data – New Reference Architectures for Information Management
Luis Campos
Big Data Solutions Lead, Oracle EMEA

Published in: Technology

Big Data – New Reference Architectures for Information Management Luis Campos Big Data Solutions Lead, Oracle EMEA

  1. 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1 Big Data – New Reference Architectures for Information Management Luis Campos Big Data Solutions Lead, Oracle EMEA @luigicampos
  2. 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2 Big Data New Reference Architectures for Information Management AGENDA - The New Information - From 3-Tier to N-Tier Architecture - What about High Performance Computing? - The New Reference Architectures - New Technologies and the role of Oracle Corp. - Challenges of the main industries.
  3. 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3 Where’s the New Information? New Integrations of Data New Sources from Any Data New Analytics on All Data New Orchestrations Any Computing model
  4. 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4 What does “New Data” really means? Any Data, Any Source Absorb All Dimensions of Data = 360º
  5. 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5 What does “All Data” really means? Any Data, Any Source Stop Throwing Data Away = Know More About What’s Going On in your Business
  6. 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6 What does “Any Data” really means? Any Data, Any Source Tap Any Data = New Revenue Streams
  7. 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7 From 3-Tier to N-Tier Architecture Presentation Tier Logic Tier Data Tier • Created partly to split Presentation and Logic Layer • Pushing away Logic from the Data created new challenges • Data would need to be moved around in massive amounts, using a plethora of protocols and caching layers
  8. 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8 What about High Performance Computing? Does everyone have the need for Supercomputing? • HPC: solving extraordinary real life problems with extraordinary computing power • Vertical Computing: Supercomputers • Distributed Computing: Massive Computer Clusters
  9. 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9 The New Reference Architectures Emerging Challenges Call for New Solution Mix • Low Latency Systems • Pattern Recognition • Data Science as a Service
  10. 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10 Low Latency Systems Real Time processing for the masses • Mobile Computing • Critical element in User Experience • Element of responsiveness in any user interface Users don’t need this message anymore: “Your request is being processed...”
  11. 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11 Pattern Recognition Predictive systems • Act on pattern, not re-act • Elements: • Agents (Sensors) • Event Processing engine • Rules Engine • Action Broadcast system • Self Learning mixed with Supervised Learning Input: Lots of Low Density Data Output: Immediate Actions inside a context Examples: Guided navigation, While-you-browse recommendations, manufacturing lines, retail in-store promos
  12. 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12 Data Science as a Service Lend the power of science and technology to everyday problems • Incorporate non-deterministic data When you can’t ask questions outside the function • Generation G: “I need the system to tell them what I want” Enterprise Applications: • Government Intelligence • Enterprise Security • Fraud Detection
  13. 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13 Oracle Big Data Reference Architecture Security and Metadata Source Data Layer External COTS/ERP Processes Enterprise Data Warehouse Data Integration Staging Data Layer Performance Layer Embedded Data Marts Data Quality Strongly Typed Data Weakly Typed Data Information Access BIAbstraction&QueryFederation Alerts, Dashboards, Reporting Services Performance Management Advanced Analysis & Data Science Information Discovery Streaming Sensors Social/Text Foundation Layer Enterprise Data with full history Knowledge Discovery Layer Rapid Dev SandboxData Mining Sandbox
  14. 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14 Translated into Oracle Product Architecture Security and Metadata Source Data Layer External COTS/ERP Processes Streaming Sensors Social/Text Enterprise Data Warehouse Data Integration Staging Data Layer Performance Layer Knowledge Discovery Layer Embedded Data Marts Data Quality Strongly Typed Data Weakly Typed Data Information Access BIAbstraction&QueryFederation Alerts, Dashboards, Reporting Advanced Analysis & Data Science Services Performance Management Information Discovery Foundation Layer Enterprise Data with full history Rapid Dev SandboxData Mining Sandbox Oracle NoSQL Database CDH Oracle BI Foundation Endeca Information Discovery Oracle Big Data Connectors Oracle Database -Advanced Analytics & OLAP - Spatial and Graph - Industry Models Endeca Information Discovery
  15. 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15 Translated into Oracle Engineered Systems Security and Metadata Source Data Layer External COTS/ERP Processes Streaming Sensors Social/Text Enterprise Data Warehouse Data Integration Staging Data Layer Performance Layer Knowledge Discovery Layer Embedded Data Marts Data Quality Strongly Typed Data Weakly Typed Data Information Access BIAbstraction&QueryFederation Alerts, Dashboards, Reporting Advanced Analysis & Data Science Services Performance Management Information Discovery Foundation Layer Enterprise Data with full history Rapid Dev SandboxData Mining Sandbox Oracle Big Data Appliance Oracle Exalytics Big Data Connectors Oracle Exadata Endeca
  16. 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16 Big Data Appliance Hadoop Ecosystem for the Enterprises Oracle Big Data Appliance Cloudera Dist. Hadoop Oracle NoSQL BD Connectors 18 Nodes 648TB, 288 CPUs 12 Nodes (U) 6 Nodes 216TB, 96 CPUs
  17. 17. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.17 Hadoop Oracle Database Oracle’s Big Data Connectors Unlock the power of Hadoop integration Oracle Big Data Connectors
  18. 18. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.18 (1) Oracle Data Integrator Application Adapters for Hadoop Transforms Via MapReduce(HIVE) Loads Activates Oracle Loader for Hadoop Oracle Data Integrator Benefits  Consistent tooling across BI/DW, SOA, Integration and Big Data  Reduce complexities : graphical tooling  Improves productivity Oracle Database Improving Productivity and Efficiency for Big Data
  19. 19. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.19 (2) Oracle SQL Connector for Hadoop Accessing HDFS Data from Oracle Database External TableODCH ODCH ODCH SQL Query HDFS Client HDFS Oracle Database Features Access and analyze data in place on HDFS Query and join data on HDFS with database resident data Load into the database using SQL if required Automatic load balancing to maximize performance Access or load into the database in parallel using external table mechanism
  20. 20. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.20 (3) Oracle R Connector for Hadoop R Analytics leveraging Hadoop and HDFS Linearly Scale a Robust Set of R Algorithms Leverage MapReduce for R Calculations Compute Intensive Parallelism for SimulationsHDFS Hadoop Oracle R Client MAPMAP MAPMAP REDUCE REDUCE
  21. 21. What is ? • Brings R’s statistical functionality to the Oracle Database • Eliminates R’s memory constraints • Allows R to run on very large data sets • Oracle R is architected for enterprise production infrastructure • Automatically exploits database parallelism without requiring parallel R programming • Oracle R leverages the latest R algorithms and packages • R is an embedded component of the DBMS server • Part of Oracle Advanced Analytics (+ODM)
  22. 22. Oracle R Architecture • Leverages SQL for data prep, analysis and enhanced statistics engine • R engine runs on database nodes for production enablement of R models • Leverages Exadata—Oracle R workloads run in-database and can be bound to database nodes for workload isolation • Enriches OBIEE dashboards with Oracle R statistics and analytics Function push-down – data transformation & statistics R workspace console Oracle statistics engine OBIEE, Web Services Development Production Consumption
  23. 23. Oracle Data Mining (ODM) • Data Mining: Insight from discovering relationships • Knowledge about what happened in the past • Characterization, segmentation, comparisons, discrimination • Descriptive models of patterns • Predictive Analytics: Making better decisions and forecasts • Knowledge about what is happening right now and in the future • Classification and prediction of patterns • Rule-and-model driven Data mining can answer questions that cannot be addressed through simple query and reporting techniques.
  24. 24. Data Mining – Some Definitions Supervised Learning Problem Classification Sample Problem Classification Predict customer response to an affinity card program Regression Predict customer’s age Attribute Importance Find the most significant predictors, data preparation A1 A2 A3 A4 A5 A6 A7
  25. 25. Data Mining – Some Definitions Unsupervised Learning Problem Classification Sample Problem Anomaly Detection Identify customer purchasing behavior that is significantly different from the norm Association Rules Find the items that tend to be purchased together and specify their relationship – market basket analysis Clustering Segment demographic data into clusters and rank the probability that an individual will belong to a given cluster Feature Extraction Group the attributes into general characteristics of the customersF1 F2 F3 F4
  26. 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26 Endeca Information Discovery Sandbox and Production mode Endeca Information Discovery Studio Endeca MDEX Server Intergration Suite
  27. 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27 What is the world doing today Large Spanish Clothes Manufacturer • Automation • Sensory Event Processing • Quality Assurance
  28. 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28 What is the world doing today Second Largest Bank in United States of America • Analysis of data xLoB: Loans, Insurance, on-line banking, card products • Market assessment • Risk Analysis • Revenue lift for new & existing products
  29. 29. © 2012 Oracle Corporation – Proprietary and Confidential Telco Industry Deep, Big and Fast Deep • SNA*, Find Influencers, RA** Big • Network Optimization, • CDR Analysis Fast • Sentiment Analysis • Location Based Services • Click stream Analysis * Social Network Analysis (Rate plan optimization) ** Revenue Assurance
  30. 30. © 2012 Oracle Corporation – Proprietary and Confidential Retail Industry Marketing, Merchandising and Supply Chain Marketing • In-store behaviour analysis • Sentiment Analysis + Micro segmentation Merchandising • Assortment optimization Supply Chain • Distribution and logistics optimization • Informing supplier negotiations
  31. 31. 31 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oil and Gas Use Cases Hadoop and Seismic Data Processing
  32. 32. © 2012 Oracle Corporation – Proprietary and Confidential Life Sciences / Pharmaceutical Life Sciences • DNA Sequencing, Diseases Correlation Pharmaceutical • Clinical Trial – meds simulation
  33. 33. © 2012 Oracle Corporation – Proprietary and Confidential Wrap Up  New Challenges and the New Information  N-Tier, HPC  New Reference Architectures for New Data  The role of Oracle Corp in developing New Technologies  Challenges across all industries
  34. 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34 Thank You @luigicampos
  35. 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35

×