Big Data for the Enterprise - What is under the hood

1,656 views

Published on

Oracle Day 2011

Published in: Technology, Business
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,656
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Big Data for the Enterprise - What is under the hood

  1. 1. Big Data – Are You Ready?Michał Jerzy Kostrzewa (Michal.Kostrzewa@Oracle.com)ECE Business Development Manager
  2. 2. The following is intended to outline our general product direction.It is intended for information purposes only, and may not beincorporated into any contract. It is not a commitment to deliverany material, code, or functionality, and should not be relied uponin making purchasing decisions. The development, release, andtiming of any features or functionality described for Oracle’sproducts remains at the sole discretion of Oracle.
  3. 3. Big Data Use Cases Today’s Challenge New Data What’s Possible Healthcare Preventive care, reduced Remote patient monitoring Expensive office visits hospitalization Manufacturing Product sensors Automated diagnosis, support In-person support Location-Based Services Geo-advertising, traffic, local Real time location data Based on home zip code search Public Sector Tailored services, Citizen surveys Standardized services cost reductions Retail Sentiment analysis Social mediaOne size fits all marketing segmentation
  4. 4. What Makes it Big Data? SOCIAL BLOG 101100101001 001001101010 SMART 101011100101 METER 010100100101VOLUME VELOCITY VARIETY VALUE
  5. 5. Why Is Big Data Important?US HEALTH CARE MANUFACTURING GLOBAL PERSONAL EUROPE PUBLIC US RETAIL LOCATION DATA SECTOR ADMINIncrease industry Decrease dev., Increase service Increase industry Increase netvalue per year by assembly costs by provider revenue by value per year by margin by$300 B –50% $100 B €250 B 60+% Source: * McKinsey Global Institute: Big Data – The next frontier for innovation, competition and productivity (May 2011)
  6. 6. Big Data in Action DECIDE ACQUIRE Make Better Decisions Using Big DataANALYZE ORGANIZE
  7. 7. Big Data in Action DECIDE ACQUIRE Acquire all available dataANALYZE ORGANIZE
  8. 8. Acquiring Big Data ChallengeNeed to process Application will need Must scale out to high volume, low- to change meet aggressivedensity information frequently roll out plan
  9. 9. Oracle NoSQL Database Application Application Key value pair database NoSQL Driver NoSQL Driver Dynamic data model Update Delete Read Read Highly scalable, available Transparent load balancing Built using BerkeleyDBNodes Nodes Nodes Nodes Nodes… West Central East …
  10. 10. Big Data in Action DECIDE ACQUIRE Oracle NoSQL DatabaseANALYZE ORGANIZE
  11. 11. Big Data in Action DECIDE ACQUIRE Organize and distill big data using massive parallelismANALYZE ORGANIZE
  12. 12. Organizing Big Data ChallengeHave existing Also want to Can’t negatively Oracle data perform analysis impact data warehouse on big data warehouse SLAs
  13. 13. Analysis Sandbox Provides analysis workspace Controlled access to resources and data Doesn’t impact production system
  14. 14. Sandboxing with Oracle Enterprise Manager Simple to set up Efficient server utilization Secure and scalable Accountable via charge back Ideal for Oracle Exadata
  15. 15. Organizing and Distilling Big Data ChallengeMust transform big Want to avoid Need to load datadata into something writing lots of quickly into Oracle easily analyzed Hadoop code Data Warehouse
  16. 16. Hadoop Architecture Management/Monitoring Distributed file system with redundant storage Map/Reduce programming paradigm MapReduce Highly scalable data processing Cost-effective model for high Hadoop Distributed File System (HDFS) volume, low density data
  17. 17. A Map/Reduce PipelineINPUT OUTPUT 1 1 MAP MAP MAP REDUCE REDUCE MAP REDUCE MAP REDUCE MAP MAP SHUFFLE REDUCE MAP REDUCE REDUCE MAP /SORT SHUFFLE MAP /SORT MAP MAP MAP REDUCE MAP REDUCE MAP REDUCE MAP REDUCE MAP REDUCE MAP SHUFFLE SHUFFLE MAP /SORT REDUCE MAP /SORT SHUFFLE /SORTINPUT OUTPUT 2 2
  18. 18. Oracle Loader for Hadoop Reduces Hadoop complexities throughINPUT 1 MAP MAP graphical tooling (ODI) MAP REDUCE REDUCE MAP MAP REDUCE MAP REDUCE MAP MAP REDUCE REDUCE SHUFFLE REDUCE SHUFFLE MAP /SORT MAP /SORT MAP MAP MAP REDUCE MAP REDUCE MAP REDUCE MAP REDUCE MAP MAP REDUCE SHUFFLE SHUFFLE MAP /SORT MAP /SORT REDUCE SHUFFLE /SORTINPUT 2
  19. 19. Big Data in Action DECIDE ACQUIRE Oracle NoSQL Database Oracle Enterprise Manager Oracle Data Integrator Oracle Loader for HadoopANALYZE ORGANIZE
  20. 20. Big Data in Action DECIDE ACQUIRE Analyze all your data, at onceANALYZE ORGANIZE
  21. 21. Analyzing Big Data Challenge Want to perform Doing analysis on aRequire access to statistical analysis laptop is slow and all data using R not secure
  22. 22. R Statistical Programming Language Open source language and environment Used for statistical computing and graphics Strength in easily producing publication-quality graphs Highly extensible
  23. 23. Oracle R Enterprise Approach Models run in-database Processes large data sets Uses the power of Oracle Database 11g and Exadata Same code, much faster
  24. 24. Big Data in Action DECIDE ACQUIRE Oracle NoSQL Database Oracle Enterprise Manager Oracle Data Integrator Oracle Loader for Hadoop Oracle R EnterpriseANALYZE ORGANIZE
  25. 25. Big Data in Action DECIDE ACQUIRE Decide based on real-time big dataANALYZE ORGANIZE
  26. 26. Making Decisions Based on Big Data ChallengeBig data has been Want to add new How do we quickly transformed into insights into BI integrate R analytics actionable insight dashboard into dashboard?
  27. 27. Dashboard Analytics• Oracle Business Intelligence Enterprise Edition ‒Advanced dashboard visualization ‒Runs BI and EPM applications• Integrating R Analytics ‒Embed R script’s web interface in BI dashboard ‒Graphics will stream to BI dashboard
  28. 28. Oracle Integrated Solution Stack for Big Data HDFS Hadoop (MapReduce) In-Database AnalyticsOracle NoSQL Oracle Loader Data Analytic Database for Hadoop Warehouse Applications Enterprise Oracle DataApplications Integrator ACQUIRE ORGANIZE ANALYZE DECIDE
  29. 29. Oracle Integrated Solution Stack Engineered SystemsACQUIRE ORGANIZE ANALYZE DECIDE
  30. 30. Oracle Engineered Systems Big Data Appliance Hardware• 18 Sun X4270 M2 Servers –48 GB memory per node = 864 GB memory –12 Intel cores per node = 216 cores –24 TB storage per node = 432 TB storage• 40 Gb p/sec InfiniBand• 10 Gb p/sec Ethernet
  31. 31. Oracle Big Data Appliance Software • Oracle Linux • Java Hotspot VM • Apache Hadoop Distribution • R Distribution • Oracle NoSQL Database • Oracle Data Integrator for Hadoop • Oracle Loader for Hadoop
  32. 32. Oracle Exalytics HardwareEngineered for extreme analytics• 40 Intel processor cores• 1 Terabyte main memory• 40 Gb InfiniBand connection to Oracle Exadata
  33. 33. Oracle Exalytics Software• Oracle TimesTen In-Memory Database ‒Adaptive in-memory caching of analytics ‒In-memory columnar compression ‒Tightly integrated with Oracle Exadata ‒Enables speed-of-thought visualization• Oracle Business Intelligence Foundation Suite
  34. 34. Maximizing the Value of Enterprise Big Data•Hardware and software for Big Data•Integrates all enterprise data –Structured and unstructured –SQL and NoSQL•Fastest time-to-market•Single vendor support
  35. 35. The preceding is intended to outline our general productdirection. It is intended for information purposes only, and maynot be incorporated into any contract. It is not a commitment todeliver any material, code, or functionality, and should not berelied upon in making purchasing decisions. The development,release, and timing of any features or functionality described forOracle’s products remains at the sole discretion of Oracle.

×