Hadoop's Role in Enterprise Architecture

7,479 views

Published on

With the rise of Apache Hadoop, a next-generation enterprise data architecture is emerging that connects the systems powering business transactions and business intelligence. Hadoop is uniquely capable of storing, aggregating, and refining multi-structured data sources into formats that fuel new business insights. Organizations that embrace solution architectures focused on maximizing the value from ALL data will put themselves in a position to drive more business, enhance productivity, or discover new and lucrative business opportunities. Over the coming years, Hadoop could be in a position to process more than half the world’s data. There is still much work to be done, however, if Hadoop is to achieve this lofty goal. In this talk Shaun Connolly, VP Corporate Strategy for Hortonworks, will look at Hadoop’s role in the enterprise architecture and how it compliments existing enterprise systems.

Published in: Technology

Hadoop's Role in Enterprise Architecture

  1. 1. Hadoop’s Role in theEnterprise ArchitectureShaun ConnollyHortonworks VP Strategy@shaunconnolly
  2. 2. What is Big Data? What is Big Data?
  3. 3. Transactions InteractionsObservations
  4. 4. What is Big Data? Transactions + InteractionsPetabytes BIG DATA Mobile Web + Observations Sentiment SMS/MMS User Click Stream = BIG DATA Speech to Text Social Interactions & Feeds Terabytes WEB Web logs Spatial & GPS Coordinates A/B testing Sensors / RFID / Devices Behavioral Targeting Gigabytes CRM Business Data Feeds Dynamic Pricing Segmentation External Demographics Search Marketing Customer Touches User Generated Content ERP Megabytes Affiliate Networks Purchase detail Support Contacts HD Video, Audio, Images Dynamic Funnels Purchase record Offer details Offer history Product/Service Logs Payment record Increasing Data Variety and Complexity
  5. 5. Big Data Market Drivers Business1 Enable new business models & drive faster growth (20%+)2 Find insights for competitive advantage & optimal returns Technical3 Data continues to grow exponentially4 Data is increasingly everywhere and in many formats5 Traditional solutions not designed for new requirements Financial6 Cost of data systems, as % of IT spend, continues to grow7 Cost advantages of commodity hardware & open source
  6. 6. Is This Your Big Data Strategy?BIG DATA you
  7. 7. Next-Generation Data Architecture Unstructured Business CRM, ERP Data Transactions Web, Mobile & Interactions Point of sale Log files Enterprise Hadoop Exhaust Data Platform Classic Data Integration & ETL Social Media Sensors, devices Business Dashboards, Intelligence Reports, & Analytics Visualization, … DB data1 Capture Big Data 2 Process & Structure 3 Distribute Results 4 Feedback & Retain
  8. 8. Making Hadoop Enterprise Ready OPERATIONAL DATA SERVICES SERVICES Manage & Store, Operate at Process and Scale Access Data Distributed HADOOP CORE Storage & Processing Enterprise Readiness: HA, PLATFORM SERVICES DR, Snapshots, Security, … ENTERPRISE HADOOP PLATFORM OS / VM Cloud Appliance
  9. 9. Existing Data ArchitectureAPPLICATIONS Business Custom Enterprise Analytics Applications Applications DEV & DATA TOOLS BUILD & TESTDATA SYSTEMS OPERATIONAL TOOLS MANAGE & RDBMS EDW MP MONITOR TRADITIONAL REPOS PDATA SOURCES Traditional Sources OLTP,(RDBMS, OLTP, OLAP) POS SYSTEMS
  10. 10. An Emerging Data ArchitectureAPPLICATIONS Business Custom Enterprise Analytics Applications Applications DEV & DATA TOOLS BUILD & TESTDATA SYSTEMS OPERATIONAL TOOLS ENTERPRISE MANAGE & HADOOP PLATFORM MONITOR RDBMS EDW MP TRADITIONAL REPOS PDATA SOURCES Traditional Sources New Sources OLTP,(RDBMS, OLTP, OLAP) (web logs, email, sensors, social media) MOBILE POS DATA SYSTEMS
  11. 11. [Integrating Hadoop withexisting IT investments isvitally important.] Larry Feinsmith
  12. 12. Interoperating With Your ToolsAPPLICATIONS Microsoft Applications DEV & DATA TOOLSDATA SYSTEMS OPERATIONAL TOOLS ENTERPRISE HADOOP PLATFORM TRADITIONAL REPOS ViewpointDATA SOURCES Traditional Sources New Sources OLTP,(RDBMS, OLTP, OLAP) (web logs, email, sensors, social media) MOBILE POS DATA SYSTEMS
  13. 13. Big Data Tag Team!Your EnterpriseTools Hadoop
  14. 14. Hadoop Common Patterns of Use Business Cases “Right-time” Access to Data Batch Interactive Online Refine Explore Enrich ENTERPRISE HADOOP PLATFORM Big Data Transactions, Interactions, Observations
  15. 15. Operational Data Refinery Enric Refine Explore hAPPLICATIONS Business Custom Enterprise Transform & refine ALL Analytics Applications Applications sources of data Also known as Data Reservoir or Catch Basin 3DATA SYSTEMS ENTERPRISE HADOOP 2 1 Capture RDBMS EDW MPP PLATFORM TRADITIONAL REPOS 2 Process 1DATA SOURCES Traditional Sources New Sources 3 Distribute & Retain (RDBMS, OLTP, OLAP) (web logs, email, sensor data, social media)
  16. 16. Big Data Exploration & Visualization Refine Explore EnrichAPPLICATIONS Business Custom Enterprise Leverage “data lake” Analytics Applications Applications to perform iterative investigation for value 3DATA SYSTEMS ENTERPRISE HADOOP 2 1 Capture RDBMS EDW MPP PLATFORM TRADITIONAL REPOS 2 Process 1DATA SOURCES Traditional Sources New Sources 3 Explore & Visualize (RDBMS, OLTP, OLAP) (web logs, email, sensor data, social media)
  17. 17. Application Enrichment Refine Explore EnrichAPPLICATIONS Custom Enterprise Create intelligent Applications Applications applications 3 Collect data, create analytical models and deliver to online appsDATA SYSTEMS ENTERPRISE HADOOP 2 1 Capture RDBMS EDW MPP NOSQL PLATFORM TRADITIONAL REPOS 2 Process & Compute 1DATA SOURCES Traditional Sources New Sources 3 Deliver Model (RDBMS, OLTP, OLAP) (web logs, email, sensor data, social media)
  18. 18. Big Data: Optimize Outcomes at Scale Media o p ti m i z e Content Intelligence o p ti m i z e Detection Finance o p ti m i z e Algorithms Advertising o p ti m i z e Performance Fraud o p ti m i z e Prevention Retail / Wholesale o p ti m i z e Inventory turns Manufacturing o p ti m i z e Supply chains Healthcare o p ti m i z e Patient outcomes Education o p ti m i z e Learning outcomes Government o p ti m i z e Citizen services Source: Geoffrey Moore. Hadoop Summit 2012 keynote presentation.
  19. 19. Market Transitioning into Early Majority relative %customers The CHASM Innovators, Early Early Late majority, Laggards, technology adopters, majority, conservatives Skeptics enthusiasts visionaries pragmatists time Customers want Customers want technology & performance solutions & convenience Source: Geoffrey Moore - Crossing the Chasm
  20. 20. At Hortonworks, we believe that by the end of 2015, more than half the worlds data will be processed by Apache Hadoop. Welcome to Hadoop Summit and Enjoy the Conference!

×