Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Value of the Modern Data Architecture with Apache Hadoop and Teradata

9,317 views

Published on

This webinar discusses why Apache Hadoop most typically the technology underpinning "Big Data". How it fits in a modern data architecture and the current landscape of databases and data warehouses that are already in use.

Published in: Technology, Business
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

The Value of the Modern Data Architecture with Apache Hadoop and Teradata

  1. 1. The Value of a Modern Data Architecture with Apache Hadoop and Teradata © Hortonworks Inc. 2013 Page 1
  2. 2. Today’s Topics • Introduction • Drivers for the Modern Data Architecture (MDA) • Apache Hadoop’s role in the MDA • EDW’s role in the MDA • Q&A © Hortonworks Inc. 2013 Page 2
  3. 3. DATA  SOURCES   Sources   DATA  SYSTEMS   Data  Systems   APPLICATIONS   Applica/ons   Existing Data Architecture Packaged     Analy/c  App   RDBMS   EDW   Custom     Analy/c  App   Discovery   PlaEorm   Tradi/onal  Sources     (RDBMS,  OLTP,  OLAP)   © Hortonworks Inc. 2013 Page 3
  4. 4. Big Data Market Trends & Projections 15x 1 Zettabyte (ZB) = 1 Billion TBs growth rate of machine generated data by 2020 The US has 1/3 of the world’s data Big Data is 1 of 5 US GDP Game Changers $325 billion incremental annual GDP from big data analytics in retail and manufacturing by 2020 Big Data Explosion © Hortonworks Inc. 2013 ñ 20% % by which org’s leveraging modern info management systems outperform peers by 2015 Page 4
  5. 5. APPLICATIONS   Traditional Data Architecture Pressured Custom   Applica/ons   Business   Analy/cs   Packaged   Applica/ons   DATA  SYSTEMS   2.8  ZB  in  2012   85%  from  New  Data  Types   RDBMS   EDW   Discovery   PlaEorm   15x  Machine  Data  by  2020   40  ZB  by  2020   DATA  SOURCES   Source: IDC                 Tradi/onal                                          New  Sources       YSTEMS   OLTP,  POS  S(RDBMS,  OLTP,  OLAP)                              (sen/ment,  click,  geo,  sensor,  …)   © Hortonworks Inc. 2013   Page 5
  6. 6. APPLICATIONS   Modern Data Architecture Enabled Custom   Applica/ons   Business   Analy/cs   Packaged   Applica/ons   DEV  &  DATA   TOOLS   DATA  SOURCES   DATA  SYSTEMS   BUILD  &   TEST   OPERATIONAL   TOOLS   RDBMS                 EDW   MANAGE  &   MONITOR   Discovery   PlaEorm     Tradi/onal                                          New  Sources     OLTP,  POS    (RDBMS,  OLTP,  OLAP)                              (sen/ment,  click,  geo,  sensor,  …)   SYSTEMS   © Hortonworks Inc. 2013   Page 6
  7. 7. Today’s Topics • Introduction • Drivers for the Modern Data Architecture (MDA) • Apache Hadoop’s role in the MDA • EDW’s role in the MDA • Q&A © Hortonworks Inc. 2013 Page 7
  8. 8. What Data is Being Stored in Hadoop? 1.  Social Understand how your customers feel about your brand and products – right now 2.  Clickstream Capture and analyze website visitors’ data trails and optimize your website 3.  Sensor/Machine Discover patterns in data streaming automatically from remote sensors and machines 4.  Geolocation Analyze location-based data to manage operations where they occur Value 5.  Server Logs Research logs to diagnose process failures and prevent security breaches 6.  Unstructured (text, video, pictures, etc..) Understand patterns in text across millions of unstructured work products: web pages, emails, video, pictures and documents © Hortonworks Inc. 2013 Page 8
  9. 9. Modern Data Architecture Applied DATA  SOURCES   Sources   DATA  Systems   Data   SYSTEMS   APPLICATIONS   Applica/ons   Shared Data Lake Packaged     Analy/c  App   RDBMS   EDW   Custom     Analy/c  App   Discovery   PlaEorm   Tradi/onal                                        New  Sources                    (RDBMS,  OLTP,  OLAP)                    (sen/ment,  click,  geo,  sensor,  …)   © Hortonworks Inc. 2013 •  Store all data and build/ enable applications on shared “data lake” •  As orgs mature they move to this as a goal for Hadoop Infrastructure  -­‐  Data  Lake   Modern  Data  Architecture   •  Delivers broad value across the enterprise Page 9
  10. 10. Drivers for Hadoop Adoption Modern Data Architecture Hadoop has a central role in next generation data architectures while integrating with existing data systems Driving Efficiency Business Applications Use Hadoop to extract insights that enable new customer value and competitive edge Driving Opportunity Big Data Sets Existing Traditional Server log Clickstream © Hortonworks Inc. 2013 Emerging Sentiment/Social Machine/Sensor Geo-locations Page 10
  11. 11. 3 Requirements for Hadoop Adoption Requirements for Hadoop’s Role in the Modern Data Architecture Integrated Interoperable with existing data center investments Key Services Skills Platform, operational and data services essential for the enterprise Leverage your existing skills: development, operations, analytics © Hortonworks Inc. 2013 Page 11
  12. 12. APPLICATIONS   Interoperating With Your Tools DATA  SOURCES   DATA  SYSTEMS   Microsoft Applications DEV  &  DATA   TOOLS   OPERATIONAL   TOOLS   Viewpoint   Tradi/onal                                          New  Sources                    (RDBMS,  OLTP,  OLAP)                              (sen/ment,  click,  geo,  sensor,  …)     © Hortonworks Inc. 2013 Page 12
  13. 13. Today’s Topics • Introduction • Drivers for the Modern Data Architecture (MDA) • Apache Hadoop’s role in the MDA • EDW’s role in the MDA • Q&A © Hortonworks Inc. 2013 Page 13
  14. 14. Shift from a Single Platform to an Ecosystem "Logical" Data Warehouse “We will abandon the old models based on the desire to implement for high-value analytic applications.” “Big Data requirements are solved by a range of platforms including analytical databases, discovery platforms, and NoSQL solutions beyond Hadoop.” Source: “Big Data Comes of Age”. EMA and 9sight Consulting. Nov 2012. 14 2/28/14 Teradata Confidential
  15. 15. UNIFIED DATA ARCHITECTURE ERP MANAGE MOVE ACCESS Marketing Marketing Executives Applications Operational Systems SCM INTEGRATED DATA WAREHOUSE CRM Images DATA PLATFORM Business Intelligence Audio and Video Data Mining Frontline Workers Customers Partners Engineers Machine Logs DISCOVERY PLATFORM Math and Stats Data Scientists Text Languages Business Analysts Web and Social USERS SOURCES ANALYTIC TOOLS
  16. 16. UNIFIED DATA ARCHITECTURE ERP MANAGE MOVE ACCESS Marketing Marketing Executives Applications Operational Systems SCM INTEGRATED DATA WAREHOUSE CRM Images DATA PLATFORM Business Intelligence Predictive Analytics Business Intelligence Operational Intelligence Audio and Video Machine Logs Text Fast Loading Data Mining Filtering and Processing Online Archival Customers Partners Engineers DISCOVERY PLATFORM Math and Stats Data Scientists Data Discovery Path, graph, time-series analysis Web and Social Frontline Workers Languages Business Analysts Pattern Detection USERS SOURCES ANALYTIC TOOLS
  17. 17. TERADATA UNIFIED DATA ARCHITECTURE ERP MANAGE MOVE ACCESS Marketing Marketing Executives Applications Operational Systems SCM INTEGRATED DATA WAREHOUSE CRM Images DATA PLATFORM Business Intelligence Audio and Video Data Mining Frontline Workers Customers Partners Engineers Machine Logs DISCOVERY PLATFORM Math and Stats Data Scientists Text Languages Business Analysts Web and Social USERS SOURCES ANALYTIC TOOLS
  18. 18. Teradata Appliance for Hadoop Value-Added Software Bringing Hadoop to Enterprise Access: SQL-H™, Teradata Studio Management: Viewpoint, TVI Administration: Hadoop Builder, Intelligent start/stop, DataNode swap, deferred drive replace High Availability : NameNode HA, Master Machine Failover Refining, Metadata, Entity Resolution HCatalog 18 2/28/14 Security & Data Access Kerberos Kerberos Teradata Confidential
  19. 19. Modern Data Architecture Details TVI – Proactive system monitoring tied to Teradata customer support Alerts Viewpoint SOURCE DATA Sensor Log Data System Health Node Health Space Usage Capacity Heatmap Metrics Analysis Query/Visualization/ Reporting/Analytical Tools and Apps DB JDBC/ODBC Compliant Tool KNOX AMBARI File MAPREDUCE Customer/ Inventory Data JMS YARN Clickstream Data REST Flat Files Services HDFS LOAD SQOOP EXTRACT REFINE HIVE FLUME PIG NFS ETL Web HDFS HCATALOG (metadata services) CUSTOM HTTP Sentiment Analysis Streaming Data STRUCTURING © Hortonworks Inc. 2013 EXPORT INTERACTIVE Teradata SQL-H Analytical Platforms Aster Discovery Platform LOAD Teradata IDW SQOOP / HIVE TDCH Page 19
  20. 20. Teradata Vital Infrastructure (TVI) PROACTIVE RELIABILITY, AVAILABILITY, AND MANAGEABILITY 1U server virtualizes system and cabinet management software Server Management VMS •  Cabinet Management Interface Controller (CMIC) •  Service Work Station (SWS) •  Automatically installed on base/first cabinet VMS allows full rack solutions without additional cabinet for traditional SWS Eliminates need for expansion racks, reducing customers’ floor space and energy costs Supports Teradata hardware and Hadoop software TVI Support for Hadoop 62–70% of Incidents Discovered through TVI 20 2/28/14 Teradata Confidential
  21. 21. Standard SQL Access to Hadoop Data Give business users on-the-fly access to data in Hadoop Teradata SQL-H Aster SQL-H Hadoop MR •  Fast: Queries run on Teradata or Aster, data accessed from Hadoop Data •  Standard: 100% ANSI SQL access to Hadoop data Data Filtering •  Trusted: Use existing tools/skills and enable self-service BI with granular security HCatalog Hive Pig •  Efficient: Intelligent data access leveraging the Hadoop HCatalog 21 2/28/14 Teradata Confidential Hadoop Layer: HDFS
  22. 22. Teradata Unified Data Architecture™ Partners Support Many Layers 22 2/28/14 Teradata Confidential
  23. 23. Teradata Aster Discovery Portfolio: Accelerate Time to Insights Some of the 80+ out-of-the-box analytical apps PATH ANALYSIS TEXT ANALYSIS Discover Patterns in Rows of Sequential Data Derive Patterns and Extract Features in Textual Data STATISTICAL ANALYSIS High-Performance Processing of Common Statistical Calculations SEGMENTATION MARKETING ANALYTICS DATA TRANSFORMATION Analyze Customer Interactions to Optimize Marketing Decisions Graph Analysis Graph analytics processing and visualization 23 2/28/14 Teradata Confidential Discover Natural Groupings of Data Points Transform Data for More Advanced Analysis SQL-MapReduce Visualization Graphing and visualization tools linked to key functions of the MapReduce analytics library
  24. 24. More Accurate Customer Churn Prevention Hadoop captures, stores and transforms social, images and call records SOCIAL FEEDS Multi-Structured Raw Data Call Center Voice Records CLICKSTREAM DATA Call Data Hadoop Sentiment Scores Aster Discovery Platform Data Sources ETL Tools 24 2/28/14 Teradata Confidential Analytic Results Traditional Data Flow Capture, Retain and Refine Layer Dimensional Data Check Data eMail Aster does path and pattern analysis Teradata Integrated DW Analysis + Marketing Automation (Customer Retention Campaign)
  25. 25. MPP RDBMS + Hadoop Customer Successes 25 2/28/14 Teradata Confidential
  26. 26. Key Considerations For EDW and Hadoop MPP RDBMS Hadoop Stable Schema Evolving Schema Leverages Structured Data Structure Agnostic ANSI SQL Flexible Programming Iterative Analysis Batch Analysis Fine Grain Security N/A Cleansed Data Raw Data Seeks Scans Updates/Deletes Ingest Service Level Agreements Flexibility Core Data All Data Complex Joins Efficient Use of CPU/IO 26 Complex Processing Low Cost of Storage 2/28/14 Teradata Confidential
  27. 27. Complete Consulting and Training Services Teradata Analytic Architecture Services Services to scope, design, build, operate and maintain an optimal UDA approach for Teradata, Aster, and Hadoop Teradata DI Optimization Assess structured/non-structured data, discuss data loading techniques, determine best platform, optimize load scripts/processes Teradata Big Analytics Assess data value/cost of capture, identify source of “exhaust” data, create conceptual architecture, refine and enrich the data, implement initial analytics in Aster or best-fit tool Teradata Workshop for Hadoop Introduction workshop (across all of UDA) Teradata Data Staging for Hadoop Load data into landing-area; set-up data exploration/refining area; Scope architecture and analytics; set-up Hadoop repository; Load sample data Teradata Platform for Hadoop Installation guidance and mentoring for Hadoop platform, D-I-Y after installation Teradata Managed Services for Hadoop Operations, management, administration, backup, security, process control for Hadoop Teradata Training Courses for Hadoop 27 Areas of Focus Two comprehensive, multi-day training offerings: 1) Administration of Apache Hadoop and 2) Developing Solutions Using Apache Hadoop 2/28/14 Teradata Confidential
  28. 28. Discovering Deep Insights in Retail Transforming Web Walks into DNA Sequences Impact Situation Large retailer with 700M visits/ year, 2M customers / day look at 1M products online Problem Increase ability of web content owners to self-serve insights Solution Treat web walks like DNA sequences of simple patterns. 28 2/28/14 •  Data: loaded logs into Hortonworks •  Loaded 2 months of raw data in 1 hour, vs. 1 day on old system •  Can load a day’s log data in 60 sec •  Sessionize: Creates sequence for visit, e.g., boils 20 customer clicks down to 1 line: •  <Home –Search -Look at Product Add to Basket – Pay – Exit> •  Analyze: Business analysts can now do path analysis •  Act: •  Segmentations by behavior can increase conversion rates by 5-10%. •  Web design changes can drive another 10-20% more visitors into the sales funnel Teradata Confidential
  29. 29. Demo Demo 29 2/28/14 Teradata Confidential

×