Downloaded 58 times
Uploaded byHortonworks
Delivering a Flexible IT Infrastructure for Analytics on IBM Power Systems
The document discusses the partnership between Hortonworks and IBM to provide the Hortonworks Data Platform (HDP) on IBM Power Systems, emphasizing its capabilities in big data processing and customer analytics. It highlights the advantages of the open-source technology and hardware solutions, such as reduced costs and improved performance in analytics and server management. Additionally, it showcases customer success stories, particularly in healthcare, demonstrating the operational benefits of adopting this flexible IT infrastructure.
More Related Content
Similar to Delivering a Flexible IT Infrastructure for Analytics on IBM Power Systems
Delivering a Flexible IT Infrastructure for Analytics on IBM Power Systems
- 1. Delivering a FlexibleIT Infrastructure for Analytics with HDP on IBM Power Systems Nov 30, 2016
- 2. Agenda • Hortonworks andIBM Power Systems Partnership • Customer Analytics Journey • Open Community Innovation • Leading Time to Insights 2
- 3. Hortonworks, IBM Collaborateto Offer Open Source Distribution on Power Systems Latest Hortonworks Data Platform (HDP) to provide IBM customers with more choice in open source Hadoop distribution for big data processing 3 Las Vegas, NV (IBM Edge) - 19 Sep 2016: IBM (NYSE: IBM) and Hortonworks (NASDAQ: HDP) today announced the planned availability of Hortonworks Data Platform (HDP®) for IBM Power Systems enabling POWER8 clients to support a broad range of new applications while enriching existing ones with additional data sources. Scott Gnau, CTO, Hortonworks at Edge. Youtube: http://bit.ly/2dSOliW James Wade, Director of Application Hosting, Florida Blue. Youtube: http://bit.ly/2dxVHIY © 2016 IBM Corporation
- 4. Customer Story –Guidewell Health - Florida Blue • Business Problem – Transformational journey resulting in rapid expansion of business models – Technology innovation required to keep up with the business expansion while improving client satisfaction, reducing costs and supporting the company’s green IT initiatives o Existing x86 server sprawl not sustainable • Solution with Hortonworks, IBM OpenPOWER servers and Sage Solutions Consulting – Embraces the open software and hardware model adopted by Florida Blue – Hortonworks supporting new fraud analytics initiative to reduce costs and client premiums – OpenPOWER to enable smaller datacenter footprint with stronger reliability 4 See the full story in this Hortonworks Blog post. © 2016 IBM Corporation
- 5. 5 © Hortonworks Inc.2011 – 2016. All Rights Reserved Payment Tracking Due Diligence Social Mapping Product Design M & A Call Analysis Machine Data Defect Detecting Factory Yields Customer Support Basket Analysis Segments Customer Retention Sentiment Analysis Optimize Inventories Supply Chain Cross- Sell Vendor Scorecards Ad Placement Cyber Security Disaster Mitigation Investment Planning Ad Placement Risk Modeling Proactive Repair Inventory Predictions Next Product Recs OPEX Reduction Historical Records Mainframe Offloads Device Data Ingest Rapid Reporting Digital Protection Data as a Service Fraud Prevention Public Data Capture INNOVATE RENOVATE EXPLORE OPTIMIZE TRANSFORM ACTIVE ARCHIVE ETL ONBOARD DATA ENRICHMENT DATA DISCOVERY SINGLE VIEW PREDICTIVE ANALYTICS M&A Storage Blending M&A Ingest Integration
- 6. About Hortonworks The Leaderin Connected Data Platforms Publicly traded on NASDAQ: HDP Hortonworks DataFlow for data in motion Hortonworks Data Platform for data at rest Powering new modern data applications Partnering for Customer Success Leader in open-source community, focused on innovation to meet enterprise needs Unrivaled support subscriptions © Hortonworks Inc. 2011 – 2016. All Rights Reserved Founded in 2011 Original 24 Architects, Developers, Operators of Hadoop from Yahoo! 1000+ Employees 1500+ Ecosystem Partners
- 7. HPD is a100% Open Source Connected Data Platform Eliminates Risk of vendor lock-in by delivering 100% Apache open source technology Maximizes Community Innovation with hundreds of developers across hundreds of companies Integrates Seamlessly through committed co-engineering partnerships with other leading technologies © Hortonworks Inc. 2011 – 2016. All Rights Reserved
- 8. Apache Hadoop Committers WeEmploy the Committers one third of all committers to the Apache® Hadoop™ project, and a majority in other important projects Our Committers Innovate and expand Open Enterprise Hadoop We Influence the Hadoop Roadmap by communicating important requirements to the community through our leaders Hortonworks Influences the Apache Community
- 9. Hortonworks Nourishes theCommunity and Ecosystem Hortonworks Community Connection Hortonworks Partnerworks • Community Q/A Resources • Articles & Code Repos! • Community of (big data) developers • Open Ecosystem of Big Data for vendors & end-users • Advance Apache™ Hadoop® • Enable more Big Data Apps • World class partner program • Network of partners providing best-in-class solutions Hadoop & Big data ecosystem © Hortonworks Inc. 2011 – 2016. All Rights Reserved
- 10. Hortonworks Delivers ProactiveSupport Hortonworks SmartSense™ with machine learning and predictive analytics on your cluster Integrated Customer Portal with knowledge base and on-demand training © Hortonworks Inc. 2011 – 2016. All Rights Reserved
- 11. Client Value propositionfor HDP on Power: • 100% Open Source Hadoop running on OpenPOWER Hardware Platform – OpenPOWER is open-source hardware solution for open source SW – Intel x86 – perceived as default/commodity option - but x86 is not open – Open = no vendor lock-in and flexibility • Processor and Server Architecture optimized for big data processing – 2x per core performance compared to intel x86 o Fewer cores & servers needed: contain server sprawl o Improved HW price/performance – Higher server & data reliability – designed to run the enterprise 11
- 12. 5 IBMers contributingto Linux and Apache Projects 1999 IBM is investing in the Linux ecosystems & open innovation 270+ OpenPOWER-based innovations under way 2016 50k+ IBMers contributing to 150+ open organizations 1. Source: https://developer.ibm.com/start/ 1 Blockchain Hyperledger Open Source Databases 12
- 13. 13 13 Accelerated innovationthrough collaboration of partners Accelerated innovation through collaboration of partners Amplified capabilities driving industry performance leadership Amplified capabilities driving industry performance leadership Vibrant ecosystem through open development Vibrant ecosystem through open development Cloud Computing Hyperscale & Large scale Datacenters High Performance Computing & Analytics Domestic IT Agendas Industry adoption, Open choice OpenPOWER Strategy Moore’s law no longer satisfies performance gain Numerous IT consumption models Growing workload demands Mature Open software ecosystem Market Shifts OpenPOWER is an open development community, using the POWER Architecture to serve the evolving needs of customers. OpenPOWER, a catalyst for Open Innovation
- 14. Fueling an OpenDevelopment Community 14 Chip / SOC Boards Systems I/O / Storage Acceleration System / Software Integration Implementation / HPC / Research
- 15. - Design andcost optimized for deployments of multiples (cloud and cluster) - Broad number of optimal solutions - Co-Designed with the OpenPOWER Ecosystem Supported by Canonical IBM Support Community / 3rd Party Support running The LC Line The L Line PurePower Enterprise & IFLs IaaS Scale-Out, Linux-Only Converged Infrastructure Scale-Up - Enterprise level RAS for single system deployments - Solutions for Big Data & Analytics - Converged infrastructure offering - Rapid time to value and simplicity of management - Enterprise level robustness and IFL capability - Solution editions for in memory databases - (HANA, DB2 BLU) - Hosted cloud and hybrid cloud solutions - Rapid deployments and POCs The IBM Power Systems Linux Portfolio Pipeline of innovation Broad Linux portfolio delivers all your Linux deployment needs POWER8 is designed for the Big Data era and delivers price-performance leadership to the Linux Market! 155 © 2016 IBM Corporation
- 16. Introducing the IBMPower Systems LC Line S822LC For High Performance Computing •Incorporates the new POWER8 processor with NVIDIA NVLink •Delivers 2.8X the bandwidth to GPUs accelerators •Up to 4 integrated NVIDIA “Pascal” GPUs S822LC For Big Data •Ideal for storage-centric and high data through- put workloads •Brings 2 POWER8 sockets for Big Data workloads •Big data acceleration with work CAPI and GPUs S821LC •Storage rich single socket system for big data applications •Memory Intensive workloads S822LC •2X memory bandwidth of Intel x86 systems •Memory Intensive workloads S812LC •2 POWER8 sockets in a 1U form factor •Ideal for environments requiring dense computing NEW NEW NEWBig Data High Performance Computing ComputeIntensive Announce 9/8, GA 9/26 Announce and GA 9/8 Announce and GA 9/8 OpenPOWER servers for cloud and cluster deployments that are different by design
- 17. Innovation Pervasive inthe Design Power Systems S822LC for Big Data NVIDIA: Tesla K80 GPU Accelerator Red Hat, Ubuntu, SUSE: Linux OS Mellanox: InfiniBand/Ethernet Connectivity in and out of server HGST: Optional NVMe Adapters Alpha Data with Xilinx FPGA: Optional CAPI Accelerator Broadcom: Optional PCIe Adapters QLogic: Optional Fiber Channel PCIe Samsung: SSDs & NVMe Hynix, Samsung, Micron: DDR4 IBM: POWER8 CPU 17
- 18. Leading Operational DBMSsAvailable & Optimized for Linux on Power In-Memory, NoSQL SAP HANA MongoDB Neo4j EnterpriseDB MariaDB Open Source IBM DB2 BLU RedisLabs Cassandra PostgreSQL 18
- 19. 19 IBM is yoursingle, trusted vendor to support and help you manage your Linux infrastructure 1 Based on IBM internal data. 2 Original equipment manufacturer
- 20. Price/Performance Moore’s Law Processor Technology 2000 2020 Firmware/ OS Accelerators Software Storage Network 20 Data holds competitive valueFull system and stack open innovation required You are here 44 zettabytes unstructured data 2010 2020 structured data DataGrowth Today’s challenges demand innovation
- 21. 4X Threads per core 4X Mem.Bandwidth1 6X More cache2 @ Lower Latency SMT=Simultaneous Multi-Threading OLTP = On-Line Transaction Processing These design decisions result in best performance for data centric workloads like: Spark, Hadoop, Database, NoSQL, Big Data Analytics, OLTP POWER8: Designed for data to deliver breakthrough performance POWER8 SMT8 x86 Hyperthread Parallel Processing POWER8 pipe Data flow x86 pipe POWER8 x86 POWER8 + OpenPOWER x86 21 1. Up to 4X depending on specific x86 and POWER8 servers being compared 2. Up to 6X more cache comparing Intel e7-8890 servers to 12 core POWER8 servers. See speaker notes for more details © 2016 IBM Corporation
- 22. SAP S&D industrybenchmark demonstrates that x86 technologies have NOT improved performance per core Source: http://global.sap.com/solutions/benchmark/sd2tier.epx Certification # 2016023 Date: 05/10/2016 Fujitsu PRIMEQUEST 2800E3 192 Cores & 384 Threads E7-8890 v4 at 2.20 GHz 2048 GB Memory Users: 74,000, SAPS: 404,200 Windows Server 2012 R2 SQL Server 2012 Sandy Bridge EP Ivy Bridge EP Ivy Bridge EX Haswell EP Haswell EX Broadwell EP Broadwell EX Certification # 2014034 Date: 10/3/2014 IBM Power E870 80 Cores & 640 Threads POWER8 at 4.19 GHz 2 TB Memory Users: 79,750, SAPS: 436,100 AIX 7.1 DB2 10.5 POWER8 2.58x Certification # 2013039 Date: 12/10/2013 Cisco UCS C420 M3 32 Cores & 64 Threads E5-4650 at 2.70 GHz 256 GB Memory Users: 13,010, SAPS: 71,170 Windows Server 2012 SQL Server 2012 Certification # 2014017 Date: 05/05/2014 Dell PowerEdge R720 24 Cores & 48 Threads E5-2697 v2 at 2.70 GHz 256 GB Memory Users: 10,253, SAPS: 55,970 RHEL 6.5 SAP ASE 16 Certification # 2014018 Date: 05/05/2014 Cisco UCS B260 M4 30 Cores & 60 threads E7-4890 v2 at 2.80 GHz 512 GB Memory Users: 12,280, SAPS: 67,020 Windows Server 2012 SQL Server 2012 Certification # 2014033 Date: 09/10/2014 Dell PowerEdge R730 36 Cores & 72 threads E5-2699 v3 at 2.3 Ghz 256 GB Memory Users: 16,500, SAPS: 90,120 RHEL 7.0 SAP ASE 16 Certification # 2015012 Date: 05/05/2015 Dell PowerEdge R930 72 Cores & 144 threads E7-8890 v3 at 2.5 GHz 1 TB Memory Users: 31,000, SAPS: 170,030 RHEL 7.1 SAP ASE 16 Certification # 2016006 Date: 03/31/2016 Cisco UCS C240 M4 44 Cores & 88 Threads E5-2699 v4 at 2.20 GHz 512 GB Memory Users: 21,210, SAPS: 115,820 Windows Server 2012 R2 DB2 10.1
- 23. RelativeRPE2**PerCore POWER6=1.0 **Gartner RPE2 Details: http://www.gartner.com/technology/research/RPE2-methodology-details.jsp RPE2**numbers are derived from the following six benchmark inputs: SAP SD Two-Tier, TPC-C, TPC-H, SPECjbb2006 and two SPEC CPU2006 components The data on this chart is derived from RPE2 from Gartner, Inc.'s Competitive Profile tool. © 2014 Gartner, Inc. and/or its affiliates. All rights reserved. This was a POWER8 Design Goal POWER7 3.55 GHz 16 cores POWER7+ 4.2 GHz 16 cores POWER8 3.52 GHz 24 cores POWER6 4.2 GHz 16 cores POWER Performance per core is increasing! Increasing Performance The servers shown are best in each category (sockets and number of cores) 1.0
- 24. Example: OpenPOWER accelerationwith NVLink Current CPU to GPU PCIe Attachment System Bottleneck Graphics Memory New POWER8 CPU New POWER8 CPU P100 Tesla GPU P100 Tesla GPU NVLink NVLink NVLink 115GB/s 80 GB/s 80GB/s 80GB/s New POWER8 with NVLink Processor Technology CPUCPU GPUGPU PCIe 32GB/s Graphics Memory Graphics Memory POWER8 with NVLink delivers 2.8X the bandwidth PCIe Data Pipe POWER8 NVLink Data Pipe P100 Tesla GPUz P100 Tesla GPUz © 2016 IBM Corporation
- 25. YCSB running MongoDBon POWER8 delivers leadership performance and 2.2X better price-performance than Intel Xeon E5-2690 v3 Haswell IBM Power S822LC (16-core, 256GB) HP DL380 Gen9 (24-core, 256GB) Server web price* -3-year warranty $16,295 $24,615 System Cost -Server + RHEL OS + MongoDB Annual Subscription $29,584 ($16,295 + $1,299 + $11,990) $37,904 ($24,615 + $1,299 + $11,990) MongoDB YCSB (total operations per second) 297.5 k ops 169.5 k ops $ / Op per Sec 100 $ / ops 223 $ / ops 2.2X better 2.2X Better Price-Performance 33% Lower HW costs and maintenance 75% More Performance per Server @ •Based on IBM internal testing of single system and OS image running Yahoo Cloud Services Benchmark (YCSB) 0.6.0, 1M record workload at 50/50 read/write factor. Conducted under laboratory condition, individual result can vary based on workload size, use of storage subsystems & other conditions. • IBM Power System S822LC; 16 cores (2 x 8c chips) / 128 threads, POWER8; 3.3 GHz, 256 GB memory, MongoDB 3.3.4, RHEL 7.2. Competitive stack: HP Proliant DL380 Gen9; 24 cores (2 x 12c chips) / 48 threads; Intel E5-2690 v3; 2.6 GHz; 256 GB memory, MongoDB 3.3, RHEL 7.2 . Both server priced with 2 x 1TB SATA 7.2K rpm HDD, 1 Gb 2-port, 2 x 16gbps FCA. Configurations represent the highest processor frequency for that specific processor running the MongoDB server on 1 socket & the YCSB application workload on the 2nd socket. RAM disk was used to focus testing on processor technology differences. Pricing is based on web pricing for S822LC http://www-03.ibm.com/systems/power/hardware/s822lc-commercial/buy.html and HP DL380 Gen9 https://h22174.www2.hp.com/SimplifiedConfig/Index MongoDB https://www.mongodb.com/compare/mongodb-oracle Page: 6
- 26. POWER Advantages forSpark • Streaming and SQL benefit from High Thread Density and Concurrency • Machine Learning benefits from Large Caches and Memory Bandwidth • Graph also benefits from Large Caches, Memory Bandwidth and Higher Computational Strength 26 Machine Learning SQL Graph 2X Core-to-Core Advantage Machine Learning SQL Graph 1.5X Price Performance Advantage 7-Node S812LC 10-core vs. 7-Node E5-2690 v3 12-core © 2016 IBM Corporation
- 27. Future Proof YourHadoop Infrastructure • Total Cost of Ownership benefits of a Linux on Power decision – Less infrastructure means reduced costs in many areas: o Energy, cooling, server administration, floor space, SW licensing – Position for future growth, avoid hitting the data center wall with cluster sprawl – As your workloads evolve, POWER8 gives you options: o Scale up each node by exploiting the memory bandwidth and multi-threading o Add new workload optimized servers to the cluster (such a GPU with NVLink) 27
- 28. HDP and OpenPOWER– Better Together • Leading Open Hadoop and Spark distribution on the leading Open Server platform – built for Big Data, driving continuous innovation in the community • Industry Experience to Lead Clients on an Analytics Journey – Discovery workshops to identify the key business use cases and the client progression • Mission Critical Support – Stable, trusted Hadoop platform on proven Power System with outstanding client support • Performance optimized for enterprise scale – Price performance leadership with POWER8 – Exploiting POWER8 advantages for key workloads such as SQL and Spark – Future exploitation of accelerators to maintain leadership 28© 2016 IBM Corporation
- 29. 29 Hortonworks (HDP) onPower Roadmap 3Q2016 Ramp Up Program HDP early code on Power Head start on your Analytics Journey IBM and Hortonworks assistance for non-prod POCs Ramp Up Program HDP early code on Power Head start on your Analytics Journey IBM and Hortonworks assistance for non-prod POCs General Availability HDP on Power Proven Power Reference Configs Hortonworks and IBM production Support General Availability HDP on Power Proven Power Reference Configs Hortonworks and IBM production Support General Availability HDP next on Power Simultaneous releases General Availability HDP next on Power Simultaneous releases 4Q2016 Q1 2017 later 2017 Partnership Announce Edge Ann Sept 19 Partnership Announce Edge Ann Sept 19 Note: Timing and content subject to change. © 2016 IBM Corporation
- 30. How to GetStarted with HDP on OpenPOWER Systems • Join the Hortonworks Community: https://community.hortonworks.com/ • Learn more about the benefits of Hortonworks: http://hortonworks.com/training/ • Learn more about the benefits of IBM Power Systems and OpenPOWER • If you are interested in discussing a HDP on Power Systems option or proposal, talk to your Hortonworks or Power sales reps 30© 2016 IBM Corporation
Editor's Notes
- #2 TALK TRACK This presentation provides an overview of the Hortonworks and IBM Power partnership and how it will enable clients rapid access to open innovation and drive key business value on their analytics journey.
- #3 TALK TRACK 1st I will review the partnership announce at IBM Edge in Sept 2016 2nd I will look at an example of a client’s planned journey with HDP and Power Systems and the renovate to innovate philosophy Next I will review how Hortonworks leads the open community innovation for Hadoop and supports their client with business success Lastly I will overview how OpenPower innovation delivers outstanding time to insights for analytics
- #4 TALK TRACK Overview the recently announced partnership between IBM and Hortonworks that will bring open Hadoop to OpenPOWER Encourage clients to view the videos of Scott Gnau talking about Hortonworks perspective and James Wade who talked about how Florida Blue is leveraging open source HDP and OpenPower to power his analytics journey.
- #5 TALK TRACK Refer to the linked blog for the full background: http://hortonworks.com/blog/delivering-flexible-infrastructure-analytics-hortonworks-hdp-ibm-power-systems/ Summary of Florida Blue’s three new business models (expanding from one to four businesses): started with a B2B insurance model, entering retail insurance, now handling 32% of US government Medicare claims and opening provider clinics in Florida Florida Blue has seen outstanding 3-5x performance results of running MongoDB on IBM OpenPOWER servers to support their call center initiatives. Note that they are using both Hortonworks and IBM Power today and are planning to bring these capabilities together as one of the 1st adopters of the HDP 2.5 for Linux on Power within the early client program.
- #6 TALK TRACK At Hortonworks, we partner with our customers and guide them on their Journey to Actionable Intelligence. You can start your journey anywhere you want. You can renovate your IT architecture to reduce costs and boost functionality. Or you can innovate modern data applications that you use at your own company or sell on the open market. You can start with the most sophisticated use cases if your team is experienced, or you can build your expertise by beginning with less complex use cases that bring quick results. As you build your team’s expertise and comfort with Hortonworks Data Platform and Hortonworks DataFlow, you can then tackle more challenging aspects of your road map. We will help you plan the right path to meet your objectives. Now I’d like to tell you about some Hortonworks customers and how they are transforming their industries with the actionable intelligence that comes from Connected Data Platforms. [NEXT SLIDE] DISCUSSION STRATEGY [For Business Prospects] Focus questions What business problems can we help you solve? Which use case would you like to tackle first? What type of challenge is most important: data discovery, building a single view or creating predictive analytics? Calls to action Recommend the Jumpstart package: http://hortonworks.com/products/subscriptions/jumpstart/ Schedule a use case workshop and plan your journey across your most important use cases. Give them an industry-specific White Paper to read. [For IT Prospects] Focus questions Where do you face the most cost pressure to store and process data? Which use case would you like to tackle first: active archive, ETL offload or data enrichment? Calls to Action Recommend that they download Hortonworks Sandbox: http://hortonworks.com/products/hortonworks-sandbox/ Schedule a use case workshop Give them the EDW Optimization White Paper to read: http://hortonworks.com/solutions/edw-optimization/
- #7 TALK TRACK When you choose a vendor, you choose a platform that will be with you for a long time. As the originators of both the Apache Hadoop and Apache NiFi technologies as well as the Open Enterprise Hadoop category, Hortonworks is uniquely positioned to help you transform your business with actionable intelligence. We are the only publicly traded pure-play Hadoop company Our momentum is accelerating. Our customers come to us for all the reasons I’ve described: our superior technology that evolves at the pace of open innovation, our proven model for partnering with our customers and our dedication to helping them succeed. I’d like to come back for a larger meeting and a use case workshop to start the journey together. Would next week work for you? [NEXT SLIDE] SOURCE: http://hortonworks.com/about-us/quick-facts/
- #8 TALK TRACK Hortonworks has always followed a 100-percent open approach to Hadoop and Hortonworks Data Platform is the only genuinely open Hadoop distribution. Our open approach: Eliminates risk of becoming locked in with one proprietary vendors It maximizes innovation by tapping into the ingenuity of the largest number of talented developers with the broadest perspective of the capabilities and areas for improvement It integrates seamlessly with other datacenter technologies. Like our customers, our technology partners also want to avoid getting locked in to a proprietary approach. [NEXT SLIDE] SUPPORTING DETAIL Eliminates Risk Why this matters to our customers: Their interests in Hadoop align with Hortonworks’ incentives to deliver world-class support and constantly improve Hadoop Proof point: Spotify was running Cloudera without support. As the importance of Hadoop grew in their business, they needed support. They chose to migrate from CDH AND to begin paying Hortonworks for support, instead of staying on CDH unsupported Citation: "[Hortonworks'] true open source approach and the work they have done to improve the Apache Hive data warehouse system aligns well with our needs," said Wouter de Bie, team lead for data infrastructure at Spotify, in a statement. "We use Hive extensively for ad-hoc queries and for the analysis of large data sets.” | http://www.informationweek.com/software/information-management/spotify-embraces-hortonworks-dumps-cloudera/d/d-id/1111563? Maximizes Community Innovation Why this matters to our customers: They benefit from the fastest possible innovation that comes from the real-world needs felt by our HDP subscribers Proof point: The Stinger Initiative organized the efforts of 144 developers at 45 companies to write 390,000 lines of java code in 13 months. This combined effort made the most important Hive queries 100X faster. Citation: “Combining the capabilities of HARMAN services and Hortonworks, automakers and their suppliers will have access to a scalable platform for, real-time insights, new innovative service creation and predictive analytics-based solutions that can minimize the risk of costly recalls and reduce warranty expenses,” said Sanjay Dhawan, President, HARMAN Services Division. “This is a very exciting step forward in the evolution of the connected car as we speed up the time to market with powerful functionalities that benefit the OEMs and their drivers.” Integrates seamlessly Why this matters to our customers: Rapid, easy adoption that reinforces your existing datacenter technologies Proof point: More than 130 technology partners are HDP-certified, including: EMC, HP, Microsoft, Red Hat, SAP, Teradata, Cisco, SAS and Google Cloud Platform Citation: “The new features in the Hortonworks Data Platform 2.3 allow us to empower our joint customers with access to more data through Apache Hadoop,” said Randy Guard, Vice President of Product Management, SAS. “This allows them to perform deeper analysis on their data with extensive security and data governance capabilities in place. We want our customers to know that we support the platforms and environments they need to handle their data-driven workloads in a flexible, secure and efficient way.” | http://hortonworks.com/press-releases/hortonworks-accelerates-big-data-transformations-with-hortonworks-data-platform-2-3/
- #9 TALK TRACK Hortonworks employs the largest number of project committers to Apache Hadoop, Apache NiFi and their related Apache projects. Our committers innovate with other leaders working through the Apache Software Foundation. We have more than 90 Apache committers, and they interact with each other every day in meetings, in our hallways or over lunch, And our leadership in the Apache community also helps us influence the roadmap, to meet the needs that we learn about from our hundreds of enterprise subscribers. The Apache way is democratic, but we have the largest voting block. [NEXT SLIDE] SUPPORTING DETAIL We Employ the Committers Why this matters to our customers: No company understands Hadoop better than Hortonworks. This correlates directly to the quality of our enterprise support. Proof point: Specific Apache Hadoop committer numbers as of January 2016 (# of committer seats / % of total committers): Hadoop: Hortonworks = 30/31%, Cloudera = 17/71% Customer quote: Paul Boal, “Whenever our developers need access to some knowledge, and we say ‘Could we talk to somebody about that?’, Hortonworks says, ‘Sure, here are a few of our committers. Why don’t you talk to them?’“ | https://youtu.be/iQ2V3FuqxHg @ 3:04 Our Committers Innovate Why this matters: Choose the team that leads Hadoop innovation, so you can benefit from a rapid flow of new features that directly benefit your business. Proof points: YARN – YARN is the data operating system of Hadoop 2. Arun Murthy originally conceived of YARN in the JIRA ticket MapReduce 279. The founding architects of Hortonworks developed YARN until it was ready for release as part of Apache Hadoop 2.0, which was GA in October 2013. | http://hortonworks.com/blog/introducing-apache-hadoop-yarn/ The Stinger Initiative: In 2013, our customers told us that Hive was too slow and that it didn’t include all the semantics they needed. So we spearheaded the Stinger Initiative to improve the speed, semantics and scale of Hive queries. Over 13 months and three versions of Hive (0.11, 0.12 and 0.13) – 144 developers from 45 companies wrote 390 lines of java code. This improved Hive’s speed by more than 50x and added the missing SQL semantics. | http://hortonworks.com/blog/announcing-apache-hive-0-13-completion-stinger-initiative/ Partner quote: “Hortonworks has partnered with Red Hat since its early days. They have been a champion of the open source model and a key enabler of the communities needed to support the big data ecosystem,” said Greg Kleiman, director of big data strategy at Red Hat. “As a strategic partner, Hortonworks brought their flavor of open source innovation to build more agile and cost effective big data solutions with Red Hat.” | http://hortonworks.com/press-releases/hortonworks-recognized-as-crn-emerging-vendor/ We Influence the Roadmap Why this matters to our customers: They can influence the roadmap and attract top technical talent to join their teams Proof point: The Data Governance Initiative initiative led to the creation of Apache Ranger. Hortonworks encouraged its subscribers Schlumberger, Aetna, Target and Merck to join the initiative to speak for their requirements and to engage their teams to solve for those requirements. This led directly to creation of the Apache Atlas project, and each of these customers employs at least one committer to Atlas. | http://hortonworks.com/blog/apache-atlas-project-proposed-for-hadoop-governance/ Analyst Quote: “Applying old-school dictatorial-style governance to Hadoop would be a disaster for enterprises because it would tamp down the agility of Hadoop,” he said. “The Data Governance Initiative is intriguing because it brings the Hadoop community together with real enterprises. I hope they partner in a way that create ‘minimally viable governance.’ That would keep Hadoop flexible while giving enterprises the governance they need to prevent data pandemonium.” | http://www.bigdatatechcon.com/news/data-governance-initiative-expands-the-hadoop-ecosystem
- #10 TALK TRACK We know that the community powers the ongoing success of our technology, and so we provide a platform for that community through Hortonworks Community Connection. As a founding member of ODPI, we help influence the larger ecosystem that is standardizing around Apache Hadoop as they develop new Big Data applications. And we engage our partner ecosystem community through the Hortonworks Partnerworks program. [NEXT SLIDE]
- #11 TALK TRACK Hortonworks Support Subscriptions come with Hortonworks SmartSense to help you proactively and predictively optimize your HDP cluster. SmartSense analyzes the data on how you use your cluster, compares that to best practices and makes automated recommendations about how you can optimize your investment in HDP. We also offer a variety of self-service tools through our integrated customer portal. And of course, you can calls us 24x7. We’ll always pick up, and for any tough issues, our support engineers have access to our committers. [NEXT SLIDE] SUPPORTING DETAIL Hortonworks SmartSense Why this matters to our customers: Hortonworks SmartSense helps our customers make the most of their Hadoop cluster – using data. Proof point: Just like our customers use machine data for preventative maintenance on their telco infrastructures, oil rigs or military aircraft, they can use the same type predictive analytics using data from their own cluster to optimize that resource. In fact, Hortonworks subscribers benefit (through SmartSense) from the usage data from all other Hortonworks customers. Citation: “SmartSense’s capabilities, says Cheolho Minale, vice president of technology at The Mobile Majority will allow his Hadoop team to optimize it’s HDP cluster’s ad performance, ‘At The Mobile Majority, we have been using Hortonworks Data Platform to optimize ad performance on behalf of our customers. We’re excited to look into Hortonworks SmartSense as a way to continuously optimize our HDP cluster as it grows over time.’” | Source: http://hortonworks.com/blog/introducing-availability-of-hdp-2-3-part-3/ Integrated customer portal Why this matters to our customers: Hortonworks subscribers can scale their organization’s Hadoop expertise as quickly as they want.
- #16 TALK TRACK IBM Power Systems, with our OpenPOWER partners, offer a full range of server solutions from public cloud to distributed scale out servers to 192 core scale up servers LC and L line servers have been developed specifically for scale out Linux applications LC servers provide clear price/performance leadership with form factors and price points competitive with commodity Intel Unlike commodity Intel, Power Systems supports an open ecosystems via the OpenPOWER initiative which ensures continuous leading innovation For example: Intel per core performance generally stays flat each new generation while the number of cores per server increases driving up SW costs Power per core performance has demonstrated significant improvement with each successive generation. [NEXT SLIDE]
- #20 We know that IT system downtime is much more than an inconvenience or aggravation. When your systems are down or not performing optimally, people across your company can’t do their jobs effectively—if at all. As a result, revenue, profit, company reputation and customer loyalty can suffer. We know that you need one dependable, trusted provider to help manage and resolve problems regardless of the platform or vendor. In particular, you need open source and Linux infrastructure specialists. IBM has been committed to open technologies from the onset, bringing more than 16 years of experience supporting open source environments. Through our alliances with SUSE, Ubuntu and Red Hat, we demonstrate our commitment to the success of open technologies. We bring a virtually unparalleled expertise in supporting Linux across all IBM systems and OEM x86 platforms certified for Linux. Our well-established global support infrastructure is key. With that you can gain: In-depth skills and experience that encompass the many products and technologies that make up your IT environments A streamlined process that expedites problem determination and resolution A proven track record in delivering result-producing support services The ability to access your support provider as often as needed, around the clock and around the world With IBM, you can gain accountability. When you report a problem, you can feel confident that it will be handled professionally and expeditiously, and that our specialists will manage it diligently from initial report through resolution.
- #21 We know that IT system downtime is much more than an inconvenience or aggravation. When your systems are down or not performing optimally, people across your company can’t do their jobs effectively—if at all. As a result, revenue, profit, company reputation and customer loyalty can suffer. We know that you need one dependable, trusted provider to help manage and resolve problems regardless of the platform or vendor. In particular, you need open source and Linux infrastructure specialists. IBM has been committed to open technologies from the onset, bringing more than 16 years of experience supporting open source environments. Through our alliances with SUSE, Ubuntu and Red Hat, we demonstrate our commitment to the success of open technologies. We bring a virtually unparalleled expertise in supporting Linux across all IBM systems and OEM x86 platforms certified for Linux. Our well-established global support infrastructure is key. With that you can gain: In-depth skills and experience that encompass the many products and technologies that make up your IT environments A streamlined process that expedites problem determination and resolution A proven track record in delivering result-producing support services The ability to access your support provider as often as needed, around the clock and around the world With IBM, you can gain accountability. When you report a problem, you can feel confident that it will be handled professionally and expeditiously, and that our specialists will manage it diligently from initial report through resolution.
- #22 TALK TRACK POWER8’s leading IO and Memory capabilities power fast data access and movement across a wide range of BDA applications. POWER8 is the first microprocessor designed for Big Data and Analytics. When systems are designed for big data, there are a couple of key attributes that are important to create a balanced system design. First having the processing capability, second having the memory space, the workspace, and the third is having the bandwidth, the ability to move the information in and out of the system at the rapid speeds required. POWER8 delivers 4 times more threads per core vs. commodity infrastructure. We can easily support a growing number of users who need reports, or to perform ad hoc analytics. This is because the processor can run more concurrent queries in parallel faster, across multiple cores with more threads per core. We’re delivering up to 4 times more memory bandwidth. Increased memory bandwidth to access up to 1 TB of memory for data operations and enlarged cache in every processor. This delivers the levels of performance your teams need to make decisions in real time. We’re delivering faster IO and high speed caching to ingest, move and access large volumes of data so that analytics results are available faster. Moore’s law is no longer delivering the processor speed up needed to meet the demands of Big Data, the OpenPOWER foundation is driving innovations such as hardware accelerators that will deliver on a continuous improvement in price/performance leadership. Power Systems provide the capabilities needed to handle the varying analytics initiatives your business requires. Broad range of data and analytics – from operational to computational to business analytics, as well as cognitive solutions leveraging IBM Watson technology, Power Systems are optimized for performance and can scale to support demanding and growing workloads. These solutions help you capitalize on the currency of data by finding business insights faster and more efficiently. Supporting Data: threads per core vs. x86 (up to 1536 threads per system) memory bandwidth vs. x86 (up to 32TB of memory) more cache vs. x86 (>19MB cache per core) The POWER8 cache numbers are L1 - 96 KB per Core ==> L2 - 512KB per Core ==> L3 - 8 MB per core ==>(8192KB) L4 - 128 MB per Socket 128/12 = 10.7MB per core (10922KB per core) Total Cache per Core - 19.26MB (19722KB) The Intel side (Broadwell-EP) also has L1 and L2... L1 - 64 KB per Core ==> L2 - 256KB per Core ==> L3 - 55 MB per socket ==> 2.5MB per Core (2560KB per core) Total Cache per Core – 2.81MB (2880KB) Ratio is 19722/2880 = 6.85 round down to 6X
- #25 TALK TRACK - One good example of OpenPOWER innovation that could be exploited by GPU aware applications on Hadoop is the POWER8 NVLink support - In partnership with IBM’s OpenPOWER partner Nvidia, we released a POWER8 Linux server in Sept 2016 with 2.8X the IO throughput from CPU to GPU of what is possible on x86 - This removes the PCIe data pipe bottleneck to ensure full exploitation of the GPUs and is only available today on OpenPOWER servers Links: IBM S822LC for HPC with NVLink: http://www-03.ibm.com/systems/power/hardware/s822lc-hpc/ Related press: https://www.hpcwire.com/2016/09/08/ibm-debuts-power8-chip-nvlink-3-new-systems/ http://www.pcworld.com/article/3117718/ibms-new-power8-server-packs-in-nvidias-speedy-nvlink-interconnect.html
- #26 Note: This chart compares 16-c S822LC to the 36-c HP DL380. So the below System-level performance is more compelling when this is considered. S822LC system-level total through put is 20%+ higher than DL380. The price Performance shown is outcome of this higher performance and the lower HW street price of S822LC. On Trans/$ graph, note that the VM quantities show 20/24 VMs on Power and 16/20 VMs on x86 are intentionally different. As the goal is to show virtual machines that Produce similar per VM performance through put level. The slight raise in x86’s second bar is not significant and is due to the fact that x86 system is under allocated on first data point and not on second, where as p8 is already fully allocated on both data points shown. Memory: 256GB for S822LC and 256 GB for HP DL380 HBA: 2 x 16 gbps FCA for S822LC and HP Eth: 2port 10GbHDD: 2 x 1TB for S822LC & 2x 1TB for HP Price (match the configuration) BUT 2 x 300GB SAS 10k SFF for HP testcase Power S822LC delivers leadership performance vs Intel Xeon E5-2699 v3 2.7X more transactions/second per core Improve your revenue potential – Power S822LC delivers 25% more VMs in the same rack space as HP DL380 25% more virtual machines per server than HP DL380 76 more revenue generating virtual machines per rack than HP DL380
- #27 TALK TRACK - Spark workloads are ideal to gain advantage from the core capabilities of the POWER8 processor for multithreading and large memory - Streaming and SQL workload can be broken down into many parallel threads and exploit the 8 thread per core unique to POWER8 - Complex, memory intensive workloads like machine learning and graph, can benefit from the large memory bandwidth and cache - The result shown here are from an open source Spark Bench workload (contributed by IBM) which drives a set of representative workloads across machine learning, SQL and Graph on Spark. (https://github.com/SparkTC/spark-bench) - You can see that overall POWER8 provides a 2X per core advantage and 1.5X price performance advantage in these 7 node cluster results
- #29 TALK TRACK - To summarize, the combination of HDP and OpenPOWER offers the fastest route to exploitation through open community innovation - ODPi and OpenPOWER foundations ensure no risk of vendor lock-in and no other Hadoop + System combination can offer this pairing - Hortonworks and IBM both have deep industry experience and a proven history of outstanding dedication to client success and support - Ultimately, clients need fast time to insights and OpenPOWER delivers on performance with a reduced infrastructure cost
- #30 TALK TRACK - As mentioned earlier, the Hortonworks and IBM Power Systems partnership was announced at IBM Edge conference in Sept 2016 - Early adopter clients will gain access to technical preview builds in late 4Q2016 - General availability for HDP 2.5 for Linux on Power will follow in 1Q2017 - Future HDP releases will provide Linux on Power support at the same time as the Linux on Intel releases
- #31 TALK TRACK - If you are not already a member, join the Hortonworks community to access to valuable training resources and member insights - Get to know more about the value of Power Systems and OpenPOWER and how it can provide a unique, open and optimized alternative for your Big Data and Analytics workloads - To learn more about HDP and Power Systems, please join us for an upcoming webiniar or catch the replay.