The recently launched YARN Ready Program will accelerate multi-workload Hadoop in the Enterprise. The program enables developers to integrate new and existing applications with YARN-based Hadoop. We will cover: --the program and it's benefits --why it is important to customers --tools and guides to help you get started --technical resources to support you --marketing recognition you can leverage

1. Page1 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Introduction to YARN Ready
John Kreisa – VP Strategic Marketing
Ajay Singh - Director, Partner Solutions

2. Page2 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
YARN Ready Program Overview
John Kreisa

3. Page3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Why YARN Ready? -> Apache Hadoop YARN
• YARN is the architecture center of Apache Hadoop 2 enabling more
efficient cluster utilization
• More and more customers are asking about tools and applications that
are integrated into YARN to maximize the value of their Hadoop cluster
• YARN Ready is a program offered by Hortonworks the leading
developers of YARN
–YARN was conceived of by Arun Murthy Hortonworks founder
–He submitted JIRA MapReduce 279 in January 2008 and has been working on it since
–Hortonworks has written 80% of the YARN code
Page 3

4. Page4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
The Two YARN Ready Program Goals
 Expand the Hadoop ecosystem by enabling ISVs to build & validate
YARN-based applications, using Apache technologies
 Allow enterprises to know which technologies maximize Hadoop
resources in an ecosystem-friendly way

5. Page5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Two Achievements Available
 HDP Certified
 YARN Ready
Badge Description
HDP (Certification) Integrates with HDP 2.x
YARN Ready (Validation) Uses the YARN resource scheduler

6. Page6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
YARN Technical Overview and Value
Ajay Singh

7. Page7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Hadoop 1.0: Batch
HADOOP 1.0
Built for Web-Scale Batch Apps
Single App
BATCH
HDFS
Single App
INTERACTIVE
Single App
BATCH
HDFS
• All other usage patterns must
leverage that same
infrastructure
• Forces the creation of silos for
managing mixed workloads
Single App
BATCH
HDFS
Single App
ONLINE

8. Page8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Hadoop 2.0: Multi Workload
HADOOP 1.0
HDFS
(redundant, reliable storage)
MapReduce
(cluster resource management
& data processing)
HDFS2
(redundant, reliable storage)
YARN
(cluster resource management)
MapReduce
(data processing)
Others
(data processing)
HADOOP 2.0
Single Use System
Batch Apps
Multi Purpose Platform
Batch, Interactive, Online, Streaming, …

9. Page9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Key Driver Of Hadoop Adoption: Enterprise Data Lake
Flexible
Enables other purpose-built data processing
models beyond MapReduce (batch), such
as interactive and streaming
Efficient
Double processing IN Hadoop on the same
hardware while providing predictable
performance & quality of service
Shared
Provides a stable, reliable, secure
foundation and shared operational
services across multiple workloads
Data Processing Engines Run Natively IN Hadoop
BATCH
MapReduce
INTERACTIVE
Tez
STREAMING
Storm
IN-MEMORY
Spark
GRAPH
Giraph
ONLINE
HBase, Accumulo
OTHERS
HDFS: Redundant, Reliable Storage
YARN: Cluster Resource Management

10. Page10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
YARN Development Framework
YARN : Data Operating System
°1 ° ° ° ° ° ° ° °
° ° ° ° ° ° ° ° °
°
°°
° ° ° ° ° ° °
° ° ° ° ° ° N
HDFS
(Hadoop Distributed File System)
System
Batch
MapReduce
Interactive
Tez
Engine Long Running
Slider
Direct
ISV
Apps
Scripting
Pig
SQL
Hive
Cascading
Java
Scala
NoSQL
HBase
Accumulo
Stream
Storm
API
ISV
Apps
ISV
Aps
Applications
Others
Spark
ISV Apps
ISV
Apps

11. Page11 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Tez – Next Generation Data Processing Framework
YARN ApplicationMaster to run DAG of Tez Tasks
Task with pluggable Input, Processor and Output
Tez Task - <Input, Processor, Output>
Task
ProcessorInput Output

12. Page12 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Slider – Onboarding Long Running Services To YARN
 A YARN application to deploy existing distributed applications, monitor
them and make them larger or smaller as desired - even while the
cluster is running
 Has a plug-in provider architecture to support different applications
 Automatic Deploy, Manage, Monitor via Ambari
– Standard management APIs for Ambari to use
– Standard ways to collect metrics
 Clusters can be stopped, "frozen" and restarted, "thawed" later
– Distribution of the deployed application across the YARN cluster is persisted.

13. Page13 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
# of Apps / Ease of Integration
Control
Integration Option Tradeoffs
YARN Native Apps
Tez Apps
Services on YARN (Slider)
MapReduce Apps
Hive

14. Page14 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
HADOOP
YARN
HDFS
SQL
Datanode
HDFS
Visual Data
& Analytics
Workflow
Actian Analytics Platform – YARN Native Integration Example
Transform Hadoop into a High Performance Analytics Platform
Datanode
HDFS
Datanode
HDFS
Datanode
HDFS
X100X100X100
Read
Load
Actian Vector
Blend &
Enrich
Data Science
& Analytics
Datanode
HDFS
X100
HDFS
Vector
• Original file format
• Standard block
replication
• Column-based
blocks
• Binary
• Compressed
• Partitioned
• Faster Loading
• Faster SQL
• Standard SQL
• Better Scaling
Namenode
High Performance,
Industrialized SQL
Database
High Performance,
Data Science &
Analytics

15. Page15 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Partners with YARN Ready Apps

16. Page16 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
YARN Ready Program Details
John Kreisa

17. Page17 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Partner Benefits - Marketing
 Recognition
 Recognition as Hortonworks Certified Technology Partner on our website along with the YARN Ready
badge
 Hortonworks quote for reference in partner marketing assets
 Promotion to customers in Hortonworks led events and activities
 Joint Marketing & Field Alignment
 Invitation to Modern Data Architecture Webinar Series
 Hortonworks Sales Briefing and alignment with Field BD
 Automatic participation in digital marketing initiatives
 LinkedIn and FB campaigns
 Twitter and blog activity

18. Page18 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Partner Benefits - Technical
 Tech Expertise
 YARN implementation guide and sample code
 Integration best practice guidance
 Access to the architects and builders of YARN
 Roadmap Input
 YARN, Tez and Stinger roadmap visibility
 Input into the direction of YARN and the YARN engines
 Access to pre-release builds to speed integration with new features

19. Page19 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Process for Partners
 Become a Hortonworks partner
 Sign up at http://hortonworks.com/partners/become-a-partner/
 Become Certified
 Download the certification kit (a .zip archive) from the partner portal
 Validate with HDP (Sandbox OK)
 Complete the provided templates (samples are included) for
 Solution Overview
 Reference Architecture
 Certification Report
 Submit the three documents to technology-certification@hortonworks.com.
 SLA: 10 working days for Hortonworks response.
 Collaborate with Hortonworks on joint marketing – via ChannelMarketing@Hortonworks.com

20. Page20 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Certification Kit
 A .zip file available to partners
 Contents:
 Program Guide
 Solution Overview template (DOC)
 Reference Architecture template (PPT)
 Certification Report template & sample (DOC)
 Sample YARN App code*
 Sample Slider provider code*
 YARN app test recommendations*
 Logos (available after certification)
 PR Template (available after certification)
 README file (text)
*YARN Ready, Coming – target July

21. Page21 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Calls to Action
 Engage us on your YARN initiative
 Become HDP Certified and YARN Ready
 Develop a joint Sandbox Tutorial
 Collaborate on “Art of Possible” Demonstration
 Co-Market & Co-Sell

22. Page22 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Questions?
Join us for the next series of webinars on integrating to YARN:
YARN native July 24 @ 9am PT
Slider integration August 7 @ 9am PT
Tez integration August 21 @ 9am PT
Page 22

23. Page23 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Thank You