The document discusses the adoption and integration of Apache Spark within major Hadoop distributions, highlighting the need for simplified big data solutions. It outlines the Databricks platform, which offers a unified workspace for managing Spark applications and enhances user experience through training programs and on-demand cluster management. The goal is to make big data processing more accessible and efficient for organizations seeking to leverage their data effectively.

1. Spark Summit
June 2014

2. Apache Spark and Databricks

3. Adoption
All major Hadoop distributions include Spark
Beyond Hadoop

4. Partnerships
Partner with Spark distributors to provide great
experience to every Spark user
Partners

5. Certification
Build a strong application ecosystem
Spark API
Spark Distros
…
Distros Cert
Spark Apps
… App Cert

6. Certification
Free certification process
Scripts for certifying Spark distributions
• Developed by community
• Open-source
Anyone will be able to certify any Spark distribution

7. Training
We’ve been teaching Spark since 2012
• 400+ people this year through Databricks
Just launched a new training program
• Already hold workshops in 5 cities
300+ people signed up for training on Wednesday

8. Solve Big Data Challenges

9. Big Promise
Great successes using Big Data

10. Big Promise
Great successes using Big Data
Every organization collects dataYour company here!

11. Big Challenge
Great successes using Big Data
Google, Facebook spend billions $ to develop,
implement, and run data analysis tools and products
Your company here!
Every organization collects data

12. Typical Story
Your company starts a Big Data initiative
You are tasked to…
1) Build a Hadoop cluster
2) Build a data pipeline
3) Get insights &
build data products
Clusters hard to set up
and manage
Need to integrate a zoo
of tools
Tools are hard to use
(IT)
(engineers, data scientists)
(engineers, data scientists, analysts)

13. Typical Data Pipeline
Data
ETL
Exploration
Dashboards
& Reports
Data
Products
Advanced
Analytics
Integrate disparate, clunky tools
Hard to navigate data, develop and deploy apps

14. Vision
Make big data easy

15. From Challenges to Solutions
Challenges
Solutions
Hosted platform
Apache Spark
Clusters hard to set up
and manage
Need to integrate a zoo
of tools
Tools are hard to useInteractive Workspace

16. Databricks Cloud
Databricks Workspace
Databricks Cloud
Databricks Platform

17. Databricks Platform
Databricks Workspace
Databricks Platform
…
…

18. Databricks Platform
Start clusters in seconds
Zero-cost management
Dynamically scale up & down

19. Apache Spark
Unifies
• Streaming
• SQL
• Machine learning
• Graphs
Single system,
Databricks Workspace
Databricks Platformsingle API

20. Databricks Workspace
Databricks Workspace
Notebooks
Dashboards
Jobs
Apps
Databricks Platform

21. Notebooks
Support Python, SQL, Scala
Interactive commands & plots
On-line collaboration

22. Dashboards
WYSIWYG builder
Interactive plots
One-click publishing

23. Job Launcher
Run arbitrary Spark jobs, programmatically

24. Dramatically Simplify Data Pipeline
Data
ETL
Exploration
Advanced Analytics
Dashboards & Reports
Data Products
Cloud

25. Dramatically Simplify Data Pipeline
Data
ETL
Exploration
Advanced Analytics
Dashboards & Reports
Data Products
Cloud
Free users to focus on
finding answers & building products

26. Demo

27. Availability
Started closed beta program earlier this year
Limited availability soon
• Gradually ramping up
• Sign up on databricks.com!

28. 3rd Party Apps
Databricks
Workspace
Databricks Platform

29. 3rd Party Apps
Databricks Workspace
Apps
…
Databricks Platform

30. Databricks Cloud and Spark
Databricks Cloud runs 100% Apache Spark
• No lock in: any Databricks Cloud app runs on any
certified Spark distribution
Databricks Cloud accelerates Spark adoption
• Provide easiest way to learn and use Apache Spark

31. Databricks Cloud
Databricks Workspace
Databricks Platform
Dramatically simplify
• analyzing big data
• building data products
Fuel growth of Spark ecosystem
Make big data easy

32. Thank You!