Hadoop and Spark are big data frameworks used to extract useful span a variety of scenarios from ingestion, data prep, data management, processing, analyzing and visualizing data. Each step requires specialized toolsets to be productive. In this talk I will share solution examples in the Big Data ecosystem such as Cask, StreamSets, Datameer, AtScale, Dataiku on Microsoft’s Azure HDInsight that simplify your Big Data solutions. Azure HDInsight is a cloud Spark and Hadoop service for the enterprise and take advantage of all the benefits of HDInsight giving you the best of both worlds. Join this session for practical information that will enable faster time to insights for you and your business.

3. Big data is hard
Buy
Servers
Install
OSS
Secure Configure
Optimize
Debug
Success
Scale up

4. HDInsight makes it easy
Provide
Cluster
details
HDInsight
Cluster
 100% open source
 Optimized
 Highly available
 Secure
 Scalable
 Dedicated
 Managed
 Certified ISVs
 Customizable
Browse to
Azure Portal

5. Managed Open Source Analytics for the
cloud with a 99.9% SLA.
100% Open Source
Clusters up and running in minutes
63% lower TCO than deploy your own Hadoop on-
premises
Separation of compute and store allows you to scale
clusters to exponentially reduce costs
Open Source for the Enterprise

6. Multi Region Availability
Available in >25 regions world-wide
Launched most recently in US West 2, and UK regions
Available in China, Europe and US Government clouds
Deploy Globally Within Minutes

7. Perimeter Level Security
Virtual Networks
Network Security Groups (firewalls)
Authentication
Azure Active Directory
Kerberos authentication
Authorization
Apache Ranger
RBAC for Admin
POSIX ACLs for Data Plane Data Security
Server-Side encryption at rest
HTTPS/TLS In-transit
Security and Compliance to Enable OSS for Enterprises

8. Plugins for HDI available for most popular IDEs for agile
development and debugging
Rich support for powerful notebooks used by data
scientists
Develop in C#, deploy on Linux in Java via HDI
developed SCP.Net technology
Rich Developer Ecosystem

9. Recognized by
Top Analysts
Forrester Wave for Big Data
Hadoop Cloud
• Named industry leader by
Forrester with the most
comprehensive, scalable, and
integrated platforms*
• Recognized for its cloud-first
strategy that is paying off*
*The Forrester WaveTM: Big Data Hadoop Cloud Solutions, Q2 2016.

12. Hive meta store
Azure SQL database
Azure Storage or
Data Lake Store
Client
machines
HDInsight cluster
Gateway
nodes
Head
nodes
Worker
nodes
Edge
nodes
Zookeeper nodes

14. • Manage Data drift (unexpected changes) at
scale.
• Run at enterprise grade & scale on HDInsight

15. Machine Learning
and Analytics
Big Data Stores
Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards &
Visualizations
Cortana
Bot
Framework
Cognitive
Services
Power BI
Information
Management
Event Hubs
HDInsight
(Hadoop and Spark)
Stream Analytics
Intelligence
Data Lake
Analytics
Machine
Learning
SQL Data
Warehouse
Data Lake Store
Data
Sources
Apps
Sensors
and
devices
Data
STREAMSETS DATACOLLECTOR
*
Data Catalog
*
*
*
*
Data Lake Store

17. • Run at enterprise grade & scale on HDInsight

19. • No data movement for BI scenarios
• Use any BI tool
• Scale out OLAP with Hadoop & Spark
• No Hadoop/ Spark expertise required
• Target audience: Analysts
• Run at enterprise grade & scale on HDInsight

21. • Run at enterprise grade & scale on HDInsight

23. • Run at enterprise grade & scale on HDInsight

25. Big Data Stores Machine Learning
and Analytics
Cask and Microsoft:
Enabling Customers’ Journey from Data Ingestion to Action
Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards &
Visualizations
Cortana
Bot
Framework
Cognitive Services
Power BI
Information
Management
Event Hubs
Data Catalog
Data Factory
HDInsight
(Hadoop and Spark)
Stream Analytics
Intelligence
Data Lake Analytics
Machine Learning
SQL Data
Warehouse
Data Lake Store
Data
Sources
Apps
Sensors
and
devices
Data
Capture, track and
analyze (meta)data
Enable governed
self-service
Develop, test, debug
& deploy apps
Ingest, explore &
serve data

26. Azure HDInsight application platform
Install custom HDInsight applications
Try SDC from StreamSets on HDInsight
Try Datameer on HDInsight
Try AtScale on HDInsight
Try DSS from Dataiku on HDInsight
Try H2O.ai on HDInsight
Try CDAP from Cask on HDInsight