Azure Databricks is a platform for running Apache Spark and analytics workloads in the cloud. It provides a managed Spark cluster, tools for data engineering and science, and integrates with other Azure services. The document discusses features of Databricks like the workspace, workflows, runtime, security, and how it can be used for SQL, NoSQL, streaming, machine learning, and connecting various data sources.

1. Azure Databricks
@vishwasnarayan5 Vishwas N
https://hacksterdude.web.app/

2. About me
I am a Podcaster : http://tiny.cc/vnrpodcast
I love to talking to techies
A Bibliophile
Passionate about Image dataset - Computer Vision
Now exploring Azure cloud

3. Spark SQL
+ Data
Frames
Streaming
MLLIb
Machine
Learning
Graph X
Graph
Computation
Eco - System
Spark Core API

4. Azure DataBricks Platform
DataBricks WorkSpace DataBricks WorkFlows
DataBricks RunTime
Data Bricks I/O(DBIO) DataBricks Serverless
DataBricks Enterprise Security
…….

5. NoSQL
Storage
SQL
H
a
d
o
o
p
In Memory
MPP
Machine
Learning
ETL
Queuing
Service
Move Data to/From
SQL Databases
Streaming
Streaming
Distributed
System
Coordination
B
A
T
C
H
Real Time
Full
Text
Search
Hadoop EcoSystem : This is it

6. Spark Cluster Architecture
Source : https://spark.apache.org/docs/latest/cluster-overview.html

7. Spark Core Engine
Spark SQL
Interactive
Queries
Spark Structured
Streaming
Stream processing
Spark MLlib
Machine
Learning
Yarn Mesos
Standalone
Scheduler
Spark
MLlib
Machine
Learning
Spark
Streaming
Stream
processing
GraphX
Graph
Computation

10. What is it all about?

11. Where MLFlow Makes
a Lot of Influence