SlideShare a Scribd company logo
A presentation by
W H Inmon
DATA LAKEHOUSE –
THE BASIC ELEMENTS
All data in the corporation
Structured
data
Textual
data
Analog/IoT
data
Structured
data
Textual
data
Analog/IoT
data
Each of the different types of data have their
own unique characteristics
Structured
data Usually transaction based
record
key attribute
index
Bank transactions
Point of sale
Telephone call
Payments made
Payments received
…………………….
Structured
data
The same record type is repeated
Each record has different contents
Textual
data
Medical records
Contracts
Internet
Call centers
Warranty claims
Insurance claims
Email
………………..
Text is found everywhere
English
Spanish
Portuguese
French
Mandarin
Korean
German
Formal language
Slang
Acronyms
……………..
Voice
Written
Internet
Video
………………..
Textual
data
Textual
ETL
taxonomies
Text is transformed
Into a structured
format
Analog/IoT
Machine generated
Drones
Electric eye
Temperature gauge
Speed
Mechanical
Telemetry
…………………….
Analog/IoT
telemetry
Date Sept 2, 2021
Time 11:21 am
Location from Denver
Location to Co Spgs
Elevation
786
792
812
854
901
978
1012
1256
1469
1672
2018
2259
2871
……..
Speed
0
35
79
124
197
276
367
416
521
702
835
915
…..
Telemetry data is generated as the
rocket is launched and is measured
throughout the flight
Analog/IoT
The data lake is created by throwing data
all the data into the lake
Textual
data
Structured
data
Analog/IoT
Soon the data lake
turned into a swamp
Analog/IoT
The data swamp was not good for anyone….
Analog/IoT
The data lake needs to be turned
into a lakehouse
Analog/IoT
All this education and 95% of my job
is being a data garbageman
Data scientist
Analog/IoT
Data scientist
Ah, that’s more
like it
infrastructure
Analog/IoT
Machine generated
Time – 0912
Time – 0916
Time – 1002
Time – 1008
Time – 1017
…………….
Basic, raw measurements
High probability
High performance
Low probability
Bulk storage
Analog/IoT data is often
segmented
High probability
High performance
Low probability
Bulk storage
Date of launch
Ultimate speed
Ultimate height
Final landing point
Second by second
measurements
Structured
data
Textual
data
Analog/IoT
data
Relative volumes of data in each sector
Structured
data
Textual
data
Analog/IoT
data
Business value and the volumes of data
Structured
data
Textual
data
Analog/IoT
data
Relational format Raw data format
From a format standpoint, the structured and the textual
environments are very different from the analog/IoT
environment
Format compatibility
Structured
data
Textual
data
Analog/IoT
data
Key compatibility – very unintegrated
Content compatibility
Structured
data
Textual
data
Analog/IoT
data
In order to do analytics, there must be
some common data on which to do a
comparison
Without common data it is very difficult
to do a meaningful comparison
Structured
data
Textual
data
Analog/IoT
data
The problem is that there may be no obvious,
easy way to isolate common identifiers
Structured
data
Textual
data
Analog/IoT
data
Fortunately there are such things as
universal common connectors
Structured
data
Textual
data
Analog/IoT
data
Universal common connectors exist regardless of the
way that data has been collected
Structured
data
Textual
data
Analog/IoT
data
Universal common connector for anything
geography
time
dollar amount
General common connectors
Structured
data
Textual
data
Analog/IoT
data
Universal common connector for humans
gender
age
race
Common connectors for humans
Structured
data
Textual
data
Analog/IoT
data
Universal common connector for physical objects
weight
color
cost
size
shape
Common connectors for objects
SOME EXAMPLES
Universal common connector
Healthcare – outcomes analysis
Did the medicine work?
Did the vaccination work?
Did the operation have the right effect?
Outcome analysis
Structured
data
Textual
data
Analog/IoT
data
Prolia
Estrogen
Vitamin D
Algaecal
Calcitonin
Sales of -
Doctor’s notes
tests
diagnosis
procedure
medication
history
……………
X rays
date
location
patient age
examination results
Structured
data
Textual
data
Analog/IoT
data
What medicines
have been
purchased
What medicines
have been
prescribed and/or
discussed with
doctors
By state
By age
By gender
By state
By age
By gender What outcomes have
been achieved
By state
By age
By gender
What medicines
have been
purchased
By state
By age
By gender
What medicines
have been
prescribed and/or
discussed with
doctors
By state
By age
By gender
What outcomes have
been achieved
By state
By age
By gender
Analyses –
how does treatment in Utah vary from treatment in Oregon?
is Prolia more effective than estrogen?
when patients are treated with Algaecal, what other side effects are noticed?
do women have better results than men?
how much does age affect –
the types of treatment for osteoporosis
the effectiveness of treatment
whether men react differently than women
What medicines
have been
purchased
By state
By age
By gender
What medicines
have been
prescribed and/or
discussed with
doctors
By state
By age
By gender
What outcomes have
been achieved
By state
By age
By gender
When you have both treatment and outcome data together, you can
answer – for the first time – important questions about treatment,
medication, dosage, side, effects, demographics of treatment
You can match outcome with treatment
What medicines
have been
purchased
By state
By age
By gender
What medicines
have been
prescribed and/or
discussed with
doctors
By state
By age
By gender
What outcomes have
been achieved
By state
By age
By gender
The result is healthier people
and longer life and better quality
of life
Manufacturing
Structured
data
Textual
data
Analog/IoT
data
Sales data
unit sold
date of sale
location of sale
customer address
Warranty claims
unit
unit type
defect
severity
in use desc
Manufacturing data
unit id
lot id
date of manufacture
machine used
operator
Textual
data
Structured
data
Analog/IoT
data
Units sold
Date of sale
Location of sale
Unit id
Defect description
Date of warranty
Unit id
Machine used for manufacture
Date of manufacture
Operator
Lot id
Manufacture telemetry
Unit id
Unit id
Unit id
Units sold
Date of sale
Location of sale
Unit id
Defect description
Date of warranty
Unit id
Machine used for manufacture
Date of manufacture
Operator
Lot id
Manufacture telemetry
Analyses –
what manufacturing machines are producing defects
what manufacturing machines are not producing defects
what operators are producing defects
what operators are not producing defects
what telemetry needs to be adjusted
under what conditions are defects created
…………………………………………………………
Units sold
Date of sale
Location of sale
Unit id
Defect description
Date of warranty
Unit id
Machine used for manufacture
Date of manufacture
Operator
Lot id
Manufacture telemetry
With all of this data together and able to be analyzed
you can now tell what defects can be corrected and what
conditions cause defects to occur. The manufacturing
process can be materially improved
Units sold
Date of sale
Location of sale
Unit id
Defect description
Date of warranty
Unit id
Machine used for manufacture
Date of manufacture
Operator
Lot id
Manufacture telemetry
Now manufacturing can be done
efficiently and in a cost effective
manner
With analytics from the data lakehouse, you can improve the
lives and livelihood of many people

More Related Content

What's hot

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
Databricks
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
James Serra
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
Databricks
 
Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3
Jeffrey T. Pollock
 
Is the traditional data warehouse dead?
Is the traditional data warehouse dead?Is the traditional data warehouse dead?
Is the traditional data warehouse dead?
James Serra
 
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
DataScienceConferenc1
 
Data platform architecture
Data platform architectureData platform architecture
Data platform architecture
Sudheer Kondla
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
James Serra
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
James Serra
 
Data Lake: A simple introduction
Data Lake: A simple introductionData Lake: A simple introduction
Data Lake: A simple introduction
IBM Analytics
 
Mapping Data Flows Training deck Q1 CY22
Mapping Data Flows Training deck Q1 CY22Mapping Data Flows Training deck Q1 CY22
Mapping Data Flows Training deck Q1 CY22
Mark Kromer
 
Building Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft AzureBuilding Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft Azure
Dmitry Anoshin
 
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data Pipelines
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data PipelinesPutting the Ops in DataOps: Orchestrate the Flow of Data Across Data Pipelines
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data Pipelines
DATAVERSITY
 
Data Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshData Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to Mesh
Jeffrey T. Pollock
 
Intro to Delta Lake
Intro to Delta LakeIntro to Delta Lake
Intro to Delta Lake
Databricks
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
Databricks
 
Azure Synapse Analytics
Azure Synapse AnalyticsAzure Synapse Analytics
Azure Synapse Analytics
WinWire Technologies Inc
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
Matillion
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data Mesh
LibbySchulze
 
Snowflake Datawarehouse Architecturing
Snowflake Datawarehouse ArchitecturingSnowflake Datawarehouse Architecturing
Snowflake Datawarehouse Architecturing
Ishan Bhawantha Hewanayake
 

What's hot (20)

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
 
Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3
 
Is the traditional data warehouse dead?
Is the traditional data warehouse dead?Is the traditional data warehouse dead?
Is the traditional data warehouse dead?
 
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...
 
Data platform architecture
Data platform architectureData platform architecture
Data platform architecture
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
Data Lake: A simple introduction
Data Lake: A simple introductionData Lake: A simple introduction
Data Lake: A simple introduction
 
Mapping Data Flows Training deck Q1 CY22
Mapping Data Flows Training deck Q1 CY22Mapping Data Flows Training deck Q1 CY22
Mapping Data Flows Training deck Q1 CY22
 
Building Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft AzureBuilding Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft Azure
 
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data Pipelines
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data PipelinesPutting the Ops in DataOps: Orchestrate the Flow of Data Across Data Pipelines
Putting the Ops in DataOps: Orchestrate the Flow of Data Across Data Pipelines
 
Data Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshData Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to Mesh
 
Intro to Delta Lake
Intro to Delta LakeIntro to Delta Lake
Intro to Delta Lake
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
 
Azure Synapse Analytics
Azure Synapse AnalyticsAzure Synapse Analytics
Azure Synapse Analytics
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data Mesh
 
Snowflake Datawarehouse Architecturing
Snowflake Datawarehouse ArchitecturingSnowflake Datawarehouse Architecturing
Snowflake Datawarehouse Architecturing
 

Similar to Data Lakehouse Symposium | Day 1 | Part 1

A View on AI in Insurance - Chris Madsen - H2O AI World London 2018
A View on AI in Insurance - Chris Madsen - H2O AI World London 2018A View on AI in Insurance - Chris Madsen - H2O AI World London 2018
A View on AI in Insurance - Chris Madsen - H2O AI World London 2018
Sri Ambati
 
Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...
Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...
Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...
Aggregage
 
Smartphone Forensic Challenges
Smartphone Forensic ChallengesSmartphone Forensic Challenges
Smartphone Forensic Challenges
CSCJournals
 
Developing a Federal Vision for Identity Management
Developing a Federal Vision for Identity ManagementDeveloping a Federal Vision for Identity Management
Developing a Federal Vision for Identity Management
Duane Blackburn
 
Intel HIMSS WoHIT mhealth
Intel HIMSS WoHIT mhealthIntel HIMSS WoHIT mhealth
Intel HIMSS WoHIT mhealth
rcnossen
 
Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...
Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...
Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...
Coert Du Plessis (杜康)
 
Improving Life With Connected Medical Devices
Improving Life With Connected Medical DevicesImproving Life With Connected Medical Devices
Improving Life With Connected Medical Devices
AryanRaj496746
 
So, My FitBit is Clinical Trial Grade Right?
So, My FitBit is Clinical Trial Grade Right?So, My FitBit is Clinical Trial Grade Right?
So, My FitBit is Clinical Trial Grade Right?
PAREXEL International
 
Preparing Testimony about Cellebrite UFED In a Daubert or Frye Hearing
Preparing Testimony about Cellebrite UFED In a Daubert or Frye HearingPreparing Testimony about Cellebrite UFED In a Daubert or Frye Hearing
Preparing Testimony about Cellebrite UFED In a Daubert or Frye Hearing
Cellebrite
 
Fast and fire-walled IOT healthcare-Baseer
Fast and fire-walled  IOT healthcare-BaseerFast and fire-walled  IOT healthcare-Baseer
Fast and fire-walled IOT healthcare-Baseer
AbdulBaseer (Baseer) Mohammed
 
Architecting, designing and building medical devices in an outcomes focused B...
Architecting, designing and building medical devices in an outcomes focused B...Architecting, designing and building medical devices in an outcomes focused B...
Architecting, designing and building medical devices in an outcomes focused B...
Shahid Shah
 
eBook-IoTPractice
eBook-IoTPracticeeBook-IoTPractice
eBook-IoTPractice
Shargeel sohaib
 
Big data analytics for life insurers
Big data analytics for life insurersBig data analytics for life insurers
Big data analytics for life insurers
dipak sahoo
 
Big_data_analytics_for_life_insurers_published
Big_data_analytics_for_life_insurers_publishedBig_data_analytics_for_life_insurers_published
Big_data_analytics_for_life_insurers_published
Shradha Verma
 
Enterprise Digital Writing
Enterprise Digital WritingEnterprise Digital Writing
Enterprise Digital Writing
mcrussell
 
497secondary
497secondary497secondary
497secondary
Meenakshi Singh
 
Practical Guide - www.devicematters.com
Practical Guide - www.devicematters.comPractical Guide - www.devicematters.com
Practical Guide - www.devicematters.com
PowerViz
 
Big data in IoT for healthcare - www.pepgra.com
Big data in IoT for healthcare - www.pepgra.comBig data in IoT for healthcare - www.pepgra.com
Big data in IoT for healthcare - www.pepgra.com
PEPGRA Healthcare
 
Trends in Wireless Working
Trends in Wireless WorkingTrends in Wireless Working
Trends in Wireless Working
Wheatstone
 
AIMeetup #3: Cortana intelligence suite - tchnij życie w swoje dane
AIMeetup #3: Cortana intelligence suite - tchnij życie w swoje daneAIMeetup #3: Cortana intelligence suite - tchnij życie w swoje dane
AIMeetup #3: Cortana intelligence suite - tchnij życie w swoje dane
2040.io
 

Similar to Data Lakehouse Symposium | Day 1 | Part 1 (20)

A View on AI in Insurance - Chris Madsen - H2O AI World London 2018
A View on AI in Insurance - Chris Madsen - H2O AI World London 2018A View on AI in Insurance - Chris Madsen - H2O AI World London 2018
A View on AI in Insurance - Chris Madsen - H2O AI World London 2018
 
Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...
Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...
Critical Relationships for HR Professionals to Mitigate Risks and Navigate Ch...
 
Smartphone Forensic Challenges
Smartphone Forensic ChallengesSmartphone Forensic Challenges
Smartphone Forensic Challenges
 
Developing a Federal Vision for Identity Management
Developing a Federal Vision for Identity ManagementDeveloping a Federal Vision for Identity Management
Developing a Federal Vision for Identity Management
 
Intel HIMSS WoHIT mhealth
Intel HIMSS WoHIT mhealthIntel HIMSS WoHIT mhealth
Intel HIMSS WoHIT mhealth
 
Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...
Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...
Data Driven Disruption - Why Marketing and Advertising in WA lags - ADMA WA 2...
 
Improving Life With Connected Medical Devices
Improving Life With Connected Medical DevicesImproving Life With Connected Medical Devices
Improving Life With Connected Medical Devices
 
So, My FitBit is Clinical Trial Grade Right?
So, My FitBit is Clinical Trial Grade Right?So, My FitBit is Clinical Trial Grade Right?
So, My FitBit is Clinical Trial Grade Right?
 
Preparing Testimony about Cellebrite UFED In a Daubert or Frye Hearing
Preparing Testimony about Cellebrite UFED In a Daubert or Frye HearingPreparing Testimony about Cellebrite UFED In a Daubert or Frye Hearing
Preparing Testimony about Cellebrite UFED In a Daubert or Frye Hearing
 
Fast and fire-walled IOT healthcare-Baseer
Fast and fire-walled  IOT healthcare-BaseerFast and fire-walled  IOT healthcare-Baseer
Fast and fire-walled IOT healthcare-Baseer
 
Architecting, designing and building medical devices in an outcomes focused B...
Architecting, designing and building medical devices in an outcomes focused B...Architecting, designing and building medical devices in an outcomes focused B...
Architecting, designing and building medical devices in an outcomes focused B...
 
eBook-IoTPractice
eBook-IoTPracticeeBook-IoTPractice
eBook-IoTPractice
 
Big data analytics for life insurers
Big data analytics for life insurersBig data analytics for life insurers
Big data analytics for life insurers
 
Big_data_analytics_for_life_insurers_published
Big_data_analytics_for_life_insurers_publishedBig_data_analytics_for_life_insurers_published
Big_data_analytics_for_life_insurers_published
 
Enterprise Digital Writing
Enterprise Digital WritingEnterprise Digital Writing
Enterprise Digital Writing
 
497secondary
497secondary497secondary
497secondary
 
Practical Guide - www.devicematters.com
Practical Guide - www.devicematters.comPractical Guide - www.devicematters.com
Practical Guide - www.devicematters.com
 
Big data in IoT for healthcare - www.pepgra.com
Big data in IoT for healthcare - www.pepgra.comBig data in IoT for healthcare - www.pepgra.com
Big data in IoT for healthcare - www.pepgra.com
 
Trends in Wireless Working
Trends in Wireless WorkingTrends in Wireless Working
Trends in Wireless Working
 
AIMeetup #3: Cortana intelligence suite - tchnij życie w swoje dane
AIMeetup #3: Cortana intelligence suite - tchnij życie w swoje daneAIMeetup #3: Cortana intelligence suite - tchnij życie w swoje dane
AIMeetup #3: Cortana intelligence suite - tchnij życie w swoje dane
 

More from Databricks

Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
Databricks
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
Databricks
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
Databricks
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
Databricks
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch FixThe Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
Databricks
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
Databricks
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Databricks
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
Databricks
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Databricks
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature Aggregations
Databricks
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
Databricks
 
Raven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction QueriesRaven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction Queries
Databricks
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 
Machine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionMachine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack Detection
Databricks
 
Jeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and QualityJeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and Quality
Databricks
 
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + FugueIntuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
Databricks
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload Deployment
Databricks
 
Improving Apache Spark for Dynamic Allocation and Spot Instances
Improving Apache Spark for Dynamic Allocation and Spot InstancesImproving Apache Spark for Dynamic Allocation and Spot Instances
Improving Apache Spark for Dynamic Allocation and Spot Instances
Databricks
 
Importance of ML Reproducibility & Applications with MLfLow
Importance of ML Reproducibility & Applications with MLfLowImportance of ML Reproducibility & Applications with MLfLow
Importance of ML Reproducibility & Applications with MLfLow
Databricks
 

More from Databricks (20)

Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch FixThe Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature Aggregations
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
 
Raven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction QueriesRaven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction Queries
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
 
Machine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionMachine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack Detection
 
Jeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and QualityJeeves Grows Up: An AI Chatbot for Performance and Quality
Jeeves Grows Up: An AI Chatbot for Performance and Quality
 
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + FugueIntuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload Deployment
 
Improving Apache Spark for Dynamic Allocation and Spot Instances
Improving Apache Spark for Dynamic Allocation and Spot InstancesImproving Apache Spark for Dynamic Allocation and Spot Instances
Improving Apache Spark for Dynamic Allocation and Spot Instances
 
Importance of ML Reproducibility & Applications with MLfLow
Importance of ML Reproducibility & Applications with MLfLowImportance of ML Reproducibility & Applications with MLfLow
Importance of ML Reproducibility & Applications with MLfLow
 

Recently uploaded

Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Influencer Marketing Master Class - Alexis Andreasik
Influencer Marketing Master Class - Alexis AndreasikInfluencer Marketing Master Class - Alexis Andreasik
Influencer Marketing Master Class - Alexis Andreasik
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Efficient Website Management for Digital Marketing Pros
Efficient Website Management for Digital Marketing ProsEfficient Website Management for Digital Marketing Pros
Efficient Website Management for Digital Marketing Pros
Lauren Polinsky
 
Evaluating the Effectiveness of Women-Focused Marketing
Evaluating the Effectiveness of Women-Focused MarketingEvaluating the Effectiveness of Women-Focused Marketing
Evaluating the Effectiveness of Women-Focused Marketing
HighViz PR
 
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
5ys5mvlp
 
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdfTop Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
1Solutions Pvt. Ltd.
 
Mindfulness Techniques Cultivating Calm in a Chaotic World.pptx
Mindfulness Techniques Cultivating Calm in a Chaotic World.pptxMindfulness Techniques Cultivating Calm in a Chaotic World.pptx
Mindfulness Techniques Cultivating Calm in a Chaotic World.pptx
elizabethella096
 
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
Jorge Calmett
 
AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024
Amanda Farrell
 
Practical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven KingpdfPractical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven Kingpdf
william charnock
 
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
omywaf
 
原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样
原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样
原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样
5ys5mvlp
 
Mastering SEO for Google in the AI Era - Dennis Yu
Mastering SEO for Google in the AI Era - Dennis YuMastering SEO for Google in the AI Era - Dennis Yu
How to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.etHow to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.et
Y
 
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptxINTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
Giorgio Chiesa
 
Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...
Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...
Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Snapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdfSnapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdf
Eastern Online-iSURVEY
 
Embark on style journeys Indian clothing store denver guide.pptx
Embark on style journeys Indian clothing store denver guide.pptxEmbark on style journeys Indian clothing store denver guide.pptx
Embark on style journeys Indian clothing store denver guide.pptx
Omnama Fashions
 
How to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social CommerceHow to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social Commerce
Vbout.com
 

Recently uploaded (20)

Unleash the Power of Storytelling - Win Hearts, Change Minds, Get Results - R...
Unleash the Power of Storytelling - Win Hearts, Change Minds, Get Results - R...Unleash the Power of Storytelling - Win Hearts, Change Minds, Get Results - R...
Unleash the Power of Storytelling - Win Hearts, Change Minds, Get Results - R...
 
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge...
 
Influencer Marketing Master Class - Alexis Andreasik
Influencer Marketing Master Class - Alexis AndreasikInfluencer Marketing Master Class - Alexis Andreasik
Influencer Marketing Master Class - Alexis Andreasik
 
Efficient Website Management for Digital Marketing Pros
Efficient Website Management for Digital Marketing ProsEfficient Website Management for Digital Marketing Pros
Efficient Website Management for Digital Marketing Pros
 
Evaluating the Effectiveness of Women-Focused Marketing
Evaluating the Effectiveness of Women-Focused MarketingEvaluating the Effectiveness of Women-Focused Marketing
Evaluating the Effectiveness of Women-Focused Marketing
 
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
 
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdfTop Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
 
Mindfulness Techniques Cultivating Calm in a Chaotic World.pptx
Mindfulness Techniques Cultivating Calm in a Chaotic World.pptxMindfulness Techniques Cultivating Calm in a Chaotic World.pptx
Mindfulness Techniques Cultivating Calm in a Chaotic World.pptx
 
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
 
AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024
 
Practical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven KingpdfPractical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven Kingpdf
 
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
 
原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样
原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样
原版制作(Sunderland毕业证书)桑德兰大学毕业证录取通知书一模一样
 
Mastering SEO for Google in the AI Era - Dennis Yu
Mastering SEO for Google in the AI Era - Dennis YuMastering SEO for Google in the AI Era - Dennis Yu
Mastering SEO for Google in the AI Era - Dennis Yu
 
How to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.etHow to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.et
 
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptxINTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
 
Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...
Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...
Get Off the Bandwagon - Separating Digital Marketing Myths from Truth - Scott...
 
Snapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdfSnapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of May 2024-EOLiSurvey (EN).pdf
 
Embark on style journeys Indian clothing store denver guide.pptx
Embark on style journeys Indian clothing store denver guide.pptxEmbark on style journeys Indian clothing store denver guide.pptx
Embark on style journeys Indian clothing store denver guide.pptx
 
How to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social CommerceHow to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social Commerce
 

Data Lakehouse Symposium | Day 1 | Part 1