SlideShare a Scribd company logo
#LuminisDay @thiagoavadore
Thiago de Faria
DataOps & AI Lead @LINKIT
ML, TECHCRUNCH, HYPE,
AI AND NO IRONIES
#LuminisDay @thiagoavadore
AI STARTUPS.
LEGACY & THE REAL WORLD.
@thiagoavadore#LuminisDay
FROM WINTER TO HYPE.
A JOURNEY THROUGH KIMBALL,
SVM, DEEPBLUE & CLOUD.
#LuminisDay @thiagoavadore
GOOGLE AI MAKES PHONE CALLS??!!
CAN YOU DO SOMETHING LIKE THAT
USING THAT TENSORFLOW STUFF?
IT WOULD REALLY HELP US GET
MORE FUNDING.
- PERSON A, VP OF PRODUCT, LAST
WEEK
@thiagoavadore#LuminisDay
WAIT… CAN YOU ALSO DO THAT IN A
CONTAINER USING BLOCKCHAIN?
- PERSON B, ONE DAY
LATER
* PERSON B WAS APPOINTED VP OF PRODUCT
#LuminisDay @thiagoavadore
SO OBVIOUS! WE WILL CREATE AN AI
DEVOPS APPLICATION ON TOP OF
KUBERNETES RUNNING ON THE
CLOUD USING BLOCKCHAIN THAT
THE USER WILL EXPERIENCE WITH A
VR HEADSET AND THIS WILL BE OUR
DIGITAL TRANSFORMATION
DISRUPTIVE STRATEGY!
- PERSON C, YESTERDAY
* PERSON C IS THE NEW
CEO
@thiagoavadore#LuminisDay
MAKE COMPUTERS
CAPABLE OF DOING
THINGS THAT WHEN
DONE BY A HUMAN,
WOULD BE THOUGHT
TO REQUIRE
INTELLIGENCE
AI
#LuminisDay @thiagoavadore
MAKE MACHINES
FIND PATTERNS
WITHOUT
EXPLICITLY
PROGRAMMING
THEM TO DO SO
AI
MACHIN
E
LEARNIN
G
@thiagoavadore#LuminisDay
YOU KNOW
THE OUTPUTS
AI
MACHINE LEARNING
SUPERVIS
ED
LEARNING REGRESSION &
CLASSIFICATIO
N
#LuminisDay @thiagoavadore
EXPLORATORY
AI
MACHINE LEARNING
SUPERVI
SED
LEARNIN
G
CLUSTERING
UNSUPERVI
SED
LEARNING
NO LABELS
@thiagoavadore#LuminisDay
RETRO FEED THE
MODEL
ACCORDING TO A
REWARD
AI
MACHINE LEARNING
SUPERVI
SED
LEARNIN
G
DOESN’T RELY
ON A LABELED
DATASET
UNSUPERV
ISED
LEARNING
REINFORCE
MENT
LEARNING
#LuminisDay @thiagoavadore
AI
MACHINE LEARNING
SUPERVISED
LEARNING
UNSUPERVI
SED
LEARNING
REINFORCE
MENT
LEARNING
@thiagoavadore#LuminisDay
AI
MACHINE LEARNING
SUPERVISED
LEARNING
SVM, Decision
Trees, LinReg,
Neural
Networks, …
UNSUPERVISED
LEARNING
K-Means,
Association
Rules, Neural
Networks, …
REINFORCEMENT
LEARNING
Q-Learning,
Adversarial
Networks, …
#LuminisDay @thiagoavadore
NARROW
@thiagoavadore#LuminisDay
TRADITIONAL DEVELOPMENT
REVOLVES AROUND IF/THEN, LOOPS
AND TRADITIONAL TESTING
#LuminisDay @thiagoavadore
#LuminisDay @thiagoavadore
@thiagoavadore#LuminisDay
FEAR IS THE PATH
TO THE DARK SIDE
@thiagoavadore#LuminisDay
#LuminisDay @thiagoavadore
ML DEVELOPMENT REVOLVES
AROUND HYPERPARAMETERS AND
THE DATA PIPELINE
@thiagoavadore#LuminisDay
DATA
PIPELINE
MODEL DEVOPERATIONS
INGEST
EXPLORE
CLEAN
FEATURE
ENGINEERING
TRAIN
TEST
TUNE
PUBLISH
CONSUME
MONITOR
#LuminisDay @thiagoavadore
DATA PIPELINE
MODEL DEV
OPERATIONS
INGEST
EXPLORE
CLEAN
FEATURE
ENGINEERING
TRAIN
TEST
TUNE
PUBLISH
CONSUME
MONITOR
@thiagoavadore#LuminisDay
#LuminisDay @thiagoavadore
WT
F
@thiagoavadore#LuminisDay
DO NOT WATERFALL INTO IT.
DO NOT OVERPROMISE.
DAY ZERO = PRODUCTION READY!
#LuminisDay @thiagoavadore
80% IS NOT THAT HARD.
90% DEMANDS A BIT OF A TIME.
95% IS QUITE TRICKY.
99% IS AN HOMERIC TASK.
@thiagoavadore#LuminisDay
GARBAGE IN, GARBAGE OUT.
#LuminisDay @thiagoavadore
DATA PIPELINE IS DATA
ENGINEERING!
AND GUESS WHAT? SOFTWARE
ENGINEERING BEST PRACTICES
APPLY!
@thiagoavadore#LuminisDay
DATA
PIPELINE
MODEL DEVOPERATIONS
INGEST
EXPLORE
CLEAN
FEATURE
ENGINEERING
TRAIN
TEST
TUNE
PUBLISH
CONSUME
MONITOR
#LuminisDay @thiagoavadore
IT IS LIKELY THAT YOUR DATA
PIPELINE WILL DEPEND ON A JVM
@thiagoavadore#LuminisDay
DATA
PIPELINE
MODEL DEVOPERATIONS
INGEST
EXPLORE
CLEAN
FEATURE
ENGINEERING
TRAIN
TEST
TUNE
PUBLISH
CONSUME
MONITOR
#LuminisDay @thiagoavadore
PROBABLY ON PYTHON AND
COMPLETELY DEPENDENT ON
THE DATA
@thiagoavadore#LuminisDay
DATA SCIENTISTS ARE NOT
DEVELOPERS
PROVIDE A SAFE
ENVIRONMENT WITH THE DATA
THEY NEED
#LuminisDay @thiagoavadore
DATA HAS GRAVITY
@thiagoavadore#LuminisDay
DISCUSS WHAT TO TEST &
HOW TO TEST THE ML MODELS
AN ML MODEL SHOULD RELY
ON FUNCTIONS & ENDPOINTS
#LuminisDay @thiagoavadore
MODEL READY! VICTORY, RIGHT?
@thiagoavadore#LuminisDay
DATA
PIPELINE
MODEL DEVOPERATIONS
INGEST
EXPLORE
CLEAN
FEATURE
ENGINEERING
TRAIN
TEST
TUNE
PUBLISH
CONSUME
MONITOR
#LuminisDay @thiagoavadore
@thiagoavadore#LuminisDay
SYNC HOW TO SERVE THE MODEL
REST or SOAP or RPC
FEATURE ENGINEERING BEFORE?
#LuminisDay @thiagoavadore
PROBLEMS I HAVE SEEN(some I may have even
caused…)
DIFFERENT DATA
WRONG MODEL PUBLISHED
LOCAL FEATURE
ENGINEERING
MODEL DRIFT AND FRAGILITY
ENDPOINTS CAN’T HANDLE CALLS
@thiagoavadore#LuminisDay
THESE SUGGESTIONS ARE NOT A
SILVER BULLET, BUT THEY WILL
DECREASE YOUR TIME-TO-MARKET
AND SHORTEN YOUR FIX TIME
#LuminisDay @thiagoavadore
RETHINK AND BUILD A NEW CD
PIPELINE FOR ML MODELS
@thiagoavadore#LuminisDay
LOCAL DATA SCIENCE IS DEAD
#LuminisDay @thiagoavadore
TESTS WON’T SAVE YOUR MODEL
DRIFT OR PRODUCTION PROBLEMS
RELATED TO DATA… CHANGE YOUR
MIND AND KNOW, MISTAKES WILL
HAPPEN!
@thiagoavadore#LuminisDay
SPARK 2.3
[SPARK-
18278]
#LuminisDay @thiagoavadore
HADOOP 3.1 WILL HAVE DOCKER
CONTAINERS
@thiagoavadore#LuminisDay
#LuminisDay @thiagoavadore
Demo Time!
#LuminisDay @thiagoavadore
#LuminisDay @thiagoavadore
THANKS!!
GO TO
ITNEXT.IO

More Related Content

What's hot

Candy for everybody - APIDays Mediterranea 2015
Candy for everybody - APIDays Mediterranea 2015Candy for everybody - APIDays Mediterranea 2015
Candy for everybody - APIDays Mediterranea 2015
Alberto López Martín
 
[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober
[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober
[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober
Searchmetrics
 
Developing a Mobile Content Strategy That Works
Developing a Mobile Content Strategy That WorksDeveloping a Mobile Content Strategy That Works
Developing a Mobile Content Strategy That Works
Ahava Leibtag
 
A Software Engineer's Guide to DevOps
A Software Engineer's Guide to DevOpsA Software Engineer's Guide to DevOps
A Software Engineer's Guide to DevOps
Laurie Barth
 
Software Engineer's Guide to DevOps
Software Engineer's Guide to DevOpsSoftware Engineer's Guide to DevOps
Software Engineer's Guide to DevOps
Laurie Barth
 
ISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitswelt
ISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitsweltISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitswelt
ISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitswelt
ISARNETZ
 
Contributing to the success of open stack
Contributing to the success of open stackContributing to the success of open stack
Contributing to the success of open stack
Alexis Monville
 
Scrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindsetScrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindset
Michael Palotas
 
The Ongoing Reinvention of Shopping
The Ongoing Reinvention of ShoppingThe Ongoing Reinvention of Shopping
The Ongoing Reinvention of Shopping
sinnerschrader
 
How to Build and Drive Revenue Focused SaaS Marketing Operations
How to Build and Drive Revenue Focused SaaS Marketing OperationsHow to Build and Drive Revenue Focused SaaS Marketing Operations
How to Build and Drive Revenue Focused SaaS Marketing Operations
HelloMeets
 

What's hot (10)

Candy for everybody - APIDays Mediterranea 2015
Candy for everybody - APIDays Mediterranea 2015Candy for everybody - APIDays Mediterranea 2015
Candy for everybody - APIDays Mediterranea 2015
 
[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober
[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober
[US] 2015 Mobile Ranking Factors and Google Mobile Update - Marcus Tober
 
Developing a Mobile Content Strategy That Works
Developing a Mobile Content Strategy That WorksDeveloping a Mobile Content Strategy That Works
Developing a Mobile Content Strategy That Works
 
A Software Engineer's Guide to DevOps
A Software Engineer's Guide to DevOpsA Software Engineer's Guide to DevOps
A Software Engineer's Guide to DevOps
 
Software Engineer's Guide to DevOps
Software Engineer's Guide to DevOpsSoftware Engineer's Guide to DevOps
Software Engineer's Guide to DevOps
 
ISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitswelt
ISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitsweltISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitswelt
ISARCAMP: elance / Katrin Hippler - shareconomy in der arbeitswelt
 
Contributing to the success of open stack
Contributing to the success of open stackContributing to the success of open stack
Contributing to the success of open stack
 
Scrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindsetScrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindset
 
The Ongoing Reinvention of Shopping
The Ongoing Reinvention of ShoppingThe Ongoing Reinvention of Shopping
The Ongoing Reinvention of Shopping
 
How to Build and Drive Revenue Focused SaaS Marketing Operations
How to Build and Drive Revenue Focused SaaS Marketing OperationsHow to Build and Drive Revenue Focused SaaS Marketing Operations
How to Build and Drive Revenue Focused SaaS Marketing Operations
 

Similar to Luminis Session - ML, TechCrunch, Hype, AI and no ironies

devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...
devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...
devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...
Thiago de Faria
 
DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...
DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...
DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...
DevOpsDays Riga
 
IBM Tech Session: DevOps - 2009 in Ghent, 2018 in Hypeland
IBM Tech Session: DevOps - 2009 in Ghent, 2018 in HypelandIBM Tech Session: DevOps - 2009 in Ghent, 2018 in Hypeland
IBM Tech Session: DevOps - 2009 in Ghent, 2018 in Hypeland
Thiago de Faria
 
DevOpsDays Istanbul 2019 - AI with a devops mindset
DevOpsDays Istanbul 2019 - AI with a devops mindsetDevOpsDays Istanbul 2019 - AI with a devops mindset
DevOpsDays Istanbul 2019 - AI with a devops mindset
Thiago de Faria
 
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Codemotion
 
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
Thiago de Faria
 
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Codemotion
 
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
Thiago de Faria
 
Digital transformation
Digital transformationDigital transformation
Digital transformation
Scopernia
 
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
Thiago de Faria
 
AI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
AI with a DevOps Mindset - Experimentation, sharing & easy ML deploymentAI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
AI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
Thiago de Faria
 
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your CareerBecoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Andrea Goulet
 
Digital Transformation Before and After Seminar, 11th August, Melbourne.
Digital Transformation Before and After Seminar, 11th August, Melbourne. Digital Transformation Before and After Seminar, 11th August, Melbourne.
Digital Transformation Before and After Seminar, 11th August, Melbourne.
Precedent
 
The Corporate Startup
The Corporate StartupThe Corporate Startup
The Corporate Startup
Duval Union Consulting
 
The corporate startup
The corporate startupThe corporate startup
The corporate startup
Scopernia
 
devopsdays Warsaw 2018 - Chaos while deploying ML
devopsdays Warsaw 2018 - Chaos while deploying MLdevopsdays Warsaw 2018 - Chaos while deploying ML
devopsdays Warsaw 2018 - Chaos while deploying ML
Thiago de Faria
 
Forget disruption, it's time for Transformation
Forget disruption, it's time for TransformationForget disruption, it's time for Transformation
Forget disruption, it's time for Transformation
Scopernia
 
Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.
Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.
Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.
Citizen, Inc.
 
Digital Transformation in Governments
Digital Transformation in GovernmentsDigital Transformation in Governments
Digital Transformation in Governments
Scopernia
 
Understanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and FoundersUnderstanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and Founders
Andrea Goulet
 

Similar to Luminis Session - ML, TechCrunch, Hype, AI and no ironies (20)

devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...
devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...
devopsdays Kiel 2018 - Can the AI hype & ML algorithms harm your devops initi...
 
DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...
DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...
DevOpsDaysRiga 2018: Thiago de Faria - Chaos while deploying ML and making su...
 
IBM Tech Session: DevOps - 2009 in Ghent, 2018 in Hypeland
IBM Tech Session: DevOps - 2009 in Ghent, 2018 in HypelandIBM Tech Session: DevOps - 2009 in Ghent, 2018 in Hypeland
IBM Tech Session: DevOps - 2009 in Ghent, 2018 in Hypeland
 
DevOpsDays Istanbul 2019 - AI with a devops mindset
DevOpsDays Istanbul 2019 - AI with a devops mindsetDevOpsDays Istanbul 2019 - AI with a devops mindset
DevOpsDays Istanbul 2019 - AI with a devops mindset
 
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
 
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
 
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
Thiago de Faria - AI with a devops mindset - experimentation, sharing and eas...
 
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
 
Digital transformation
Digital transformationDigital transformation
Digital transformation
 
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
 
AI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
AI with a DevOps Mindset - Experimentation, sharing & easy ML deploymentAI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
AI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
 
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your CareerBecoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
 
Digital Transformation Before and After Seminar, 11th August, Melbourne.
Digital Transformation Before and After Seminar, 11th August, Melbourne. Digital Transformation Before and After Seminar, 11th August, Melbourne.
Digital Transformation Before and After Seminar, 11th August, Melbourne.
 
The Corporate Startup
The Corporate StartupThe Corporate Startup
The Corporate Startup
 
The corporate startup
The corporate startupThe corporate startup
The corporate startup
 
devopsdays Warsaw 2018 - Chaos while deploying ML
devopsdays Warsaw 2018 - Chaos while deploying MLdevopsdays Warsaw 2018 - Chaos while deploying ML
devopsdays Warsaw 2018 - Chaos while deploying ML
 
Forget disruption, it's time for Transformation
Forget disruption, it's time for TransformationForget disruption, it's time for Transformation
Forget disruption, it's time for Transformation
 
Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.
Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.
Internet of Things: The Next Bubble? a talk by Sce Pike, CXO, Citizen, Inc.
 
Digital Transformation in Governments
Digital Transformation in GovernmentsDigital Transformation in Governments
Digital Transformation in Governments
 
Understanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and FoundersUnderstanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and Founders
 

Recently uploaded

GT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma TranscriptGT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma Transcript
xmevus
 
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
Lucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall ServiceLucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall Service
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
veenita788
 
UW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma TranscriptUW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma Transcript
xmevus
 
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
parichopra4
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
rawankhanlove256
 
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
sankisogandhi
 
Chapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdf
Chapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdfChapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdf
Chapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdf
Adroit PMC
 
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
seenaoberoi
 
UCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma TranscriptUCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma Transcript
xmevus
 
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
kishanaaani
 
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
DrAdoGarba
 
UC Davis biyezheng degree offer diploma Transcript
UC Davis biyezheng degree offer diploma TranscriptUC Davis biyezheng degree offer diploma Transcript
UC Davis biyezheng degree offer diploma Transcript
xmevus
 
@ℂall Lucknow @Girls Chinhat 08630512678
@ℂall Lucknow  @Girls Chinhat 08630512678 @ℂall Lucknow  @Girls Chinhat 08630512678
@ℂall Lucknow @Girls Chinhat 08630512678
veenita788
 
VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...
sukaniyasunnu
 
Curtin Cert degree offer diploma
Curtin Cert degree offer diploma Curtin Cert degree offer diploma
Curtin Cert degree offer diploma
popecap
 
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
saroohilakhatariroy
 
Strategies for Adoption of SDGs in organizations
Strategies for Adoption of SDGs in organizationsStrategies for Adoption of SDGs in organizations
Strategies for Adoption of SDGs in organizations
Amgad Morgan
 
Flinders Cert degree offer diploma
Flinders Cert degree offer diploma Flinders Cert degree offer diploma
Flinders Cert degree offer diploma
popecap
 
2024-07-14 Transformed 07 (shared slides).pptx
2024-07-14 Transformed 07 (shared slides).pptx2024-07-14 Transformed 07 (shared slides).pptx
2024-07-14 Transformed 07 (shared slides).pptx
Dale Wells
 
Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...
Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...
Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...
seenaoberoi
 

Recently uploaded (20)

GT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma TranscriptGT biyezheng degree offer diploma Transcript
GT biyezheng degree offer diploma Transcript
 
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
Lucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall ServiceLucknow @Girls @ℂall  Gomti Nagar 08630512678  @Girls @ℂall Service
Lucknow @Girls @ℂall Gomti Nagar 08630512678 @Girls @ℂall Service
 
UW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma TranscriptUW biyezheng degree offer diploma Transcript
UW biyezheng degree offer diploma Transcript
 
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
Varanasi Girls Call Varanasi 0X0000000X Payment On Delevery Cash Hot Premium ...
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
Dehradun Girls Call Dehradun 0X0000000X Unlimited Short Providing Girls Servi...
 
Chapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdf
Chapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdfChapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdf
Chapter 1 - Comparsion of Dispute Resolution Technique - Reading Material.pdf
 
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
Mysore Girls Call Mysore 0X0000000X Payment On Delevery Cash Hot Premium Genu...
 
UCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma TranscriptUCI biyezheng degree offer diploma Transcript
UCI biyezheng degree offer diploma Transcript
 
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
Chandigarh Girls Call Chandigarh 0X0000000X Provide Best And Top Girl Service...
 
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
ANALYSIS OF LIVELIHOOD DIVERSIFICATION STRATEGIES AMONG WOMEN CROP FARMERS IN...
 
UC Davis biyezheng degree offer diploma Transcript
UC Davis biyezheng degree offer diploma TranscriptUC Davis biyezheng degree offer diploma Transcript
UC Davis biyezheng degree offer diploma Transcript
 
@ℂall Lucknow @Girls Chinhat 08630512678
@ℂall Lucknow  @Girls Chinhat 08630512678 @ℂall Lucknow  @Girls Chinhat 08630512678
@ℂall Lucknow @Girls Chinhat 08630512678
 
VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Shimla Girls Call Shimla 0X0000000X Doorstep High-Profile Girl Service Ca...
 
Curtin Cert degree offer diploma
Curtin Cert degree offer diploma Curtin Cert degree offer diploma
Curtin Cert degree offer diploma
 
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
VIP Nashik Girls Call Nashik 0X0000000X Doorstep High-Profile Girl Service Ca...
 
Strategies for Adoption of SDGs in organizations
Strategies for Adoption of SDGs in organizationsStrategies for Adoption of SDGs in organizations
Strategies for Adoption of SDGs in organizations
 
Flinders Cert degree offer diploma
Flinders Cert degree offer diploma Flinders Cert degree offer diploma
Flinders Cert degree offer diploma
 
2024-07-14 Transformed 07 (shared slides).pptx
2024-07-14 Transformed 07 (shared slides).pptx2024-07-14 Transformed 07 (shared slides).pptx
2024-07-14 Transformed 07 (shared slides).pptx
 
Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...
Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...
Vadodara Girls Call Vadodara 0X0000000X Payment On Delevery Cash Hot Premium ...
 

Luminis Session - ML, TechCrunch, Hype, AI and no ironies

Editor's Notes

  1. Starting with AI from scratch, as a startup is a different problem… as a startup you can shape everything from beginning and you will have scaling issues as you cut corner to deliver a problem, satisfy your own issues and whatever comes from it… that is cool and not my talk. The majority of us, need to handle with legacy, changes in direction and decisions that are still not taken by us engineers at some point. Why? Because we are in the middle of the hype! But wasn’t AI always a hype.
  2. But first, what is AI? The belle époque of AI - 50’s and 60’s there was a lot of investment and growth. Solved high school algebra and geometry problems. The first steps in NLP were made here and a researches said “machines will be capable, within twenty years, of doing any work a man can do.” It looks like a techcrunch article from today right? But in the end of 70s this optimist paid back and it was faded as a scifi product, losing investment. Computer power and? What else? Data, labeled data, were holding them back… So we had the first AI WINTER! On the 80s investment returned with a lot of money from the Japanese government and it lasted less time. The embodiment of an AI raised and some research continued to build the hype in 90s… The AI research was coined and very much limited by the media as something far away… even though Deep Blue’s success. But there were companies starting with something that would change it all in the 200s: BI, data warehouses, MCMC and gathering data. The fancy models were being replaced by statistics… SVM. Again, decision theory and maximising success was the core thought behind. In the 2000’s until now something changed? What? Commodification of storage and CPU led to Cloud computing and access to large amounts of data. Cloud and BIG DATA!!! Deep Learning and multiple layers of neural networks that were unthinkable to run in the 50’s were being used, improved and tested. This led to AlphaGo, translate, chatbots, image recognition and self driving cars. Now again, we are in a booming era of AI… TechCrunch and regular media cover everything we do and release stories about. This sells and make me listen the most awkward requests that I narrowed down to this.
  3. The hype is here and you will see this happening more and more…
  4. And also this… with that person being the one that decides the products you build. But joke aside, you are probably working with something called AI, using ML. If not, there is a big chance that something with it will be in your PROD in the upcoming months… So first, let’s try to clarify some concepts and understand what is AI, ML, DL and cases that you should use it!
  5. And also this… with that person being the one that decides the products you build. But joke aside, you are probably working with something called AI, using ML. If not, there is a big chance that something with it will be in your PROD in the upcoming months… So first, let’s try to clarify some concepts and understand what is AI, ML, DL and cases that you should use it!
  6. And also this… with that person being the one that decides the products you build. But joke aside, you are probably working with something called AI, using ML. If not, there is a big chance that something with it will be in your PROD in the upcoming months… So first, let’s try to clarify some concepts and understand what is AI, ML, DL and cases that you should use it!
  7. Narrow AI is the only form of Artificial Intelligence that humanity has achieved so far.  good at performing a single task, such as playing chess or Go, Computer vision — seeing stuff and image tagging Natural language processing is the start… automatic translation is happening. You have NLG, imagine how brilliant that will be. Speech recognition Personal assistants and all their use Self-driving car technology is still considered a type of narrow AI, or more precisely, a coordination of several narrow AIs. In essence, narrow AI works within a very limited context, and can’t take on tasks beyond its field. So you can’t expect the same engine that transcripts audio and video files to, say, order pizza for you. That’s the task of another AI. Narrow AI is sometimes also referred to as “weak AI.” However, that doesn’t mean that narrow AI is inefficient. In contrary, it is very good at routine jobs, both physical and cognitive. It’s narrow AI that is threatening to replace (or rather displace) many human jobs. And it’s narrow AI that can ferret out patterns and correlations from data that would take eons for humans to find. But it’s still not human-level AI. That’s the stuff of Artificial General Intelligence.
  8. Let’s make a small comparison between our regular software engineering systems and ML systems… The majority of the software, coding and scripting done is a succession of events that will lead to an output. In an agile way you see the user stories: as someone I would like to see this and that, the PO breaks it down to small stories and you can start your TDD approach and come out with a workable version of that piece. You use functions, loops, catches and exceptions to pinpoint some possible outcomes.
  9. In old school algorithm classes when I studied math I remember having to think of possible algorithms for doing some things and break down in this graphs. You have a binary decision if it is True you start a sequence of processes/functions and return something… if false you check another binary function and you move on into it until you covered your possibilities. You pick this algorithm and start coding in the language you would like to, test it and move on with your life. There will be bugs because somethings were not considered and when you start your debugging process you will see one thing. You will be able to pinpoint where on this process the bug resides… you can isolate points.
  10. This ability to pinpoint where the problems are caused and still caused a lot of fear. A fear of the unknown… a fear that if you code something that causes a problem to your company you can be pinpointed… after all, your code generated the mistake. So, to avoid mistakes let’s plan everything and do everything by the book… let’s put the managers and CEO and PO’s responsible for everything… why? Because I am scared to break stuff. The Devops cultural movement came here to show that NO, that is not the way!
  11. Going fast and breaking things fast is the way to go! That we should go fast, break stuff and fix it… that everyone makes mistakes… that you can put yourself in the other position. That you can be empathic and constructive! That is the Continuous Delivery revolution and a lot of people mistake that with Devops… CD is a “frameworks/tools” that together with blameless post-mortem, engagement, freedom and ownership will generate an avalanche of well-being at your team and environment!
  12. If you want to see the data an impact of CD, blameless and other things to see how impactful that can be! Check it out this amazing book!
  13. But how an ML algorithm works? First, why we use a ML algorithm? The most important reason is: the desired behavior cannot be effectively implemented in software logic without dependency on external data. Unless you are developing a new type of algorithm, that I think only in PhD levels or new implementations of paper things you will do, you will be using pre built algorithms. Tensor flow, keras, caffe, Paddle and other frameworks designed to do so. If you will use Support Vector machines, General linear models, ensemble or deep learning… these are pre built here. Of course you can go into the C++ level of tensor flow and help it optimise it, but the majority of us do not go into that layer. It is like saying you do something in terraform… you change parameters in the config file, you build different flows but not everyone goes into the real implementation behind a terraform plan… you know how it does and tweak it. In ML you have lots of hyperparameters according to the algorithm: learning rates, folds, distance measures… you can play with activation functions, layers, loss functions, how to divide the set, how to compare the train… Let’s say you come with a deep learning model that identifies and caption images… you have an astonishing 98% accuracy on test, successful deploy it and you start getting feedback from your users… But how is the process here?
  14. But let’s look to a bigger picture. When you see a tutorial on deep learning, there is a big chance you will get the mnist dataset… the handwritten digits data and start on your model. Or your imagine data for images and so on… but that is really far from the real life. On a real work you have data scattered from public sources, your own application, things you built or use… you have to build your ingestion mechanism, extract that data so you can prepare it. After that, you need to iterate on it. Explore, Clean, Feature Engineer and then you have a data set to use for your mode development. There you will select the models to train, test and tune it… changing the model or the hyper parameters, so you can publish this! In this deployment part you need to know the model will be consumed, monitor the health of this serving structure and gather info about usage and new data to be added your model so you can start it over again.
  15. You have all the regular issues here! Legacy applications Different API’s that will change Specific silos of knowledge that are not mapped Business requirements that may change and political fights. Yup…. If you think that being a developer is just pushing code to git, you are doing it wrong… But more than that… you have the fear! The fear of deployment! Deploying it and making it a part of a running ecosystem is a messy terrible job!
  16. And sometimes you is just the engineer that wants to get things done… you see in one side, you have the (BATMAN IMAGE) night paladin. The dark knight that works alone at night, that knows how the packaging systems works and misses the time where people were not trying to go to that unsafe place: THE CLOUD. On the other hands, you have the people’s hero… the one that convinced that CEO that AI would save the world and stop the company running on RED {SUPERMAN} The PO/VP OF Product/PRODUCT MANAGER and always smiling proposing team outings and drinks and fun. The one thing that he secretly hates is to find out. These 2 characters clash, and you are usually and the middle here. Trying to find a balance between not going breaking something, or doing something not GDPR and being called out and on the other hands, not being the grumpy person that always says no. After all… you just want to build stuff and add value right?
  17. And sometimes you is just the engineer that wants to get things done… you see in one side, you have the (BATMAN IMAGE) night paladin. The dark knight that works alone at night, that knows how the packaging systems works and misses the time where people were not trying to go to that unsafe place: THE CLOUD. On the other hands, you have the people’s hero… the one that convinced that CEO that AI would save the world and stop the company running on RED {SUPERMAN} The PO/VP OF Product/PRODUCT MANAGER and always smiling proposing team outings and drinks and fun. The one thing that he secretly hates is to find out. These 2 characters clash, and you are usually and the middle here. Trying to find a balance between not going breaking something, or doing something not GDPR and being called out and on the other hands, not being the grumpy person that always says no. After all… you just want to build stuff and add value right?
  18. So don’t go into the traps… do not waterfall into it and try to predict all the things that happen! You can’t do this… There are unknowns unknowns in every project and Production is a mystery land that we do not know! At the same time… this is somewhat uncharted territory. Until you have the data, build the initial drafts of the model and release into the wild you do not know how much this willl work… so do not overpromise!! But that cannot stop you to build and help data scientists do things that are ready from day zero!
  19. Because when we are talking about ML models, it is something like this:
  20. But at the same time, as we saw before, the data pipeline and ingestion is the best thing you will have. Sometimes you will get companies that have 10 years of data… exabytes of it! But they changed databases, added applications, deprecated columns, changed important columns variable, discontinued some things, pivoted their business model… until you can get a consistency data set…
  21. That means that you can use regular CD, built modularised, optimise the performance, do Unit Tests, integration Tests, performance tests…
  22. The majority of the apache projects related to data pipelines are built on top of the JVM. It is likely you will Kafka or something similar to it for your event bus. It is likely that you will use other tools like Fink, Storm, Nifi, Spark core, Spark Streaming on it. All these tools have API’s and you can use regular TDD, Unit Tests and all of it… that is not a problem. The problem resides when your data scientist, that is not aware of testing and making test an integral part of your pipeline, the one that handles this! So one thing… building a data pipeline? Use Software Engineering standards!
  23. Data is MASTER. Data is KING. GARBAGE IN, OUT. All these things happen a lot here and some things differ a lot here.
  24. Don’t force a Python code from a data scientist to run on your new brand kubernetes cluster on the cloud if the data source is still on-premises… IO, pushing data along it all it does not scale. Consider all that when doing your pipeline and providing an environment for the data scientists.
  25. Lots of data scientists will extract data from a DWH system, work on their things and after weeks they will present this amazing model to the CEO. 99% accuracy. The CEO schedule a Press Release and announce to the world that we embraced AI. Victory, right? Everybody is happy… but a model ready on a Python notebook is not a industrialised model that can be published and serve stuff in a constant way… {GO BACK AND SHOW PUBLISH} You need to overcome this always…
  26. Data is MASTER. Data is KING. GARBAGE IN, OUT. All these things happen a lot here and some things differ a lot here.
  27. Lots of data scientists will extract data from a DWH system, work on their things and after weeks they will present this amazing model to the CEO. 99% accuracy. The CEO say
  28. Lots of data scientists will extract data from a DWH system, work on their things and after weeks they will present this amazing model to the CEO. 99% accuracy. The CEO say