SlideShare a Scribd company logo

Introduction to Data Science

Niko Vuokko
Niko Vuokko
Niko VuokkoChief of Data at Sharper Shape

Two hour lecture I gave at the Jyväskylä Summer School. The purpose of the talk is to give a quick non-technical overview of concepts and methodologies in data science. Topics include a wide overview of both pattern mining and machine learning. See also Part 2 of the lecture: Industrial Data Science. You can find it in my profile (click the face)

Introduction to Data Science

1 of 31
Download to read offline
INTRODUCTION TO DATA SCIENCE
NIKO VUOKKO
JYVÄSKYLÄ SUMMER SCHOOL
AUGUST 2013
DATA SCIENCE WITH A BROAD BRUSH
Concepts and methodologies
DATA SCIENCE IS AN UMBRELLA, A FUSION
• Databases and infrastructure
• Pattern mining
• Statistics
• Machine learning
• Numerical optimization
• Stochastic modeling
• Data visualization
… of specialties needed
for data-driven
business optimization
DATA SCIENTIST
• Data scientist is defined as DS : business problem  data solution
• Combination of strong programming, math, computational and business skills
• Recipe for success
1. Convert vague business requirements into measurable technical targets
2. Develop a solution to reach the targets
3. Communicate business results
4. Deploy the solution in production
UNDERSTANDING DATA
Monday 19 August 2013
PATTERN MINING AND DATA ANALYSIS
Ad

Recommended

How to Become a Data Scientist
How to Become a Data ScientistHow to Become a Data Scientist
How to Become a Data Scientistryanorban
 
What Is Data Science? | Introduction to Data Science | Data Science For Begin...
What Is Data Science? | Introduction to Data Science | Data Science For Begin...What Is Data Science? | Introduction to Data Science | Data Science For Begin...
What Is Data Science? | Introduction to Data Science | Data Science For Begin...Simplilearn
 
Introduction of Data Science
Introduction of Data ScienceIntroduction of Data Science
Introduction of Data ScienceJason Geng
 
Data Science Tutorial | Introduction To Data Science | Data Science Training ...
Data Science Tutorial | Introduction To Data Science | Data Science Training ...Data Science Tutorial | Introduction To Data Science | Data Science Training ...
Data Science Tutorial | Introduction To Data Science | Data Science Training ...Edureka!
 
Data science & data scientist
Data science & data scientistData science & data scientist
Data science & data scientistVijayMohan Vasu
 
introduction to data science
introduction to data scienceintroduction to data science
introduction to data sciencebhavesh lande
 
Introduction to data science.pptx
Introduction to data science.pptxIntroduction to data science.pptx
Introduction to data science.pptxSadhanaParameswaran
 

More Related Content

What's hot

Demystifying data engineering
Demystifying data engineeringDemystifying data engineering
Demystifying data engineeringThang Bui (Bob)
 
Introduction to data science club
Introduction to data science clubIntroduction to data science club
Introduction to data science clubData Science Club
 
What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...
What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...
What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...Edureka!
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data scienceSampath Kumar
 
Big Data [sorry] & Data Science: What Does a Data Scientist Do?
Big Data [sorry] & Data Science: What Does a Data Scientist Do?Big Data [sorry] & Data Science: What Does a Data Scientist Do?
Big Data [sorry] & Data Science: What Does a Data Scientist Do?Data Science London
 
Introduction on Data Science
Introduction on Data ScienceIntroduction on Data Science
Introduction on Data ScienceEdureka!
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data ScienceSrishti44
 
Data Science Introduction
Data Science IntroductionData Science Introduction
Data Science IntroductionGang Tao
 
The Evolution of Data Science
The Evolution of Data ScienceThe Evolution of Data Science
The Evolution of Data ScienceKenny Daniel
 
Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...
Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...
Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...Edureka!
 
Introduction To Data Science
Introduction To Data ScienceIntroduction To Data Science
Introduction To Data ScienceSpotle.ai
 
Data science presentation
Data science presentationData science presentation
Data science presentationMSDEVMTL
 
Data Science Training | Data Science Tutorial | Data Science Certification | ...
Data Science Training | Data Science Tutorial | Data Science Certification | ...Data Science Training | Data Science Tutorial | Data Science Certification | ...
Data Science Training | Data Science Tutorial | Data Science Certification | ...Edureka!
 
Data science | What is Data science
Data science | What is Data scienceData science | What is Data science
Data science | What is Data scienceShilpaKrishna6
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data ScienceEdureka!
 
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...Edureka!
 

What's hot (20)

Demystifying data engineering
Demystifying data engineeringDemystifying data engineering
Demystifying data engineering
 
Introduction to data science club
Introduction to data science clubIntroduction to data science club
Introduction to data science club
 
What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...
What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...
What Is Data Science? Data Science Course - Data Science Tutorial For Beginne...
 
Data science
Data scienceData science
Data science
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data science
 
Big Data [sorry] & Data Science: What Does a Data Scientist Do?
Big Data [sorry] & Data Science: What Does a Data Scientist Do?Big Data [sorry] & Data Science: What Does a Data Scientist Do?
Big Data [sorry] & Data Science: What Does a Data Scientist Do?
 
Introduction on Data Science
Introduction on Data ScienceIntroduction on Data Science
Introduction on Data Science
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Data Science Introduction
Data Science IntroductionData Science Introduction
Data Science Introduction
 
Data science
Data scienceData science
Data science
 
The Evolution of Data Science
The Evolution of Data ScienceThe Evolution of Data Science
The Evolution of Data Science
 
Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...
Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...
Data Science For Beginners | Who Is A Data Scientist? | Data Science Tutorial...
 
Introduction To Data Science
Introduction To Data ScienceIntroduction To Data Science
Introduction To Data Science
 
Data science presentation
Data science presentationData science presentation
Data science presentation
 
Data Science Training | Data Science Tutorial | Data Science Certification | ...
Data Science Training | Data Science Tutorial | Data Science Certification | ...Data Science Training | Data Science Tutorial | Data Science Certification | ...
Data Science Training | Data Science Tutorial | Data Science Certification | ...
 
Data science | What is Data science
Data science | What is Data scienceData science | What is Data science
Data science | What is Data science
 
Data science
Data scienceData science
Data science
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Data science - An Introduction
Data science - An IntroductionData science - An Introduction
Data science - An Introduction
 
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
 

Similar to Introduction to Data Science

Improving AI Development - Dave Litwiller - Jan 11 2022 - Public
Improving AI Development - Dave Litwiller - Jan 11 2022 - PublicImproving AI Development - Dave Litwiller - Jan 11 2022 - Public
Improving AI Development - Dave Litwiller - Jan 11 2022 - PublicDave Litwiller
 
Choosing a Machine Learning technique to solve your need
Choosing a Machine Learning technique to solve your needChoosing a Machine Learning technique to solve your need
Choosing a Machine Learning technique to solve your needGibDevs
 
Industrial Data Science
Industrial Data ScienceIndustrial Data Science
Industrial Data ScienceNiko Vuokko
 
Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013
Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013
Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013Neo4j
 
Lecture 2 Data mining process.pdf
Lecture 2 Data mining process.pdfLecture 2 Data mining process.pdf
Lecture 2 Data mining process.pdfKaushik Kundu
 
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)MAHIRA
 
The zen of predictive modelling
The zen of predictive modellingThe zen of predictive modelling
The zen of predictive modellingQuinton Anderson
 
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...Lucas Jellema
 
"Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f...
"Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f..."Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f...
"Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f...Edge AI and Vision Alliance
 
02-Lifecycle.pptx
02-Lifecycle.pptx02-Lifecycle.pptx
02-Lifecycle.pptxShree Shree
 
MACHINE LEARNING YEAR DL SECOND PART.pptx
MACHINE LEARNING YEAR DL SECOND PART.pptxMACHINE LEARNING YEAR DL SECOND PART.pptx
MACHINE LEARNING YEAR DL SECOND PART.pptxNAGARAJANS68
 
Machine Learning in the Financial Industry
Machine Learning in the Financial IndustryMachine Learning in the Financial Industry
Machine Learning in the Financial IndustrySubrat Panda, PhD
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptxharikaramisetty3
 
Large Scale Modeling Overview
Large Scale Modeling OverviewLarge Scale Modeling Overview
Large Scale Modeling OverviewFerris Jumah
 
Unit 3 part ii Data mining
Unit 3 part ii Data miningUnit 3 part ii Data mining
Unit 3 part ii Data miningDhilsath Fathima
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptxAbderrahmanABID2
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptxsnigdhaagrawal11
 

Similar to Introduction to Data Science (20)

Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Improving AI Development - Dave Litwiller - Jan 11 2022 - Public
Improving AI Development - Dave Litwiller - Jan 11 2022 - PublicImproving AI Development - Dave Litwiller - Jan 11 2022 - Public
Improving AI Development - Dave Litwiller - Jan 11 2022 - Public
 
Choosing a Machine Learning technique to solve your need
Choosing a Machine Learning technique to solve your needChoosing a Machine Learning technique to solve your need
Choosing a Machine Learning technique to solve your need
 
Industrial Data Science
Industrial Data ScienceIndustrial Data Science
Industrial Data Science
 
Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013
Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013
Neo4j Theory and Practice - Tareq Abedrabbo @ GraphConnect London 2013
 
Lecture 2 Data mining process.pdf
Lecture 2 Data mining process.pdfLecture 2 Data mining process.pdf
Lecture 2 Data mining process.pdf
 
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
 
The zen of predictive modelling
The zen of predictive modellingThe zen of predictive modelling
The zen of predictive modelling
 
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
 
"Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f...
"Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f..."Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f...
"Solving Vision Tasks Using Deep Learning: An Introduction," a Presentation f...
 
02-Lifecycle.pptx
02-Lifecycle.pptx02-Lifecycle.pptx
02-Lifecycle.pptx
 
MACHINE LEARNING YEAR DL SECOND PART.pptx
MACHINE LEARNING YEAR DL SECOND PART.pptxMACHINE LEARNING YEAR DL SECOND PART.pptx
MACHINE LEARNING YEAR DL SECOND PART.pptx
 
Machine Learning in the Financial Industry
Machine Learning in the Financial IndustryMachine Learning in the Financial Industry
Machine Learning in the Financial Industry
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptx
 
Large Scale Modeling Overview
Large Scale Modeling OverviewLarge Scale Modeling Overview
Large Scale Modeling Overview
 
Ml2 production
Ml2 productionMl2 production
Ml2 production
 
Unit 3 part ii Data mining
Unit 3 part ii Data miningUnit 3 part ii Data mining
Unit 3 part ii Data mining
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptx
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptx
 
MachineLearningSparkML.pptx
MachineLearningSparkML.pptxMachineLearningSparkML.pptx
MachineLearningSparkML.pptx
 

More from Niko Vuokko

Analytics in business
Analytics in businessAnalytics in business
Analytics in businessNiko Vuokko
 
Drones in real use
Drones in real useDrones in real use
Drones in real useNiko Vuokko
 
Analytiikka bisneksessä
Analytiikka bisneksessäAnalytiikka bisneksessä
Analytiikka bisneksessäNiko Vuokko
 
Sensor Data in Business
Sensor Data in BusinessSensor Data in Business
Sensor Data in BusinessNiko Vuokko
 
Sensoridatan ja liiketoiminnan tulevaisuus
Sensoridatan ja liiketoiminnan tulevaisuusSensoridatan ja liiketoiminnan tulevaisuus
Sensoridatan ja liiketoiminnan tulevaisuusNiko Vuokko
 
Metrics @ App Academy
Metrics @ App AcademyMetrics @ App Academy
Metrics @ App AcademyNiko Vuokko
 
Big Data Rampage
Big Data RampageBig Data Rampage
Big Data RampageNiko Vuokko
 

More from Niko Vuokko (7)

Analytics in business
Analytics in businessAnalytics in business
Analytics in business
 
Drones in real use
Drones in real useDrones in real use
Drones in real use
 
Analytiikka bisneksessä
Analytiikka bisneksessäAnalytiikka bisneksessä
Analytiikka bisneksessä
 
Sensor Data in Business
Sensor Data in BusinessSensor Data in Business
Sensor Data in Business
 
Sensoridatan ja liiketoiminnan tulevaisuus
Sensoridatan ja liiketoiminnan tulevaisuusSensoridatan ja liiketoiminnan tulevaisuus
Sensoridatan ja liiketoiminnan tulevaisuus
 
Metrics @ App Academy
Metrics @ App AcademyMetrics @ App Academy
Metrics @ App Academy
 
Big Data Rampage
Big Data RampageBig Data Rampage
Big Data Rampage
 

Recently uploaded

2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch TuesdayIvanti
 
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfThomas Poetter
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!XfilesPro
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, GoogleISPMAIndia
 
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEDNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEandreiandasan
 
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)Memory Fabric Forum
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERNRonnelBaroc
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stackSummit
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfDomotica daVinci
 
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxEvolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxKyle Willson
 
Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Daniel Toomey
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfDomotica daVinci
 
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Product School
 
Relationship Counselling: From Disjointed Features to Product-First Thinking
Relationship Counselling: From Disjointed Features to Product-First ThinkingRelationship Counselling: From Disjointed Features to Product-First Thinking
Relationship Counselling: From Disjointed Features to Product-First ThinkingProduct School
 
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes", Volodymyr TsapFwdays
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfIsidro Navarro
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...shaiyuvasv
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupMemory Fabric Forum
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfDomotica daVinci
 

Recently uploaded (20)

2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch Tuesday
 
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
 
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEDNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
 
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stack
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
 
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxEvolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
 
Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
 
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
 
Relationship Counselling: From Disjointed Features to Product-First Thinking
Relationship Counselling: From Disjointed Features to Product-First ThinkingRelationship Counselling: From Disjointed Features to Product-First Thinking
Relationship Counselling: From Disjointed Features to Product-First Thinking
 
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdf
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product Lineup
 
5 Tech Trend to Notice in ESG Landscape- 47Billion
5 Tech Trend to Notice in ESG Landscape- 47Billion5 Tech Trend to Notice in ESG Landscape- 47Billion
5 Tech Trend to Notice in ESG Landscape- 47Billion
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdf
 

Introduction to Data Science

  • 1. INTRODUCTION TO DATA SCIENCE NIKO VUOKKO JYVÄSKYLÄ SUMMER SCHOOL AUGUST 2013
  • 2. DATA SCIENCE WITH A BROAD BRUSH Concepts and methodologies
  • 3. DATA SCIENCE IS AN UMBRELLA, A FUSION • Databases and infrastructure • Pattern mining • Statistics • Machine learning • Numerical optimization • Stochastic modeling • Data visualization … of specialties needed for data-driven business optimization
  • 4. DATA SCIENTIST • Data scientist is defined as DS : business problem  data solution • Combination of strong programming, math, computational and business skills • Recipe for success 1. Convert vague business requirements into measurable technical targets 2. Develop a solution to reach the targets 3. Communicate business results 4. Deploy the solution in production
  • 6. PATTERN MINING AND DATA ANALYSIS
  • 7. UNSUPERVISED LEARNING • Could be called pattern recognition or structure discovery • What kind of a process could have produced this data? • Discovery of “interesting” phenomena in a dataset • Now how do you define interesting? • Learning algorithms exist for a huge collection of pattern types • Analogy: You decide if you want to see westerns or comedies, but the machine picks the movies • But does “interesting” imply useful and significant?
  • 8. EXAMPLES OF STRUCTURES IN DATA • Clustering and mixture models: separation of data into parts • Dictionary learning: a compact grammar of the dataset • Single class learning: learn the natural boundaries of data Example: Early detection of machine failure or network intrusion • Latent allocation: learn hidden preferences driving purchase decisions • Source separation: find independent generators of the data Example: Independent phenomena affecting exchange rates
  • 9. MORE EXAMPLES OF “INTERESTING” PATTERNS • { charcoal, mustard } ⇒ sausage • Grocery customer types with differing paths around the trading floor • Pricing trend change in a web ad exchange • Communities and topics in a social network • Distinct features of a person’s face and fingerprints • Objects emerging in front of a moving car
  • 10. KNOW YOUR EIGENS AND SINGULARS • Eigenvalue and singular value decompositions are central data analysis tools • They describe the energy distribution and static core structures of data Examples • Face detection, speaker adaptation • Google PageRank is basically just the world’s largest EVD • Zombie outbreak risk is determined by its eigenvalues • As a sub-component in every second learning algorithm
  • 11. DIMENSION REDUCTION • Some applications encounter large dimension counts up to millions • Dimension reduction may either 1. Retain space: preserve the most “descriptive” dimensions 2. Transform space: trade interpretability for powerful rendition • Usually transformations work oblivious to data (they are simple functions) • Curvilinear transformations try to see how the data is “folded” and build new dimensions specific to the given dataset
  • 12. DIMENSION REDUCTION EXAMPLE • Singular value decomposition is commonly used to remove the “noise dimensions” with little energy • Example: gene expression data and movie preferences have lots of these • After this more complex methods can be used for unfolding the data
  • 14. BLIND SOURCE SEPARATION • Find latent sources that generated the data • Tries to discover the real truth beneath all noise and convolution • Examples: • Air defense missile guidance systems • Error-correcting codes • Language modeling • Brain activity factors • Industrial process dynamics • Factors behind climate change
  • 15. (STATISTICAL) SIGNIFICANCE TESTING • Example: Rejection rate increase in a manufacturing plant • “What is the probability of observing this increase if everything was OK?” • “What is the probability of having a valid alert if there really was something wrong?” • Reliability of significance testing results is wholly dependent on correct modeling of the data source and pattern type • Statistical significance is different from material significance
  • 16. CORRELATION IS NOT CAUSALITY A correlation may hide an almost arbitrary truth • Cities with more firemen have more fires • Companies spending more in marketing have higher revenues • Marsupials exist mainly in Australia • However, making successful predictions does not require causality
  • 18. SUPERVISED LEARNING • Simplistically task is to find function f : f(input) = output • Examples: spam filtering, speech recognition, steel strength estimation • Risks for different types of errors can be very skewed • Complex inputs may confuse or slow down models • Unsupervised methods often useful in improving results by simplifying the input
  • 19. SEMI-SUPERVISED LEARNING • Only a part of data is labeled • Needed when labeling data is expensive • Understanding the structure of unlabeled data enhances learning by bringing diversity and generalization and by constraining learning • Relates to multi-source learning, some sources labeled, some not • Examples: • Object detection from a video feed • Web page categorization • Sentiment analysis • Transfer learning between domains
  • 20. TRAINING, TESTING, VALIDATION • A model is trained using a training dataset • The quality of the model is measured by using it on a separate testing dataset • A model often contains hyper-parameters chosen by the user • A separate validation dataset is split off from the training data • Validation data is used for testing and finding good hyper-parameter values • Cross-validation is common practice and asymptotically unbiased
  • 21. BIAS AND VARIANCE • Squared error of predictions consists of bias and variance (and noise) • BIAS Model incapability of approximating the underlying truth • VARIANCE Model reliance on whims of the observed data • Complex models often have low bias and high variance • Simple models often have high bias and low variance • Having more data instances (rows) may reduce variance • Having more detailed data (variables) may reduce bias • Testing different types of models can explain how to improve your data
  • 22. TRAINING AND TESTING, BIAS AND VARIANCE Complex modelSimple model Minimal testing error Minimal training error
  • 24. THE KERNEL TRICK • Many learning methods rely on inner products of data points • The “kernel trick” maps the data to an implicitly defined, high dimension space • Kernel is the matrix of the new inner products in this space • Mapping itself often left unknown • Example: Gaussian kernel associates local Euclidean neighborhoods to similarity • Example: String kernels are used for modeling DNA sequence structure • Kernels can be combined and custom built to match expert knowledge A kernel is a dataset-specific space transformation, success depends on good understanding of the dataset
  • 25. ENSEMBLE LEARNING • The power of many: combine multiple models into one • Wide and strong proof of superior performance • Extra bonus: often trivially parallelizable OUR EXPERIENCE IS THAT MOST EFFORTS SHOULD BE CONCENTRATED IN DERIVING SUBSTANTIALLY DIFFERENT APPROACHES, RATHER THAN REFINING A SINGLE TECHNIQUE. Netflix $1M prize winner (ensemble of 107 models) “ “
  • 26. ENSEMBLE LEARNING IN PRACTICE • Boosting: weigh (⇒ low bias) focused (⇒ low bias) simple models (⇒ low bias) • Bagging: average (⇒ low variance) results of simple models (⇒ low bias) • What aspect of the data am I still missing? • Variable mixing, discretized jumps, independent factors, transformations, etc. • Questions about practical implementability and ROI • Failure: Netflix winner solution never taken to production • Success: Official US hurricane model is an ensemble of 43
  • 27. RANDOMIZED LEARNING • Motivation: random variation beats expert guidance surprisingly often • Introducing randomness can improve generalization performance (smaller variance) • Randomness allows methods to discover unexpected success • Examples: genetic models, simulated annealing, parallel tempering • Increasingly useful to allow scale-out for large datasets • Many successful methods combine random models as an ensemble • Example: combining random projections or transformations can often beat optimized unsupervised models
  • 28. ONLINE LEARNING • Instead of ingesting a training dataset, adjust the data model after every incoming (instance, label) pair • Allows quick adaptation and “always-on” operation • Finds good models fast, but may miss the great one ⟹ suitable also as a burn-in for other models • Useful especially for the present trend towards analyzing data streams
  • 29. BAYESIAN BASICS • Bayesians see data as fixed and parameters as distributions • Parameters have prior assumptions that can encode expert knowledge • Data is used as evidence for possible parameter values • Final output is a set of posterior distributions for the parameters • Models may employ only the most probable parameter values or their full probability distribution • Variational Bayes approximates the posterior with a simpler distribution
  • 30. MODEL COMPLEXITY • Limiting model size and complexity can be used to avoid excessive bias • Minimum description length and Akaike/Bayesian information criteria are the Occam’s razor of data science • VC dimension of a model provides a theoretical limit for generalization error • Regularization can limit instance weights or parameter sizes • Bayesian models use hyper-parameters to limit parameter overfit