SlideShare a Scribd company logo
Web Intelligence and Big Data
Introduction to the Web Intelligence
Mangesh R. Wanjari
Lecture 1
1/4/2017 1
1/4/2017 2
Outline
• Turing test
• Intelligence of a machine
• Web scale AI applications
• Big Data?
• Analytics
1/4/2017 3
Turing test
Textual typewritten
messages only
Party Game
Which is man
and which is
woman?
1/4/2017 4
Turing test
Textual typewritten
messages only
Party Game: Replace one of the humen with a Machine
Which is man
and which is
machine?
1/4/2017 5
Turing test
Textual typewritten
messages only
Party Game: Replace Judge with a machine
Which is man
and which is
machine?
1/4/2017 6
Reverse Turing test
Textual typewritten
messages only
Like/Dislike Shopper/Surfer Rich/Poor
Which is man
and which is
woman?
1/4/2017 7
Web Scale AI
1/4/2017 8
Moore’s Law
Moore's law refers to an observation made by
Intel co-founder Gordon Moore in 1965. He
noticed that the number of transistors per
square inch on integrated circuits had doubled
every year since their invention.
1/4/2017 9
Kryder’s Law
The density of information on hard drives
has been growing at an even faster rate,
increasing by a factor of 1000 in 10.5 years,
which corresponds to a doubling roughly
every 13 months
1/4/2017 10
Big Data?
• Lots of web pages
• A billion Facebook users
• Billion+ facebook pages
• Hundreds of millions Twitter account
• Hundreds of millions Tweets per day
• Billions of Google queries per day-may be
more
• Millions of servers, petabytes of data
1/4/2017 11
Big corporations work on
• 5000-50000 servers, may be some
more
• Terabytes of data, millions
transactions per day
1/4/2017 12
Big Data technology
• Traditional BI using databases
• Google, Facebook, LinkedIn, eBay,
Amazon, … did not use traditional
databases for their data
Statistical
report
Databases Data
Warehouse
More
Databases
1/4/2017 13
What they use?
• Parallel programming
• Massive parallelism
• Map-Reduce paradigm
1/4/2017 14
Data and intelligence
• “Any fool can know. The point is to
understand.”
-Albert Einstein
The goal of understanding is to
predict.
1. Reactive Intelligence
2. Predictive intelligence
1/4/2017 15
Data and intelligence
Look
Listen
Learn
Connect
Predict
Correct
1/4/2017 16
Thank you!!!

More Related Content

Viewers also liked

Competitive Intelligence & Big Data
Competitive Intelligence & Big DataCompetitive Intelligence & Big Data
Competitive Intelligence & Big Data
CID GmbH
 
Project england
Project englandProject england
Project england
ivancampo
 
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
Amazon Web Services
 
M.tech. cse csesy
M.tech.   cse csesyM.tech.   cse csesy
M.tech. cse csesy
Jampala Nag
 
Amazon Item-to-Item Recommendations
Amazon Item-to-Item RecommendationsAmazon Item-to-Item Recommendations
Amazon Item-to-Item Recommendations
Roger Chen
 
How to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on SparkHow to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on Spark
Caserta
 

Viewers also liked (9)

Competitive Intelligence & Big Data
Competitive Intelligence & Big DataCompetitive Intelligence & Big Data
Competitive Intelligence & Big Data
 
12
1212
12
 
11
1111
11
 
2
22
2
 
Project england
Project englandProject england
Project england
 
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
AWS re:Invent 2016: Zillow Group: Developing Classification and Recommendatio...
 
M.tech. cse csesy
M.tech.   cse csesyM.tech.   cse csesy
M.tech. cse csesy
 
Amazon Item-to-Item Recommendations
Amazon Item-to-Item RecommendationsAmazon Item-to-Item Recommendations
Amazon Item-to-Item Recommendations
 
How to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on SparkHow to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on Spark
 

Similar to Lecture 2

The Promise and Perils of Data Science: A Workshop on Ethical Thinking
The Promise and Perils of Data Science: A Workshop on Ethical ThinkingThe Promise and Perils of Data Science: A Workshop on Ethical Thinking
The Promise and Perils of Data Science: A Workshop on Ethical Thinking
Data Science Studies UW
 
Value Drivers for Your Data – Big, Fast, or Smart
Value Drivers for Your Data – Big, Fast, or SmartValue Drivers for Your Data – Big, Fast, or Smart
Value Drivers for Your Data – Big, Fast, or Smart
IDEAS - Int'l Data Engineering and Science Association
 
Using Big Data to Find Anomalies and Fight Crime
Using Big Data to Find Anomalies and Fight CrimeUsing Big Data to Find Anomalies and Fight Crime
Using Big Data to Find Anomalies and Fight Crime
FITC
 
BIG DATA -- The Next Big Thing!
BIG DATA -- The Next Big Thing!BIG DATA -- The Next Big Thing!
BIG DATA -- The Next Big Thing!
Extentia Information Technology
 
Introduction to Internet Governance
Introduction to Internet GovernanceIntroduction to Internet Governance
Introduction to Internet Governance
RIPE NCC
 
Janet in a changing world
Janet in a changing world Janet in a changing world
Janet in a changing world
Jisc
 
Diffusion in platform-based markets: big data driven agent-based model
Diffusion in platform-based markets: big data driven agent-based modelDiffusion in platform-based markets: big data driven agent-based model
Diffusion in platform-based markets: big data driven agent-based model
Jari Jussila
 
NGOpen Content Strategy
NGOpen Content StrategyNGOpen Content Strategy
NGOpen Content Strategy
Russ Maschmeyer
 

Similar to Lecture 2 (8)

The Promise and Perils of Data Science: A Workshop on Ethical Thinking
The Promise and Perils of Data Science: A Workshop on Ethical ThinkingThe Promise and Perils of Data Science: A Workshop on Ethical Thinking
The Promise and Perils of Data Science: A Workshop on Ethical Thinking
 
Value Drivers for Your Data – Big, Fast, or Smart
Value Drivers for Your Data – Big, Fast, or SmartValue Drivers for Your Data – Big, Fast, or Smart
Value Drivers for Your Data – Big, Fast, or Smart
 
Using Big Data to Find Anomalies and Fight Crime
Using Big Data to Find Anomalies and Fight CrimeUsing Big Data to Find Anomalies and Fight Crime
Using Big Data to Find Anomalies and Fight Crime
 
BIG DATA -- The Next Big Thing!
BIG DATA -- The Next Big Thing!BIG DATA -- The Next Big Thing!
BIG DATA -- The Next Big Thing!
 
Introduction to Internet Governance
Introduction to Internet GovernanceIntroduction to Internet Governance
Introduction to Internet Governance
 
Janet in a changing world
Janet in a changing world Janet in a changing world
Janet in a changing world
 
Diffusion in platform-based markets: big data driven agent-based model
Diffusion in platform-based markets: big data driven agent-based modelDiffusion in platform-based markets: big data driven agent-based model
Diffusion in platform-based markets: big data driven agent-based model
 
NGOpen Content Strategy
NGOpen Content StrategyNGOpen Content Strategy
NGOpen Content Strategy
 

Recently uploaded

一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
1tyxnjpia
 
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing  Sample .pdfDigital Marketing Performance Marketing  Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
Vineet
 
Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
TeukuEriSyahputra
 
一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理
一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理
一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理
z6osjkqvd
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
Vietnam Cotton & Spinning Association
 
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docxDATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
SaffaIbrahim1
 
REUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptx
REUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptxREUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptx
REUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptx
KiriakiENikolaidou
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
aguty
 
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCAModule 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
yuvarajkumar334
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
keesa2
 
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
hyfjgavov
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
Timothy Spann
 
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
oaxefes
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
eudsoh
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
slg6lamcq
 
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google CloudBuild applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
Márton Kodok
 
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative ClassifiersML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
MastanaihnaiduYasam
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
bmucuha
 
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
blueshagoo1
 

Recently uploaded (20)

一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
 
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing  Sample .pdfDigital Marketing Performance Marketing  Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
 
Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
 
一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理
一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理
一比一原版英属哥伦比亚大学毕业证(UBC毕业证书)学历如何办理
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
 
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docxDATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
 
REUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptx
REUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptxREUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptx
REUSE-SCHOOL-DATA-INTEGRATED-SYSTEMS.pptx
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
 
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCAModule 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
 
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
 
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
 
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
 
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google CloudBuild applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
 
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative ClassifiersML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
 

Lecture 2

  • 1. Web Intelligence and Big Data Introduction to the Web Intelligence Mangesh R. Wanjari Lecture 1 1/4/2017 1
  • 2. 1/4/2017 2 Outline • Turing test • Intelligence of a machine • Web scale AI applications • Big Data? • Analytics
  • 3. 1/4/2017 3 Turing test Textual typewritten messages only Party Game Which is man and which is woman?
  • 4. 1/4/2017 4 Turing test Textual typewritten messages only Party Game: Replace one of the humen with a Machine Which is man and which is machine?
  • 5. 1/4/2017 5 Turing test Textual typewritten messages only Party Game: Replace Judge with a machine Which is man and which is machine?
  • 6. 1/4/2017 6 Reverse Turing test Textual typewritten messages only Like/Dislike Shopper/Surfer Rich/Poor Which is man and which is woman?
  • 8. 1/4/2017 8 Moore’s Law Moore's law refers to an observation made by Intel co-founder Gordon Moore in 1965. He noticed that the number of transistors per square inch on integrated circuits had doubled every year since their invention.
  • 9. 1/4/2017 9 Kryder’s Law The density of information on hard drives has been growing at an even faster rate, increasing by a factor of 1000 in 10.5 years, which corresponds to a doubling roughly every 13 months
  • 10. 1/4/2017 10 Big Data? • Lots of web pages • A billion Facebook users • Billion+ facebook pages • Hundreds of millions Twitter account • Hundreds of millions Tweets per day • Billions of Google queries per day-may be more • Millions of servers, petabytes of data
  • 11. 1/4/2017 11 Big corporations work on • 5000-50000 servers, may be some more • Terabytes of data, millions transactions per day
  • 12. 1/4/2017 12 Big Data technology • Traditional BI using databases • Google, Facebook, LinkedIn, eBay, Amazon, … did not use traditional databases for their data Statistical report Databases Data Warehouse More Databases
  • 13. 1/4/2017 13 What they use? • Parallel programming • Massive parallelism • Map-Reduce paradigm
  • 14. 1/4/2017 14 Data and intelligence • “Any fool can know. The point is to understand.” -Albert Einstein The goal of understanding is to predict. 1. Reactive Intelligence 2. Predictive intelligence
  • 15. 1/4/2017 15 Data and intelligence Look Listen Learn Connect Predict Correct