SlideShare a Scribd company logo
1 of 29
big data
people, technologies, jobs and hong kong
scott brady drummonds
scott@infoincog.com
technologies

definitions

timing and
hype

future
hong kong

people
the common definition

volume
velocity
variety
veracity
the subjective definition
the easy definition
the real challenge:
what to do with all these data?
components: creating value from data
servers

traditional
dbms

visualization

storage

columnar db’s

network
s

hardware

software

nosql

platforms
hadoop
appliances

people
traditional
it
platform
architects

comp.
scientists
stats.
people
hot components
servers

traditional
dbms

visualization

storage

columnar db’s

network
s

hardware

software

nosql

platforms
hadoop
appliances

people
traditional
IT
platform
architects

comp.
scientists
stats.
people
applications
a few examples
why now?
•
•
•
•

cheap storage
unbounded compute
data accessibility and world datafication
internet scale: Yahoo! and Google
– offspring of Hadoop
visibility/expectations

adapted from gartner hype cycle

time
trigger

inflated
disillusionment
productivity
expectations
enlightenment
big data: where are we
today?

adapted from gartner hype cycle

visibility/expectations

this will be caused by
a lack of data scientists

time
trigger

inflated
disillusionment
productivity
expectations
enlightenment
asia
enterprise it adoption
global
usa
business interest

asia
hong kong

time
http://www.infoincog.com/big-data-in-asia-pacific/
Hong Kong
thinking about work?
cs and ds skills most valued ($$$)
us, most disruptive (fun!) techs
it tough to manage visa

us or
hk

it or
other

consider grad
school for visa
and route to
employment

cs skills not valued; ds skills valued ($$)
us, non-disruptive tech; back-end support
non-it tough to manage visa

us companies hiring sales teams, not developers
hk, very few it companies based in hk
it few skilled people, skills not heavily in demand
many jobs in finance
hk, non-disruptive tech; back-end support
non-it easiest path to stay in hk

consider
start-up
want to get involved?
• want to learn more?
– join hkbd* community, ask scott for reading
recommendations

• want to meet other developers?
– come to odhk meetings and hkbd meetings

• want a project for fyp, to build your resume, or
just for fun?
– datasets and software projects are everywhere! as
scott for more info

• want a job?
– scott knows some recruiters and hiring businesses
(*) Hong Kong Big Data on Google+
want more?
Google+: Hong Kong Big Data
http://www.infoincog.com/
scott@infoincog.com

all content by Scott Brady Drummonds – scott@infoincog.com

More Related Content

What's hot

"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr..."Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
Edge AI and Vision Alliance
 

What's hot (20)

Big Data LDN 2017: Machine Learning: What Works And What They Won’t Tell You
Big Data LDN 2017: Machine Learning: What Works And What They Won’t Tell YouBig Data LDN 2017: Machine Learning: What Works And What They Won’t Tell You
Big Data LDN 2017: Machine Learning: What Works And What They Won’t Tell You
 
BigData Meets the Federal Data Center
BigData Meets the Federal Data CenterBigData Meets the Federal Data Center
BigData Meets the Federal Data Center
 
A novel approach to big data veracity using crowd-sourcing techniques
A novel approach to big data veracity using crowd-sourcing techniques A novel approach to big data veracity using crowd-sourcing techniques
A novel approach to big data veracity using crowd-sourcing techniques
 
Dataiku - for Data Geek Paris@Criteo - Close the Data Circle
Dataiku  - for Data Geek Paris@Criteo - Close the Data CircleDataiku  - for Data Geek Paris@Criteo - Close the Data Circle
Dataiku - for Data Geek Paris@Criteo - Close the Data Circle
 
Is It A Right Time For Me To Learn Hadoop. Find out ?
Is It A Right Time For Me To Learn Hadoop. Find out ?Is It A Right Time For Me To Learn Hadoop. Find out ?
Is It A Right Time For Me To Learn Hadoop. Find out ?
 
Big data analysis using map/reduce
Big data analysis using map/reduceBig data analysis using map/reduce
Big data analysis using map/reduce
 
The Rise of the DataOps - Dataiku - J On the Beach 2016
The Rise of the DataOps - Dataiku - J On the Beach 2016 The Rise of the DataOps - Dataiku - J On the Beach 2016
The Rise of the DataOps - Dataiku - J On the Beach 2016
 
Big Data on Public Cloud
Big Data on Public CloudBig Data on Public Cloud
Big Data on Public Cloud
 
A Big Data Timeline
A Big Data TimelineA Big Data Timeline
A Big Data Timeline
 
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATIONKNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
 
Data Culture Series - Keynote - 24th feb
Data Culture Series - Keynote - 24th febData Culture Series - Keynote - 24th feb
Data Culture Series - Keynote - 24th feb
 
Presentation About Big Data (DBMS)
Presentation About Big Data (DBMS)Presentation About Big Data (DBMS)
Presentation About Big Data (DBMS)
 
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr..."Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
 
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataBig Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on Data
 
PASS Summit Data Storytelling with R Power BI and AzureML
PASS Summit Data Storytelling with R Power BI and AzureMLPASS Summit Data Storytelling with R Power BI and AzureML
PASS Summit Data Storytelling with R Power BI and AzureML
 
Introduction of Big data and Hadoop
Introduction of Big data and Hadoop Introduction of Big data and Hadoop
Introduction of Big data and Hadoop
 
A brief history of "big data"
A brief history of "big data"A brief history of "big data"
A brief history of "big data"
 
Big data
Big dataBig data
Big data
 
BreizhJUG - Janvier 2014 - Big Data - Dataiku - Pages Jaunes
BreizhJUG - Janvier 2014 - Big Data -  Dataiku - Pages JaunesBreizhJUG - Janvier 2014 - Big Data -  Dataiku - Pages Jaunes
BreizhJUG - Janvier 2014 - Big Data - Dataiku - Pages Jaunes
 
What is big data?
What is big data?What is big data?
What is big data?
 

Viewers also liked (18)

Charles C. Resume(1-27-16)
Charles C. Resume(1-27-16)Charles C. Resume(1-27-16)
Charles C. Resume(1-27-16)
 
KT Resume_2014
KT Resume_2014KT Resume_2014
KT Resume_2014
 
Moe Aboelela - Resume - May 17 2015 - Director
Moe Aboelela - Resume - May 17 2015 - DirectorMoe Aboelela - Resume - May 17 2015 - Director
Moe Aboelela - Resume - May 17 2015 - Director
 
joseph resume
joseph resumejoseph resume
joseph resume
 
David Harper CV - 2015
David Harper CV - 2015David Harper CV - 2015
David Harper CV - 2015
 
My Resume
My ResumeMy Resume
My Resume
 
Jerry Jones 2016b
Jerry Jones 2016bJerry Jones 2016b
Jerry Jones 2016b
 
R Anthony 9-2016 Resume
R Anthony 9-2016 ResumeR Anthony 9-2016 Resume
R Anthony 9-2016 Resume
 
IDG Media Kit
IDG Media KitIDG Media Kit
IDG Media Kit
 
Mac Picar resume
Mac Picar resumeMac Picar resume
Mac Picar resume
 
Jordan Garth general resume 2015
Jordan Garth general resume 2015Jordan Garth general resume 2015
Jordan Garth general resume 2015
 
VirginiaResume201600323_VCNew
VirginiaResume201600323_VCNewVirginiaResume201600323_VCNew
VirginiaResume201600323_VCNew
 
RajeshGautamResearchResume-Apr-2016
RajeshGautamResearchResume-Apr-2016RajeshGautamResearchResume-Apr-2016
RajeshGautamResearchResume-Apr-2016
 
Ed Peterson 2015 Technical Resume iCloud
Ed Peterson 2015 Technical Resume iCloudEd Peterson 2015 Technical Resume iCloud
Ed Peterson 2015 Technical Resume iCloud
 
RES.Damron CV Jan 2016
RES.Damron CV Jan 2016RES.Damron CV Jan 2016
RES.Damron CV Jan 2016
 
Alexander Wang Beauty - Extension Line Development
Alexander Wang Beauty - Extension Line DevelopmentAlexander Wang Beauty - Extension Line Development
Alexander Wang Beauty - Extension Line Development
 
John Westmorelands Resume
John Westmorelands ResumeJohn Westmorelands Resume
John Westmorelands Resume
 
Janet Resume
Janet ResumeJanet Resume
Janet Resume
 

Similar to Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology

Hadoop is Happening
Hadoop is HappeningHadoop is Happening
Hadoop is Happening
Precisely
 
Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...
Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...
Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...
Shirshanka Das
 
Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...
Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...
Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...
Yael Garten
 

Similar to Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology (20)

Big Data World Singapore 2017 - Moving Towards Digitization & Artificial Inte...
Big Data World Singapore 2017 - Moving Towards Digitization & Artificial Inte...Big Data World Singapore 2017 - Moving Towards Digitization & Artificial Inte...
Big Data World Singapore 2017 - Moving Towards Digitization & Artificial Inte...
 
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
 
Data Science at Scale - The DevOps Approach
Data Science at Scale - The DevOps ApproachData Science at Scale - The DevOps Approach
Data Science at Scale - The DevOps Approach
 
Big Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-AriBig Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-Ari
 
Enabling Data centric Teams
Enabling Data centric TeamsEnabling Data centric Teams
Enabling Data centric Teams
 
Hadoop is Happening
Hadoop is HappeningHadoop is Happening
Hadoop is Happening
 
Moving Targets: Harnessing Real-time Value from Data in Motion
Moving Targets: Harnessing Real-time Value from Data in Motion Moving Targets: Harnessing Real-time Value from Data in Motion
Moving Targets: Harnessing Real-time Value from Data in Motion
 
Geschäftliches Potential für System-Integratoren und Berater - Graphdatenban...
Geschäftliches Potential für System-Integratoren und Berater -  Graphdatenban...Geschäftliches Potential für System-Integratoren und Berater -  Graphdatenban...
Geschäftliches Potential für System-Integratoren und Berater - Graphdatenban...
 
Level Seven - Expedient Big Data presentation
Level Seven - Expedient Big Data presentationLevel Seven - Expedient Big Data presentation
Level Seven - Expedient Big Data presentation
 
Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...
Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...
Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...
 
Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...
Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...
Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...
 
Platform for Big Data Analytics and Visual Analytics: CSIRO use cases. Februa...
Platform for Big Data Analytics and Visual Analytics: CSIRO use cases. Februa...Platform for Big Data Analytics and Visual Analytics: CSIRO use cases. Februa...
Platform for Big Data Analytics and Visual Analytics: CSIRO use cases. Februa...
 
Agents for Agility - The Just-in-Time Enterprise Has Arrived
Agents for Agility - The Just-in-Time Enterprise Has ArrivedAgents for Agility - The Just-in-Time Enterprise Has Arrived
Agents for Agility - The Just-in-Time Enterprise Has Arrived
 
Big data business case
Big data   business caseBig data   business case
Big data business case
 
Introduction to Cloud computing and Big Data-Hadoop
Introduction to Cloud computing and  Big Data-HadoopIntroduction to Cloud computing and  Big Data-Hadoop
Introduction to Cloud computing and Big Data-Hadoop
 
Introduction to Azure DocumentDB
Introduction to Azure DocumentDBIntroduction to Azure DocumentDB
Introduction to Azure DocumentDB
 
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
 
Data science and Artificial Intelligence
Data science and Artificial IntelligenceData science and Artificial Intelligence
Data science and Artificial Intelligence
 
[Webinar] Getting to Insights Faster: A Framework for Agile Big Data
[Webinar] Getting to Insights Faster: A Framework for Agile Big Data[Webinar] Getting to Insights Faster: A Framework for Agile Big Data
[Webinar] Getting to Insights Faster: A Framework for Agile Big Data
 
Data Science-Why?What?How? By Hari Prasad
Data Science-Why?What?How? By Hari PrasadData Science-Why?What?How? By Hari Prasad
Data Science-Why?What?How? By Hari Prasad
 

Recently uploaded

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Recently uploaded (20)

Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 

Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology

Editor's Notes

  1. $6.3B in 2012, $48.3B by 2018, CAGR 40.5% (http://www.prweb.com/releases/2013/7/prweb10905352.htm)
  2. 2009: http://www.nature.com/nature/journal/v457/n7232/full/nature07634.htmlCDC, 60 years old, dept. of health and human services
  3. Gartner, 2001, three v’s. veracity added by others later.
  4. “too big from which to derive value”
  5. 76% of analysts use MS Excel: http://www.billingviews.com/microsoft-excel-king-analytics-hill/
  6. http://www.intel.com/content/www/us/en/communications/internet-minute-infographic.html
  7. Retail:CRM – Customer Scoring,Store Siting and Layout,Fraud Detection / Prevention,Supply Chain OptimizationFinancial Services:Algorithmic Trading,Risk Analysis,Fraud Detection,Portfolio AnalysisManufacturing:Product Research,Engineering Analytics,Process & Quality Analysis,Distribution Optimization (GE and Wikibon think manufacturing/industrial growing 2x faster than any other segment: http://online.wsj.com/article/PR-CO-20130618-908554.html)Government:Market Governance,Counter-Terrorism,Econometrics,Health InformaticsEnergy:Smart Grid,ExplorationHealthcare & Life Sciences:Pharmaco-Genomics,Bio-Informatics,Pharmaceutical Research,Clinical Outcomes ResearchAdvertising & Public Relations:Demand Signaling,Ad Targeting,Sentiment Analysis,Customer AcquisitionMedia & Telecommunications:Network Optimization,Customer Scoring,Churn Prevention,Fraud Prevention
  8. http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=1&_r=1&hp
  9. Doug Cutting and Mike Cafarella in 2005
  10. US leads the globe by about six months.Asia trails the US by about 18 monthsHong Kong trails Asia by about six months
  11. http://www.gov.hk/en/theme/psi/datasets/questions:what is the relationship between traffic and air pollution? (data joining)how does property value lead/trail changes in population (historical analysis)what are the trends for weather-related closures (trending)
  12. http://www.gov.hk/en/theme/psi/datasets/questions:what is the relationship between traffic and air pollution? (data joining)how does property value lead/trail changes in population (historical analysis)what are the trends for weather-related closures (trending)
  13. 博文约礼
  14. http://www.gov.hk/en/theme/psi/datasets/questions:what is the relationship between traffic and air pollution? (data joining)how does property value lead/trail changes in population (historical analysis)what are the trends for weather-related closures (trending)
  15. * Hong Kong Big Data