big data
people, technologies, jobs and hong kong
scott brady drummonds
scott@infoincog.com
technologies

definitions

timing and
hype

future
hong kong

people
the common definition

volume
velocity
variety
veracity
the subjective definition
the easy definition
the real challenge:
what to do with all these data?
components: creating value from data
servers

traditional
dbms

visualization

storage

columnar db’s

network
s

hardware...
hot components
servers

traditional
dbms

visualization

storage

columnar db’s

network
s

hardware

software

nosql

pla...
applications
a few examples
why now?
•
•
•
•

cheap storage
unbounded compute
data accessibility and world datafication
internet scale: Yahoo! and Goo...
visibility/expectations

adapted from gartner hype cycle

time
trigger

inflated
disillusionment
productivity
expectations...
big data: where are we
today?

adapted from gartner hype cycle

visibility/expectations

this will be caused by
a lack of ...
asia
enterprise it adoption
global
usa
business interest

asia
hong kong

time
http://www.infoincog.com/big-data-in-asia-pacifi...
Hong Kong
thinking about work?
cs and ds skills most valued ($$$)
us, most disruptive (fun!) techs
it tough to manage visa

us or
hk...
want to get involved?
• want to learn more?
– join hkbd* community, ask scott for reading
recommendations

• want to meet ...
want more?
Google+: Hong Kong Big Data
http://www.infoincog.com/
scott@infoincog.com

all content by Scott Brady Drummonds...
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology
Upcoming SlideShare
Loading in …5
×

Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology

3,019 views

Published on

This is the student-focused version of the talk I have been delivering in Hong Kong lately. Very similar to the talk I gave to the CUHK Statistics department. In this one I provide a general description of big data, talk about enabling technologies, give examples of big data in action, and talk about big and open data projects right here in Hong Kong.

For more info: http://infoincog.com

0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,019
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
32
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide
  • $6.3B in 2012, $48.3B by 2018, CAGR 40.5% (http://www.prweb.com/releases/2013/7/prweb10905352.htm)
  • 2009: http://www.nature.com/nature/journal/v457/n7232/full/nature07634.htmlCDC, 60 years old, dept. of health and human services
  • Gartner, 2001, three v’s. veracity added by others later.
  • “too big from which to derive value”
  • 76% of analysts use MS Excel: http://www.billingviews.com/microsoft-excel-king-analytics-hill/
  • http://www.intel.com/content/www/us/en/communications/internet-minute-infographic.html
  • Retail:CRM – Customer Scoring,Store Siting and Layout,Fraud Detection / Prevention,Supply Chain OptimizationFinancial Services:Algorithmic Trading,Risk Analysis,Fraud Detection,Portfolio AnalysisManufacturing:Product Research,Engineering Analytics,Process & Quality Analysis,Distribution Optimization (GE and Wikibon think manufacturing/industrial growing 2x faster than any other segment: http://online.wsj.com/article/PR-CO-20130618-908554.html)Government:Market Governance,Counter-Terrorism,Econometrics,Health InformaticsEnergy:Smart Grid,ExplorationHealthcare & Life Sciences:Pharmaco-Genomics,Bio-Informatics,Pharmaceutical Research,Clinical Outcomes ResearchAdvertising & Public Relations:Demand Signaling,Ad Targeting,Sentiment Analysis,Customer AcquisitionMedia & Telecommunications:Network Optimization,Customer Scoring,Churn Prevention,Fraud Prevention
  • http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=1&_r=1&hp
  • Doug Cutting and Mike Cafarella in 2005
  • US leads the globe by about six months.Asia trails the US by about 18 monthsHong Kong trails Asia by about six months
  • http://www.gov.hk/en/theme/psi/datasets/questions:what is the relationship between traffic and air pollution? (data joining)how does property value lead/trail changes in population (historical analysis)what are the trends for weather-related closures (trending)
  • http://www.gov.hk/en/theme/psi/datasets/questions:what is the relationship between traffic and air pollution? (data joining)how does property value lead/trail changes in population (historical analysis)what are the trends for weather-related closures (trending)
  • 博文约礼
  • http://www.gov.hk/en/theme/psi/datasets/questions:what is the relationship between traffic and air pollution? (data joining)how does property value lead/trail changes in population (historical analysis)what are the trends for weather-related closures (trending)
  • * Hong Kong Big Data
  • Big Data Overview for Chinese University of Hong Kong Centre for Innovation and Technology

    1. big data people, technologies, jobs and hong kong scott brady drummonds scott@infoincog.com
    2. technologies definitions timing and hype future hong kong people
    3. the common definition volume velocity variety veracity
    4. the subjective definition
    5. the easy definition
    6. the real challenge: what to do with all these data?
    7. components: creating value from data servers traditional dbms visualization storage columnar db’s network s hardware software nosql platforms hadoop appliances people traditional it platform architects comp. scientists stats. people
    8. hot components servers traditional dbms visualization storage columnar db’s network s hardware software nosql platforms hadoop appliances people traditional IT platform architects comp. scientists stats. people
    9. applications
    10. a few examples
    11. why now? • • • • cheap storage unbounded compute data accessibility and world datafication internet scale: Yahoo! and Google – offspring of Hadoop
    12. visibility/expectations adapted from gartner hype cycle time trigger inflated disillusionment productivity expectations enlightenment
    13. big data: where are we today? adapted from gartner hype cycle visibility/expectations this will be caused by a lack of data scientists time trigger inflated disillusionment productivity expectations enlightenment
    14. asia
    15. enterprise it adoption global usa business interest asia hong kong time http://www.infoincog.com/big-data-in-asia-pacific/
    16. Hong Kong
    17. thinking about work? cs and ds skills most valued ($$$) us, most disruptive (fun!) techs it tough to manage visa us or hk it or other consider grad school for visa and route to employment cs skills not valued; ds skills valued ($$) us, non-disruptive tech; back-end support non-it tough to manage visa us companies hiring sales teams, not developers hk, very few it companies based in hk it few skilled people, skills not heavily in demand many jobs in finance hk, non-disruptive tech; back-end support non-it easiest path to stay in hk consider start-up
    18. want to get involved? • want to learn more? – join hkbd* community, ask scott for reading recommendations • want to meet other developers? – come to odhk meetings and hkbd meetings • want a project for fyp, to build your resume, or just for fun? – datasets and software projects are everywhere! as scott for more info • want a job? – scott knows some recruiters and hiring businesses (*) Hong Kong Big Data on Google+
    19. want more? Google+: Hong Kong Big Data http://www.infoincog.com/ scott@infoincog.com all content by Scott Brady Drummonds – scott@infoincog.com

    ×