Winning With Big Data: Secrets of the Successful Data Scientist

by Dataspora on Jun 16, 2010

1,020 views

The world is experiencing an Industrial Revolution of Data. In any given minute the machines around us are tracking billions of mouse clicks, credit card swipes, and GPS coordinates. And increasingly this data is being saved, aggregated, and analyzed. These massive data flows present big challenges to firms, but also new opportunities for deriving insights.

Presented at the June 2010 gathering of the Bay Area's Business Intelligence Special Interest Group.

Less

1 Embed 1

http://www.slideshare.net

Statistics

Favorites: 1
Downloads: 63
Comments: 0
Embed Views: 1
Views on SlideShare: 1,019
Total Views: 1,020

Winning With Big Data: Secrets of the Successful Data Scientist — Presentation Transcript

WINNING
WITH
BIG
DATA
Secrets of the Successful
Data Scientist
SDForum BI SIG
June 15, 2010
Michael Driscoll
@dataspora

WHY DATA
MATTERS
NOW

THE INDUSTRIAL
AGE
OF
DATA

WHAT IS
BIG DATA?
Data that is distributed.

WHAT IS
DATA
SCIENCE?

WHY DATA SCIENCE
IS SEXY

“The sexy job in the next ten years will be statisticians…”
- Hal Varian
=
+

data
model
1000 bytes
2 bytes

9 WAYS TO WIN
WITH DATA

1. CHOOSE THE
RIGHT TOOL
You don’t need a chainsaw to cut butter.

2. COMPRESS EVERYTHING
mysqldump -u myuser -p mypasssourceDB |
gzip | ssh mike@dataspora.com "cat - |
gunzip | mysql -u myuser -p mypasstargetDB"
The world is IO-bound.

3. SPLIT UP
YOUR DATA
Split, apply, combine.

4. WORK
WITH SAMPLES
perl -ne "print if (rand() < 0.01)"
data.csv > sample.csv
Big Data is heavy,
samples are light.

5. USE
STATISTICS

COPY
FROM OTHERS
git clone git://github.com/kevinweil/hadoop-lzo
Use open source.

7. ESCHEW CHART TYPOLOGIES
Charts are compositions,
not containers.

8. COLORWITH CARE
Color can enhance
or insult.

9. TELL A STORY
People are listening.

ONE
SUCCESS
STORY

WHY DO TELCO CUSTOMERS LEAVE?
Sign up
Leave
Goal: “less churn.”

DATA:
BILLIONS
OF CALLS
… and millions of callers.

DOES CALL
QUALITY
MATTER?
… a difference,
but not significant.

WHAT ABOUT
SOCIAL
NETWORKS?
Hmmm...

BUILD THE
CALL GRAPH
… but is it predictive?

EVOLUTION OF A CALL GRAPH
April

EVOLUTION OF A CALL GRAPH
May

EVOLUTION OF A CALL GRAPH
June

EVOLUTION OF A CALL GRAPH
July

700% INCREASE
IN CHURN
when a cancellation
occurs in a call network.

FINAL
THOUGHTS

THE BIG DATA STACK
Actions
Data Products
(Content Filters, Rec Engines)
Analytics
(R, SPSS, SAS, SAP)
Insights
Big Data
Dedicated RDBMS
Data

THANKS!
QUESTIONS?
Michael Driscoll
med@dataspora.com
@dataspora on Twitter
http://www.dataspora.com/blog
SDForum BI SIG
June 15, 2010

Winning With Big Data: Secrets of the Successful Data Scientist

by Dataspora on Jun 16, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

Winning With Big Data: Secrets of the Successful Data Scientist — Presentation Transcript