Data Science 101

Ofer Ron | DevCon – October 2013
Just what is data science anyway?

Data science has many different
manifestations:
• BI

–find new markets and niches to t...
Building data driven products

Increase each user’s
exposure to the social
network by easing him
in.
Building data driven products

Increase revenues through
recommendations.
Building data driven products
Building data driven products

Targeting visitors
needing assistance.
Toys, beautiful toys

What is the right question to ask?
Data science is not about technology.

It is about using data to ...
That BIG DATA thing

What is Big Data?

“Big data” is when the size of the
data itself becomes part of the
problem.
Small ...
Hypothesize – test - iterate

The basics of data science
• Examine the data

• Perform basic analysis
• Formulate a hypoth...
Hypothesize – test - iterate - cnt’d

The principles of data science
• Iterate quickly – know your tools
• Don’t be afraid...
Practical Examples

When should I tweet?
The problem: Give the tweeter user an indication
when he is most likely to get a ...
Practical Example – the workflow

• Choose features

• Split data into training and evaluation
• Model the training data

...
Practical Example – features – the simple choice

October

th,
8

2013 16:43
hour of day = 16

day of week = Tuesday
Practical Example – the modeling (on training)

A decision tree using the features
28.2% (259)
Hour
37.2% (145)
Day

16.7%...
Practical Example – evaluation

On Training

AUC=.667

On Evaluation

AUC=.576
Practical Example – what now?

• More features – #,NLP on tweets?

• More data
• Validate across users, cluster users?
Etc...
LivePerson’s Ecosystem

developer.liveperson.com

apps.liveperson.com
LivePerson’s Ecosystem

YouTube.com/LivePersonDev
Twitter.com/LivePersonDev

Facebook.com/LivePersonDev
Thank You

And we’re hiring!
Upcoming SlideShare
Loading in...5
×

Introduction to Data Science

385

Published on

Ofer Ron, senior data scientist at LivePerson.
Recently, I've had the pleasure of presenting an introduction to Data Science and data driven products at DevconTLV

I focused this talk around the basic ideas of data science, not the technology used, since I thought that far too many times companies and developers rush to play around with "big data" related technologies, instead of figuring out what questions they want to answer, and whether these answers form a successful product.

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
385
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Global research of 600 consumers’ expectation of help – when purchasing online, 1/3 expect to be helped immediately, whilewhen purchasing online, ¾ expect to be able to access help within 5 minutesif a response is not delivered in the expected timeframe 50% will shop elsewhere online, or abandon altogether
  • Transcript of "Introduction to Data Science"

    1. 1. Data Science 101 Ofer Ron | DevCon – October 2013
    2. 2. Just what is data science anyway? Data science has many different manifestations: • BI –find new markets and niches to target. • Information • Predictive retrieval – search engines. modeling –personalization of the user experience.
    3. 3. Building data driven products Increase each user’s exposure to the social network by easing him in.
    4. 4. Building data driven products Increase revenues through recommendations.
    5. 5. Building data driven products
    6. 6. Building data driven products Targeting visitors needing assistance.
    7. 7. Toys, beautiful toys What is the right question to ask? Data science is not about technology. It is about using data to solve business needs.
    8. 8. That BIG DATA thing What is Big Data? “Big data” is when the size of the data itself becomes part of the problem. Small social graph Large distributed social graphs
    9. 9. Hypothesize – test - iterate The basics of data science • Examine the data • Perform basic analysis • Formulate a hypothesis • Validate the hypothesis • Productize
    10. 10. Hypothesize – test - iterate - cnt’d The principles of data science • Iterate quickly – know your tools • Don’t be afraid of statistics • Don’t fall in love with your ideas
    11. 11. Practical Examples When should I tweet? The problem: Give the tweeter user an indication when he is most likely to get a retweet. The input: The users tweet stream, with all features.
    12. 12. Practical Example – the workflow • Choose features • Split data into training and evaluation • Model the training data • Validate the hypothesis • What now?
    13. 13. Practical Example – features – the simple choice October th, 8 2013 16:43 hour of day = 16 day of week = Tuesday
    14. 14. Practical Example – the modeling (on training) A decision tree using the features 28.2% (259) Hour 37.2% (145) Day 16.7% (114) 34.1% (132) 30.8% (104) 69.2% (13) Hour Day 33.3% (18) 46.4% (28) 70.0% (10)
    15. 15. Practical Example – evaluation On Training AUC=.667 On Evaluation AUC=.576
    16. 16. Practical Example – what now? • More features – #,NLP on tweets? • More data • Validate across users, cluster users? Etc. etc.
    17. 17. LivePerson’s Ecosystem developer.liveperson.com apps.liveperson.com
    18. 18. LivePerson’s Ecosystem YouTube.com/LivePersonDev Twitter.com/LivePersonDev Facebook.com/LivePersonDev
    19. 19. Thank You And we’re hiring!

    ×