ANALYZING LARGE-SCALE USER DATA                    SPEAKER: Aaron Kimball                             CTO                 ...
Friday, July 27, 2012
Analyzing	  Large-­‐Scale	  User	  Data                      with	  Hadoop	  and	  HBase                        Aaron	  Ki...
We	  can	  now	  collect	                          more	  data	  than	  at	                          any	  Dme	  in	  hist...
Yesterday’s	  engineering	      challenge:	  FiJng	  the	      problem	  into	  the	      hardware.Friday, July 27, 2012
Today’s	  constrained	                 resource	  is	                 understanding.Friday, July 27, 2012
How	  do	  we	  best	  apply	            data                          …to	  beMer	  serving	  our	  Friday, July 27, 2012
The	  best	  products	  are	  user-­‐         • IntuiDve	  UI         • ConDnuously	  learning                   – Guided	...
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
What	  are	  we	  building	  Friday, July 27, 2012
Requirements                 1.	  Understand	  the	  user	                   populaDonFriday, July 27, 2012
Requirements                        2.	  Respond	  to	                                users	  in	  real	                  ...
Requirements                 3.	  Support	  graceful	  data	                   evoluDonFriday, July 27, 2012
Large-­‐scale	  data	  science	  is	           • What	  does	  a	  user	  look	  like?                   – What	  data	  i...
Friday, July 27, 2012
Tools	  of	  the	  trade         • Store	  all	  data	  about	  a	             user	  in	  one	  place         • Support	 ...
Tools	  of	  the	  trade                        • Use	  complex	  data	                            types	  to	  model	    ...
Tools	  of	  the	  trade         • Abstract	  computaDonal	             model	  away	  from	             MapReduce        ...
Friday, July 27, 2012
Friday, July 27, 2012
Friday, July 27, 2012
Friday, July 27, 2012
Friday, July 27, 2012
Friday, July 27, 2012
Friday, July 27, 2012
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
 	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  	  :	  for	  set-­‐top	  boxes                ...
The	  future         • More	  personalizaDon         • AdapDve	  UIs	  (self	  arranging	             dashboards)         ...
Conclusions         • ApplicaDons	  are	  becoming	             increasingly	             user-­‐centric         • Data	  ...
www.wibidata.com	  /	                          Aaron	  Kimball	  –	  aaron@wibidata.comFriday, July 27, 2012
Friday, July 27, 2012
Upcoming SlideShare
Loading in...5
×

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012

358

Published on

Presentation from Aaron Kimball, WibiData
#dataconf
More at http://event.gigaom.com/structuredata/

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
358
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012

  1. 1. ANALYZING LARGE-SCALE USER DATA SPEAKER: Aaron Kimball CTO WibiDataFriday, July 27, 2012
  2. 2. Friday, July 27, 2012
  3. 3. Analyzing  Large-­‐Scale  User  Data with  Hadoop  and  HBase Aaron  Kimball  –  CTO WibiData,  Friday, July 27, 2012
  4. 4. We  can  now  collect   more  data  than  at   any  Dme  in  history.Friday, July 27, 2012
  5. 5. Yesterday’s  engineering   challenge:  FiJng  the   problem  into  the   hardware.Friday, July 27, 2012
  6. 6. Today’s  constrained   resource  is   understanding.Friday, July 27, 2012
  7. 7. How  do  we  best  apply   data …to  beMer  serving  our  Friday, July 27, 2012
  8. 8. The  best  products  are  user-­‐ • IntuiDve  UI • ConDnuously  learning – Guided  search – Smarter  recommenda1ons • More  effec1ve  serviceFriday, July 27, 2012
  9. 9. What  are  we  building  Friday, July 27, 2012
  10. 10. What  are  we  building  Friday, July 27, 2012
  11. 11. What  are  we  building  Friday, July 27, 2012
  12. 12. What  are  we  building  Friday, July 27, 2012
  13. 13. What  are  we  building  Friday, July 27, 2012
  14. 14. What  are  we  building  Friday, July 27, 2012
  15. 15. What  are  we  building  Friday, July 27, 2012
  16. 16. What  are  we  building  Friday, July 27, 2012
  17. 17. What  are  we  building  Friday, July 27, 2012
  18. 18. What  are  we  building  Friday, July 27, 2012
  19. 19. Requirements 1.  Understand  the  user   populaDonFriday, July 27, 2012
  20. 20. Requirements 2.  Respond  to   users  in  real   DmeFriday, July 27, 2012
  21. 21. Requirements 3.  Support  graceful  data   evoluDonFriday, July 27, 2012
  22. 22. Large-­‐scale  data  science  is   • What  does  a  user  look  like? – What  data  is  available  about  the  user? – Which  features  are  important? – Which  features  are  correlated? • How  do  I  model  this  in  MapReduce? • How  do  I  serve  results  in  a  Dmely  Friday, July 27, 2012
  23. 23. Friday, July 27, 2012
  24. 24. Tools  of  the  trade • Store  all  data  about  a   user  in  one  place • Support  real-­‐Dme   get/put,  as  well  as   MapReduceFriday, July 27, 2012
  25. 25. Tools  of  the  trade • Use  complex  data   types  to  model   complex  data • Support  extended   data  models  over   DmeFriday, July 27, 2012
  26. 26. Tools  of  the  trade • Abstract  computaDonal   model  away  from   MapReduce • Support  computaDon   over  all  users…  or  one   user  at  a  DmeFriday, July 27, 2012
  27. 27. Friday, July 27, 2012
  28. 28. Friday, July 27, 2012
  29. 29. Friday, July 27, 2012
  30. 30. Friday, July 27, 2012
  31. 31. Friday, July 27, 2012
  32. 32. Friday, July 27, 2012
  33. 33. Friday, July 27, 2012
  34. 34.                                                      :  for  set-­‐top  boxes Viewing/recording   historyFriday, July 27, 2012
  35. 35.                                                      :  for  set-­‐top  boxes Viewing/recording   historyFriday, July 27, 2012
  36. 36.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=onsFriday, July 27, 2012
  37. 37.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=onsFriday, July 27, 2012
  38. 38.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=ons Analysis  for   product   roadmapFriday, July 27, 2012
  39. 39.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=ons Analysis  for   product   roadmapFriday, July 27, 2012
  40. 40.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=ons Analysis  for   product   Tech  support   roadmap portalFriday, July 27, 2012
  41. 41.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=ons Analysis  for   product   Tech  support   roadmap portalFriday, July 27, 2012
  42. 42.                                                      :  for  set-­‐top  boxes          Libraries Device  and  User  Analysis Viewing/recording   history Personalized  offers   and   recommenda=ons Improve Analysis  for   d  reports   product   Tech  support   for   roadmap portalFriday, July 27, 2012 adver=se
  43. 43. The  future • More  personalizaDon • AdapDve  UIs  (self  arranging   dashboards) • Targeted  content,  ads • More  effecDve  customer  serviceFriday, July 27, 2012
  44. 44. Conclusions • ApplicaDons  are  becoming   increasingly   user-­‐centric • Data  drives  this  capability,  but   harnessing  it  requires  a  new   distributed  architectureFriday, July 27, 2012
  45. 45. www.wibidata.com  /   Aaron  Kimball  –  aaron@wibidata.comFriday, July 27, 2012
  46. 46. Friday, July 27, 2012

×