WUD2008 - The Numbers Revolution and its Effect on the Web


Published on

Published in: Technology, Education
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • WUD2008 - The Numbers Revolution and its Effect on the Web

    1. 1. The Numbers Revolution and its effect on the Web Rich Miller Research Scientist, LexisNexis World Usability Day – 11/13/2008
    2. 2. What we will cover <ul><li>What the numbers revolution is and how it came about </li></ul><ul><li>How it is affecting the world and the web </li></ul><ul><li>How it is changing both the user experience and design approaches </li></ul><ul><li>How it is being manifested in tools and applications </li></ul>
    3. 3. The numbers revolution – what is it? <ul><li>Using data and statistics to better see and affect reality </li></ul><ul><li>Revolutionary impact due to the maturation of the web and its enabling technologies </li></ul><ul><ul><li>More data and metadata available – more sensing/measuring </li></ul></ul><ul><ul><li>Faster networks </li></ul></ul><ul><ul><li>More powerful displays </li></ul></ul><ul><ul><li>User interface rendering technologies – e.g. Flash </li></ul></ul><ul><li>Involves… </li></ul><ul><ul><li>Capturing and organizing data, often large amounts </li></ul></ul><ul><ul><li>Crunching data to make predictions </li></ul></ul><ul><ul><li>Slicing, dicing and visualizing data to aid decision-making and discovery </li></ul></ul><ul><li>Information overload revisited </li></ul><ul><ul><li>More information (in the form of metadata) needed to enable the consumption of large amounts of data </li></ul></ul>
    4. 4. Process and related domains
    5. 5. Hans Rosling and trendalyzer <ul><li>http:// www.youtube.com/watch?v =hVimVzgtD6w </li></ul><ul><ul><li>Start at 2:15, goto 5:15 </li></ul></ul><ul><ul><li>Sequel introducing predictive analytics </li></ul></ul><ul><li>demo </li></ul>
    6. 6. How is it changing how people think? <ul><li>Focus thinking on what humans do well and let the computers handle the tough thinking jobs </li></ul><ul><ul><li>Less guessing and hypothesis-testing, more discovery </li></ul></ul><ul><ul><li>Predicting based on data and sometimes relegating expertise to asking the right questions </li></ul></ul><ul><li>Analytics-powered thinking essentially amplifies the main advantage of the internet: information at your fingertips </li></ul><ul><ul><li>there is no reason that both information and analysis should not be at a user's fingertips </li></ul></ul>
    7. 7. Where is it having the most impact? <ul><li>Organization decision-making and strategy </li></ul><ul><ul><li>Includes business intelligence and CRM </li></ul></ul><ul><ul><li>Predictive modeling </li></ul></ul><ul><li>Inform general product/UI design and usability </li></ul><ul><ul><li>User behavior analytics </li></ul></ul><ul><ul><ul><li>Navigation </li></ul></ul></ul><ul><ul><ul><li>Consumption </li></ul></ul></ul><ul><ul><ul><li>Personalization </li></ul></ul></ul><ul><ul><li>Feedback and testing </li></ul></ul><ul><ul><ul><li>e.g. randomized web tests </li></ul></ul></ul><ul><li>Enable more powerful, analytics-fueled applications </li></ul><ul><ul><li>Decision-support tools </li></ul></ul><ul><ul><li>Recommendation systems </li></ul></ul><ul><ul><li>Interactive visual interfaces to enhance… </li></ul></ul><ul><ul><ul><li>Consuming content </li></ul></ul></ul><ul><ul><ul><li>Socializing and collaborating </li></ul></ul></ul>
    8. 8. Importance of the metadata layer <ul><li>As the amount of data available through the web is continuing to increase, a new layer of metadata is being created around it </li></ul><ul><li>The metadata layer yields multiple benefits, e.g. </li></ul><ul><ul><li>framework for organizing data </li></ul></ul><ul><ul><li>Fuel for UIs </li></ul></ul><ul><ul><li>personalization </li></ul></ul><ul><ul><li>individual user productivity and decision-support </li></ul></ul>
    9. 9. Numbers-related publications
    10. 10. Moneyball , Baseball Abstract <ul><li>Sports geeks as pioneers </li></ul><ul><ul><li>Evaluating talent </li></ul></ul><ul><ul><li>Determining strategy </li></ul></ul><ul><li>1980s - Bill James </li></ul><ul><ul><li>Challenged conventional wisdom </li></ul></ul><ul><li>1990s – Fantasy Sports </li></ul><ul><ul><li>The Lonious Monks </li></ul></ul><ul><li>2000s – Major League Baseball </li></ul><ul><ul><li>Moneyball – Oakland A’s </li></ul></ul><ul><ul><li>Red Sox hire Epstein, James </li></ul></ul><ul><li>Today – part of the game </li></ul><ul><ul><li>NBA – e.g. Houston Rockets Science </li></ul></ul>
    11. 11. Freakonomics <ul><li>Applying economic analysis to understand human behavior </li></ul><ul><li>Like Bill James, Stephen Levitt challenges conventional wisdom </li></ul><ul><li>Investigations include… </li></ul><ul><ul><li>Why do teachers and sumo wrestlers cheat? </li></ul></ul><ul><ul><li>Why should you be suspicious of your real-estate agent? </li></ul></ul><ul><ul><li>Why do most crack dealers live with their mothers? </li></ul></ul><ul><ul><li>Is there a link between abortion and crime rate? </li></ul></ul><ul><ul><li>Why does good parenting not really affect educational performance? </li></ul></ul><ul><ul><li>How is a child’s name affected by his/her socioeconomic position? </li></ul></ul><ul><ul><li>Why it is safer to own a gun than a swimming pool? </li></ul></ul><ul><li>Freakonomics blog </li></ul>
    12. 12. Super Crunchers , Ian Ayres <ul><li>Ayres = Yale law professor </li></ul><ul><ul><li>Collaborates with Freakonomics guys </li></ul></ul><ul><li>Focus on very large datasets and multiple regression analysis </li></ul><ul><li>Goes beyond BI by combining predictive models with randomized experiments </li></ul><ul><li>How statistical evidence can supplement/replace human judgement. </li></ul><ul><ul><li>exposes experts’ limitations in predicting </li></ul></ul><ul><ul><li>seeking decision-making help from computers should be normal part of business. </li></ul></ul><ul><li>Topics include medicine, education, business, sports, and winemaking. </li></ul>
    13. 13. The Wisdom of Crowds <ul><li>In this case, the numbers are the people in the crowd…the more the better </li></ul><ul><li>Under certain conditions, the crowd’s decisions are superior to the individual’s </li></ul><ul><li>4 conditions necessary </li></ul><ul><ul><li>diversity of opinion </li></ul></ul><ul><ul><li>independence of opinions </li></ul></ul><ul><ul><li>decentralization of power </li></ul></ul><ul><ul><li>aggregation into a group answer </li></ul></ul><ul><li>Again, experts exposed as inferior – to crowd in predicting </li></ul><ul><ul><li>… and “less-bright” folks are essential! </li></ul></ul><ul><li>Examples include </li></ul><ul><ul><li>auto traffic behavior </li></ul></ul><ul><ul><li>disease tracking and treating </li></ul></ul><ul><ul><li>navigating the internet </li></ul></ul>
    14. 14. Competing on Analytics <ul><li>How leading companies collect, analyze, and act on data </li></ul><ul><li>It takes an investment, a plan,and the discipline to stick to it. </li></ul><ul><li>Applications include… </li></ul><ul><ul><li>Supply chain, customer relations, pricing, HR, product quality, R&D. </li></ul></ul><ul><li>Companies include </li></ul><ul><ul><li>Marriot International </li></ul></ul><ul><ul><li>Capital One </li></ul></ul><ul><ul><li>Oakland A’s, Boston Red Sox </li></ul></ul><ul><ul><li>Procter & Gamble </li></ul></ul><ul><li>Bottom line </li></ul><ul><ul><li>Make it a normal part of your business </li></ul></ul>
    15. 15. WIRED: The End of Science <ul><li>“ All models are wrong, but some are useful” – George Box (statistician) </li></ul><ul><li>“ All models are wrong, and you can do without them” – Peter Norvig (Google) </li></ul><ul><li>A shift from traditional scientific models to ones based on collecting and analyzing large bodies of data – “the petabyte age” </li></ul><ul><ul><li>essentially discovering the truth as opposed to making predictions and testing them. </li></ul></ul><ul><li>Example applications include </li></ul><ul><ul><li>disease surveillance, farming, physics, legal discovery, news/event monitoring, astronomy, archaeology, airfare analysis, politics/elections, web data management, terrorism insurance. </li></ul></ul><ul><li>Counterpoint: “If the data wasn't collected to fit the model, the model is probably wrong” – Mark Wasson (LexisNexis) </li></ul><ul><li>Also, we need a better measure of database size </li></ul><ul><ul><li>“ information objects” rather than storage space </li></ul></ul>
    16. 16. Related (but unread by presenter) <ul><li>Similar </li></ul><ul><ul><li>Wikinomics : How Mass Collaboration Changes Everything </li></ul></ul><ul><ul><li>The Long Tail : Why the Future of Business is Selling Less of More </li></ul></ul><ul><li>Differing perspectives </li></ul><ul><ul><li>Nassim Nicholas Taleb </li></ul></ul><ul><ul><ul><li>The Black Swan : The Impact of the Highly Improbable </li></ul></ul></ul><ul><ul><ul><li>Fooled by Randomness : The Hidden Role of Chance in Life and in the Markets  </li></ul></ul></ul>
    17. 17. How is it affecting business/web? <ul><li>Using numbers to get smarter is quickly becoming a competency necessary to effectively compete </li></ul><ul><li>Numbers used for improving both… </li></ul><ul><ul><li>Internal decision-making </li></ul></ul><ul><ul><li>Quality of products for customers </li></ul></ul><ul><li>Provides fuel for, and often demands existence of, visualization </li></ul><ul><li>Giving rise to new wave of tools and apps </li></ul><ul><ul><li>Business intelligence </li></ul></ul><ul><ul><li>Charting and social/sharing/collaboration </li></ul></ul><ul><ul><li>Decision-support </li></ul></ul><ul><ul><li>Information-seeking and research </li></ul></ul>
    18. 18. Implications for product/UI design <ul><li>Opportunities </li></ul><ul><ul><li>bridging the gap between normal users and complex statistical analysis tools </li></ul></ul><ul><ul><ul><li>allowing human brain to do what is does best and let the computer play an augmenting role </li></ul></ul></ul><ul><ul><li>Creating compelling visual information spaces to explore </li></ul></ul><ul><li>Challenges </li></ul><ul><ul><li>adding power while preserving UI simplicity </li></ul></ul><ul><ul><li>Screen space allocation and management </li></ul></ul><ul><ul><li>avoiding misleading conclusions or data views…pitfalls include </li></ul></ul><ul><ul><ul><li>small sample sizes </li></ul></ul></ul><ul><ul><ul><li>inappropriate mapping of data to charts </li></ul></ul></ul><ul><ul><li>managing data clean-up issues </li></ul></ul><ul><ul><ul><li>data normalization – consistent label per entity </li></ul></ul></ul><ul><ul><ul><li>label abbreviation – so they can fit into charts </li></ul></ul></ul><ul><ul><li>facilitate learning by integrating new tools with familiar tools </li></ul></ul>
    19. 19. Best answer vs. best picture <ul><li>The information-seeking UX lies along a continuum from a single answer to an information space </li></ul><ul><li>Continuum anchored by different views/interfaces </li></ul><ul><ul><li>Providing best answer – crunching and analyzing data and coming up with the best answer </li></ul></ul><ul><ul><ul><li>e.g. how old is a person expected to live? </li></ul></ul></ul><ul><ul><li>Providing a view – organizing and rendering data in a way for users to best understand it and use it. </li></ul></ul><ul><ul><ul><li>e.g. voting patterns by county in Ohio </li></ul></ul></ul><ul><li>The continuum reflects the degree of interaction a user has with an application </li></ul><ul><ul><li>i.e. the broader the answer space, the more the user needs control over manipulating a view of the space </li></ul></ul>
    20. 20. UI-related technologies/approaches <ul><li>Advances in visualization and infographics </li></ul><ul><ul><li>2 fields merging as computer graphics improve </li></ul></ul><ul><ul><li>New York Times setting the pace </li></ul></ul><ul><li>Rich internet applications (RIAs) </li></ul><ul><ul><li>Provide virtually unlimited rendering of graphics and visualization </li></ul></ul><ul><li>Large displays and new pointing devices </li></ul>
    21. 21. Application landscape
    22. 22. BI visualization and dashboards Example = Tableau Desktop
    23. 23. Ian Ayres prediction tools <ul><li>Featured in Super Crunchers </li></ul><ul><ul><li>Predict the Value of Bordeaux  (Ayres - may require  free JAVA download ) </li></ul></ul><ul><li>Personal/Family </li></ul><ul><ul><li>Predict how long your marriage will last  (Political Calculations) </li></ul></ul><ul><ul><li>Predict Your Child's Adult Height  (University of Saskatchewan) </li></ul></ul><ul><li>Consumer Applications </li></ul><ul><ul><li>Predict the Market Value of Your Home  (Zillow) </li></ul></ul><ul><li>Fun/Sports (and gambling?) </li></ul><ul><ul><li>Predict NFL game  (NFL Picker) </li></ul></ul><ul><li>Politics/Government </li></ul><ul><ul><li>Predict Where Your Tax Money Is Spent  (Tax Break Down) </li></ul></ul><ul><li>Media </li></ul><ul><ul><li>Predict Demographics of Who Will Use a Webpage  (Microsoft adCenter) </li></ul></ul><ul><li>Health </li></ul><ul><ul><li>Predict Your Liklihood of Illness or Disability  (Northwestern Mutual) </li></ul></ul><ul><li>Money/Business </li></ul><ul><ul><li>Predict whether a publicly traded company will file for bankruptcy (Political Calculations) </li></ul></ul><ul><li>Macroeconomic </li></ul><ul><ul><li>Predict the odds of a U.S. recession in the next 12 months  (Political Calculations) </li></ul></ul>
    24. 24. Predictive Analytics - SAS
    25. 25. Small-scale decision-support - visual i|o Should the pitcher be replaced?
    26. 26. Fantasy sports decision-support CBS Sportsline Fantasy Basketball The outcome of my decisions Who should I start? Who should I add to my team? What moves are others making?
    27. 27. Infographics - Catalogtree.net Candidate for a geo view
    28. 28. Infographics – weighted electoral map
    29. 29. Infographics-visualization convergence New York Times
    30. 30. Hans Rosling and trendalyzer <ul><li>http:// www.youtube.com/watch?v =hVimVzgtD6w </li></ul><ul><ul><li>Start at 2:15, goto 5:15 </li></ul></ul><ul><ul><li>sequel </li></ul></ul><ul><li>demo </li></ul>
    31. 31. Info space browsers - Bestiario
    32. 32. Data+viz sharing - Many-eyes Powers New York Times Visualization Lab example
    33. 33. Data+viz sharing - Swivel
    34. 34. Data+viz sharing - lastfm
    35. 35. Professional info-seeking - LexisNexis LNIS - volume of European news coverage of US Banks over time Total Patent # of patents over time by authority/country Courtlink Strategic Profiles - types of matters handled by law firm
    36. 36. Pro info-seeking - ILOG/Elixir CIA World Factbook
    37. 37. Consumer info-seeking – Stamen viz <ul><ul><ul><li>trulia </li></ul></ul></ul><ul><li>See document: trulia </li></ul><ul><ul><ul><ul><li>oreilly post on trulia map </li></ul></ul></ul></ul><ul><li>See document: stamens-map- for.html </li></ul><ul><ul><ul><li>adobe </li></ul></ul></ul><ul><ul><ul><ul><li>tour of california RIA </li></ul></ul></ul></ul><ul><li>See document: adobe </li></ul><ul><ul><ul><li>digg labs </li></ul></ul></ul><ul><li>See document: digg </li></ul><ul><ul><ul><li>global business network </li></ul></ul></ul><ul><li>See document: gbn </li></ul><ul><ul><ul><li>root markets </li></ul></ul></ul><ul><li>See document: root </li></ul>
    38. 38. What about usability in transportation? <ul><li>A ripe area for using numbers to better view and affect reality </li></ul><ul><li>Applications </li></ul><ul><ul><li>Traffic analysis – optimizing traffic efficiency </li></ul></ul><ul><ul><li>Public transportation planning </li></ul></ul><ul><ul><li>Trip planning </li></ul></ul><ul><ul><li>Monitoring vessels/vehicles – intelisea </li></ul></ul><ul><ul><li>Geo-maps with info layers </li></ul></ul><ul><ul><li>GPS location trackers – e.g. where is the family car? </li></ul></ul>
    39. 39. Recommendations for UX pros <ul><li>Acknowledge and discover your newly expanded solution space </li></ul><ul><li>Seek out and use number-fueled apps </li></ul><ul><li>Understand user cognitive limits </li></ul><ul><ul><li>… and provide tools for tasks they need help with </li></ul></ul><ul><li>View visualization design as an extension of UI design </li></ul><ul><li>Get a handle on data cleanup and formatting </li></ul><ul><li>Embrace RIAs and the tools to make them </li></ul>
    40. 40. Predictions for 2009 and beyond <ul><li>Mining and analyzing data for intelligence becomes common practice for more companies </li></ul><ul><li>RIA technologies gain momentum </li></ul><ul><li>Visualization continues to proliferate </li></ul><ul><li>The analytics shakeout begins as users discover which numbers are not as useful </li></ul><ul><ul><li>Broken Social Scene – social apps will get hit the hardest </li></ul></ul><ul><li>Decision-support apps hit the mainstream </li></ul><ul><li>Increased emphasis on data cleanup </li></ul><ul><ul><li>Companies with nimble data better compete </li></ul></ul><ul><li>The world becomes smarter, more objective </li></ul>
    41. 41. Questions and related resources <ul><li>If we run out of time for questions, feel free to contact me… </li></ul><ul><ul><li>in person later today </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><li>Previous public talks </li></ul><ul><ul><li>WUD 2007 talk on web applications </li></ul></ul><ul><ul><li>MVCS talk on Web 2.0 </li></ul></ul><ul><ul><li>SOASIS talk on Visual Thinking </li></ul></ul><ul><ul><li>Miami-IMS talk on Trends in HCI </li></ul></ul>