WebQuilt: Capturing and Visualizing the Web Experience at WWW10
•Download as PPT, PDF•
0 likes•123 views
Research I did a while back on using a web proxy to capture web interactions remotely and then visualizing those interactions. Basically, WebQuilt is a tool to support remote usability testing of web sites. WebQuilt is a web logging and visualization system that helps web design teams run usability tests (both local and remote) and analyze the collected data. Logging is done through a proxy, overcoming many of the problems with server-side and client-side logging. Captured usage traces can be aggregated and visualized in a zooming interface that shows the web pages people viewed. The visualization also shows the most common paths taken through the website for a given task, as well as the optimal path for that task as designated by the designer. This paper discusses the architecture of WebQuilt and also describes how it can be extended for new kinds of analyses and visualizations. Authors are Jason Hong and James Landay
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to WebQuilt: Capturing and Visualizing the Web Experience at WWW10
Similar to WebQuilt: Capturing and Visualizing the Web Experience at WWW10 (20)
Recently uploaded
Recently uploaded (20)
WebQuilt: Capturing and Visualizing the Web Experience at WWW10
- 1. WebQuilt Capturing and Visualizing the Web Experience Jason I. Hong James A. Landay Group for User Interface Research EECS Department University of California at Berkeley World Wide Web 10
- 2. May 04 2001 2 Motivation • Many websites have usability problems 62% web shoppers gave up past month (Spool) 39% failed in buying attempts (Creative Good) • Two problems all web designers face Understanding users' tasks Understanding obstacles in completing tasks • Many methods for understanding tasks E.g. interviews, ethnographic observations, surveys, focus groups • Focus here is on understanding obstacles
- 3. May 04 2001 3 Understanding Obstacles Today • Traditional usability tests Extremely useful qualitative information Lots of time, small websites, few people, local • Serverside logging Easy to collect, remote testing, lots of tools Restricted access, little on tasks and problems • Clientside logging Can track everything, remote testing Installation, platformdependent, analysis tools
- 4. May 04 2001 4 Streamlining Current Practices • Fast and easy to deploy on any website • Compatible with range of OS and browsers • Better tools for analyzing the data
- 5. May 04 2001 5 WebQuilt Approach • Fast and easy to deploy on any website • Compatible with range of OS and browsers • Better tools for analyzing the data Client Browser Web Server Request Response
- 6. May 04 2001 6 WebQuilt Approach • Fast and easy to deploy on any website • Compatible with range of OS and browsers • Better tools for analyzing the data WebQuilt Log ProxyClient Browser Web Server
- 7. May 04 2001 7 WebQuilt Approach • Fast and easy to deploy on any website • Compatible with range of OS and browsers • Better tools for analyzing the data
- 8. May 04 2001 8 WebQuilt Usage • Setup several tasks, recruit 20–100 people • Email participants a URL that uses the proxy • Ask them to complete the predefined tasks • Collect lots of remote (or local) data • Aggregate, view, and interact with data • Find problems, fix, repeat Evaluate Design Prototype
- 9. May 04 2001 9 Outline Background and Motivation WebQuilt Architecture Usage Experience and Visualizations Summary and Future Work
- 10. May 04 2001 10 Overall Architecture Proxy Logger Graph Layout Viz Graph Merger Action Inferencer Log Files Online Offline
- 11. May 04 2001 11 Proxy Proxy Logger Graph Layout Viz Graph Merger Action Inferencer • Lies between browser and server http://domain.com/webquilt? replace=http://www.yahoo.com • One log file per user session • Currently use Java servlets Important part is log file format
- 12. May 04 2001 12 Time (ms) From TID To TID Parent ID HTTP Response Fram e ID Link ID HTTP Method URL + Query 6062 0 1 1 200 1 1 GET http://www.google.com 11191 1 2 1 200 1 1 GET http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky 167525 2 3 1 200 1 1 GET http://www.phish.com/bios.html 31043 3 4 1 200 1 2 GET https://www.phish.com/bin/catal og.cgi 68772 2 5 1 200 1 15 GET http://www.emusic.com/features /phish Log File Format
- 13. May 04 2001 13 Time From TID To TID Parent ID HTTP Response Fram e ID Link ID HTTP Method URL + Query 6062 0 1 1 200 1 1 GET http://www.google.com 11191 1 2 1 200 1 1 GET http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky 167525 2 3 1 200 1 1 GET http://www.phish.com/bios.html 31043 3 4 1 200 1 2 GET https://www.phish.com/bin/catal og.cgi 68772 2 5 1 200 1 15 GET http://www.emusic.com/features /phish Time From TID To TID Parent ID HTTP Response 6062 0 1 1 200 (ms) Log File Format
- 14. May 04 2001 14 Time (ms) From TID To TID Parent ID HTTP Response Fram e ID Link ID HTTP Method URL + Query 6062 0 1 1 200 1 1 GET http://www.google.com 11191 1 2 1 200 1 1 GET http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky 167525 2 3 1 200 1 1 GET http://www.phish.com/bios.html 31043 3 4 1 200 1 2 GET https://www.phish.com/bin/catal og.cgi 68772 2 5 1 200 1 15 GET http://www.emusic.com/features /phish Frame ID Link ID HTTP Method URL + Query 1 1 GET http://www.google.com Log File Format
- 15. May 04 2001 15 Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package Store The Proxy at Runtime
- 16. May 04 2001 16 Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package 1. Process Client Request Store The Proxy at Runtime
- 17. May 04 2001 17 Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package 2. Retrieve Requested Document Store The Proxy at Runtime
- 18. May 04 2001 18 Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package 3. Process and return the page Store The Proxy at Runtime
- 19. May 04 2001 19 Start with: <A HREF="computers.html"> End up with: <A HREF="http://tasmania.cs.berkeley.edu/webquilt? replace=http://www.yahoo.com/computers.html &tid=1&linkid=12"> The Proxy at Runtime
- 20. May 04 2001 20 Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package Client Browser Web ServerWebQuilt Proxy Proxy Editor Cached Pages WebQuilt Logs WebProxy Servlet 1 2 3 4 5 HTTPClient Package 4. Store the page 5. Log the transaction Store The Proxy at Runtime
- 21. May 04 2001 21 Additional Proxy Functionality • Handling Cookies Cookies only sent from browser back to web server that put it there Proxy Logger Graph Layout Viz Graph Merger Action Inferencer User ID Domain Cookie AAA yahoo.com xyzzy AAA google.com asdfg BBB yahoo.com abcde
- 22. May 04 2001 22 Additional Proxy Functionality • Handling Cookies Cookies only sent from browser back to web server that put it there • Handling Secure Socket Layer Encrypts page requests and data E.g. Shopping Carts, Financials Proxy Logger Graph Layout Viz Graph Merger Action Inferencer Client Browser Web Server SSL
- 23. May 04 2001 23 Additional Proxy Functionality • Handling Cookies Cookies only sent from browser back to web server that put it there • Handling Secure Socket Layer Encrypts page requests and data E.g. Shopping Carts, Financials Split into two SSL requests Proxy Logger Graph Layout Viz Graph Merger Action Inferencer ProxyClient Browser Web Server SSL SSL
- 24. May 04 2001 24 Action Inferencer • Takes a single log file and converts into a list of actions "Clicked on link" or "Hit back button" • Inference because still must guess Back and forward actions local Proxy Logger Graph Layout Viz Graph Merger Action Inferencer
- 25. May 04 2001 25 Time From TID To TID Parent ID HTTP Response Fram e ID Link ID HTTP Method URL + Query 6062 0 1 1 200 1 1 GET http://www.google.com 11191 1 2 1 200 1 1 GET http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky 167525 2 3 1 200 1 1 GET http://www.phish.com/bios.html 31043 3 4 1 200 1 2 GET https://www.phish.com/bin/catal og.cgi 68772 2 5 1 200 1 15 GET http://www.emusic.com/features /phish ReAssembling User Actions
- 26. May 04 2001 26 From TID To TID Parent ID HTTP Response Fram e ID Link ID HTTP Method 0 1 1 200 1 1 GET 2 1 200 1 1 GET 3 1 200 1 1 GET 4 1 200 1 2 GET 5 1 200 1 15 GET URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish Time 6062 11191 167525 31043 68772 1 2 3 2 1 2 3 4 5 ReAssembling User Actions
- 27. May 04 2001 27 From TID To TID 0 1 2 3 3 4 2 5 URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish 1 2 3 4 5 1 2 3 2 ReAssembling User Actions
- 28. May 04 2001 28 From TID To TID 0 1 2 3 3 4 2 5 URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish 1 2 3 4 5 1 2 3 2 Start 1 ReAssembling User Actions
- 29. May 04 2001 29 From TID To TID 0 1 2 3 3 4 2 5 URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish 1 2 3 4 5 1 2 3 2 Start 1 2 ReAssembling User Actions
- 30. May 04 2001 30 From TID To TID 0 1 2 3 3 4 2 5 URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish 1 2 3 4 5 1 2 3 2 Start 1 2 3 ReAssembling User Actions
- 31. May 04 2001 31 From TID To TID 0 1 2 3 3 4 2 5 URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish 1 2 3 4 5 1 2 3 2 Start 1 2 3 4 ReAssembling User Actions
- 32. May 04 2001 32 From TID To TID 0 1 2 3 3 4 2 5 URL + Query http://www.google.com http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky http://www.phish.com/bios.html https://www.phish.com/bin/catal og.cgi http://www.emusic.com/features /phish 1 2 3 4 5 1 2 3 2 Start 1 2 3 4 5 ReAssembling User Actions
- 33. May 04 2001 33 Start 1 2 3 4 5 Action Inferencer
- 34. May 04 2001 34 1 2 3 4 3 2Start 5 Start 1 2 3 4 5 Case 1 Link Back Link Action Inferencer
- 35. May 04 2001 35 Start 1 2 3 4 5 1 2 3 4 3 2Start 1 2 5 Case 2 Link Back LinkFwd Action Inferencer
- 36. May 04 2001 36 1 2 3 4 3 2Start 5 Start 1 2 3 4 5 Case 1 by default (shortest path) Action Inferencer
- 37. May 04 2001 37 Merger • Combines multiple log files into a single directed graph Web pages are nodes Actions are edges Proxy Logger Graph Layout Viz Graph Merger Action Inferencer
- 38. May 04 2001 38 Graph Layout • Assign (x,y) to all nodes • Forcedirected placement Keep connected nodes close Push unconnected nodes far apart • Edgeweighted depthfirst Most traffic along top Less followed paths below Grid to help organize and align • Plugin new algorithms here Proxy Logger Graph Layout Viz Graph Merger Action Inferencer
- 39. May 04 2001 39 Visualization Proxy Logger Graph Layout Viz Graph Merger Action Inferencer
- 40. May 04 2001 40
- 41. May 04 2001 41
- 42. May 04 2001 42
- 43. May 04 2001 43
- 44. May 04 2001 44
- 45. May 04 2001 45
- 46. May 04 2001 46
- 47. May 04 2001 47
- 48. May 04 2001 48
- 49. May 04 2001 49 Future Work • More sophisticated logging Lower level events (e.g. AT&T WET) Personalized web pages • More sophisticated visualizations More use of semantic zooming Dynamic filtering • Continue getting feedback from designers Initiated interviews with web designers Still need to do evaluations
- 50. May 04 2001 50 Take Home Ideas • Need more tools for improving web site usability • Proxy logging Logging where task is already known Any website, any browser, remote testing • Visualizing logged data Aggregates large data sets Interact with in a zooming interface • Pluggable architecture
- 51. May 04 2001 51 Acknowlegements • Special thanks to Jeff Heer, Tim Sohn, and Sarah Waterson Group for User Interface Research EECS Department University of California at Berkeley Download WebQuilt at: http://guir.berkeley.edu/webquilt
- 52. May 04 2001 52 Extra Slides
- 53. May 04 2001 53 Berkeley Website A
- 54. May 04 2001 54
- 55. May 04 2001 55
- 56. May 04 2001 56
- 57. May 04 2001 57
- 58. May 04 2001 58
- 59. May 04 2001 59 Casa de Fruta A
- 60. May 04 2001 60
- 61. May 04 2001 61
- 62. May 04 2001 62
- 63. May 04 2001 63 Casa de Fruta B
- 64. May 04 2001 64
- 65. May 04 2001 65
- 66. May 04 2001 66
- 67. May 04 2001 67
- 68. May 04 2001 68 Time From TID To TID Parent ID HTTP Response Fram e ID Link ID HTTP Method URL + Query 6062 0 1 -1 200 -1 -1 GET http://www.google.com 11191 1 2 -1 200 -1 -1 GET http://www.phish.com/index.htm q=Phish&btnI=I %27m+Feeling+Lucky 167525 2 3 -1 200 -1 1 GET http://www.phish.com/bios.html 31043 3 4 -1 200 -1 2 GET https://www.phish.com/bin/catal og.cgi 68772 2 5 -1 200 -1 15 GET http://www.emusic.com/features /phish Log File Format
- 69. May 04 2001 69 In Case You're Feeling Evil… • URLs can be of the form: http://userid@domain/page.html • Most web servers ignore the userid part, but… http://www.yahoo.com@tasmania.cs…/… • Can auto-track people's actions once they hit your page without their knowledge or consent
Editor's Notes
- Web usability is a serious problem Two problems web designers face in trying to improve the usability of a website First is understanding the task, that is what people want to do on a website An example task could be something like "I want to purchase an mp3 player and spend less than $500" Understanding obstacles in completing the task For example, I might not be able to find an mp3 player on an ecommerce website because I thought it would be in the "music" section instead of the "electronics" section You really need to understand both Could know that my task is to buy an mp3 player, but this isn't useful if you don't know what problems are preventing me from successfully completing the task On the other hand, you might know that the problem is that I'm abandoning lots of shopping carts, but this isn't useful if you don't know my task. Instead of trying to buy an mp3 player, maybe I was doing a price comparison or was just showing someone a feature your website has in the shopping cart Knowing both the task and the obstacles are important in designing websites that people find both useful and usable Lots of methods for understanding tasks out there Our focus is on understanding obstacles
- Traditional Usability Tests A usability specialist recruits and schedules several participants, and then brings them into a usability lab Asks them to complete a few predefined tasks Observes what stumbling blocks people come across Follows up with interviews and surveys Takes a lot of time to schedule participants, run tests, and then analyze data Furthermore, while usability tests are extremely useful, they don't scale well if you want to test a large website. You would have to bring in hundreds of users to cover all the possible tasks that a large website supports. Consequently, traditional usability tests tend to use only a few dozen people at most, and only cover small portions of a website Server-Side Logging This just means analyzing the log file generated by the web server Easy to collect data for large numbers of people, automatically down by most web servers You can also do remote testing, that is people can surf from their home computers instead of having to be brought into a usability lab Also, there are lots of tools available, at least 90 research and commercial tools out there today Restricted access (can't do a competitive analysis) / organizational bureacracy (can't get access to server logs) Also, most tools give you information that is fairly useless in terms of usability. Knowing that 90% of my visitors come from a ".com" domain doesn't tell me how to improve the website There are some newer tools that provide information about at user needs at a statistical level, such as the information scent project at Xerox PARC, but in these cases, the users' tasks and goals are still ambiguous Client Side Logging End-users download special software, such as a modified web browser or a custom-made web browser, to capture all transactions The advantage to this approach is that you can track literally everything, including keystrokes and mouse movement Also, client-side logging allows for remote testing, making it easier to scale up to many users However, there are two downsides to this approach First, it requires special software to be installed, which people may be unwilling or unable to do Can severely limit usability test participants to experienced users, which may not be representative of the target audience Second, the special software is usually platform dependent, meaning that it only works for a specific operating system or web browser This limits the overall reach of the logging software Lastly, there aren't a lot of analysis tools here for aggregating the data It's easy to collect gigabytes of data using client-side logging, but it's not easy to understand it
- To address the first two issues, we created a proxy logger. So here's how HTTP works today.
- We introduce a level of indirection here with the web proxy, having all transactions transparently go through a web proxy and logging those transactions. Traditionally, proxies are used for things like caching and for firewalls. What we've done here is to use a web proxy for usability purposes, and later on I'll talk a little more about some of the special features we've added to make the logging more useful for usability analysis
- To address the third issue, we created a visualization system that aggregates all of the user logs and displays it in an interactive visualization. So here's a screenshot of the current visualization, but we'll get back to this again towards the end of the talk.
- Talk about iterative design, why it's important WebQuilt focuses on the Evaluate part of iterative design
- Only online part is the proxy, that is the only part that is running when participants go through their tasks Offline parts are for designers, can be done anytime Components can be replaced in case anyone has a better algorithm
- Some browsers have a proxy field that can be set internally within the browser WebQuilt uses a URL based approach for proxies instead of having end users set a field in their web browser We chose this approach, again because we thought novices would be unwilling or unable to do this Instead, the WebQuilt proxy is URL-based This approach has been done by others, there's nothing new here
- Time is the time in milliseconds since starting the session The next two fields are the "From Transaction ID" and the "To Transaction ID" Transaction IDs represent the Nth page that a person has requested So a transaction ID of 1 means that this is the first page that a person requested, 2 means the second page that a person requested, etc The FromTID represents the page that a person came from The ToTID represents the current page that the person is at The transaction IDs are used to recreate what a person did in terms of page requests and going backwards and forwards The ParentID specifies the frame parent of the current page. This number is the TID of the frameset to which the current page belongs, or –1 if the current page is not a frame The HTTP response field is just the response from the server, such as "200 ok" and "404 not found"
- The Frame ID field indicates which frame in an enclosing frameset the current page is in. A frame ID of 0 indicates the first frame in the frameset, 1 is the second, and so on. The LinkID field specifies which link was clicked on according to the Document Object Model. In this representation, the first link has a LinkID of 0, the second one has a value of 1, and so on. This technique is useful if there are redundant links, as it lets us identify exactly which link a person used. Multiple links to the same page but… Link not being used because below the fold Or link not being used because it's a graphic image but people don't know that they can click on it
- In the first phase, an HTTP request is received from the client by the proxy All WebQuilt parameters, such as the destination URL and link ID are extracted and saved The proxy also collects other data that is saved in the log file, such as the transaction IDs and parent ID
- In the next phase, the proxy attempts to retrieve the actual document The rest of the data needed for the log entry is also stored at this time, including the HTTP response code and final destination URL in case of server redirects
- Before the downloaded page is sent back to the client, it has to be edited so that all the links are redirected to use the proxy again We first check if the content type of the page is text/html If it isn't, then we just pass it through without any modifications If it is, then we first add or update the <BASE> tag so that all relative links and images work correctly Then we go through the entire document and update all page hyperlinks, frames, and form actions to go through the proxy again Thus, once a person starts to use the proxy, all of the links thereafter will automatically be rewritten to continue using the proxy We also add transaction ID tags and Link ID tags to each of the links, as well as any parent ID and Frame ID parameters
- In case you're feeling particularly insidious you can do something like: http://www.yahoo.com/computers.html@tasmania.cs.berkeley.edu/webquilt?replace=http://www.yahoo.com/computers.html&tid=1&linkid=12
- Saves a local copy of the web page to a cache And then writes to the log
- Have to have some additional functionality in order for proxy to work correctly with existing websites One feature is handling cookies How cookies are done today In WebQuilt, we store the cookie in the proxy, keyed off of the User ID and the domain
- Also have to handle SSL
- Reassemble what a person did
- WebQuilt assumes this case by default But if there is a better algorithm, then someone can just plug it in here
- Berkeley website Colors used to indicate average amount of tie before traversing a link Colors closer to yellow mean short amounts of time, on the order of a few seconds Colors closer to red mean longer amounts of time, on the order of a hundred seconds Thicker lines mean more people traversed a link
- Casa de Fruta Task was to find a specific item and add it to the shopping cart
- Shopping task, add items and then check out
- Shopping task, add items and then check out
- Reassemble what a person did
- In case you're feeling particularly insidious you can do something like: http://www.yahoo.com/computers.html@tasmania.cs.berkeley.edu/webquilt?replace=http://www.yahoo.com/computers.html&tid=1&linkid=12