Bachelor thesis Project Design and Development of Learning Content andCollaborative tool for card sorting technique in Usability Engineering Kshitij Pandey, Final Year, B.Des IITG Manas Modi, Final Year, B.Des IITG Under the guidance of Prof. Pradeep Yammiyavar 1
Recap Introduction to card sorting technique Technique used to generate a category tree. It is a useful approach for designing information architecture, workflows, menu structure, or web site navigation paths. Types of card sorting Open, Closed, Reverse, Delphi Need analysis •Lack of interactive tools for learning card sorting technique •No tools available for collaborative card sorting Aim •To design a computer based tool that assists a novice user to learn card sorting technique. •To design an online tool to perform card sorting technique in a collaborative, way. Deliverables Working prototype which can be used as a) Learning content for card sorting b) Online tool for collaborative card sorting 2
Recap Group vs. Individual card sorting Pros and Cons of online card sorting Quantitative and qualitative valuation of the existing online tools on the basis of tasks effectiveness and efficiency; and featuresOptimal sort UX Sort User Zoom WebSort 3
Recap Experiment for collaborative card sorting Purpose To understand and observe how participants and users, who were known or unknown to each other perform the card sorting activity Method Participants performed the activity on their own with no interference from the moderator. The whole session was documented through photos, videos and notes Procedure 24 cards and 6 categories 15 participants, 5 groups of 3. Observations and Inferences/Functional requirements •An activity lasts for 15 min approx •Focused discussions in groups where participants are unknown to each other •Need for an application which can help card sorting activity with participants sitting at different places •Moderator should be able to make cards and categories beforehand. •System should be able to Assimilate, compile and store the card sorting results 4
Recap Concept 1: Moderator Controlled, Mod sort Environment is totally controlled by the moderator. He acts according to the group consensus Concept 2: Participant Controlled, Collab Sort Environment parameters are set by moderator. Card sorting is done by the participants themselves Concept 3: Optimized Model, Co Sort Environment parameters are set by moderator. Card sorting is initially done individually by participants, and followed by a group sort. Selected concept: Concept 3, Co Sort •Participant control is present •Easy to determine individual mental model •Less time taken, as the common cards are already sorted •Good for large scale, when information is very large in number, and there are over 50 cards 5
Selected ConceptCo-Sort It allows participants to work individually as well as collaboratively Moderator can distinguish between every participant’s mental model Cards which are common between participants after individual activity are placed beforehand on the group interface, hence, saving time. Versatile for small scale sort to large scale card sorting algorithms where complex systems are involved 10
WireframesThis screen isused by themoderator to setparameters forthe card sort. 11
WireframesThe participants sortout the cardsindividually first in apre set time frame 12
WireframesModerator can see theparticipants’ activityremotely. 13
Wireframes•Shared screen• All the participantscan drag and drop thecards.•There are 3 mousepointers on the screen. 14
Prototyping Phase I, Card Sort: UI design and prototyping a tool To help a novice user learn card sorting Phase II, Co Sort: UI design and prototyping a tool for Collaborative card sorting
Technology Adobe flash(AS3), PHP, MySQL database. PHPPHP feeds the Also, it brings theuser activity data updated data theon the front end database back to reflectinto the MySQL the changes in the frontdatabase. MySQL Flash end.
Phase I User Flow • Introduction to card sorting1 • Stepwise learning content for card sorting2 • An exercise where the participant gets to choose an existing3 website to perform card sorting activity • A comparison of participants result and actual website4 architecture • Repeat step 3 with same or different website5 • Or exit
Phase IExercise Stepwise instructions Card Status Progress bar
Phase II Results gathered from from Phase I individual card-sorting Collaboration added through dynamically refreshing the user screens through PHP scripts and MySQL Database.
Phase II CoSort HomeInformationArchitecture Moderator Participant Set up new Introduction to View calendar Reminder card sort card sorting Change View previous Send invites View details Learning guide schedule results Individual card sorting Collaborative card sorting Submit results
Phase II Moderator’s dashboard Reminder for next sort Setup a new card sort (inline feature)Calendar to viewschedule and previoussorting
Phase II Collaboration Glow representing active mouse activityVideo of the participantOption to delete a card Option to add a card
Phase II Sorting ResultsCollaborative result Individual results
Implementation A MySQL database is maintained on the server. The database is initialized through the interface. The user interfaces of the system are synchronized with the data base using PHP scripts. As the users interact with the system, the database is constantly modified again using PHP scripts. This way all the participant screens operating from different geographical locations are synchronized and serve as a real time collaborative platform.
User Testing Testing of Phase I, Card Sort Testing of Phase II, Co Sort
Results Sum of ratings between 1-5(1-lowest, 5 highest) from 10 users50454035302520151050 Easyness of Knowledge Effectiveness Efficiency of Extent of Enjoyability of How effective Easyness of How satisfying Extent of error navigation gained of task task confusion exercise can it be a exercise were the freeness freeness source of results knowledge
Observations Most of the users enjoyed the activity and could relate to the real websites A participant took an average of 4 minutes to learn as well as perform card sorting activity. There is a bug in one module of exercise which need to be corrected Users wanted to know “a score” since they played “a game” Users expected the cards to snap one above other
UI ChangesCards stacking one aboveanother Cards separated, no overlapping
Testing Phase II15 cards, 6 categories6 participants in 2groupsUsers of IMDb websiteAge group: 18-22yearsServer on a laptopSkype conference call
Results Sorting was finished successfully by the participants Participants could easily discuss and operate the cards. A slight lag was experienced in the synchronization of the cards and pointers due to server limitations . There were times when users got confused due to the lag An average of 10 minutes was taken during sorting
Changes required A better server needed which can handle the requests from the users An optimised algorithm which is lightweight from the current algorithm.
Summary A learning tool for card-sorting technique was successfully conceptualized, developed and tested. The learning is hosted on the IIT-G virtual lab in the form of an interactive experiment. A new model of card-sorting technique was conceptualized and a working prototype was successfully developed and tested with participants. Based on the user-testing data, the implied changes were implemented to improve the interaction on the tool.
Conclusion The learning tool was effective in educating participants about card-sorting. Participants rated it 41/50 as a source of knowledge. The new model of collaborative card sorting was tested with participants and the results obtained were coherent with the theory proposed. Both the phases of the proposed model went successful. There were a few problems while testing the collaborative prototype like slight lag in the synchronization in the 3 participant’s screens. But it did prove to be a major hindrance and the sessions could be completed successfully.
Future Work The future work involves extending this platform and incorporating intercultural elements into it. Features such as language options, themes, icons etc. would impart a cultural dimension to this tool. A further extension of this tool would be for mobile devices on various platforms like Android, iOS, Windows Phone etc.
Acknowledgement Prof. Pradeep Yammiyavar for his guidance, encouragement and gracious support Paranjay Shrivastava, Praveen Alevoor, and Yogesh Deshpande for constantly helping us in the times of need.
References Jakob Nielsen. "Card Sorting: How Many Users to Test".Useit. 2004. [online] http://www.useit.com/alertbox/20040719.html [Accessed 10 Oct 2011] Neil Turner. UX for the masses. [online] http://www.uxforthemasses.com/online-card-sorting/ [Accessed 10 Oct, 2011.] Hiroshi Ishii, Minoru Kobayashi, Kazuho Arita. Iterative design of seamless collaboration Media. August 1994, vol. 37, No.8. Kildare, R., Williams, R. N., Hartnett, J.,2006. An Online Tool for Learning Collaboration and Learning While Collaborating. Eighth Australasian Computing Education Conference (ACE2006), Vol. 52. Sutcliffe, A., 2005. Applying Small Group Theory to Analysis and Design of CSCW Systems. Human and Social Factors of Software Engineering (HSSE). Katsanos, C., Tselios, N., Avouris, N.,2008. AutoCardSorter: Designing the Information Architecture of a Web Site Using Latent Semantic Analysis. CHI 2008, April 5–10, 2008, Florence, Italy. Copyright 2008 ACM 978-1-60558-011-1/08/04. 42
References Joly, A.,V., Pemberton, L., Griffiths, R.,2009. Card Sorting Activities with Preschool Children. BCS-HCI 09: Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology. Jakob Nielsen. "Card Sorting to Discover the Users Model of the Information Space".Useit. 1995.[online] http://www.useit.com/papers/sun/cardsort.html [Accessed on 10 Oct 2011] Schadewitz, N., Zakaria, N., 2009. Cross-cultural Collaboration Wiki – Evolving Knowledge about International Teamwork. IWIC’09, February 20–21, 2009, Palo Alto, California, USA. ACM 978-1-60558-502-4/09/02. Pekkola, S., 2003. Designed for Unanticipated Use: Common Artefacts as Design Principle for CSCW Applications. GROUP’03, November 9–12, 2003, Sanibel Island, Florida, USA. Rusu, C., Roncagliolo,S., Tapia, D., Hayvar, D., Rusu, V.,, Gorgan, D., 2010. Evaluating the Usability of Intercultural Collaboration Platforms: Grid Computing Applications. ICIC10, August 19– 20, 2010, Copenhagen, Denmark. 43
References Card Sorting- Introduction[Online]. Available from http://www.syntagm.co.uk/design/cardsortintro.shtml [accessed 25 August 2011] Miller J. K., 2010. Intercultural Interaction and Collaboration in a Virtual World. ICIC’10, August 19– 20, 2010, Copenhagen, Denmark. Lewis, S., Ellis, J.B., Kellogg, W.A. 2010. Using Virtual Interactions to Explore Leadership and Collaboration in Globally Distributed Teams. ICIC’10, August 19–20, 2010, Copenhagen, Denmark. Donna Spencer and Todd Warfel .Boxes and Arrows.[online] http://www.boxesandarrows.com/view/card_sorting_a_definitive_gu ide.[Accessed 25 August, 2011.] Hudson, W., 2007. Old Cards, New Tricks: Applied Techniques in Card Sorting. Published by the British Computer Society Volume 2 Proceedings of the 21st BCS HCI Group, HCI 2007, 3-7 September 2007, Lancaster University, UK. Kodagoda, N., Wong, B. L. W, Khan N., 2010. Open-Card Sort to Explain Why Low-Literate Users abandon their Web Searches Early. BCS 10: Proceedings of the 24th BCS Interaction Specialist Group Conference. 44
References Sinha, R., Boutelle, J., 2004. Rapid Information Architecture Prototyping. DIS2004, August 1- 4, 2004, Cambridge, Massachusetts, USA. Qian, X., Yang, Y., Gong, Y., 2011. The Art of Metaphor: A Method for Interface Design Based on Mental Models. VRCAI 2011, Hong Kong, China, December 11 – 12, 2011. Chandler, H.E., 2001. The Complexity of Online Groups: A Case Study of Asynchronous Distributed Collaboration. ACM Journal of Computer Documentation, 2001,25:1-2. Tullis, T. and Wood, L., 2004. How Many Users Are Enough for a Card-Sorting Study? In Proc UPA’2004. Dong, J.M., Shirley, M. and Waldo, P. A user input and analysis tool for information architecture. In Proc CHI 2001, ACM Press (2001), 23-24. Rosenfeld, L. and Morville, P. Information Architecture for the World Wide Web, 3rd edition, O’Reilly, 2006. Hudson, W. Playing your cards right: getting the most from card sorting for navigation design, ACM, 2005, 56-58. 45