Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Shannon Holloway | Research Portfolio


Published on

Research Portfolio

Published in: Design, Technology, Business
  • Be the first to comment

Shannon Holloway | Research Portfolio

  2. 2. Shannon Holloway | Research + Design 2017 Hi! I’m Shannon Holloway. Thank you for taking the time to review my portfolio. I'm passionate about solving complex problems with human-centered design methods. I lead interdisciplinary teams to develop high-impact research strategies, translate research findings into actionable insights, design human- centered solutions, and promote knowledge sharing. I'm a self-starter with 8+ years of experience realizing new initiatives in industry, academia, and government. I received my M.S. from New York University Tandon School of Engineering in Integrated Digital Media and my B.A. from the University of California at Berkeley in the Practice of Art. I'm a James Dyson Design Leader and member of the Human Factors and Ergonomics Society and the Environmental Design Research Association. In my free time I enjoy working on my own artistic and DIY projects, riding my bike and getting lost in a good sci-fi novel. 2
  3. 3. Shannon Holloway | Research Portfolio 2017 3 Skills Stakeholder Workshops Insight Generation Product Strategy Cross-Functional Teamwork Agile Development Process Planning and Roadmapping Presentations and Storytelling Customer Journey Mapping Project Management Best Practices and Standards Competitive Analysis Interviews and Ethnography Surveys and Analytics Usability Testing Heuristic Evaluation Experimental Research Methods Web Accessibility UX and Interaction Design Service and Organization Design Information Architecture Qualtrics Google Analytics Omnigraffle Invision Axure Balsamiq InDesign Photoshop Adobe Premiere JIRA, Rally, Trello HTML, CSS JavaScript Markdown Python, R Sublime Text Git Xcode Wordpress Google Apps MS Office Suite Tools
  4. 4. Research Lead NYU Ability Project
  5. 5. Shannon Holloway | Research Portfolio 2017 The NYU Ability Project partnered with Charter Spectrum and the NYC Media Lab to research universal design in home entertainment. My responsibilities as Research Lead include leading an interdisciplinary team in conducting user research, rapid prototyping and testing with individuals with disabilities to inform Charter’s accessibility strategy. 5
  6. 6. Shannon Holloway | Research Portfolio 2017 Charter/Spectrum provided our team the following goals: 1. Humanize access issues and our research (less abstraction, more stories) 2. Consider a range of disabilities 3. Model a framework for inclusive design 4. Present ideas across platforms 5. Address metadata issues 6. Address institution-wide adoption of accessible design and development principles From its inception, priority was given to establishing participation and eliciting feedback from community members with disabilities. Partnerships with several New York-based organizations were formed to recruit participants who have first- hand experience with access issues in home entertainment. 6 Project Goals
  7. 7. Shannon Holloway | Research Portfolio 2017 We researched media distributors, in particular their strategic partnerships with smart home technologies and acquisitions of artificial intelligence solutions to streamline metadata. We also researched applications for emerging technologies in assistive tech and overall trends in media consumption to understand the direction of accessibility and home entertainment. 7 Competitive Analysis
  8. 8. Shannon Holloway | Research Portfolio 2017 Usability testing of Spectrum’s products was conducted on the TV set-top box, Roku, Xbox, iPad and website. Stations for each device were set up around the lab with at least one member of the Ability Team. After conducting product-specific task analyses, we identified user goals around searching for live TV and on-demand programming, watching video, and adjusting the accessibility settings to suit their needs. The goals informed testing tasks and scripts that were consistent across devices. 8 Usability Testing
  9. 9. Shannon Holloway | Research Portfolio 2017 9 The first Spectrum testing session had a total of 9 participants. Before testing, participants were briefly interviewed about their demographic and ability-related information. Participants in this first round of testing were between the ages 28-64, and reported watching TV an average of 5 times a week, using a variety of methods (streaming online, using mobile apps, set-top boxes and digital media players). Participants spent approx 10 minutes at each station and were asked to ‘think aloud’ as they interacted with each system.
  10. 10. Shannon Holloway | Research Portfolio 2017 10 TV Set-Top Box The remote was found to have too many buttons, requiring users to begin most tasks by carefully searching and reading each button on the remote to find an option they could use. The remote usually required two hands to use because the user couldn’t access all the buttons with one finger. The size of the buttons made them difficult to read and difficult to press and lead to frequent user error. On the TV UI, search tools were not easy to find and in fact, none of the users knew it was available. Users completed the search task by changing channels or scrolling through the guide. Users struggled to move forward and back through menus, frequently exited menus by accident and had to start over. This was caused in part by difficulties using the remote control and the lack of guidance/direction in the menus themselves. The TV STB and Charter remote provide very little accommodation for users with limited visual abilities. This is a Priority 1 accessibility issue that makes the system unusable and excludes this user group entirely.
  11. 11. Shannon Holloway | Research Portfolio 2017 11 Xbox The XBox One Controller was difficult for users to manipulate due to the size and shape of the controller, the size and placement of buttons, and the required force to press a button. Users experienced difficulty navigating through menus and understanding which option was currently selected, mistaking the Spectrum logo as the current option due to low contrast colors. Users were confused by menus and controls not intuitively mapped to user inputs on the controller. Timeouts of screens required users to act quickly or lose the screen they were at and not be able to complete a task. The Spectrum app was completely inaccessible to screen reader users and could not be tested with them. This is a Priority 1 accessibility issue that makes the system unusable and excludes this user group entirely.
  12. 12. Shannon Holloway | Research Portfolio 2017 12 Roku Two out of four users with mobility impairments required the remote to be taped to their wheelchair. The buttons required too much pressure to push down with a stylus, and the rounded bottom of the remote made it slide away from the user. The Search feature required tedious input using the arrows on the remote to select letters on an on- screen keypad. On Demand was organized for browsing and made it difficult to find specific content. Screen readers accessed the Voice Guide with headphones plugged into the remote. All users found it difficult to hear the Voice Guide over the Live TV in the background upon launch and the inability to control separate volumes left users disoriented. User described the vocal tone as “robotic”, “horrible”, “muffled”, and “the voice speed is killing me”. The lack of crisp pronunciation, such as “buddin” instead of “button”, contributed to user error.
  13. 13. Shannon Holloway | Research Portfolio 2017 13 Website All users reported that the font size was too small and could not be adjusted. Testing administrators had to zoom to 125% to conduct testing. It took a lot of scrolling and clicking to find content and begin playback. Search results poorly labelled which content is available or upcoming, with the invitation to “Watch the latest episode” as standard copy on every program page, even when content is not yet available. Testing with screen reader users revealed that some primary features of were incompatible with screen reader software. Users reported higher overall success using JAWS software rather than VoiceOver. However, users noted that the site organization is unconventional and recommended creating a JAWS screen reader guide for navigating Screen reader users reported physical and mental fatigue from trying to use The error message overlay interrupted site functionality, but could not be detected by or dismissed by the screen reader. The error message could only be dismissed by mouse clicking on “OK,” which the testing administrator had to intervene.
  14. 14. Shannon Holloway | Research Portfolio 2017 14 iPad Users with mobility impairments experienced difficulty using the iPad app mainly due to the motions and precision required while interacting with the controls. Users reported exhaustion from extensive scrolling and multiple failed attempts at selecting icons that were too small, including letters on the keypad for the Search feature. Users reported difficulty reading due to small text size and low color contrast. For screen reader users, the iPad app was difficult to use because the page layout is not compatible with screen readers. For example, the search function was hard to find. Users were unable to hear what they had typed into the search field and the results of their search. Many key functional buttons lacked labels for screen readers including Mute, Back, Closed Captioning, and Full Screen. In On Demand, the “On Demand” button does not make it clear that its function is to play content. Some elements appeared to be buttons but weren’t actually buttons. Categories in menus were not labeled as headings. For Live TV, the app does not announce the channels in which certain shows are playing.
  15. 15. Shannon Holloway | Research Portfolio 2017 15 Insight: Users fall into three main groups based on their watching habits. We identified 3 personas based on user behavior: the traditional user, exploratory user, and streamer. Insight: Users prioritized certain features aligned with their goals. A traditional user prioritizes Live TV features and does not adjust settings, while a streamer cares about personalized content and accessibility settings. Insights
  16. 16. Shannon Holloway | Research Portfolio 2017 16 Insight: Users prefer to use their personal devices already customized for their needs. Integrate with their devices already customized for use with their preferred hand(s), voice tone, text size, color contrast, etc. Insight: Users multitask with various devices while watching. Mobiles devices serve as second screens that provide an opportunity space for engaging and interactive experiences. Insight: Users want to find their favorite content quickly and easily. Prioritize search and bookmarking features to easily find favorite shows and channels. Insight: Users want to find content fitting their accessibility needs quickly and easily. Clearly denote accessible content and make sure its availability is accurate. Insight: Partial access means no access. Meeting the accessibility requirements mandated by law does not guarantee an accessible end-to-end user experience in implementation. Insight: Good UX is critical for user acceptance. Even if accessible, users’ overall attitude towards the system impacts whether they use the system altogether. Insight: Users are adept at learning new systems and would like guidance. Onboarding materials can provide an overview for users to quickly and easily learn how to use them. Insight: Access should include the entire customer journey. Diverse user needs should be considered from set-up to usage to troubleshooting problems.
  17. 17. Shannon Holloway | Research Portfolio 2017 17 Experimental Prototyping + Testing Based on these insights, the team built experimental prototypes that ranged from accessibility modifications to new concepts. 1. GUI prototype 2. HTML screen reader-compatible prototype 3. Remote control prototypes 4. Onboarding experience prototype 5. AI + crowdsourced descriptions prototype In our second testing session we had a total of 11 participants and their ages ranged from 30-66. Participants reported a range of disabilities including blindness, low-vision, deafness, cognitive and mobility issues. Each user reported watching television an average of 6 times a week using set-top boxes, Rokus, mobile apps and other streaming sources.
  18. 18. Shannon Holloway | Research Portfolio 2017 18 GUI Prototype The Graphical user interface (GUI) prototype was built with accessible design enhancements for the iPad and laptop. The design incorporated WCAG 2.0 standards for 18pt font or 14pt bold text. The prototype featured a splash page upon launch for users to orient themselves, a global navigation with a prominent Search feature, and a universal access icon for easy access to accessibility settings. Users did not experience difficulty completing tasks during testing.
  19. 19. Shannon Holloway | Research Portfolio 2017 19 Screen Reader Compatible Prototype A screen reader compatible prototype was developed in HTML for the iPad and laptop intended for screen reader users. The design focused on the screen reader user pain points revealed in usability testing: difficulty finding the search feature, improper labelling of page elements, and inconsistent information architecture that made it difficult or at times impossible to navigate. The screen reader prototype had a prominent Search feature that loaded the page with the search bar already selected so any typing is captured and immediately inputs text. The prototype was designed with only one global navigation included in the “head” of the HTML rather than the “main” body of the page to avoid repeating the menu when navigating to different pages. Title text was included for every navigable item on the page, including images, to enable screen reader users to search for specific items rather than browsing. The HTML prototype optimized for screen reader compatibility was tested with three screen reader users on the iPad and laptop. This text-only prototype had a clearly structured information architecture that enabled users to complete all the tasks without difficulty during testing.
  20. 20. Shannon Holloway | Research Portfolio 2017 20 Remote Control Prototyping Participatory prototyping techniques were employed in reimagining the remote control at the “Build Your Own Remote” station. Users talked through their process while prototyping to facilitate co-creation between designers and users. There were strong existing expectations for the layout of the remote which guided each participant’s design and perceived ease of use. Users had a tendency to create two separate sections - a control panel and a direct input area.
  21. 21. Shannon Holloway | Research Portfolio 2017 21 Based on the findings from the participatory prototyping session, the team proposed concepts for short and long term. A remote control app features control panel and direct input areas and would be implementable in the near-term. A future concept out of malleable smart material would mold to the surface the user places it on to help it stay put during use, provide haptic feedback when input is received, is rechargeable and easy to find when lost. “Agent picks up dog behind crime scene tape.”
  22. 22. Shannon Holloway | Research Portfolio 2017 22 Onboarding Experience Prototype At the onboarding station, participants were asked for their feedback on the onboarding experience prototype, chatbot feature, and friendly error messaging. Users expressed that they like onboarding that is quick, fun, easy, and informative. They preferred the onboarding experience to focus on elements in the UI/UX that are not necessarily the most obvious features. All users wanted the option to dismiss onboarding—no matter where in the UI/UX it is presented. When asked which features the users think are important to include in the onboarding, the responses included: all accessibility features, especially screen reader features (most importantly), the position of buttons on the remote control (as this can “eliminate the need for additional onboarding in many cases”), the general settings, and the privacy settings. Importantly, the accessibility settings page should be organized by feature and not by disability since users with multiple disabilities or no disabilities adjust these settings.
  23. 23. Shannon Holloway | Research Portfolio 2017 23 Video Description Prototype “Agent picks up dog behind crime scene tape.” Finding content with second audio programming (SAP) and video descriptions was very difficult for users. Our prototype proposed crowdsourcing video descriptions and using IBM Watson to filter and detect bias. Video playback delivered an extended video description (EVD) when complex visual information was presented in a scene. Screen reader users were asked to watch an original Law and Order video clip by itself and then watch it with the EVD prototype. The action in the scene focuses on a dog running under a police tape and cannot be understood int he dialogue alone. After watching the first time, users said they felt “frustrated” and “that I’m missing a lot, I wish I knew what was going on”. Users had overwhelmingly positive responses about the quality of the descriptions and made comments like “that was so awesome. I wish I could take this home” and “I would have liked to watch the whole thing”.
  24. 24. Shannon Holloway | Research Portfolio 2017 24 Metadata Issues Gracenote metadata is provided to Charter as a package of XML files, the schema of which informs the structure of the product navigation UI across Charter’s product line, and the content of which is used to populate TV schedules, program summaries, allow for searching by, e.g. primary cast members, and many other useful features. Our analysis revealed accessibility features (Closed Captioning and Descriptive Video Service, tagged as 'CC' and 'DVS', respectively) are identified within the broadcast schedule metadata ('schedules.xml') within a single field (<quals>). For comparison, the ratings / advisory metadata provided by Gracenote on a per-program basis is quite comprehensive, listing content descriptors such as ‘Graphic Language’ under a well-defined <ratings> heading in the 'programs' file. While a remedy to this situation lies upstream from Charter, we endorsed that Charter, as Gracenote's customer, could request that the metadata provider reorganize and prioritize how it delivers accessibility information.
  25. 25. Shannon Holloway | Research Portfolio 2017 25 UX Recommendations Small changes can result in dramatic improvements to the overall user experience. Support the user’s ability to achieve their goals. Leverage user behavior by adhering to design conventions. Emerging technologies present rich opportunities for accessibility. Improvements in access benefit users of all abilities. Company-Wide Recommendations Broad mandate to engage with employees on the topic of accessibility. New employees introduced to the Accessibility Team as part of onboarding. Existing employees are regularly invited to engage with Accessibility Team. The Accessibility Team reviews all new products and has the mandate to review all products. The Accessibility Team has sufficient resources to evaluate the entire product line
  26. 26. Research Assistant NYU Tandon MakerSpace
  27. 27. Shannon Holloway | Research Portfolio 2017 27 The NYU Tandon School of Engineering MakerSpace is a collaborative workspace and lab that encourages students and faculty to engage in innovative and entrepreneurial activities. As a Research Assistant I worked with Professor Anne-Laure Fayard on qualitative research and Victoria Bill, the MakerSpace manager and quantitative researcher, on understanding user needs and optimizing the new space.
  28. 28. Shannon Holloway | Research Portfolio 2017 28 I conducted observations and in situ interviews in the MakerSpace and comparative maker spaces on campus. A total of twenty-one open-ended interviews were conducted. At the beginning of the project, stakeholder interviews were conducted with participants in the design of the MakerSpace: three with members of the administration, three faculty members, and three students (N=9). Additional interviews with students were conducted (N=12). Methods
  29. 29. Shannon Holloway | Research Portfolio 2017 29 A Welcoming Environment The open plan space and concierge desk were designed to invite newcomers to ask for help. However, the observations and in situ interviews indicated that many users coming to the space did not feel welcome. Often newcomers’ first interaction with the staff was negative, telling them no food allowed or pointing to a machine to swipe their ID card. Many students ended up sitting in the foyer area outside of the MakerSpace, which has become a buffer space for students to meet and food is allowed. While the space had furniture on wheels for flexibility, the cleanliness of the space and persistence of the original layout in rows of tables likened a library more than a place for creativity and experimentation. Students reported feeling on display and did not want to mess up.
  30. 30. Shannon Holloway | Research Portfolio 2017 30 Equipment and Space Usage Most students used the space for homework or 3D printing personal projects, mainly on the Ultimakers since the materials were free. A variety of implicit barriers impact student participation including fear of failure, cost, gender, and lack of training. Students reported feeling intimidated and wanted to see examples of student projects and additional entry-level trainings. The concierge displayed some student work but many students inquired about purchasing objects rather than how to make them. One team project was showcased in the space while other teams did not have a visible presence to demonstrate the work students can and are doing in the space.
  31. 31. Shannon Holloway | Research Portfolio 2017 31 Reported usage in the quantitative data was consistent with our qualitative findings. Most users attended the safety orientation, which is required before use of the 3D printers and attending other training sessions, and attendance for other trainings dropped off sharply. Working on personal projects was the top reason students came to the MakerSpace with 63% of respondents reporting this, the second highest reason being for homework at 55%. By far 3D printers were the most popular equipment used by 62% of respondents, which was reflected in observations of the Ultimakers as the hub of activity.
  32. 32. Shannon Holloway | Research Portfolio 2017 It was found – in line with the literature on communities of practice – that the MakerSpace community is composed of four main group of users: the student TAs, who are a very closed group; a small group of core expert users (15 to 20 members, mostly male students) who are often had prior connections with the TAs; regular users who seemed to be using the space mostly for its co-working and meeting facilities, and some infrequent users. The observations and informal conversations with other students in the space suggest that there was a strong in-group vs. out-group distinction that made it difficult for students who were not part of the TA or core users group to engage with these groups. Additional trainings, workshops, and student club collaborations could create new pathways that might increase serendipitous interactions among different sub- communities. 32 Communities of Users
  33. 33. Shannon Holloway | Research Portfolio 2017 33 The neutral and negative responses to meeting other majors, meeting others with similar interests, and preference to work alone suggest the limited community involvement between core users and newcomers.
  34. 34. Shannon Holloway | Research Portfolio 2017 Turning insights into action, we partnered with the MakerSpace management to design, test and iterate on spatial and organizational design solutions to foster a collaborative and inclusive environment. For example, moving the ID card swipe machine contributed to more welcoming and positive interactions with staff members, as well as an increase in compliance with ID card policy and attendance data collected. The lounge chairs were moved from the middle of the space, where students were observed sleeping, to the entrance of the space. This served the need for an informal social meeting place observed in student use of the outside foyer, and observations of this change were positive. 34 Prototyping
  35. 35. Shannon Holloway | Research Portfolio 2017 Spatial and organizational recommendations included: • create areas with a sense of privacy to encourage experimentation • display available tools and materials in the open • showcase student projects in an exhibition area and on screens • create a “needs and wants” wall for students to build teams • train TAs to mentor and partner with students on projects • have a standard system in place for when ID is required and reduce number of requests that require ID 35 Recommendations
  36. 36. Master’s Thesis NYU Tandon School of Engineering
  37. 37. Shannon Holloway | Research Portfolio 2017 The line is blurring between our physical and digital worlds. There is a new hybrid existence being created by the proliferation of displays in our real environment and the integration of the real world in virtual environments. Mixed reality (MR) is a continuum of environments between completely real environments and virtual ones [1]. MR poses new challenges for interactions in both our physical and virtual worlds. In MR environments, humans interact with real and virtual objects together in context. However the technologies that enable MR are designed for direct user interaction rather than considering the larger context within which a human interacts with a system. This creates a fragmented user experience and forces users to be split between real and virtual worlds. A design process focused on human needs in context contributes to more positive user experience outcomes and user acceptance of MR technologies. How might we promote the human-centered design of mixed reality environments? 37 Research Question
  38. 38. Shannon Holloway | Research Portfolio 2017 Failures in software projects cost the US economy an estimated $25 to $75 billion annually, and many of the reasons for failure are preventable [2]. Delivery without adequate requirements from customers and users is a problem found in 73% of failed software projects [3] and can be avoided with UX research. UX research is necessary to understand users, whose needs change little over time while the features and technologies change rapidly [4]. UX is a major factor in whether or not people use a system. as illustrated in Davis’ technology acceptance model [5]. Today’s agile development methods prioritize shipping product over proper UCD process [6]. Lightweight tools for the UCD process, such as scenarios and personas, are effective ways to make it more agile and communicate a strategic UX vision with team members and stakeholders [7]. MR poses additional challenges as it requires advanced programming skills [8] and high-fidelity prototypes don’t always solicit helpful feedback [9]. There is currently a lack of structured approaches to MR design and trial and error is the best choice [10]. 38 Background
  39. 39. Shannon Holloway | Research Portfolio 2017 39 I researched the available tools to understand user context as this is critical for MR environments. Personas outline user needs, goals and tasks but do not provide details on context of use. The Empathy Map [11] has been adapted for use in the design process [12] but is better suited for market research rather than user research. The Activity Checklist [13] is a comprehensive list of user context considerations but is difficult to use and does not map to the design process. The MUSiC method [14] provides a valuable framework for capturing the context of use but does not offer insight from the perspective of the user to inform UX. Clockwise from top left: persona, the Empathy Map, the Activity Checklist, the MUSiC method. Comparative Analysis
  40. 40. Shannon Holloway | Research Portfolio 2017 40 I decided to try rapid prototyping in the context of use to explore new methods for user research in mixed reality environments. These low- fidelity prototypes focused on navigation as this continues to be a critical usability issue with web design and with wayfinding systems in physical spaces. However, I found each prototype follows the assumptions of the specific technologies delivering the experience and it was difficult to generalize my findings across them. Clockwise from top left: Neon duct tape, digital mockup, “Protorama” physical interface, mobile AR navigational aid. Rapid Prototyping
  41. 41. Shannon Holloway | Research Portfolio 2017 The existing method that is closest to capturing user context in a way that supports the design process for user experiences is activity theory. Activity theory posits that human activity is motivated by transforming an object into an outcome, which is mediated by tools and embedded in its context [15]. I designed a large-format Activity System printed as an 11x17 poster so it was large enough to use with familiar tools in the research and design process: post-its an sharpies. I decided the only way to see if it was effective was to test it in a case study with a real user in an MR environment. 41 Activity Theory in Practice
  42. 42. Shannon Holloway | Research Portfolio 2017 I interviewed a smart home user who lives in a one-bedroom apartment with his cat. His motivation to use smart home technologies was to automate processes to adjust his environment to his preferences while he was at home and save energy while he was away. I created a scenario around his arrival at home, when he would adjust the lighting and temperature through voice input and locomotion. Analyzing this scenario in the Activity System gave me the perspective I was searching for. It provided a clear definition of the activity and the contextual factors involved. The Rules section provided rich UX insights in the scenario. As the head of the household, the user felt undermined when he had to repeat himself to the device and said “I need it to respect my authority”. Such rules have direct implications on possible design enhancements for the Amazon Echo. Some design solutions could be to increase the radius and volume to which it responds to voice input, accommodate a wider vocal range and various pronunciations, and have it respond in a way that is more deferential to the user’s authority. 42 Case Study 1: Smart Home Environment
  43. 43. Shannon Holloway | Research Portfolio 2017 43 I found there to be some redundancy in the findings under “Division of Labor” and “Community” as the unit of analysis in this framework is much larger than the scenario I was using it for. I also found that contextual factors in the physical space were overlooked. I tested out replacing “Division of Labor” with “Environment” and found this change enabled me to include findings that were more relevant to the UX research and design process.
  44. 44. Shannon Holloway | Research Portfolio 2017 I had the opportunity to test out my v2 prototype with UX researchers at the NYC UX/User Researcher Meetup. First I gave a talk on mixed reality environments and introduced my Activity System as a tool I have found useful in my research. Afterward 35 UX researchers split into groups for an interactive workshop using MR scenarios. Printed copies of my Activity System were on the tables for the groups’ use. Many groups ignored it while they discussed technical constraints and research methods they were more familiar with. Finally, each team shared their research approaches and there was consensus that MR environments were very different from their current work. Out of 35 users the majority did not actively engage with the system during their team research planning. However after the workshop, 5 people I will call “super users” took copies of the system with them and it really resonated with them. What these users had in common was some background in order to understand the system’s value - either they were familiar with the theoretical side or they had experience in the MR space. 44 Testing 1: UX Researchers
  45. 45. Shannon Holloway | Research Portfolio 2017 45 User testing provided some key findings to consider in improving my prototype. First, the design of the document was not approachable or easy to understand. Second, the system requires more thorough explanation/ onboarding before use. Third, it appeared that the system was more suitable for the synthesis phase rather than at the outset of a project. The scenarios had sparse information on the user and researchers would need more information in order to find the Activity System useful. The Activity System v2
  46. 46. Shannon Holloway | Research Portfolio 2017 46 Activity theory provided a good foundation for analyzing context. However I needed to adapt it to understand a user’s context in their interaction with a system incorporating digital and physical components. I designed the Concept Map as a worksheet to easily fill in research findings. I represented the user with a full body to reinforce that consciousness is intertwined with human activity in context. Interaction with these systems is not an isolated cerebral process but can happen with inputs such as gaze, voice, gesture, locomotion, etc. The Context Map v1
  47. 47. Shannon Holloway | Research Portfolio 2017 For my MR HMD case study, my user was a graduate student at NYU Tandon School of Engineering using the Microsoft HoloLens in a classroom at MAGNET. In the interview she described her research on virtual avatars and the importance of being able to see and interact with them in 3D space. I created a scenario around her workflow in which she would record a POV video of her interaction with a 3D hologram in space to provide feedback to collaborators or design managers, hologram, placed it on a desk, resized it and was then able to inspect it from various angles. As an observer I could not see the holograms she was interacting with. It was critical to get her POV video not only for the scenario but also for documentation. While the task interacting with a hologram was straightforward, recording a POV video proved to be the most challenging part of the scenario. The user had to already be in the Holograms app and start recording by voice command, “Hey Cortana, record a video.” Cortana did not always recognize the voice command and would pull up search results in a web browser. The system provided little feedback on its status. This case study reflected some of the key concerns with sensing systems raised by Bellotti [16]. We were unsure of whether it recognized the command, was processing the request, or recording the video. 47 Case Study 2: MR Head-Mounted Display
  48. 48. Shannon Holloway | Research Portfolio 2017 48 Analyzing the MR HMD case study with the Context Map was easier with the updated language. Again, the Rules section provided valuable UX insights that could inform future design enhancements. These implicit rules included pronouncing English a certain way to be recognized by the system, staying within the spatially mapped area, and having the consent to take video from others in the room. Improvements to the software and hardware could include support for a wider range of English pronunciation and improved feedback when recording video.
  49. 49. Shannon Holloway | Research Portfolio 2017 I tested the Context Map with 19 students in the graduate level UX course at NYU Tandon School of Engineering. The class was midway through an MR design challenge partnered with Microsoft HoloLens acting as their client and familiar with the unique challenges in designing MR user experiences. Isked them to try using the Context Map with a HoloLens scenario they were already familiar with in class. After the exercise we had a group discussion on their experience using the Context Map. The majority of users said they could see themselves using the Context Map for future projects. Some students said they wished their team had this tool at the beginning of their client project in order to make sure they considered all the contextual factors in the MR experience they designed. They had encountered problems with their design working with the user’s physical space and believed the Context Map would have guided them to design around this earlier in the process.They wanted to see more case studies using the Context Map to see it in action and better understand how to use it. 49 Testing 2: MR UX Designers
  50. 50. Shannon Holloway | Research Portfolio 2017 50 User testing provided some key findings for improvements to the Context Map. First, the Context Map was intimidating to new users. It was not clear how to translate the information provided in the scenario to the Context Map format. Second, the Context Map required more thorough onboarding/education before use. Many suggested providing more examples of the Context Map in action to see what types of information fit where. Third, the Context Map proved to be useful for the research and planning phases of a project. The students’ experience with designing MR experiences enabled them to recognize the value of this tool in the design process.
  51. 51. Shannon Holloway | Research Portfolio 2017 51 My iteration on the Context Map reconceptualized the original triangular structure inherited from the activity system. I restructured the hierarchy of contextual factors relative to the needs for the user experience. At the base of the pyramid is Tools, which are system components involved in the activity. Tools represent the minimum system requirements for the interaction to take place, i.e. hardware, software, wi-fi, etc. The next level is Physical Space, the objects in the space as well as the natural and built environment. This includes some factors required for interaction to take place such as sufficient visible light for holograms to display and defined spaces for spatial tracking, but is also an area to take note of surfaces and furniture, the relationships of connected devices to each other, and other affordances and constraints in the user’s physical environment. The next level of the hierarchy is People, which includes stakeholders and community members. Identification of the relevant network of stakeholders and their relationships to the system and each other is an important step in requirements gathering [17]. Included are parties involved in the background such as hardware and software providers, third-party data buyers and advertisers. The apex of the hierarchy is User Needs such as preferences and social norms. The previous title, Rules, caused confusion for users and the findings usually pointed to implicit user needs and pain points.
  52. 52. Shannon Holloway | Research Portfolio 2017 52 Once I established the relationships of the contextual factors to each other, I sketched out ways to visualize them while focusing on the user journey. I looked at diagrams of ecosystems for inspiration on how to map out relationships in complex interdependent systems. The resulting diagram of user interaction in the context of use provides a framework for describing the context in which a human interacts with a system. I separated User Input from the Tools category and placed it next to the figure of the user, interfacing with the Touchpoint of the system and pointing toward the user’s Goal. Emanating from the user are rings of contextual factors in hierarchical order of their impact on the user’s experience.
  53. 53. Shannon Holloway | Research Portfolio 2017 53 Next, I created a new version of the Context Map. The 8.5x11 worksheet format is more approachable and easier to fill out multiple worksheets for various contexts throughout a user journey. The focus is on the user journey by featuring the User Needs, Touchpoint, and Goal at the top. The hierarchy of contextual factors and descriptions are in the bottom-right corner of the worksheet to guide the user. I tested it with my MR HMD case study. The 8.5x11 format made the tool more approachable and I could easily complete another one if it needed revision. The Context Map v2
  54. 54. Shannon Holloway | Research Portfolio 2017 I returned to the graduate-level UX course at NYU Tandon School of Engineering to test v2 of the Context Map. I asked them to analyze the same motorcycle designer scenario from the last round of testing. Once students completed the exercise they completed a 9-question survey designed to evaluate the Context Map and solicit student feedback. Questions 1-5 measured student comprehension and perceived usefulness of the Context Map. Questions 6-8 were open response items about strengths and weaknesses of the Context Map. The survey results show that students had a positive experience using the Context Map. While v1 was challenging for some students to translate the information in the scenario to the worksheet, v2 was easier to understand. 75% of respondents agreed or strongly agreed with the statement “I understood how to use the Context Map worksheet for the scenario.” The Context Map was deemed useful in the design process by 69% of students. Overall, 62% of students reported that they would use the Context Map in their projects. 54 Testing 3: UX Designers
  55. 55. Shannon Holloway | Research Portfolio 2017 55 Answers to the open response items suggested changes to the terminology and examples for each category would improve comprehension. Many students wrote about how they did not understand the involvement of other people at all. One student wrote, “I don't understand how/why the stakeholders and community fit into what the user is doing. The stakeholders are not the ones using the app, the user is. So why are they included in the functionality?” These statements shed light on students’ mental model of direct one-to-one interaction between a user and system. Further research is needed to demonstrate that the new range of input modalities in MR environments, regardless of the technologies enabling them, impact the people and environment in the context of use. The survey results on the students’ backgrounds were illuminating. Most students identified their background was in Design or both Design and Engineering, and positive responses correlated with these respondents. Negative responses correlated with those who identified their background was in Engineering. This finding points to a gap in reaching this particular audience. Further effort is needed to understand the engineering mental model and reasons why they disagreed with the terminology, logic, or visual representation of the Context Map.
  56. 56. Shannon Holloway | Research Portfolio 2017 [1]. Milgram, Paul, and Fumio Kishino. "A taxonomy of mixed reality visual displays.”IEICE TRANSACTIONS on Information and Systems 77.12 (1994): 1321-1329. [2]. Charette, R. n. "Why Software Fails." IEEE Spectrum 42.9 (2005): 36-43. Web. Oct 23, 2016. [3]. Cerpa, Narciso, and June M. Verner. "Why did your project fail?" Communications of the ACM 52.12 (2009): 130-134. [4]. Beyer, Hugh, Karen Holtzblatt, and Lisa Baker. "An agile customer-centered method: rapid contextual design." Conference on Extreme Programming and Agile Methods. Springer Berlin Heidelberg, 2004. [5]. Davis, Fred D. "User acceptance of information technology; system characteristics, user perceptions and behavioral impacts.” International Journal of Man—Machine Studies 38 (1993): 475-487. [6]. McInerney, Paul, and Frank Maurer. "UCD in agile projects: dream team or odd couple?” Interactions 12.6 (2005): 19-23. [7]. Kollmann, Johanna, Helen Sharp, and Ann Blandford. "The importance of identity and vision to user experience designers on agile projects." Agile Conference AGILE’09, IEEE, 2009. [8]. Abawi, Daniel F., et al. "Efficient mixed reality application development." 1st European Conference on Visual Media Production (CVMP). 2004. [9]. Rettig, Marc. "Prototyping for tiny fingers." Communications of the ACM 37.4 (1994): 21-27. [10]. Geiger, Christian, et al. "Rapid prototyping of mixed reality applications that entertain and inform." Entertainment Computing. Springer US, 2003. 479-486. [11]. Osterwalder, Alexander, and Yves Pigneur. Business Model Generation: a Handbook for Visionaries, Game Changers, and Challengers. John Wiley & Sons, 2010. [12]. Ferreira, Bruna, et al. "Designing Personas with Empathy Map." SEKE. 2015. [13]. Kaptelinin, Victor, Bonnie A. Nardi, and Catriona Macaulay. "The Activity Checklist: A Tool for Representing the “Space” of Context.” Interactions, vol 6, no. 4, 1994, pp. 27-39 [14]. Bevan, Nigel, and Miles Macleod. "Usability measurement in context." Behaviour & Information Technology 13.1-2 (1994): 132-145. [15]. Kutti, Kari. "Activity Theory as a Potential Framework for Human-Computer Interaction Research.” Context and Consciousness. Activity Theory and Human-Computer Interaction, edited by Bonnie A. Nardi, MIT Press, 1996, pp. 17-44 [16]. Bellotti, Victoria, et al. "Making sense of sensing systems: five questions for designers and researchers." Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2002. [17]. Sharp, Helen, Anthony Finkelstein, and Galal Galal. "Stakeholder identification in the requirements engineering process." Database and Expert Systems Applications, Proceedings from the Tenth International Workshop on IEEE, 1999. 56 References
  57. 57. Shannon Holloway | Research + Design 2017 Thank you for taking the time to review my portfolio. Think we might strike a good match? I would welcome the opportunity to discuss further. Contact 57