Your SlideShare is downloading. ×
9/13/13
1
Improving ERP Usability
Through User-System
Collaboration
Tamara Babaian, Wendy Lucas,
Heikki Topi, and Jennifer...
9/13/13
2
Background
¢  Anecdotal evidence of poor usability of
ERP systems
¢  Poorly designed user interfaces have
nega...
9/13/13
3
Motivation
¢  User’s spend weeks (months) in training, yet
often characterize ERP systems as ‘brutal,’
‘unforgi...
9/13/13
4
Broader Problematic System
Characteristics
¢  Users lack an understanding of how their
narrowly defined tasks a...
9/13/13
5
User notes – example 1
User notes – example 2 –process
description
and 11 more
pages
9/13/13
6
General Approach
¢  Apply Collaboration Theory to the
design and evaluation of ERP user
interfaces
l Collabora...
9/13/13
7
Conceptual Model of
System-User Collaboration
Current Research Program
¢  Three-year NSF grant (2008-2011)
with...
9/13/13
8
Three Broad Principles
¢  Greater collaborative strength of
software yields greater usability
¢  Embedding kno...
9/13/13
9
Research Goal
¢  Improve the usability of ERP systems by
increasing the collaborative capabilities of
their int...
9/13/13
10
Sample Publications and
Working Papers
¢  Xu, J. & Topi, H. “A Conceptual Model for Collaborative ERP
Systems,...
9/13/13
11
Connections with UEF
Projects
¢  “The main aim of DAVID is to enhance
corporate leadership by combining
leader...
9/13/13
12
Example: Illustrations of
CJA, MR, and CMS
CJA-User
¢  How do you know that this is your debit piece and this
...
9/13/13
13
CJA-User – Survey Items
¢  I try to stay aware of the progress of
my tasks
¢  In general, I know how to compl...
9/13/13
14
CJA-System
¢  User: Depending on what I'm going to try to do here. I
forget if I'm in account payable or accou...
9/13/13
15
CJA-Total – Survey Items
¢  <Enterprise System> and I have a shared
understanding about how to complete
busine...
9/13/13
16
MR-User – Survey Items
¢  I have learned how to use <Enterprise
System> in order to adapt to its way of
workin...
9/13/13
17
MR-Systems – Survey Items
¢  <Enterprise System> is able to
learn from how I work with it
¢  <Enterprise Syst...
9/13/13
18
CMS-System – Survey Items
¢  In case of an error, <Enterprise
System> provides mechanisms for
fixing the probl...
9/13/13
19
Example: Proposed Design
Principles (Babaian, Lucas,
Xu, and Topi, DESRIST
2010)
Design Principle 1
¢  DP1. Th...
9/13/13
20
Design Principle 2
¢  DP2. The system should provide navigational
and progress guidance to a user performing a...
9/13/13
21
Design Principle 4
¢  DP4. In presenting selection choices, the
system should utilize what it knows about
the ...
9/13/13
22
Collaborative Critique
¢  A new usability walkthrough method
¢  Questions that the evaluators answer
were for...
9/13/13
23
Viewing Cognitive Walkthrough
Through a Collaborative Lens
¢  Based on the user’s overall goal, will the syste...
9/13/13
24
Example: Use of Process
Graphs to Improve System-
User Communication
(Babaian, Lucas, and Topi,
2007)
Typical E...
9/13/13
25
49
Improving System-User
Communication Via Process Graphs
Add Material
Edit Material
Add Vendor
Process Graphs ...
9/13/13
26
Benefits
¢  Improves system-user communication
of system-prescribed processes and
progress through task
¢  Si...
9/13/13
27
Improving System-User
Communication via Usage Logs
¢  Embed model capturing relationships
between users, actio...
9/13/13
28
Usage Log Examples
¢  Identify required fields based on prior usage
¢  Reconstruct completed processes
l  Re...
9/13/13
29
Automated Usability
Evaluation
¢  Keystroke analysis
l  Components that are prone to user entry
errors
¢  Ti...
9/13/13
30
Work that Continues in the
Project
¢  More in-depth analysis of the interview data
¢  Further formalization o...
Upcoming SlideShare
Loading in...5
×

Improving ERS usability through user-system collaboration

159

Published on

Presentation of Prof. Heikki Topi at edTech Symposium, September 12

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
159
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Improving ERS usability through user-system collaboration"

  1. 1. 9/13/13 1 Improving ERP Usability Through User-System Collaboration Tamara Babaian, Wendy Lucas, Heikki Topi, and Jennifer Xu CIS Department, Bentley University Presentation by Heikki Topi University of Eastern Finland September 12, 2013 Background ¢  Enterprise Resource Planning (ERP) systems integrate business processes across the entire enterprise ¢  ERP tasks automate business processes that typically: l  Have well-defined hierarchical sub-task structures l  Involve multiple users and system modules l  Can extend over days/weeks ¢  Many other types of large-scale enterprise systems share characteristics with ERPs
  2. 2. 9/13/13 2 Background ¢  Anecdotal evidence of poor usability of ERP systems ¢  Poorly designed user interfaces have negative impact on bottom line due to: l  Increases in new user training time l  Decreases in end-user productivity l  Poor user adoption rates (from Forrester Research studies of enterprise and business application usability - Ragsdale 2004; Herbert 2006) Motivation ¢  Research initiative driven by lack of attention paid to addressing significant usability shortcomings of ERP systems by both manufacturers and the usability community ¢  Focus on understanding ways in which users interact with ERP software and degree to which interaction model supports tasks being performed ¢  Address gap between capabilities of ERP systems and how they meet each user’s objectives
  3. 3. 9/13/13 3 Motivation ¢  User’s spend weeks (months) in training, yet often characterize ERP systems as ‘brutal,’ ‘unforgiving,’ and ‘unintuitive,’ because l  System’s logic (process and data flow) opaque to users l  Relationships and interdependencies between tasks difficult to determine l  Semantics of fields, forms, and interfaces hard to understand l  Little if any support for error resolution ¢  Users rely on in-house made documentation, notes, and support by IT departments Categorization of Usability Problems from a Field Study* ¢  Difficulty in identifying and accessing the correct functionality ¢  Lack of transaction execution support ¢  System output limitations ¢  Inadequate support in error situations ¢  Incompatibility between the users’ and the system’s terminology ¢  Usage-related problems arising from the overall complexity of the system *Topi, Lucas, Babaian. Identifying Usability Issues with an ERP Implementation" in Proc. ICEIS’2005.
  4. 4. 9/13/13 4 Broader Problematic System Characteristics ¢  Users lack an understanding of how their narrowly defined tasks are related to the broader process in which they are taking part ¢  Underlying business processes and their relationships are hidden from users ¢  Data-to-process and process-to-process relationships too complex for a user to understand ¢  Local requirements lead to manual tailoring efforts that are very expensive, particularly over time Typical ERP Screens “Easy” Access Menus Data entry form
  5. 5. 9/13/13 5 User notes – example 1 User notes – example 2 –process description and 11 more pages
  6. 6. 9/13/13 6 General Approach ¢  Apply Collaboration Theory to the design and evaluation of ERP user interfaces l Collaboration between user and the system, with system acting as a collaborative partner helping the user achieve system-related goals •  Note that this is not CSCW Human-Computer Collaboration Theory* ¢  Commitment to Joint Activity (CJA) l  Each party in the interaction commits to the joint activity and is aware of the context surrounding their collaboration ¢  Mutual Responsiveness (MR) l  Each party responds to the intentions and actions of the other by adjusting his own behavior based on the behavior of the other’s ¢  Commitment to Mutual Support (CMS) l  Each party is committed to supporting the efforts of the other when that party needs help and will recognize the need and provide assistance *Bratman, 1992; Grosz & Kraus, 1996
  7. 7. 9/13/13 7 Conceptual Model of System-User Collaboration Current Research Program ¢  Three-year NSF grant (2008-2011) with a one-year no-cost extension (2011-12) ¢  Multiple research perspectives and methods ¢  Four faculty members (two with a CS background; two with an IS background) and two IS doctoral students actively involved
  8. 8. 9/13/13 8 Three Broad Principles ¢  Greater collaborative strength of software yields greater usability ¢  Embedding knowledge about the users, tasks, processes, domains, prior usage, and characteristics of interface components into an ERP systems will strengthen its collaborative capabilities ¢  Usage logs are a true reflection of reality Four Primary Methods ¢  Field studies (both interview and survey) to identify usability issues with existing ERP systems from the end users’ perspective ¢  Formulation of design principles based on user input and collaboration theory to guide interface design ¢  Applying design principles to the development of illustrative prototypes and evaluating them in laboratory studies ¢  Specifying new usability evaluation methodologies and evaluating them in the lab and in the field
  9. 9. 9/13/13 9 Research Goal ¢  Improve the usability of ERP systems by increasing the collaborative capabilities of their interfaces through: l  Development of design guidelines and interface evaluation techniques based on collaboration theory l  Implementation of prototype interfaces to validate design and evaluation methodologies Related Research ¢  Human-Computer Collaboration l  Terveen (1995) ¢  Human factors issues and usability characteristics of ERP systems l  Bishu et al. (1999) l  Calisir and Calisir (2004) ¢  User-centered/human-centered design l  Maguire (2001) ¢  Activity modeling l  Constantine (2006): Activity theory & usage-centered design ¢  TAM, UTAUT l  Davis (1989), Venkatesh & al. (2003)
  10. 10. 9/13/13 10 Sample Publications and Working Papers ¢  Xu, J. & Topi, H. “A Conceptual Model for Collaborative ERP Systems,” Bentley University Working Paper. ¢  Lucas, W., Xu, J., and Babaian, T. Visualizing ERP Usage Logs in Real Time. In ICEIS ’13. ¢  Lucas, W. and Babaian, T. “Implementing Design Principles for Collaborative ERP Systems,” In DESRIST ‘12 ¢  Babaian, Lucas, Xu, and Topi, 2010,” Usability through System- User Collaboration: Deriving Design Principles for Greater ERP Usability,” In DESRIST ’10 ¢  Cooprider, Topi, Xu, Dias, Babaian, and Lucas, 2010, “A Collaboration Model for ERP User-System Interaction,” In HICSS ‘10 ¢  Lucas and Babaian, 2009, “Reasoning for Intelligent System- User Interactions with Enterprise Resource Planning Systems,” IJCAI 2009 Workshop on Intelligence and Interaction Connections with UEF Projects ¢  We believe that the key principles of the human-computer collaboration model can be successfully applied to all types of enterprise systems ¢  We are interested in testing the ideas developed in the ERP context in other environments
  11. 11. 9/13/13 11 Connections with UEF Projects ¢  “The main aim of DAVID is to enhance corporate leadership by combining leadership technologies, findings of information systems research, visualization and language technologies.” (Dai, 2013, p. 77) ¢  These goals are, at the general level, not far from the goals of enterprise systems à could we apply HCC principles to evaluate and potentially improve DAVID? Connections with UEF Projects ¢  Expansion of the usability concept beyond user interface issues – our research encourages all designers to consider the collaborative relationship between the users and the system from very early stages of the design l  Moving from ease of use to usability and user experience
  12. 12. 9/13/13 12 Example: Illustrations of CJA, MR, and CMS CJA-User ¢  How do you know that this is your debit piece and this is your credit piece? You wouldn't know that until you walked through it all and then you understood conceptually. (need for commitment to learn) ¢  So, that's the one drawback I would say versus some other systems that's not very intuitive. So, you need to know the screens. And, you need to be trained in what the process is. (commitment to get training) ¢  No. No. I think it's just (up to) you to understand what you're trying to do and say 'Oh, yeah. Your (journal actions are going to have to be) balanced, of course. (commitment to certain subtasks)
  13. 13. 9/13/13 13 CJA-User – Survey Items ¢  I try to stay aware of the progress of my tasks ¢  In general, I know how to complete tasks in <Enterprise System> ¢  I know how to perform my tasks with <Enterprise System> ¢  I know the business goals I am trying to achieve using <Enterprise System> CJA-System ¢  And so, you can drill in here and see why you short- paid someone, or why you paid the amount you did. It makes sense. It's like a story when you go through them. (+) ¢  I think that the system is actually very helpful. This little help button right here, which will launch into either the definitions for where you are, if you're on a particular field, but then it will also launch into help with the overall applications, so I actually think it's very helpful. (+)
  14. 14. 9/13/13 14 CJA-System ¢  User: Depending on what I'm going to try to do here. I forget if I'm in account payable or accounts receivable here. Interviewer: How can you tell if you need to know? User: I actually don't know. I'd probably just go back and then I'd look at the menu path that I was in. So here I'm in accounts receivable. (-) ¢  When you first see this layout stuff, it doesn't make sense. But then once you know what it is it does make sense. (-) ¢  Honestly, for me personally, every time I've gone and tried to drill in through the folders, I can never get to where I think I need to be. (-) CJA-System – Survey Items ¢  <Enterprise System> always informs me of the progress of my tasks ¢  In general, I feel that <Enterprise System> knows how the tasks should be completed ¢  <Enterprise System> knows how business tasks should be completed
  15. 15. 9/13/13 15 CJA-Total – Survey Items ¢  <Enterprise System> and I have a shared understanding about how to complete business tasks while we work on them together ¢  <Enterprise System> and I share goals for tasks that need to be completed when we work together ¢  <Enterprise System> and I share understanding about the progress of our tasks while we work on them together MR-User ¢  If you're used to doing an upload of the same thing, you might have to manually type that in, if the upload doesn't work. You always have to adapt to the system. ¢  Interviewer: Are there any examples of situations where the system forces you to do things in a certain way that is different from your preferred way of doing it, accomplishing a certain goal? User: Geez, I don’t know. I think after all these years I think the system has just beaten me down.
  16. 16. 9/13/13 16 MR-User – Survey Items ¢  I have learned how to use <Enterprise System> in order to adapt to its way of working ¢  I take into consideration how <Enterprise System> does things ¢  I understand how <Enterprise System> typically gets things done and adjust my work accordingly MR-System ¢  The system just doesn't know what the problem is. It doesn't know how to diagnose those. So this one says, no suitable documents found. Message [XXXXX]. (-) ¢  I would say just getting rid of some of the fields that you don’t need to look at [for this task] instead of having to go through them all. It’s just a waste of time. (-)
  17. 17. 9/13/13 17 MR-Systems – Survey Items ¢  <Enterprise System> is able to learn from how I work with it ¢  <Enterprise System> adjusts to the way I perform tasks ¢  <Enterprise System> takes into consideration my previous actions CMS-System ¢  Yes, it can give you error signs, saying something is not right, or this field is incorrect, or this has already been inputted. (+) ¢  No, it doesn't give you like error messages that say hey, you need to do x, y, z. (-) ¢  The thing is that in this case [of getting lost], the system is not reaching out to you saying that you obviously need help. It’s me having to go find it. (-)
  18. 18. 9/13/13 18 CMS-System – Survey Items ¢  In case of an error, <Enterprise System> provides mechanisms for fixing the problem ¢  <Enterprise System>'s error messages provide detailed guidance and instructions to help me solve problems ¢  <Enterprise System>'s error messages use terms that I can understand CMS-User – Survey Items ¢  I recognize the error situations in which <Enterprise System> needs additional input from me ¢  I do my best to understand <Enterprise System>'s error messages in order to solve the problem ¢  I do my best to diagnose and solve problems in error situations
  19. 19. 9/13/13 19 Example: Proposed Design Principles (Babaian, Lucas, Xu, and Topi, DESRIST 2010) Design Principle 1 ¢  DP1. The user interface should provide a mechanism for customizing the vocabulary of terms used by the system in its communication to the user, the composition of business transactions, and the content of the system's informational output to match the practices of the organization. There should be a mechanism for incorporating the customizations from an earlier version of the system to a later one.
  20. 20. 9/13/13 20 Design Principle 2 ¢  DP2. The system should provide navigational and progress guidance to a user performing a transaction, indicating the broader context of each interaction in terms of the related business process components and specifying the completed and remaining parts. A sufficiently competent user should be able to turn off this guidance if it becomes a distraction. Design Principle 3 ¢  DP3. When the system detects a problem, it should identify the possible causes and ways of resolving it. If the fix is obvious, the system should inform the user and perform it. If it isn't obvious, the possible causes and resolution scenarios should be presented to the user and be readily executable. If the system is unable to identify resolution strategies, it should present the user with the relevant data and transactions.
  21. 21. 9/13/13 21 Design Principle 4 ¢  DP4. In presenting selection choices, the system should utilize what it knows about the user, the organization, the task, and the context, and provide faster access to the more likely choices than the less likely ones. Where the choice of data or action is obvious, the system should have an option of not waiting for the user to enact it. The user should have an option to replace/cancel the system's provided choice of data/action. Example: Moving towards a Collaborative Walkthrough / Collaborative Critique
  22. 22. 9/13/13 22 Collaborative Critique ¢  A new usability walkthrough method ¢  Questions that the evaluators answer were formulated from the human- computer collaboration paradigm ¢  The first version of the method has been developed and gone through the first round of testing ¢  CHI’12 Extended Abstract Interface Evaluation: Cognitive Walkthrough* ¢  Will the user try to achieve the right effect? ¢  Will the user notice that the correct action is available? ¢  Will the user associate the correct action with the effect trying to be achieved? ¢  If the correct action is performed, will the user see that progress is being made toward solution of the task? *[Wharton et al, 1994]
  23. 23. 9/13/13 23 Viewing Cognitive Walkthrough Through a Collaborative Lens ¢  Based on the user’s overall goal, will the system recognize the next step in the process and either act to perform that step or, if the user’s input is necessary, present a set of alternative actions from which the user may make a selection? ¢  Does the system help the user identify the next action and present it in a highly visible manner? ¢  Does the system present a meaningful set of alternative actions based on the user’s overall goal? ¢  Will the system keep the user informed about the consequences of actions taken by either the user or the system, as they relate to progress made toward the achievement of the task? Key Questions ¢  Does the allocation of responsibilities between the system and its user within the task align with their respective natural strengths? ¢  Is the system-user communication mechanism effective for both novice and experienced users? ¢  Does the system make effective use of its resources in the process of working on the task and communicating with the user? ¢  What are the system capabilities and information that could be added to improve interaction?
  24. 24. 9/13/13 24 Example: Use of Process Graphs to Improve System- User Communication (Babaian, Lucas, and Topi, 2007) Typical ERP task interface
  25. 25. 9/13/13 25 49 Improving System-User Communication Via Process Graphs Add Material Edit Material Add Vendor Process Graphs Form Hierarchies
  26. 26. 9/13/13 26 Benefits ¢  Improves system-user communication of system-prescribed processes and progress through task ¢  Simplifies navigation to related tasks ¢  Enables preview of future steps and revisiting of previously completed steps Example: Improving System- User Communication based on Usage Logs (Babaian and Lucas, 2011)
  27. 27. 9/13/13 27 Improving System-User Communication via Usage Logs ¢  Embed model capturing relationships between users, actions, tasks, and processes into the system so that it can reason about itself and its users l  Capture usage data via input-aware components à supporting users with basic system usage and in error situations l  Ongoing usability evaluations l  Automated planning for diagnosing and resolving errors Usage Logs + Task-Interface Model ¢  Enables l  playback of past interactions l  usage log mining l  estimation of user’s experience level ¢  Makes relevant usage histories available to all users: l  Show how Jack posted a goods receipt. ¢  Enables automated usage-based support mechanisms l  Show all account numbers used by inspectors l  Which fields in the GR are always filled? Business Processes, Tasks Interface components Usage log Users ERP database
  28. 28. 9/13/13 28 Usage Log Examples ¢  Identify required fields based on prior usage ¢  Reconstruct completed processes l  Remind user how process was completed before via a “playable” reconstruction process l  Show user how others completed the same process ¢  Adapt level of instructional/error support based on user experience Data Model Represents and links: o  Domain data o  User data o  Task data o  UI data o  User input log
  29. 29. 9/13/13 29 Automated Usability Evaluation ¢  Keystroke analysis l  Components that are prone to user entry errors ¢  Time to complete a task l  Subtasks that are more difficult to complete ¢  Time between tasks l  Navigation difficulties ¢  Number of times a task is revisited l  Tasks requiring additional support Concluding Observations ¢  Underlying theoretical framework as a strong guide for a research program that approaches the same underlying research question from multiple perspectives ¢  A planned multi-method approach will build a much stronger evidentiary foundation than any single method would ¢  The empirical evidence gathered so far provides support for our key propositions, but getting to a point where we can conclusively reject alternative hypotheses will still take time ¢  Design principles and guidance for design of future enterprise systems will be in the center of the research program outcomes ¢  The benefits of a multidisciplinary research team are significant
  30. 30. 9/13/13 30 Work that Continues in the Project ¢  More in-depth analysis of the interview data ¢  Further formalization of collaboration theory applied to the ERP usability context ¢  Finalizing the survey instrument to enable positivist theory testing ¢  Further development of collaborative critique for evaluating interface usability ¢  Continuation of lab studies with experimental collaborative interfaces ¢  Continued development and testing of the prototypes ¢  Further articulation of the design principles Acknowledgements ¢  This material is based in part upon work supported by the National Science Foundation under Grant No. 0819333. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

×