Defining and Evaluating the Usability of CMS - Saurabh Kudesia

Uploaded on


More in: Design , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Defining and Evaluating Usability Criteria forContent Management SystemsSaurabh KudesiaCustomer Documentation ManagerNokia Siemens Networks, BangaloreAugust 20111 © Nokia Siemens Networks 2011 Presented at STC India Usability Conference 2011, Bangalore, India
  • 2. About the Speaker• Customer Documentation Manager, Nokia Siemens Networks, Bangalore• Co-founder, former editor-in-chief, KnowGenesis International Journal for Technical Communication (IJTC)• Former associate editor of Directives, a newsletter published by the (STC) Management SIG• Reviewer: ISO/IEC 25612 standards for International Standards Organization (ISO)• Bachelor of Electronics and a Certified Scrum Master (CSM)• Alumnus of Symbiosis Institute (Pune) and IIM-B• Part of Author-it DevHub Community2 © Nokia Siemens Networks 2011
  • 3. In this Session Generic Content Management Usability Usability Testing Framework (CMF) Measurement Tactics for CMS and Usability Methodology Perspectives3 © Nokia Siemens Networks 2011
  • 4. Content Management is Universal Micro Macro The information viewpoint applies at different levels4 © Nokia Siemens Networks 2011
  • 5. Content Management Perspectives Distributes Business Value Business Perspective Balances Organizational Forces Organizational Perspective Content Combines Content-related disciplines Content Perspective Collects/Manages/Publishes/Distributes Information Process Perspective Technical Infrastructure Technical Perspective5 © Nokia Siemens Networks 2011
  • 6. Content Management Framework (CMF) Content/Process View Collection System Management System Publication System Print Syndication Electronic Web Impacts publication Impacts component Impacts access structure designs designs designs6 © Nokia Siemens Networks 2011
  • 7. Content Management Framework (CMF) Technical View Access Interface Workflow Auditing Formats/Standards Evaluation Analysis Quality Format Conversion Assurance Audit Data Access Attributes Version Tracking R&D Relational Links Implementation Design Usage/Performance Statistics Auditors Tools Development Issue Categorization/Tracking Automatic AdvancedSupport Teams Notifications Search Security Customers Feedback/Issue Reporting Configuration and Administration7 © Nokia Siemens Networks 2011
  • 8. Usability Mea Culpa (or “That’s Not Architectural”) “Making a system’s user interface clear and easy to use is primarily a matter of getting the details of a user’s interaction correct…but these details are not architectural” - Bass, Clements, and Kazman [Software Architecture in Practice, first edition]8 © Nokia Siemens Networks 2011
  • 9. The ‘Planes’ of User Experience Surface brings everything together visually: What will the finished Surface product look like? Skeleton makes structure concrete: What components will enable Skeleton people to use this site? Structure gives shape to scope: How will the pieces of the system fit Structure together and behave? Scope transforms strategy into requirements: What features will the Scope system need to include? Strategy is where it all begins: What do we want to get our of the Strategy system? What do our users want?9 © Nokia Siemens Networks 2011
  • 10. The ‘Planes’ of User Experience Discrete ‘Planes’ of User Experience Concurrent ‘Planes’ of User Experience Requiring work on each plane to finish before work on the next can start A better approach is to have work on each plane finish before work on the leads to unsatisfactory results for the designers and the users. next can be finish. EffortEffort Strategy Scope Structure Skeleton Surface Strategy Scope Structure Skeleton Surface Time Time10 © Nokia Siemens Networks 2011
  • 11. Usability Tactics Runtime Tactics Design time TacticsSeparate User Support User Support System Localize changes to User Interface Interface Initiative Initiative code Cancel User Model Undo System Model Aggregate Task Model Runtime Tactics Design time Tactics • Maintain a model of the task • Localize changes to user interface code • Maintain a model of the system • Use supported software architecture patterns like Model-View-Controller • Maintain a model of the user (MVC), Presentation-Abstraction-Control (PAC)11 © Nokia Siemens Networks 2011
  • 12. Quality Attributes Stimuli Availability Modifiability Performance Security Unexpected Event Functionality Periodic Display Services Non occurrence of expected event Platform (Portability) Stochastic Modify Services Quality attribute Sporadic Modify Information Capacity (Scalability) Modify Access Reduce availability to system services12 © Nokia Siemens Networks 2011
  • 13. Usability General Scenarios Stimulus Response Artifact Source Environment Response MeasureSource Stimulus Artifact Response Response MeasureEnd Users Learn system features System Support “learn system features” Task Time Use system efficiently Support “use system efficiently Number of errors Minimize error impact Minimize error impact Number of problems solved Adapt system Environment Adapt system User satisfaction Feel comfortable Feel comfortable Gain of user knowledge Run time Proportion of Successful operation Configure time Amount of time/data lost13 © Nokia Siemens Networks 2011
  • 14. Usability Measurement MethodologyIdentification of Measurement and Result analysis andDimensional scale, Selecting the Validation of Scale estimation ofSub constructs and Measurement model based on usabilityitem pool measurement model14 © Nokia Siemens Networks 2011
  • 15. Usability Measurement is Fuzzy15 © Nokia Siemens Networks 2011
  • 16. Dimensional Scales, Sub Constructs and item pool Simplicity Familiarity Consistency Visual Feedback Responsiveness Fault Tolerance Scalability 1. Visual Efficiency and Consistency 3. Interface-Task Performance Consistent Interface Design labeling Simple structure to accomplish a task Consistent Interface Response UI screens don’t include too many elements Screen design inline with visual design principles Understandable Interface language 4. Interface Familiarity Interface resemble user’s prior experience 6. Efficient/Flexible Access to Content Interface behaves similar to what user is accustomed to Easy interface design for accessing content No specialized training required to perform functions Shortcuts to access frequently used features 5. Interface Operation Performance Multiple options to perform the same action CMS loads fast Prompt CMS responses to users’ prompts CMS has a flexible use Interface design changes are localized Less user interaction required to complete a task 7. Efficiency of Navigation 2. Error Handling and Functional Efficiency Easy access to critical navigational buttons Easy to understand error messages Navigation remains unaffected with increase in content volume Explicit error messages No problems with CMS functions16 © Nokia Siemens Networks 2011
  • 17. Measurement Models Model 1: Strictly Uni- Model 2: Group-Factor Model 3: Essentially Model 4: Multidimensional dimensional Model Unidimensional Structural Model E1 X1 E2 X2 E3 X3 C : : Ek Xk17 © Nokia Siemens Networks 2011
  • 18. Fit Indices of ModelsCovariance Matrix Sub 1 Sub 2 Sub 3 Sub 4 Sub 5 Sub 6 Sub 7 0.57 Sub 1Sub 1 1.00 0.66Sub 2 0.59 1.00 0.17 Sub 2 0.91Sub 3 0.01 -0.01 1.00 0.57 1.00Sub 4 0.37 0.52 -0.01 1.00 0.68 Sub 4 Upper 1Sub 5 0.01 -0.01 0.64 -0.01 1.00 0.93Sub 6 0.61 0.84 -0.01 0.53 -0.01 1.00 0.14 Sub 6 0.66Sub 7 0.43 0.60 -0.01 0.38 -0.01 0.62 1.00 -0.02Upper 1 (ζ) 0.66 0.91 -0.02 0.57 -0.01 0.93 0.66 0.56 Sub 7 Goodness-of-fitness Lack-of-fitnessUpper 2 (ζ) -0.01 -0.02 0.86 -0.01 0.74 -0.02 -0.01Models NNFI CFI RMSEA SRMR 0.25 0.86Model 1 0.85 0.86 0.134 0.121 Sub 3Model 2 0.92 0.93 0.075 0.073 Upper 2 0.74 1.00Model 3 0.92 0.92 0.078 0.075 Sub 5 0.46Model 4 0.92 0.93 0.075 0.062 18 © Nokia Siemens Networks 2011
  • 19. Important Guidelines• Don’t mix content presentation and architectural design; both are perceived as separate constructs by participants• Usability scale is multidimensional and should be treated accordingly in benchmarking usability of CMS• Consider context dependencies of User’s perceptions and functional use of CMS into usability calculation• Generalizability and transferability of ideas across contexts, extensive collaboration is essential for the multidimensionality of usability scales19 © Nokia Siemens Networks 2011
  • 20. For further discussion: Phone: +91-9620208216 Email: saurabhkudesia@gmail.com20 © Nokia Siemens Networks 2011