CORE: Cognitive Organization for Requirements Elicitation

How to get out of a mess: Use the CORE methodology Cognitive Organization for Requirements Elicitation Scott M. Confer Information Architect Author contact: joanna.wiebe@orbitz.com 312.260.8274 Joanna Wiebe Information Architect

CORE overview

Integrates two analytical methodologies:

Conceptual graphing (Gordon, Gill, Schmeier, 1993)

Collaborative soft systems inquiry framework (Checkland, 1981).

Conceptual Graphing Soft Systems C.O.R.E. Method

What is the CORE Method?

CORE

An emerging methodology and conceptual toolkit at Orbitz

Allows the IA to lead the team to

Clarify

Elicit

Iterate the initial vague requirements.

Advantages of the CORE method

Centers on the user, drawn as a key actor in Rich Picture.

Cross-disciplinary teams can work collaboratively to define quality requirements.

Other advantages include:

Scalable for large or small projects

Is domain-agnostic

Rules-based formal grammar for cognitive graphing ensures a thorough approach

Based on empirically validated methods

Ecologically valid combination of two methods widely used in industry and business today

Above all, CORE is a driver for innovation

CORE has seven steps

Step 1: Unstructured situation

Step 2: Rich Picture

Step 3: Relevant systems

Step 4: Conceptual Graph Structures (CGS)

Step 5: Preliminary requirements

Step 6: Requirements consensus

Step 7: Translation to Information Architecture

The scenario

Final details of the contract with the third party had not yet been resolved, one reason that the requirements were not complete at kickoff …

Sound familiar?

So, you find yourself needing better requirements?

What to do next?

The IA didn’t have a formal organizational method for requirements development.

The ‘flat’ organizational structure at Orbitz demanded team consensus for the project to move forward

What to do?

Add structure!

Step 1: The Unstructured Situation Also known as the “Mess TM ” in soft systems In an unstructured situation, “…human perceptions, behavior or actions seemed to be the dominating factors…goals, objectives and even interpretation of events are all problematic.” (Naughton, 2005)

Some attributes of a requirements “mess”

Undefined or implicit features

Featuritis

Speed to market is the driver

“ We don’t need no requirements!”

“ Requirements” are:

Voluminous (i.e. have Featuritis)

Solutions-focused

Vague

Disorganized

Missing

Conflicting

Interdependent

Not prioritized (or all=HIGH)

Too little, incomplete, or missing:

User research

Functional requirements

Use cases

Magically coordinate with other projects / dependencies

Some attributes of requirements with “quality”

Correct

Unambiguous

Complete

Consistent

Prioritized for importance and stability

Verifiable

Modifiable

Traceable

Understandable

Reference: Leffingwell, D. and Widrig D. (2000). Managing Software Requirements - a Unified Approach

To structure an “unstructured situation” . . .

Identify key roles

investigator

subject matter expert

client

problem-owner

problem-solver

(one person may wear many hats)

Form your team

Do the research

What is this thing?

What’s happening?

What to do?

Where is it?

Where can I find out more?

(more question probes, next slide)

Question probes for primary research

Questions from: Gordon et al., & San Diego State University http://coe.sdsu.edu/EDTEC544/Images/probes.gif

Step 2: Create a “Rich Picture”

Case study example: Rich Picture

Drawing the picture is a way to methodically walk through some or all of key aspects of the project.

A Rich Picture is Checkland’s term for a visual capture of:

Structure

Function

Process

Environment. The IA worked from her raw interview notes, whiteboard sketches and assumptions, to create the Rich Picture, using Visio.

Rich Picture elements

Key actors

Structure of actor interactions

New Features

Integration with existing ones

Process flow

Environmental factors

“ Hard” or “soft” information

Hard information: verifiable data and knowledge

Soft information: feelings, perceptions, opinions, values—often key to project success or failure

Types of requirements:

functional, data, content, creative, data safety, testing performance, user interface, localization, technical, financial, temporal, managerial

Tasks involved in understanding each requirement type

1 2 4 6 6 3 8 7 5

Key Actors 1

Structure of actor interactions 2

New Features 3

Process flow 4

Environmental factors 5

“ Hard” or “soft” information 6 6

Types of requirements 7

Tasks to understand the requirements 8

Step 3: Write root definitions of relevant systems

How X does Y to accomplish Z.

The definition for each relevant system describes the Rich Picture elements of structure, function, process and environment.

Case study example: Relevant Systems

System A (Customer perspective): A system to enable customers to book and manage travel online at the same time that they register for meetings without having to log into another application

System B (Business perspective): A system to provide incremental transactional revenue and revenue share offerings for Orbitz and third parties

System C (Development perspective) A system to add features to the online travel booking web application.

Step 4: Create conceptual graphs CGS method is broadly applicable to human endeavors The grammar is based on the cognitive psychology and story comprehension research by Arthur Graesser.

Conceptual graphing phase

1. Primary and secondary goals and goal-actions are broken out from the root definition of each relevant system.

2. Relevant system(s) are then graphed.

Conceptual Graph Structures (CGS)

Key elements define the grammar

Six types of nodes graphically represent one piece of knowledge.

Eighteen arc types connect the nodes, indicating the relationship between the nodes.

Templates for CGS structure contain “legal” arc-node combinations.

There are four types of knowledge substructures for a conceptual graph :

Goal hierarchy

Taxonomy

Causal network

Spatial relationship

Process flows vs. conceptual graph with a grammar

Unlike process flows, conceptual graphs can be non-linear yet show procedures at the same time .

You can see concepts, goals, and procedures. Spatial relations are possible for actual interface layout or physical products.

Could be called “Concept Goal Procedure” networks since those sub-structures are depicted.

Examples of concept mapping found online… 1 2 3 theyrule.net Network Diagrams of Conspiracy Mark Lombardi Internet Search Dubberly Design Office 4 The Budget Graph Jesse Bachman … enable discovery but don’t necessarily show user goals or use formal grammar

Four types of CGS substructures

Substructures are created from templates

The template provides basic form of substructure

Cohesive and predictable unit with prototypical node-arc-node triplets

Types of CGS templates:

Goal hierarchies

Taxonomic substructures

Causal network substructures

Spatial relations

Goal hierarchy example

Substructure 1. Goal hierarchy example Goal Hierarchies are our ‘bread and butter’ for system design.

Procedure for creating a graph

Identify the document to be graphed

Read for content and context

Note types of knowledge (taxonomic, goal, spatial, causal)

Finish reading. Go back and start the studying and creation.

Study each sentence and break into phrases

Identify nodes and links

Create nodes and links using judgment to cover these cases:

Easy: Both nodes and arc are in the phrase or sentence

Linkage may be clear in your mind, but more nodes/arcs required on the graph to represent it than in the doc

Hard: Ambiguous situation

As each sentence is read, associate a substructure to it that will contain node-arc-node triplets

By using the template you will see gaps in the structure to be filled.

Each entry in the graph is attached to something else. Islands show what is unanswered, unrelated or out of scope.

CGS Construction Kit with Visio stencils

CGS creation is supported by the custom Visio stencil

A notable feature is that when a ‘Smartarc’ is attached to two nodes, the legal arc types are suggested from a contextual menu on the arc (right click).

Question probes for conducting your own SME interviews

Download visio tools at onemind.wetpaint.com

Key question probes for primary research Start with goals, add in supporting concepts. For innovation, focus on manual and machine-aided tasks that are difficult, time intensive, repetitive, and pet peeves.

Step 5 Iterate graphs/develop preliminary requirements The process of analysis that is encouraged to find and develop new solutions serves as a scaffolding for the “a-ha” moment.

Case study example: Preliminary requirements

CGS process led to cognitive breakthroughs, discoveries and innovations to meet user goals.

The CGSs were iterated between team meetings

CGSs capture, clarify, and relate concepts as well as goals

Case study example: Preliminary requirements

CGS process led to cognitive breakthroughs, discoveries and innovations to meet user goals.

The CGSs were iterated between team meetings

CGSs capture, clarify, and relate concepts as well as goals

Requirements drafted

Analysis and synthesis from concept graphing

Served dual purposes for analysis of both macro and micro perspectives from the CGS itself.

Highlighted inconsistencies and/or problems with the existing situations/systems.

Issues were managed in an Issues Log.

New requirement issues logged; brought back to team for clarification.

Implicit relationships were revealed

Visualize what was known and

Missing items in the CGS via the templates

(e.g. goals, taxonomies, and causal networks.)

Preliminary requirements drafted and walked through with individual team members

After iteration cycles, the IA was able to make a list of preliminary requirements.

Case study example: Preliminary requirements Thirty-five pages of a “requirements doc” were transformed into succinct, prioritized requirements

Step 6 Reach team agreement on requirements

By Step 6 in the CORE methodology, a project team usually can efficiently reach requirements consensus.

The requirements now have

Total team buy-in

Documented

Stored with version control

Shared

Requirements are developed iteratively, published and are accessible to all project participants.

Online wiki (open source)

MS Word documents

At this point stakeholders are invested in the process and it will not be too difficult to reach agreement on the final requirements.

Step 7 Translate to Information Architecture

Implementing IA changes software and environment

Whether it is a new or enhanced system, the changes are seen in:

Software

Structure

Function

Process

Environment

Business intelligence

Third-party agreements

Partnerships

Contracts

Work processes

Attitude and policy change consequences

The result is a set of changes of the original situation as collaboratively defined

Conclusion

CORE has broad utility for all types of design contexts:

physical products

services

training program design

software development

CORE method helps designers define:

what information and inputs are needed,

at what time they are needed,

in order for users to make decisions .

Lather, rinse, repeat

And now you have a new “mess” …

To do a new project, this current situation is to be cognitively structured anew, these cognitive structures have to be agreed on, and crystallized into the form of new requirements

References

Atlantic Systems Guild Limited (2006). Volere Method Requirements Specification Template, Edition 11. Available at http://www.systemsguild.com and http://www.volere.co.uk.

Beyer, H. & Holtzblatt, K. (1998). Contextual Design: A Customer-Centered Approach to Systems Designs . Morgan Kaufmann.

Carroll J.M., Rosson M.B., Chin G., Koenemann J. (1997). Requirements Development: Stages of opportunity for collaboration needs discovery. Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques. ACM Press. New York, NY, USA.

--> 5 stages of Requirements Development instead of gathering (Genesis & Types) on p.63

Checkland, P B (1972). Towards a Systems-based methodology for real-world problem-solving. Journal of Systems Engineering, vol. 3, no. 2.

Checkland, P.B. (1981). Systems Thinking, Systems Practice . John Wiley & Sons.

Constantine, L. & Lockwood, L. (2000). Usage-Centered Design: Practical Abstract Modeling with Use Cases. ACM CHI 2000 Tutorial.

DuFour, R., & Eaker, R. (1998). Professional Learning Communities at Work: Best Practices for Enhancing Student Achievement. National Educational Service.

Gordon, S. E. & Gill, R. T. (1997). Cognitive Task Analysis. In C. Zsambok & G. Klein, (Eds.), Naturalistic Decision Making (pp. 131-140). Hillsdale, NJ: Lawrence Erlbaum.

Gordon, S.E., Schmierer, K.A., & Gill, R. T. (1993). Conceptual graph analysis: Knowledge acquisition for instructional system design. Human Factors, 35, 459-481.

Graesser, A.C., Golding, J.M., & Long, D.L. (1991). Narrative representation and comprehension. In R. Barr, M.L. Kamil, P. Mosenthal, and P.D. Pearson (Eds.), Handbook of Reading Research (pp.191-205). White Plains, NY: Longman.

Hackos, J. and Redish, J. (1998). User and Task Analysis for Interface Design . John Wiley and Sons. Bloomington, IN: National Educational Service.

Holtzblatt, K. & Beyer, H. (1995). Requirements gathering: the human factor. Communications of the ACM. Volume 38, Issue 5. Available at http://doi.acm.org/10.1145/203356.203361.

Hutchings, F. & Knox, S. (1995). Creating Products Customers Demand. Communications of the ACM. Volume 38 , Issue 5, Pages: 72 - 80.

Leffingwell, D. & Widrig D. (2000). Managing Software Requirements - a Unified Approach. Object Technology Series (Series editors Booch, Jacobson, & Rumbaugh). Addison-Wesley.

Lengler R., Eppler M. (2007). Towards A Periodic Table of Visualization Methods for Management. IASTED Proceedings of the Conference on Graphics and Visualization in Engineering (GVE 2007), Clearwater, Florida, USA. Available at http://www.visual-iteracy.org/periodic_table/periodic_table.pdf.

Naughton, J. (2005). Soft systems analysis: An introductory guide. Milton Keynes: The Open University.

Robertson S. & Robertson J. (2006). Mastering the Requirements Process—Second Edition . Addison-Wesley. Available at http://systemsguild.com/GuildSite/Robs/RMPBookPage.html.