TENCompetence Workshop, Manchester 2007 Designing Systems for Managing Collaborative Research Processes Scott Wilson, CETIS; Yoichi Takayama, Ernie Ghiglione, James Dalziel, MELCOE This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence

Context Two linked projects: RAMS & DRAMA Research by MELCOE Funded by DEST as part of programme backing Australia’s research capability

Two linked projects: RAMS & DRAMA

Research by MELCOE

Funded by DEST as part of programme backing Australia’s research capability

Why collaborative research workflow? Plenty of work to date in process automation, especially data analysis Relatively little work on communication and collaboration-heavy processes Emphasis in funding on collaborative work (e.g. multi-partner) but patchy practice Possibility of scaffolding collaborative processes to improve effectiveness

Plenty of work to date in process automation, especially data analysis

Relatively little work on communication and collaboration-heavy processes

Emphasis in funding on collaborative work (e.g. multi-partner) but patchy practice

Possibility of scaffolding collaborative processes to improve effectiveness

Which kinds of processes? Research management Research quality assessment Online research collaboration Data collection Monitoring, positioning new opportunities Proposal development Scheduling

Research management

Research quality assessment

Online research collaboration

Data collection

Monitoring, positioning new opportunities

Proposal development

Scheduling

Research process characteristics May be weakly structured While some processes may be pre-determined, the majority exhibit dynamic properties, and may diverge from design at runtime Alternate periods of individual and team activity

May be weakly structured

While some processes may be pre-determined, the majority exhibit dynamic properties, and may diverge from design at runtime

Alternate periods of individual and team activity

Potential benefits Greater standardisation of common or repeatable research processes, leading to higher quality outcomes and improved efficiency; The ability to share descriptions of common research processes both within institutions, and between institutions – including the ability to adapt and localise shared research processes; Greatly improved accountability and audit for processes involving multiple actors across multiple steps – such as for research assessment (e.g., RQF/RAE assessor workflows), as well as for research itself (e.g., as a deterrent to academic fraud); and Providing a process-oriented checklist to ensure the ordered completion of relevant research tasks.

Greater standardisation of common or repeatable research processes, leading to higher quality outcomes and improved efficiency;

The ability to share descriptions of common research processes both within institutions, and between institutions – including the ability to adapt and localise shared research processes;

Greatly improved accountability and audit for processes involving multiple actors across multiple steps – such as for research assessment (e.g., RQF/RAE assessor workflows), as well as for research itself (e.g., as a deterrent to academic fraud); and

Providing a process-oriented checklist to ensure the ordered completion of relevant research tasks.

Potential risks The technology may be costly when compared to the actual benefits The technology may require new competences to devise and execute research processes, placing control of such processes into the hands of IT specialists rather than researchers themselves The technology may restrict unnecessarily the diversity of activities and approaches to conducting research The technology may be perceived as an attempt to control rather than enable the work of researchers

The technology may be costly when compared to the actual benefits

The technology may require new competences to devise and execute research processes, placing control of such processes into the hands of IT specialists rather than researchers themselves

The technology may restrict unnecessarily the diversity of activities and approaches to conducting research

The technology may be perceived as an attempt to control rather than enable the work of researchers

Six Lenses (so far…) BPM HIM BPOKM CSC Learning Design VSM

BPM

HIM

BPOKM

CSC

Learning Design

VSM

Simple workflow…

… can exhibit complex behaviour

Is it worth the trouble? “ One might argue that in a world where routine processes have been automated by systems, these value-adding, creative, innovative human activities are what business is all about. An organisation in which people interacted only via scripts loaded into machines could not think, could not respond, could not change and could not possibly provide effective support for its customers. And if human-driven processes are indeed critical, they deserve effective computer support.” (Harrison-Broninski (2004)

“ One might argue that in a world where routine processes have been automated by systems, these value-adding, creative, innovative human activities are what business is all about. An organisation in which people interacted only via scripts loaded into machines could not think, could not respond, could not change and could not possibly provide effective support for its customers. And if human-driven processes are indeed critical, they deserve effective computer support.” (Harrison-Broninski (2004)

Cybernetic Analysis using the VSM Attempt to identify design principles using structured approach the VSM by Stafford Beer Previously applied in pedagogic analysis of elearning systems (Liber & Britain, 1999 & 2004)

Attempt to identify design principles using structured approach

the VSM by Stafford Beer

Previously applied in pedagogic analysis of elearning systems (Liber & Britain, 1999 & 2004)

Simplified VSM (Liber & Britain, 2004)

Topics of analysis Distribution of work Monitoring Audit Self-organisation Reverberation Strategy

Distribution of work

Monitoring

Audit

Self-organisation

Reverberation

Strategy

Distribution of work How do users end up in process roles? How are resources managed? Useful perspectives: Winograd and Flores (1986): CFA Russell et al (2004): workflow resource patterns

How do users end up in process roles?

How are resources managed?

Useful perspectives:

Winograd and Flores (1986): CFA

Russell et al (2004): workflow resource patterns

Simplistic concepts of work distribution abound in IT systems…

But simplicity can be deceptive! In some cases it just pushes complexity to the outside

Other considerations in work distribution Capability management Role-capability mapping and user-capability assessment

Capability management

Role-capability mapping and user-capability assessment

Monitoring (system 3) Monitoring for variety management Monitoring and self-regulation Challenge: identifying process states in dynamic, diverging processes Solutions: minimal representations (van der Alst, 2001); analog roll-up; declaration Issues: assumptions of control Requirement for openness in monitoring

Monitoring for variety management

Monitoring and self-regulation

Challenge: identifying process states in dynamic, diverging processes

Solutions: minimal representations (van der Alst, 2001); analog roll-up; declaration

Issues: assumptions of control

Requirement for openness in monitoring

Audit (System 3*) Process auditing Rewind and reflect Sampling Intervention

Process auditing

Rewind and reflect

Sampling

Intervention

Self-organisation Self-organization can dramatically reduce the management demands of a system Delegation of process design Delegation of process initiation Requirement for workflow tools to be usable by process participants, not specialists

Self-organization can dramatically reduce the management demands of a system

Delegation of process design

Delegation of process initiation

Requirement for workflow tools to be usable by process participants, not specialists

Reverberation Avoiding ‘process blindness’ or blinkered viewpoint leading to clashing activities Cross-cutting communication; openness of processes across related groups

Avoiding ‘process blindness’ or blinkered viewpoint leading to clashing activities

Cross-cutting communication; openness of processes across related groups

Unforeseen events In work distribution, requiring reallocation, escalation, delegation… In process development, requiring redesign, termination, suspension… Monitoring and intervention Algedonic channel

In work distribution, requiring reallocation, escalation, delegation…

In process development, requiring redesign, termination, suspension…

Monitoring and intervention

Algedonic channel

Strategy (System 4) Forecasting Monitoring trends in the wider world Implementing strategy

Forecasting

Monitoring trends in the wider world

Implementing strategy

Recursions and perspectives Work Item Work Package Project Team Unit Department University/Company Consortium

Work Item

Work Package

Project

Team

Unit

Department

University/Company

Consortium

From LAMS to RAMS Flexible role definitions supporting dynamic processes and shared ownership Choice of work distribution models, not just “simple assignment” Processes able to be redesigned while running by the participants Monitoring interfaces available for all participants, not just “teacher” role Analog and asserted digital indicators for monitoring Dashboard for multiple-level monitoring Tools to interact with more structured processes (e.g. bid/RQF submission, grid data processes)

Flexible role definitions supporting dynamic processes and shared ownership

Choice of work distribution models, not just “simple assignment”

Processes able to be redesigned while running by the participants

Monitoring interfaces available for all participants, not just “teacher” role

Analog and asserted digital indicators for monitoring

Dashboard for multiple-level monitoring

Tools to interact with more structured processes (e.g. bid/RQF submission, grid data processes)

Modified WfMC Reference Model

Observations Collaborative processes do not lend themselves to todays BPM methods Requirements for eResearch turned up using VSM-driven approach are also useful in eLearning

Collaborative processes do not lend themselves to todays BPM methods

Requirements for eResearch turned up using VSM-driven approach are also useful in eLearning

Future work Aligning process and policy (and the gray area inbetween) Capability management Validation of the approach

Aligning process and policy (and the gray area inbetween)

Capability management

Validation of the approach

Cheers! [email_address] http://www.cetis.ac.uk/members/scott

[email_address]

http://www.cetis.ac.uk/members/scott