Chapter 9 Case study: tools for visualising                   designs        Chapter 9 Case study: tools for visualising d...
valuable insights into the design process that cluster into five overarching themes (Figure1).Figure 1: The five overarchi...
I was building a sense of what the new course might be ... we must remember to       do x, or a url of relevance [Intervie...
available and which would best fit your intention and provide you with a map -            thats when a map would work, the...
ings from this work were similar to the examples provided above. Analysis of the data re-vealed a number of themes that ar...
P7: “So probably having different layers of visualization of the same structure           could help filter the relevant i...
•only able to capture partial details in the example representations; and,•needing additional information for clarificatio...
play and edit details of a node including its text), right clicking (to display a menu offer-ing actions and operations to...
ping a learning activity in this way involves the user in a cognitive process of external-ising their understanding of the...
activities including tasks with discuss in their label. Further help is provided by theAbout.. buttons. These buttons init...
Figure 5: Help relevant to a particular toolFigure 6 represents a screen shot of part of the learning activity associated ...
Figure 6: Visual representation of part of a collaborative role play activityOur ultimate goal is to provide adaptive and ...
Feedback was also positive about our approach to helping teachers/designer consider inmore detail the general issues and u...
A media designer had the OU used CompendiumLD to draw the tools that were being      used in a course. The benefits noted ...
•It is easy to change the layout•It is useful for communicating the overall structure of a course•It is good for sharing i...
dent instructions and for planning the logistics of a course – particularly for complexcourses. Negative comments about th...
Upcoming SlideShare
Loading in …5
×

Chapter 9 case study tools for visualising designs

2,118 views
2,030 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,118
On SlideShare
0
From Embeds
0
Number of Embeds
39
Actions
Shares
0
Downloads
40
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Chapter 9 case study tools for visualising designs

  1. 1. Chapter 9 Case study: tools for visualising designs Chapter 9 Case study: tools for visualising designs...............................................1 Introduction.............................................................................................................1 Practitioners’ approaches to design.......................................................................1 Repurposing an Open Education Resource..............................................................4 The development of CompendiumLD..........................................................................7 Evaluation of the use of CompendiumLD................................................................13 Use of CompendiumLD by practitioners...............................................13 Use of CompendiumLD by students.......................................................15 Other visualisation tools .....................................................................................16 Conclusion.............................................................................................................16IntroductionThis chapter presents a case study on the development, use and evaluation of tools forrepresenting learning designs, CompendiumLD. A rationale for the development of thetool will be provided, along with a description of the tool, its functionality and use. Acomparison will be provided of related visualisation tools. The chapter will conclude witha discussion of the benefits of such tools, along with the challenges they present.Practitioners’ approaches to designThe extensive range of data collected in the OULDI described earlier in this bookprovides a rich body of empirical evidence to inform our thinking and the development ofappropriate tools for design. In summary we have conducted a series of interviews, work-shops and focus groups with practitioners to elicit their approach to design and any asso-ciated challenges. In terms of guiding and representing learning designs, we have adaptedan argumentation and visualisation tool (Compendium) to create a visualising tool fordesign, CompendiumLD. This section provides a summary of some of the key findingsfrom the empirical data, in terms of how it has informed and developed our thinking inthe development of the CompendiumLD tool, a more detailed discussion of some of thefindings from the interviews with teachers/designers in provided elsewhere Cross et al.(2008) and Clarke and Cross (2010).The empirical data provided a rich picture of the way in which teachers design. It wasevident from the data that there was no one perfect tool for design and that individualshad different preferences of how they went about the design process - some sketchingideas out and linking them, others working systematically from learning outcomes, whilstothers using the subject content as a baseline for development. Some used a combinationof approaches at different stages of the design. The interviews and case studies provided 1
  2. 2. valuable insights into the design process that cluster into five overarching themes (Figure1).Figure 1: The five overarching themes on design approaches and methodsThe most prominent finding from the interviews was that design is a messy, creative andinteractive process, and that even when working in teams there is a large element of indi-viduality in the design process. Teachers design at different levels of granularity and fo-cus on different aspects of design over the design lifecycle. Both the interviews and theworkshops gave us a clearer understanding of the design strategies that teachers adopt.Foci for design include: looking at learning outcomes and mapping these to assessmentstrategies, integrating the use of externally resources with locally authored materials,designing activities to test understanding, integrating a range of tools and approaches, ad-dressing different learner preferences and levels of competence, mapping to externallyprescribed professional requirements. The following quote is illustrative of this: Its not in one direction. Not sure if I always start with aims, sometimes I do! Broad aims, then thinking about the mix, go to the palette and look at existing re- sources, what will the budget allow us to do (chairs hat on), what additional re- sources do we need, which would be most effective to teach certain things. For example, we need this software to help teach linguistic analysis. We might want some video analysis, so think about how to bring in video sequences, what video- ing needs to be done. Then start writing. Its chicken and egg. Sometimes start with study guide and then think about activities, and then think I need this bit of video. But you dont always have luxury of working in this direction or budget to do filming so start looking for other sequences and build activities around those. [Interview 160607]The following quote gives an example of how a teacher iteratively develops their conceptof the course over a period of time and how they kept an evolving record of relevant re-sources and materials for the course. 2
  3. 3. I was building a sense of what the new course might be ... we must remember to do x, or a url of relevance [Interview 160607]It was also evident that design for a new course is very different to design when redevel-oping a course based on interpretation of student feedback and evaluation. The inter-views revealed that there was no simple route to teachers accessing support and guidanceon the design process. Little use appeared to be made of online resources and networks -most adopted a serendipitous approach, relying on peer practitioners and close colleaguesfor ideas. One interviewee from the case studies conducted by Wilson said: This says more about me than it does about the stuff really but I preferred the cor- ridor conversations. It was a way of ... I had invested quite a lot of money in cof- fee and so there were a whole set of people across the university who I took to coffee and pumped them for what I could really. [Case study Interview, 210107]Those interviewed recognised the value of sharing and reuse, but little evidence emergedof a significant amount of sharing and reuse. Different forms of representation of learningactivities (textual, visual, etc.) all had different pros and cons and there was evidently adistinction between the process of producing a design and design as an artefact. Whenshown visual presentations of learning activities for example, many of those interviewedfound it difficult to interpret them, to apply/adapt them to their own context. However, onfurther probing they could see a genuine benefit in using visual tools as a means of map-ping their own practice, as is evident in the following quote from one of the interviews. [On the value of a visual representation] It always needs to be brought to life, to have some form of enactment... Would I want to see what someone else has done, yes I suppose so. [Interview, 141107]The conflict between the process of dynamic creation of an activity and the associatedsense of ownership the designer has in the process, contrasts with design as a product, astatic artefact. For example one interviewee struggled to see the benefit of a visual repres-entation of someone else’s design, even though it was an activity in her subject area. Shecontinued later in the interview to argue for the need for a mediation role to help interpretdesigns and as she says ‘make them come alive’: [One being shown a visual representation of a learning activity] Its such a differ- ent context and level. This is language teaching rather than linguistic teaching. And there isnt the contextual information, even with you having just explained a little, which helped, without you there Id be looking at this and thinking... I think thered be too much work to look in to this plus the recontextualisation. I wouldnt spend the time to be honest. I really think you need someone who goes to the course team, although not neces- sarily staying with them. And sits down, not right at the start but a little way in, and asks what are you teaching and what resources are you going to use alone or in combination and that person would go away do some work and come back - have some insight into bringing together their knowledge of the technologies 1
  4. 4. available and which would best fit your intention and provide you with a map - thats when a map would work, theyd be bringing it alive. [Interview, 141107] The interactive and holistic nature of the design process came out strongly across the data: One of the difficulties is mapping the whole process I have tried to approach course design using a holistic approach [Interview, 121107] Teachers differed in the extent to which they worked visually or textually, although evid- ently the institutional quality audit and validation processes require textual representa- tions of some description for courses. Some used software, others sketched or wrote ideas, one teacher had a scrapbook which he used as he was developing his design ideas: Its in words, not diagrams a dumping ground for thoughts - [to] capture thoughts [Interview, 121107] Others used visualisation as a means of mapping different elements of the design process: List of words clustered into blocks, arrows...can you have clusters link to TMAs [Assignments] [Interview, 141107] Start from assessment strategies and learning outcomes and get an alignment [In- terview, 151007] I tend to sit and doodle a map - will draw the logic and flow of the course on pa- per and then go to Compendium. Then the problem is sharing it [Interview, 291107] The interviews also highlighted a number of contradictions about the process of design, forms of representation for design and the nature/type of support, which teachers wanted:•A tension between design as process and design as artefact•The difficulty of capturing what is inherently an implicit process•The demand for subject-specific case studies and examples, which are then not used and ad-apted•The variety of influences on the different forms of representation and individuals interpreta-tions of them•The desire for specific, just-in-time help and support and the difficulty of capturing supportin an online tool•How to map the evolving, dynamic and changing nature of design. Repurposing an Open Education Resource In related work, we explored teachers conceptions of design in terms of how they might repurpose stand-alone Open Educational Resources (OER) to support their used in col- laborative learning activities (Conole, McAndrew, & Dimitriadis, 2010). A series of workshops were run, in which participants explored existing OER and used a set of col- laborative Pedagogical Patterns to redesign the OER for use in a collaborative learning context. The workshops were video recorded and the discussions transcribed. The find- 2
  5. 5. ings from this work were similar to the examples provided above. Analysis of the data re-vealed a number of themes that are discussed here. Part of our approach is predicated onthe notion that OER have inherent designs and that if we can make those designs moreexplicit this will aid repurposing. A number of themes emerged with respect to this,which are discussed in this section. In the following sections participants are representedas P1, P2, etc. while the workshop facilitators are indicated as F1, F2, etc. In the follow-ing section references to CSCL patterns are indicated in italics.It was evident that there were a number of ways in which textual representations couldemphasise different aspects of the design – some were descriptive in nature, others weremore metaphorical and others still more operational – for example a bullet listarticulating steps in a learning sequence. A common approach adopted by the participantswas to have a temporal sequence. Another strategy was to focus mainly on the contentand associated resources. Participants started from different perspectives; some began byconsidering the learning objectives, whilst others started with the content or activities. P2: “My resource is a design by itself. So, it is the design of an activity, it is the representation of that, a few bullet points and then a graphical representation. …. So the resource basically represents arrows pointing into a sequence of the activit- ies.”It was interesting to see the extent to which each of the representations was easilysharable with others. More often than not a dialogic engagement was necessary to helpmake meaning of the design and to clarify misunderstandings. The exercise andsubsequent discussion enabled us to tease out both the main facets of design andparticipants’ different perspectives and approaches. In addition to articulating objectives,content and tasks, some of the participants evidenced a subtler level of design –associated with the inherent principles of the design. P3: “My resource is task-driven, so that is the principle and also it integrates many pedagogies into the content, so, and also it is question based.”In terms of principles we explored whether or not they had articulated a principle aroundindividuality/collaboration. A range of characteristics was identified as being associatedwith the design – the objectives, generic characteristics, sequence of tasks undertaken,and whether it had an individual or collaborative focus. Participants recognised that itwas important to focus in terms of clarifying what information was essential tocommunicate so that the activity could be subsequently taken up and adapted by others. F1: “Just try to think again of what elements you wrote down and what elements you used when you tried to explain it to your neighbour and try to think whether they were mainly based on objectives, mainly based on the characteristics of the activities, of a temporal sequence or …”One of the participants suggested that it would be valuable to have multiple views of thesame design each view representing a different aspect. 1
  6. 6. P7: “So probably having different layers of visualization of the same structure could help filter the relevant information if you are looking at the learning objectives, or if you are looking at interactions, something like that, so, other thing that we were thinking about it probably what is missing is a legend of the different items, because we understood that there is a mixing of 2 layers, one is devoted to the designer, for example, all the questions in blue are annotations for the designers while for example it is very clear that the sequence for students is talking to the student verbally, it is talking to him, so probably having the legends saying ok, question mark annotation for the designer and the red bits are feedbacks we had from one evaluation and then filtering visually this information according to the task you are following.” This participant also argued that visualisation potentially has additional power, if a semantic dimension is included. P7: “A semantic of visualizations, really we understood that some of the connection are more related to cognitive activities of the design where as others are tactical activities of the use (missing comment) and cause and some other connection are like database connections with the resources and what they are looking, so probably having different semantic of the connections and representations.” Another aspect of importance that participants mentioned was identifying the quality and provenance of the resource; i.e. designs need to do more that display the sequence of activities, users need some indication of how effective and fit for purpose it is. There are two ways in which this can be included. Firstly, in the design representation itself, however the more detail that is included in the design the more complex it is. Secondly, an alternative is to have a wrap-around dialogue about the resource and its design, in a tool such as Cloudworks. The data revealed that deconstruction and subsequent reconstruction of OER is complex, indeed it is possible to identify four layers that need to be considered to make most ef- fective repurposing of an OER: 1.Visual representation of the design. How can the implicit OER design be made more explicit and hence shareable? 2.Opinion of goodness. How appropriate is the OER for different contexts? 3.Transferability through Pedagogical Patterns. How can generic patterns be applied to specific contexts? 4.Layer of discussion, critique and contextualisation. How might social networking sites like Cloudworks act as a supporting structure to foster debate between those using the same OER? In conclusion describing design was seen as a difficult and unfamiliar task:•having multiple solutions;•many options for what to include;•being hard to interpret in a consistent way; 2
  7. 7. •only able to capture partial details in the example representations; and,•needing additional information for clarification. The development of CompendiumLD CompendiumLD has been developed out of our interpretation of the empirical data sum- marised in the previous section and a realisation that visualisation is under-utilised as an approach to adopting a creative approach to the design process. Brasher et al. (2008) and Conole et al. (2008) provide more detailed information on the tool and associated tech- nical development; only the salient features are described here. Also see Cross et al. which provides details of the technical development of the tool, the functionality and a summary of the evaluation of its use.(S. E. Cross, 2010) We wanted to use a flexible tool as the basis for our initial prototype. We considered vari- ous drawing packages, as well as more specialised mind mapping tools (such as Inspira- tion and MindManager). In the end we choose to use Compendium (http://compendi- um.open.ac.uk/institute/), a visual representation tool, originally developed for enabling group argumentation, which was produced by researchers at our own institution. We selected Compendium for a number of reasons. Firstly because it was produced at the Open University, we felt there was more opportunity for further tool development, spe- cifically in terms of learning design requirements. Secondly, Compendium supports the creation of a range of visual mapping techniques, including mind maps, concept maps, web maps and argumentation maps (2008), which we felt offered the potential for a range of flexible approaches to the design process. Compendium comes with a predefined set of icons (question, answer, map, list, pros, cons, reference, notes, decision, and argument). The creation of a map is simple; users drag icons across and drop them onto the main window thus creating a node. Relationships between the nodes are built up by dragging between nodes thus creating a connecting arrow. Each node can have an associated name attached and displayed; if a more detailed textual description is associated with the node an asterisk appears next to the node. If the user hovers their mouse over this the content inside the node is revealed. Other types of electronic files can also be easily incorporated into the map such as images, Word files or PowerPoint presentations. The reference node enables you to link directly to external websites. Icons can also be meta-tagged using either a pre-defined set of key words or through user generated terms. Maps can be ex- ported in a variety of ways from simple diagrammatic jpeg files through to inter-linked websites. Compendium provides a utility by which users can create and share new sets of icons, for use as nodes. These sets, know as stencils, contain items where an item defines certain properties of a potential node such as its image icon and label. In the standard version of Compendium, each item inherits the behaviour of one of the standard node types. These standard node types are node which has an icon, text label and other descriptive textual information, link node which links from a node to another node, and view which is a col- lection of nodes and can be displayed either as a map or a list. There are several different mechanisms by which a user can interact with nodes. These include drag and drop (e.g. to instantiate a node as described in the preceding paragraph), double-clicking (e.g. to dis- 1
  8. 8. play and edit details of a node including its text), right clicking (to display a menu offer-ing actions and operations to apply to the node), left-click (to select a node, or allow othermenu driven operations to be executed on the node). We adapted Compendium to make itmore explicit in terms of its use for learning design and this version of the tool is referredto as CompendiumLD – it includes additional functionality such as tailored LD stencilsets and in situ help. In CompendiumLD, behaviour specific to learning design has beenimplemented for these modes of interaction as explained in the next few paragraphs. Fig-ure 2 is a screenshot of CompendiumLD, showing the LD-OU stencil towards the lefthand side, and a map describing each item in the main window.Figure 2: A screenshot of CompendiumLDIn addition to the standard icon set available in Compendium, we have created a series ofstencils specifically for learning design:•Approaches to learning design•LD-Conditional stencil•LD-OU•Learning Design icons•Learning Design templates•Sequence mapping – a stencil to help with laying ut learning activitiesCompendiumLD enables its users to visually represent learning activities in a flexibleway. They can map connections between tutors and learners, tasks, resources and tools,and a variety of notes and links to external websites or documents. The process of map- 2
  9. 9. ping a learning activity in this way involves the user in a cognitive process of external-ising their understanding of the learning activity. This facilitates and drives developmentof their own understanding of the nature of the activity, and the map facilitates commu-nication of this understanding with colleagues. We contend that the process of mappingusing CompendiumLD can improve the quality of activities that will be realised.In addition to providing a visual representation of the design process, we also wanted thetool to provide some form of in-built scaffolding and support to guide decisions at vari-ous points in the process. This we have achieved in a number of ways - by providing sug-gestions for each of the different types of nodes, additional resources and examples, andaccess to a restricted searchable set of additional help features. As an example of the firstkind, when a user drags and drops a "role" node onto the main design area, they arepresented with a menu to select the type of role as shown in figure 3(a). Therefore thissimple prompt reminds them of typical kinds of roles which they might want to include intheir design sequence. The users are not restricted to these roles however and can chooseto type in an alternative role of their specification. This sensitive balance between guidedscaffolding and user flexibility/creativity is an important design principle for our develop-ment of CompendiumLD. A similar form of scaffolding is available for the "tool mode.When a user drags and drops a tool node onto the main design area, they will be presen-ted with a menu to select the type of tool as shown in figure 3(b). Note that the optionsfor tools include ‘Other’, which enables users to specify a tool of their own choosing. Theother tool types available for selection are those currently available with the Open Uni-versitys VLE. The "Other" type allows the designer to specify a tool for face-to-face in-teractions, or a tool not currently supported by the VLE. The tool type selected is storedin CompendiumLDs data model, and tools to query the contents of this data model couldbe used to examine tool usage.Figure 3: Prompts presented for role and tool nodesIn terms of provided additional help, users of the system have the option of letting Com-pendiumLD offer context-sensitive help. For example, as the designer types into a taskdescription label, the words typed are scanned and help related to selected verbs (e.g. col-laborate, consider. discuss, reflect etc.) pops up. An example of such a help window isshown in Figure 4. In this example, the designer has typed Discuss into the task label:this prompts the application to pop up a window showing tools to support discussing andexisting activities that include tasks which include the word discuss. The set of toolsshown in this help window are selected using a verb-to-tool look-up table based on verbswithin a task taxonomy similar to that described by Falconer et al. (2006); the set ofactivities is generated by searching the database maintained by CompendiumLD for 1
  10. 10. activities including tasks with discuss in their label. Further help is provided by theAbout.. buttons. These buttons initiate a customised Google search of selected web sites(http://www.google.com/coop/cse?cx=000971387191123125524%3Alworuyth0qs). Theweb sites were chosen because of the quantity and quality of the information they provideabout use of tools in learning and include sites such as http://www.learning-designs.uow.edu.au/ and http://www.educause.edu/. We adopted this pragmatic approachfor a number of reasons. To create our own hand crafted text would not only be time con-suming but would suffer from quickly becoming dated. However the alternative of a freeGoogle search arguably produced a daunting and untargeted set of resources. The middleapproach we have adopted enables us to focus in on a small set of quality assured sites,which we have checked for relevance and which are likely to be sustained and updated inthe near future. Using a customised search allows means that potentially other institutionsinstalling versions of CompendiumLD could choose to select and include their own tailormade set of resources, which might include institution-specific examples. In our own casewe have a set of tailored resources on tools and their uses within the OU context - thelearn about guides, as well as a set of institutional cases studies on specific uses of VLEtools.Figure 4: Help relevant to a particular activityHelp related to tools that the designer drags and drops onto the window may also beshown. Figure 5 shows an example of help presented when the designer selects Wiki forthe tool type. 2
  11. 11. Figure 5: Help relevant to a particular toolFigure 6 represents a screen shot of part of the learning activity associated with a third-level environmental course (i.e. equivalent to the final year of a full-time, three year de-gree course). Two roles are shown (student and tutor) along with their respective tasks.Tools, resources and outputs (i.e. assets) associated with each task are shown alongside,with arrows indicating connections. 1
  12. 12. Figure 6: Visual representation of part of a collaborative role play activityOur ultimate goal is to provide adaptive and contextualised information on different as-pects of the design process, tailored to individual needs and delivered on a just-in-timebasis. We have now undertaken an extensive number of workshops enabling practitionersto explore CompenidumLD. These have included workshops within the OU, as well asexternally (including the University of Porto, the University of Cyprus, for the EdTechcommunity in Canada, and at numerous conferences). Evaluation of feedback on the useof the tool has enabled us to improve it. We were surprised at how far the participants gotin representing their designs and it did seem during the sessions that CompendiumLD ac-ted as a useful tool to help them articulate and share their thought processes. A few parti-cipants however commented that they did not find representing their designs visuallyhelpful, stating that, for them, pencil and paper/discussion would be preferable. It islikely that such a focus on the visual aspects of the design process will not suit everyone,but overall most participants were positive both during the session and in their evaluationfeedback. 2
  13. 13. Feedback was also positive about our approach to helping teachers/designer consider inmore detail the general issues and use of visualisation and its value in improving thepractice of design. There were some disadvantages noted regarding visualisation butthese were ones we anticipated and provide further valuable food for thought (i.e.someone said some designs may be difficult to describe using this visualisation). Muchof the focus of our use of CompendiumLD during the workshops has been designing atthe level of an individual learning activity, whereas a number of attendees also saw thevalue in stressing course-level design techniques and process as much as for individualactivities and felt that this would have a lot of appeal to teachers. However, whilst theprinciples were appreciated those new to CompendiumLD did encounter some usabilityissues and asked for more guidance and support. In view of this, it is planned that furthersupport will be provided within the application. For example, a movie with a commentarydescribing the basics of creating a learning activity will be provided with the next release.Evaluation of the use of CompendiumLDUse of CompendiumLD by practitionersCompendiumLD has being used in a range of situations with both practitioners and stu-dents. This section provides a summary of the evaluations of the use of CompendiumLD.Data was collected in a variety of ways: via surveys, evaluation of workshops, and ana-lysis of comments on CompendiumLD in a discussion forum. As part of the JISC OULDIproject we have been working with four other institutions (Brunel, Cambridge, LondonSouth Bank and Reading universities) to explore the use of our learning design tools andresources, including CompendiumLD. This has included the running of a series of work-shops, which are available on the Cloudworks site.Maria Papaefthimiou, from Reading University, also carried out an interview with one ofthe course participants to elicit how they were using CompendiumLD. He stated that hefound the tool useful because it enabled him to do what he was doing anyway, but in amore directed way. It also prompted him into thinking and We’ll probably do it on the board to begin with and then I can translate it, or do it at the same time depending on how easy it is into this kind of thing. Which will make people think about “How do we set learning outcomes?” will ... you know “How does that link to that?” You know? I think the intention is to use it again regularly as it is – you know, for other jobs to be done...you know... it should be again part of the process it makes you think about the different components of the learning process in a way that is structured and it makes people address those issues and discuss themWhat is particularly encouraging is that he concludes the interview by saying that usingCompendiumLD has transformed his design practice: you could say no it hasn’t...but I think it’s actually...and my view is that its revolu- tionised our thinking within the school of Estates and Planning to learning and teaching 1
  14. 14. A media designer had the OU used CompendiumLD to draw the tools that were being used in a course. The benefits noted by the designer included:•Enabled the course manager and media developer to determine that there was not likely to beexcessive demands placed on students•Gathered the authors knowledge of the course in one place (the diagram)•Useful for creating an on-the-fly diagram, especially as in this case time was pressing•Clarified the course structure•Diagram represented another vision of the specification (text only specifications often don’tcommunicate what is required that well).•The need for the template to be more flexible was one of the prompts for the creation of theSwim-lane stencil described earlier. We have also routinely gathered information about downloads of CompendiumLD. These show that there has been a steady increase in the number of downloads and that the tool is being used by both course teams, associate lecturers, as well as external individuals (Table 1). Table 1: CompendiumLD download statistics B efore early Fro m 1 N ov e m b er N ove m b er 2009 2009 to m i d-April Total n u m b er of d ownloads 620 332 (nu m b er of d ownloads by N u m b er of O U staff (thos e with 86 22 an @ o p e n.ac.uk ad dress) N u m b er of Associate Lecturers 9 9 (thos e with a .op e n.ac.uk ad- A number of surveys have been carried out after CompendiumLD workshops; these are of two types a workshop survey and an impact survey).(S. E. Cross, 2010) Cross et al. (2010) reflecting on the lessons learnt from the evaluations recognised the importance of having good quality training materials to get users started with the tool. Some of the pos- itive comments included: •That the basis functionality of the tool is straightforward •The value of having a background template that can be used to ensure that a design con- forms to a set of learning design norms 2
  15. 15. •It is easy to change the layout•It is useful for communicating the overall structure of a course•It is good for sharing ideas with others•The drag and drop facility and the ability to link nodes•The ability to ass time values to tasks•The way in which you can export designs in a variety of formats.On the negative side respondents cited the following:•No normal cut, copy and paste facilities•Difficult to save and export and hence share with others•It is not always obvious what the icons should be used for•Better quality image exportsUse of CompendiumLD by studentsThis section will summarise a detailed evaluation of the use of CompendiumLD by stu-dents on the OU’s MAODE H800 course in 2009-2010. Andrew Brasher, the Compendi-umLD technical developer, carried out the evaluation of CompendiumLD.(A. Brasher,2010) The main points are summarised here.The analysis focussed on student and tutor postings in the course discussion forum. 92 ofthe 136 students registered on the course participated in the course forum, of these 78 cre-ated a CompendiumLD map, 2 created a map and a visual representation using anothertool, 5 created a visual represented using another tool and 4 did not create anything.Most students thought that the tool was user-friendly, although it required an investmentin time to become familiar with the interface. They liked the way in which colour wasused and the different nodes for different elements in a design. They also liked the way inwhich the tool enabled them to produce a clear, structured output. Students felt that therepresentation produced was useful in that it did summarise the essence of an activity, en-couraging they also felt that the visual representation could reveal aspects of a design thatare not obvious from a textual representation. They stated that it was particularly usefulfor brainstorming a design.On the negative side, some students found the tool very frustrating and time consumingto learn and use. Others felt that other mapping tools, like MindManger, Cmap or Twine,were more intuitive. Some of the students felt that the representation was essential linearin nature and hence couldn’t be used to produce more circular designs or ones with mul-tiple pathways.Potential uses cited included: sharing design ideas amongst a team of tutors, for a com-plex design the CompendiumLD representation could provide a useful summary, for stu- 1
  16. 16. dent instructions and for planning the logistics of a course – particularly for complexcourses. Negative comments about the tool included: the fact that there is a potential tooverdesign and hence get too focused on mechanism/process and some very sceptical ofthe return on investment of mapping out learning activities, stating that a textual lessonplan is quicker to produce and arguably more useful.Other visualisation toolsConclusionThe empirical evidence we gathered on practitioners’ design practice has informed ourdevelopment of the CompendiumLD tool. We believe that there is no one perfect toolfor design and instead prefer to adopt a pick and mix approach to the design process. Ourinitial findings are positive; however it is clear that there is a need for further research -practitioners are crying out for examples of good practice and guidance in design.However previous research shows that representing learning design practice and provid-ing appropriate support for learning designers is both difficult and contested. By bringingtogether both narrative accounts of learning designs with notational maps showing thedesign visually, we hope to address and find practical ways of approaching the key issuesin this area. CompendiumLD seems to provide an easy to use visual tool to help representdifferent learning designs.However, it is also evident that there are a number of drawbacks with a tool like Compen-diumLD. It is available to download to PC, Mac and Linux platforms. However the toolis relative difficult to learn and it not always intuitive to use. We have had considerablesuccess in recent workshops using paper-based print outs of the icons, rather than thesoftware per se. Another issue is that CompendiumLD is not able to represent the fullrange of design representations, which were discussed in Chapter 5. A better solutionwould be to have a web-based tool, which enables users to oscillate easily between thedifferent design views. In addition, despite our best efforts to include scaffolded guidanceand support, the help facility at the moment is limited and is not as comprehensive as thatavailable in pedagogical planner tools such as DialogPlus, Phoebe and the LDSE dis-cussed elsewhere in this book. Using Cloudworks as a form of pedagogical wrapperaround CompendiumLD is one way of addressing this shortcoming and has been usedsuccessful in a number of our workshops. For example in a workshop at Brunel Uni-versity on 9th November 2009, participants shared and discussed the designs they createdusing CompendiumLD (http://cloudworks.ac.uk/cloud/view/2639). 2

×