Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cardiff OER slides


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Cardiff OER slides

  1. 1. Paul Maharg Glasgow Graduate School of Law simulation: why use it in higher education? paul maharg
  2. 2. why s imulations…? <ul><li>They… </li></ul><ul><li>are close to the world of practice , but safe from the (possible) realities of malpractice and negligent representation. </li></ul><ul><li>enable students to practise transactions , discuss the transactions with other tutors, students, and use a variety of instruments or tools, online or textual, to help them understand the nature and consequences of their actions </li></ul><ul><li>facilitate a wide variety of assessment , from high-stakes assignments with automatic fail points, to coursework that can double as a learning zone and an assessment assignment </li></ul><ul><li>encourage collaborative learning . The guilds and groups of hunters/players in multi-player online games can be replicated for very different purposes in FE & HE. </li></ul><ul><li>students begin to see the potential for the C in ICT ; and that technology is not merely a matter of word-processed essays, reports & quizzes, but a form of learning that changes quite fundamentally what and how they learn . </li></ul>
  3. 3. <ul><li>They: </li></ul><ul><li>are often complex and difficult to build , especially first-time round </li></ul><ul><li>are time-consuming to plan </li></ul><ul><li>can be expensive to run </li></ul><ul><li>can be difficult to embed alongside more conventional forms of teaching and learning </li></ul><ul><li>are disruptive of settled modes of feedback </li></ul>why not s imulations…?
  4. 4. authenticity as transactional learning… <ul><li>active learning </li></ul><ul><li>through performance in authentic transactions </li></ul><ul><li>involving reflection in & on learning, </li></ul><ul><li>deep collaborative learning , and </li></ul><ul><li>holistic or process learning, </li></ul><ul><li>with relevant professional assessment </li></ul><ul><li>that includes ethical standards </li></ul>
  5. 5. <ul><li>Personal Injury Negotiation transaction </li></ul><ul><li>Standardised Client </li></ul><ul><li>Conclusions… </li></ul>
  6. 6. 1. personal injury negotiation transaction <ul><li>Administration: </li></ul><ul><li>272 students, 68 firms, 8 anonymous information sources – PI mentors </li></ul><ul><li>68 document sets, 34 transactions </li></ul><ul><li>Each scenario has embedded variables, called from a document server, making it similar, but also unique in critical ways </li></ul><ul><li>students have 12 weeks to achieve settlement </li></ul><ul><li>introductory & feedback lectures </li></ul><ul><li>discussion forums </li></ul><ul><li>FAQs & transaction guideline flowcharts </li></ul><ul><li>voluntary face-to-face surgeries with a PI solicitor </li></ul>
  7. 7. correspondence file
  8. 8. Ardcalloch directory
  9. 9. map of Ardcalloch
  10. 10. PI project: assessment criteria <ul><li>We require from each student firm a body of evidence consisting of: </li></ul><ul><li>fact-finding – from information sources in the virtual community) </li></ul><ul><li>professional legal research – using WestLaw + paperworld sources </li></ul><ul><li>formation of negotiation strategy – extending range of Foundation Course learning </li></ul><ul><li>performance of strategy – correspondence + optional f2f meeting, recorded </li></ul>
  11. 14. PI project: (some of) what students learned <ul><li>extended team working </li></ul><ul><li>real legal fact-finding </li></ul><ul><li>real legal research </li></ul><ul><li>process thinking in the project </li></ul><ul><li>setting out negotiation strategies in the context of (un)known information </li></ul><ul><li>writing to specific audiences </li></ul><ul><li>handling project alongside other work commitments </li></ul><ul><li>structuring the argument of a case from start to finish </li></ul><ul><li>keeping cool in face-to-face negotiations </li></ul><ul><li>more effective delegation </li></ul><ul><li>keeping files </li></ul><ul><li>taking notes on the process... </li></ul>
  12. 15. PI project: what students would have done differently… <ul><li>‘ In tackling this project I think that our group made two main mistakes. The first mistake we made was in approaching the task as law students as opposed to Lawyers. By this I mean we tried to find the answer and work our way back. Immediately we were thinking about claims and quantum and blame. I don't think we actually initiated a claim until a week before the final settlement. I think the phrase &quot;like a bull in a china shop&quot; would aptly describe the way we approached the problem. […] Our group knew what area of law and tests to apply yet we ended up often being ahead of ourselves and having to back-pedal </li></ul><ul><li>The second mistake we made was estimating how long it would take to gather information. We started our project quite late on and began to run out of time towards the end. None of us appreciated the length of time it would take to gather information and on top of this we would often have to write two or three letters to the same person as the initial letter would not ask the right question.’ </li></ul>
  13. 16. PI project: what students would have done differently… <ul><li>‘ At the beginning we thought we perhaps lost sight of the fact that we had a client whom we had a duty to advise and inform. On reflection we should have issued terms of engagement and advised the client better in monetary terms what the likely outcome was going to be.’ </li></ul><ul><li>‘ […] unlike other group projects I was involved in at undergraduate level I feel that I derived genuine benefit from this exercise in several ways: </li></ul><ul><ul><li>1. reinforcing letter-writing, negotiation, time-management and IT skills </li></ul></ul><ul><ul><li>2. conducting legal research into issues of quantum </li></ul></ul><ul><ul><li>working effectively in a group as a group - not delegating tasks at the first meeting and then putting together pieces of work at the second meeting.’ </li></ul></ul>
  14. 17. key issue: simulation tempo & complexity
  15. 18. key issue: simulation tempo & complexity
  16. 20. Plagiarism … ? <ul><li>Plagiarism is not just about students being selfish, or narcissistic behaviour, or academic cheating or a syndrome or lack of integrity or anything else </li></ul><ul><li>We create fertile conditions for it to flourish by our teaching & assessment designs: </li></ul><ul><ul><li>lack of apprenticeship models </li></ul></ul><ul><ul><li>insufficient situated learning & assessment </li></ul></ul><ul><ul><li>poor academic literacy support within disciplines </li></ul></ul><ul><li>We need to re-design the ecology of learning, eg : </li></ul><ul><ul><li>trading zones, for students > students, staff > staff, students > staff </li></ul></ul><ul><ul><li>teach rhetorical models via games, sims, debriefs, PBL, etc. </li></ul></ul><ul><ul><li>transactional learning </li></ul></ul><ul><li>One attempt to change practice: SIMPLE </li></ul>
  17. 21. correspondence file
  18. 22. <ul><li>Students co-opted to community-police plagiarism </li></ul><ul><li>Students carry out authentic client-based work, not artificial, assessment-led tasks </li></ul><ul><li>ICT is used to create multiple versions of tasks via document variables + support for tasks: feed forward </li></ul><ul><li>Students take responsibility for their transactional learning, their files, their clients, their firm, ie assessment: </li></ul><ul><ul><li>encourages ownership, not submission </li></ul></ul><ul><ul><li>enhances collaboration, not plagiarism </li></ul></ul><ul><li>Staff take responsibility for designing transactional learning </li></ul>Plagiarism … ?
  19. 23. <ul><li>Start with a simple scenario for the first attempt. </li></ul><ul><li>Run a pilot before letting students loose on it. </li></ul><ul><li>Don’t underestimate the skills you might need to get things up and running. </li></ul><ul><li>Begin the process of developing the scenario as early as possible. </li></ul><ul><li>Think in advance about how sim responses will be managed ie when/who/how often: set clear guidelines to students about how this will work. </li></ul><ul><li>Plan & organise well in advance </li></ul>staff advice… staff experiences on SIMPLE sims?
  20. 24. SIMPLE project conclusions: simulation environments <ul><li>Sims can enable more engaged and deeper learning in students, both at undergraduate and postgraduate levels </li></ul><ul><li>Sims can be used to learn and assess conceptual and second-order symbolic knowledge, practice-based skills and personal achievement of integrated skills. </li></ul><ul><li>Students adapt best to new learning environments when they are aware of the expectations of them in the new arena. </li></ul><ul><li>Simulation is a disruptive heuristic and requires support. </li></ul><ul><li>Although initial workload is heavy there is payback in later years </li></ul><ul><li>T here are serious implications for institutional change and innovation </li></ul>
  21. 25. 2. standardised client initiative <ul><li>Training lay people to simulate clients, and doing two things well: </li></ul><ul><ul><li>Discussing their case with the (trainee) lawyer in a way that is standard across the cohort of students that the SC meets </li></ul></ul><ul><ul><li>Assessing the client-facing skills of the lawyer </li></ul></ul>
  22. 26. <ul><li>Large body of research literature criticised oral exams beginning in 1960s </li></ul><ul><li>‘ A test that is not reliable cannot be valid’ e.g. NBME (USA) studies exams of 10,000 medical examiners over 3 years and found correlations between 2 examiners in one encounter <0.25 </li></ul><ul><li>Use of Standardised Patients since 1963 </li></ul><ul><li>Now used in high-stakes competency examination for licensure in USA and Canada </li></ul><ul><li>Extensively used in final exam ‘OSCE’ stations in UK medical schools </li></ul>evidence from medical education
  23. 27. SCI: our hypothesis <ul><li>With proper training and carefully designed assessment procedures, Standardised Clients (SCs) could assess important aspects of client interviewing with validity and reliability comparable to assessment by law teachers </li></ul>
  24. 28. aims <ul><li>develop a practical and cost-effective method to assess the effectiveness of lawyer-client communication which correlates assessment with the degree of client satisfaction. </li></ul><ul><li>ie answer the following questions… </li></ul><ul><ul><li>Is our current system of teaching and assessing interviewing skills sufficiently reliable and valid ? </li></ul></ul><ul><ul><li>Can the Standardised Patient method be translated successfully to the legal domain? </li></ul></ul><ul><ul><li>Is the method of Standardised Client training and assessment cost-effective? </li></ul></ul><ul><ul><li>Is the method of Standardised Client training and assessment more reliable, valid and cost-effective than the current system? </li></ul></ul>
  25. 29. development of new assessment instrument <ul><li>PART A: Global Ratings (process) </li></ul><ul><ul><li>Started with existing GGSL list – quickly discarded </li></ul></ul><ul><ul><li>Attempted to use ELCC form – but unsuited to our needs </li></ul></ul><ul><ul><li>Deep analysis of client-centred experience, ie : </li></ul></ul><ul><ul><ul><li>What constituted good practice re client-centred interviewing </li></ul></ul></ul><ul><ul><ul><li>What were the isolable features of this that could be identified, remembered, analysed in context </li></ul></ul></ul><ul><ul><ul><li>What were the standards we wanted to apply to these features </li></ul></ul></ul><ul><ul><ul><li>How we were going to communicate these to our SCs </li></ul></ul></ul>phase two
  26. 30. development of new assessment instrument <ul><li>PART B: Checklist (content) </li></ul><ul><ul><li>yes/no response on crucial items of information that should have been obtained during course of interview </li></ul></ul><ul><li>PART C: Note to File (advice giving) </li></ul><ul><ul><li>marked by tutor </li></ul></ul><ul><li>A = 80%; B+C= 20% </li></ul>phase two
  27. 31. recruitment of SCs <ul><li>Special agendas – possible screening process needed </li></ul><ul><li>Age/gender role-dependent </li></ul><ul><li>Accessible (tel, email) </li></ul><ul><li>Able to keep data confidential </li></ul><ul><li>Able to accept feedback and to learn from it </li></ul><ul><li>Currently, a group of around 12 SCs are on file </li></ul>phase two
  28. 32. training of SCs <ul><li>‘ The best way to learn how to do standardized patients is to do it along side of someone who has already done it before. It’s [the] apprenticeship system.’ </li></ul><ul><li>Wallace, P. (1997) Following the threads of an innovation: the history of standardized patients in medical education, Caduceus , A Humanities Journal for Medicine and the Health Sciences , Department of Medical Humanities, Southern Illinois University School of Medicine, 13, 2, 5-28. </li></ul>phase two
  29. 33. training the trainers workshop <ul><li>Held by Dr Jean Ker, University of Dundee Medical Faculty, for GGSL staff and SCs. </li></ul><ul><li>Aims: by the end of this introductory workshop the participants: </li></ul><ul><ul><li>recognised how a script develops a character and what part they can play in that development </li></ul></ul><ul><ul><li>recognised the importance of non-verbal cues in maintaining realism </li></ul></ul><ul><ul><li>knew how to practise using a script </li></ul></ul><ul><ul><li>had discussed the importance of a checklist in assessing a student’s performance </li></ul></ul>phase two
  30. 34. formative & summative role-play <ul><li>Practice in formative settings: </li></ul><ul><ul><li>Clients in ICCC </li></ul></ul><ul><ul><li>Clients in Professional Competence Course (PCC) </li></ul></ul><ul><li>Preparation for summative role – the stages included: </li></ul><ul><ul><li>read script as group </li></ul></ul><ul><ul><li>discussed the role </li></ul></ul><ul><ul><li>discussed feelings, reactions </li></ul></ul><ul><ul><li>cleared up ambiguities re role of lawyer </li></ul></ul><ul><ul><li>used feedback to modify the scenario </li></ul></ul><ul><ul><li>practised the role (staff as lawyer), observed by other clients who provided feedback </li></ul></ul>phase two
  31. 35. training as clients <ul><li>There’s a need for the SCs to calibrate: </li></ul><ul><li>Body language </li></ul><ul><li>Tone of voice </li></ul><ul><li>Attitudinal swings </li></ul><ul><li>Dealing with the lawyer’s open questions… </li></ul><ul><li>Improvising on closed questions… </li></ul><ul><li>Performance analysis: ‘What prompted you to say…?’ ‘How did you feel…?’ </li></ul><ul><li>And to: </li></ul><ul><li>Be aware of orientation towards lawyer at first sight </li></ul><ul><li>Respond congruently to the lawyer </li></ul><ul><li>Consult the internal ‘invigilator’… </li></ul>phase two
  32. 36. training to assess <ul><li>We explained marking system, discussed and formed common understanding </li></ul><ul><li>SCs viewed and marked videos, comparing to ‘standard’ </li></ul><ul><li>Viewed each others’ ‘live’ performances and marked </li></ul><ul><li>Process repeated until everyone has role-played once </li></ul><ul><li>Comment on performance </li></ul><ul><li>Marks were collated in the room (suspense factor…) </li></ul>phase two
  33. 37. Client interviewing assessment form phase two
  34. 39. results… <ul><li>Phase three: Jan 2006 – main SC trial </li></ul><ul><ul><li>270+ interviews with SCs & students </li></ul></ul><ul><ul><li>Used new assessment form with students, SCs and tutors </li></ul></ul><ul><ul><li>Carried out statistical analysis </li></ul></ul><ul><ul><li>RESULTS: </li></ul></ul><ul><ul><li>Correlations between SCs/tutor assessment were highly statistically significant across all items </li></ul></ul><ul><ul><li>Correlations between student self-assessment and SCs/tutors were poor </li></ul></ul>phase three
  35. 40. conclusions <ul><li>Use of SCs is as reliable and valid as tutor assessments </li></ul><ul><li>We make what the client thinks important in the most salient way for the student: a high-stakes assessment where most of the grade is given by the client </li></ul><ul><li>We do not conclude that all aspects of client interviewing can be assessed by SCs </li></ul><ul><ul><li>We focused the assessment instrument on aspects we believe could be accurately evaluated by non-lawyers </li></ul></ul><ul><ul><li>Focused the assessment on initial interview </li></ul></ul><ul><li>This has changed the way we enable students to learn interviewing… </li></ul>
  36. 42. next steps <ul><li>Use SCs in other institutions, eg College of Law in England & Wales (York), Northumbria Law School, Franklin Pierce Law Center, New Hampshire, USA. </li></ul><ul><li>Form a network of legal educators interested in using SCs </li></ul><ul><li>Cascade training methods </li></ul><ul><li>Embed methods in local learning cultures </li></ul>
  37. 44. signature pedagogies (Lee Shulman) Sullivan, W.M., Colby, A., Wegner, J.W., Bond, L., Shulman, L.S. (2007) Educating Lawyers. Preparation for the Profession of Law, Jossey-Bass, p. 24
  38. 45. Transforming Legal Education: four key themes
  39. 46. sims affect learning, institutions, practices <ul><li>T here is no such thing as experiential learning. </li></ul><ul><ul><li>Learning is distributed among expanded environments, tools, roles, tasks, social relations. </li></ul></ul><ul><li>T here is no spoon: curriculum is technology. </li></ul><ul><li>Staff role-change vs conventional teaching/admin roles. </li></ul><ul><li>3. The role of the institution changes. </li></ul><ul><li>The question is no longer why conventional learning vs sim, clinic, PBL, etc; but in an era where Wikipedia & SourceForge flourish against all odds, why are we not collaborating at all levels in teaching & learning? </li></ul>
  40. 47. aims of the simSHARE project? <ul><li>Collation of simulation resources which are repurposed as open educational content </li></ul><ul><li>Creation of guidelines for future publication of simulation projects </li></ul><ul><li>Help staff to use simulation more widely and effectively through staff development . </li></ul><ul><li>Create methodologies that will help staff to see more clearly how simulation OER can be interpreted and in particular how to: </li></ul><ul><ul><li>Generate or re-purpose a simulation </li></ul></ul><ul><ul><li>Archive a simulation </li></ul></ul><ul><ul><li>Retrieve a simulation and analyse its component parts for educational value and purpose </li></ul></ul>
  41. 49. my profile
  42. 55. future plans <ul><li>Collation of as many interdisciplinary sims as we can get </li></ul><ul><li>We’re about to enter upload & then dissemination phases of project </li></ul><ul><li>simSHARE adds value to open-source SIMPLE, by disseminating SIMPLE blueprints as open resources </li></ul><ul><li>next step is to add further value to the Open sim environment by adding an open-source e-portfolio, eg Mahara. </li></ul>
  43. 56. transformation of HE? <ul><li>Would you use simSHARE resources? </li></ul><ul><li>Will you share your resources with others? </li></ul><ul><li>Do you think simSHARE OER will help you improve teaching & learning practices? </li></ul><ul><li>Will this help to transform HE? </li></ul>