More Related Content
Similar to Emerging Practices in e-Assessment
Emerging Practices in e-Assessment
- 1. EMERGING PRACTICES IN E-ASSESSMENT NAV HUNDAL LEARNING TECHNOLOGIST FACULTY OF MEDICINE DENTISTRY & HEALTH UNIVERSITY OF SHEFFIELD
- 2. WHAT WILL WECOVER Who are we? What drives us to develop our online learning and e-assessment? What challenges are we addressing? Key areas of e-assessment focus • Collaborative e-Assessment • Portfolio based e-Assessment • e-Submission, e-Marking & e-Feedback What are we doing in relation to these themes, feedback and recommendations What does the future hold?
- 3. FACULTY OF MEDICINE DENTISTRY& HEALTH Students 2013/14 Undergraduate 2382 (FT 2104 PT 278) Postgraduate 1192 (PGR FT 216 PT 89) PGT FT 359 PT 528 Distance Learning Masters in Dental Public Health Masters in International Health Technology Assessment MSc in LACIC (Language & Communication Impairment in Children MSc Speech Difficulties Masters in Public Health MRes Musculoskeletal Ageing MMid Midwifery
- 4. DRIVERS & AIMS Inline with Faculty Strategic Plans such as ‘Learning & Teaching Strategy’ and ‘eLearning Strategy’ - Promotion of learning, teaching and assessment using technology - Sustainable and flexible structures and online environments - Increasing PGT numbers - Increasing flexibility and accessibility to resources Annual Reflections, feedback from staff and students. Institutional lines of inquiry into electronic assessment methods and tools. Widespread use of Turnitin Further embed Turnitin and Grademark Portfolio based systems
- 5. CHALLENGES • Providingan engaging online environment for group work and collaborative working. • Using technology that is easy to use and accessible for students based across different locations around the globe. • Providing an e-submission mechanism for students to submit assignments online. • Providing a mechanism to deliver
- 6. THEMES Collaborative Portfolio eMarking & eFeedback
- 7. COLLABORATIVE E-ASSESSMENT Trends • Moving away from VLE tools but still administering and instructing via VLE. • ‘ a Google institution’ • Making use of full app suite • Collaborative assessment Tensions • Learning new tools • Google seen as third party • Hangouts • VLE glitches
- 8. FEEDBACK & RECOMMENDATIONS • Raising awareness of Google for education to staff and students. • Stigma of 3rd party • Blend of tools • Benefits of ease of use – some students are using Google already • Little staff development • Student choice – creating communities and flexibility “I feel supported and part of a group” “Enables me to add more variety to the online programmes and encourages inclusion and interaction between students”
- 9. EVIDENCE OR PORTFOLIO BASED E-ASSESSMENT Trends • TNA • Supervisory meetings • Replacing treatment and clinic logbooks • Student progress monitoring Tensions • Engagement • Students needed to see why they are doing it, big task. • Staff time management Dentistry Portfolio • Bespoke e-portfolio for UG • Student centered • Recording treatments • Automation of log books • Staff and student monitoring of progression • Entries are visible, recorded and they get feedback!
- 10. FEEDBACK & RECOMMENDATIONS • Make it meaningful • Explain why they are using the tool! Portfolios can be quite a task • Interact with students, feedback on their evidence • Keep it student focused “Advantages include the flexibility to allow students complete this in conjunction with ongoing research projects. Disadvantages include getting the students to engage with it initially” “Students have better knowledge of their own progress” “More of a flexible way of undertaking tutorial data collection of a physical event”
- 11. E-SUBMISSION, E-MARKING & E-FEEDBACK Tensions • Offline marking • Marking on the iPad • On screen marking • Double Blind marking Trends • Using through VLE • Maybe preferred over from VLE submission • Grademark for richer feedback • Quickmark comments
- 12. TECHNIQUES 1. SecondMarking – Feedback provided by first marker in Grademark. No grade given. Grade held in shared spread sheet. Second marker grades, discussion between two. Mark delivered via VLE. 2. Moderation – Feedback via Grademark. Marks and feedback held in a Google spreadsheet shared between moderators and markers. 3. Print and go paper based post submission! External examiner access
- 13. RECOMMENDATIONS & FEEDBACK • Staff development • Changing marking practice – marking online • Student anxiety – fear of the originality report! • Benefits of different modes of feedback within one tool “Using Grademark has made in-text commenting a lot quicker/easier and while I know some faculty don't like having to be screen-based for marking (they like the break from a computer screen by doing paper reading/marking), I actually prefer not having so much paper to deal with - it makes me feel more organised (whether or not I really am!).” “Encourages in text comments; initial concerns years ago that it restricted WHERE you could mark, with several staff saying they used to mark on trains, or in other places where IT access was limited. Agree but quickly adapt.” “Students are generally happy that they do not have to travel to the University with two printed versions of their work. Submitting online saves them travel costs, time and paper.”
- 14. ON THE HORIZON… OSCE examinations via iPad. • Reliability of marking • Storing marks and scores • Feedback for students • Making the scores meaningful • Software that allows integration with other systems • Software that allows analysis of scores • Wifi – offline syncing capabilities What are we encouraging and supporting? • Greater use of Tii/Grademark • Promoting different learning technologies for e-assessment -TEL FEST • eLearning SIG and Teaching circles for e-Assessment sharing • Sharing techniques • Raising awareness of different methods of eFeedback
- 15. QUESTIONS & DISCUSSION THANK YOU @navhundal n.k.hundal@sheffield.ac.uk
Editor's Notes
- #4 We are the Faculty of Medicine Dentistry & Health. We have departments such as the ScHARR, Dentistry, Medicine, Nursing, Oncology, Orthotics, Human Communications Sciences, Human Metabolism, Cardiovascular Science and Neuroscience. Our undergraduate population for 2013/14 – 2382.. … our UG student population is pre set in terms of student number we can take on. Breakdown and our PG. We have a mix of post graduate taught and post graduate research, as you can see see mix of our FT and PT PGT students is heavier on the PT route. And over the years we have seen an increase of part time PGT students coming into the faculty to study whilst working in their professional roles within Medicine, Dentistry and Health Insert data? So we have a lot of PT PGT students and one way of attracting our students is by offering a range of Distance Learning courses that they can study whilst working. We have a mix of DL programmes within ScHARR, Medicine, Nursing, Orthoptics, HCS and Dentistry …. Here are just a few well established programmes to give you an idea of the sorts of DL programmes we offer across the departments.
- #5 As a driver we as a faculty are keen that that PGT grows, therefor keen to carry on developing our DL provision as a way of attracting working health care professionals. And this growth and supporting the growth of DL is eflected in many ways within Faculty strategic Plans such as our e-Learning Strategy and Learning and Teaching Strategy. The use of innovative technologies to support LTA is something that is central in delivering these aims. Supporting staff to make these developments is something that is important to us to continue and we do this through of sharing practice, growing learning technologist provision and staff and curriculum development. Some of the key points in relation to this from theses strategies are to…. (read bullets) Annual Reflections feedback from Departments highlights current e-assessment specifically as well as feedforward info ----- So why are we interested in e-assessment?
- #6 We have various challenges that we face in delivering online learning and e-assessment. Some of the common challenges we face (and I’m sure many of us face)….. Include the following:… for each one state where we are at now. We currently use Blackboard as our institutional VLE and are also a ‘Google Uni’ We have the VLE MOLE, Google tools and webinar technologies that aid us in doing this We use the VLE assignments tools, Tii and some departments are still doing some form of paper submission And again we have Tii/ Grademark and the assignment tools in MOLE, some staff use email and annotate electronically through MS word etc too.
- #7 I’m going to talk about e-assessment in relation to 3 themes which seem to be growing within the Faculty. These are: Collaborative e-assessment Portfolio or evidence based e-assessment and reflection eSubmission, eMarking and efeedback I’ve picked these key areas of eAssessment to focus in relation to the strand emerging practices, techniques and tensions as each area of e-assessment is developing for us, and this is happening due to technology and tools changing, student feedback and changing practices and in some ways each have their own challenges when it comes to the delivery of e-assessment.
- #8 Collaborative e-Assessment Group work forms a large part of our programme assessment in one form or another, and what we are finding is since the introduction of Google Apps we’re able to use a host of flexible creative apps to deliver this. We have a few strong examples, for example the Health Promotion Module in Dentistry is using Google Docs in stead of the typical VLE wiki tool. Administration and instruction of the task is delivered with via the VLE, so students are allocated groups within the VLE using the randomization of groups ‘tool’, they are given a google Doc to perform a group task and collect and collaborate within the document. Interestingly they also had a discussion forum within the VLE and used this area to go back and discuss their task rather than using the comments facility within a google doc. Similarly we have just taken this one step further with orthoptics, although this piece of assessment is set to run next month and what we are setting up there is a google site. So students are allocated case studies around ethical issues surrounding the subject, it could be patient care or using social media, and rather than using Google docs to replace the wiki we have created a google site and each group of student will be given a page on the google site. This allows the students to collaborate online, they are able to discuss using the page comments, they are free to choose what ever kind of format of content which they will construct around their case study. Using google sites allows them to use text, link to other resources, docs, presentations and videos. So we are hoping this assessment will produce a really nice end resource. Once the groups have completed the task we will open up the page restrictions and they can view eachothers work. Traditionally students would use a wiki, and need to do a final presentation and an individual essay. They are still required to do this, but they will be given more flexibility to present their site instead, or a video etc. And we have a couple more examples of using Google docs in Nursing and Scharr in a very similar way. So what our Google Apps suite is allowing us to do is give the students a little more flexibility in the way in which they want to produce a resource for assessment, so allowing for different levels of creativity and learning styles, but also we are finding that students really enjoy using Google as it’s a tool they are already familiar with. So.. Our trends are….. Our Tensions are Nursing - Collaboration of Google docs, discussions and connect to explain the assessment.
- #9 From following this approach we found: (READ OUT)
- #10 Again we have varying types of ePortfiolo based e-Assessment in that faculty, it’s used with our UG, PGT, PGR. We use PebblePad at the university but it’s by no means in full swing. We have had mixed feedback about it and I’ll talk about that in a sec as to why. I’ll just start with our UG Dentistry Portfolio, it’s a bespoke system maintained by a software developer which has designed and developed it. It’s essentially the automation of log books, so the Dentists go off and carry out treatments, they need to have completed a certain amount and different types of treatments, previously all recorded in a paper book, this portfolio has brought all of that online. It’s very much student centered. Staff can then monitor their progress. It takes them time, but they have to be recorded and now everything is visible for staff and students so they get a lot more feedback on their progress. (clinical log books and targets.... type in their treatments so staff are up to speed with it and monitor automation of books, student progress... 160,000 treatments 18500 unique patients recorded treatments undergrad dentistry... hygiene and therapy diploma ) Our main portfolio is PebblePad…. It’s used across PGR and PGT in pockets for things such as TNA, supervisory meetings and progress tracking. For our CIMA MRes in Musculoskeletal Ageing the e-portfolio was used for summative assessment in the TNA process, The aims were to teach students to identify gaps in their knowledge and skills; to identify the training opportunities to fill that gaps; to reflect on the learning experience and plan how to embed what they learnt in their professional life. This was performed throughout the year. We have assessed 5 students. The students had to perform a training needs analysis based on the RDF, prepare a personal development plan with their supervisors, and give evidence of level of skills at the start with examples, review progression in their skills, log any training activity performed and reflection on what they had learnt and what they were going to do next The e-portfolio was assessed for the quality of the evidence of the level of skills in the TNA, quality of the PDP and number of activities and for the quality of the reflection. I think for us this area is something that we will see grow, however right now it seems to be something that takes a lot of thought into administering. And it’s important to use any e-portfolio tool to it’s full potential because they can be a very powerful tool, so not just something as an evidence store. As this seems to be the case in some examples we have found and it’s a time consuming task for students if they aren’t getting anything from it such as feedback on their evidence, reflection and able to see their own progress. This is something we hope to embed further on the CIMA Mres.
- #12 The big one. Tii has been around for years and although it different for us in that it’s more established in our faculty as opposed to Google and PebblePad, I’ve decided to talk about it because it’s still a tool that continually is developing, is liked by students and is something we want to embed further in our Faculty. Does everyone here know what Tii is? Lots of great examples of the use of Tii and Grademark within the Faculty. People like it’s dual functionality, the originality reports and the grading aspect. Grademark provides a rich blend of feedback opportunities for for staff to use, so audio, quick marks, generic feedback and the rubric. We have no policy on whether or not you must use Tii but it seems to becoming standard practice now. AR feedback suggests it’s an area in which students want to have feedback delivered online and via Grademark. It’s accessible, legible, online. And for us, having a robust e-submission mechanism is key for our DL students. So you can see why we use it…. I’m not going to focus to heavily on it’s functionality and benefits because I think it’s something most people know about but I want to look more at what we need to over come in the future to further embed the tool. Read out Tensions: Using audio… using ipad for offline marking… lots of staff mark on trains. Still getting people interested in the audio aspect. Many staff think 3 minutes isn’t enough although or scared of the voice recording being stored. Double blind marking is a big tension as it is for a lot of institutions. Although this is a feature which is in development it has hindered uptake of Tii. We have various work around for this…. How do we get around double blind marking…. Next slide
- #13 So a big thing for us and I know for many institutions is the inability to overcome the double marking, or blind marking. These are just techniques, I have no solution until this feature is available in Turnitin! Any other suggestions welcome! Scharr Nursing dentistry
- #15 Looking into the future, emerging practices and trends that are developing within the health education sector in-line with the increase of mobile technology adoption. Innovations in mobile application development and the ubiquity of the internet open up a range of possibilities, with this in mind we give consideration to possible future changes in e-assessment for conducting OSCE examinations using mobile devices to overcome some of issues surrounding traditional methods such as reliability of marking, storing scores and providing feedback. In addition to emerging trends we look at the approach taken at both at a faculty and institutional level to support future practice