Building eAssessmentBuilding eAssessment
Michael Gwyther
Connect with the Centre:
twitter.com/VET_Centre
facebook.com/VETDevelopmentCentre
linkedin.com/company/vet-development-centre
vetcentre.vic.edu.au/research-and-reports
Subscribe to VET news and updates:
• What is e-assessment?
• Facilitating webinar based assessments
• Facilitating discussion forum based assessments
• Group work projects
• Principles and tools for mobile evidence capture and
submission
• Student productivity and digital information organisation
• Tools and approaches for student placement reporting
• Principles for developing self paced assessment tasks
Today’s Agenda
ASQAASQA
ASQAASQA
When I hear the
phrase e-Assessment
I think……………….
Burning QuestionsBurning Questions
I am using e Assessments for:
•A I don’t use e Assessments
•B Self Paced Learning
•C Group Tasks
•D Skills Demonstration
•E Something else…
Burning Questions
What is e-Assessment?
E-assessment is the use of information technology
for any assessment-related activity.
Assessment
Distribution
& Collection
Assessment
Distribution
& Collection
Assessment
Response
Assessment
Response
Assessment
Management
Assessment
Management
Assessment
Verification
&
Moderation
Assessment
Verification
&
Moderation
Learners need to know:
1.What they must demonstrate to you as the
trainer to show workplace competency
2. How they need to demonstrate competency
3. What methods you as the trainer will use to
measure and assess their competency.
eAssessment
• Elements and performance criteria
• Required skills and knowledge
• Critical aspects of evidence
• Employability Skills
(usually embedded in the unit of competency)
Assessment Design Tool Principles
Assessment Mapping Tool
Direct Evidence – things that we, as assessor,
observes first-hand, e.g., observation, work samples
Indirect Evidence – things that someone else has
observed and reported to us, e.g., third party reports
Supplementary Evidence – other things that can
indicate performance, such as training records,
questions, written work, portfolios
Types of Evidence
Develop an assessment model for a
blended delivery approach
Face to
Face
Face to
Face
Self
Paced
Self
Paced
Online
Facilitated
Online
Facilitated
Modes of DeliveryModes of Delivery
Face to Face Self Paced
Online
Facilitated
Content
Activities
Assessment
Skills, Knowledge, Critical aspectsSkills, Knowledge, Critical aspects
Face to Face Self Paced
Online
Facilitated
Content
Skills KnowledgeActivities
Assessment
Applying a blendApplying a blend
Direct
demonstration/observation
Indirect demonstration
Products
Workplace documents
Questions - written and
oral
Assignments
Third party reports
Self-assessment
Simulation
Portfolios
Indirect demonstration
Products
Workplace documents
Assignments
Third party reports
Self-assessment
Simulation
Portfolios
Products
Questions - written and
oral
Assignments
Third party reports
Examine your assessmentsExamine your assessments
Work in pairs, 10 minutes
to discuss and list:
•What do we need to do to
ensure our e-assessments
are valid, current,
sufficient and authentic?
•Enter answers at
http://tinyurl.com/qxku23n
eAssessment Principles
Issues and Concerns
• Identity
• Cheating
• Skills
• Variety of assessment
• Information retrieval
• Just online
• Content
• Isolation
• Teachers
• Storage
• Literacy & Numeracy skills
Issues and Concerns
• Identity – Personal data and questions
• Cheating – Turnitin & random tutor contact
• Skills – Online for Knowledge, F2F for skills
• Variety of assessment -
• Information retrieval – Problems, DMT
• Just online – Blend with webinars & F2F
• Content – Mix of video, case study, procedures,
problems.
• Isolation – Social network, forum, FAQ, webinars
• Teachers – Webinar, contact, forum support
• Storage –intuitive navigation, location, privacy law,
integration with SMS systems and services
e-Assessment Scenarios
Work in pairs, 10 minutes each. 1
talk, the other document at:
http://tinyurl.com/p5k75fr
Think of an existing assessment
task ready for an e-assessment
treatment. Describe the existing
task and list:
A What the student needs to
demonstrate?
B Type of Assessment method
C Type of evidence you need to
collect
D Potential issues in using e with
this assessment
Your eAssessment
e-Assessment methods
Trends - Quiz
• Tend to be knowledge retrieval
• Align to required knowledge in work context
• Put comprehension questions in context
• Make a sub set of questions reflecting workplace
process
• Mix up types
(MC, T or F, Matching, Short answer, cloze)
• Ask less questions & focus on good distracters
• Attempts, Feedback and randomisation
Self Paced Content – Writing Quizzes
Trends - Webinar
Plan an assessment activity for webinar delivery
In pairs, share 3 ways student focused webinars
could contribute to assessment at your RTO for
a specific qualification
http://tinyurl.com/ltcgh8l
Suggestion
Trends – Mobile Evidence
What do we need to consider when giving
instructions to learners who use Video, Audio,
Images and Text to demonstrate a skill for
Assessment?
Demonstration Assessment Principles
•Visual evidence (Camera Phone)
•Documented Evidence & Checklists
(Evernote, Writable PDFs, Google forms)
•Phone Casting (iPadio)
•Group Work verification (Wikis)
Assessment
• ipadio combines the telephone with the blog to
create an audio “phonecast” which is streamed Live
to the Internet.
You can phonecast from any phone, anywhere in the
world - no need for a computer or even access to
the Internet.
• Learner can call in their responses via phone call.
iPadio then converts the audio into a blog post
accessible online. No software or app is required by
the caller.
iPadio
• Use your mobile
• Call 1800-802569
• Enter registered number
1161409317625 or 1161353317625
• Enter PIN – 5676
• Talk when prompted
• When finished access at
http://www.ipadio.com/phlogs/MichaelGwyther/
Share an opportunity to use eAssessment in your course delivery?Share an opportunity to use eAssessment in your course delivery?
Phlogging Example
Mobile Evidence Apps
How might your RTO verify evidence capture
from mobile devices or apps off site or from
remote learners?
Question
Trends – Discussion & Social Learning
Give an example of a discussion forum topic
that could work with your learners
Activity 6
•Link to assessment
•Base on participants sharing experiences & prior knowledge
– “learners stories”
•Application to workplace or experience
•Make pre readings short
•Mix of problems, case studies, reflections
•Commenting/Building on postings
•Use to demonstrate employability skills
•Track to determine contribution to group
•Peer Review
Discussions
Work in pairs – 15 minutes
Get involved in the discussion at:
http://tinyurl.com/k3dqegc
Describe a task you could use as a discussion forum
assessment.
Consider the relevant factors
What are the major issues for you to make this work?
Burning Questions Discussions
Workplace futures
•Digital collaboration
•Moving from Training to Performance
Support
•Social and Collaborative Learning
•Information management
Student productivity & digital information
organisation
•Digital Information collation,
storage and retrieval
•Sharing of resources
•Paper less assessment and
reporting
Productivity
•Cloud Storage (e.g. DropBox)
•Social Media (Twitter, LinkedIn, Instagram, Facebook)
•Social Bookmarks
•Blogging
•Electronic signing
Productivity Tools
•Learner/Trainer Digital Literacy
•Cloud Storage
•Responsiveness/Visibility
•Knowledge management and “curation”
Productivity Issues
What productivity tools do you/could you use?
Productivity Issues
Take 10 minutes to review your original e-
assessment idea.
You will talk to these points:
A What the student needs to demonstrate?
B Type of Assessment method
C Type of evidence you need to collect
D Potential issues in using e with this assessment
After each presentation we’ll complete a short peer
review to feed back on the task and suggest ideas!
http://tinyurl.com/kcgsx5n
Don’t be critical – be positive and seek to improve
the task for the presenter
Share your eAssessment for peer review!
Trends - Simulation
Trends - Questioning
Trends - Placement
•Common Issues
•Make assessment responses visible
•Broaden industry knowledge
•Third party reports
•Confidentiality and non disclosure
Sharing placement
Sharing placement
Rubric
Map
Marking Guide
Presentations/Submis
sion
Collect peer review
Share & Asses
Trends – Group work & Peer review
How do you use Group work assessments?
What information do you need to collect to
gauge an individuals contribution to the group’s
output?
Group work assessments
Trends – Demonstration becomes content
Trends – Decision Making Trees
Quiz
Topic Topic Topic
Sequence (Comprehension)
Topics
Topic 1
Topic 2
Topic 3
Branch (Sequence & Choice)
Catch
up
Catch
up
Catch
up
Conditional (Decision making tree)
Content in Decision making trees
Catch
up
Catch
up
Catch
up
Complex Conditional (Decision making tree)
Topic Topic
Conditional (time/mark)
Decision making treesSelf Paced Content
Self paced content
– Cathy Moore
Pull not PushSelf Paced Content
Pull not PushSelf Paced Content
Think of a difficult or dull process you need your
learners to understand.
Describe a scenario that might work for this
content
Burning Question
Draft a scenario
Draft a scenario
Closing remarks & Evaluation
You will receive an email introduction to our
Resource Portal where you can download your
certificate of attendance and other resources from
today’s workshop

E-Assessment ~ Vet Development Centre

  • 1.
  • 2.
    Connect with theCentre: twitter.com/VET_Centre facebook.com/VETDevelopmentCentre linkedin.com/company/vet-development-centre vetcentre.vic.edu.au/research-and-reports Subscribe to VET news and updates:
  • 3.
    • What ise-assessment? • Facilitating webinar based assessments • Facilitating discussion forum based assessments • Group work projects • Principles and tools for mobile evidence capture and submission • Student productivity and digital information organisation • Tools and approaches for student placement reporting • Principles for developing self paced assessment tasks Today’s Agenda
  • 4.
  • 5.
  • 6.
    When I hearthe phrase e-Assessment I think………………. Burning QuestionsBurning Questions
  • 7.
    I am usinge Assessments for: •A I don’t use e Assessments •B Self Paced Learning •C Group Tasks •D Skills Demonstration •E Something else… Burning Questions
  • 8.
    What is e-Assessment? E-assessmentis the use of information technology for any assessment-related activity. Assessment Distribution & Collection Assessment Distribution & Collection Assessment Response Assessment Response Assessment Management Assessment Management Assessment Verification & Moderation Assessment Verification & Moderation
  • 9.
    Learners need toknow: 1.What they must demonstrate to you as the trainer to show workplace competency 2. How they need to demonstrate competency 3. What methods you as the trainer will use to measure and assess their competency. eAssessment
  • 10.
    • Elements andperformance criteria • Required skills and knowledge • Critical aspects of evidence • Employability Skills (usually embedded in the unit of competency) Assessment Design Tool Principles
  • 11.
  • 12.
    Direct Evidence –things that we, as assessor, observes first-hand, e.g., observation, work samples Indirect Evidence – things that someone else has observed and reported to us, e.g., third party reports Supplementary Evidence – other things that can indicate performance, such as training records, questions, written work, portfolios Types of Evidence
  • 13.
    Develop an assessmentmodel for a blended delivery approach
  • 14.
  • 15.
    Face to FaceSelf Paced Online Facilitated Content Activities Assessment Skills, Knowledge, Critical aspectsSkills, Knowledge, Critical aspects
  • 16.
    Face to FaceSelf Paced Online Facilitated Content Skills KnowledgeActivities Assessment Applying a blendApplying a blend
  • 17.
    Direct demonstration/observation Indirect demonstration Products Workplace documents Questions- written and oral Assignments Third party reports Self-assessment Simulation Portfolios Indirect demonstration Products Workplace documents Assignments Third party reports Self-assessment Simulation Portfolios Products Questions - written and oral Assignments Third party reports
  • 18.
  • 19.
    Work in pairs,10 minutes to discuss and list: •What do we need to do to ensure our e-assessments are valid, current, sufficient and authentic? •Enter answers at http://tinyurl.com/qxku23n eAssessment Principles
  • 21.
    Issues and Concerns •Identity • Cheating • Skills • Variety of assessment • Information retrieval • Just online • Content • Isolation • Teachers • Storage • Literacy & Numeracy skills
  • 22.
    Issues and Concerns •Identity – Personal data and questions • Cheating – Turnitin & random tutor contact • Skills – Online for Knowledge, F2F for skills • Variety of assessment - • Information retrieval – Problems, DMT • Just online – Blend with webinars & F2F • Content – Mix of video, case study, procedures, problems. • Isolation – Social network, forum, FAQ, webinars • Teachers – Webinar, contact, forum support • Storage –intuitive navigation, location, privacy law, integration with SMS systems and services
  • 23.
  • 24.
    Work in pairs,10 minutes each. 1 talk, the other document at: http://tinyurl.com/p5k75fr Think of an existing assessment task ready for an e-assessment treatment. Describe the existing task and list: A What the student needs to demonstrate? B Type of Assessment method C Type of evidence you need to collect D Potential issues in using e with this assessment Your eAssessment
  • 25.
  • 26.
  • 27.
    • Tend tobe knowledge retrieval • Align to required knowledge in work context • Put comprehension questions in context • Make a sub set of questions reflecting workplace process • Mix up types (MC, T or F, Matching, Short answer, cloze) • Ask less questions & focus on good distracters • Attempts, Feedback and randomisation Self Paced Content – Writing Quizzes
  • 28.
  • 29.
    Plan an assessmentactivity for webinar delivery
  • 30.
    In pairs, share3 ways student focused webinars could contribute to assessment at your RTO for a specific qualification http://tinyurl.com/ltcgh8l Suggestion
  • 31.
  • 32.
    What do weneed to consider when giving instructions to learners who use Video, Audio, Images and Text to demonstrate a skill for Assessment? Demonstration Assessment Principles
  • 33.
    •Visual evidence (CameraPhone) •Documented Evidence & Checklists (Evernote, Writable PDFs, Google forms) •Phone Casting (iPadio) •Group Work verification (Wikis) Assessment
  • 34.
    • ipadio combinesthe telephone with the blog to create an audio “phonecast” which is streamed Live to the Internet. You can phonecast from any phone, anywhere in the world - no need for a computer or even access to the Internet. • Learner can call in their responses via phone call. iPadio then converts the audio into a blog post accessible online. No software or app is required by the caller. iPadio
  • 35.
    • Use yourmobile • Call 1800-802569 • Enter registered number 1161409317625 or 1161353317625 • Enter PIN – 5676 • Talk when prompted • When finished access at http://www.ipadio.com/phlogs/MichaelGwyther/ Share an opportunity to use eAssessment in your course delivery?Share an opportunity to use eAssessment in your course delivery? Phlogging Example
  • 36.
  • 37.
    How might yourRTO verify evidence capture from mobile devices or apps off site or from remote learners? Question
  • 38.
    Trends – Discussion& Social Learning
  • 39.
    Give an exampleof a discussion forum topic that could work with your learners Activity 6
  • 40.
    •Link to assessment •Baseon participants sharing experiences & prior knowledge – “learners stories” •Application to workplace or experience •Make pre readings short •Mix of problems, case studies, reflections •Commenting/Building on postings •Use to demonstrate employability skills •Track to determine contribution to group •Peer Review Discussions
  • 41.
    Work in pairs– 15 minutes Get involved in the discussion at: http://tinyurl.com/k3dqegc Describe a task you could use as a discussion forum assessment. Consider the relevant factors What are the major issues for you to make this work? Burning Questions Discussions
  • 42.
    Workplace futures •Digital collaboration •Movingfrom Training to Performance Support •Social and Collaborative Learning •Information management Student productivity & digital information organisation
  • 44.
    •Digital Information collation, storageand retrieval •Sharing of resources •Paper less assessment and reporting Productivity
  • 45.
    •Cloud Storage (e.g.DropBox) •Social Media (Twitter, LinkedIn, Instagram, Facebook) •Social Bookmarks •Blogging •Electronic signing Productivity Tools
  • 46.
    •Learner/Trainer Digital Literacy •CloudStorage •Responsiveness/Visibility •Knowledge management and “curation” Productivity Issues
  • 47.
    What productivity toolsdo you/could you use? Productivity Issues
  • 48.
    Take 10 minutesto review your original e- assessment idea. You will talk to these points: A What the student needs to demonstrate? B Type of Assessment method C Type of evidence you need to collect D Potential issues in using e with this assessment After each presentation we’ll complete a short peer review to feed back on the task and suggest ideas! http://tinyurl.com/kcgsx5n Don’t be critical – be positive and seek to improve the task for the presenter Share your eAssessment for peer review!
  • 49.
  • 50.
  • 51.
  • 52.
    •Common Issues •Make assessmentresponses visible •Broaden industry knowledge •Third party reports •Confidentiality and non disclosure Sharing placement
  • 53.
  • 54.
    Rubric Map Marking Guide Presentations/Submis sion Collect peerreview Share & Asses Trends – Group work & Peer review
  • 55.
    How do youuse Group work assessments? What information do you need to collect to gauge an individuals contribution to the group’s output? Group work assessments
  • 56.
    Trends – Demonstrationbecomes content
  • 57.
    Trends – DecisionMaking Trees
  • 58.
  • 59.
  • 60.
    Topics Topic 1 Topic 2 Topic3 Branch (Sequence & Choice)
  • 61.
  • 62.
    Content in Decisionmaking trees
  • 63.
  • 64.
  • 65.
  • 66.
  • 68.
    Pull not PushSelfPaced Content
  • 69.
    Pull not PushSelfPaced Content
  • 70.
    Think of adifficult or dull process you need your learners to understand. Describe a scenario that might work for this content Burning Question
  • 71.
  • 72.
  • 73.
    Closing remarks &Evaluation You will receive an email introduction to our Resource Portal where you can download your certificate of attendance and other resources from today’s workshop

Editor's Notes

  • #9 Assessment activities include the design of assessment tools, the delivery of assessments and the reporting, storing and transferring of assessment data. The Three Types of Assessment   Whatever the pathway that we choose to take, assessment should always be based on clearly defined processes and specific standards rather than on opinions and feelings. There are three types of assessment:     formative assessment summative assessment holistic assessment.   Formative Assessment Formative assessment takes place throughout a training program. The learner is assessed and given feedback as they learn rather than at the end of the program. Summative Assessment Summative assessment is described as assessment conducted at predetermined points in the learning process or at the end. Holistic Assessment Assessment of a range of skills and knowledge together is known as holistic assessment. The methods and tools may assess a number of elements of competence or more than one competency unit at a time. You may know this as integrated assessment. http://www.wikitaa.taa04.com.au/tiki-index.php?page=Types+of+Assessment
  • #10 Potential benefits of e-assessment E-assessment offers a range of potential benefits for RTOs, candidates, industry, assessors, AQTF2 auditors and assessment developers. Benefits for RTOs:  More efficient assessment moderation processes – examples include the use of cloud computing, wikis and document sharing to facilitate exchange of materials between assessors.  Improved consistency and speed in processing assessment evidence through combining human and computer marking – examples include the use of online quizzes and Learning Management Systems (LMS).  Reliable submission and storage of assessment evidence - examples include the use of cloud computing and external storage devices to store and retrieve evidence.  Improved reliability of assessment through using computer marking – examples include the use of online quizzes, SCORM learning objects and LMS.  Enhanced learner engagement through interactive assessment with adaptive feedback – examples include the use of online quizzes and LMS.  Provision of accurate and timely information on the effectiveness of training program design and delivery – examples include the use of wikis, blogs and online surveys to gather candidate and industry feedback. Benefits for candidates:  Improved explanation of competency requirements – examples include the use of forums, blogs, virtual classrooms, video streaming and voice over internet protocols (VoIP).  Gaining immediate feedback – examples include the use of virtual classrooms, online quizzes and LMS.  Improved opportunities for online peer assessment – examples include the use of email, wikis, blogs, voice boards, virtual classrooms and VoIP.  Increased opportunities for self-assessment – examples include use of digital stories, wikis, blogs and online quizzes.  Improved feedback by including links to online support materials – examples include the use of LMS and virtual classrooms. Benefits for industry:  Increased capacity to provide on the job assessment and assessment in remote areas – examples include the use of LMS, mobile phones and tablets/e-readers.  Enhanced capacity to participate in assessment validation – examples include the use of online forums and discussion groups.  Greater flexibility in the timing of assessments – examples include the use of mobile phones, tablets/e-readers and LMS to facilitate on the job assessment. Benefits for assessors:  Rapid sharing of assessment materials between assessors may reduce unnecessary duplication of effort and improve consistency in assessment processes – examples include cloud computing, wikis and document sharing.  Production of rapid and reliable information on candidate progress – examples include mobile phones, LMS, document sharing and voice over internet protocols.  Collection of evidence on skills and knowledge not easily assessed by other means – examples include the use of e-portfolios, gaming, simulations, online role-play and virtual worlds.  Rapid retrieval of assessment documentation – examples include the use of LMS and document sharing systems.  Ease of establishing authenticity of evidence through improved audit trails – examples include the use of email, cloud computing and e-portfolios. Benefits for assessment developers include:  Enhanced question styles which incorporate interactivity and multimedia – examples include the use of digital images and audio files to support questioning.  Greater variety and authenticity in the design of assessments – examples include the use of simulations and virtual worlds.
  • #11 Potential benefits of e-assessment E-assessment offers a range of potential benefits for RTOs, candidates, industry, assessors, AQTF2 auditors and assessment developers. Benefits for RTOs:  More efficient assessment moderation processes – examples include the use of cloud computing, wikis and document sharing to facilitate exchange of materials between assessors.  Improved consistency and speed in processing assessment evidence through combining human and computer marking – examples include the use of online quizzes and Learning Management Systems (LMS).  Reliable submission and storage of assessment evidence - examples include the use of cloud computing and external storage devices to store and retrieve evidence.  Improved reliability of assessment through using computer marking – examples include the use of online quizzes, SCORM learning objects and LMS.  Enhanced learner engagement through interactive assessment with adaptive feedback – examples include the use of online quizzes and LMS.  Provision of accurate and timely information on the effectiveness of training program design and delivery – examples include the use of wikis, blogs and online surveys to gather candidate and industry feedback. Benefits for candidates:  Improved explanation of competency requirements – examples include the use of forums, blogs, virtual classrooms, video streaming and voice over internet protocols (VoIP).  Gaining immediate feedback – examples include the use of virtual classrooms, online quizzes and LMS.  Improved opportunities for online peer assessment – examples include the use of email, wikis, blogs, voice boards, virtual classrooms and VoIP.  Increased opportunities for self-assessment – examples include use of digital stories, wikis, blogs and online quizzes.  Improved feedback by including links to online support materials – examples include the use of LMS and virtual classrooms. Benefits for industry:  Increased capacity to provide on the job assessment and assessment in remote areas – examples include the use of LMS, mobile phones and tablets/e-readers.  Enhanced capacity to participate in assessment validation – examples include the use of online forums and discussion groups.  Greater flexibility in the timing of assessments – examples include the use of mobile phones, tablets/e-readers and LMS to facilitate on the job assessment. Benefits for assessors:  Rapid sharing of assessment materials between assessors may reduce unnecessary duplication of effort and improve consistency in assessment processes – examples include cloud computing, wikis and document sharing.  Production of rapid and reliable information on candidate progress – examples include mobile phones, LMS, document sharing and voice over internet protocols.  Collection of evidence on skills and knowledge not easily assessed by other means – examples include the use of e-portfolios, gaming, simulations, online role-play and virtual worlds.  Rapid retrieval of assessment documentation – examples include the use of LMS and document sharing systems.  Ease of establishing authenticity of evidence through improved audit trails – examples include the use of email, cloud computing and e-portfolios. Benefits for assessment developers include:  Enhanced question styles which incorporate interactivity and multimedia – examples include the use of digital images and audio files to support questioning.  Greater variety and authenticity in the design of assessments – examples include the use of simulations and virtual worlds.
  • #15 It is easier to think of Elearning as a suite of Blended delivery choices available to RTOS: These modes can be any mix of face to face and either or a combination of online self paced, facilitated or group work sessions. Students participate in both traditional face to face classes and virtual classes such as online tutorials or discussion forum activities.   Face to Face – Trainers and learners participate in online activities or resources during face to face sessions. Flexible, Self Paced Trainer or computer assessed – Learner works through prepared learning content that is assessed as they progress. Access to trainer generally for support or higher level assessment task feedback. Facilitated online – learning is conducted entirely online through course material provision provided through a Learner Management System and online activities provided though either virtual classroom tutorials or discussion forums. Not such an attractive model for RTOs as it does not attract Government funding incentives for training placements.   Many RTOs commence using Elearning via the provision to learners of a Resource repository – learners access content online and refer to resources that are used in a face to face class. Not really considered elearning.     Informal networks based on knowledge sharing (also known as social learning) – “The revolution that is social media means that now everyone can have access to the Social Web and a range of services and applications to support their own as well as their team’s learning, performance and productivity. “ Jane Hart.  
  • #16 For each course we can determine the best mix of delivery modes according to: Infrastructure available to deliver in each mode Capability of our organisation and our learners to teach and learn in each mode Strategic choices about how to package courses for blended learning The types of content we will need to delivery Learning activities Assessment methods and potential for assessment submission.
  • #17 For each course we can determine the best mix of delivery modes according to: Infrastructure available to deliver in each mode Capability of our organisation and our learners to teach and learn in each mode Strategic choices about how to package courses for blended learning The types of content we will need to delivery Learning activities Assessment methods and potential for assessment submission.
  • #22 Identify is a perennial question in e-Assessment. Is the person submitting the assessment the person who is enrolled in the course? This has ramifications for both the integrity of the RTO but also the application of knowledge and skills in the workplace, not to mention potential liability. While identify of submitted written assessments is always an issue for RTO it is especially magnified in the online delivery. Cheating – Turnitin can help with written assessments, however for shorter compliance courses tutor contact is essential to back up what the online system is telling you. If this is not possible, do a random selection of learners. Call and interview especially around the key knowledge so you have an extra observation to back up their results from the online system. Skills – You can only use a simulated environment where the unit permits you too. Otherwise you’ll need to either do it in person or develop a strategy for remote evidence collection (apps, phone video, podcast) Varity of Assessment – too many online courses rely soley on quizzes. While good for information retrieval and knowledge concepts, they are notoriously poor indicators of problem solving ability and information gathering to assist with new workplace processes. Decision making trees and problem based learning are useful cross checks here. Just online – Much of corporate learning is heading towards just in time learning kiosks to support employees to solve many of their own workplace needs. The provision of a portal with video, short job aids, case studies, access to a trainer si where they are heading. Formal training is being saved for the big ticket items. Content – A lot of online content I see from RTOs is deadly boring. Text with few or no images, little if any video, occasional use of sound to support learning concepts and assessment based purely on short answer comprehension and quizzes. Its very structure encourages learners to click through – it fails to immerse learners to think, absorb and apply information to vocational contexts. It also fails to show learners how to use the resources around them in the workplace (people, procedures, SOPS and manuals) to solve their own problems on the job. This was one of the key planks of the employability skills. Using webinars (live or recorded), PowerPoint's supported by audio, simple decision making trees (using the Lesson in Moodle or other rapid elearning software) Isolation – Online learners generally work in isolation where the RTO has focused purely on self paced delivery mode without use of any F2F or online facilitated. Consider how learners may learn from one another by either sharing their workplace context or by running regular mandatory interactive webinars which will help you get a sense of the learners in your group and their progress toward competency. Teachers – They are invisible in online self paced courses. The voice of the resource is generally that of the subject matter expert or instructional designer. Consider how to make the trainer active through forums, webinars and phone contact.
  • #23 Identify is a perennial question in e-Assessment. Is the person submitting the assessment the person who is enrolled in the course? This has ramifications for both the integrity of the RTO but also the application of knowledge and skills in the workplace, not to mention potential liability. While identify of submitted written assessments is always an issue for RTO it is especially magnified in the online delivery. Identity – the perennial issue raised by every dissenter to elearning. Need strategies in place to check and verify identity. The face to face savings need to be diverted to more stringent identity checks. Phone calls, support, weekly emails. Cheating – Turnitin can help with written assessments, however for shorter compliance courses tutor contact is essential to back up what the online system is telling you. If this is not possible, do a random selection of learners. Call and interview especially around the key knowledge so you have an extra observation to back up their results from the online system. Skills – You can only use a simulated environment where the unit permits you too. Otherwise you’ll need to either do it in person or develop a strategy for remote evidence collection (apps, phone video, podcast) Varity of Assessment – too many online courses rely soley on quizzes. While good for information retrieval and knowledge concepts, they are notoriously poor indicators of problem solving ability and information gathering to assist with new workplace processes. Decision making trees and problem based learning are useful cross checks here. Just online – Much of corporate learning is heading towards just in time learning kiosks to support employees to solve many of their own workplace needs. The provision of a portal with video, short job aids, case studies, access to a trainer si where they are heading. Formal training is being saved for the big ticket items. Content – A lot of online content I see from RTOs is deadly boring. Text with few or no images, little if any video, occasional use of sound to support learning concepts and assessment based purely on short answer comprehension and quizzes. Its very structure encourages learners to click through – it fails to immerse learners to think, absorb and apply information to vocational contexts. It also fails to show learners how to use the resources around them in the workplace (people, procedures, SOPS and manuals) to solve their own problems on the job. This was one of the key planks of the employability skills. Using webinars (live or recorded), PowerPoint's supported by audio, simple decision making trees (using the Lesson in Moodle or other rapid elearning software) Isolation – Online learners generally work in isolation where the RTO has focused purely on self paced delivery mode without use of any F2F or online facilitated. Consider how learners may learn from one another by either sharing their workplace context or by running regular mandatory interactive webinars which will help you get a sense of the learners in your group and their progress toward competency. Teachers – They are invisible in online self paced courses. The voice of the resource is generally that of the subject matter expert or instructional designer. Consider how to make the trainer active through forums, webinars and phone contact.
  • #27 Quizzes are distributed to learners via a LMS and added to mop up bits and pieces of knowledge that may not fit precisely into a skills demonstration. Quizzing is the most popular eAssessment method but they are notoriously difficult to write if you are going to go beyond simple knowledge retrieval and checking. Writing the question with a realistic set of detractors to the correct answer takes skill. Do you need to add feedback if learners are correct? Quizzes are useful for: Replacing short answer questions that require marking Breaking down processes into a set of stepped quiz questions Meaning of words and phrases Scaffold off a video or other presentation (PowerPoint) Checking knowledge progression through a course.
  • #29 When you offer a webinar to your learners you are truly entering blended learning territory. Not only are they wonderful for building a community of learners and delivering content and offering opportunities for clarifying your expectations, webinars also provide e-assessment opportunities especially for CIV and Diploma level learners. They are very much about the students voice and if you run remote courses, provide you with good evidence on learner progress you can cross match against other submitted evidence for direct follow up or additional questioning where you have doubts on a student’s progress or identity. Methods include: Remote skills demonstration Assessment Task Pitch Group problem solving, Case studies or scenarios Peer Review of presentations and Group projects Communication role plays Webinars can be recorded and stored either as files or direct links to supplier cloud hosting. For many people the experience of teaching and learning online has been a particularly isolated and isolating one. Virtual classroom technologies offer teachers opportunities to provide a more human, real time interaction. It is possible to give learners a sense of intimacy with you, their teacher, and co learners. It also allows you to offer immediate support and counselling to learners. Finally these technologies allow you to build greater interactivity into your online learning programs and to include remote experts in the delivery. Features   usually include live audio and text communication between trainers and learners. Participants use headsets to communicate through audio can include graphic material like PowerPoint slides which is broadcast to all participants. Can include the running of polls to gain input or feedback from them Enables trainers control of the microphone or “first in best voiced” settings depending on formality of session the presenter share an application (e.g. Word or Excel) running on their desktop so that the group can work together collaboratively on a document or other resource. The presenter can often hand over application sharing to another member of the group may include a shared whiteboard space for text and drawing the presenter and participants can co-browse the Web and conduct web tours the presenter can share files with the participants. the presenters and participants can display live video from a web cam, may include “break out” rooms that enable small group work online can be archived or recorded to enable later reviewing of a trainer/learner led session.
  • #32 Evidence capture of skills involves the use of a combination of video, audio, text. Capturing skills evidence via mobile phones, tablets, HD portable cameras is an effective method when used with a specific evidence framework: No hoodies! Speaking to camera and identify self and processes working through Reinforced by pictures and text or audio from the learner Comments from Work supervisor Guided by checklist provided by the learner Before and after skills demonstration views of the work/site/ etc Answering of supplementary questions either verbally within the mobile evidence method or later via quiz or short answer
  • #37 Visual evidence (Camera Phone) Documented Evidence & Checklists (Evernote, Writable PDFs) Phone Casting (iPadio) Group Work verification (Wikis)
  • #39 Course information & updates Engaging learners in warm-up activities Problem-based learning (debates, role-plays, problem-based activities, projects, feedback, resources) Learners stories (industry knowledge) Group work Projects/journal Feedback and evaluation on the course Discussion activities can be undertaken using a variety of methods: Discussion forums Social networks Voice boards Instant Messaging Comments posted to wikis or blogs or LMS Need to be mapped to Assessment if content driven! Link to assessment Base on participants sharing experiences & prior knowledge – “learners stories” Application to workplace or experience Make pre readings short Mix of problems, case studies, reflections Commenting/Building on postings Use to demonstrate employability skills Track to determine contribution to group Peer Review (Moodle post rating)
  • #50 These can range from simple games and decision making trees to highly developed workplace simulations or virtual worlds. Expensive to implement and usually requires industry/government support at RTO level – for example the White Card game http://www.whitecardgame.com.au/
  • #51 Dialogues between trainer and student at higher level qualifications can support learners to refine their assessment response to major written assessments, project pitches, placement questions etc. Time consuming but excellent way to record learners journey to summative assessment competence as well as identify verification. Dialogue tool in Moodle or use of targetted social media tool such as Yammer.
  • #52 Share experiences Wiki/LMS/Social Media Common issues Variety in the sector Private V Shared Ethics & Confidentiality practised and applied “Case Study” style assessment creates learning for subsequent learners Common Issues on the job Make assessment responses visible Third party reports Confidentiality and non disclosure
  • #54 Two ideas Google calendar as a diary Learner group private wiki
  • #55 Group work always leaves the teacher vulnerable at audit for decisions made in determining the amount and quality of each learner’s contribution.   Peer Review tools allow teachers to survey learners anonymously on each of their team member’s contribution to group process and project outcomes as well as rate the quality of content/ideas/resources they have provided or contributed to. Each learner as part of their assessment would rate each of their team members on contribution to group processes quality of work product contributed This gives the teacher additional information for assessment. A model that can work is: Develop a rubric with learners Check for mapping Distribute back to learners as marking guide Rehearse the marking guide Conduct presentations (summative) Collect peer review Redraft and resubmit Share and assess Moodle has an underutilised resource called Workshop in which you can develop, distribute and mark peer review assessments. The process of participating in the peer review can be mapped to critical aspects of assessment that involve performance reflection, communication and working in groups, and industry knowledge.
  • #57 Film learners/third party/supervisor demonstrations Ensure learner speaks to camera to describe what they have done, job requirements, conditions and special aspects of job Add to LMS as content and or as part of peer review assessment where learners contribute to assessing appropriateness of tools, equipment, OHS, sustainability, quality and finish on a job. Learners view same job in variety of conditions and settings
  • #58 While quiz based, DST goes beyond the linear/right/wrong nature of quizzes enabling powerful mazes of understanding to be created that test both the decision making capability of the learners as well as their resourcefulness of acquiring and apply knowledge from sources the trainer provides. Quiz based but wraps a story around a workplace process you want to check for a learners understanding of applying problem solving and use of external resources and materials (procedures, policies etc). Scenario driven and problem based. Usually built into a LMS but many applications allow you to easily develop and distribute DMT in conjunction with content (Captivate, Articulate Storyline, iSpring etc). Articulate can also house and score the content for you in the cloud. DMT useful for both formative and summative assessment Decision-making trees are sometimes referred to as Action mazes. Similar in concept to a 'choose your own adventure' novel, they provide a highly effective structure within which you can lay out options. Learners can investigate the possible outcomes of choosing those options. They help the learner to form a balanced picture of the risks and outcomes associated with each possible course of action.   Students can safely explore the 'cause and effect' nature of a scenario or process. Consider:   an occupational health and safety scenario a simulation of a practical activity that might involve dangerous chemicals or equipment a scenario, such as a human relations dispute, that might take weeks to 'play out' in real time.
  • #63 Surround the learner with content to help them decide the right course of action Manuals FAQx Videos Supervisors or Experts Opinions Easy Job Aids (PDF)
  • #66 This is how the DMT differs from the quiz
  • #67 An example of content integrated with activity and assessment. Starts with a problem that a learner has to solve by investing the provided content or by exploring the answers. “Ask Joe” is content in audio-visual form to assist the learner understand approaches to this particular problem. A wide variety of existing content can be used in DMT
  • #68 DMT are most effective when a workplace issue is contained within a story with specific characters. Learners are usually seeing the action from a workers perspective in the role their training is preparing them to occupy. This enables learners to explore process, communication, information and the application of policy and procedure in a simulated workplace setting.