SlideShare a Scribd company logo
1 of 9
Download to read offline
SmartEvidence
A Mahara Development Proposal
Shane Nuessler and Misty Kirby
shane.nuessler@canberra.edu.au and misty.kirby@canberra.edu.au
University of Canberra
Background
Anecdotal evidence that
● Students needed just-in-time interventions (instructions, guidance on what evidence is, etc)
when building their portfolios for assessment.
● Students found Mahara difficult to use unless they had step by step instructions.
● Various disciplines have adopted a template approach to provide interventions and scaffolding,
however students found Mahara unintuitive and did not engage unless required.
● A “how to use it” training approach took time away from the most important task of students
learning the standards, what appropriate evidence consists of, how to collect evidence,
confidentiality and privacy, ethics, etc.
● Excessive time taken to assess portfolios due to:
○ multiple ways students present evidence, regardless of the template
○ having to scan the portfolios visually linking artefacts on screen to particular standards and
competencies
○ spending time assessing portfolios with missing or inadequate rationale for the presence of
the evidence.
Preliminary Investigation Findings
Themes from initial conversation between UC and Catalyst
● Concerns around quality of evidence being submitted
● Teachers need to locate evidence efficiently
● Need multiple ways to sort/search/display evidence
● Need to re-communicate interventions through system
● Want students to be able to self assess
● Need to support/encourage reflective process
● Need simple way for students to display and share
evidence/rationale/self assessment etc.
● UC has multiple disciplines with competency/standards
● Take into account use of additional strategies
Themes from literature
● Reliability and validity of ePortfolios
● Reflective component of portfolio practice is critical to deep
learning
● Purpose needs to be clearly defined
● Assessment criteria (and use) needs to be clearly defined
● Need multiple engagement/intervention strategies
Preliminary Investigation Findings
Findings from focus groups and interviews with teachers and students
Students
● Had trouble visually connecting groups of evidence together on a page
● Had need for technical training/support
● Needed guidance around what constitutes good evidence (on screen)
● Need for more personalisation
● Students don’t realise they need to be making connections from evidence to multiple standards
● Some want a standard template approach/ others prefer creating their own
Teachers
● Workflows vary from teacher to teacher
● Would like different ways to access and display evidence e.g. grid view of the standards.
● Clicks/complexity was an issue for everyone
● Need to provide clear purpose and direction to students
● Students generally good at describing the evidence but not deriving meaning/connections esp to multiple standards.
● Fear that standardisation would lessen student creativity
Proposed Solution
Underpinning concepts:
1. Usefulness:
○ Develop student critical thinking through provision of relevant evidence interventions
that are built into the interface, displayed on demand and when students attempt to provide
evidence against a standard.
○ Mahara provides mechanism to store multiple standards frameworks and related
interventions (making tacit expectations and requirements explicit).
○ Staff can efficiently assess student performance against any given standard.
2. Usability:
○ Students are able to choose the standard(s) that apply to their context, limiting student
exposure to only those standards relevant to them.
○ Mahara makes it easy for students to link evidence to standards - providing an intuitive
and simple means so students can focus on the primary task.
○ Staff can choose how Mahara displays evidence provided by students - e.g. on a student
organised portfolio, or a drill down through matrix of students, or through a matrix of
standards, via reports that highlight lack of evidence against certain standards, etc, etc.
Mockups-AdminSettings
Mockups-StudentSettings
Mockups-Providingevidence
Mockups-DisplayingEvidence

More Related Content

What's hot

Distance Education
Distance EducationDistance Education
Distance Education
Özge Topal
 
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
MedCouncilCan
 
Overview and technology tools for online assessment
Overview and technology tools for online assessmentOverview and technology tools for online assessment
Overview and technology tools for online assessment
ostromae
 
Distance education (1734342 abdulkadir çepiç )
Distance education (1734342 abdulkadir çepiç )Distance education (1734342 abdulkadir çepiç )
Distance education (1734342 abdulkadir çepiç )
e173434
 

What's hot (20)

ESL learner independence using 21st Century skills_ESL Pedagogy Research Proj...
ESL learner independence using 21st Century skills_ESL Pedagogy Research Proj...ESL learner independence using 21st Century skills_ESL Pedagogy Research Proj...
ESL learner independence using 21st Century skills_ESL Pedagogy Research Proj...
 
Distance Education
Distance EducationDistance Education
Distance Education
 
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
 
Assessments - Data Driven Instruction
Assessments - Data Driven InstructionAssessments - Data Driven Instruction
Assessments - Data Driven Instruction
 
Alternative assessments
Alternative assessmentsAlternative assessments
Alternative assessments
 
Internal assessment & formative assessment
Internal assessment & formative assessmentInternal assessment & formative assessment
Internal assessment & formative assessment
 
Formative assessment
Formative assessmentFormative assessment
Formative assessment
 
San Juan College OTLC - Alexa Wheeler and Elaine Clark - Handout 4
San Juan College OTLC - Alexa Wheeler and Elaine Clark - Handout 4San Juan College OTLC - Alexa Wheeler and Elaine Clark - Handout 4
San Juan College OTLC - Alexa Wheeler and Elaine Clark - Handout 4
 
Formative and summative assessments presentation
Formative and summative assessments presentationFormative and summative assessments presentation
Formative and summative assessments presentation
 
Overview and technology tools for online assessment
Overview and technology tools for online assessmentOverview and technology tools for online assessment
Overview and technology tools for online assessment
 
Evaluation of Teaching Performance for Improvement and Accountablility
Evaluation of Teaching Performance for Improvement and AccountablilityEvaluation of Teaching Performance for Improvement and Accountablility
Evaluation of Teaching Performance for Improvement and Accountablility
 
Designing good assessment
Designing good assessmentDesigning good assessment
Designing good assessment
 
Distance education (1734342 abdulkadir çepiç )
Distance education (1734342 abdulkadir çepiç )Distance education (1734342 abdulkadir çepiç )
Distance education (1734342 abdulkadir çepiç )
 
Mentoring in practice portfolio guidance
Mentoring in practice portfolio guidanceMentoring in practice portfolio guidance
Mentoring in practice portfolio guidance
 
The Student and Distance Education
The Student and Distance EducationThe Student and Distance Education
The Student and Distance Education
 
Formative and Summative Assessment - Faculty Development
Formative and Summative Assessment - Faculty DevelopmentFormative and Summative Assessment - Faculty Development
Formative and Summative Assessment - Faculty Development
 
Assessment & Feedback Reflective Tool
Assessment & Feedback Reflective ToolAssessment & Feedback Reflective Tool
Assessment & Feedback Reflective Tool
 
Programme Evaluation in Open and Distance Learning
Programme Evaluation in Open and Distance LearningProgramme Evaluation in Open and Distance Learning
Programme Evaluation in Open and Distance Learning
 
CEIT 321 Distance education
CEIT 321 Distance education CEIT 321 Distance education
CEIT 321 Distance education
 
It’s time to ‘Face’ the truth. Is Facebook’s Survey Monkey a legitimate rese...
It’s time to ‘Face’ the truth.  Is Facebook’s Survey Monkey a legitimate rese...It’s time to ‘Face’ the truth.  Is Facebook’s Survey Monkey a legitimate rese...
It’s time to ‘Face’ the truth. Is Facebook’s Survey Monkey a legitimate rese...
 

Viewers also liked

Exercícios determinantes pronomes e quantificadores
Exercícios determinantes pronomes e quantificadoresExercícios determinantes pronomes e quantificadores
Exercícios determinantes pronomes e quantificadores
Célia Gonçalves
 

Viewers also liked (9)

SmartEvidence: Enhancing ePortfolios with evidence maps
SmartEvidence: Enhancing ePortfolios with evidence mapsSmartEvidence: Enhancing ePortfolios with evidence maps
SmartEvidence: Enhancing ePortfolios with evidence maps
 
Using Mahara for cross-curricular learning at Perins Secondary School
Using Mahara for cross-curricular learning at Perins Secondary SchoolUsing Mahara for cross-curricular learning at Perins Secondary School
Using Mahara for cross-curricular learning at Perins Secondary School
 
Mahara Hui 14 - What to consider when introducing educators to Mahara - 200214
Mahara Hui 14 - What to consider when introducing educators to Mahara - 200214Mahara Hui 14 - What to consider when introducing educators to Mahara - 200214
Mahara Hui 14 - What to consider when introducing educators to Mahara - 200214
 
MAHARA
MAHARAMAHARA
MAHARA
 
Mahara: Portfolios for everyone
Mahara: Portfolios for everyoneMahara: Portfolios for everyone
Mahara: Portfolios for everyone
 
Shining our Mahara boots for a new era
Shining our Mahara boots for a new eraShining our Mahara boots for a new era
Shining our Mahara boots for a new era
 
Mahoodle in the "SUNY Nebula"
Mahoodle in the "SUNY Nebula"Mahoodle in the "SUNY Nebula"
Mahoodle in the "SUNY Nebula"
 
Exercícios determinantes pronomes e quantificadores
Exercícios determinantes pronomes e quantificadoresExercícios determinantes pronomes e quantificadores
Exercícios determinantes pronomes e quantificadores
 
ePortfolios and Mahara
ePortfolios and MaharaePortfolios and Mahara
ePortfolios and Mahara
 

Similar to SmartEvidence: Improving ePortfolio quality through intervention

BdEurope2011_BlendedLearningWorkshop.pptx
BdEurope2011_BlendedLearningWorkshop.pptxBdEurope2011_BlendedLearningWorkshop.pptx
BdEurope2011_BlendedLearningWorkshop.pptx
Ains11
 
Evaluating the impact of the Pandemic on departmental uses of learning techno...
Evaluating the impact of the Pandemic on departmental uses of learning techno...Evaluating the impact of the Pandemic on departmental uses of learning techno...
Evaluating the impact of the Pandemic on departmental uses of learning techno...
RichardM_Walker
 

Similar to SmartEvidence: Improving ePortfolio quality through intervention (20)

ADOVH Validity and Reliability of Online Assessments.pdf
ADOVH Validity and Reliability of Online Assessments.pdfADOVH Validity and Reliability of Online Assessments.pdf
ADOVH Validity and Reliability of Online Assessments.pdf
 
Technology & Social Inclusion: Enhancing the First Year Experience
Technology & Social Inclusion: Enhancing the First Year Experience Technology & Social Inclusion: Enhancing the First Year Experience
Technology & Social Inclusion: Enhancing the First Year Experience
 
Curriculum development
Curriculum development Curriculum development
Curriculum development
 
Chapter 20
Chapter 20Chapter 20
Chapter 20
 
Unit-3-Development-of-Assessment-Tools.pdf
Unit-3-Development-of-Assessment-Tools.pdfUnit-3-Development-of-Assessment-Tools.pdf
Unit-3-Development-of-Assessment-Tools.pdf
 
performance based assessment.pptx
performance based assessment.pptxperformance based assessment.pptx
performance based assessment.pptx
 
Effective Creation, Mediation and Use of Knowledge in and about Education.
Effective Creation, Mediation and Use of Knowledge in and about Education.Effective Creation, Mediation and Use of Knowledge in and about Education.
Effective Creation, Mediation and Use of Knowledge in and about Education.
 
Designing and conducting formative evaluations
Designing and conducting formative evaluationsDesigning and conducting formative evaluations
Designing and conducting formative evaluations
 
Supervising and working with PhD students - July 2020
Supervising and working with PhD students - July 2020Supervising and working with PhD students - July 2020
Supervising and working with PhD students - July 2020
 
BdEurope2011_BlendedLearningWorkshop.pptx
BdEurope2011_BlendedLearningWorkshop.pptxBdEurope2011_BlendedLearningWorkshop.pptx
BdEurope2011_BlendedLearningWorkshop.pptx
 
NON-SCIENTIFIC MODELS OF CURRICULUM DEVELOPMENT
NON-SCIENTIFIC MODELS OF CURRICULUM DEVELOPMENT NON-SCIENTIFIC MODELS OF CURRICULUM DEVELOPMENT
NON-SCIENTIFIC MODELS OF CURRICULUM DEVELOPMENT
 
Foundations for sustaining learning-centered practices
Foundations for sustaining learning-centered practicesFoundations for sustaining learning-centered practices
Foundations for sustaining learning-centered practices
 
Doctoral Education Online: What Should We Strive For? How Could It Be Better?
Doctoral Education Online: What Should We Strive For? How Could It Be Better?Doctoral Education Online: What Should We Strive For? How Could It Be Better?
Doctoral Education Online: What Should We Strive For? How Could It Be Better?
 
Specific techniques of curriculum evaluation
Specific techniques of curriculum evaluationSpecific techniques of curriculum evaluation
Specific techniques of curriculum evaluation
 
Feed-forward approaches for enhancing assessment and feedback
Feed-forward approaches for enhancing assessment and feedbackFeed-forward approaches for enhancing assessment and feedback
Feed-forward approaches for enhancing assessment and feedback
 
Assessment
AssessmentAssessment
Assessment
 
Learning analytics bij de Universiteit van Amsterdam en ROC Noorderpoort
Learning analytics bij de Universiteit van Amsterdam en ROC NoorderpoortLearning analytics bij de Universiteit van Amsterdam en ROC Noorderpoort
Learning analytics bij de Universiteit van Amsterdam en ROC Noorderpoort
 
ABLE - the NTU Student Dashboard - University of Derby
ABLE - the NTU Student Dashboard - University of DerbyABLE - the NTU Student Dashboard - University of Derby
ABLE - the NTU Student Dashboard - University of Derby
 
Curriculum Evaluation
Curriculum EvaluationCurriculum Evaluation
Curriculum Evaluation
 
Evaluating the impact of the Pandemic on departmental uses of learning techno...
Evaluating the impact of the Pandemic on departmental uses of learning techno...Evaluating the impact of the Pandemic on departmental uses of learning techno...
Evaluating the impact of the Pandemic on departmental uses of learning techno...
 

More from Mahara Hui

More from Mahara Hui (20)

Eportfolios for INTRA placement - a pilot study in the Schools of Biotechnolo...
Eportfolios for INTRA placement - a pilot study in the Schools of Biotechnolo...Eportfolios for INTRA placement - a pilot study in the Schools of Biotechnolo...
Eportfolios for INTRA placement - a pilot study in the Schools of Biotechnolo...
 
Organisation-centric Portfolios
Organisation-centric PortfoliosOrganisation-centric Portfolios
Organisation-centric Portfolios
 
Mahara and Collaboration: Building communities of practice.
Mahara and Collaboration: Building communities of practice.Mahara and Collaboration: Building communities of practice.
Mahara and Collaboration: Building communities of practice.
 
Mahara on Holiday
Mahara on HolidayMahara on Holiday
Mahara on Holiday
 
Exploring e-Portfolios to produce educated employable graduates
Exploring e-Portfolios to produce educated employable graduatesExploring e-Portfolios to produce educated employable graduates
Exploring e-Portfolios to produce educated employable graduates
 
eZines and ePortfolios:Struggling with design issues in ePortfolio content cr...
eZines and ePortfolios:Struggling with design issues in ePortfolio content cr...eZines and ePortfolios:Struggling with design issues in ePortfolio content cr...
eZines and ePortfolios:Struggling with design issues in ePortfolio content cr...
 
Our Mahara Hikoi: Experiences from both sides of the fence
Our Mahara Hikoi: Experiences from both sides of the fenceOur Mahara Hikoi: Experiences from both sides of the fence
Our Mahara Hikoi: Experiences from both sides of the fence
 
To Train or To Support?: Teacher's e-capability development in a tertiary edu...
To Train or To Support?: Teacher's e-capability development in a tertiary edu...To Train or To Support?: Teacher's e-capability development in a tertiary edu...
To Train or To Support?: Teacher's e-capability development in a tertiary edu...
 
Using ePortfolios for Integrated Learning
Using ePortfolios for Integrated LearningUsing ePortfolios for Integrated Learning
Using ePortfolios for Integrated Learning
 
Using Mahara to demonstrate skills acquired in a hospitality workplace practicum
Using Mahara to demonstrate skills acquired in a hospitality workplace practicumUsing Mahara to demonstrate skills acquired in a hospitality workplace practicum
Using Mahara to demonstrate skills acquired in a hospitality workplace practicum
 
Using Mahara ePortfolios for Vocational Industry Currency
Using Mahara ePortfolios for Vocational Industry CurrencyUsing Mahara ePortfolios for Vocational Industry Currency
Using Mahara ePortfolios for Vocational Industry Currency
 
Bridging the Gaps
Bridging the GapsBridging the Gaps
Bridging the Gaps
 
Creating spaces for students to demonstrate competencies in their profession
Creating spaces for students to demonstrate competencies in their professionCreating spaces for students to demonstrate competencies in their profession
Creating spaces for students to demonstrate competencies in their profession
 
Make the connection with your LMS
Make the connection with your LMSMake the connection with your LMS
Make the connection with your LMS
 
Developing a Mahara Bootcamp for Educators
Developing a Mahara Bootcamp for EducatorsDeveloping a Mahara Bootcamp for Educators
Developing a Mahara Bootcamp for Educators
 
Mahara-Juvenile Delinquent or Prodigal ??
Mahara-Juvenile Delinquent or Prodigal ??Mahara-Juvenile Delinquent or Prodigal ??
Mahara-Juvenile Delinquent or Prodigal ??
 
ePortfolios and managing privacy
ePortfolios and managing privacyePortfolios and managing privacy
ePortfolios and managing privacy
 
Cloud hosting your ePortfolio
Cloud hosting your ePortfolioCloud hosting your ePortfolio
Cloud hosting your ePortfolio
 
Embedding ePortfolios in Course Delivery
Embedding ePortfolios in Course DeliveryEmbedding ePortfolios in Course Delivery
Embedding ePortfolios in Course Delivery
 
A proposal for technique to use common terms among multiple systems
A proposal for technique to use common terms among multiple systemsA proposal for technique to use common terms among multiple systems
A proposal for technique to use common terms among multiple systems
 

Recently uploaded

SPLICE Working Group: Reusable Code Examples
SPLICE Working Group:Reusable Code ExamplesSPLICE Working Group:Reusable Code Examples
SPLICE Working Group: Reusable Code Examples
Peter Brusilovsky
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project research
CaitlinCummins3
 
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaPersonalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
EADTU
 
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
EADTU
 

Recently uploaded (20)

DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUMDEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
 
SPLICE Working Group: Reusable Code Examples
SPLICE Working Group:Reusable Code ExamplesSPLICE Working Group:Reusable Code Examples
SPLICE Working Group: Reusable Code Examples
 
ESSENTIAL of (CS/IT/IS) class 07 (Networks)
ESSENTIAL of (CS/IT/IS) class 07 (Networks)ESSENTIAL of (CS/IT/IS) class 07 (Networks)
ESSENTIAL of (CS/IT/IS) class 07 (Networks)
 
8 Tips for Effective Working Capital Management
8 Tips for Effective Working Capital Management8 Tips for Effective Working Capital Management
8 Tips for Effective Working Capital Management
 
How to Manage Website in Odoo 17 Studio App.pptx
How to Manage Website in Odoo 17 Studio App.pptxHow to Manage Website in Odoo 17 Studio App.pptx
How to Manage Website in Odoo 17 Studio App.pptx
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project research
 
An overview of the various scriptures in Hinduism
An overview of the various scriptures in HinduismAn overview of the various scriptures in Hinduism
An overview of the various scriptures in Hinduism
 
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaPersonalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
 
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
 
e-Sealing at EADTU by Kamakshi Rajagopal
e-Sealing at EADTU by Kamakshi Rajagopale-Sealing at EADTU by Kamakshi Rajagopal
e-Sealing at EADTU by Kamakshi Rajagopal
 
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdfFICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
 
Major project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategiesMajor project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategies
 
How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17
 
An Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge AppAn Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge App
 
The Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDFThe Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDF
 
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
 
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of TransportBasic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
 
Đề tieng anh thpt 2024 danh cho cac ban hoc sinh
Đề tieng anh thpt 2024 danh cho cac ban hoc sinhĐề tieng anh thpt 2024 danh cho cac ban hoc sinh
Đề tieng anh thpt 2024 danh cho cac ban hoc sinh
 
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptxAnalyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
 
Mattingly "AI and Prompt Design: LLMs with NER"
Mattingly "AI and Prompt Design: LLMs with NER"Mattingly "AI and Prompt Design: LLMs with NER"
Mattingly "AI and Prompt Design: LLMs with NER"
 

SmartEvidence: Improving ePortfolio quality through intervention

  • 1. SmartEvidence A Mahara Development Proposal Shane Nuessler and Misty Kirby shane.nuessler@canberra.edu.au and misty.kirby@canberra.edu.au University of Canberra
  • 2. Background Anecdotal evidence that ● Students needed just-in-time interventions (instructions, guidance on what evidence is, etc) when building their portfolios for assessment. ● Students found Mahara difficult to use unless they had step by step instructions. ● Various disciplines have adopted a template approach to provide interventions and scaffolding, however students found Mahara unintuitive and did not engage unless required. ● A “how to use it” training approach took time away from the most important task of students learning the standards, what appropriate evidence consists of, how to collect evidence, confidentiality and privacy, ethics, etc. ● Excessive time taken to assess portfolios due to: ○ multiple ways students present evidence, regardless of the template ○ having to scan the portfolios visually linking artefacts on screen to particular standards and competencies ○ spending time assessing portfolios with missing or inadequate rationale for the presence of the evidence.
  • 3. Preliminary Investigation Findings Themes from initial conversation between UC and Catalyst ● Concerns around quality of evidence being submitted ● Teachers need to locate evidence efficiently ● Need multiple ways to sort/search/display evidence ● Need to re-communicate interventions through system ● Want students to be able to self assess ● Need to support/encourage reflective process ● Need simple way for students to display and share evidence/rationale/self assessment etc. ● UC has multiple disciplines with competency/standards ● Take into account use of additional strategies Themes from literature ● Reliability and validity of ePortfolios ● Reflective component of portfolio practice is critical to deep learning ● Purpose needs to be clearly defined ● Assessment criteria (and use) needs to be clearly defined ● Need multiple engagement/intervention strategies
  • 4. Preliminary Investigation Findings Findings from focus groups and interviews with teachers and students Students ● Had trouble visually connecting groups of evidence together on a page ● Had need for technical training/support ● Needed guidance around what constitutes good evidence (on screen) ● Need for more personalisation ● Students don’t realise they need to be making connections from evidence to multiple standards ● Some want a standard template approach/ others prefer creating their own Teachers ● Workflows vary from teacher to teacher ● Would like different ways to access and display evidence e.g. grid view of the standards. ● Clicks/complexity was an issue for everyone ● Need to provide clear purpose and direction to students ● Students generally good at describing the evidence but not deriving meaning/connections esp to multiple standards. ● Fear that standardisation would lessen student creativity
  • 5. Proposed Solution Underpinning concepts: 1. Usefulness: ○ Develop student critical thinking through provision of relevant evidence interventions that are built into the interface, displayed on demand and when students attempt to provide evidence against a standard. ○ Mahara provides mechanism to store multiple standards frameworks and related interventions (making tacit expectations and requirements explicit). ○ Staff can efficiently assess student performance against any given standard. 2. Usability: ○ Students are able to choose the standard(s) that apply to their context, limiting student exposure to only those standards relevant to them. ○ Mahara makes it easy for students to link evidence to standards - providing an intuitive and simple means so students can focus on the primary task. ○ Staff can choose how Mahara displays evidence provided by students - e.g. on a student organised portfolio, or a drill down through matrix of students, or through a matrix of standards, via reports that highlight lack of evidence against certain standards, etc, etc.