This document proposes the development of a tool called SmartEvidence to improve the use of ePortfolios for assessment in Mahara. It summarizes findings that students struggle to present appropriate evidence in their portfolios and need just-in-time guidance. The proposed solution would integrate interventions and instructions into Mahara to help students link evidence to standards more easily. It would also allow different ways to organize, search, and view evidence to help both students and teachers. Mockups provide examples of how the admin and student interfaces could display standards, evidence, and assessments.

1. SmartEvidence
A Mahara Development Proposal
Shane Nuessler and Misty Kirby
shane.nuessler@canberra.edu.au and misty.kirby@canberra.edu.au
University of Canberra

2. Background
Anecdotal evidence that
● Students needed just-in-time interventions (instructions, guidance on what evidence is, etc)
when building their portfolios for assessment.
● Students found Mahara difficult to use unless they had step by step instructions.
● Various disciplines have adopted a template approach to provide interventions and scaffolding,
however students found Mahara unintuitive and did not engage unless required.
● A “how to use it” training approach took time away from the most important task of students
learning the standards, what appropriate evidence consists of, how to collect evidence,
confidentiality and privacy, ethics, etc.
● Excessive time taken to assess portfolios due to:
○ multiple ways students present evidence, regardless of the template
○ having to scan the portfolios visually linking artefacts on screen to particular standards and
competencies
○ spending time assessing portfolios with missing or inadequate rationale for the presence of
the evidence.

3. Preliminary Investigation Findings
Themes from initial conversation between UC and Catalyst
● Concerns around quality of evidence being submitted
● Teachers need to locate evidence efficiently
● Need multiple ways to sort/search/display evidence
● Need to re-communicate interventions through system
● Want students to be able to self assess
● Need to support/encourage reflective process
● Need simple way for students to display and share
evidence/rationale/self assessment etc.
● UC has multiple disciplines with competency/standards
● Take into account use of additional strategies
Themes from literature
● Reliability and validity of ePortfolios
● Reflective component of portfolio practice is critical to deep
learning
● Purpose needs to be clearly defined
● Assessment criteria (and use) needs to be clearly defined
● Need multiple engagement/intervention strategies

4. Preliminary Investigation Findings
Findings from focus groups and interviews with teachers and students
Students
● Had trouble visually connecting groups of evidence together on a page
● Had need for technical training/support
● Needed guidance around what constitutes good evidence (on screen)
● Need for more personalisation
● Students don’t realise they need to be making connections from evidence to multiple standards
● Some want a standard template approach/ others prefer creating their own
Teachers
● Workflows vary from teacher to teacher
● Would like different ways to access and display evidence e.g. grid view of the standards.
● Clicks/complexity was an issue for everyone
● Need to provide clear purpose and direction to students
● Students generally good at describing the evidence but not deriving meaning/connections esp to multiple standards.
● Fear that standardisation would lessen student creativity

5. Proposed Solution
Underpinning concepts:
1. Usefulness:
○ Develop student critical thinking through provision of relevant evidence interventions
that are built into the interface, displayed on demand and when students attempt to provide
evidence against a standard.
○ Mahara provides mechanism to store multiple standards frameworks and related
interventions (making tacit expectations and requirements explicit).
○ Staff can efficiently assess student performance against any given standard.
2. Usability:
○ Students are able to choose the standard(s) that apply to their context, limiting student
exposure to only those standards relevant to them.
○ Mahara makes it easy for students to link evidence to standards - providing an intuitive
and simple means so students can focus on the primary task.
○ Staff can choose how Mahara displays evidence provided by students - e.g. on a student
organised portfolio, or a drill down through matrix of students, or through a matrix of
standards, via reports that highlight lack of evidence against certain standards, etc, etc.

6. Mockups-AdminSettings

7. Mockups-StudentSettings

8. Mockups-Providingevidence

9. Mockups-DisplayingEvidence