This is a useful start because participants can begin to get to know each other. Introductions help them to feel at ease in the group and ready to contribute to discussions. Facilitators can note concerns (perhaps on a flip chart) and ensure that they are met during the session. If you’re using this presentation for private study, consider why you want to use portfolios for assessment. You might like to note down a few points. Also list any concerns you have. You can go back to these when you finish reading the slides and check that you’ve managed to get what you wanted out of the session. If not you might want to consult some of the literature or links (see slides 30 and 31).
These learning outcomes provide participants with a structure for the session. If you’re using this alone, this is what you should be able to do when you’ve studied the slides.
It’s important that portfolios provide evidence of the attainment of the learning outcomes of a course. Portfolios would naturally be useless if they consisted of lots of random writing. They need to be carefully planned, delimited and relevant to course content.
The potential for the student to select work for inclusion in a portfolio is an appealing characteristic of this mode of assessment. The process is comparable to a designer selecting their most exciting designs to show to a prospective employer. Students can play to their strengths and draft and redraft portfolio content. The preparation of a portfolio therefore provides multiple learning opportunities. Diverse aims can be met through portfolio production, ranging from raising awareness of professional learning through reflection to providing an overview of achievements (eg outcomes of personal development planning/key skills development).
Portfolios can be regarded as a flexible vehicle for the provision of evidence of the attainment of learning outcomes. Their content aligns closely to what the student has done to learn, ie their learning activities. Portfolio assessment harmonises with learning outcomes and learning activities. In contrast, a written exam can tend to be something of an add-on at the end of a course. Traditional exams often favour students with poise and the ability to remember facts and write quickly under pressure. Steady learners who find cramming difficult may not always do well in exams. Portfolios invite students to adopt a more profound and analytical approach to learning.
Portfolio content can be assessed formatively before inclusion in the final portfolio which is assessed summatively.
This figure shows how the student is at the centre of a feedback dialogue focusing on their work. Students reflect on their work and redraft (self assessment), discuss their work with peers (peer assessment) and also get feedback from their tutor/lecturer. They ask questions and discuss any troublesome areas, and redraft. Portfolios can provide the opportunity to ‘close the feedback loop’, in this case acting on feedback immediately to improve current work prior to inclusion in the portfolio for summative assessment. This is an excellent way of learning from any mistakes and eliminating weak areas or misunderstandings.
This slide underlines the way portfolios can both evidence and support learning. These ideas have been introduced in earlier slides. Baume’s point about the integration of learning from ‘beyond’ the course is particularly relevant to work-based learning and off-campus learning. Students go out into the world and subsequently use notes, diaries, video or sound recordings to create a narrative of their experiences, relating them to course content. This type of narrative lends itself well to inclusion in a portfolio.
The introduction to portfolios is now complete. The focus moves to outcome 1 on planning so that students can benefit from portfolio assessment.
This slide illustrates the link between learning outcomes, teaching and learning activities, and assessment methods. Activity lies at the heart of learning. Factual knowledge is the bedrock of learning but students need to know how to approach, apply and use knowledge as appropriate for their discipline/s.
This slide is useful as a basis for analytical approaches to learning in different subject areas. Participants should consider how the approaches to using knowledge suggested on the right hand side of the table relate to their discipline and their students’ learning. If you are using this resource for private study, consider how you can encourage your students to engage in ‘adaptive’ learning.
When we set assessment tasks, we often focus on assessing knowledge of course content. We don’t always aim to assess higher order learning. However, university learning should result in higher order learning such as the ability to analyse and solve problems. Such abilities should be developed through learning activities and tested through assessment. Scenarios and projects create more authentic contexts for assessment. They result in richer texts than those which simply require the descriptive replication of factual knowledge acquired through reading or lectures. The texts themselves and reflection on them can be included in portfolios.
This model (reproduced by kind permission of the authors) illustrates a cycle of assessment and feedback. It is based on developing student understanding of assessment criteria. Criteria stipulate the aspects of an assessment task which will be evaluated and detail the quality of the task (writing, presentation, performance, etc) required for a pass/distinction and/or other grades. Descriptors, which detail the characteristics of work of a certain standard, can also serve to clarify assessors’ expectations. O’Donovan and Rust have given students the opportunity to apply assessment criteria to examples of written work. Students’ deepened understanding of what is required has resulted in improved work. This approach is particularly useful when assessment tasks focus on testing higher order learning. Students may not know what analysis is, for example, until they see an example of analysis in another student’s work. Assessment criteria should be available to students at the start of a course. They act as a road map which helps students to achieve intended learning outcomes. Criteria do not prevent short excursions off the main route to stimulating areas of interest. Such diverting activities are to be encouraged. Criteria ensure that core outcomes are articulated and understood so that it is more likely that students will attain them. Many diverse examples of assessment criteria are available. It is important that assessors write their own criteria, share them with colleagues, and that they are not too complex to work in practice. Ulster’s comprehensive assessment handbook is available online and includes a range of examples. http://www.ulster.ac.uk/academicoffice/download/Handbooks/Assessment%20Handbook.pdf
The focus now moves to an important rationale for using portfolios for assessment, improved student learning.
Burns’ research relates back to the dicussion of the portfolio as a vehicle for learning and the assessment of learning both formatively and summatively (see slides 9-11 above).
Woodward’s work is useful because it emphasises the importance of clear and strict guidelines and frameworks for portfolios, whether electronic or paper.
This slide illustrates the range of approaches to the content of electronic portfolios. The point about the emphasis on evidence of learning rather than on elegance or sophistication of presentation is central. Assessment criteria, based on clear guidelines, can provide information to students on the weighting of the graphics or layout of the portfolio. It is conceivable that design may be an important aspect of the portfolio, depending on the topic of the course. It is essential for the course team to decide what is important and ensure that the essentials are reflected in the portfolio assessment criteria.
Wilhelm focuses on the usefulness of portfolios for teachers. Student work can help programme leaders to develop their courses on the basis of student performance as reflected in portfolios.
Jones’s study confirms that the production of portfolios enables students to engage in reflection. They will then be able to consider specific aspects of their practice and ways of developing them.
This is an appealing approach beause it provides students with the opportunity to expand on portfolio content during an interview. Examiners can explore specific areas in more depth. This is rather resource-intensive, however.
This outcome enables participants to link presentation content to their practice.
These how-to steps build on previous slides. The final point about limiting the extent/volume of portfolio content is vital. Quality is the aim. Extracts of excellent earlier work (anonymised) can be uploaded to a virtual learning environment to provide students with examples (provided permission is obtained from the authors).
This slide relates to slide 16 above.
This illustration shows how choice of content can be embedded into portfolio assessment. Choice is motivating for learners.
Baume’s points focus on the student experience of portfolio assessment. Students are often enthusiastic about producing portfolios. If this presentation is used as part of a workshop, discussion can follow. If there is time, participants can sketch initial plans for introducing portfolios into the assessment diet on one of their courses and share the results with colleagues. If you are working alone, consider if/how you might develop portfolio assessment for one of your courses.
Assessing student learning in diverse ways: Portfolios Rosalind Duhs Centre for the Advancement of Learning and Teaching (CALT) This document is licensed under the Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales license, available at http://creativecommons.org/licenses/by-nc-sa/2.0/uk/.
support the development, demonstration and valid assessment of a wide range of personal, professional and academic capabilities, both inside and outside a programme of study; (Personal Development Planning PDP)
provide evidence of work done and learning achieved;
show reflection on and analysis of evidence and learning;
support the integration of learning from different parts of the course and beyond.
Plan portfolio assessment in line with intended learning outcomes and course learning and teaching activities to enhance higher order learning
Planning aligned assessment methods Think and act like a biologist, historian, or computer scientist, etc. Teaching and Learning Activities Assessment methods Learning outcomes, learning activities and assessment are tightly linked. Learning outcomes Act like a biologist, historian or computer scientist Learn to act like a biologist, historian or computer scientist
What is learning/higher order learning (HOL)? From: http://www2.warwick.ac.uk/services/cap/resources/pubs/eguides/eskills/guidelines/higher/ Originality, Creativity and Innovation Established Principles and Relationships Synthesis and Conceptualisation Organisation and Structure Arguments, Reasoning and Justification Techniques and Procedures Evaluation and Decisions Terminology, Language and Protocols Personal Interpretation and Meaning Facts, Assertions, Rules and Laws Formation and generation of Knowlege and practice of ADAPTIVE LEARNING Knowledge created = HOL ADOPTIVE LEARNING Knowledge transmitted
Base assessment tasks on using rather than replicating knowledge, e.g. assess through problem-solving which requires knowledge-based analysis and judgement
Assess in varied ways and include tasks (eg Modified Essay Questions - MEQs, projects) which require deep engagement and relate to real-life roles and competencies
Embed feedback and promote student understanding of how work is assessed Figure: Berry O’Donovan & Chris Rust ASKe Centre for Excellence in Teaching and Learning, (Assessment Standards Knowledge exchange) Oxford Brookes University
Evaluate the potential impact of portfolios on student learning
A systematic review of the use of portfolios (in Burns 2008)
Available evidence demonstrates that portfolios can support both the learning and assessment of general competencies ...
Success factors included: concurrent use in both formative promotion of learning and summative assessment; (mentor)
Summative assessment of the portfolio was important in ensuring portfolio learning maintained its status alongside other assessed material.
E-portfolios: flexibility (Woodward 2004) Digital portfolios: Fact or fashion?
‘ One of the inherent dangers with digital portfolios, for example, is that the technological novelty of the product could overshadow the purpose of the portfolio. The danger is that learning to use the technology itself could then subsume the learning opportunities of portfolio construction.’
‘ There is strong evidence from this research that digital portfolios need to be developed within a carefully designed framework in the same way the paper-based portfolios have been developed …’
E-portfolios: flexibility (Woodward 2004) Digital portfolios: Fact or fashion? Cont …
… the combination of text, audio, graphic and video based representation of information collectively termed ‘multimedia’ student engagement in learning
Ownership of author and user: ‘hyper-linked portfolios offer choice to their audience’ (p.230)
‘ Hartnell-Young and Morris, however, caution that a “multimedia portfolio is not expected to be a graphic designer’s dream, the emphasis should be on learning” (1999, p. 28).’
E-portfolios: comparison three universities (Wilhelm 2006 p.70)
‘ Implementing e-portfolios helps to develop a “culture of evidence” (Barrett & Wilkerson, 2004) for ongoing program improvement. E-portfolios are a useful assessment tool in this process.’
‘… reflection is a process of critically examining one’s present and past practices as a means of building one’s knowledge and understanding in order to improve practice.’
‘… a more comprehensive understanding of reflection was evident and many … reported that articulating their personal theory (often for the first time in their career) impacted positively on their ability to reflect on practice.’
The effectiveness of the use of a portfolio is enhanced by combining portfolio assessment with interview.
‘ a single-examiner portfolio interview focusing on standardised questions and a global rating scale is a feasible portfolio assessment method that can be used to assess clinical reasoning skills in an integrated, professionally authentic manner.’
Baume, D. (2001). A briefing on assessment of portfolios [Electronic Version]. Learning and Teaching Support Network Generic Series Assessment 6 from http://www.bioscience.heacademy.ac.uk/ftp/Resources/gc/assess06portfolios.pdf
Biggs, J. (2003). Teaching for Quality Learning at University. 2nd ed. Buckingham: The Society for Research into Higher Education & Open University Press.
Burch VC, Seggie JL. (2008) Use of a structured interview to assess portfolio-based learning. Medical Education, Vol. 42 :894-900
Jones, E. (2010). Personal theory and reflection in a professional practice portfolio [Electronic Version]. Assessment & Evaluation in Higher Education , 35, 699-710 from http://dx.doi.org/10.1080/02602930902977731
Klenowski, V., Askew, S., & Carnell, E. (2006). Portfolios for learning, assessment and professional development in higher education. Assessment & Evaluation in Higher Education, 31 (3), 267-286.
Mentowski, M. and Associates (2000). Learning that lasts: integrating learning development, and performance in college and beyond . San Francisco: Jossey-Bass
O’Donovan, B., Price, M., and Rust, C. (2004) Know what I mean? Enhancing student understanding of assessment standards and criteria. Teaching in Higher Education , Vol. 9, No. 3.
Rees, C. and Sheard, C. (2002) The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: the Nottingham experience. Medical Education, Vol. 38, No. 2: 138-144
Wilhelm, L., Puckett, K., Beisser, S., Wishart, W., Merideth, E., & Sivakumaran, T. (2006). Lessons Learned from the Implementation of Electronic Portfolios at Three Universities [Electronic Version]. Tech Trends , 50 from http://www.springerlink.com/content/l3412700x44l4752/fulltext.pdf
Woodward, H., & Nanlohy, P. (2004). Digital portfolios: fact or fashion? [Electronic Version]. Assessment & Evaluation in Higher Education , 29, 227-238 from http://dx.doi.org/10.1080/0260293042000188492
Links from LTSS: e-portfolios http://www.ucl.ac.uk/ltss-blog/?cat=74