Overview of ePortfolios
Simon Cotterill
Learning Technologies for Medical Sciences
School of Medical Sciences Education Development
Objectives

   To provide/discuss a general overview:

   •   What are portfolios?
   •   Background/ Drivers
   •   Engagement / Embedding
   •   Brief overview of current Newcastle University context
What are Portfolios ?
   “..a purposeful collection of student work that
   exhibits to the student, or others, [their] efforts or
   achievement in one or more areas.” Arter and
   Spandel (1991)


   The Journey – the process of completing the
                 portfolio as a learning experience

   The end-point – the evidence of learning /
                  competencies

      Artefacts (eg. essays, logs of experiences, artwork, videos,
           audits, records of prior-learning/ qualifications).
Diversity of Portfolios
        Unstructured          Structured

           Formative          Summative

Factual / Quantitative        Reflective / Interpretive

              Sample          All work

           Best work          Representative

     Learner Owned            Employer Owned

             Episodic         Life-long
ePortfolios: defined by their purpose(s)
                                                       Portfolio for
                                                       Presentation
   Portfolio for        Sharing / dialogue
   Assessment
                                                          Portfolio for
                                                           Application
 Portfolio for               Learner’s                   (job / promotion)
Accreditation/              ‘repository’
 Revalidation


                            Institutional
                                Data

                                                          Portfolio for
                                                          Appraisal
  Central data:
     Transcript
     MIS/ HR data                                       PDP
                                                        (shared)
  Programme data:
                                             PDP / Reflective
     Granular assessment data
                                             (private)
     Outcomes / skills sets
ePortfolios at Newcastle: >10 yr experience

                                     FDTL-4
   JISC ePET project
                                     Project

                                                          FDTL          T&L Fund Projects x 2
                          Mini project                Transferability
JISC EPICS regional                                      Project
                          (Bioscience)                                  Postgraduate Research
    ePortfolio project
                                                                        Training ePortfolio
                          Mini Projects
                          (Medicine)
                                                                        Teaching Fellowship
                                                                        ePortfolio for appraisal
                                                         Mobile
                         Hosted           Dental         portfolio
                         services        ePortfolio      projects       Wide uptake in other
JISC EPICS-2 regional                                                   undergraduate courses
    ePortfolio project
Aims of our FDTL-4 ePortfolio project (2002)
• To develop an on-line portfolio system to support a reflective
  approach to evidencing the attainment of programme
  outcomes.

• To promote the development of the reflective capabilities of
  medical students, giving greater responsibility for managing
  their own learning and preparing for aspects of work-based
  and lifelong learning.

• To develop strategies to facilitate assessment of curriculum
  outcomes that are not amenable to traditional instruments of
  assessment.
Personal
          Development Planning (PDP)

 ‘a means by which students can monitor, build and
reflect upon their personal development’.
Dearing Review (NCIHE 1997)


 ‘a structured and supported process undertaken by
an individual to reflect upon their own
learning, performance and/or achievement and to plan
for their personal, educational and career development’.
(QAA, 2001)

Links with other agendas: employability, widening participation
How much structure?

• Intrinsic structure
   – „scaffolding‟
   – structure aids assessment
       • structured skill sets, outcomes and competencies
   – but too much structure
       • perceived as ‘form filling’
       • risks lower engagement

• Extrinsic structure
   – guidelines / assessment criteria
   – guidance / direction from tutors & mentors
Changing habits and expectations…

Web 2.0
• Interactive / participative
• Social / conversational
• Simple – unstructured

Our response:
• Unstructured Blog but with explicit links to skills/outcomes

    Cotterill SJ et al. Beyond the Blog: getting the right level of structure
    in an ePortfolio to support learning. Proc. ePortfolios 2007

Mobile / Wireless Technologies
• Our early pilots of PDAs for logbooks problematic
• Successful use in Dentistry – assessment at the point of learning
Factors related to engagement with ePortfolio / blog
Questionnaire results n=163 (30% response rate, 3 subjects – non-medical)


                                                                                EPICS-2 Project
                                                                                http://www.epics.ac.uk



                                                                                    Group 1      Group 2      Group 3

                                                                                     High        Medium       Modest
                                                                                  Engagement   Engagement   Engagement

I have a clear understanding of the purpose of the ePortfolio                        87%          74%          58%

I have a clear understanding of how the ePortfolio is used in my programme           91%          76%          42%

I received adequate information on how to use the ePortfolio                         58%          50%          58%

I had a clear understanding of the skills being evidenced in the ePortfolio          66%          65%          17%

The skills included in the ePortfolio are important in studying for my degree        69%          94%          25%

The skills included in the ePortfolio are important for my longer-term career        59%          94%          33%

The ePortfolio is important for my programme                                         75%          82%          33%

Course handbooks and study guides refer to the ePortfolio                            58%          65%          25%

Teaching staff regularly refer to the ePortfolio                                     84%          41%          33%
Embedding in the curriculum = better engagement
and effective use of ePortfolios
  Driessen E et al. Portfolios in medical education: why do they meet
  with mixed success? A systematic review. Med Educ. 2007;41:1224-
  33.

  Atlay, M. Embedding PDP practice in the curriculum, in Personal
  Development Planning and Employability, Higher Education
  Academy, York. 2006




… but, what do we mean by embedding?
Atlay (2009) classified ways in which PDP can be
             associated with curricula:
Discrete     ‘bolt on’ – optional and additional to    + Low cost
             the curriculum                            - Risk low engagement


Linked       Run in parallel to the curriculum but     + No disruption to curric.
             linked to it                              - Risks low engagement

Embedded     Included in specific module(s)            + Better consistency of
                                                         experience
                                                       - Risks being fragmented

Integrated   Underpins and used throughout the         + Becomes integrated part
             curriculum                                  of staff/student thinking
                                                       - Difficult to achieve!

Extended     Integrates activities in the curriculum   + Draws together study,
             along with wider life experiences           work & life experiences
                                                       - Unpredictable outcomes
Project 2012 – University portfolio
Governance: University ePortfolio Steering Group


      2011/12                   2012/13                   2013/14
Student evaluation         Students will have        Students will have the
                           access to an ePortfolio   opportunity to use their
                           to support personal       ePortfolio in meetings
Consultations with staff
                           reflection and            with their Personal Tutor
                           development of            to support the student in
Jan 2012: enhanced         employability skills      achieving personal,
ePortfolio live                                      academic and
Semester 2 2011/12                                   professional goals
pilots of ePortfolio use
Opportunities
Sharing and community features
• Additional information for tutors + recording of meetings
• Channel for additional feedback
                                                              }   opportunity
                                                                  & threat!
• Could help support seminar group / peer mentoring

Integration with other systems
•   Integration with TSE/LSE e.g. SSC supervisors
•   Viewing released component marks + integrated reflect/plan or feedback
•   Feeding undergraduate portfolio information on into Foundation portfolio
•   Portfolio linked to curriculum map (Learning Maps)

Feeding information into the portfolio
• Longstanding example: PGR workshop attendance records are added to CV
• Could feed in assessment and feedback from diverse activities
   • e.g. Phase 1 ethics: peer assessment / peer feedback
   • e.g. Facility for staff to add adhoc feedback
Further information                                                Simon.Cotterill@ncl.ac.uk

Our Information
Portfolio Service: https://portfolio.ncl.ac.uk (or access via TSE/LSE or Blackboard)

Our publications: http://www.eportfolios.ac.uk/docs
Over 15 papers, chapters and conference papers e.g.

Cotterill SJ, Bradley PM, Stacy R. Using ePortfolios to support annual appraisal in undergraduate medicine.
Proc. ePortfolios, identity and personalised learning in healthcare education. 2008 Newcastle

Cotterill SJ, McDonald AM, Hammond GR, Bradley PM. ePortfolio to support planning, learning and
summative assessment for student selected components in Medicine: 4 years experience. . Proc. ePortfolios,
identity and personalised learning in healthcare education. 2008 Newcastle

Cotterill SJ, Aiton J, Bradley PM, Hammond GR, McDonald AM, Struthers J, Whiten S. A flexible component-
based ePortfolio: adapting and embedding in the curriculum. In: In Jafari A, Kaufman C, ed. Handbook of
Research on ePortfolios. Pennsylvania: Idea Group Inc, 2006.


External Sources
      JISC ePortfolios Toolkit: http://www.jiscinfonet.ac.uk/e-portfolios

                         Centre for Recording Achievement
                                    http://www.recordingachievement.org/
10 ‘value added’ features
of an online approach to portfolios
•   Sharable / facilitate interaction
•   Transportable - supporting continuity in LLL
•   Highly customisable
•   Multi-purpose eg. formative & summative = reduced duplication
•   Multiple structures / views
•   Easier cross-referencing
•   Searchable
•   Reduced admin
•   Secure access from a range of locations
•   Not left on the bus !

ePortfolios overview

  • 1.
    Overview of ePortfolios SimonCotterill Learning Technologies for Medical Sciences School of Medical Sciences Education Development
  • 2.
    Objectives To provide/discuss a general overview: • What are portfolios? • Background/ Drivers • Engagement / Embedding • Brief overview of current Newcastle University context
  • 3.
    What are Portfolios? “..a purposeful collection of student work that exhibits to the student, or others, [their] efforts or achievement in one or more areas.” Arter and Spandel (1991) The Journey – the process of completing the portfolio as a learning experience The end-point – the evidence of learning / competencies Artefacts (eg. essays, logs of experiences, artwork, videos, audits, records of prior-learning/ qualifications).
  • 4.
    Diversity of Portfolios Unstructured Structured Formative Summative Factual / Quantitative Reflective / Interpretive Sample All work Best work Representative Learner Owned Employer Owned Episodic Life-long
  • 5.
    ePortfolios: defined bytheir purpose(s) Portfolio for Presentation Portfolio for Sharing / dialogue Assessment Portfolio for Application Portfolio for Learner’s (job / promotion) Accreditation/ ‘repository’ Revalidation Institutional Data Portfolio for Appraisal Central data: Transcript MIS/ HR data PDP (shared) Programme data: PDP / Reflective Granular assessment data (private) Outcomes / skills sets
  • 6.
    ePortfolios at Newcastle:>10 yr experience FDTL-4 JISC ePET project Project FDTL T&L Fund Projects x 2 Mini project Transferability JISC EPICS regional Project (Bioscience) Postgraduate Research ePortfolio project Training ePortfolio Mini Projects (Medicine) Teaching Fellowship ePortfolio for appraisal Mobile Hosted Dental portfolio services ePortfolio projects Wide uptake in other JISC EPICS-2 regional undergraduate courses ePortfolio project
  • 7.
    Aims of ourFDTL-4 ePortfolio project (2002) • To develop an on-line portfolio system to support a reflective approach to evidencing the attainment of programme outcomes. • To promote the development of the reflective capabilities of medical students, giving greater responsibility for managing their own learning and preparing for aspects of work-based and lifelong learning. • To develop strategies to facilitate assessment of curriculum outcomes that are not amenable to traditional instruments of assessment.
  • 8.
    Personal Development Planning (PDP)  ‘a means by which students can monitor, build and reflect upon their personal development’. Dearing Review (NCIHE 1997)  ‘a structured and supported process undertaken by an individual to reflect upon their own learning, performance and/or achievement and to plan for their personal, educational and career development’. (QAA, 2001) Links with other agendas: employability, widening participation
  • 9.
    How much structure? •Intrinsic structure – „scaffolding‟ – structure aids assessment • structured skill sets, outcomes and competencies – but too much structure • perceived as ‘form filling’ • risks lower engagement • Extrinsic structure – guidelines / assessment criteria – guidance / direction from tutors & mentors
  • 10.
    Changing habits andexpectations… Web 2.0 • Interactive / participative • Social / conversational • Simple – unstructured Our response: • Unstructured Blog but with explicit links to skills/outcomes Cotterill SJ et al. Beyond the Blog: getting the right level of structure in an ePortfolio to support learning. Proc. ePortfolios 2007 Mobile / Wireless Technologies • Our early pilots of PDAs for logbooks problematic • Successful use in Dentistry – assessment at the point of learning
  • 11.
    Factors related toengagement with ePortfolio / blog Questionnaire results n=163 (30% response rate, 3 subjects – non-medical) EPICS-2 Project http://www.epics.ac.uk Group 1 Group 2 Group 3 High Medium Modest Engagement Engagement Engagement I have a clear understanding of the purpose of the ePortfolio 87% 74% 58% I have a clear understanding of how the ePortfolio is used in my programme 91% 76% 42% I received adequate information on how to use the ePortfolio 58% 50% 58% I had a clear understanding of the skills being evidenced in the ePortfolio 66% 65% 17% The skills included in the ePortfolio are important in studying for my degree 69% 94% 25% The skills included in the ePortfolio are important for my longer-term career 59% 94% 33% The ePortfolio is important for my programme 75% 82% 33% Course handbooks and study guides refer to the ePortfolio 58% 65% 25% Teaching staff regularly refer to the ePortfolio 84% 41% 33%
  • 12.
    Embedding in thecurriculum = better engagement and effective use of ePortfolios Driessen E et al. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ. 2007;41:1224- 33. Atlay, M. Embedding PDP practice in the curriculum, in Personal Development Planning and Employability, Higher Education Academy, York. 2006 … but, what do we mean by embedding?
  • 13.
    Atlay (2009) classifiedways in which PDP can be associated with curricula: Discrete ‘bolt on’ – optional and additional to + Low cost the curriculum - Risk low engagement Linked Run in parallel to the curriculum but + No disruption to curric. linked to it - Risks low engagement Embedded Included in specific module(s) + Better consistency of experience - Risks being fragmented Integrated Underpins and used throughout the + Becomes integrated part curriculum of staff/student thinking - Difficult to achieve! Extended Integrates activities in the curriculum + Draws together study, along with wider life experiences work & life experiences - Unpredictable outcomes
  • 14.
    Project 2012 –University portfolio Governance: University ePortfolio Steering Group 2011/12 2012/13 2013/14 Student evaluation Students will have Students will have the access to an ePortfolio opportunity to use their to support personal ePortfolio in meetings Consultations with staff reflection and with their Personal Tutor development of to support the student in Jan 2012: enhanced employability skills achieving personal, ePortfolio live academic and Semester 2 2011/12 professional goals pilots of ePortfolio use
  • 15.
    Opportunities Sharing and communityfeatures • Additional information for tutors + recording of meetings • Channel for additional feedback } opportunity & threat! • Could help support seminar group / peer mentoring Integration with other systems • Integration with TSE/LSE e.g. SSC supervisors • Viewing released component marks + integrated reflect/plan or feedback • Feeding undergraduate portfolio information on into Foundation portfolio • Portfolio linked to curriculum map (Learning Maps) Feeding information into the portfolio • Longstanding example: PGR workshop attendance records are added to CV • Could feed in assessment and feedback from diverse activities • e.g. Phase 1 ethics: peer assessment / peer feedback • e.g. Facility for staff to add adhoc feedback
  • 16.
    Further information Simon.Cotterill@ncl.ac.uk Our Information Portfolio Service: https://portfolio.ncl.ac.uk (or access via TSE/LSE or Blackboard) Our publications: http://www.eportfolios.ac.uk/docs Over 15 papers, chapters and conference papers e.g. Cotterill SJ, Bradley PM, Stacy R. Using ePortfolios to support annual appraisal in undergraduate medicine. Proc. ePortfolios, identity and personalised learning in healthcare education. 2008 Newcastle Cotterill SJ, McDonald AM, Hammond GR, Bradley PM. ePortfolio to support planning, learning and summative assessment for student selected components in Medicine: 4 years experience. . Proc. ePortfolios, identity and personalised learning in healthcare education. 2008 Newcastle Cotterill SJ, Aiton J, Bradley PM, Hammond GR, McDonald AM, Struthers J, Whiten S. A flexible component- based ePortfolio: adapting and embedding in the curriculum. In: In Jafari A, Kaufman C, ed. Handbook of Research on ePortfolios. Pennsylvania: Idea Group Inc, 2006. External Sources JISC ePortfolios Toolkit: http://www.jiscinfonet.ac.uk/e-portfolios Centre for Recording Achievement http://www.recordingachievement.org/
  • 17.
    10 ‘value added’features of an online approach to portfolios • Sharable / facilitate interaction • Transportable - supporting continuity in LLL • Highly customisable • Multi-purpose eg. formative & summative = reduced duplication • Multiple structures / views • Easier cross-referencing • Searchable • Reduced admin • Secure access from a range of locations • Not left on the bus !