During their course of study, medical science students are generally unaware that they are developing professional skills related to graduate capabilities. Interestingly, at a program level the institution finds it difficult to view the development of these capabilities. In this session we will discuss our own learning journey as discipline specific teachers who have worked collaboratively to implement ePortfolios and rubrics across courses and within the medical science degree program at UNSW Australia. Our approach to supporting student learning and development of reflective practice and professional skills in teamwork by cross-discipline alignment of assessment coupled with ePortfolio thinking and doing will be presented.

1. School of Medical Sciences
Patsie: Lead facilitator: ePortfolios pedagogy/rationale/skills in the Medical Sciences and beyond
Jia-Lin: Lead Career learning and ePortfolios, Data/statistics / spokesperson
Thuan: Project manager/ spokesperson - Aligning Assessment Project implementation across
disciplines, 2015
Annie: Project manager/ spokesperson - Fellowship implementation across programs, 2016
Richard, Trevor, Nicole, Cristan, Thomas, Julian, Sue:
Discipline Specific Experts: will lead table discussions and discuss their course and task specific
approaches
Stories and Strategies from UNSW Australia
Collaborative, Program-wide Alignment of Assessments and
ePortfolios to Build Teamwork Skills and Reflective Practice for
Medical Sciences Students:

2. ePortfolios Australia: 29/09/16 Workshop 2 Room 2174, 2-4:30pm
Title: Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build
Teamwork Skills and Reflective Practice for Medical Science Students: Stories and
Strategies from UNSW Australia
Themes: Working across disciplines, building teamwork skills, scaffolding reflective
practice, ePortfolios for future use beyond university , extracting data from ePortfolios
Schedule
• Workshop overview: 15 min
• Group discussion: 30 min
• Group presentation: 30 min (5 min per group/table)
• Discussion: Design, Development, Implementation (30 mins)
• Q & A (30 min)
• Summary (15 min)

3. AIMS: Workshop
• To engage colleagues in thinking about how to align
assessment tasks across programs and disciplines
• Implementation of ePortfolio use linked to assessment tasks
• Discuss the idea of ePortfolios, rubrics and data to recognise
attainment of skills

4. ePORTFOLIO / Reflective Blogging
Select
Collect
ReflectCurate
Present
Self appraisal
Gathering
occupational
information
Goal setting and
selection
Planning and
problem solving
Self
awareness
Decision
making
Opportunity
awareness
Transition
learning
Yang et al, 2014
Polly et al., 2015

5. UNSW – Institutional Priorities for Graduates
Global Citizens
Scholars
Leaders Professionals
UNSW, Strategic Intent, 2013
UNSW, Key Priorities, 2012-2016
UNSW Strategy, 2025
The ‘Why’
P. Polly et al, 2013
Understanding of
their discipline in its
Interdisciplinary context
Capable of
initiating and
embracing
innovation and
change
Capable of ethical, self-
directed practice and
independent lifelong learning
Graduates who are
culturally adept and
capable of respecting
diversity

6. 2:00-3:15pm
Workshop Part 1

7. 2:10-2:40pm
Group discussion: 30 min
• What is your understanding of a student ePortfolio?
• What is your experience in ePortfolio study and/or practice?
• What are 3 important issues in current ePortfolio study and
practice from your points of view?
• What are your 3 suggestions improving ePortfolio study and
practice in educational institutes in Australia?

8. 2:40-3:15pm (35 min, 5 min/table)
Group feedback
ePortfolio definition
Three important issues
Resolving issues

9. 3:15-3:30pm
Break

10. 3:30-4:30pm
Workshop Part 2

11. 3:30-4:00pm (30 min) What have we done in resolving
the issues? UNSW Stories and strategies
PATSIE: Overview (3-5mins)
PATSIE: Design:
Top down from school incorporating teamwork skill development and ePortfolio in curriculum of science program
PATSIE: Development
• Teamwork, Modified AAC&U Rubric
• ePorfolio, template questions, reflection rubric
JIA-LIN: Career development learning, CDMSE survey
Implementation:
THUAN (APP; 3mins): Moodle workshop, self-/peer assessment, WordPress
ANNIE (TF; 3mins): Moodle as an LMS, central data collection
JIA-LIN (3mins): Statistical analysis, ePortfolio reflection analysis (N Vivo)
Discipline Expert Stories: 3 mins each – suggestion, assessment strategy linked to ePortfolio slide, data slide
TREVOR: Neuro Physiology - 3rd Year
RICHARD: Neuro Physiology – 2nd Year
NICOLE: Pharmacology – 3rd Year
THOMAS: Anatomy - 3rd Year
CRISTAN: Pathology – 2nd Year
SUE: Medical Science – 1st Year

12. ISSUE and WHY we use ePortfolio
• Students find teamwork activities challenging
• Assessment tasks coupled with reflective practice are critical
for building professional skills such as teamwork
• Address graduate attributes to launch into professional skills
and career learning
• Capturing, tracking and extracting data from ePortfolios is a
challenge

13. Longitudinal,Program-wideuseofePortfolio
Professional
Readiness
Pathways
 Further studies
 Research
 Industry
Year 1
Year 2
Year 3
Year 3
Year 4
Career
Awareness
ePortfolio Use in the Medical Sciences
Cross-Discipline Approaches – WordPress 2015, Moodle 2016
Skills Building
Anatomy Pathology Pharmacology Physiology
Transverse, Cross-context and cross-discipline
UNSW L&T Innovation and Development Grant, Round 2 2014
Aligning assessment to track development of graduate capabilities, professional skills and life-long
learning in the biomedical sciences
UNSW Ethics Approval
HC15134
P. Polly et al, 2014
Identity
Narrative
Employability

14. Teacher Involvement – Medical Science Program
Aligning Assessments
P. Polly et al, 2014

15. ePortfolio: a high impact practice in higher education:
Statistical analysis and ePortfolio assessment
Jia Lin Yang
ePortfolio Forum Workshop
28 September 2016

16. ePortfolios impact on learning: Statistical analysis
• Scholarship practice (educational publications)
• Hypothesis testing and errors
• Descriptive and inferential statistics
• Selection of correct statistical analysis methods (Yang et al.
2010 eSTATS-CHOICE:
http://dev.adaptiveelearning.com/demos/DTv3/)

17. ePortfolio assessment-
https://teaching.unsw.edu.au/printpdf/535
• ePortfolio – digitised collection of artefacts (Lorenzo &
Ittelson, 2005)
• Software (eg. NVivo)
• Rubrics (Different focuses)
• Others (eg. Badging)

18. Rubric - https://www2.uwstout.edu/content/profdev/rubrics/eportfoliorubric.html
• Selection of Artefacts
• Descriptive Text
• Reflective Commentary
• Citations
• Navigation
• Usability and Accessibility: Text Elements, Layout, and
Colour
• Writing Conventions
• Multimedia Elements (Optional)
© COPYRIGHT 2001-2016 Joan Vandervelde
All Rights Reserved.
University of Wisconsin
Revised: Monday, January 11, 2016

19. NVivo - http://www.qsrinternational.com/nvivo-product
NVivo is software that supports qualitative and mixed methods
research. It’s designed to help you organize, analyze and find
insights in unstructured, or qualitative data like: interviews,
open-ended survey responses, articles, social media and web
content.
A systemic approach:
• Converting data into digital transcripts
• Coding/recoding
• Constant comparing
• Clustering
• Storing
• Facilitating the connection
• Pilot inductive and interpretive analysis
• Within/across cases
• Commonalities and exceptionalities

20. Badging
A badge is a visual representation of an accomplishment,
achievement or skill acquisition—more granular than a formal
degree, but helps to make incremental learning more visible.
(http://www.educause.edu/badging)
A digital badge is a validated indicator of accomplishment, skill,
quality, or interest that can be earned in many learning
environments. Open digital badging makes it easy for anyone
to issue, earn, and display badges across the web—through an
infrastructure that uses shared and open technical technical
standards.
(https://www.hastac.org/initiatives/digital-badges)

21. Career Development Learning
across Year 3 Science Courses
Jia Lin Yang
Research article: Jia-Lin Yang, Patsie Polly, Thomas Fath, Nicole Jones, John
Power.2016. Int J Science Math Tech Learning. 23(3), 33-46.
http://ijlsmtl.cgpublisher.com/product/pub.266/prod.111

22. Pilot study: Educational interventions on career
development learning (2012, PATH3208)
Yang et al. Int J Adult, Commun Prof Learning Vol 22 (1), 2015, pp 1-17.
Table 4: Results of CDSME Survey Pre- and Post the Study Intervention
Total (n=32) Female (n=21) Male (n=11)
mean SD p mean SD p mean SD p
Self-appraisal pre 18.1 3.1 0.014 18.3 2.6 0.019 17.8 3.9 0.240
post 19.8 2.6 19.8 2.0 19.7 3.6
Occupational
information
pre 15.9 2.8 0.001 16.2 2.4 0.002 15.4 3.5 0.122
post 18.2 3.1 18.7 3.0 17.3 3.1
Goal selection pre 18.4 3.1 0.507 18.9 3.4 0.794 17.6 2.5 0.438
post 18.8 2.6 19.1 2.4 18.3 3.0
Planning pre 17.0 2.9 0.002 17.8 2.7 0.048 15.4 2.6 0.009
post 19.2 3.1 19.3 3.0 18.9 3.4
Problem
solving
pre 17.1 3.0 0.016 17.6 3.4 0.162 16.1 1.9 0.024
post 18.3 2.5 18.5 2.8 18.1 1.9
Source(s): Yang, Coleman, Das and Hawkins, 2013

23. Aim
1. To enable more students in Stage 3 courses at the School of
Medical Sciences to be engaged in a career development
learning process in order to develop career awareness,
employability and professional skills.
2. To perform a controlled study in large sample populations.
1. There would be no improvement in CDMSE scales within
individual courses, pre- and post-course.
2. There would be no difference in variation of CDMSE
scales in CDL intervention or control group.
Null Hypotheses

24. CDL intervention
CDL
tutorials
personal/group
counselling
Professional
knowledge and
skills
CDMSE
survey
CDL assessment
Career and
reflective
ePortfolio
Learning centred
ePortfolio

25. CDMSE Survey
 Gathering occupational information: 1, 10, 15, 19, 23
 Goal selection: 2, 6, 11, 16, 20
 Planning: 3, 7, 12, 21, 24
 Self-appraisal: 5, 9, 14, 18, 22
 Problem solving: 4, 8, 13, 17, 25
(Betz & Taylor, 1983; Scott & Ciani, 2008)

26. 2013 Project Design
Pre/post-course
CDMSE Survey
CDL
intervention
(n=135)
ANAT3212
(n=11)
PATH3208
(n=37)
PHAR3202
(n=35)
PHAR3101
(n=52)
Non-CDL
intervention
control
NEOR3221
(n=30)

27. Results 1: CDMSE scales within individual courses
0 5 10 15 20 25
occupational info
goal selection
planning
self appraisal
problem solving
ANAT3212 (n=11)
post-course pre-course
0 5 10 15 20 25 30
occupational info
goal selection
planning
self appraisal
problem solving
PATH3208 (n=37)
post-course pre-course
0 5 10 15 20 25
occupational info
goal selection
planning
self appraisal
problem solving
PHAR3101 (n=52)
post-course pre-course
0 5 10 15 20 25 30
occupational info
goal selection
planning
self appraisal
problem solving
PHAR3202 (n=35)
post-course pre-course
0 5 10 15 20 25
occupational info
goal selection
planning
self appraisal
problem solving
NUER3221 (n=30)
post-course pre-course
Control group
* p<0.05, ** p<0.01
**

28. Result 2: CDMSE scales obtained pre- and post-course
for CDL intervention (n=135) and control (n=30) groups
**
* p<0.05, ** p<0.01

29. Conclusion
• The introduction of this pedagogy and model has
proved successful in evidencing graduate learning
outcomes for improving student confidence in
CDMSE.
• This study supports further extension of this approach
to senior undergraduate courses at UNSW and other
universities.
• Using ePortfolio to deliver CDL will result in directive
and structural reflection by learners.

30. 4:00- 4:20pm Q & A
workshop: leaders and facilitators

31. 4:20-4:30pm
Patsie: Summary of Workshop Outcomes

32. Acknowledgements and Outcomes
UNSW Australia funding
• 2015 : SEF#4 1x LNT Innovation and Development
• 2016 : SEF#3 UNSW Teaching Fellowship
Presentations
• 2015 Yang et al, MedEd Forum, Faculty of Medicine, Dec
• 2015 Yang, Jia-Lin, and Patsie Polly. ‘Career development learning across year 3 science courses – Pedagogy and
curriculum: Improving student self-efficacy in employability skills’ 22nd Learners Conference, University of St Paulo,
Madrid.
• 2016 Polly et al, AAEEBL International ePortfolio Conference, Boston, US, Jul
• 2016 Polly et al, ePIC, International ePortfolio and Identity Conference, Invited Panelist, Invited Speaker, Bologna, Oct
Published Journal
• Naumann…Polly et al, (2016). Virtual Patient Consultations and the use of ePortfolios to support student learning of
integrated professional skills. (Focus on Health Professional Education, under final review).
• J.L. Yang, P. Polly, T. Fath, N. Jones, J. Power (2016). ISA Model and Integrative Career Development Learning in Year 3
Science Courses. The International Journal of Science, Mathematics and Technology Learning (Vol 23, Issue, 3, p33-46).
Book Chapters
• P. Polly, et al (2016). ePortfolios, Assessment and Professional Skills in the Medical Sciences at UNSW Australia. In
ePortfolios in Australian Universities. Springer.
• P. Polly, et al (2015). Creative teaching, learning and assessment in Medical Science: ePortfolios to support skills
development in scientists beyond just knowing their own discipline content. Capturing Creativity through Creative
teaching. The Learner Series.