Measuring Graduate Attributes - Lit Review for Tracker

1. How to measure graduate attribute development – A literature Review & Scope
Following our meeting with Dr. Moira Maguire, Head of Teaching @CELT, we completed a
literature review and scope to highlight issues pertinent to the measurement of graduate
attribute development.
Some of the issues highlighted in that literature scope, similar to what our discussion attended
were:
• A measurement tool needs to be specific enough to reliably measure the graduate
attributes it purports to.
• A measurement tool is only of use if it is informative for benchmarking, validation and
recommending improvements. In other words, it needs to be simple enough to be
comprehensible to its users but have enough teeth to provide opportunities to
authentically validate graduate attribute development.
• Tensions between developing and measuring subject-specific graduate attributes and
generic graduate attributes for the workplace are inevitable, but many authors have
worked out ways to resolve many of these contradictions.
• We need to actively encourage curriculum revision for improvement and counter
notions of intrusion or watering down because rejections of any incursions by generic
skills into programme content tend to be protective of time spent on basic skills, e.g.,
literacy and numeracy.
Graduate Attribute Development: Creating a Reliable and Valid Measurement Tool
What should a measurement tool attend to when informing benchmarking, validation and
improvements:
1. Specificity – Any graduate attribute measurement tool must be specific enough to
measure graduate attributes deemed relevant. We need to be clear that there will be
some overlap between subject-specific evaluation and institution-led evaluation, but
that they are separate and complementary exercises. We must communicate that
graduate attribute development is extra work – adds value and puts an institution into
or maintains an institution’s competitive position in a labour market.

2. 2. Composite Scores – additive on a scale of multiple items to draw broader conclusions
about meaningful generic skills development. The DkIT PCs Framework follows this
model.
3. Incremental approaches, e.g., measuring annually from students’ perspective. Self-
assessment by students is key. There are scales/inventories in the literature that capture
student insight relevant to their graduate attribute development, both specific to their
degree programme and generally to their university experience.
4. Thresholds can be used to identify, map and monitor graduate attribute development
via multiple thresholds. For example, compositive scores for each sub-attribute per
capstone attribute category.
5. Self-assessment by students will counterbalance any strategies conceived by teaching
strategy to slip into a box-ticking exercise. Key are the questions we ask.
6. Multi-perspective measurement – employers, students and staff feedback are optimal.
Awareness Raising – Strategic Integration
• Graduate Attribute Development must represent a key part of the institute's goals and
development agenda. Our Programme Design & Validation policy category for the DkIT
Employability Statement & Graduate Attribute Framework emphasises the achievement
of this goal.
• Successful transition to work and managing continued employability in a rapidly
changing occupational world is the aim of graduate attributes, be they subject specific
or workplace general.
• Rapidly evolving information technology significantly influences the capabilities
expected from graduates in pursuit of careers, and graduate attribute development will
facilitate graduate’s adaptability to this new world of work. Awareness across and
between all stakeholders to support the development of graduate attributes, their
measurement and their relevance within an environment of evolving graduate attribute
priority is key.
• That said, no matter how supportive services or management might be for a new
initiative that aligns with an existing institutional agenda, unless the project team is able
to generate support at all levels of the university its implementation simply will not
happen.
• Heightening awareness between stakeholders is key, so that communicating the results
of a measurement tool, or suite of complementary tools across stakeholders is
perceived to have utility.
• Ambiguity of the terminology used to discuss generic skills at the programme specific
level and the institute-led graduate attribute level will require a strategy and raising
awareness agenda. Terminology varies within and between disciplines and varies among
institutional stakeholders. Employers and academics tend to talk past on another in

3. endless debates about appropriate language. Likely to feature in a workshop we will
create to introduce stakeholders to embedding employability via the lens of graduate
attributes.
Operationalising Graduate Attribute Measurement:
Multi-Item Inventory Scale implemented via Excel Tracker:
• Separate Measurement Tool or Subscales by Audience
• Separate Measurement Tool by Levels of Relevance
1. TOOL 1: Our Tool(s) – 2 Audiences (Staff and Students) and 4 Levels of Relevance
(Module, Programme, School, and Institute)
2. TOOL 2: Employability 10-dot Matrix is separate re Embedding Employability only. It
may be used for an Activity, Assessment, Module or Programme.
Bibliography
Barrie, S.C. (2004). A research-based approach to generic graduate attributes policy. Higher
Education Research & Development, 23(3), 261–275.
Coetzee, M. (2012). A framework for developing student graduateness and employability in the
economic and management sciences at the University of South Africa. In M. Coetzee, J. Botha,
N. Eccles, N. Holtzhausen, & H. Nienaber (Eds.), Developing student graduateness and
employability: Issues, provocations, theory and practical guidelines (pp. 119–152). Randburg:
Knowres.
Coetzee, M. (2014). Measuring student graduateness: Reliability and construct validity of the
Graduate Skills & Attribute Scale. Higher Education Research & Development, 33(5): 887-902.
Coetzee, M., & Schreuder, A.M.G. (2011). The relation between career anchors, emotional
intelligence and employability satisfaction among workers in the service industry. Southern
African Business Review, 15(3), 76–97.
Coetzee, M., & Schreuder, A.M.G. (2012, July). Employees’ graduate and employability
attributes in relation to their emotional intelligence and career anchors. Poster presented at the
30th international congress of psychology, Cape Town, South Africa.
Crebert, R.G. (2000, July 17-19). A snapshot of generic skills development at Griffith University.
Paper presented at the inaugural international lifelong learning conference, Yeppoon,
Queensland, Australia; under the auspices of Central Queensland University.

4. DkIT’s Employability Statement & Framework (2021). [Retrieved online 1st
Dec, 2021: DkIT-
Employability-Statement.pdf]
Drummond, I., Nixon, I., & Wiltshire, J. (1998). Personal transferable skills in higher education:
The problems of implementing good practice. Quality Assurance in Education, 6 (1), 119-27.
Ipperciel, D., & ElAtia, S. (2014). Assessing Graduate Attributes: Building Criteria-Based
Composite Model. International Journal of Higher Education, 3(3): 27-38.
Lipan, G. I., Irwin, A., Nordman, E., Perkins, J. (In press). The 7i Model: A Common Language for
Graduate Attributes. PsyArXiv Preprints. [Retrieved Online 29th
Dec, 2021: PsyArXiv Preprints |
The 7i model: A common language for graduate attributes]
Nelson, N. & Brennan, R. (2018). Graduate Attributes and Threshold Concepts: Are We
Measuring What’s Important? Proceedings of the Canadian Society of Mechanical Engineering
International Congress, CSME 2018. [Retrieved Online 26th
Dec 2021: Paper Title (use style:
paper title) (yorku.ca)].
Thompson, D., Treleaven, L., Kamvounias, P., Beem, B., & Hill, E. (2008). Integrating graduate
attributes with assessment criteria in business education: Using an online assessment system.
Journal of University Teaching & Learning Practice, 5(1), 1–14.
Yorke, M., & Knight, P. (2007). Evidence-informed pedagogy and the enhancement of student
employability. Teaching in Higher Education, 12(2), 157–170.