3. 1.2 There are robust internal
processes for design and approval
of the course of study which take
account of external standards
and requirements, e.g. published
discipline standards …, and
comparable standards at other
higher education providers.
Scott Ray 4/4/09 http://www.flickr.com/photos/15075617@N05/3418528723
6. 5.5 The academic
standards intended to be
achieved by students and
the outcomes actually
achieved by students in
the course of study are
benchmarked against
similar accredited courses
of study.”
Martin Morris 20/2/12 http://www.flickr.com/photos/martin55/6929721749/
“external
AoL”
8. Calibration – Task validity
Written communication
91%
Individual confidence pre-workshop
Individual results pre-workshop
• Min & max (n=26)
NA
A
• Mean 1 SD
Group results at workshop
• Small groups (n=5)
• Consensus
NA
A
9. Calibration – UG student 1
....S2, S3,
Written communication S4, S5
78%
Individual confidence pre-workshop
Individual results pre-workshop
• Min & max (n=26)
• Mean 1 SD
NM
M
NM
M
Group results at workshop
• Small groups (n=5)
• Consensus
10. Provider selects
reviewers
Project selects
standards &
degree for
review
Provider uses
results
(Project ensures
consensus)
3 anonymous
reviewers submit
(Project allocates
2 externals)
External
calibration &
review
Provider selects
tasks
Project generates
random student
ID to submit
Reviewers
calibrate
Provider submits
de-identified data
13. Your uni predicts that the BSc (BioMed)
is likely to be selected by TEQSA as a
sample degree for scrutiny in 2015. How
would you demonstrate external AOL?
1. Rely on sample data reviewed as part of QVS
2. Select benchmarking partner dept to assess sample
assignments & exams in final UoS for 5% students in each
grade band
3. Pay an external academic to assess
4. Lobby ACDS to establish a national ‘double blind peer
review’ benchmarking process that includes calibration
5. Lobby CUBEnet to do DBPR benchmarking & calibration
6. Do nothing
14. Achievement Matters: External Peer Review of Accounting
Learning Standards
Australian Business Deans Council
Institute of Chartered Accountants in Australia
CPA Australia
Office for Learning and Teaching
Project team : Adelaide (B.Howieson), Deakin (K.Watty), RMIT
(B.O’Connell, P.de Lange), Sydney (M.Freeman), UWA (P.
Hancock), UWS (A.Abraham)
Participants from 17 providers
Websites
achievementmatters.com.au
disciplinestandards.pbworks.com
Disclaimer
The views expressed in this presentation do not
necessarily reflect the views of any of these stakeholders
In order to play in many fields you need to meet a minimum standard.For example: Womens high jump introduced in 1928 Olympics. – Ethel Catherwood (Canada) jumped 1.59m (5 ft 2 inches). But 2012 Olympics qualification standard was 1.96m (6ft 5inches – set by Intern Assoc of Athletics Federation). Like deregulation of banking sector required APRA’s role to be strengthened, moving to demand-driven HE system has meant a new & stronger regulator. TEQSA (on video) has invited us to take responsibility for our future by setting and assessing academics standards Outline for today – we will cover some Background (including process)State of playLearningsNext steps
15:28-22:3042:13-
External – AQF and Discipline standards
2010Interprets AQF ie. eg. develop TLO for accounting bachelors (lev 7) and coursework masters (lev 9)degreesBuilds on previous program focus eg. AACSB assurance of learning where program LO matter (not unit of study LO)Highly consultative - Meet with multiple levels of distributed leaders. Multiple cycles. In total 2,100 participants nationally, representing 38 Australian universities, 21 private/other providers, 20 others (eg professional and peak bodies)Five LS - Fewer thresholds than any other discipline. Acctngdepts encouraged to diversely build (eg. Internationalisation)Resulted in jointly-funded initiative to assess learning outcomes benchmarked against standardsResulted in other business disciplines setting standards using same approach (eg. Marketing, econs)
2010Interprets AQF ie. eg. develop TLO for accounting bachelors (lev 7) and coursework masters (lev 9)degreesBuilds on previous program focus eg. AACSB assurance of learning where program LO matter (not unit of study LO)Highly consultative - Meet with multiple levels of distributed leaders. Multiple cycles. In total 2,100 participants nationally, representing 38 Australian universities, 21 private/other providers, 20 others (eg professional and peak bodies)Five LS - Fewer thresholds than any other discipline. Acctngdepts encouraged to diversely build (eg. Internationalisation)Resulted in jointly-funded initiative to assess learning outcomes benchmarked against standardsResulted in other business disciplines setting standards using same approach (eg. Marketing, econs)
HE is quite unlike other top export industries (iron ore, coal and gold), since the quality or standard of HE can not be measured exactly. Hence the importance for TEQSA to monitor and protect “Brand Australia”.HE is not like high jump where the standard can be measured exactly. It is more like diving where experienced judges make independent decisions. In fact this mirrors what is valued in vetting a manuscript for publication – it is double-blind peer reviewed.
“I am confident rating the capacity of assessment requirements to allow students to demonstrate the national learning standard for oral communication in accounting”Indiv: 64707263938758799287757479477669635753708280777790 Av=73.36Stdev=12.17881Teams:3643413945Consensus: 40
“I am confident rating students' written communication ability benchmarked against the national standard”Indiv: 64707263938758799287757479477669635753708280777790 Av=73.36Stdev=12.17881Teams:3643413945Consensus: 40