CDE conference 9 February 2009 Assessment and evaluation in UG Laws: new directions and old challenges

Assessment and evaluation in UG Laws:

new directions and old challenges

London LLB/Dip ‘classic’ picture Long standing examination Board Traditional assessment – unseen 3 hour examinations Large syllabi Very little guidance from London Teaching conducted by third party, commercial institutions Difficulty of communication

Long standing examination Board

Traditional assessment – unseen 3 hour examinations

Large syllabi

Very little guidance from London

Teaching conducted by third party, commercial institutions

Difficulty of communication

Process of redvelopment in to F&DL from 2001 Ist stage: increase learning resources cultural change both in EISA and Laws community to embrace ‘distance learning’ as governing description Involvement of outside consultants Work with private institutions, i.e. visit, lecture, attend convocations

Ist stage:

increase learning resources

cultural change both in EISA and Laws community to embrace ‘distance learning’ as governing description

Involvement of outside consultants

Work with private institutions, i.e. visit, lecture, attend convocations

Overall constraint ‘… that as far as possible practices on external programmes should reflect those in place for the internal programme’ But what if the internal programme carries particular conservative practices? Most assessment in the Laws Schools is by unseen 3 hour examinations Marking and progression is ‘by part’ and not modular Have to pass ‘all’ the examinations taken in one go to progress Not at all dl practices

‘… that as far as possible practices on external programmes should reflect those in place for the internal programme’

But what if the internal programme carries particular conservative practices?

Most assessment in the Laws Schools is by unseen 3 hour examinations

Marking and progression is ‘by part’ and not modular

Have to pass ‘all’ the examinations taken in one go to progress

Not at all dl practices

numbers Now c. 14,500 students 2007 marked 36,000 examination scripts All double marked i.e. 72,000 markings c. 5000 candidates in 1 st year Only 50% of year 1 go to year 2 Only 60 % of year 2 go to year 3

Now c. 14,500 students

2007 marked 36,000 examination scripts

All double marked i.e. 72,000 markings

c. 5000 candidates in 1 st year

Only 50% of year 1 go to year 2

Only 60 % of year 2 go to year 3

Institutions ‘teaching’ to examination papers Materials have developed BUT are they used? Still ‘classic assessment outcomes, e.g. Answers that look like from script Many similar answers (including same typos and grammatical mistakes Are we assessing the student or the tutor? The ‘jury question’ paradigm.

Materials have developed BUT are they used? Still ‘classic assessment outcomes, e.g.

Answers that look like from script

Many similar answers (including same typos and grammatical mistakes

Are we assessing the student or the tutor?

The ‘jury question’ paradigm.

2005 onwards radical re-appriasal In part due to external pressures (JASB revalidation) review of assessment to see Fit with minimum levels of learning materials for QLD degree Fit with ‘skills agenda’, both subject specific and transferable

In part due to external pressures (JASB revalidation) review of assessment to see

Fit with minimum levels of learning materials for QLD degree

Fit with ‘skills agenda’, both subject specific and transferable

outcomes Division of degree: QLD and non-QLD ‘streams’ by student choosing to do add on skills activities (the Law Skills portfolio by pathway 1 or 2. All students confront research activities in year 1 subject (CLRI) If want QLD then choose one of the Skills pathways in final year

Division of degree: QLD and non-QLD ‘streams’ by student choosing to do add on skills activities (the Law Skills portfolio by pathway 1 or 2.

All students confront research activities in year 1 subject (CLRI)

If want QLD then choose one of the Skills pathways in final year

Other outcome All questions on the table! Importance of technology New single sign on portal introduced in 2008 but ‘teething issues’ Difficulties of communication remain Need effective academic guidance

All questions on the table!

Importance of technology

New single sign on portal introduced in 2008 but ‘teething issues’

Difficulties of communication remain

Need effective academic guidance

Questions multiply What is the reality of the immediate learning or not-learning environment? Do students get practice and feedback on all major learning outcomes prior to final assessment? Is feedback aligned with what London would do? Is self-assessment incorporated into the learning of the course unit?

What is the reality of the immediate learning or not-learning environment?

Do students get practice and feedback on all major learning outcomes prior to final assessment?

Is feedback aligned with what London would do?

Is self-assessment incorporated into the learning of the course unit?

Academic guidance Do student know how to understand what London expects? Functional literacy (not literacy per se.) Are students shown how to read messages and spot cues about criteria for attainment from the nature of the activities they are given? Are there structured opportunities for students to discuss their learning with peers and benefit from peer feedback?

Do student know how to understand what London expects?

Functional literacy (not literacy per se.)

Are students shown how to read messages and spot cues about criteria for attainment from the nature of the activities they are given?

Are there structured opportunities for students to discuss their learning with peers and benefit from peer feedback?

headings for a subject ‘guide’: part 1 These are the aims of this subject... The reason for having this subject and for having it at this level or point in the course is... By the end of studying this subject you will be able to... These are the main ways of learning and teaching which will help you to achieve the learning outcomes... This is how the learning outcomes will be assessed...

These are the aims of this subject...

The reason for having this subject and for having it at this

level or point in the course is...

By the end of studying this subject you will be able to...

These are the main ways of learning and teaching which will

help you to achieve the learning outcomes...

This is how the learning outcomes will be assessed...

Headings for subject guide; part 2 This is how and at what points you will be given feedback on your performance... To achieve a pass you should... These are examples of the content of the unit... These are examples of the main learning resources you will use...

This is how and at what points you will be given feedback

on your performance...

To achieve a pass you should...

These are examples of the content of the unit...

These are examples of the main learning resources you will use...

Aims (information – practice) Learning Outcomes What should the students be able to know, understand, do? Standards Learning and Teaching Strategy What methods will help students achieve the learning outcomes? Assessment Criteria On what basis will you judge if students have met the learning outcomes and how well? Assessment Strategy How will you assess if students have met the learning outcomes? (source: Rosie Bingham 2001)

Constructive alignment (easy in theory, but feedback loop is crucial)

Peer assessment What opportunities for peer assessment exist? Work with ‘teaching institutions’ and research shows complexity of relationship/context of operation Extremely difficult to achieve blended learning

What opportunities for peer assessment exist?

Work with ‘teaching institutions’ and research shows complexity of relationship/context of operation

Extremely difficult to achieve blended learning

Evaluation?