Assessment Workshop, Plenary 1.pptx

Professor Paul Maharg
paulmaharg.com/slides
https://www.zotero.org/groups/4722732/lsi_assessment_workshop_27.6.22/library
Law Society of Ireland
Workshop on Assessment
27.6.22
Plenary 1
Zeugma
legal education::technology::
writing::professional education

preview of workshop
2
Times Session
1000-1040 1. Introductions (10 mins)
2. Plenary 1: Research, ideas, applications (RIA): Framing research & concepts (20)
3. Brief discussion (10)
1040-1115 Breakout groups: Group discussions (20), plenary discussion (15)
1115-1130 Break
1130-1200 Recap on breakout issues.
Plenary 2: Examples of the architecture of assessment
1200-1230 Breakout groups: Group discussions
1230-1300 Plenary discussion of ways forward & possible projects
Zeugma

in this plenary – some framing research &
concepts
Zeugma
ideas
application
research
Some key questions:
1. Is the research out there? Is it
relevant to my subject?
2. Are my ideas feasible re time,
finance, effort?
3. Is the application practical?
4. Will the application improve
professional learning?
5. Is it sustainable?
6. How will it fit programme
learning?
7. How do I think students will
remember my assessments?

conventional teaching & assessment
If learning … then assessment is often…
1 Is teacher-focused Teacher-centred, not learner-centred.
2 Follows a transmission model
of education
Focused only on what’s supposed to have arrived /
been delivered
3 Focuses only on the individual Individual, alienating, where collaborative, peer-
review or self-review can’t take place
4 Consists of monolithic &
substantive law content
Lacking interdisciplinarity, with little assessment of
skills, values, attitudes as well as knowledge
5 Sits in the contested relations
between practice & academy
Problematic, because content & forms of academic
assessments can’t transfer well to professional
learning and formation of identity
Zeugma

signature pedagogies, Lee Shulman
Surface
structure
•Observable,
behavioural
features
Tacit structure
•Values and
dispositions that
the behaviour
implicitly models
Deep structure
•Underlying
intentions,
rationale or theory
that the behaviour
models
Shadow
structure
•The absent
pedagogy that is,
or is only weakly,
engaged
Sullivan, W.M., Colby, A., Wegner, J.W.,
Bond, L., Shulman, L.S. (2007) Educating
Lawyers. Preparation for the Profession of
Law, Jossey-Bass, p. 24
Zeugma

transforming pedagogies
Experience of…
•law in the world
•interdisciplinary trading
zones
•creative, purposeful acts
Ethics in…
•an integrated curriculum
•habitual action
•reclamation of moral
spaces in the curriculum
Technology for…
•our discipline, our
curricula
•learner-centred control
•transactional learning
Collaboration
between…
•students
•institutions
•academic & professional
learning
•open-access cultures
Zeugma

1. Framing research & concepts

1. high impact strategies (Kuh)
1. Investing time and effort
2. Interacting with faculty and peers about substantive matters
3. Experiencing diversity
4. Responding to more frequent feedback
5. Reflecting and integrating learning
6. Discovering the relevance of learning through real-world
applications
(Kuh, 2008, 14-17)
Zeugma

2. student engagement
Learning begins with student engagement, which in turn leads to knowledge and
understanding. Once someone understands, he or she becomes capable of performance or
action. Critical reflection on one's practice and understanding leads to higher-order
thinking in the form of a capacity to exercise judgment in the face of uncertainty and to
create designs in the presence of constraints and unpredictability. Ultimately, the exercise
of judgment makes possible the development of commitment. In commitment, we become
capable of professing our understandings and our values, our faith and our love, our
skepticism and our doubts, internalizing those attributes and making them integral to out
identities. These commitments, in turn, make new engagements possible - and even
necessary... [Thus] engagement is not solely a proxy; it can also be an end in itself. Our
institutions of higher education are settings where students can encounter a range of
people and ideas and human experiences that they have never been exposed to before.
Engagement in this sense is not just a proxy for learning but a fundamental purpose of
education.
(Shulman, Foreword, in Gess 1999, my emphases)
Zeugma

3. Entrustable Professional Activities (EPAs)
Zeugma
• An EPA is a unit of professional practice that can be entrusted
to a sufficiently competent learner or professional
• It requires proficiency in multiple competences
simultaneously, and is a more suitable focus for assessment
than separate competencies
• EPA-based assessment results in summative entrustment
decisions to act under a specified level of supervision
• Mobile technologies and e-Portfolios can be used to support
EPA-related feedback and entrustment decision-making
AMEE Guide, 2015

IAALS is approaching an EPA approach…
https://iaals.du.edu/sites/defa
ult/files/documents/publicatio
ns/think_like_a_client.pdf
Zeugma

4. Programmatic assessment
Zeugma
Programmatic assessment:
1. optimizes the coaching, learning, and decision-making functions of
assessment.
2. It utilizes multiple data points which on their own guide learning, but
taken together form the basis of holistic decision-making.
3. While principles are broadly agreed on, implementation varies according
to context.
4. It involves programmatic assessment framework, tracking and making
sense of data, how academic decisions are made, and how data guide
coaching and tailored support and learning plans for learners
5. But it’s resource-intensive and requires substantial support from
professional staff

2. Examples of 1-4: PREP & its education design principles

Canada is a federal system
• 10 provinces and 3 territories
• No national regulator or educational
standard for domestic students
• Each province and territory regulates the
legal profession and sets standards for legal
education
• Law Societies are still the
provincial/territorial regulators
Zeugma

Requirements of provincial/territorial Law
Societies for entry to the legal profession
• Obtain university degree in law
• Pass one or more exams
• Experiential training (usually articling)
• Some provinces/territories also have bar
admission courses
Zeugma

CPLED & PREP
• Practice Readiness Education Program (PREP)
• Bar admission course Alberta, Saskatchewan,
Manitoba, Nova Scotia
• Practical lawyering skills, including Sim Client
interviews
• In interviewing, each student 2 practice interviews
then final exam interview (Capstone)
• Practice interviews formative and summative
• Capstone summative only
• 800-1,000 students learning entirely online during the
pandemic
Zeugma

PREP schedule
Zeugma

design principles
Fusion of:
– Distance + intimacy
– online + f2f
– singleton + collaborative learning
– variety of media and platforms
– academic + professional + personal
knowledge
– Learning about the job + learning
within the job
– pervasive ethics in sims
Zeugma

Typical fusion of assessment criteria with
learning
21 Professor Paul Maharg | CC BY-NC-ND 2.5 CANADA
Zeugma

assessment in more detail…
1. Phase 1: Foundation modules: online, singleton largely
2. Phase 2: Foundation w/shops: f2f + collaboration
3. Phase 3: Virtual firm transactions: collaborative + individual learning, which is
foregrounded + formative assessments of
collaboration & individual effort (mandatory
completion)
4. Phase 4: Capstone: singleton assessment of all skills and
knowledge. Reflective e-Portfolio used as
part of assessment tools
22
Zeugma

sims in a spiral curriculum
• PREP’s four phases are designed
as more & more sophisticated
spirals
• Digital sophistication matches
cognitive & skilled complexities
• Digital use matches cognitive &
skill levels: there is matching
sophistication.
• All assets will be available all the
time (cp LMSs…)
• Reflection in the e-portfolio is the golden
thread throughout.
Harden, RM, Stamper, N. (1999).
What is a spiral curriculum?
Medical Teacher, 21, 2, 141-3, 142.
Zeugma

16.6.17
Resource-building:
from this…
Zeugma

to this…
Zeugma

In PREP,
tutors are
apomediators
Eysenbach (2008)
Zeugma

Academics as apomediators
Apomediation…?
1. Essentially the replacement of traditional intermediaries by
apomediaries, tools and peers standing by to guide consumers to
trustworthy information, or add credibility to information.
2. Helps users navigate informational overload
3. Uses collaboration to scale, collaborative filtering, recommender system,
allows bookmarking and scholarly folksonomies
4. Sophistication of reader means that intermediaries may be preferred at
first; but as expertise grows, apomediation is needed.
5. With pre-print publication, journals themselves could be
disintermediated, though publishers contesting this strongly.
Zeugma

Learning and time
‘”Time”, he said, ‘is what keeps everything from happening at once””
Ray Cummings, The Girl in the Golden Atom (1922)

1. chronology of the curriculum
Zeugma

• students needed time to:
– explore – don’t read
recursively
– listen to their expectations,
and to ours
– become adept in context
– develop good habits
2. designing for time
Zeugma

3. space and time:
bounded field &
open field
transactions

bounded field open field
Zeugma

Assessment Workshop, Plenary 1.pptx

Recommended

Recommended

More Related Content

Similar to Assessment Workshop, Plenary 1.pptx

Similar to Assessment Workshop, Plenary 1.pptx (20)

More from York University - Osgoode Hall Law School

More from York University - Osgoode Hall Law School (20)

Recently uploaded

Recently uploaded (20)

Assessment Workshop, Plenary 1.pptx