Beyond the ‘adhocracy’: building a platform for legal education research and evidence-based policy-making in the Common Law world

1. Commonwealth Legal Education Association
Conference, Melbourne 2017
Beyond the ‘adhocracy’: building a platform for legal
education research and evidence-based policy-making
in the Common Law world
Paul Maharg Julian Webb

2. Context: ubiquity of reform
• USA
– Carnegie Foundation report (2007)
– ABA Task Force on the Future of Legal
Education (Report 2014)
– [ABA Commission on the Future of Legal
Services (2016)]
• UK
– ACLEC ‘First Report’ (1996)
– Law Society Training Framework Review
(2001-08)
– Legal Education and Training Review
Report (2013)
• Implementation by SRA/BSB/CILEx
2013-18
• Hong Kong
– Redmond-Roper Report (1992)
– SCLET Review 2015-17 – changes by 2021?
• Australia
– [Productivity Commission Report (2014),
Access to Justice Arrangements]
– Law Admissions Consultative Committee?
• Canada
– Federation of LS Canada Task Force Report
(2009)
– [CBA Legal Futures Initiative Report
(2014)]

3. Context: drivers for reform
• Costs of education and training (and impact on practice/A2J)
• ‘Over-supply’ of graduates
• Concerns about practice readiness [innovation + cost of training]
• Diversity and access to the profession
• Quality + scope and focus of regulation (moves towards more
competence- and/or risk-based regulation)
• Impact of new technologies (on practice and learning)
• Impact of new practice models: who will be tomorrow’s lawyers and
who will train them?
• Neoliberal policy agendas: competition, privatisation, flexibilisation of
HE

4. Our experience: LETR
• Driver: Legal Services Act 2007
(E&W) –
– ‘independent’ regulation and
market liberalisation agenda
• Push from oversight regulator
(LSB)
• Greater focus on regulation
• Lack of clear problem definition
• Multiple stakeholders/interest
groups
• Relatively fragmented system
of LET

5. Address the following issues:
1.What are the skills/knowledge/experience currently required by the legal services sector?
2.What skills/knowledge/experience will be required by the legal services sector in 2020?
3.What kind of legal education and training (LET) system(s) will deliver the regulatory objectives
of the Legal Services Act 2007?
4.What kind of LET system(s) will promote flexibility, social mobility and diversity?
5.What will be required to ensure the responsiveness of the LET system to emerging needs?
6.What scope is there to move towards sector-wide outcomes/activity-based regulation?
7.What need is there (if any) for extension of regulation to currently non-regulated groups?
1 LETR Remit

6. Pathways to qualification (E&W)

7. LETR’s ‘new paradigm’: understanding
(and responding to) social complexity
Lessons from complexity science,
work on bounded rationality,
‘wicked problems’, systems
theory, and modern regulatory
theory….
(eg, Majone, 1989; Reed & Harvey,
1992; Sanderson, 2002, 2006;
Baldwin & Black, 2008; Head,
2008, Wegner, 2009)
• Law of unintended consequences and the
limits of regulatory/organisational
steering
• Dampening effects on policy reform of
internal system design/complexity
• Impact of complexity of social and
governance networks
• One size does not fit all: possibly greater
effectiveness of small-scale, participatory
and localised interventions
• Beyond rational choice: values, ethical-
moral choices, and desired ends matter

8. LET as a ‘socially complex’ problem
Characteristics of socially complex
problems
Corresponding features of LSET (eg)
There is no definitive definition of the problem Some agreement over a need for reform, but widespread
disagreement over the extent, priorities and nature of the changes
required
They tend to be intractable General lack of effect from a number of recent education and
training reviews
Specific intractable problems:
•Achieving consistency of standards
•Reducing costs of training
•Managing increasing numbers
The information needed to make sense of the
problem is often ill-defined, changing and may be
difficult to put into use
Currently operating in rapidly changing work and educational
environments
Relative lack of robust, especially longitudinal, data
Costs of deriving meaningful information are relatively high

9. From Webb et al (2013), Table 1.1
Characteristics of socially complex
problems
Corresponding features of LSET (eg)
They emerge in fields where there are multiple
stakeholders; limited consensus as to who the
legitimate stakeholders and/or problem-solvers
are, and stakeholders are likely to have different
criteria of success
Large number of stakeholders, with different understandings of the
problem(s), and different levels of engagement with the process
Legitimacy questions exist, eg, over the extent of professional and
regulatory interest in the Bachelor of Laws (LLB)
Evidence of different stakeholders having different ‘objectives’ for
the review
Every attempt at a solution matters significantly Reform tends to be a ‘one-shot’ operation so relatively high risk
Exacerbated by uncertainties about the new regulatory environment,
and the tendency of LSET system to operate as a relatively low trust
environment

10. Complexity: how not to regulate…
1. Debate around ABA Standard 306 (now 311), restricting distance learning
vis-à-vis classroom time:
Standard 311. DISTANCE LEARNING
Distance education is an educational process in which more than one-third of the
instruction of the course is characterized by: (1) the separation in time or place, or both,
between instructor and student; and (2) the use of technology to deliver instruction.
2. But now see:
1. ABA Task Force Report
2. Mitchell Law School Variation, under Standard 802 (http://bit.ly/1kvD6uR):
Interpretation 802-1(b) states the Council may grant a variance for an experimental program if that
program is well-designed and the benefits of the experimental program outweigh its risks.

11. Colin Scott’s approach:
‘a more fruitful approach would be to seek to understand where the capacities lie
within the existing regimes, and perhaps to strengthen those which appear to pull in
the right direction and seek to inhibit those that pull in the wrong way’
‘meta-review’: ‘all social and economic spheres in which governments or others might
have an interest in controlling already have within mechanisms of steering – whether
through hierarchy, competition, community, design or some combination thereof’
(2008, 27).
Regulation and ‘modalities of
control’

12. Norms Feedback Behaviour-al modifi-
cation
Example Variant
Hierarchical Legal Rules Monitoring
Powers/Duties
Legal Sanctions Classic Agency Model Contractual Rule-
making &
Enforcement
Competition Price / Quality
Ratio
Outcomes of
Competition
Striving to Perform
Better
Markets Promotion Systems
Community Social Norms Social Observation Social Sanctions, eg
Ostrac-ization
Villages, Clubs Professional Ordering
Design Fixed with
Architecture
Lack of Response Physical Inhibition Parking Bollards Software Code
Modalities of control (Murray & Scott 2002)

13. Regulatory alternatives?
Shared spaces concept in traffic zones:
• Redistributes risk among road users
• Treats road users as responsible, imaginative, human
• Holds that environment is a stronger influence on behaviour than formal rules &
legislation.
‘All those signs are saying to cars,
“this is your space, and we have
organized your behavior so that
as long as you behave this way,
nothing can happen to you”.
That is the wrong story’.
Hans Monderman,
http://bit.ly/1p8fC3u
The Art & Science of Shared Streets, http://bit.ly/1p8fr8r See also
Hamilton-Baillie (2008).
redesign
relations

14. Participative regulation
• Portrait of the regulator as:
– Not QA but QE – Quality Enhancer, to focus on culture shifts towards
innovation, imagination, change for a democratic society
– A hub of creativity, shared research, shared practices & guardian of
debate around that hub
– Initiating cycles of funding, research, feedback, feedforward
– Archive of ed tech memory in the discipline
– Founder of interdisciplinary, inter-professional trading zones
• Regulator as democratic designer
redesign
relations
2

15. We’re not the only ones….
“A resounding sentiment was that there needs to be a permanent
conversation about the future of the profession – from broad and creative
imaginings of future business structures, to assessing emerging technologies,
to pedagogical reform in the training of lawyers – to encourage lawyers to
innovate within their own practice setting. More often than not, participants
affirmed that there should be space within the CBA for these issues to be
discussed on a more regular basis.” (Canadian Bar Association, 2014:24)
Legal Services Forum 2016 15

16. The need for structure(s)
... the fundamental tension between education of lawyers as delivering public value and education of
lawyers as delivering private value is structural. The tension may manifest itself in different ways under
different conditions, but it will always be with us and must always be managed. Other matters likely to
continually give rise to stresses, challenges, and the need for managing change are: the economics of law
schools; the rapid evolution in the market for legal services; the function and value of accreditation
standards; the financing of legal education; the role of parties other than law schools in legal education;
and the role of media in understanding legal education and communicating with the public.
Since these forces and factors will always be with us, it is prudent for the system of legal education to
institutionalize the process of dealing with them. All parties involved in legal education should support a
framework for the continual assessment of strengths and weaknesses and of conditions affecting legal
education, and for fostering continual improvement. The process should ensure that not only law schools,
but also practicing lawyers, judges, and other interested actors have a voice and an opportunity for
meaningful contribution.”
ABA Legal Education Task Force, 2014:29
Legal Services Forum 2016 16

17. Structures, systems and culture matter
“Innovation is an ongoing process that requires sustained effort
and resources as well as a culture that is open to change. To
sustain and cultivate future innovation, the ABA should establish
a Center for Innovation.”
ABA Commission on the Future of Legal Services – Final Report, 2016: 48
Legal Services Forum 2016 17

18. Future research needs?
1. Map the field & create
taxonomies for research data
3 improve
research
2. Organise systematic data collection on
law school stats across entry/exit
points, across jurisdictions (eg using
Big Data Project methods)

19. Future research needs?
3. Focus on learning,
not NSS league tables – see US
LSSSE… and include longitudinal
research data, not just
snapshots of place & time
3 improve
research
4. Provide meta-reviews
and systematic summaries
of research, where
appropriate; literature
guides

20. Themis.space
• Collaborative (and cross-disciplinary) research
(Melbourne/ANU/Flinders)
– Original research and meta-analysis: the changing profession/market for legal services
– Original research and meta-analysis: developments and best practices in legal education
and training
– Synthesis: consequences for LET of regulatory/organisational/technological change in
legal services
• Databanking
• Resources and training
– Working Paper Series
– Systematic Reviews Series
– Methods Series
20

21. Funding needs
1. Targeted funding for research initiatives, systematic reviews, eg Cochrane
Collaboration type of initiative
2. Funding & admin support to start-up and analyze innovation – eg PBL,
public education in law,
legal informatics,
data visualization, etc
3. Financial & other support to enable
round table meetings with regulators
and comparative work with other
jurisdictions – globally
4. Creation and maintenance
of a digital hub.
3 improve
research

22. If we build it will they
(you) come?
22