Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (18)
Similar to Prof Liz Johnson: Growing the ACDS T&L Centre
Similar to Prof Liz Johnson: Growing the ACDS T&L Centre (20)
Recently uploaded
Recently uploaded (20)
Prof Liz Johnson: Growing the ACDS T&L Centre
- 1. ACDS projects in learning & teaching - 2015 Liz Johnson Director, ACDS TL Centre
- 2. Key topics for science learning & teaching … • Quality assurance and benchmarking • Assessment as the key to curriculum • Building capability, and advice on resources
- 3. The Faculty role Faculty: environment for learning • Standards and quality assurance • Resources for staff and students • Expertise and capacity for teaching Faculties deliver whole degrees Faculties operate across disciplines to: • create equivalent outcomes • foster cross-disciplinary links Context for learning: Institution/Faculty Discipline content and skills: Disciplines
- 4. The role of the ACDS TL Centre • Build a picture of good practice in learning and teaching in science and mathematics • Provide condensed authoritative advice to Faculties of Science: evidence-based practice • Develop advice for regulatory, funding and policy bodies • Build and support links amongst science and maths education leaders
- 5. Quality assurance and benchmarking • The Standards environment • Benchmarking projects • Using the Science Threshold Learning outcomes
- 6. Shared standards: Reference points and Benchmarks INPUTS OUTCOMES Guide for curriculum design & teaching delivery Evidence for claimed levels of achievement
- 7. Science Threshold Learning Outcomes • Statements of graduate outcomes across science disciplines • Apply to a pass level bachelor degree • Commissioned by ALTC, endorsed by ACDS
- 8. Standards environment… Govt + Institution + Discipline Accreditation of institution Self-accreditation of degrees Professional accreditation of degrees TEQSA legislation Internal policies and procedures Science associations: Recommended curricula Australian Qualifications Framework Institutional priorities Standards Higher Education Standards ESOS compliance MyUni website
- 9. Australian Standards 1. Higher Education Standards: Course Design, Learning Outcomes and Assessment Minimum standards for design & delivery of teaching http://www.hestandards.gov.au/ 2. Australian Qualifications Framework Minimum standards for degree levels: bachelor vs masters vs PhD http://www.aqf.edu.au/ 3. Science Threshold Learning Outcomes Descriptors of minimum knowledge and skills for science graduates http://www.acds.edu.au/tlcentre/links-publications/ or http://www.olt.gov.au
- 10. Australian Standards 1. Higher Education Standards: Course Design, Learning Outcomes and Assessment Minimum standards for design & delivery of teaching Learning outcomes • are specific and informed by national/international http://www.hestandards.gov.au/ 2. Australian comparators Qualifications Framework Minimum standards for degree levels: bachelor vs masters vs PhD • Cover discipline-specific and generic outcomes • Are demonstrated by students on completion http://www.aqf.edu.au/ 3. Science Threshold Learning Outcomes Descriptors of minimum knowledge and skills for science graduates http://www.acds.edu.au/tlcentre/links-publications/ or http://www.olt.gov.au
- 11. Evidence for QA Curriculum design Curriculum documents, study guides, assessment tasks & rubrics Student achievement Student retention Student success (pass/fail rates) Degree benchmarking between institutions Assessment pieces, portfolios Student feedback Satisfaction (UES, CEQ, institutional surveys) Self-appraisal & confidence Graduate outcomes Graduate employment/ further study (GDS) Employer satisfaction
- 12. Benchmarking projects Cross-marking of sample work Moderation of assessment tasks, rubrics/marking schemes Development of assessor expertise Institutions • Institutional benchmarking: eg QVS (GO8),IRU, Learning & Teaching Standards, IRU, Scott (OLT Fellowship 2014) • National External Peer Review of Assessment Network Disciplines • Accounting: Freeman (“Assessment Matters”) • Chemistry workshops with TLOs (RACI)
- 13. Science Threshold Learning Outcomes Require interpretation and application Translation into assessment tasks and learning activities
- 14. Using the Science TLOs Science TLOs Disciplinary TLOs Biology Biomedical Science Chemistry Mathematics Physics … Agricultural Science Environmental Science Good practice guides Description of TLO Examples/case studies Resources for curriculum design See: www.acds.edu.au/tlcentre ... Projects …TLOs in science VIBEnet, CUBEnet, Chemnet, AMSLaT, Physics Education Network, AgLTAS, Environmental Science Network Sue Jones, UTas Brian Yates, UTas
- 15. ACDS TL Centre and standards Foster implementation of Science TLOs • New disciplines • Dissemination of discipline network projects • Connect partners Funding for implementation projects? • Local projects test strategies • National projects (eg Mind the Gap - testing maths achievement) Deb King, UMelb Michael Jennings, UQ Adam Bridgman, USyd John Rice, ACDS
- 16. Assessment: key to the curriculum • Assessment drives learning • Relevant examples of assessment practice
- 17. Assessment in learning Learning Objectives Assessment Learning activities Assessment indicates “importance” to students and teachers Assessment drives student behaviour Assessment produces evidence of learning
- 18. Assessment as evidence Formative assessment monitors progress (educators), directs learning (students) Summative assessment measures achievement and learning outcomes • Benchmarking: parity of awards and outcomes • Relative rank: scholarships • Individual student achievement: • Pass/fail – minimum standards • Excellence – out of 100? • Prospective employers – marks (transcript) or demonstration (portfolio, competencies)
- 19. Authentic assessment • Mimics real-world tasks • Complex, requiring integrated skills and knowledge • Inquiry tasks reflect scientific process – what assessment is appropriate?
- 20. Multi-tasking assessment Report to Uni (internal)to monitor comparable student achievement writing Inquiry/research Snapshot measurement of an intended outcome by an assessment task Report to student to give feedback on progress Demonstrate learning outcomes externally
- 21. Measuring outcomes: assessment rubrics Published rubrics • Link assessment to objectives • Standardize assessment practice • Space for professional judgement • Inter-marker moderation Moderation with student work
- 22. ACDS assessment project? • Consolidated assessment advice • Disciplinary variation and examples/case studies
- 23. Building capability • Resources for teaching academics • Building capability through Faculty projects
- 24. Resources for teaching Decades of work available! SMART Directory (Science & Maths Annotated Resources for Teaching) Lists online compiled resources for science and maths Annotated with resource type, discipline Potentially include peer rating system? Kirsten Zimbardi, UQ Kay Colthorpe, UQ Malcolm Campbell, Deakin
- 25. Capacity building: seed projects SaMnet • Science and Maths Network of Australian University Educators • Leadership development • Advice and mentoring for Faculty learning & teaching projects • Scholarship of learning and teaching (evidence-based practice) • Peer network ACDS mentoring scheme (SaMnet 2.0)? • Identify priority development areas (eg assessment of TLOs) • Seed funding for Faculty-based projects • Share outcomes through ACDS TL Centre/SaMnet • Mentoring by Science TL leaders
- 26. So…possible 2015 projects… • Mind the Gap – diagnostic testing in maths • SMART Directory development – annotated resources • ACDS consolidated assessment advice project • ACDS seed projects in TLO assessment (SaMnet 2.0) What should be the top priorities? Other ideas?
- 27. Thanks to… Prof John Rice Prof Russell Crawford & the ACDS Executive 2014 ACDS Project teams Emma Yench, Webmaster Stephanie Beames, Conference Organization
- 29. Science is not one discipline • Common themes • Foundation skills • Overlapping content • Specialist knowledge and skills Figure 5. Map of science derived from clickstream data. Bollen J, Van de Sompel H, Hagberg A, Bettencourt L, et al. (2009) Clickstream Data Yields High-Resolution Maps of Science. PLoS ONE 4(3): e4803. doi:10.1371/journal.pone.0004803 http://www.plosone.org/article/info:doi/10.1371/journal.pone.0004803
- 30. Science associations in education Associations offering accreditation (excluding professional registration) 7 (agriculture, mathematics, chemistry, physics, psychology, statistics, nutrition) Associations offering resources and education events 21 Associations offering resources 12 Science Education networks 7+ (biomedical science, biology, chemistry, mathematical sciences, physics, environmental science, agriculture…and growing) Who should define standards for Science degrees? How can Faculties balance different disciplinary requirements? How do students manage competing disciplines?
- 31. ACDS Teaching & Learning Centre Centre website TL meetings Projects ADTLs ACDS members Science TL leaders
- 32. Curriculum for diverse outcomes
- 33. What do employers think? • Employment: A mix of cognitive and behavioural skills • Content is the context for skill development Graduate capability Employer ranking Graduate ranking Faculty ranking Ability & willingness to learn 1 1 1 Teamwork and co-operation 2 2 5 Initiative 3 7 8 Analytical thinking 4 3 3 Technical expertise 13 17 6 Coll and Zegward (2007) Research in Science & Technological Education 24:1, 29-58 See also: Harris (2012) A Background in Science, CSHE
- 34. Tertiary Education Landscape TEQSA Act Higher Education Standards Australian Qualifications Framework Other HE Providers (~130) • TAFEs • Private providers • Professional Associations Self-accrediting (43) • Public • Universities (37) • Private • Universities (3) • Overseas Universities (2)
- 35. Discipline education networks Discipline experts • Core concepts & skills and their interconnection • Disciplinary thinking • Pedagogical content knowledge • Effective Teaching practice Physics Education Network
- 36. Thank you to… • Australian Council of Deans of Science and the Associate Deans Teaching and Learning • Office for Learning and Teaching • Staff and students of the Faculty of Science, Technology and Engineering at La Trobe University • Discipline networks: CUBE, VIBE, Chemnet, Physics Education Network, AMSLaT
- 37. The ACDS Teaching and Learning Centre Liz Johnson OLT National Teaching Fellow Support for this activity has been provided by the Australian Government Office for Learning and Teaching. The views expressed in this presentation do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.
- 38. In the talk… The role of the ACDS TL Centre Operation of the ACDS Centre • Centre website • Centre projects • A learning and teaching hub Priorities for the ACDS Centre • feedback from consultation • projects for 2014
- 39. Developing the model National Conferences (ACDS TL conference, ACDS AGM, ACSME, HERDSA) 7 ADTL/institutional leader interviews 10 State-based/discipline workshop 12 Institutional presentations 4
- 40. A hunger for consolidated and authoritative information… Resources Evaluated good practice Practice/ideas exchange Info: funding, conference Evaluating Actions Link networks/disciplines Voice & vision for Sci Funding Endorsement Careers/mentoring People & organization College of experts? Character X –disciplinary Connected, sustainable
- 41. ACDS TL Centre Operation ACDS ACDS Executive ACDS TL Centre Centre Director Communications Officer ADTL network ACDS TL Conference ADTL Professional Development Workshops Science & Maths educators Information hub Practice exchange ACDS TL project teams ACDS Position papers National Statements/ Reports on Learning and Teaching in Science and mathematics Distributed leadership Facilitation, Progression Authority, whole-of-sector
- 42. Testing the model: 2013 • Participation: newsletter (36+), meetings (~50) • ACDS TL Centre Website • Projects: ACDS TLO project • Meetings • ACDS TL Conference • Discipline Network Roundtable • Project Workshops • ACSME Centre website TL meetings Projects ADTLs ACDS members Science TL leaders
- 43. www.acds.edu.au/tlcentre Centre Projects: Information, outcomes Resources: ACDS reports & statements Useful links News: meetings, projects, publications, ideas Networks: Science Education, ADTL Science TL Calendar Discussion board Ideas exchange
- 44. ACDS TLO Project Objectives: • Advice on using the national Science Threshold Learning Outcomes (Science TLOs) in course design • Forward planning for sector-wide uptake Action: • Advancing the TLOs workshop, Feb 2013 • ACDS response to HESP draft standards • Advice for Faculties in preparation • Publications: • Discipline draft TLO statements • TLO good practice guides
- 45. Meetings • Develop collective view • Disseminate information • Initiate and progress projects • Build on existing meetings ACDS TL Conference • ADTLs, TL leaders Workshops • Specific purpose: TLO workshop, Discipline Network Roundtable; ACDE/ACDS joint meeting • Professional development? ACSME • Broad audience of science and mathematics educators
- 46. Priorities for the ACDS TL Centre
- 47. Recurring themes…issues for STEM? Curriculum • New modes, online teaching • Measuring achievement of learning outcomes • Ubiquity of information changes the role of Uni Students • Diversity and underpreparation • Large classes Staff • Professional development and cultural change
- 48. Recurring themes…valuable info? Authoritative advice: • Position paper/ executive summary • Pedagogies, PD for staff, student diversity & underpreparation Tools: • Good practice/ideas exchange, resource toolkit, benchmarking projects, evaluation, links to resources Networking/ sector information • What’s happening?, regulation and standards • Conferences, discussion forum
- 49. Where to next? • ACDS TL Centre is growing and
Editor's Notes
- I’m going to stray away from the topic in your program to concentrate on setting the background for your discussion of curriculum development For me, curriculum includes all the resources and activities that contribute to student learning. It includes the content of your courses, the learning activities the students undertake and the assessment which tells about student progress and achievement. Curriculum design considers all these issues. In this talk I will focus on external guidance about what belongs in your curriculum, how that information can be used productively and a little about teaching delivery issues.
- Core roles for the Centre were established early and tested through consultation with ADTLs and with learning and teaching leaders. There was very good consensus that these roles were both appropriate for the ACDS Centre and needed in the sector. So what should the ACDS Centre be doing? …. Two further ideas about the role the ACDS sector emerged during the project: the idea of aligning influences on teachers and institutions and the idea of a the Centre as a hub for activities across the sector.
- We can use shared standards for curriculum in two ways. Standards provide reference points to guide planning and delivery. Standards provide the benchmarks for measurement of outcomes In a national system, standards should be consistent so we can be sure that the quality of our curriculum will be recognized elsewhere. Of course, this is very helpful because it fosters productive collaboration between institutions and within disciplines. Standards provide guidance – where do they come from?
- In 2011, the Australian Learning and Teaching Council kicked off the Learning and Teaching Academic Standards Project to develop Australian subject standards statements. They appointed 2 discipline scholars for science – Brian Yates and Sue Jones. Brian and Sue consulted VERY extensively to develop the science TLOs which are the agreed minimum standards for a B Sc graduate from an Australian university. This landmark statement has been the starting point for a lot of curriculum design work and thinking.
- Standards come from multiple sources Different standards focus on different parts of the curriculum and at different levels. Apart from the regulatory standards they don’t prescribe what we teach
- The design of courses is governed by two sets of Govt Standards Higher education standards are currently in re-development and are being re-organized with focus on outcomes as well as inputs I’ve paraphrased some of the standard on learning outcomes here CLICK
- The design of courses is governed by two sets of Govt Standards Higher education standards are currently in re-development and are being re-organized with focus on outcomes as well as inputs I’ve paraphrased some of the standard on learning outcomes here CLICK
- In 2011, the Australian Learning and Teaching Council kicked off the Learning and Teaching Academic Standards Project to develop Australian subject standards statements. They appointed 2 discipline scholars for science – Brian Yates and Sue Jones. Brian and Sue consulted VERY extensively to develop the science TLOs which are the agreed minimum standards for a B Sc graduate from an Australian university. This landmark statement has been the starting point for a lot of curriculum design work and thinking.
- Assessment is, of course, crucial to successful curriculum design. It provokes strong feelings about standards, workload and value. Here’s one view of assessment – as evidence for learning Read through slide Changing assessment is one of the most powerful ways to shift learning outcomes. If we cope with large class sizes by increasing reliance on MCQ exams we should not be surprised that student learning changes. We need to carefully monitor what our assessment regime is really telling students to concentrate on.
- There is no room for lazy assessment in a science curriculum. Assessment tasks generally need to cover more than one intended learning outcome. Student work is the direct evidence of student learning so assessment provides the evidence base for achievement. At La Trobe we are using assessment as evidence in three ways. We use assessment to help students monitor their own progress, to monitor how our curriculum is developing graduate capabilities and to provide evidence to external groups like TEQSA that our courses do deliver what we claim. Of course, easier said than done!
- Use of assessment as evidence requires that assessment practice is robust. Assessment rubrics give students and markers clear guidance about the relative importance of each assessment criteria and the level of achievement required. Personally I use a mix of descriptive statements and quantitation (number of marks). There are challenges with both. The most important thing is to build consensus amongst markers – often a group of hard-working and under-appreciated sessional staff – and to periodically defend the assessment standards with colleagues outside the immediate teaching team.
- And continue… Dissemination of news, events and TL activity Links with discipline networks Website updates
- The tricky bit is that Science is not one discipline. This map shows connectivity and clustering between disciplines of science. It was built on an analysis of how researchers moved around in scholarly web portals such as the web of science and was gathered from web logs showing behaviour. So this a functional map of how science research areas are related. Its complex! So, how does a generic list of outcomes for science graduate turn into something useful for curriculum design on the ground? It needs translation and adaptation to the local environment – your classroom. There are a lot of tools available to help you.
- There is a lot of information on good curriculum design and delivery available but it is largely scattered. Good work in biology does not easily reach physics educators. And the array of disciplines in science and mathematics is wide. This snapshot is taken from the science organizations listed on the Science and Technology Australia site. All of these professional societies are providing advice to their members at some level. Why does this matter? Because our disciplines are interdependent, we all rely on foundation studies in year 1. It also matters because students study in multiple disciplines in parallel. It is far more effective to reinforce learning outcomes across multiple subjects than to operate independently. And thirdly it matter because good ideas in science teaching, especially around practice, are valuable and should be shared to promote good practice. As well as connecting disciplines a science teaching hub can bring together information on projects working in similar space either national or international. The OLT repository contains…. So as well as its roles in distilling information, constructing authoritative advice for Faculties and external groups, the ACDS Teaching and Learning Centre offers us cross-disciplinary view of Science education and a hub for activity. Now I’d like to move to the operating model I have proposed in the paper for the AGM. I’ll start with some key ideas that came out of my consultation with the sector.
- This is the model for the Teaching and Learning Centre that I presented to the ACDS AGM in October 2012. It is a virtual centre of teaching and learning leaders from Faculties of Science, including ADTLs, ACDS members and science teaching and learning education specialists and project leaders – people who are innovators, passionate about student learning The work of the Centre is delivered through three types of activities The Centre Website = face of the Centre Teaching and Learning meetings which bring together Centre members and other stakeholders and drive engagement and work Projects that tackle issues important to sci and maths education in a targeted and finite body of work The primary target for the work of the Centre are Faculties of Science and its work must complement the work of other groups concerned with learning and teaching in higher education. We do not want to reinvent the wheel or to compete for scarce resources to support learning and teaching. So the first step was to establish a clear role for the Centre
- Science students use their degrees in very diverse ways. How many students enrol at Monash in the B Biomed Sci? How many will go into research? How many stay in research for the long-term? Science degrees need to foster a range of skills and knowledge.
- Employers value skills above technical knowledge. This study comes from New Zealand and compares perceptions about graduate capabilties in graduates from B Sci, B Sci and Tech and B Engineering. Comparison of the perceptions of academics against employers and graduates shows a couple of interesting things. Firstly, all agree that a willingness to learn is the very important graduate asset. Secondly, academic staff thought that technical competence was much more important than employers did. This tallies with suggestions on on one hand that science courses focus on content and on the other that employers are more willing to have graduates learn specific content on the job. But it doesn’t have to be one or the other. Disciplinary content should be the context for skills development.
- Universities are self-accrediting institutions governed by the TEQSA act. Why does this matter? We are given the capacity to decide for ourselves what meets national criteria for the quality of education in our institutions. Our curriculum must stand up to external scrutiny The National Framework is now moving away from focus on criteria about inputs to the curriculum to focus on learning outcomes. So, what does that mean for curriculum design?
- Locally discipline education networks have been developing disciplinary interpretations of the Australian Science TLOs. I hope you are all familiar with CUBEnet whose work includes development of biomedical science threshold learning outcomes led by Janet.
- Thank you for inviting me Today I would like to talk about the work I have been doing for the last 15 months on the establishment of the ACDS Teaching and Learning Centre. The Centre was the vision of John Rice. It seeks to achieve sector-wide improvements in learning outcomes for science and mathematics students …in the face of declining interest in advanced level mathematics and fundamental sciences such as physics and chemistry. My task has been to establish the Centre and to make sure it had good engagement with Faculties of Science and other stakeholders.
- In this talk I would like to present the Centre as it currently stands and to look forward to what it could achieve next. I will explain how the role of the Centre has evolved I will present an operating model that has evolved through discussion with teaching and learning leaders in science and mathematics We have tested the model this year through a number of activities which have refined my thinking. Finally I will share with you some of the suggestions made for future work for the Centre
- The Centre model was developed through a range of activities. I began with the ADTLS at the ACDS TL conference and then through direct interviews. Detailed reports from the ACDS TL conferences are on the Centre website. The second group were the SaMnet scholars. Interesting cross-section of learning and teaching leaders… action-learning teams… The consultation collected suggestions for how the Centre could best operate, ideas about what were the most pressing issues in science and maths teaching and what projects the Centre should focus on in the near future. Distributed leadership: shared leadership is more robust & productive; diverse expertise is recognized Communities of practice: : action emerges from common interest and peer interaction Sustainable provision / self-renewing: not dependent on a single driver
- I summarized some recurring themes from the feedback on the operation of the Centre in this slide The bottom two boxes are about what the Centre should be. TL leaders valued the cross-disciplinarity that the Centre could provide and were also keen to have a well-connected and sustainable presence that was not dependent on finite project funding. They wanted the Centre to be authoritative through contribution of recognized expertise. The top two boxes are about what the Centre should do. TL leaders wanted the Centre to either provide or consolidate links to credible resources for teaching. They felt the Centre had the capacity through the ACDS to be a strong voice for Science and Mathematics education. They strongly felt the Centre had a role in fostering excellence through endorsement and funding of projects and in developing future leaders. It’s a big agenda. Out of the consultation and the experience of trialling some activities, I’ve constructed this operating model.
- This operational model fleshes out the original proposal to the ACDS Executive in Feb 2012. Balance between a minimal governance structure (point to top two layers) and the work of the Centre delivered through Centre members (point to bottom layer) … CLICK The structure is built on three roles: members bring diverse expertise and interest to the work of the Centre and take leadership of discrete activities remembering that all of the members have busy positions already; the Centre director and the communications officer faciltate each activity: bring teams together, cross-pollinate ideas; keep activities on track, inform the ACDS and provide consolidated advice, liaise with external partners and interest groups; The ACDS and its executive hold the governance role for the Centre; set priorities, review work and products, lead external interactions and, of course, decide on funding This model has been tested through a number of Centre activities in 2013.
- During 2013 we tested the Centre operational model through a series of activities. We built the website which continues to develop We ran teaching and learning meetings building on the ACDS TL Conference And we are now running the first Centre project Now we’ll have a quick look at each of these activity areas
- Uses WordPress so relatively easy to manage and add material Little feedback from viewers yet Hits mid Jul-mid Oct = 223 visitors 18 news stories, 3 news features including extended video footage Where to next… News stories or major links have been published for: 2 current OLT projects in science and mathematics 4 OLT fellowships 6 science peer networks
- Advancing the TLOs workshop, Feb 2013 Project launched Mar 2013
- This wordle is constructed from feedback during my fellowship from workshops and interviews. I found it interesting that practice of teaching was the major concern but you can see other ideas surfacing about capacity – academics and staff…