Towards a culture of data collection and analysis  in mathematics support centres  Professor Tony Croft Director, Mathemat...
 
background
background <ul><li>“ higher education has little option but to accommodate to the students emerging from the current GCE [...
Overview <ul><li>Growth in activity </li></ul><ul><li>Difficulties of getting good and convincing data </li></ul><ul><li>W...
Caveats <ul><li>Shouldn’t be too fixated on trying to find hard measures of success. </li></ul><ul><li>We are professional...
Why do we have these ? <ul><li>Because we care! </li></ul><ul><li>Because we want to encourage and support our students </...
Continuing growth in UK activity <ul><li>Surveys: </li></ul><ul><li>Demand: When sigma announced a competition for funding...
Around the world <ul><li>Australia: there are 39 universities, of which 32 have some form of Mathematics & Statistics Lear...
Around the world <ul><li>Switzerland’s first….. Opened September 2008 </li></ul>
Evaluation, accountability <ul><li>So with huge and rapid growth of mathematics support activity it is right and proper th...
Difficulties of getting data:  systems and staff <ul><li>This activity is still very young. </li></ul><ul><li>Drop-in acti...
Difficulties of getting data: departments and students <ul><li>Departments often reluctant to give out data. </li></ul><ul...
Difficulties of getting data <ul><li>Sent: 30 September 2008 12:43 To: mlsc@lboro.ac.uk Subject: Maths for Physics Hi, I’v...
Resource Issues / financial constraints <ul><li>Almost all centres were established on very modest budgets and their purpo...
What has been written on Evaluation ? <ul><li>[Mathematics Learning Support] needs sufficient security to attract, train a...
Measuring and modelling success <ul><li>I will describe both soft and hard measures. </li></ul>
Measures of success – soft measures <ul><li>Growth in the number of centres </li></ul><ul><li>Usage Data – attendance reco...
Usage Data <ul><li>Student visits </li></ul>
Usage Data <ul><li>Students </li></ul><ul><li>Visits </li></ul><ul><li>Departments </li></ul>
External Review <ul><li>The  Institutional Audit  report of 2004 records: </li></ul><ul><ul><ul><li>“ The audit team ident...
NSS <ul><li>National Student Survey 2008 </li></ul><ul><li>Comments relating to “maths support” </li></ul><ul><li>Departme...
The student voice <ul><li>video </li></ul>
Modelling success   –  hard measures <ul><li>Studies with more substance are beginning to emerge which might inform method...
Modelling success   –  hard measures <ul><li>What sort of things might we count ? </li></ul><ul><li>% of support centre us...
Robinson
Bamforth et al (2007)
Bamforth et al (2007)
Bamforth et al (2007)
Dowling & Nolan (DCU) <ul><li>Published in CETL MSOR 2006 Conference Proceedings </li></ul><ul><li>Effectiveness of the Ma...
 
 
Pell & Croft <ul><li>Anecdotally: Some of the students most in need of support fail to access it at all. Many of those who...
Pell & Croft <ul><li>21 students attended >=2 and achieved grade D </li></ul><ul><li>63 out of 74 failing students did not...
Pell & Croft <ul><li>35% of those achieving A* sought maths help more than once </li></ul><ul><li>15% of E Grade Students ...
National Audit Office 2007…. <ul><li>Staying the course: the retention of students in higher education. </li></ul><ul><li>...
There are some things you just can’t measure…. <ul><li>I just wanted to share my good news that after five years of study ...
Upcoming SlideShare
Loading in …5
×

(PowerPoint) Professor Tony Croft, Loughborough University:Towards a culture of data collection & analysis in Mathematics Support Centres.

1,213 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,213
On SlideShare
0
From Embeds
0
Number of Embeds
23
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • (PowerPoint) Professor Tony Croft, Loughborough University:Towards a culture of data collection & analysis in Mathematics Support Centres.

    1. 1. Towards a culture of data collection and analysis in mathematics support centres Professor Tony Croft Director, Mathematics Education Centre, Loughborough University www.sigma-cetl.ac.uk
    2. 3. background
    3. 4. background <ul><li>“ higher education has little option but to accommodate to the students emerging from the current GCE [ie pre-university schooling] process” Smith, 2004, Section 4.39, </li></ul>
    4. 5. Overview <ul><li>Growth in activity </li></ul><ul><li>Difficulties of getting good and convincing data </li></ul><ul><li>What has been written about this ? </li></ul><ul><li>Measuring and modelling success </li></ul>
    5. 6. Caveats <ul><li>Shouldn’t be too fixated on trying to find hard measures of success. </li></ul><ul><li>We are professionals – we recognise the difficulties our students have and respond to the challenges in different ways. </li></ul><ul><li>Mathematics support centres are but one possible way! </li></ul><ul><li>HE institutions have all sorts of support services – counselling, careers, personal tutors, English Language, and we don’t necessarily require these to justify their existence. Why do we have these ? </li></ul>
    6. 7. Why do we have these ? <ul><li>Because we care! </li></ul><ul><li>Because we want to encourage and support our students </li></ul><ul><li>Provision of excellent support is expected in a well-rounded institution, at a time of inclusion and widening participation So in what follows, whilst I focus on collecting data and measuring/modelling success, I would not want to give the impression that I think these things are the most important aspect of what we do! </li></ul>
    7. 8. Continuing growth in UK activity <ul><li>Surveys: </li></ul><ul><li>Demand: When sigma announced a competition for funding to develop new or existing centres, 14 high quality bids were received. </li></ul><ul><li>Sigma has now helped to establish new centres at </li></ul>Beveridge 1993 42 Lawson et al 2001 46 Perkin & Croft 2004 66 Bath http://www.bath.ac.uk/study/mash/ Sheffield http://www.shef.ac.uk/mash/
    8. 9. Around the world <ul><li>Australia: there are 39 universities, of which 32 have some form of Mathematics & Statistics Learning Support. (Helen MacGillivray, QUT, 2008) </li></ul><ul><li>Ireland: 13 institutions with mathematics support provision are reported in the audit of Gill Donoghue & Johnson </li></ul><ul><li>(CEMTL 2008) </li></ul>
    9. 10. Around the world <ul><li>Switzerland’s first….. Opened September 2008 </li></ul>
    10. 11. Evaluation, accountability <ul><li>So with huge and rapid growth of mathematics support activity it is right and proper that we ask… Are our efforts worthwhile, and how would we know ? Can we justify this activity ? </li></ul><ul><li>What data is it possible to collect ? </li></ul><ul><li>Is trying to collect data worth the effort ? </li></ul><ul><li>How can we interpret the data ? </li></ul><ul><li>Modelling </li></ul>
    11. 12. Difficulties of getting data: systems and staff <ul><li>This activity is still very young. </li></ul><ul><li>Drop-in activities were invariably started by concerned teachers who found a few hours in the week to offer additional support to students. </li></ul><ul><li>Staff if they were paid, were paid to tutor not to gather data, and certainly not to analyse it. </li></ul><ul><li>Very little long term strategy – cinderella service wherein staff rarely knew if they would continue to be employed from one year to the next. </li></ul>
    12. 13. Difficulties of getting data: departments and students <ul><li>Departments often reluctant to give out data. </li></ul><ul><li>Cooperation of departments is not always forthcoming. </li></ul><ul><li>Difficulties of accessing the data. </li></ul><ul><li>Moving goal posts from year to year. </li></ul><ul><li>There are many factors which influence students’ success in a mathematics module: curriculum change; recruitment and selection criteria change; staff change; students change!; even things like time of day change! </li></ul><ul><li>There may be lots of students seeking help, but they come from many different courses, with very different needs, so it can be difficult to compare </li></ul>
    13. 14. Difficulties of getting data <ul><li>Sent: 30 September 2008 12:43 To: mlsc@lboro.ac.uk Subject: Maths for Physics Hi, I’ve just started a Physics and Management course and am quite worried about the maths side of it as I am one of the very few if not the only undergraduate that doesn’t have a maths A level . I am willing to work very hard to get the skills needed but with maths tests already coming up for my course I want to get started as soon as possible. What it the best way to go about achieving my goal ? And is there anyone I can speak to about exactly what is needed for the course ? ( I haven’t yet spoken to my maths lecturer but will be seeing him tomorrow). Any help or advice about what’s the best course of action to take would be appreciated as it would be a weight off my mind. </li></ul>
    14. 15. Resource Issues / financial constraints <ul><li>Almost all centres were established on very modest budgets and their purpose was to be up-front helping students. </li></ul><ul><li>“ more than half of centres [in the Australian survey] do not have a full-time person employed” . </li></ul><ul><li>Many staff are sessional only (term only). </li></ul><ul><li>Short term funding. </li></ul><ul><li>Often staff recruited to maths support work do not have any experience of data collection and analysis, and are attracted to the work so that they can help students with their maths. </li></ul>
    15. 16. What has been written on Evaluation ? <ul><li>[Mathematics Learning Support] needs sufficient security to attract, train and retain staff, and to play its part in the ongoing and longitudinal data collection and analysis that should be an integral part of its contribution to the university. All universities should ensure that such data collection and analysis are undertaken and performed correctly to provide vital information for university academic management. However, as reported, few of the facilities currently have the resources to undertake this important work . Helen MacGillivray (QUT, Australia 2008) </li></ul><ul><li>In the audit of Gill et al… measures of success are listed, but [Carlow] there is no definitive way of measuring the effectiveness …messiness of the data.. [DCU]. …LSU’s should have a research and development function that is adequately recognised for funding (ITT Dublin) </li></ul>
    16. 17. Measuring and modelling success <ul><li>I will describe both soft and hard measures. </li></ul>
    17. 18. Measures of success – soft measures <ul><li>Growth in the number of centres </li></ul><ul><li>Usage Data – attendance records, return visits, year of study, department, problem </li></ul><ul><li>Student Feedback Questionnaires – biased and usually positive </li></ul><ul><li>Module Feedback Questionnaires – reach a broader group of students, but often little is said about maths support. </li></ul><ul><li>External Review </li></ul>
    18. 19. Usage Data <ul><li>Student visits </li></ul>
    19. 20. Usage Data <ul><li>Students </li></ul><ul><li>Visits </li></ul><ul><li>Departments </li></ul>
    20. 21. External Review <ul><li>The Institutional Audit report of 2004 records: </li></ul><ul><ul><ul><li>“ The audit team identified the following areas as being good practice in the context of the University: the work of the Mathematics Education Centre...” </li></ul></ul></ul><ul><ul><ul><li>“ Having discussed the work of the MEC with members of staff across the University, the team came to the view that [its] contributions to the University's resources for staff development, and their work more generally, constituted a feature of good practice.” </li></ul></ul></ul><ul><li>http://www.qaa.ac.uk/reviews/reports/institutional/ loughborough2004/Rg063loughboroughUni.pdf </li></ul>The Institutional Audit report of 2008 records: sigma CETL has its origins in the Mathematics Education Centre but has widened its concern from the teaching of mathematics to engineers to include support for mathematics education across the University. The ready accessibility of useful help was praised by both undergraduate and postgraduate students that met the audit team. Other students described the benefits of the support rooms and associated equipment. Postgraduate students were appreciative of the one-to-one help and individual study programmes provided for them by the Centre. http://www.qaa.ac.uk/reviews/reports/institutional/Loughborough08/RG378Loughborough.pdf
    21. 22. NSS <ul><li>National Student Survey 2008 </li></ul><ul><li>Comments relating to “maths support” </li></ul><ul><li>Department: Mathematical Sciences: 2 positive comments referring to MLSC (unspecified) </li></ul><ul><li>Department: Economics Two positive comments citing “excellent maths support” (one) and “superb support in the form of the maths support centre” (one) </li></ul><ul><li>Department: Electronic and Electrical Eng: One positive comment: “the uni is more than equipped to help and bring me up to speed with my maths skills that I may have previously missed or forgotten”. </li></ul>
    22. 23. The student voice <ul><li>video </li></ul>
    23. 24. Modelling success – hard measures <ul><li>Studies with more substance are beginning to emerge which might inform methodology of data collection and analysis: </li></ul><ul><li>Dowling & Nolan </li></ul><ul><li>Patel </li></ul><ul><li>Croft & Pell </li></ul><ul><li>Lee, Harrison, Pell & Robinson </li></ul><ul><li>Parsons </li></ul><ul><li>MacGillivray </li></ul>
    24. 25. Modelling success – hard measures <ul><li>What sort of things might we count ? </li></ul><ul><li>% of support centre users who pass, against % of non-users who pass. </li></ul><ul><li>Mean exam score of users against mean exam score of non-users. </li></ul><ul><li>Pass rates of at-risk students who use the centre against those at risk that don’t. </li></ul><ul><li>Module grades of users. </li></ul><ul><li>Number of users who just scrape a pass. </li></ul><ul><li>Examine failing students to see if they attended the centre. </li></ul><ul><li>Retention rates longitudinally </li></ul>
    25. 26. Robinson
    26. 27. Bamforth et al (2007)
    27. 28. Bamforth et al (2007)
    28. 29. Bamforth et al (2007)
    29. 30. Dowling & Nolan (DCU) <ul><li>Published in CETL MSOR 2006 Conference Proceedings </li></ul><ul><li>Effectiveness of the Maths Learning Centre measured by usage statistics, feedback including interviews. </li></ul><ul><li>Model: pass rates of “at risk” students who did / did not visit the centre were compared over two years. </li></ul>2004/5 2005/6 Number at risk 80 161 Number at risk who visited MLSC 41 95 Pass rate at risk who visited 53% 60% Pass rate at risk who did not visit 25% 49%
    30. 33. Pell & Croft <ul><li>Anecdotally: Some of the students most in need of support fail to access it at all. Many of those who make good use of the centres would pass anyway. </li></ul><ul><li>Analysed five first year engineering maths modules in 2004-5. 644 students. </li></ul><ul><li>Data obtained on their module grade, number of visits to the centre. No access to entry grades. </li></ul>
    31. 34. Pell & Croft <ul><li>21 students attended >=2 and achieved grade D </li></ul><ul><li>63 out of 74 failing students did not attend more than once </li></ul>
    32. 35. Pell & Croft <ul><li>35% of those achieving A* sought maths help more than once </li></ul><ul><li>15% of E Grade Students and 15% of F Grade students sought help more than once </li></ul><ul><li>fail grade students, in addition to having ability problems have attitudinal problems </li></ul>
    33. 36. National Audit Office 2007…. <ul><li>Staying the course: the retention of students in higher education. </li></ul><ul><li>“ A university’s approach to retention should be a positive one and it should provide students with opportunities to improve their grades rather than just addressing gaps in their knowledge” </li></ul><ul><li>The data from Pell would suggest the support centres are already doing this rather well! </li></ul>
    34. 37. There are some things you just can’t measure…. <ul><li>I just wanted to share my good news that after five years of study at Loughborough I managed to gain a first class honours in Product Design and Manufacture. I believe that without the hours you dedicated to the maths learning support centre I would not have been able to pass the maths modules on my foundation year and first year. Your support, patience and encouragement were invaluable when it came to a subject that I had little confidence in when I first arrived at Loughborough. Kathryn </li></ul>

    ×