Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Assessment in SEBD schools
Similar to Assessment in SEBD schools (20)
Recently uploaded
Recently uploaded (20)
Assessment in SEBD schools
- 1. An investigation into the use of assessment materials in schools for pupils with Social, Emotional and Behavioural Difficulties An online survey of current practice Gavin Weedy: Teacher at Goldwyn School, Ashford, Kent
- 2. Why bother with this survey?  “Without the careful identification, classification, and selection that should be a part of good assessment, social behaviour interventions are likely to be haphazard and disorganized at best and ineffective at worst. Clearly, practitioners and researchers who are serious about understanding and dealing effectively with children’s social skills and related social– behavioral issues must provide the proper foundation for their work through carefully planned assessment activities.” (Merrell, 2001)
- 3. My role ď‚— As a practitioner I am directly involved in the assessment of pupils with SEBD in a school ď‚— I am in contact with many colleagues in other settings ď‚— Driven by providing the best outcomes for the pupils and thence the school. ď‚— I am interested in spreading the word about best practice
- 4. Methodology of survey  Using a Critical Realist paradigm:  Looks for causal mechanisms rather than social laws – the same causal powers can generate different outcomes  Allow for scope for change as research develops  Allows for both a Quantitative approach, such as “How many schools are using X, Y or Z assessment?”  And a Qualitative approach, such as “Why are they using X, Y or Z assessment?”
- 5. Design of the questionnaire used in the survey  Trialled with colleagues  10 simple questions – some with “yes” or “no” answers to get basic quantitative data  Others with scope for more in depth to generate qualitative answers  Quick to complete  Online for ease of completion  Potential for computer analysis
- 6. How was the survey carried out? ď‚— Online survey of day schools, residential schools and Pupil Referral Units. ď‚— Initial introductions via Kent Special School Heads group ď‚— Introductions via colleagues such Educational Psychologists ď‚— Word of mouth through attendance at SEBDA conferences ď‚— Known colleagues in other settings
- 7. Issues with data collection  Replies needed on a timescale – perhaps missed out some potential respondents ( though the research is ongoing after the thesis is submitted)  Small scale study of invited participants. Not truly representative.  Contentious wording of “assessment”  Use of alternative language - “behaviour profile”
- 8. Data analysis methods ď‚— Online via SurveyMonkeyTM ď‚— Using AtlasTI qualitative data analysis programme
- 9. Data generated  A total of 20 establishments or individuals were asked       to complete the survey 11 respondents (in the timeframe) Represents 55% compared to average of 31-34% for online survey requests Tables of responses created and analysed Qualitative responses coded and used in code families to establish any links such as “is associated with” “is a property of”
- 10. Questionnaire results ď‚— Q1. When a new pupil is admitted to your provision, or soon after, do you carry out some form of assessment? Could you please indicate if you use pre-intake testing.
- 11. ď‚— Q2. If you do, do you use a named assessment tool(s)?
- 12. Action outlined  Q3. If you answered "Yes" to question 2, which assessment tool(s) do you use? Please outline for what purpose it is used. Reading Spelling Sentence comprehension Literacy (generic) Numeracy National Curriculum (NC) levels Speech and language screening Behaviour specific Pupil attitudes Specific learning difficulties Achievement Ability Resiliency Psychological reasons Solution focussed questions Intelligence Frequency of action in sample 3 2 2 Comment 1 2 2 1 3 2 1 1 1 1 2 1 1 One developed “in house” One for attachment disorders One for behaviour readiness for reintegration SnAP scale
- 13. Named assessment tool and its function respondent 1 Named assessment WISC-IV purpose 7 WRAT 4 - The Wide Range Measures an individual's ability Achievement Test Clinical tool for measurement to read words, comprehend sentences, spell, and of learning disabilities and attention compute solutions to maths problems. disorders 1 WIAT-II Assessment of reading, language and numerical attainment 1 Resiliency Scales 7 Num NFER National Identify and measure core personal Foundation for Educational qualities of resiliency in youth aged 9:0 Basic numeracy Research years to 18:11 years. 1 Solution Focussed Designed to help people explore their Questions 7 Measurement of a pupil’s attitudes towards and school strengths and resources rather than PASS – pupils attitude to self themselves as learners and their attitudes towards school, concentrate on their problems and deficits. 7 4 SNAP Special Needs Assessment Profile Test for different forms of intelligence Aimed at looking at behaviour relating to Specific Learning Difficulties WASI - Wechsler Boxall Profile Diagnostic Assessment and Profiling for attachment issues. 4 8 The Muntham Wheel “ for monitoring and recording progress within the Abbreviated Scale of areas of classroom conformity, task orientation and Intelligence the SEAL outcomes.” 4 Beck Youth Inventory Testing for psychological profiling 4 WRAT 4 - The Wide Measures an individual's ability Range Achievement to read words, comprehend sentences, s Test pell, and compute solutions 8 McSherry behaviour Determine readiness for pupils with SEBD to return readiness to integrate scale back into mainstream settings 9 STAR rating scale Not specified 12 CAT Cognitive Abilities Tests Test of reasoning ability to maths problems. 6 PASS – pupils attitude Measurement of a pupil’s attitudes to self and school towards themselves as learners and their attitudes towards school,
- 14.  Q4. If you named a particular assessment tool(s), can you briefly outline why you chose that one. Reason given Staff ability Suggestion from Educational Psychologist (EP) Offers best information Specific test reason “we built it” Not named “It’s very good” “best we can find” “mirrors school ethos of care” Frequency of action in sample 1 1 Comment Staff trained 1 2 1 1 1 1 1 Resiliency scales – can be built upon for intervention McSherry – good for reintegration
- 15. ď‚— Q5. Do you have a behaviour monitoring system?
- 16. ď‚— Q6. If you have a behaviour monitoring system, do you use it to directly inform your intervention strategies. For example, does the data generate "if this occurs"..."then this happens" type of intervention. Could you please give an example if it does.
- 17.  Q7. If you have a behaviour monitoring system, can you briefly explain how you track intervention strategies. Action outlined Daily recording Recorded weekly Recorded other Recorded each half term Recorded each term Recorded electronically Recorded on paper system Recorded method unspecified Use of credit / points system Recording of specific intervention Use of target setting Specific use of term “review” Mention of specific intervention measure Use of dedicated staff Frequency of action in sample 6 3 1 2 Comment Three weekly record 1 2 1 7 4 2 2 5 2 2 1 of IEP (individual education programme 1 of IAP (individual action plan) 1 of Learning mentor team 1 of “associated staff”
- 18.  Q9. If you have a behaviour monitoring system, can you briefly explain how you use the data it generates. For example, reporting to parents or showing progression, etc Action outlined Reporting to parents End of term reports Showing progress specifically Annual reviews Changing targets Inform student directly Inform key working Pupil rewards Contributes to whole school data Weekly case management Reports to governors Reports to Ofsted / DFE Other Frequency of action in sample 5 2 2 Comment Using coloured wheel 3 4 1 1 2 1 1 2 1 4 “informs diary writing” “Inform CPD” (continuing professional development) “Plan more individualised interventions” “Graphs of types of behaviours”
- 19.  Q8. How do you know your behaviour intervention strategies are achieving the desired outcome? Action outlined Reduction in incidents / physical interventions Frequency of action in sample 5 Comment Increase in appropriate behaviours 1 Not specified what “appropriate” are. Raised achievement / attainment / learning Behaviour points Behaviour review meetings Pupil progress meetings 4 Includes “qualifications” Parental reports of improvements Improvement in scaling 1 Movement in targets Attendance Exclusions Make progress 1 2 1 1 Other 2 1 1 1 1 Attendance is raised Unspecified – “children make progress” “when we have understanding and balance” “unsure”
- 20. ď‚— Q10. Please feel free to add any comments about SEBD provision and assessment.
- 21. Conclusions In this survey it is clear that SEBD schools do use assessment widely but seemingly not for specific behavioural issues. This seems a little unusual but in discussions with colleagues there would appear to be a simple lack of knowledge or indeed the confidence to administer this type of testing. If schools are able to rigorously demonstrate the pupils’ level of emotional and behavioural difficulty, and subsequent impedance on learning thereof, they will be able to go some way to defending or celebrating the academic progress made by pupils. I will be conducting further research specifically into the area of behavioural assessment and hope that by posting links to the survey on the websites of professional associations I will gain a more in depth picture of what is currently being used.
- 22. References            Ballantyne C (2005), Moving student evaluation of teaching online: reporting pilot outcomes and issues with afocus on how to increase student response rate. Paper presented at the 2005 Australasian EvaluationsForum: University Learning and Reaching: Evaluating and Enhancing the Experience, UNSW,Sydney, 28–29 November. Bhaskar, R. 1998a. "General introduction", in M. S. Archer, R. Bhaskar, A. Collier, T. Lawson, and A. Norrie, eds., Critical realism : essential readings. London: Routledge, pp. ix-xxiv Creswell, J.W.& Plano Clark V.L.(2011) Designing and Conducting mixed methods research. Thousand Oaks, CA: Sage Publications, Inc. Dommeyer, C.J., P. Baum, R.W. Hanna, and K.S. Chapman. (2004). Gathering faculty teaching evaluations by in-class and online surveys: their effects on response rates and evaluations .Assessment & Evaluation in Higher Education 29, no. 5: 611–623. Elliott, S. N., Gresham, F. M., Frank, J. L., & Beddow, P. A. (2008). Intervention Validity of Social Behavior Rating Scales: Features of Assessments That Link Results to Treatment Plans. Assessment for Effective Intervention, 34(1), 15–24. Krauss, S. E., & Putra, U. (2005). Research Paradigms and Meaning Making  A Primer, The Qualitative Report 10(4), 758–770. : Merrell, K. W. (2001). Assessment of Children’s Social Skills: Recent Developments, Best Practices, and New Directions. Exceptionality, 9(1-2), 3–18. Nair, C.S., C. Wayland, and S. Soediro. (2005). Evaluating the student experience: a leap into the future.Paper presented at the 2005 Australasian Evaluations Forum: University Learning and Teaching:Evaluating and Enhancing the Experience, UNSW, Sydney, 28–29 November Nulty D.D (2008): The adequacy of response rates to online and paper surveys: what can be done? Assessment & Evaluation in Higher Education Vol. 33, No. 3, June 2008, 301–314 Ogier, J. (2005). The response rates for online surveys—a hit and miss affair. Paper presented at the 2005. Australasian Evaluations Forum: University Learning and Teaching: Evaluating and Enhancing theExperience, UNSW, Sydney. 28–29 November Watt, S., C. Simpson, C. McKillop, and V. Nunn (2002). Electronic course surveys: does automating feedback and reporting give better results? Assessment & Evaluation in Higher Education 27, no. 4:325–337
Editor's Notes
- The audience for the presentation is made up of the whole school staff and the lead governor responsible for assessment and monitoring. The SMT had asked me to look into this because of my interest in nurture groups and their use of the Boxall profile. I suggested that I felt it would be useful to us as a school that has a very nurture oriented ethos. In view of this I decided to look at what other schools were using. Also this presentation is going to form the basis of reporting back to our National Body - the Social, Emotional and Behavioural Difficulties Association (SEBDA) at the annual conference in May 2014. By which time I will have much more data.
- The merits of this quote were discussed and I mentioned that given his track record of academic prowess it was probably a good starting point. I suggested that it was a very difficult case to argue against and used our own example of using PASS data for the last 5 years which shows whole school progression, though in a less rigorous way than I would like, which is my point of doing the study. I told them I was looking for evidence of something more concrete.
- I have been at the school for 8 years and have over 28 years experience in total. Having helped many of the staff with various projects and been mentor to some I believe I have a certain level of professional respect from my peers. Also I regularly attend association conferences and have presented before Nationally with a project demonstrating the use of opera and classical music to aid development and learning in SEBD pupils.
- Brief explanation of Critical Realism: “theoretical explanation proceeds by description of significant features, retroduction to possible causes, elimination of alternatives and identification of the generative mechanism or causal structure at work” (Bhaskar 1998a: xvii).Some discussion was needed to explain the difference between qualitative andquantitative data for some of our more junior members of small but I was able to relate it to our Individual Education Plans and how they work with numbers and general statements about behaviour.It was a real shame I could not get the interviews done in time for this because that would have generated further qualitative data. However, I would be wary of using only two people for any generalisations based on their testimony.
- In line with my RQ’s I outlined how the questions needed both quantitative and qualitative data to get what I was looking for and how a mixed methods approach (Creswell 2011) fitted in with the Critical Realism paradigm as outlined by Krauss (2005)
- I duly went through the process of applying for ethical approval from the University giving assurances that that there were no particular problems arising from the data collection in terms of vulnerability as the material is not sensitive, relating as it does to generic methods of assessment rather than specific individuals. Also that considerations were made of anonymity.The sample group is one of professionals who were invited to participate because they had some professional interest in the outcomes and were in some way involved in the assessment process in their establishment. Indeed one respondent said “ If you develop something good from your research I would love to see it” (respondent 5)
- When this survey is carried out next time it will be refined to ask questions specifically about behavioural assessment to filter out the responses I am not interested in such as reading, numeracy or similar. The contributors will be much more random as the access to the weblink for the questionnaire will be public though hosted on a couple of specialist websites. Also I have now got access to a full version of SurveyMonkeytmand so have much better potential for online analysis.
- These diagrams were surprisingly readable when put on a white board and nicely illustrated the comparison between Qualitative and Quantitative data analysis with regard to a Mixed-methods approach as suggested by Krauss (2005) . The simple yes / no questions generated the graphs and the others were analysed using AtlasTI by assigning codes to words / phrases. These codes were then grouped into families of similar concepts. Because the relationships between codes can be seen it offers some insight into the links between the text that generated them.
- I was happy that there was a good response from my requests. I was glad to see that 55% of those asked to complete the survey did so as according to University of Texas, Instructional Assessment Resources website, a figure of about 30% is usual (University of Texas 2011) this is further backed up by Watt et al (2002) 32%, Naire et al 31% and Ogier (2005) 30%. Though higher figures such as 43% (Dommeyer et al 2004) and 47% (Ballantyne 2005) are quoted, the figure of 55% stands up well even in a small survey. (in Nulty 2008)However, the data sample is too small even if everybody responded, to draw any conclusions that one might be able to apply to a general population. The next survey will be open to the general population ( or at least a specialised population) via Association websites and this should allow for more impact of the data. This was always to be considered as a pilot and it has been an extremely useful exercise in that respect.The slides that follow are ways of showing the data obtained in a quantitative manner. This was achieved by looking at numbers of code occurrences or themes.
- This is a little surprising that some respondents (20%) did not do any assessment. I think this relates to the notion of testing as outlined in the reflective report and that it is a question of recognising what is and what is not “assessment”. I would argue that any type of judgement about a behavioural function is a form of assessment.
- The only surprise here is that some people use no named assessment. That I was not expecting but realised that if an establishment developed it’s own assessment regime it was likely to use it if it felt it delivered all they needed. This could be due to the idea that people would have more faith/confidence in a system they have collaborated in the drawing up of.
- I pointed out that I was not surprised at the wide range of assessment tools but was surprised at the lack of specific behavioural ones. Once again this would only have real significance in a much larger sample and pointed this out to the audience. I suggested that as many of these are tests in some way it was perhaps more in the psyche of the respondent to easily interpret them as “assessments”. Perhaps there would have been more of they had believed their daily records constituted assessment. Perhaps I could have clarified this but definitely will be looking at data specific to behavioural assessment / measurement / profiling. The wording has yet to be decided.
- Although a little small I was able to zoom in using a remote mouse and highlight those named tools that were of interest. For me this was PASS, McSherry and Boxall. A discussion ensued relating to the compound use of PASS and Boxall. I feel this is the way to go for our establishment. I highlighted the WRAT 4 and PASS as it is one of only two with any commonality.
- I pointed out that the range of reasons in no way showed any kind of pattern. The more arbitrary comments were illuminating and I suggested that the response “we built it” was a comment of someone who believed very strongly that they had the right material for them. I would like to have had more to go on here but it’s diversity was perhaps a commentary of the general idea of assessment?
- There was no surprise here. The actual figure is really 100% in my view, as what appears to be negative was from an E.P. who comments about the whole school monitoring that they deal with rather than monitor individuals.
- This was a very easy to slide to go through though there was some discussions as to what I meant by “directly”. I gave the example of how we use our system to interventions e.g. if a child is consistently late for something, our system will flag it up automatically and then one of the support staff will be put in place to escort a pupil between lessons until they can do it alone and be on time. This is a good example of how monitoring can directly inform intervention.
- Most commonly respondents did not mention how they specifically recorded their data (70%). Is this relevant? Well only in as much as those who record electronically, I believe, will have a much better ability to manipulate the data with ease to look for trends /patterns than poring over piles of paper in long term or multiple interventions.Reviews are statutory and as such material relating to behaviour must be provided so the figure of 5 for frequency is only surprising because it is quite low in my opinion.
- Perhaps the low amount of “end of term reports” reflects the traditional approach of schools in reporting subject attainment rather than focus on behaviour. This is to me an interesting notion and one I believe stems from a perception of the need to create Ofsted and Local Authority friendly data that can demonstrate school efficacy via league tables. Personally I don’t think it is a very healthy route, but understandable nevertheless.
- No specific mention of quality of reporting. This, I believe, is important because a child can have a relatively low number of incidents, say 8, but against a baseline of none or very few, represents a dramatic change in externalising behaviour from those with a consistently high level. One needs to make sure that the recording allows for such patterns to emerge. The idea of Raised achievement / attainment / learning is also easier to demonstrate and the testing of NC levels, reading, spelling etc have been used to demonstrate this. However, nobody has tried to establish a link between the behavioural data helping to ameliorate the learning.
- Comments were made by the audience in agreement with one of the respondents that there was too little time to carry out any more assessments. I pointed out that it was “instead of” not alongside the current practice that I was looking at. There was a consensus of opinion that good assessment was a desirable thing but that the recording of progress is difficult because of the format of the assessment. In that if this data was to be used for Ofsted then there might be a desire to skew the data and this would be easy to achieve as judgements are somewhat simplistic. This broadly concurs with the findings of Elliot et al (2008 p.16) “Ratings of behaviour are evaluative judgments that are affected by the environment and a rater’s standards for behaviour.”