Data Analyst Role

6,821 views

Published on

Data Analyst Training.

Published in: Education, Technology
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,821
On SlideShare
0
From Embeds
0
Number of Embeds
163
Actions
Shares
0
Downloads
0
Comments
0
Likes
14
Embeds 0
No embeds

No notes for slide
  • It isn’t whether you have a problem, it’s whether you have the same problem again next year.
  • 03/17/10 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. 2008
  • The full TIPS model. Two parts. Implementation of Problem Solving Meeting Foundations Use of the problem solving process (strategy?)
  • 03/17/10 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. 2008
  • This slide is animated to teach the different parts of the meeting minute form each click adds the next section Most schools have the title at the top and write/type as the meeting progresses Make a point that we don’t need to document everything that happened (i.e., NM rolled her eyes KJ entered the room, SW continued to repeat the same issue, we took at 5 minute bathroom break)
  • A completed example… IF a person knows how to use the meeting minute form, the person should be able to pick these minutes up from Jan 7, 2010 and be able to organize previous items to update and facilitate creation of the Feb 3, 2010 agenda
  • 03/17/10 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. 2008
  • OK…. Building precision problem statements is a skill that is needed for using the data. organizing and interpreting the data requires another set of skills. Slides 25-47 provide a sequence of slides to illustrate different precision statements based on different pictures of the same type of data.
  • Let’s talk about accuracy of the data again. When you begin to use the data and draw comparisons, the data need to be comparable. Look at the data above. First, as a data analyst, you look and see, ‘wow.. Things are getting better, the graph is going down’…. Then you do what you are supposed to do first, and look at the label on the Y axis. This label says total office discipline referrals. It is great to compare the total ODRs, but now… look at the X-axis. There are a different number of days in each month and the number of schools day in each much has a wide range (Dec may have 10 school days, January may have 19 school days). These months, the way they are arrayed here, are not comparable and this data should not be used! If you aren’t using SWIS, do the math to get average referrals per day per month by using the total referrals and the total days each month. If you are using SWIS, do not fear….. (next slide)
  • SWIS does that calculation for you. look at the Y-axis label now. Average referrals per day per month allow us to compare months. Now look at the trend….. ‘we are going to have a wild spring term if we don’t do anything differently!). This is the same set of data on the previous slide and look at what the pattern of data does for the problem solving process. Accurate data and data that are formatted for purposes of making decisions is critical. I like to make this a bit dramatic by going back and forth between this slide and previous, telling them they are the team and they are reviewing this data…
  • Build the routine when reviewing these slides: How many students? How many hundreds? What is the per 100 rate for your school? How are you doing? Is there a trend, are there peaks, patterns?; what can we anticipate?
  • Build the routine when reviewing these slides: How many students? How many hundreds? What is the per 100 rate for your school? How are you doing? Is there a trend, are there peaks, patterns?; what can we anticipate?
  • Slides 31-33 work as a set. This slide is a precursor to the next slide to show how to start with ‘this years’ data and then use it to compare to last year (slide 32) Build the routine when reviewing these slides: How many students? How many hundreds? What is the per 100 rate for your school? How are you doing? Is there a trend, are there peaks, patterns?; what can we anticipate?
  • Build the routine when reviewing these slides: How many students? How many hundreds? What is the per 100 rate for your school? How are you doing? Is there a trend, are there peaks, patterns?; what can we anticipate? What happened last year that we don’t want to repeat?
  • Final slide of this set Build the routine when reviewing these slides: How many students? How many hundreds? What is the per 100 rate for your school? How are you doing? Is there a trend, are there peaks, patterns?; what can we anticipate? What are we going to do next fall to maintain this success?
  • Data Analyst Role

    1. 1. Presented by Anne W. Todd, UO & Rick Kirschmann, East County March 15, 2010 and April 1, 2010
    2. 2. <ul><li>“ Who is going to do it? Everyone’s just too worn out.” </li></ul><ul><li>People aren’t tired from solving problems – they’re are tired from solving the same problem over and over. </li></ul>
    3. 3. <ul><li>Authors: </li></ul><ul><ul><li>Rob Horner, Steve Newton, & Anne Todd, University of Oregon </li></ul></ul><ul><ul><li>Bob Algozzine & Kate Algozzine, University of North Carolina at Charlotte </li></ul></ul><ul><li>Background information </li></ul><ul><ul><li>Grant – “Enhancing Data-based Decision Making in Schools” </li></ul></ul><ul><ul><ul><li>Specifically to teach SW PBIS teams to use SWIS data </li></ul></ul></ul><ul><ul><li>Funded by U. S. Department of Education’s Institute of Education Sciences </li></ul></ul><ul><ul><li>4-year project </li></ul></ul><ul><li>Project goals </li></ul><ul><ul><li>Develop assessment instrument to measure decision making & problem solving of PBIS Teams </li></ul></ul><ul><ul><li>Develop problem-solving model (“TIPS”) to enhance data-based decision making </li></ul></ul><ul><ul><li>Conduct initial workshops and do field testing in Oregon & North Carolina </li></ul></ul><ul><ul><li>Conduct 2 rigorous research studies – Does instruction in use of TIPs actually enhance teams’ data-based decision making? </li></ul></ul>
    4. 4. Collect and Use Data Develop Hypothesis Discuss and Select Solutions Develop and Implement Action Plan Evaluate and Revise Action Plan Problem Solving Meeting Foundations Team Initiated Problem Solving (TIPS) Model Identify Problems
    5. 5. <ul><li>Decision making is aided by access to data (“data-based decision making”- DBDM) </li></ul><ul><li>PBIS Team meetings are a major context for DBDM </li></ul><ul><li>Providing instruction on how to embed DBDM in a problem-solving model (TIPS) will result in problem solving that is </li></ul><ul><ul><li>Thorough </li></ul></ul><ul><ul><li>Logical </li></ul></ul><ul><ul><li>Efficient </li></ul></ul><ul><ul><li>Effective </li></ul></ul><ul><li>Structure of meetings lays foundation for efficiency and effectiveness </li></ul>
    6. 6. Collect and Use Data Develop Hypothesis Discuss and Select Solutions Develop and Implement Action Plan Evaluate and Revise Action Plan Problem Solving Meeting Foundations Team Initiated Problem Solving (TIPS) Model Identify Problems
    7. 7. School A School B School C School D Meeting Foundations Score Baseline Coaching TIPS % DORA Foundations Score
    8. 8. School A School D School C Baseline Coaching TIPS Thoroughness of decision-making % DORA Thoroughness Score
    9. 9. <ul><li>Data analyst role is critical </li></ul><ul><li>We all need back up plans </li></ul><ul><li>Team training and coaching are critical </li></ul><ul><li>Training critical skills to facilitator, minute taker, and data analyst </li></ul><ul><li>Documenting decision, actions, timelines, evaluation plan is critical for sustainability </li></ul><ul><li>Launch the meeting with a data summary </li></ul>
    10. 10. <ul><li>How? </li></ul><ul><ul><li>Establish the role of a data analyst (and backup person) </li></ul></ul><ul><ul><li>Teach data analyst to develop data summary </li></ul></ul><ul><ul><ul><li>Oakes, DIBELS, SWIS…. Etc </li></ul></ul></ul><ul><ul><li>Start meeting with defining the problem with precision </li></ul></ul><ul><ul><li>Refine precision of problem statement through inferences and hypothesis </li></ul></ul><ul><ul><ul><li>Have data accessible for custom report generation during the meeting </li></ul></ul></ul>
    11. 11. Structure of meetings lays foundation for efficiency & effectiveness
    12. 12. <ul><li>We want to walk into a meeting having no previous history, review the previous meeting minutes be able to fit into any role needed </li></ul><ul><ul><li>Facilitator </li></ul></ul><ul><ul><li>Minute taker </li></ul></ul><ul><ul><li>Data analyst </li></ul></ul><ul><ul><li>Active team member </li></ul></ul><ul><li>Example…… </li></ul>
    13. 13. <ul><li>Documentation of </li></ul><ul><ul><li>Logistics of meeting (date, time, location, roles) </li></ul></ul><ul><ul><li>Agenda items for today’s meeting ( and next meeting) </li></ul></ul><ul><ul><li>Discussion items, decisions made, tasks and timelines assigned </li></ul></ul><ul><ul><li>Problem statements, solutions/decisions/tasks, people assigned to implement with timelines assigned, and an evaluation plan to determine the effect on student behavior </li></ul></ul><ul><li>Reviewing Meeting minutes </li></ul><ul><ul><li>An effective strategy for getting a snapshot of what happened at the previous meeting and what needs to be reviewed during the upcoming meeting </li></ul></ul><ul><ul><ul><li>What was the issue/problem?, What were we going to do?, Who was going to do it and by When?, and How are we measuring progress toward the goal? </li></ul></ul></ul><ul><li>Visual tracking of focus topics during and after meetings </li></ul><ul><ul><li>Prevents side conversations </li></ul></ul><ul><ul><li>Prevents repetition </li></ul></ul><ul><ul><li>Encourages completion of tasks </li></ul></ul>
    14. 14. Langley Elementary PBIS Team Meeting Minutes and Problem-Solving Action Plan Form Today’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today) Administrative/General Information and Issues Problem-Solving Action Plan Evaluation of Team Meeting (Mark your ratings with an “X”) Today’s Agenda Items Next Meeting Agenda Items 01. 02. 03. Information for Team, or Issue for Team to Address Discussion/Decision/Task (if applicable) Who? By When? Implementation and Evaluation Precise Problem Statement, based on review of data (What, When, Where, Who, Why) Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety) Who? By When? Goal, Timeline, Decision Rule, & Updates Our Rating Yes So-So No 1. Was today’s meeting a good use of our time? 2. In general, did we do a good job of tracking whether we’re completing the tasks we agreed on at previous meetings? 3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings? 4. In general, are the completed tasks having the desired effects on student behavior?
    15. 16. <ul><li>Regular meetings & regular attendance </li></ul><ul><li>The “right” people </li></ul><ul><li>The right roles </li></ul><ul><ul><li>Facilitator </li></ul></ul><ul><ul><li>Minute Taker </li></ul></ul><ul><ul><li>Data Analyst </li></ul></ul><ul><ul><li>Active Team Members </li></ul></ul><ul><li>Accomplishments – Products of successful meeting </li></ul><ul><ul><li>Meeting Minutes (record of decisions & tasks concerning administrative/general issues) </li></ul></ul><ul><ul><li>Problem-Solving Action Plan (record of decisions & tasks concerning problems identified by team) </li></ul></ul><ul><ul><li>(We’ll discuss these in more detail later in this workshop) </li></ul></ul>
    16. 17.
    17. 18.
    18. 19. From the data analysts summary report
    19. 20.
    20. 21. <ul><li>Role </li></ul><ul><ul><li>To create data summaries that will facilitate the team in </li></ul></ul><ul><ul><ul><li>determining if there are problems </li></ul></ul></ul><ul><ul><ul><li>jump starting a problem solving discussion, and </li></ul></ul></ul><ul><ul><ul><li>evaluating the impact of solutions and fidelity of implementation </li></ul></ul></ul><ul><li>General Responsibilities </li></ul><ul><ul><li>Prepares a brief written summary for distribution at meetings using each of the data sources needed for problem solving and decision making </li></ul></ul><ul><ul><li>Help to generate reports during the meeting as questions of the data arise </li></ul></ul>
    21. 22. <ul><li>Advise Backup team member in advance if unable to attend meeting, </li></ul><ul><li>Create data summaries to report </li></ul><ul><ul><li>status of solutions/decisions from previous meetings </li></ul></ul><ul><ul><li>Potential New Problems (if any) </li></ul></ul><ul><ul><li>Asks the Facilitator to add Potential New Problems to agenda for upcoming meeting </li></ul></ul><ul><li>Bring Data notebook to meeting </li></ul>
    22. 23. Problem-Solving Action Plan Creating a summary report for the status of a previous problem 1. check previous meeting minutes 2. review goal for solving the problem 3. create data summary Implementation and Evaluation Precise Problem Statement, based on review of data (What, When, Where, Who, Why) Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety) Who? By When? Goal, Timeline, Decision Rule, & Updates Many instances of disruption occurring in cafeteria between 11:30am and 12:00pm; large majority involving 6 th graders, particularly Student #10 Because (a) cafeteria overcrowded at that time, (b) sixth graders have received insufficient instruction in cafeteria expectations, and (c) disruption results in attention from adults and peers. Determine whether change in lunch schedule is possible. Reach agreement on wording of cafeteria expectations; create and post signage in cafeteria Demonstrate/teach/review expectations in 6 th grade classrooms in period before lunch; continue for at least one month; remind students of “Friday 5” prior to lunch period; remind students not to pay attention to disruptive students Determine whether we can add a cafeteria supervisor to be on duty between 11:30am and 12:00pm Initiate Friday 5; ensure Friday 5 in effect for remainder of school year Check on Behavior Support Program for Student #10 Aparicio All; Wynn All 6 th grade teachers (Aparicio to observe classrooms, see if teachers remember) Aparicio All (Wynn to observe cafeteria, see if F5 in effect) Ortiz 02/15/2008 02/15/2008; 02/22/2008 Beginning 02/22/2008 02/15/2008 Reduce disruptions in cafeteria between 11:30am and 12:00pm to no more than 6 per calendar month; maintain for remainder of school year.
    23. 24. <ul><li>Presents overview of findings from review of current data and initiates discussion of: </li></ul><ul><ul><li>Identification of new problems </li></ul></ul><ul><ul><li>Status and effectiveness of currently implemented solutions, especially as compared against team’s goal, timeline, and decision rule for a targeted problem </li></ul></ul><ul><li>Creates SWIS reports based on questions that come up in the meeting </li></ul><ul><li>Makes data available to team members via projection or hard copies </li></ul><ul><li>Is an active participant in meeting </li></ul>
    24. 25. <ul><li>Complete any assigned tasks </li></ul><ul><li>Make a note in calendar to prepare data summary for next meeting </li></ul>
    25. 26. <ul><li>Fluency in navigating through the database to generate reports and to create more customized reports based on generated questions. </li></ul><ul><li>Access to database before meetings and during the meeting </li></ul><ul><li>Can copy, paste and size images into a word or powerpoint document </li></ul>
    26. 27. <ul><li>A person or two with dedicated time for meeting the responsibilities </li></ul><ul><ul><li>A person with some time (~30 minutes) to generate data summaries before the meeting </li></ul></ul>
    27. 28. <ul><li>Data Entry </li></ul><ul><ul><ul><ul><li>School Address and Contact </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Enrollment/Ethnicity/Days per month </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Staff Information </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Student Information </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Referrals </li></ul></ul></ul></ul><ul><li>Reporting </li></ul><ul><ul><ul><ul><li>Average Referrals per Day per month </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Referrals by Problem Behavior </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Referrals by Location </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Referrals by Time </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Referrals by Student </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Other Reports </li></ul></ul></ul></ul><ul><li>Tools </li></ul>
    28. 30. <ul><li>Universal Screening Tool </li></ul><ul><ul><li>Proportion of students with </li></ul></ul><ul><ul><ul><li>0-1 Office Discipline Referrals (ODRs) </li></ul></ul></ul><ul><ul><ul><li>2-5 ODRs </li></ul></ul></ul><ul><ul><ul><li>6+ ODRs </li></ul></ul></ul><ul><li>Progress Monitoring Tool </li></ul><ul><li>Compare data across time </li></ul><ul><ul><li>Prevent previous problem patterns </li></ul></ul><ul><li>Define Problems with precision that lead to solvable problems </li></ul>
    29. 32. ~80% of Students ~15% ~5% 0-1 office discipline referral 6+ office discipline referrals 2-5 office discipline referrals Using office discipline referrals as a metric for universal screening of student social behavior
    30. 33. Total Office Discipline Referrals Total Office Discipline Referrals as of January 10
    31. 35. Grade Range Number of Schools Avg. Enrollment per school National Avg. for Major ODRs per 100 students , per school day K-6 2,162 450 .34 = about 1 Major ODR every 3 school days, or about 34 every 100 days 6-9 602 657 .85 = a little less than 1 Major ODR per school day, or about 85 every 100 days 9-12 215 887 1.27 = more than 1 Major ODR per school day, or about 127 every 100 days K- (8-12) 431 408 1.06 = about 1 Major ODR per school day, or about 106 every 100 days
    32. 36. Elementary School with 150 Students Compare with National Average 150 / 100 = 1.50 1.50 X .34 = .51
    33. 37. Elementary School with 450 Students Compare with National Average 450 / 100 = 4.50 4.50 X .34 = 1.53
    34. 38. Application Activity: Absolute Value Is there a Problem? Middle School of 625 students? Compare with national average: 625/100 = 6.25 6.25 X .85 = 5.31 Office Discipline Referrals per School Day
    35. 39. High School of 1800 students High School: Compare with National Average 1800 / 100 = 18 18 X 1.27= 22.86
    36. 40. High School of 450 students High School: Compare with National Average 450 / 100 = 4.5 4.5 X 1.27= 5.17
    37. 41. Middle School of 700 students
    38. 44. <ul><li>Our average Major ODRs per school day per month are higher than national average for a school of our enrollment size </li></ul><ul><li>Our average ODRs per school day per month are higher this year than for corresponding months of previous year </li></ul><ul><li>Our average ODRs per school day per month are showing an increasing trend </li></ul><ul><li>Faculty, parents, and students say our ODR levels are too high </li></ul>
    39. 45. <ul><li>Define problem by identifying W hat problem behaviors are involved in ODRs </li></ul><ul><li>Clarify problem by identifying </li></ul><ul><ul><li>When ODRs are occurring (time of day) </li></ul></ul><ul><ul><li>Where ODRs are occurring (location) </li></ul></ul><ul><ul><li>Who is engaging in problem behaviors that result in ODRs </li></ul></ul>
    40. 46. <ul><li>Ultimately, you want to write a “problem statement” that precisely defines the problem you identified </li></ul><ul><li>The more Ws (what, when, where, who) you incorporate into the problem statement, the more precise the problem statement will be </li></ul><ul><li>The more precise the problem statement, the easier it will be to generate a solution that “fits” the problem </li></ul>
    41. 47. 1a. Too many ODRs 1b. Too many instances of disrespect 2a. Too many ODRs between 1:00pm and 1:30pm 2b. Too many ODRs in the afternoon 3a. Too many ODRs occurring outside the classrooms 3b. Too many ODRs on the playground 4a. 25% of students have at least 2 ODRs 4b. Many students are experiencing ODRs 5a. Too many ODRs on the playground 5b. Total of 12 aggression ODRs on playground last month; more than last year & showing increasing trend this year; occurring during first recess; 8 different students involved
    42. 49. <ul><li>Are there lots of different types of problems or just a few? </li></ul><ul><li>How do the problem behaviors fit with the SW expectations? </li></ul><ul><li>What other questions do these data pose? </li></ul>
    43. 50.
    44. 51. Note that you can request a Table as well as a Graph
    45. 52. Are there lots of different types of problems or just a few? How do the problem behaviors fit with the SW expectations? What other questions do these data pose?
    46. 53. <ul><li>Are there lots of different times when problems occur or a period(s) of time when problems occur? </li></ul><ul><li>How do the times fit with our daily schedule? </li></ul><ul><li>What other questions do these data pose? </li></ul>
    47. 54.
    48. 55.
    49. 56. <ul><li>Are there lots of different times when problems occur or a period(s) of time when problems occur? </li></ul><ul><li>How do the times fit with our daily schedule? </li></ul><ul><li>What other questions do these data pose? </li></ul>
    50. 57. Many locations or just a few?
    51. 58.
    52. 59.
    53. 60. Many locations or just a few? What other questions do these data pose?
    54. 61.
    55. 62.
    56. 63.
    57. 64.
    58. 65. <ul><li>Be able to navigate through SWIS </li></ul><ul><li>Be able to generate SWIS reports </li></ul>
    59. 66. <ul><li>Middle School – Grades 6, 7, & 8 </li></ul><ul><li>565 students </li></ul>
    60. 67.
    61. 68. <ul><li>Identified problem </li></ul><ul><ul><li>for last 4 mos., Major ODRs per day higher than national avg. </li></ul></ul><ul><ul><li>increasing trend across all 5 mos. </li></ul></ul>
    62. 69. Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.)
    63. 70.
    64. 71. <ul><li>The sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention. </li></ul><ul><li>A smaller number of students engage in skipping and noncompliance/defiance in classes, (mostly in rooms 13, 14 and 18), and these behaviors appear to be maintained by escape. </li></ul><ul><li>Work on one problem at a time… prioritize </li></ul>
    65. 72. ~80% of Students ~15% ~5% 0-1 office discipline referral 6+ office discipline referrals 2-5 office discipline referrals Using office discipline referrals as a metric for universal screening of student social behavior
    66. 76. Plan Change 2/10/2010: Check out with preferred adult
    67. 78. Brian Bender SWIS Report generated 3/10
    68. 80. <ul><li>Create two data summaries before April 1 training </li></ul><ul><li>Become fluent in navigating through SWIS </li></ul><ul><ul><li>Review Swift at SWIS User’s Manual </li></ul></ul><ul><ul><ul><li>Complete reporting activities on pages 47-48 </li></ul></ul></ul><ul><li>Bring a data summary to April 1 to share with group </li></ul><ul><ul><li>Define primary problem </li></ul></ul><ul><ul><li>Define problem with precision </li></ul></ul>

    ×