Download to read offline
More Related Content
Similar to Naf powerpoint july9-2010_v5
More from NAFCareerAcads
Naf powerpoint july9-2010_v5
- 1. Student Voices &Data-Driven Decision Making Improving Your AcademyDr. Ann G. Bessell (University of Miami)Dr. Cathleen Armstead (University of Miami)National Academy Foundation Staff Development InstitutePhiladelphia, PA. July, 2010
- 2. AgendaOverview and PurposeInnovativeTechniques: PhotolanguageDoing PhotolanguageExploring Data-in-a-DayAnalyzing data setsUnderstanding DataMaking Data-Driven Decisions
- 3. Overview & PurposeToIntroduce Innovative Techniques for Self-Assessment & Evaluation – Photolanguage & Data-in-a-DayTo Explore and Practice Analyzing Data for Decision-Making Purposes
- 4. PHOTOLANGUAGEInnovative process usingblack and white photographs chosen for their: Aesthetic qualities
- 5. Capacity to stimulatethe imagination, memory and emotions
- 6. Ability to challengethe viewer for thoughtful reflection
- 7. 15Arrange PhotosShare Selections & Significance234PHOTOLANGUAGEPresent Focus QuestionView PhotosSelect Photos
- 8. Question#1Choose a picturethat best represents your image of a NAF student when compared to the general population of high school students.Choose a picture that best represents your image of a NAF teacher or leader when compared to the general population of high school teachers.Question#2
- 9. Challenges & ObstaclesLackof Longitudinal DataDifferent Data PointsMeasuring OutcomesLack of Integrated Data SystemWhat’s NextLack of Comparable DataTeachers: Not ResearchersFear of RetributionClass Time
- 10. Sidney Harris's cartoondemonstrates how to get past those pesky detail steps
- 11. Quick & EfficientImproveSchoolsParticipatoryData-in-aDayObserve multiple classrooms in a single day (20-30 Minutes)
- 12. Collect datafrom multiple sources - surveys - questions - observations -
- 13. Involve multiplestakeholders: Input from students, faculty & administrators
- 14. Facilitate a dialogueabout school improvement strategies (Ginsberg & Kimball, 2008)
- 15. MostImportantValueListeningTeamworkEncourages Critical ReflectionImportantAspectsStudent InputPromotes DialogueRepresentative Teams
- 16. Our GoalDesiredCourseWe areHereActualCourseWhere are we?What is our goal?What course should we follow?How are we doing?2002Assessment Committee12
- 17. Principal InvolvementTeachers &NAF Lead Teachers InvolvementGuidance CounselorsExplain the processEncourage participation on each teamReview scheduleFrom the Start
- 18. Classroom ObservationsReview checklistwith district and principals at beginning of yearReview with principals and teachers at the meet and greet in morning10-15 minutes of classroom observation
- 21. Time to Practice!(c)2002 Good & Kaminski
- 22. 15Review data setsand look for patternsReview & Summarize Areas to Improve2Aggregate Quantitative Data3Explore Qualitative Data4Review & Summarize StrengthsGUIDELINES
- 24. Time to Practice!10Minutes(c) 2002 Good & Kaminski
- 26. “It sort ofmakes you stop and think, doesn’t it.”
- 27. Deciding on ThemesMorethan three students mention in a classroom.There is widespread agreement through nodding or shaking headsVerbal agreement (That’s the truth).Mentioned in more than one classroom with different observers.There are positive and negative corollaries expressed.
- 28.
Editor's Notes
- #2 The University of Miami Education and Evaluation Team (UMEET) is the independent evaluator for Smaller Learning Communities in the Miami-Dade County School District. Our office provides interim reports based on mid-year site visits (which incorporates Data-in-a-Day) and an annual report for each cohort of grantees. UMEET evaluates the grant’s measurable outcomes related to student achievement, school factors, personnel roles and responsibilities, professional development for SLC implementation, continuous program improvement of SLCs along with facilitators and barriers to implementing SLCs. Multiple methods of evaluation were conducted with key stakeholders in the process: administrators, teachers, staff, and students. The multi-method approach to the evaluation provides both depth and breadth to the evaluation, as each method captures a different dimension.For our purposes today we will be focusing on collecting, compiling, and interpreting student perspectives on smaller learning communities – which can be easily translated into research/evaluation on NAF academies.
- #3 There are two main goals to our presentation: 1) To introduce two innovative evaluation techniques – Photolanguage and Data-in-a-Day and the 2) is how to understand data, and then how to use the data to make decisions.Throughout the workshop you will be asked to participate – first experiencing Photolanguage, then analyzing sample data sets compiled from our Data-in-a-dayobservations, and discussing what is important to report and how do we make decisions from these.
- #4 School wide improvements in student outcomes (student achievement, student attendance, graduation rates and closing the achievement gap) are typically incremental; substantial gains may not be evident until 5 years or more into the process. It is possible that a school could make significant progress towards implementing SLCs that are not yet evident in student outcomes. These specific evaluations concentrated on an assessment of the progress schools are making towards best practices towards implementing smaller learning communities and NAF academies.Student outcomes are more likely to be achieved by schools that are making the most progress towards successful implementation -- Schools that exhibit the characteristics of high impact SLC and NAF programs, together with desirable teaching characteristics can be expected to be schools with positive cultures that offer quality instruction and support systems to ensure that students’ academic, social, and emotional needs are met.
- #5 Sometimes it’s not what you collect but how you collect it.Facilitates interactions among individuals and encourages creative and critical thinkingAn unobtrusive probe for gathering information from sensitive populations
- #6 Discuss the significance of the picture to the participant, not the picture per se.
- #9 Participants in Photolanguage enjoy the experience and we have gotten some very important insights on NAF students and academies. There are still challenges & obstacles to collecting data – includingDifferent data points are required for different entities (District, State, NAF requirements)Teachers in general feel that taking time to collect data – takes time away from teaching and learningTeachers are also not trained researchers – leading to questions about what to collect, what is important, how to be systematic & how to record the dataLast, there is a fear of retribution from the administration – imagined or not Once the data is collected – there are even more difficulties in what to do with itLack of comparable dataLack of longitudinal dataNo user-friendly and integrated data systems.Most of all – is the question of what to do with it next. I’ve had more than one teacher tell me about their data binder – all their students and all their data – and how they carry it around.
- #10 During the 2009-2010 academic year, the UMEET team used the participatory approach, Data-in-a-Day (Ginsberg, 2001). A Data-in-a-Day approach allowed us to observe multiple classrooms in a single day, combining short classroom observations, a faculty survey, and a fifteen minute structured student questionnaire, with time for students’ unstructured feedback. Data-in-a-Day is a research tool developed with the dual purposes of involving multiple stakeholders in the self-evaluation process, and facilitating critical dialogue about strategies for school improvement (Ginsberg & Kimball, 2008). Data-in-a-Day is a quick and efficient way of collecting, analyzing, and synthesizing a broad spectrum of data that can be utilized to evaluate the implementation of educational reforms in large schools. It provides timely and effective feedback as schools attempt to improve their practices. District personnel, school faculty and administration were encouraged to conduct classrooms observations and survey students alongside our team. Our experience of conducting evaluation of educational reforms indicates that the Data-in-a-Day process greatly enhances the evaluation. Our previous evaluation of large urban schools consisted of surveys, telephone interviews with SLC design team, and focus group interviews with faculty, administration and students. For each of these methods, schools invariably put their best foot forward, providing extraordinarily positive claims that were unverifiable.The Data-in-a-Day approach to data collection and analysis actively engages those involved in the reform, facilitates critical dialogue about future steps, and enhances the credibility of an external evaluation. No longer were we, as evaluators the designated bearer of bad news, instead we were working together to determine the effectiveness of the program. When the external reviewers and those from the school had similar conclusions after gathering data, there was an immediate discussion on how to enhance the reform and adjust the process. When our conclusions were different, it led the way to fruitful dialogue regarding what was observed, what was missing, and what were the consequences. By working collaboratively, the Data-in-a-day process culminated in a critical conversation about the implementation of secondary school reform.
- #12 There are some key important aspects that make data in a day different from classroom walk-throughs: 1) teams with representatives of students, teachers and administrators on each time 2) Promotes a dialogue – as the team is walking from class to class & in the de-briefing 3) Sends a message that the school values student input in addition to teacher & administration’s perspectives 4) Most important, this process stimulates critical reflection and discussion about best practices in and out of the classroom.Best practices includeDesigning user-friendly instrumentsParticipation of the students in data in a dayParticipation of the administration in data in a dayThere is a climate of trustThere is a receptiveness to student opinion
- #17 These are the types of questions where we ask students to raise their hands.
- #18 What you have in your packets are five classroom observation worksheets which are combined with the structured and unstructured questions we ask of students. Students raise their hands for structured questions and we count hands.During the unstructured questions, students are free to make any comments they feel are useful. And they do. Often, it is how something is said that leads to a clear analysis and that is missing from the worksheets.However, much of what is said is captured verbatim and used in the interim reports.The data sets you have are not actual worksheets, rather they are compilations of typical worksheets.Your task is to decide what are the important variables from a student’s perspective, how important and why? What is the picture of smaller learning communities or academies at this particular school?HINT #1: Pay attention to the classroom as well as the students.HINT #2: Count and tabulate and look for patterns in students words
- #21 What you have in your packets are five classroom observation worksheets which are combined with the structured and unstructured questions we ask of students. Students raise their hands for structured questions and we count hands.During the unstructured questions, students are free to make any comments they feel are useful. And they do. Often, it is how something is said that leads to a clear analysis and that is missing from the worksheets.However, much of what is said is captured verbatim and used in the interim reports.The data sets you have are not actual worksheets, rather they are compilations of typical worksheets.Your task is to decide what are the important variables from a student’s perspective, how important and why? What is the picture of smaller learning communities or academies at this particular school?HINT #1: Pay attention to the classroom as well as the students.HINT #2: Count and tabulate and look for patterns in students words
- #22 Questions:How would you calculate positive experience in an academy? For example, in the first class, a required class, the percentage of positive experience could be 100% (6 out of 6) or it could be 32% - 6 out of 19. Which one better reflects the reality in the classroom.Why are the numbers higher for academy classes than for other classes?How do you explain the numbers for the early childhood academy?Note what is going on in the classroomNote also the “choice” aspectWhy would 50% of students in the NAF Finance Academy @ this particular school willing to leave if external constraints were removed?What is the most important area to improve? Why? How?
- #24 I would stay for the academy classes – Saturday school Academies (Business)University OutreachMedical magnet (Academy)Extracurricular activities (related to academy)Some [teachers] help more than othersHow do you decide on themes?Hands-on activitiesField tripsCaring teachersToo much on FCATMore programs like the health academyWe need to study what we’re interested inMore academies More than three students mention in class or there is affirmation when said (head nodding, verbal agreement) More than one class mention (note different observers) There are positive & negative corollaries to themes: e.g. we need more caring teachers similar to so me help more than others.Having administrators and teachers participate in this research drives the point home. It is one thing to hear guidance counselors are not accessible from an outside observer, it is a very different thing to hear spontaneous laughter when a question is asked.Students, when asked, are both serious and thoughtful about good classes, good teachers, what they want to learn. Yes, students mentioned the cafeteria food often, and the strict cell phone rules. But most students, most of the time, focused on academic issues such as learning, hands-on activities, tutoring (such as Saturday school)
- #25 Having administrators and teachers participate in this research drives the point home. It is one thing to hear guidance counselors are not accessible from an outside observer, it is a very different thing to hear spontaneous laughter when a question is asked.Uniforms: consistency about the rulesCell phones: especially during lunch is a huge issue and fight with students. Students, when asked, are both serious and thoughtful about good classes, good teachers, what they want to learn. Yes, students mentioned the cafeteria food often, and the strict cell phone rules. But most students, most of the time, focused on academic issues such as learning, hands-on activities, tutoring (such as Saturday school)