Welcome to the NEBHE Developmental Math Demonstration Project Kick-Off WebinarParticipant Instructions• Participants should be able to access the webinar’s audio through computer speakers or via telephone. If you have any difficulties accessing the webinar, please email Daren Follweiler at email@example.com.• To ensure good audio quality, please mute your audio when you are not speaking.• All participants are automatically muted upon joining. To ask a question, please use one of the following: • Type into the Question Box • Raise Your Hand and we will unmute you • Unmute by clicking the microphone icon (at the side of your control panel)NEBHE Developmental Math Demonstration ProjectJanuary 29, 2013
NEBHE Developmental Math Demonstration Project Presenters: Stafford Peat, Senior Consultant Monnica Chan, Director of Policy and ResearchNEBHE Developmental Math Demonstration ProjectJanuary 29, 2013
Participating Institutions/Systems• Connecticut Board of Regents for Higher Education• Community College System of New Hampshire• Community College of Vermont• Housatonic Community College• Middlesex Community College (CT)• Middlesex Community College (MA)• Mt Wachusett Community College• One additional Community College may join the network NEBHE Developmental Math 3 Demonstration Project
Memorandum of Understanding• The MOU outlines the terms of partnership between participating institutions/systems and NEBHE as well as the research questions we plan to answer.• Thanks to those institutions that have already sent in their MOUs.• Signed MOUs will be returned to the college/system with the first payment.• Signed forms are due by February 28, 2013.• Are there any questions that need to be answered before your institution/system signs the document? NEBHE Developmental Math 4 Demonstration Project
NEBHE Developmental Mathematics Demonstration Project Memorandum of UnderstandingThe New England Board of Higher Education (NEBHE) released a Letter of Intent in November2012 for community colleges, consortia of public two-year colleges or state community collegesystems to partner in the NEBHE Developmental Mathematics Demonstration Project (DMDP).Institutions and state community college systems were selected through a competitive process.The Memorandum of Understanding (MOU) below outlines the respective roles andresponsibilities of colleges/systems and NEBHE commencing in January 2013 throughSeptember 2015.Community college/state community college system pilot sites will: 1. Use Khan Academy materials, as described in the college’s/system’s Letter of Institutional Intent. 2. Help NEBHE answer research questions, listed below. 3. Adhere to the research dimensions of the project. 4. Report student data to NEBHE based upon the schedule described on pages two and three in the NEBHE Developmental Math Demonstration Project Letter of Intent. The data for participating enrolled students include: demographic information, developmental math course information including instructional approach, student grades, course enrollment patterns, grades in subsequent math courses and Khan Academy usage. For high school students participating in project pre-matriculation programs, data will be reported in the aggregate. See Attachment A for variables that will be reported. 5. Enter into a data sharing agreement with NEBHE. The data sharing agreement specifies the specific variables to be collected, the basis for how student data is transmitted, stored, analyzed, made secure, reported and destroyed once the project has ended. 6. Comply with the Family Education Rights and Privacy Act (FERPA) and other state privacy requirements. 7. Advise NEBHE in the development of model Khan Developmental Math (DM) templates, resources, and data collection tools. 8. Provide NEBHE with Khan Academy electronic platform feedback, suggestions for enhancement, or new features for use by community colleges/adult programs through formal surveys and other informal feedback. 9. Participate in a project orientation and training session, Khan Academy network users group, annual network meetings, and other online convenings. 10. Assign a site coordinator who will be responsible for communication with institutional/system leaders, participating faculty and with NEBHE. 11. Participate in a broader regional dialogue on DM and developmental education reform. 12. Share best practices and lessons learned with Khan Academy and with NEBHE. 13. Inform NEBHE of planned major changes in project design activities as described in the college’s/system’s Letter of Institutional Intent.Research Questions • To what extent, and in what ways, is student performance in DM impacted by implementation of new delivery formats and Khan technology-based resources?
• What are the most and least effective utilizations of Khan resources? • What is the impact of the pilot implementation on DM at the institution? • What are the project’s research findings, in terms of: Khan usage, in-module/course student performance? • What were the differences between the treatment and control groups? Between the varied models/implementations of Khan resources? • What impact can be attributed to implementation of Khan-based resources, delivery modes and other factors?To help answer these questions NEBHE will collect baseline, formative and summative data aswell as use information collected through staff and student surveys, and onsite visits. NEBHE will: 1. Provide demonstration sites with technical assistance and ongoing coaching. 2. Organize a community of Khan Academy network project users to share information on best practices, problem solving and for ongoing communication. 3. Host a two-day orientation and training session in Spring 2013 for demonstration site staff. NEBHE will cover participants’ accommodations and meal expenses. 4. Provide assistance to demonstration sites in collecting and reporting data to NEBHE. 5. Comply with the Family Education Rights and Privacy Act (FERPA) and other state privacy guidelines. 6. Convene all participating colleges/systems at least once per year for information sharing and training. 7. Provide opportunities for web-based convenings. 8. Identify model DM course syllabi objectives and map the course objectives to the Common Core State Standards for Mathematics and Khan Academy Videos and Practice Sets. 9. Develop other mapping tools, guides, data dictionaries and resources as needed to implement the project. 10. Identify or create other DM open educational resources. 11. Provide demonstration sites with project-related summaries of their institution/state system as well as network results based upon data submissions. 12. Provide incentive funds at the conclusion to participating institutions/state systems based upon the number of students divided by the number of students served across all pilot sites. 13. Work with participating colleges/systems to identify and develop potential next phase activities, as desired and as informed by project findings. 2
By signing below, the parties agree to the roles and responsibilities of this MOU. FOR THE INSTITUTION OR SYSTEM Entity: ___________________________________ Signature: ___________________________________ Name: ____________________________________ Title: ____________________________________ Date: ____________________________________ FOR THE NEW ENGLAND BOARD OF HIGHEREDUCATION Signature: ____________________________________ Name: ____________________________________ Title: _______________________________________ Date: ____________________________________ 3
Data Sharing Variables• Attachment to MOU.• Outlines data we would like to collect for all institutions including student age, race, developmental course taken and grade. There are also optional sets of data variables to be collected based upon submitted LOIs: – Variables for HS students in pre-matriculation programs – Variables for measuring whether Khan usage reduces institutional costs – Variables comparing student performance on placement exams and/or in treatment vs. control groups• Are there any questions about the data to be collected, including optional data sets? NEBHE Developmental Math 5 Demonstration Project
Attachment AData Sharing Agreement Variables1 I Data variables for all participating colleges/systems Variables for pre- or post- matriculation programs using Khan Academy include: • Student identification numbers using a randomized protocol developed by demonstration sites/state systems • Race/Ethnicity using IPEDS definitions • Gender • Pell Grant status (as a proxy for family income), if available • Age reported in age bands (<18,18-24, 25-34, 35+) • Name of pre-matriculation program, if applicable • Developmental course name and date taken • Developmental course number • Developmental course level • Developmental course instructional approach • Number of Khan Academy videos watched • Number of Khan Academy exercises (skills) started • Number of Khan Academy exercises (skills) proficient • Number of Khan Academy practice problems attempted • Number of Khan Academy practice problems correct • Developmental course grade • Student enrollment in a subsequent developmental or for “transfer credit” math course • Grade in subsequent math course and date taken Base-line data variables to be collected for comparative purposes by developmental course name, number and level for the 2011-12 academic year: • Student Race/Ethnicity breakdowns using IPEDS definitions • Gender ratio • % of Pell Grant recipients • Age reported in age bands (<18,18-24, 25-34, 35+) • Student enrollment rates in a subsequent developmental or for transfer credit math course • Student completion rate (C or higher) in DM course • Student completion rate (C or higher) in subsequent math course and date taken II Optional for colleges/systems that plan to serve high school students Variables to be collected for in the aggregate for students enrolled in high school participating in pre-matriculation programs include: • Name of pre-matriculation program1 NB: In addition to these quantitative data variables, NEBHE will work with all sites toadminister feedback surveys for faculty and students, per the Project Letter of Intent.
• Type of pre-matriculation program• Number of participating students reported by high school• Average number of Khan Academy videos watched• Average number of Khan Academy exercises (skills) started• Average number of Khan Academy exercises (skills) proficient• Average number of Khan Academy practice problems attempted• Average number of Khan Academy practice problems correct• Students’ enrollment in a subsequent developmental and for “transfer credit” math course at the postsecondary level, if applicable (by number and %)• Average grade, range of grades and course completion rates in subsequent developmental math and for “transfer credit” math course at the postsecondary level for pre- matriculation students, if applicable• Pre-and-post placement test pass/fail data (by number and %), if applicableIII Optional for institutions that plan to collect additional data to measure KhanAcademy usage impact on:• Institutional cost o Salaried costs before and after demonstration project o Hourly personnel costs before and after demonstration project o Fixed costs (instructional materials, etc) before and after demonstration project• Student performance relative to college placement exams o College placement exam(s), and cut-off score(s) o Student placement exam score prior to developmental math course o Student placement exam score at the end of developmental math course, if applicable• Student performance in treatment compared to control groups o Student identification numbers using a randomized protocol developed by demonstration sites/state systems o Race/Ethnicity using IPEDS definitions o Gender o Pell Grant status (as a proxy for family income) o Age reported in age bands (<18,18-24, 25-34, 35+) o Name of pre-matriculation program, if applicable o Developmental course name and date taken o Developmental course number o Developmental course level o Developmental course instructional approach o Developmental course grade o Student enrollment in a subsequent developmental or for transfer credit math course o Grade in subsequent math course and date taken
Data Sharing Memorandum of Agreement• Outlines the process for sharing data on participating students with NEBHE.• This MOA may be used as a model for institutions that don’t have a MOA template and will be emailed to participating colleges/systems coordinators next week.• May be modified to meet needs state/institutional privacy requirements.• MOA due May 31, 2013.• NEBHE doesn’t plan to collect student names/IDs, however we have been advised that a data share agreement is needed. NEBHE Developmental Math 6 Demonstration Project
March Training Workshop for Participating Sites• Sessions will take place at the Sturbridge Host Hotel and Conference Center on March 28-29, 2013.• Tentative schedule has the workshop beginning in mid- morning on March 28 and ending late morning on March 29.• Accommodations and meal charges covered by NEBHE.• A formal workshop announcement including the tentative agenda will be emailed out next week.• Institutions may send up to five individuals and state systems may send up to 10 staff.• Site representatives should include the site coordinator, IR staff person, and participating faculty members. NEBHE Developmental Math 7 Demonstration Project
March Training Workshop Tentative Agenda• Khan Academy orientation by Khan Academy staff• Flipped classroom workshop session• Data reporting orientation• Feedback from attending math faculty who have used Khan in their math courses• Presentation of training resources• Discussion of implementation plans across sites and sharing of developmental math redesign initiatives NEBHE Developmental Math 8 Demonstration Project
Upcoming Training Resources• Draft model developmental math course objectives that are aligned to the Common Core and Khan videos and practice sets• Instructor guides• Student guides• Project website with resources and list of implementation models (www.nebhe.org/devmath) NEBHE Developmental Math 9 Demonstration Project
Next Steps• Model course objectives for DM courses• Pathfinders to share experiences in using Khan Academy materials• Look for emails on MOA, March workshop announcement and registration information• On-going feedback NEBHE Developmental Math 10 Demonstration Project
Questions & Comments? Stafford Peat at firstname.lastname@example.org