With that, I’d like to close by discussing next steps. We will be sending out a communiqué to all community colleges in the region, to gauge and identify interest in participation. In addition, we’ll be putting together an advisory committee comprised of faculty, academic deans, institutional researcher and others for feedback and guidance on this project. Certainly there’s much experience in the room here—if there are nominations, self or otherwise, we’d love to hear them. For now, I’d like to [LOOK AT WATCH] invite questions [OR] invite those interested in learning/discussing this more to approach Stafford or myself—particularly if you’d be interested in participating within the first cohort. We’ll be around after the session. Recognizing that we’ve promised a 4 PM end date—I’d like to turn it over to Michael Thomas for some closing remarks. Thanks so much for your time and attention! Hope to work with you!
As a snapshot of Khan Academy’s reach, this slide provides an overview of various usage metrics as measured by Khan Academy. I’ll only point out a few today—but these slides and a recording will be available. By June 2012, Khan Academy had 42.5 million users and has been used in at least 14,000 classrooms across 2,000 schools. To date, a majority of these classroom have been in the K-12 education space. This demonstration project is one attempt to determine the utility and effectiveness of Khan Academy materials in the postsecondary arena and build a postsecondary learning community, as Stafford mentioned earlier.
Something that I’ve noticed in my work at NEBHE, is a shift towards self-paced, individualized instruction—particularly in the developmental education “redesign” efforts that are happening across region. With Khan Academy, each of the three major activities—watch, practice, coach—have real-time data affiliated with them. This data that’s available, via a “dashboard”, on the videos watched and problems practiced, combined with the open-source nature of Khan Academy materials, could be a useful tool for these redesign efforts (certainly it’s a tool that we’re hearing more and more colleges take advantage of).
To look at this “dashboard” more concretely, this is a screenshot from the Khan Academy Dashboard of my attempts to multiply polynomials through Khan Academy exercises. If I had added one of you, or a professor, as a coach, they would have access to this same data. What you see on the x-axis are the problems that I attempted. The y-axis measures the amount of time I’ve spent working on each problem. Blue bars indicate that the problem was successfully solved. Red bars indicate the problems I’ve solved incorrectly. When I hover over a bar—this is the action that’s depicted in the image you see. I was hovering over the red bar—a few facts come up. The box tells you whether or not I solved the problem, whether or not I used a hint (hints are built into the site), as well as the length of time used to solve the problem. If I were click on this bar, you would see the actual problem and in this case, the hint that I used and when I used it.In addition to this specific data, Khan Academy allows coaches to export aggregate information on a student. Such as, how many videos have been completed? How many exercises has a student solved correctly? How much has been spent watching videos and solving exercises? This aggregate information is especially of interest to us because [CLICK] of our goal to demonstrate the impact of Khan Academy materials.
Specifically: is Khan usage correlated with student success in developmental education? Many of these objectives are outlined on pages 5 – 6 on the copy of the Letter of Intent each of you should have received. I won’t go into as much detail as the project objectives outlined in the table on pages 5 through 6; however, I did want to briefly cover our primary research questions:what extent and in what ways is student performance in developmental math impacted by the implementation of new deliver formats and Khan technology-based tools? [Are more students completing a demonstration course than when Khan Academy materials aren’t used? Are students more successful in subsequent math courses?]What are the most (and least) effective utilizations of Khan-based tools? [If one demonstration site it using Khan Academy in a pre-matriculation model, another is using a flipped classroom model, and another is using blended instruction, are student results the same across each site? Or is one model perhaps more effective than another?]What is the impact of the pilot implementation on developmental math at the institution? [Part of this information will be collected through surveys—we’re planning at the moment to survey both students and faculty their experience in a way that will hopefully collect good feedback and capture any unanticipated effects. Another aspect to this question will be cost, such as the impact Khan Academy has on the cost of delivering a developmental course.]
In building on these research questions, Stafford and I wanted to give you all a visual of how NEBHE plans to go about answering these questions through partnerships with demonstration sites. This slide is thus the first of two showing a sample reporting form. While still a draft, we anticipate that some of the basic information outlined here will be the same as what we’ll ask of demonstration sites during the implementation phase—there are some tentative dates outlined on page 2 of the letter of intent as to when this might occur.Going through this, you’ll see that this form is specific to a semester. We are asking that demonstration sites provide individual student information. As Stafford mentioned, we aren’t asking for names or SSNs. Rather, each demonstration site should be able to generate a random number that would allow us get individual data on some student characteristics—like gender and age—course taking, Khan Academy usage [those middle columns on the number of videos completed, time of videos, time on exercises], and final grade.
This next form is what we anticipate being a longitudinal report that gives information on subsequent student enrollment beyond the demonstration course. You’ll see this outlined in the additional columns to the right. As mentioned in that timeline on page 2 of the LOI, this longitudinal data won’t be due until 2015—close to the end of the project. Something to note is that although both of these forms ask for student level data, we’ll only be reporting in the aggregate. The individual data is needed because if multiple sites are implementing Khan Academy in the same format, then the individual data will allow us to group the data and aggregate the information accordingly. Individual data will also allow us to slice the data differently, say by student age, the number of video completed, etc. The data and feedback that we plan to collect through this project is—If I may be immodest—what makes this project really exciting to me. It’s an opportunity for us to partner as a region and learning community towards greater student success in developmental mathematics. With the level of interest we’ve seen from institutions and states, I think it’ll be a fantastic next few years. Stafford and I are really excited for what we’ll have to review from all of you next month. But rather than gush further as an invisible webinar presenter, let me thank you for listening and now turn it over to all of you for questions and comments.
Nebhe nov 19 2012 webinar
Welcome to the NEBHE Developmental MathDemonstration Project Informational WebinarParticipant Instructions• Participants should be able to access the webinar’s audio through computer speakers or via telephone. If you have any difficulties accessing the webinar, please email Daren Follweiler at firstname.lastname@example.org.• To ensure good audio quality, please mute your audio when you are not speaking.• All participants are automatically muted upon joining. To ask a question, please use one of the following: • Type into the Question Box • Raise Your Hand and we will unmute you • Unmute by clicking the microphone icon (at the side of your control panel)NEBHE Developmental Math Demonstration ProjectNovember 19, 2012
NEBHE Developmental Math Demonstration Project Presenters: Stafford Peat, Senior Consultant & Project Director Monnica Chan, Director of Policy&ResearchNEBHE Developmental Math Demonstration ProjectNovember 19, 2012
Project Overview• Lumina Foundation awarded NEBHE a three-year grant to demonstrate the effectiveness of using Khan Academy math content and online tools to improve Developmental Math (DM) outcomes• Over three years NEBHE plans to work with 8-12 community colleges• Targets post-matriculation traditional and non-traditional DM courses (multiple levels) and pre-matriculation programs as pilots• Supports on-going developmental education reforms NEBHE Developmental Math 3 Demonstration Project
Project Goals• Increase postsecondary persistence and success in DM courses using Khan Academy content and tools• Drive reform of DM using new no cost tools• Create regional engagement and a community of learners to promote and support developmental education reform• Generate high-impact data and research findings and expand knowledge of new approaches NEBHE Developmental Math 4 Demonstration Project
Project Goals (cont.)• Produce a NEBHE document that reports on findings and best practices• Use demonstration project to enhance Khan’s usefulness for postsecondary education NEBHE Developmental Math 5 Demonstration Project
Project Implementation November 8th: Letter of Project Intent sent to On-going Work Community Colleges • Research existing NE DMcourses and learning objectives November 19th: Host webinar information session • Map and align Khan onlineresources/Common Core to a set December 15th: Letter of of identified DM models Institutional Intent due December 21st: 8-12 community colleges selected and notified NEBHE Developmental Math 6 Demonstration Project
Project ImplementationJanuary: Orientation Webinar for Demonstration Sites March: Two-day training for participating faculty and institutionsSummer Semester: Project start-up using Khan Academy with students Fall Semester: Full implementation across all participating institutions NEBHE Developmental Math 7 Demonstration Project
Incentives for Colleges to Participate• Khan Academy is no cost vs. other CAI programs• Khan can be used in a variety of instructional settings and pedagogical approaches including flipped classroom/blended learning and to prepare individuals to take Accuplacer• NEBHE will provide coaching and ongoing technical assistance and support• Each site will receive a grant and financial incentives based upon number of students served. Funds can be used to defray costs including data collection or staff travel. Funds can be also used for other purposes to be determined by pilot institution NEBHE Developmental Math 8 Demonstration Project
Letter of Project Intent – Highlights• A single community college, a consortium of colleges or state community college system is eligible to submit a Letter of Institutional Intent• Project funding includes $5K for a single institution and $15K for a consortium or state system• An additional $25K in incentives based upon number of students served will be distributed at the end of the project• Selection criteria used to select a diverse set of institutions• Project participation will include a MOU between NEBHE and participating institutions and a simple data share agreement• Student level data will be collected at the institution and sent to NEBHE through a simple data share agreement (no student names or SS #) NEBHE Developmental Math 9 Demonstration Project
Letter of Project Intent - Highlights• Letter of Institutional Intent is designed so that its completion is not burdensome• Has nine basic questions• Can be either mailed or emailed NEBHE Developmental Math 10 Demonstration Project
Watch Practice CoachOver 3300 videos210 in Arithmetic and Pre- Algebra717 in Algebra NEBHE Developmental Math 11 Demonstration Project
Khan Academy June, 2012 by the numbers 5,072,554 Videos Unique Visitors March 25 – April 25 150 million videos 65% watched to date 42.5 1,357,377 35% Exercises 500 millionmillion Hours spent on Khan Academy March 25 – April 25 problems solved to date The Khan Academy iPad app skyrocketed Unique #1 to be the #1 education app within days of launching. Over Visitors Education App 600,000 downloads to date. To Date 471,000 14,000 2,000 25 13% Users involved in Total Pilot Active users Classrooms coaching groups of size Schools Schools in schools are 10 or greater international
About Khan Academy • Data available via dashboardWatch for students and teachers on videos watched and problems practiced, as wellPractice as goals set and attained. • Tool for individualizedCoach instruction NEBHE Developmental Math 13 Demonstration Project
Khan Academy Dashboard NEBHE Developmental Math Demonstration 14 Project
Demonstrating Impact• To what extent, and in what ways, is student performance in DM impacted by implementation of new delivery formats and Khan technology-based tools?• What are the most (least) effective utilizations of Khan-based tools?• What is the impact of the pilot implementation on DM at the institution? NEBHE Developmental Math 15 Demonstration Project
Data Reporting – Sample Requirements NEBHE Developmental Math 16 Demonstration Project
Data Reporting – Sample Requirements NEBHE Developmental Math 17 Demonstration Project
Questions & Comments? Stafford Peat at email@example.com