1. COMPARISON OF SUCCESS RATES IN BASIC SKILLS MATH AT THE LOS ANGELES COMMUNITY COLLEGE DISTRICT (LACCD) Tatiana Melguizo University of Southern California Hans Bos American Institutes for Research (AIR) George Prather Los Angeles Community College District Bo Kim University of Southern California
Good afternoon! My name is Tatiana Melguizo and I am presenting with George Prather my colleague at the LACCD. I am an assistant professor at USC, and my work is centered around issues of college pathways of low-income students and students of color. In recent years I have studied issues related to community college transfers, and costs of attending community colleges. We are excited to be part of this conference. Today we are going to be describing one project that was recently funded by IES to evaluate basic skills math in the LACCD. You have two of the main researchers of the team. We are working in collaboration with Hans Bos, the Vice-President for the American Institutes for Research (AIR), and Bo Kim a PhD student in economics at USC. Given that the grant has not even started our goal is to introduce the project and share with the audience some preliminary results looking at the proportions of students placed in the different levels of math, the term in which the took these courses, the level of courses in which they enrolled, and finally the success rates. We are going to take less than the assigned 45 min because we want to have a conversation with the participants in the conference.
The results that we are presenting today are part of a two-year project recently funded by the Institute of Education Sciences.
Read the outline
I think that one thing that every single participant in this conferences agrees is that basic skill preparation matters. The RP group has devoted resources and time to come up with a number of best practices for the colleges, the state has devoted substantial funding (less now after the crisis), to really address this issue, and more recently the community colleges as well as developmental education are part of the main initiatives' of the current administration. When you go back to the literature and for those of you who have read the full volume of the “Poppy Copy.” You know that there is no agreement on the effects of remediation. Two recent state level studies have received a lot of attention in recent years. Bridget Terry Long from Harvard along with Juan Carlos Calcagno previous PhD graduate at TC have found that difference in some outcomes between remediated and non-remediated students do not vary substantially. Similar results were found for the state of Texas. Whether you agree or not with the outcomes that they are measuring, the truth is that these are two examples of good quantitative evaluations. Our study builds on these studies but unlike them we don’t think that you can link math remediation and transfer. We argue that you need to think about basic skills math as a sequence, and you should evaluate all the sequence, not just remediated versus non-remediated students. We also think that there are multiple measures of success that are more informative for the community colleges.
Our study is an evaluation of the basic skills math levels on a number of important short-term or intermediate outcomes. Read them. Our main guiding research question: READ. We cannot answer this question yet, but if you invite us in two years, we’ll have an answer
Our study is composed of two main parts: A descriptive analysis that will help us get a real detailed portrait of the placement practices, the individual characteristics of students in the different levels, as well as the course taking patterns. This will provide important background for the evaluation. Take advantage of a unique dataset compiled by LACCD that enables us to evaluate and make causal inferences related to the differences in success rates of students in the different math levels.
We collected the math sequences from the different community colleges. And I am presenting here the one developed by Harbor. I like it because it is simple and visually really powerful to indicate the different levels of math and which courses are transferable to the CSU’s and the Ucs. In the last part I report the percentages of the students placed in up to 5 levels below college level courses. One way of thinking about this levels is: Level 5=Arithmetic Level 4=Pre-Algebra Level 3=Elementary Algebra Level 2=Intermediate Algebra Level 1=Trigonometry Transfer level= Pre-calculus, calculus, linear algebra, differential equations.
Regression Discontinuity is a technique that enables the researcher to use an exogenously determined cut off score to assign students. An intuitive way to think about this technique is that students are placed using one of the 3 most common placement exams. We have that colleges have some rules to determine the cut offs and using these and other criteria defined in the matriculation policies they assign students. Regression discontinuity enables the researcher to compare the students just above and below the cut score, and test for differences in relevant short term success outcomes such as passed the course, took the following course, and so on. Our study is unique in the sense that we are testing the whole sequence of basic skills math and not just comparing the remediated versus not remediated studies.
The main task that we have in our coming months is to create the CPS with the criteria defined by the different colleges, and taking into account possible changes in the criteria over time.
One of the main requirements of RD is to have a large enough sample. We did all the power calculations and our sample enables us to identify the expected effects.
Given that the audience is mostly from CA if not LA, and that we are all familiar with the community colleges. We just wanted to summarize some specific characteristics of the district. READ.
Now, let me describe some of the preliminary descriptive statistics. We wanted to know what proportion of the non-concurrent students who are eligible for being assessed were indeed assessed. For example, a high school student taking an AP is excluded from the sample, similarly a senior citizen taking a 1 credit course is also excluded from assessment. The figure illustrates that the community colleges are doing a good job in complying with assessment requirements. About 90% of the students were assessed during the academic year. We present information for 2001 and 2007 to give some longitudinal perspective. The data suggest that assessment has increased over time.
We were also interested in looking carefully at the data to check if the students were following the assignment and if so when. This table presents again information for the 2001-02 and 2007-08 academic years. It is noteworthy that about 40% of the students enrolled in the assigned course the first term, 10% the first academic year, and additional 7% the second year, and about 8% after the second term. It is particularly important to note that over 1/3 of the students were never enrolled! The situation was a bit better for the most recent academic year but not so much. This number is problematic for the community colleges and our study. Why are the cc loosing this students? It is also problematic for our study because there is a substantial number of individuals that we won’t be able to include in our evaluation because they were NOT there! We will use the appropriate statistical corrections and acknowledge this in the limitations, but it is a very worrisome finding!
We were also interested in exploring the math level of enrollment. Basically, were students enrolling in the assigned course? The answer is for the most part. In 2001-02 over 2/3 of those who enrolled, followed the assigned level but some students decided to enroll in either a lower or higher level. The percentage of students who followed the level increased over time! Again this finding is problematic for the colleges. There is an interesting dissertation by Armstrong that show no differences in success rates of students who enrolled in a higher level math course. Then this questions the validity of the placement criteria! This is also problematic for our study and we acknowledge it, and we’ll use the appropriate statistical corrections.
We wanted to test for differences in success rates by term of enrollment. It is noteworthy that about 50% of the students succeed, and this does not change by level of enrollment. Success is defined as attaining a C or higher on a course. One would ask why are so few students succeeding. When I think about the success rate of my students it is much higher. Again what is the role of the colleges to ameliorate this leakage.
We also wanted to explore differences in success rates by enrollment level. No major differences were found there.
Finally, we wanted to share with the audience a table describing the proportion of students who were placed in each of the 5 levels below transfer in each of the nine community colleges. Remember that the levels can be associated with: Level 5=Arithmetic Level 4=Pre-Algebra Level 3=Elementary Algebra Level 2=Intermediate Algebra Level 1=Trigonometry Transfer level= Pre-calculus, calculus, linear algebra, differential equations. A couple of things stand out: Most of the need is on level 4 and 3, Pre-Algebra and Elementary Algebra. It is clear that the community college is inheriting a problem from the K-12 system, and we need to pair with high school instructor to understand the best strategies to teach these courses. There are substantial differences in the levels by colleges. These suggests that the colleges are attracting different populations of students. In trade-tech with the highest need in the lowest levels, students enter with certificate or associate degrees in mind. However, the changes in the minimum requirement of Algebra for an AA makes the need to find a solution to the problem even more relevant.
To finalize the community colleges are being successful in assessing the students. Around 45% of the students enrolled in the assign level the first term, almost 20% do it in the following 2 years, and 37% NEVER ENROLL. This is a very disturbing statistic that should be the focus of further exploration. Another statistic that should be in the radar of community college teachers, administrators and leaders is the relatively low success rates of the students. There are several potential explanations that need to be explored: 1) are students not prepared-this questions the placement policies, 2)are teachers prepared to teach the content and non-traditional students? 3)Is the curriculum appropriate? And many others.
A majority of the students are being placed in 4 levels below transfer. Assuming that they take the 4 courses in the next two academic years, this means that they won’t have college level courses to transfer. Should community colleges take 2 years to place the students at college level? There is no clear answer but this is a reality that need to be looked at. The differences in the proportion of students in the different levels in the 9 community colleges, suggest that they have to focus on different levels, but they can collaborate and share success practices at least in intermediate and pre-Algebra.
![[object Object],Evaluating the Effects of Basic Skills Mathematics Placement on Academic Outcomes of Community College Students](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)