2011 ATE Conference Panel Session

Debra D. Bragg, Office of Community CollegeResearch and Leadership, University of Illinois

Transfer BaccalaureateTerminal

Transfer Applied BaccalaureateTerminal

“…a bachelor’s degree designed to incorporateapplied associate courses and degrees onceconsidered as ‘terminal’ or non-baccalaureate levelwhile providing students with the higher-orderthinking skills and advanced technical knowledgeand skills so desired in today’s job market.” Townsend, Bragg, & Ruud (2008, p. 4)

 Julia Panke Makela, Research Specialist & Project Director Collin Ruud, Research Associate Stacy Bennett, Graduate Research Associatehttp://occrl.illinois.edu/projects/nsf_applied_baccalaureate

 Our targeted-research project aims to: ◦ Identify pathways to baccalaureate degrees in technician education ◦ Analyze pathway designs, implementation, and outcomes ◦ Describe how AB degree programs operate and meet students and employers workforce needs ◦ Identify and widely disseminate promising and exemplary practices

 Brief survey to identify established formal pathways to baccalaureate degrees Follow-up survey on identified baccalaureate degree pathways on curriculum and instruction, accreditation and evaluation, enrollments and students served, partnerships with employers and other higher education institutions, and perceived impacts of ATE. Case studies with 7–10 ATE projects and centers to uncover promising ideas and proven practices

 Contacted all NSF-ATE Principal Investigators (PIs) with grants awarded between1992 and 2011 (~700 grants) Inquired about: • degrees affiliated with the NSF-ATE project or center • fields of study • retention and recruitment of underrepresented student populations at the baccalaureate-level • access to student-level data for baccalaureate degrees Received 234 responses (36% of the sample)

 24% of survey respondents reported associate degrees affiliated with their ATE project or center with no established pathway to the baccalaureate Some survey non-participants offered insights into their decision not to participate: ◦ “Our Civil Engineering Practitioner Degree is an AAS and therefore is a terminal degree. Our participation in the survey is probably not warranted.”

 Baccalaureate degree pathways affiliated with ATE projects and centers fit both: • Traditional transfer patterns of AS or AA degrees transferring to BS or BA degrees • Emerging pathways such as applied baccalaureate (AB) and community college baccalaureate (CCB) degrees  42% (98 of all respondents) indicated that associate degree programs had established formal baccalaureate degree pathways  20% (47 of all respondents) indicated at least one pathway began from an applied associate degree

Manufacturing and Engineering Technology Computer and Information Technology Other Biotechnology Energy Electronics Environmental Technology Cyber Security and Forensics Telecommunications Nanotechnology Chemical Technology Geospatial Technology Civil and Construction Technology Multimedia Technology Transportation Technology Marine Technology Agricultural Technology 0% 5% 10% 15% 20% 25% 30% 35% Percent of Respondents Indicating Baccalaureate Degree Pathways

 Analysis of 87 of established degree pathways • Applied Associate  Technical Baccalaureate (22) • Applied Associate  Traditional Baccalaureate (32) • Traditional Associate  Technical Baccalaureate (11) • Traditional Associate  Traditional Baccalaureate (47) Degree Examples Applied Associate AAA, AAS, AAAS, AAT, AET, AT Traditional Associate AA, AS Technical Baccalaureate BAA, BAS, BAAS, BAT, BT Traditional Baccalaureate BA, BS

 20 respondents identified the following fields of study: • Biotechnology • Manufacturing and • Chemical technology engineering technology • Computer and • Marine technology information technology • Nanotechnology • Cyber security and • Telecommunications forensics • Transportation technology • Electronics • Energy CCB Defined… • Environmental Any form of baccalaureate degree awarded by an institution identified as a community college, technical technology college, two-year college, two-year or technical branch campus of a university system, or any other institution that primarily awards associate degrees.

Theoretical and Methodological Frameworks • Program Quality Unit • Educational Significance Influences • Evidence of Effectiveness and Institutional Success Influences • Replicability and External Influences Usefulness to Others Latucca & Stark (2009), Bragg et al. (2002),Contextual Influences on Academic Plans Sharing What Works: Exemplary and Promising Programs Evaluation Criteria

 Variety makes baccalaureate pathways in technician education challenging but compelling to study Many questions: • How are programs designed? • What perceived needs are they addressing? • What features contribute to their effectiveness? • What do we know about student outcomes? • What can be learned from one program that can be adopted or adapted in other settings?

• Debra D. Bragg • Email: dbragg@illinois.edu • Check out our website: OCCRL occrl.illinois.edu ◦ http://occrl.illinois.edu • Participate in our webinars ◦ PH: 217-244-9390 • Get on our listserv ◦ E-mail: occrl@illinois.edu • Receive the e-Info • Friend our Facebook • Receive our tweets

BUILDING REFLECTIVELEADERSHIP:RESEARCH INTO PRACTICES ATE LEADERSUSE TO DEVELOP AND MAINTAININDUSTRY-RELEVANT CURRICULUM,PROGRAMS, & INSTRUCTIONLouise Yarnall, Raymond McGhee, & Joseph Ames

Research goals Deepen understanding about the industry-CC collaborative cycle to develop workforce programs Analysis framed by research model based on past research and our findings; use model to:  Tell rich stories about ATE Center cases  Describe mechanisms for iteratively translating industry input into curriculum, programs, and instruction  Describe mechanisms for sustaining the curriculum, program, and instruction collaboration with industry over time  Describe common metrics of program success

Research background Title: Community College Partnership Models for Workforce Education Sustainability and Integrated Instruction 4-year project, beginning Year 3 4 ATE Centers/Projects:  Wind energy, biotechnology, engineering technology, telecommunications and information technology  Different stages of engagement with industry in instructional program development: beginning, mid-life, mature 6-7 associated colleges Case studies

Research Team and AdvisorsSRI Team and Ames Associates Evaluator and Advisory Panelists Louise Yarnall, PI  Nick Smith, Evaluator, Syracuse University Ray McGhee, co-PI  Frances Lawrenz, University of Minnesota Geneva Haertel  Cynthia Wilson, The League for Robert Murphy Innovation in the Community College  Manjari Wijenaike, former Carolyn Dornsife ATE Center director Joseph Ames, Ames  Steve Wendel, NCME Assoc.  David Jonassen, University of Missouri

Project Overview Partnership sub-study:  Evolution of relationships between industry and community college in workforce programs  Unique stories, common mechanisms to translate industry goals into instructional programs Classroom instruction sub-study:  Tracing industry and ATE Center influences on instructional programs  Characterizing range of workforce education instructional practices and curricula

Research products - Partnership Cases of ATE Center activities contributing to life cycle of collaboration with industry in workforce program development  ATE principal investigator activities  Instructional goals  Rapid development mechanisms  Sustainability challenges

Research products - Instruction Cases of ongoing, classroom-level processes that support continual instructional updates Cases of technician education instruction

Peek at findings so far Model of industry-community college instructional partnerships Partnership sub-study: Early highlights & starting cases

Model: Findings and Uses ATE community members can use this model to strengthen partnerships:  Stepping back, seeing “big picture” of your work  Using the categories in the model to “make sense” of challenges you face, identify potential opportunities Researchers use models to make sense of complex phenomena across multiple settings Models emerge from past empirical research and theory; they evolve based on current data

Model: Strategic Need

Model: Formation Processes

Model: Partnership Capital

Model: Outcomes/Outputs

ATE-CC Partnership Conceptual Model FORMATION PARTNERSHIP OUTCOMES/ PROCESSES CAPITAL OUTPUTS Establishing trust/norms/comm. Creating partnership capital Sustaining the partnership (Fusing social & org. capital) (Partnership implementation) (Producing results) Strategic Need CC support ATE center Industry Resource Student Classroom/ Workplace role community Leveraging Faculty Administrator link Certificate support for Talking with testing Degrees/certif Prepared Address ATE leader industry Productive Historic (student pays) icates offered workers labor supply presence meetings: PD, new placed needs Organizing technology, In region Degrees/certif New courses work groups standards alignment icates created Employee with faculty Articulates obtained training Retrain labor need Establish Instructional incumbent Marketing/out reach first agreements around Job materials External workers equipment, labs / placement/int development Resources Trust-building resources ernships State & local meetings Improve technician funding 1/x Instructional materials sharing training Industry adjuncts Organizational Partnership boundary Complexity maintenance -# organizations -# sectors -# states STAGES: Emergence Transition Maturity Critical Cross Roads

Partnership sub-study: Early findings Cases Uses: ATE community leaders can compare their own situations to these cases, deriving insights

Case 1: Regionally scaling a program  ATE leader role:  Facilitate regional industry, educators  Goal:  Sequence for multi-college ET program  Rapid Development Mechanisms:  Identify core courses that transfer across local fields (boating & medical devices)  Crosswalk industry standards to courses  Sustainability Challenges:  Sustain adults past 1 course

Case 2: National dissemination  ATE leader role:  Moving national industry materials to colleges  Goal:  Providelow-cost, up-to-date, industry- made IT materials  Rapid Development Mechanisms:  Identify IT platform providers with materials  Outreach to educators, pass costs to students, free training & materials  Sustainability Challenges:  Staying current

Case 3: Local industry exchange ATE leader role:  Develop instructional materials, communicating with industry Goal:  Enhance existing industry-college partnership in biotech Rapid Development Mechanisms:  “SWAT” team capacity  Division of labor around “safety training” Sustainability Challenges:  Rust belt economy  Biotech jobs pay half of old jobs  Global companies, no local loyalty

Case 4: Boot camp to program ATE leader role:  Workforce program development Goal:  Expand boot camp to college program Rapid Development Mechanisms:  DACUM Sustainability Challenges:  Timing market need: VC dry up  Keeping industry engaged  Facilitating discussions between educators/industry  “shop math” vs. “college math”

Next steps Partnership Study:  Follow up interviews with stakeholders  Development of cases, and possibly other tools Instruction Study:  Interviews to build cases: Describe 2 contrasting partnerships’ specific classroom instructional goals and programs  Classroom data to build cases: Select tech classes representing different levels of technical content and different emphases on technical vs. professional skills:  Instructional practice: Classroom observations and interviews  Curriculum: Artifacts rated by expert panels

Thank you  louise.yarnall@sri.com

Stephen Magura Kelly N. Robertson The Evaluation Center Western Michigan UniversityPresented at the 2011 National ATE PI Conference Washington, DC, October 27, 2011 Funded by NSF grant # 0832874

 Began 1992 Funding FY 11 - $64 million by NSF Approximately 40 centers & 200 projects Encompasses biotechnology, manufacturing, engineering, energy, IT Located in community colleges nationwide

1. “Producing more science & engineering technicians to meet workforce demands”2. “Improving the technical skills & general science, technology, engineering, & mathematics (STEM) preparation of these technicians” and3. “(Of) the educators who prepare them”

 Objective 1: Formulate a model for standardized measurement of outputs pertinent to ATE central goals 1, 2 and 3 that is relevant across different Projects and Centers. Objective 2: Determine which outputs individual Project and Centers are measuring as concrete steps toward achievement of ATE’s central goals and propose additional outputs that could feasibly be measured. Objective 3: Determine what types of evaluation designs individual ATE Projects and Centers are employing to determine impact and propose alternative or improved evaluation designs.

 Promote scientific assessment of effectiveness Application of objective effectiveness measurement strategies Better understanding of variations in success of grantees Return on investment of ATE portfolio to Congress

Objective 1. Existing material on ATE compiledfrom four sources:  Selected ATE Project/Center progress and final reports solicited by an NSF program official  Project/Center evaluator reports previously submitted to the ATE Resource Center  ATE Project/Center websites  ATE Projects/Centers described in the ATE Impact publications (Patton, 2008 a,b).

Objectives 2 and 3. One ATE Project was analyzed in each of ten industries and one ATE Center in each of seven industries. The Project and Center chosen within each industry based on the most information available. Purpose was to demonstrate that the proposed framework is applicable to ATE Projects and Centers across the range of applicable industries. Projects and Centers are anonymous in the report.

Post-Secondary Secondary Completed/ Completed/ Enrolled Retention Enrolled Retention Graduated Graduated (1) (2) (2 ÷ 1) (1) (2) (2 ÷ 1)A. Program ⃞ ⃞ ⃞ ⃞ ⃞ ⃞B. Course ⃞ ⃞ ⃞ ⃞ ⃞ ⃞C. Internship/ ⃞ ⃞ ⃞ ⃞ ⃞ ⃞ ApprenticeshipD. Dual Program/ ⃞ ⃞ ⃞ ⃞ ⃞ ⃞ Dual Credit Secondary Post-Secondary Exposed* Exposed*E. Software/ ⃞ ⃞ Materials

Secondary Post-SecondaryNumber of Educators who Complete… Elementary Middle High Faculty Industry ProfessionalProfessional Development Workshops ⃞ ⃞ ⃞ ⃞ ⃞ ⃞Professional Development Courses ⃞ ⃞ ⃞ ⃞ ⃞ ⃞Professional Development ⃞ ⃞ ⃞ ⃞ ⃞ ⃞Fellowships/MentoringProfessional Development ⃞ ⃞ ⃞ ⃞ ⃞ ⃞Software/Materials* Note: *Including hard copy and audio/visual materials for professional development purposes

Study Current Project Current Center Objectives Creates simulations that teach the Providing educators with Description underlying science principles of professional development in biotechnology & nanotechnology. manufacturing. Track # of teachers trained & Pre/post test to assess student2. Current self-assessment of learning. Plan achievement in relation to the topics Outputs the simulations intend to teach. to start asking teachers about implementation of learning. Quality of PD course. Test3. Recommend teacher skills, changes in Quality of the simulations. Outputs classroom practices, & student learning.4. Current Post-training satisfaction Evaluation Pre-test with repeated post-test. measures. Design5. Recommend Expert panel to assess quality of Pre-test with multiple post test simulations. Evaluation Compare student learning with for PD. Design cohort receiving standard course.

 Common ATE Project and Center outputs can be specified and potentially aggregated to yield output statistics for the national ATE program as a whole. The proposed framework, consisting of the figures and the tables in the report, narrows down and partly standardizes the types of data collected across ATE projects and centers.

 This standardization can result in meaningful aggregation of output measures that will make it possible to better determine program effectiveness. Additional instrumentation must be developed to assess the quality of STEM educational and outreach resources and their impact on students’ and educators’ learning and behavior.

 The evaluation framework is also useful because it identifies the gaps in instrumentation more precisely. The evaluation framework is very comprehensive, but all elements are not always applicable to any individual ATE Project or Center. This inherently quantitative data framework does not diminish the value of additional qualitative and narrative data that speak to the value, merit or worth of ATE programs.

 Some aspects of the proposed framework are outside the scope of any individual ATE grant and would better be pursued through targeted research. This report is not a final prescription, but may help frame further discussion of ATE evaluation.

My email:stephen.magura@wmich.eduThanks for your attention!

Ron Anderson rea@umn.edu October 27, 2011This project was funded by the National Science Foundation ATE Program forTargeted Research. The grant was to Colorado University’s DECA Project, LieselRitchie, PI, with a subcontract to Rainbow Research for Project I, Strategies forImproving Recruitment, Retention and Placement. 1

 Community College completion rates embarrassing low at 20 to 40% within 8 years. Advanced Technology Programs (ATP), while not as bad as non-ATP programs, still lose over 50% of their students before completion. Gender inequality, a serious problem in NSF ATE projects Recruitment of racial minorities improving in NSF ATE projects. NSF ATE projects neglect student advising & other strategies to retain students 2

*Data from Program Improvement Projects inWestern Michigan State annual ATE Survey bywww.evalu-ate.org 3

Data from Program Improvement Projectsin Western Michigan State annual ATESurvey: www.evalu-ate.org 4

Data from Program Improvement Projectsin Western Michigan State annual ATESurvey: www.evalu-ate.org 5

Advanced Technology Programs (ATP) fail toAttract Women. Data graphed are First-term Enrollments by Gender for ATP & Non-ATP Data are based on all students enrolled in Connecticut Community Colleges 1999-2009. (N=120,000) 6

 Many organizations are trying to address the completion/success gap in 2-year colleges Analytics movement attempting to forecast student dropouts Whitehouse Committee on Measures of Student Success ◦ Appointed in 2010 ◦ Sept. 2011 interim report ◦ April, 2012 target for preliminary report ◦ Years before impact likely 7

 Common Completion Metrics (National Governors Assoc.) Voluntary Framework of Accountability (AAAC) Foundations of Excellence in the First College Year (Gardner Institute) Complete College America Achieve, Inc (35 State network) Achieving the Dream (Database and Dashboards) Western Interstate Commission for Higher Education (WICHE) – Human Capital Database Project Gates Foundation - funded analytics initiatives National Agenda for Analytics (EDUCAUSE) 8

 Predictive Analytics (Capella U & others) Data Analytics (Sinclair Community College) Incisive Analytics (IncisiveAnalytics.com) Platinum Analytics (AstraSchedule.com) Action Analytics (Symposia in 2009 & 2010, and EDUCAUSE in 2011) Learning Analytics (1st International Conference on Learning Analytics, Feb. 27, 2011) Student Success Analytics (Purdue U., etc.) 9

 Analytics is sometime used as synonymous with ‘analysis’ to sound impressive. More precisely, ‘analytics’ refers to ‘predictive analytics,’ or analysis of trend data to predict future events of individuals or populations. Current analytics does not follow individual course-taking histories across time, thus it is weak in providing individualized information that students can use. 10

Typical Analytics Data: Trend Line, not a Trajectory(Trend lines fail to give any information about change in individual attributes overtime, only aggregates.) Percent of Students Completing Program X in each year, 2003-2008 100 90 80 70 60 50 40 46 30 40 39 37 37 20 10 0 2003 2004 2005 2006 2007 11

Cohorts Showing Student Trajectoriesfor 120,000 student histories in Conn. 12

Student-Pathway Trajectories showing Race Gaps Data are all 2,407 students first enrolled Fall, 2005 in the Community College of Rhode Island system. Completion is defined as graduation, articulation, or completion of 48+ credits within 7 terms (4.5 years). 13

 Recent, dynamic microsimulation techniques make it possible to follow individual course- taking histories (trajectories) across time Thus, using student transcript data records, models can be built that simulate student enrollment decisions term by term.. The results give information that students and student advisors can use to greatly improve their chances of completing a program successfully. 14

 Microsimulation model developed in Modgen programming language from Statistics Canada Hundreds of thousands of student transcript records from the CCs of Connecticut and Rhode Island were used as test data sets. For any given set of data, each scenario simulation is repeated for an equivalent sample of 5 million students to eliminate random variability, which only takes about 2-3 minutes. MicroCC developed with Targeted Research funds from NSF ATE program. 15

 Initial model includes 4 student choices or behaviors (details on next slide) Model’s core (predictive factors) are derived from data at hand ◦ 28 separate logistic (and ordered logit) regression models run to calculate coefficients for each factor and interaction that predicts success or completion Multiple scenarios can be simulated by modifying either ◦ starting populations (mostly demographic factors)  Gender, race, age, and initial full-/part-time status  effect coefficients for student decisions, or 16

Process Decision Points: MicroCC Completes thisDecision Sequence for each term of each Student 1) Enrollment 3) Number of /re-enrollment courses choice in each attempted term 2) Full vs Part Time 4) Successful enrollment in completion each term of each course attempted 17

 Success = completion of program (graduate, certificate, successful transfer, or completion of a required number of courses) Total courses completed = completion of 12 or more courses within 10 terms (5 years) 18

 Momentum Point One Passed - student completed 3 courses in first term Momentum Point Two Passed - student completed 6 courses in year one Stopout - student temporarily does not enroll in term X Stopouts -total terms student stopped out 19

 Used in MicroCC ◦ Gender (M/F) ◦ Race (W/B/L/O) ◦ Age (to 21/22+) ◦ Starting term enrollment full-time vs part-time Data not available in 2010 for MicroCC model ◦ Financial aid in term X ◦ Concurrent job ◦ Marital status ◦ Prior postsecondary education 20

 Data Restructuring – Creation of longitudinal file from term-level files can be done but it is time consuming. Missing Data – Records on transfer status, graduations, and certificate completions may be incomplete or nonexistent. Summer Term Challenge – can summer credits be ignored completely because there are so few regular students enroll in summer terms, or should credits and courses completed during the summer, be added into the counts for the previous term? Developmental Courses -- Developmental courses were tracked but institutions handled them differently. Transfer credits -- Are they added to new credits, and if so, when? Simultaneous enrollments -- In Connecticut we found many students enrolled in multiple colleges during a single term. 21

Screen print from MicroCC with Student Success Model for Baseline scenario with RI and CT data 22

◦ Data for MicroCC microsimulations came from two State enrollment databases:  Rhode Island Community College – 5 annual cohorts with most analysis just on the 2,502 students first enrolled in Fall 2005 for 4.5 years  Connecticut Community College system – 276,469 students in 10 cohorts beginning Fall 1999 to 2009. 23

Screen print from MicroCC with Student Pathways Models for Baseline scenario with RI and CT data Sample output table for student success rates by term Sample chart of growth of student completions from above table 0.15 % completed 0.1 0.05 0 1 2 3 4 5 6 terms 1 to 6 24

 Gaps in success can be deconstructed, identifying the student pathways that created specific portions of the gap. These results have direct relevance for students and guidance counselors, toward improving success rates. 26

Process Decision Points: MicroCC Completes thisDecision Sequence for each term of each Student 1) Enrollment 3) Number of /re-enrollment courses choice in each attempted term 2) Full vs Part Time 4) Successful enrollment in completion each term of each course attempted 27

 Most (90%) CT students in ATPs were in engineering and manufacturing programs. The remainder were in IT, network, and misc. science and technology programs. The 7,310 ATP enrollees in CT were only 6% of all CC students. As shown in the next chart, ATP students has a 17% higher completion rate than non-ATP students. 28

Source: 7,310 ATE Students in Connecticut CCs 2000-2009 29

 The amount of impact they have on success depends upon specific regions, schools, and curricular programs. If a student enrolls full time plus works full time and has children to raise, s/he might not do well in coursework and thus not keep up the momentum toward completion. 30

 But both students and their advisors need to understand how crucial these decisions are to pathway success:  1. To enroll continuously – no stop outs  2. To enroll full time  3. To take the larger numbers of courses each term, within reason  4. To pass the courses attempted. The simulation model incorporates these decisions, not just at first enrollment, but at every term in which the student is enrolled. 31

 Remaining charts from microsimulations illustrate how student decisions influence different subgroups of students within ATP programs in CT. Example 1, shows elements of gap between CT and ATP White and Hispanic men Example 2, highlights the higher completion rates of women over men in CT ATPs 32

Source: 7,310 ATE Students in Connecticut CCs 2000-2009 34

Women Outpace Men in all Race Categories - Percent of Students Completing their Programsby Gender & by Race in Conn. N=7,310 ATE students 60 50 49 50 48 43 40 37 35 30 Men Women 20 10 0 White Black Hispanic 35

 Microsimulation can uncover enrollment decisions that have huge effects on student success. These student decisions can sometimes explain demographic differences. Adding additional data, e.g., job history, financial aid and retention interventions, e.g., mentoring, as factors in the models, can make the methodology even more powerful. Enrollment forecasting can be done with greater precision. The model could also be extended to include post-schooling job trajectories as well. For More information contact Ron Anderson rea@umn.edu or 952-473-5910 36

1. The ATE program should invest in student tracking data systems, either in conjunction with existing student record systems or, better yet, a separate data system to which ATE-funded projects had to contribute.2. ATE-funded projects should be encouraged or required to address and report on student advising practices.3. Training should be developed for high school and community college student advisors regarding the needs of STEM students4. Recruitment of women (with improved advising) into STEM pathways needs to be given greater priority 37

 NSF ATE projects may be neglecting student advising & related strategies to retain students. Of the 305 projects and centers recently funded by the NSF ATE program, only two mentioned “student advising” or “guidance counseling” in their title or abstract. However, 10 projects (1%) mentioned “counselors.” ATE projects could utilize the findings of MicroCC simulations as guides for student advising. A system for student progress coaching and advising is needed with every ATE funded project 38

 Microsimulations should be run on many more States, college populations, and ATE program populations, so that findings could be tailored to specific groups of at-risk students. Input data for simulations should be expanded to include job status, financial aid, and other items relevant to student success. Microsimulation should be extended to include articulation and job acquisition processes. 40

For more information contact: Ron Anderson rea@umn.edu 41

2011 ATE Conference Panel Session

by American Association of Community Colleges

Accessibility

Categories

Upload Details

Usage Rights

Statistics

2011 ATE Conference Panel Session Presentation Transcript