Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tracking Student Access to High-Impact Practices in STEM


Published on

We know that certain “High-Impact Practices,” such as internships, undergraduate research, capstone courses, and learning communities, help undergraduate students persist and succeed. These practices have a disproportionately positive impact on students from underrepresented backgrounds. This webinar will briefly summarize the evidence for High-Impact Practices (HIPs) and share innovative efforts from California State University, Northridge and the University of South Carolina to track and analyze underrepresented student participation and outcomes.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Tracking Student Access to High-Impact Practices in STEM

  1. 1. Diversity in the Biomedical Research Workforce Webinar Series Wednesday, March 8, 1:00-2:00 p.m. ET
  2. 2.  If you cannot hear the audio, check the “Audio” pane on the control panel. You can use your speakers or dial-in using your telephone.  You may ask questions at any time using the chat box.  Handouts
  3. 3. Jennifer C. Danek, MD Senior Director, Urban Universities for HEALTH USU/APLU
  4. 4.  Webinar series on Diversity in the Biomedical Research Workforce  Share findings from the USU/APLU/AAMC report (July 2016)  Upcoming webinars: work/upcoming-events
  5. 5.  Collaborative effort of APLU/USU and AAMC, supported by NIH  70 experts from 28 universities/academic medical centers  Identify research actions for improving evidence  Examined four areas:  Diverse Faculty Hiring and Advancement  Leadership, Organizational Change, and Climate;  Diverse Student Success;  Recruitment and Admissions
  6. 6. Tia Brown McNair, Ed.D. Vice President of the Office of Diversity, Equity, and Student Success, Association of American Colleges & Universities
  7. 7. Tia McNair, Ed.D. Vice President for Diversity, Equity, and Student Success Evidence for high-impact practices (HIPs) and how they are linked with student success
  8. 8. Intentionality
  9. 9. Critical Questions • How can HIPs help students become intentional learners? • What do you want to accomplish by focusing on the design and development of HIPs? What are the outcomes? • What HIPs currently exist on your campus or in your program? • What do you know about who participates? • Who has access? Who doesn’t?
  10. 10. Intentionality of HIPs • Selection • Design • Access HIPs • Defined • Evidence Learning Outcomes • Assessment • Data Disaggregated • Integrated Equity
  12. 12. HIPs: Eight Key Elements • Performance Expectations Set at Appropriately High Levels • Significant Investment of Time and Effort by Students Over an Extended Period of Time • Interactions with Faculty and Peers about Substantive Matters • Experiences with Diversity • Frequent, Timely and Constructive Feedback • Structured Opportunities to reflect and Integrate Learning • Opportunities to Discover Relevance of Learning Through Real-World Applications • Public Demonstration of Competence Source: Kuh, George D., and Ken O’Donnell. 2013. Ensuring Quality and Taking High-Impact Practices to Scale. Washington, DC: Association of American Colleges and Universities.
  13. 13. HIPs Results of NSSE 2016 60% of first-year students surveyed participated in one HIP, with 7% of them participating in two or more HIPs. 90% of 2016 NSSE Seniors participated in one HIP, with 68% of them participating in two or more Source: NSSE 2016 High-Impact Practices (institutional report)
  14. 14. NSSE HIPs 2016 in relation to certain high-impact experiences The data to the right includes compares the percentage of students from NSSE 2016 who participated in a High-Impact Practice, including the percentage who participated overall (at least one, two or more), with experiences in internships, undergraduate research, capstone courses, and learning communities highlighted. Source: NSSE 2016 High-Impact Practices (institutional report)
  15. 15. “Ensuring Quality & Taking High- Impact Practices to Scale” “Proportionately fewer first-generation students, black and Hispanic students, and transfer students do research with a faculty member, study abroad, do an internship, or have a culminating senior experience.” (Kuh & O’Donnell, 2013)
  16. 16. Source: NSSE 2016 HIPs Institutional Report
  17. 17. What are the effects of participation in certain high-impact experiences? What are the effects of participation in multiple high-impact experiences? Finley & McNair, Assessing Underserved Students’ Engagement in High-Impact Practices, 2013.
  18. 18. Outcomes Examined • Deep Learning = Pursuit of learning beyond memorization to seek underlying meanings & relationships • Gains in General Education = Writing/speaking skills, acquire broad general educ, analyzing quant. probs • Gains in Practical Competence = Work related knowledge & skills, working effectively w/ others, use of technology, quant. problem-solving, solving complex real- world problems • Gains in Personal & Social Development = Developing ethics, understanding diff. bkgrds, understanding self, contributing to community, voting Finley & McNair, Assessing Underserved Students’ Engagement in High-Impact Practices, 2013.
  19. 19. HIP Participation vs. No Participation: Avg. Boost Across All Outcomes Finley & McNair, Assessing Underserved Students’ Engagement in High-Impact Practices, 2013.
  20. 20. Effect of participation in Multiple HIPs on Outcomes StandardiuzedScores(0-100)
  21. 21. What is the effect of participation in multiple HIPs relative to students in the same group who do NOT participate?
  22. 22. Avg % Increase in Outcomes w/ Participation in Multiple HIPs Vs. No Participation (by First-Generation & Transfer Status) 1-2 HIPs 3-4 HIPs 5-6 HIPs 0% 10% 20% 30% 40% First-Gen Transfer 11% 14% 24% 26% 35% 40% 1-2 HIPs 3-4 HIPs 5-6 HIPs Finley & McNair, Assessing Underserved Students’ Engagement in High-Impact Practices, 2013.
  23. 23. Bettina Huber, Ph.D. Director of the Office of Institutional Research, California State University, Northridge
  24. 24. (No. of respon. Percent on which HIP Attempting percent based) Service Learning 47.9 (828) Internship 48.3 (782) Senior Experience 33.3 (782) Research with Faculty 13.4 (781) Study Abroad 8.3 (782) Table 1. Percentage of CSUN Senior Respondents Attempting Selected High Impact Practices (Spring 2007 NSSE Respondents)
  25. 25. Table 2. CSUN Senior NSSE Respondents' Participation in High Impact Practices (Spring 2007 NSSE Responses) HIPs participated in Percent Number Zero (no participation) 20.3 159 One 32.7 256 Two 28.3 222 Three or more 18.8 147 Three 13.8 108 Four 4.6 36 Five 0.4 3 Total 100.0 784
  26. 26. 2.98 2.98 3.09 3.11 2.00 2.50 3.00 3.50 4.00 None 1 HIP 2 HIPs 3 or more HIPs MeanGPAatCSUNExit Figure 1a. Impact of Participation in High Impact Practices on CSUN GPA of Senior NSSE Respondents [Eta=.111 (.021)] 5.67 5.58 4.78 5.14 4.0 4.5 5.0 5.5 6.0 None 1 HIP 2 HIPs 3 or more HIPs MeanTimetoDegree Figure 1b. Impact of Participation in High Impact Practices on Time to Degree of Senior NSSE Respondents Entering as First Time Freshmen [Eta=.196 (.024)]
  27. 27. 2.00 2.20 2.40 2.60 2.80 3.00 3.20 3.40 3.60 3.80 4.00 Latina/o Respon. Other Respon. Pell Grant Recip. No Pell Received MeanGPAatCSUNExit Figure 2. Impact of Participation in High Impact Practices on CSUN GPA by Racial & Ethnic Background and Pell Grant Status of Senior NSSE Respondents None 1 HIP 2 HIPs 3 or more HIPs [Eta=.221 (.002)][Eta=.090 (NS)][Eta=.203 (.053)] [Eta=.079 (NS)]
  28. 28. 6.8 5.2 5.9 5.3 5.0 4.7 5.4 5.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 Latina/o Respondents Other Respondents MeanTimetoDegree(inYears) Figure 3. Impact of Participation in High Impact Practices on Time to Degree by Racial & Ethnic Background of Senior NSSE Respondents Entering as Freshmen None 1 HIP 2 HIPs 3 or more HIPs [Eta=.205 (.094)][Eta=.295 (.055)]
  29. 29. SHORTCOMINGS TO INITIAL APPROACH Limited number of participants drawn from one graduating class.  Little distinction in NSSE items between one-time and multiple HIP participation.  Uncertainly about how student participants interpret NSSE HIP items.
  30. 30. Pam Bowers, Ph.D. Associate Vice President for Planning, Assessment and Innovation, University of South Carolina
  31. 31. ~ 33,000 students ~ 26,000 undergraduates ~ 7,000 first-year students University of South Carolina Carnegie Classification: • RU/VH • Community Engagement
  32. 32. Beyond The Classroom Matters™ Incorporates records of each student’s involvement in non-credit, educationally purposeful support and enrichment programs into institutional data to produce a comprehensive student record. “The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extracurricular offerings on a campus,” Pascarella & Terenzini Pascarella, E. & Terenzini, P. (2005). How college affects students (Vol. 2): A third decade of research. San Francisco: Jossey-Bass.
  33. 33. • Academic Advising Programs • Alcohol & Other Drug Programs • Campus Activities Programs • Campus Religious and Spiritual Programs • Career Services • Civic Engagement & Service-Learning Programs • Clinical Health Services • Commuter and Off-Campus Living Programs • Counseling Services • Disability Resources and Services • Education Abroad Programs & Services • Financial Aid Programs • Fraternity and Sorority Advising Programs • Health Promotion Services • Housing and Residential Life Programs • International Student Programs and Services • Internship Programs • Learning Assistance Programs • Lesbian, Gay, Bisexual and Transgender Programs and Services • Multicultural Student Programs & Services • Orientation Programs • Parent and Family Programs • Recreational Sports Programs • Sexual Violence-Related Programs & Services • Student Conduct Programs • Student Leadership Programs • Transfer Student Programs and Services • TRIO & Other Educational Opportunity Programs • Undergraduate Research Programs • Veterans and Military Programs & Services CAS Categories (Council for the Advancement of Standards in Higher Education)
  34. 34. High Impact Practices – Key Elements • High performance expectations, clearly communicated • Significant investment of student time and effort, over extended period • Interactions with faculty and peers on substantive matters • Experiences with diversity, people and circumstances not familiar • Frequent, timely, constructive feedback • Periodic, structured opportunities for reflection on learning • Real-world application of learning • Public demonstration of competence Kuh, G. & O’Donnell, K. (2013) Ensuring Quality & Taking High Impact Practices to Scale, AAC&U.
  35. 35. Beyond The Classroom Matters Comprehensive Student Record Student records are - defined in catalog entry - created by sponsoring department -collected in Student Records Repository -interfaced with Banner records -managed in data warehouse COGNOS BANNER SRR EDC
  36. 36. Beyond The Classroom Matters
  37. 37. Core Principles of Improvement Carnegie Foundation Improvement Research 1. Make the work problem-specific and user-centered. 2. Variation in performance is the core problem to address. 3. See the system that produces the current outcomes. 4. We cannot improve at scale what we cannot measure. 5. Anchor practice improvement in disciplined inquiry. 6. Accelerate improvements through networked communities.
  38. 38. Student View of Records • Records are sortable by term or category • Each entry is linked to catalog description • Advisor has access to student records
  39. 39. Student can select activities (left) to create a personalized Experiential Learning Record and can store multiple documents (below)
  40. 40. Experiential Learning Record • Each entry linked to catalog description • Two ELR format options
  41. 41. Lessons Learned • It’s not a technology project, primarily • Requires defining educational purpose, intentional program structure • Requires calibration of language across multiple departments • It is a technology project Non-Credit Records Systems - Symplicity - Handshake - Maxient - Tutor-Trac - Advisor-Trac - SARS - Odyssey - CollegiateLink - Excel, Access, Google-Docs Academic Records System - Banner
  42. 42. Student Academic Support Programs Learning Community Student Leadership Roles Internship Community Service Career Development Undergraduate Research Student Degree Program Course 1 Course 2 Course 3Course 4 Course changes Major changes Comprehensive Student Record Links records of educational activities for each student, within and beyond the classroom. Beyond The Classroom Matters
  43. 43. P. Kay Lund, Ph.D. Director of the Division of Biomedical Research Workforce, National Institutes of Health
  44. 44. Division of Biomedical Research Workforce (DBRW) Office of Extramural Programs Office of Extramural Research Office of the Director NIH Perspective P. Kay Lund PhD
  45. 45. The NIH is committed to improving the diversity of the biomedical & physician scientist research workforce • Workforce composition affects the quality and impact of biomedical research, health and health disparities. • Universities play a key role in recruiting and retaining scholars into these fields. • Multiple factors impact success of students from under-represented groups at multiple career levels • A multi-faceted strategy is needed to maximize their success. • Diversity is a top priority for the Division of Biomedical Research Workforce. • Our division has launched a new extramural diversity website • We also have an interagency working group on graduate education and interacting with the interagency group re broadening participation in STEM and the biomedical sciences.
  46. 46. NIH Extramural Diversity Website New! 48 Lisa Evans and TAC
  47. 47. Programs to Enhance Diversity • By Career Stage: • F31, Predoctoral Fellowship to Promote Diversity in Health-Related Research (note due dates April 8th, August 8th, December 8th) • T32, Training Program for Institutions That Promote Diversity • T34, Maximizing Access to Undergraduate Research Careers (MARC) • K01, Mentored Career Award for Faculty at Institutions That Promote Diversity (some IC) • G12, Research Centers in Minority Institutions Program • SC1, Support of Competitive Research (SCORE) Research Advancement Award • R25, Multiple Research Education Programs (Research Initiative for Scientific Enhancement (RISE); Multiple NCI Cancer Education Programs) • Other programs • Small Business Innovation Research & Small Business Technology Transfer (SBIR/STTR) Programs • NIH Loan Repayment Programs (LRP) 49
  48. 48. Research Supplements to Promote Diversity in Health-Related Research Administrative supplement to an existing research grant, designed to: • Support candidates from underrepresented groups who “wish to develop research capabilities... participate in… career development experiences” • Support many career stages from undergraduate to faculty • Diversify the biomedical research workforce PA-16-288: 288.html • Administratively reviewed by the Institute or Center (IC) funding the original grant – Note: different ICs have different deadlines and policies 50
  49. 49. Diversity supplements at all career stages 51 Average Award Size FY2014Distribution of Awards FY2014
  50. 50. Rigorous evidence on outcomes of diversity strategies will inform policy and programs • Outcomes analyses of Diversity supplements • Evaluation of existing programs: R25 Education programs, F31 Pre-doctoral, F32 Postdoctoral, Career Development (K- Awards) • Partnerships between Research Intensive Universities & Institutions serving historically under-represented groups – Institutional Research and Academic Career Development Awards (positive outcomes:
  51. 51. NIH Research Training Website Launched in 2015, one stop for funding opportunities Useful resource for trainees and early stage faculty Modifications and integration with new DBRW website in progress
  52. 52. 54
  53. 53. For Program Leaders Building Participation 55
  54. 54. For Trainee Applicants Career Pathways 56
  55. 55. Example: NIH R25 Programs at College-Stage 57
  57. 57.  Please submit questions through the chatbox
  58. 58.  Contact info: ◦ Julia Michaels, Project Manager ◦ (202) 478-6071 ◦  Next webinar… Supporting Minority Postdocs Tuesday, April 18, 1:00-2:00 p.m. Eastern Time