First-Year ePortfolios: An Initiative to Involve Freshmen in an Assessment of Their Personal Success

First-Year ePortfolios: An Initiative to Involve Freshmen in an Assessment of Their Personal Success Tracy Gottlieb, Dean of Freshman Studies and Special Academic Programs Janet Easterling, Institutional Research Associate Mary Zedeck, Instructional Designer

Seton Hall University

A mid-sized, private, Catholic affiliated university in northern New Jersey

Main Campus in South Orange, NJ

Law School in Newark

40 off-campus sites

10,000 Students

4,500 Full Time Undergraduates (50% residential)

Laptops for all Undergrads and Faculty

Blackboard Course Management and Content System

Seton Hall Profile

Located in Suburban New Jersey on the train line to New York City

Incoming Freshman Class

Enrolls between 1050-1200 new freshmen each year

1 in 4 will commute 1 in 3 non-NJ 34 states

SAT ~1100 HS-gpa 3.3 HS-t op-10% 1/5 top-25% 2/3

Seton Hall Students and Family

Most live closer than 50 miles

50% parents with a college degree

77% live with both parents

9% English as a second language

20% will live at home

Family plays important role in college selection

“ Support from my family has made the biggest difference in the world. When I came to SHU, it was like we all received a letter of acceptance.” - T. J. Olsen, Communications Major ‘09

SHU Financial Demographics

82% family income less than $150,000

More than 70% of students have concerns about financing their education

“ I’ve worked all 4 years of college and it has only made me a stronger person. The cost of education is high but the benefits of a degree in this day outweighs the costs.” - Ivory Riley, Physician’s Assistant ‘06

Retention Background

2000: Retention identified as Presidential Priority

Academic Affairs, Enrollment Services & Student Affairs

2001-2002: Institutional Data Persistence Analysis

Identified “At Risk” populations

Researched National Best Practices

Began speaking in a language of “Pillars of Retention”

“ Pillars of Retention” Engaged Academically Integrated Socially (campus) Prepared Academically Financially Seeks/Finds Goal Clarity Supported by Family & Friends

“ Culture of Success” University-wide Data-driven Effort to Improve Retention

Institutional FACT BOOK

Persistence Data

Freshman Survey ( CIRP Cooperative I R Program )

National Student Survey of Engagement (NSSE)

Freshman Studies – student interviews

At Risk Cohorts

Intuitive Interventions

Identified At Risk Cohorts Commuter Students Students without a Declared Major Lower Academically Prepared Students Male Students

Four Key Goals #2 Increase retention to 2 nd year 80% to 84% (by 2007 for incoming 2006) #4 For students entering without a declared major 77% to 80% (by 2007 for incoming 2006) #1 Increase five and six year graduation rates from 55% to 57% (by 2007 for incoming 2002) from 57% to 60% (by 2008 for incoming 2002) #3 For commuter students 75% to 78% (by 2007 for incoming 2006)

1st year Retention of Freshmen

Freshman Studies At SHU

Three-pronged approach:

Mentor

Peer Adviser

University Life Course

Freshman Studies = Success

Help Students to

Develop networks of friendships and academic resources

Adjust to new surroundings

Clarify personal, academic and career goals

A Freshman Studies Mentor Is

A first resource for questions

An Advocate

A Professor

An Academic adviser

Most importantly, a friend

The University Life Course

Teaches students how to use Seton Hall’s vast technology network

Orients students to the library, the career center and other Seton Hall resources

Provides a forum to discuss important study skills issues about time management, learning styles and test taking

The University Life Class also

Provides a forum to discuss important social issues that confront college students

Acquaints students with the academic procedures for registration

Insures that students have weekly contact with a mentor and peer adviser

Teaches students how to plan their academic and professional career while they are at Seton Hall

“ Strives to provide a common experience for Freshmen enrolled in the University Life course”

Historically has recognized the benefits of educational technology

2004 received a Curriculum Development Initiative grant (internally funded) to redesign University Life Course

First Year Implementation

Single assignment

Students required to complete ePortfolio by semester end

Tech support available

Lessons Learned First Year Implementation (2005)

Students did not see the connection between the goal and the task (perceived as onerous)

Support came from tech people instead of from mentors and peer advisors

Portfolio was due at the end – students scrambled at the end to complete it

Students wanted to customize their pages; difficult to do in Bb environment

Second Year Implementation

Emphasized the use ePortfolio as an opportunity to evaluate first semester through reflection

Technical support came from mentors and peer advisors

Checkpoint Assignments due throughout semester

Fall 2006 ePortfolio Requirements

Required Elements

Welcome

Academics

Activities

Poetry Reflection

COMPASS Events

Career Goals

Community Service

My Thoughts

Additional Elements *

Reflection on Ethics or Computer Skills

Photo Album

Blog/MySpace

Resume

Art Work, Music or Multimedia of Own Creation

* Students must choose 2

How Far We’ve Come… Percentage Very Satisfied and Satisfied 2005 2006 2007 I understand the concept of and importance of ePortfolio as it was presented in class 52% 66.7% 76.2% The ePortfolio lesson was clearly presented in class 60.8% 79.2% 87% The system used to create the ePortfolio was easy to use 46.6% 68% 75.4% My mentor and/or peer advisor was able to answer basic technical questions regarding ePortfolio 77.1% 89.4% 92%

Allow mentors to

Gain more insight in their students

Reach out to students who need help

ePorts can Improve Connections Between Mentor and Students

How can we know which students need help? Do we know what kinds of help they need?

For the student, ePortfolios provide

a “place” to reflect on who they are, their goals, and their university experiences as they explore where they are and what they are working toward

For the university, ePortfolios provide

a “place” to see past freshman year reflections a source of qualitative info the university can explore to learn from their students experiences and needs

International/National Coalition (I/NCEPR) for Electronic Portfolio Research Across the country and around the world, at institutions of all kinds, students are creating electronic portfolios—for personal planning, in learning communities, in the major, and on and on. “ What learning is taking place as a function of digital portfolios? How do we know?” NCEPR brings together faculty from 10 campuses in successive cohorts. Each designs a campus-based research agenda, implements, reports and shares.

SHU’s ePortfolio Study: Research Qs Can electronic portfolios serve as a useable source of information concerning important student experiences in the first year? What can be learned from a systematic review of freshman ePorts? Will rubrics based on persistence theory produce meaningful data? Will these data increase our ability to predict student persistence to second year?

SHU’s ePortfolio Study: Project Design

Review of prior year freshman ePorts  create scores

non-cognitive data: student experiences & thoughts

quality of effort data: how well ePort was constructed

Scores ‘measure’ ‘levels’ - key attributes

known to associate with persistence and

reasonably present in the freshman 2006 ePortfolios

Key attributes + goal clarity + quality of effort

+ social integration + academic engagement

+ support of family & friends

Attributes reviewed in freshman ePortfolios

Persistence attributes we believed might be evident…

Goal Clarity

extent a student has career goal(s) and a clear academic plan that connects to those goals

extent current academic efforts seen as preparatory or central to carrying out that plan (Tinto)

Perceived Support from Friends and Family

The level of support from student’s friends and family for student to attend college

and particularly, to attend this college

(Cabrera et al)

Quality of Effort

Completeness (all elements as assigned)

Organization of content

Mechanics of good writing

Readability / Aesthetic Quality

(Chickering & Gamson,1983)

Academic Engagement

Extent a student finds programs and courses to be intellectually challenging, relevant and thus motivating to work hard in academic pursuits meaningful

(Tinto; Pascarella & Terenzini; Kuh)

Social Engagement

Extent the student, via participation in organizations, events and campus activities appears connected to the social and organizational fabric of SHU campus (Pascarella & Terenzini; Tinto)

ePortfolio assignments: 2 example Probes Activities What activities have you been involved in, on or off campus, that have enhanced your first year of college? Describe in 200 words or more and include images and web resources where appropriate. (Due Week 10) Career Goals Where do you see yourself after graduation? If you are unsure, this section should explore career paths that are interesting to you, and how you are going about exploring these differing areas… Describe in 200 words or more and include images and web resources where appropriate. (Due Week 13)

Attributes reviewed – links to theory

Goal clarity , academic and career goals (Tinto, 1993)

Quality of Effort (Chickering & Gamson,1983)

Support from friends and family (Cabrera et al,1993)

Social Integration (Pascarella & Terenzini,1991; 2005; Kuh, 2001, Sandler, 2000)

Academic Engagement (Pascarella & Terenzini,1991; 2005; Tinto,1993)

Methodology

Ensure meaningful sample (SHU students)

Low er Academically Prepared - all Summer Scholars (summer bridge program) - students with SAT below lower bound

High er-End Majors - admitted on entry to International Relations - admitted on entry to Biology program

Mid dle (prep levels typical for SHU) - Communication program major - Political Science major

Methodology

Ensure meaningful scores

Rubrics: realistic, useable, reliable

Realistic Initial Crafting in depth reviews of prelim sample 40 ePorts

Many prelim Scorings, many Refinements Team scoring of 40; concordance reviews Lead mentor scoring; refinements

Inter Rater reliability

50 ePorts double scored

5 Rubrics – used holistic as here (Goal C) & analytical approach (handout)

Goal Clarity (Academic and Career Goals)

4 – Mature focus of academic and career goals, enterprising and realistic

3 – Clear expression of academic and career goals

2 – Elements of career exploration, some sense of career goals or academic goals

1 – Lack of focus or clarity of purpose expressed about college (academic and/or career goals), goal uncertainty

Rubric Example SHU Integration and Engagement

Data Analysis sample n = 384

E-portfolio Scorings January (today’s presentation) and April 2008

Student Profiles

Rubric Score Profiles

Persistence Profiles

Associations – Correlations

Associations – Linear Regression scored n = 370

Student Profiles

Demographics

Gender

Race/Ethnicity

Commuter vs Residence Hall Student

Student Profiles

How Academically Prepared for College?

SAT High School GPA

Student Profiles Entering Major Other Undec Di Bio Com Pol

Score Profiles Quality of Effort – Assignments

Score Profiles Quality of Effort – Mechanics (grammar, etc)

Score Profile Quality of Effort (4 sub-scores combined)

Score Profile Goal Clarity

Score Profiles Support of Family, Friends

Score Profiles Academic Engagement

Score Profiles Social Interaction (campus)

Persistence Profiles Returned for Spring

with a 2.5 or better with a 3.0 or better

Persistence Profiles Returned for 2 nd Fall

with a 2.5 or better with a 3.0 or better

Associations – Return Minimally Good Standing (2.5) all Correlation Analysis ** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed)

Associations – Return Minimally Good Standing (2.5) all Step Lin Regression hierarchical score block last Summary of all 3 Models Coefficients for Model 3

Associations – Return Minim G Standing (2.5) low prep<950 Step Lin Regression hierarchical score block last Summary of all 2 Models Coefficients for Model 2

Associations – Return Min Good Standing (2.5) prep adeq Step Lin Regression hierarchical score block last Summary of all 2 Models Coefficients for Model 2

Associations – Return Good Standing (3.2) all Step Lin Regression hierarchical score block last Summary of all 3 Models Coefficients for Model 3

Smaller Picture – study conclusions