KMT Oct 31 2012 Assessment

Full Circle Assessment
October 2012
Summarizing FY 12 and Planning FY 13

 It assists us in our daily, quarterly and yearly
planning and forecasting.
 It creates a path of self improvement.
 It allows us to benchmark other colleges and
other Ai’s
 It has always been part of our activities. We are
working towards consistency in
◦ how we report assessment and
◦ how it will facilitate our engagement in the important
work we all do to provide a quality education for our
students.
 It is our culture.

INSTITUTIONAL EFFECTIVENESS ANNUAL REPORT – FY 2012
DEPARTMENT: Office of the President NAME: George W. Sebolt

•MISSION:
TO PROVIDE AN EDUCATION IN DESIGN, MEDIA ARTS AND CULINARY THAT PREPARES STUDENTS FOR INDIVIDUAL AND PROFSSIONAL SUCCESS.
INSTITUTIONAL GOALS (SUPPORTED):
1. Operational Effectiveness
2. Growth
3. Academic Quality
4. Regional and Global Partnerships
UNIT PURPOSE STATEMENT:
Provide leadership to all operational units on the campus.

Learning Outcomes/ Department Means of Assessment Criteria for Success Assessment Results Use of Results to Improve Outcomes
Objectives

1. Ensure development of all 1a. The 3-year strategic plan is 1a. Timely submission of 1a. Updated college Strategic 1a. Revised college mission
planning documents for the updated annually on a rolling strategic planning documents. Plan ahead of schedule. BoT statement with college
institution basis. and campus community community (campus & online)
suggested that college mission and BoT input.
1b. The IEP is updated at least 1b. Annual IEP deadlines are statement be revised to better
annually and informs the met according to published reflect institution.
Strategic Plan. timelines. 1b. Monitor for next cycle.
1b. Achieved deadlines. IE
plans tie into strategic plan.

2. Ensure appropriate 2a. Conduct at least semi- 2a. Meetings are documented. 2a. Accomplished. 2a. Continue
communication throughout the annual meetings with all staff
campus and to stakeholders and faculty. 2b. Continue. Redesigned
student lounge based on
2b. Meetings are documented. 2b. Accomplished. student body request for
2b. Conduct quarterly student larger more open space.
involvement meetings.
2c. Gap score is under 1.0 for 2c. Continue to work with
2c. ―Satisfaction‖ = gap score items 2 and 10. Did not campus staff and faculty to
2c. Results of the Noel Levitz of 1.0 or less. achieve for items 57 and 71. improve gap scores for 57 &
survey for items 2, 10, 57, and (See chart below) 71.
71.
2d. ―Satisfaction will be at or 2d. Meet with all departments
2d. Improve or maintain above system average. 2d. FY12 satisfaction scores to address issues and provide
employee feedback survey were below the system a forum for communication.
scores on employee average.
communication.

• What We Assessed
The 3-year strategic plan is updated on a rolling basis

• Summary
Updated the college strategic.
Feedback Revise Mission Statement
Formed a mission statement committee.
College Community (campus & online) and BoT provided
input.

• Next Steps
Will continue process of reviewing the strategic plan yearly.

FY’12 Accounting Institutional Effectiveness Plan

 Monitor and refine internal collection process
 Ensure that out of school accounts are collected in a
timely fashion
 Establish payment plans for students while they are
in school
 Provide information to all campus department heads
in a timely fashion
 Provide effective customer service to the college
community

#1) Monitor and refine internal collection
processes to reduce collection outsourcing

• Reviewing process determined that we were not
doing enough to communicate with students
regarding past due account balances.
• Additionally, bad debt has been steadily increasing
year over year.

 Ways to improve communication with students
regarding past due accounts?

 Implemented four step communication
approach:
 Automated Phone Call
 Personalized Letter
 Account Hold
 Personal Follow-Up Phone Call

 By implementing an effective communication
approach we hoped to:
 Reduce the number of past due accounts each month.
 Increase internal collections
 Decrease overall bad debt

While results were not as favorable as we had hoped, the
process of communication and collection on past due
account has improved slightly.

Kimbrea Browning
Chris Farrell
Jennifer O’Brien

 Recruit students with interests in line with Ai
Programs
◦ Calculated % of plan for Admissions year (reaching
100 % or exceeding)
 Prospective students will find the admissions
staff knowledgable about program offerings
◦ Noel Levitz results (students satisfied with
admissions process)

• Ensure that the admissions staff follows
practices and policies for ethical recruiting.
– Required three week training and receive 85%
accuracy on Mastery and FAQ test.
– Interview and Appointment set Observations by
SDoA or DoA. Monthly report showing observation
percentage for campus.
– Noel Levitz score (students are satisfied with the
way the campus is portrayed in recruiting practices.

 Admissions staff will demonstrate knowledge of
academic programs and will receive training to acquire
information and knowledge.
◦ Product Knowledge training done regularly for
understanding of campus program offerings.
◦ Noel Levitz score (students are satisfied with the
services that help them decide upon a career)
 Accurate and complete new student records are
transferred to the Registrar’s office in a timely manner
to facilitate processing of required
◦ Data (100%) on new student academic records
transferred to the registrar’s office from the
enrollment processor.

 Our primary objective is to continue growth of new students.
Our FY outcome fell short about 7 % which is attributed to
several factors. Our change in inquiry distribution with HS
inquiries had a major impact on our performance in the
months of July, August and October for FY 13.
 We have some opportunity in terms of ADA knowledge and
customer service to students. We will continue to initiate
more product knowledge program training versus the sales
training that was such a huge focus. Our coaching
observations will have to increase to ensure these intangible
items are happening with each and every student. The
numerical reporting is not a good indicator of customer
service.

 Primary indicators of effective performance.
 Importance of employee development and the
impact on our students.
 Importance of other departments knowledge
of ADA job role.

What was assessed?
Students Records, FERPA, transcript processing, graduate auditing, and overall satisfaction
of services provided to the AiP community.

Summary
The Registrar’s Department is meeting the benchmarks for areas of assessment
I want to develop and implement a Registrar survey. It will be given to students and
alumni . I want to know what service we are not providing.

What I learned from assessment?
Even though the method of auditing students file has changed; paper files to Image Now,
it is still looking for the accuracy of all new files.
As an established school, AiP does an incredible amount of transcripts. I am proud we
maintain the 48 hour turnaround despite changes to personnel.
Regarding the assessment process itself: making time to plan your cycle of assessment and
the methods used is the key to having a meaningful experience.

Learned?
Continue focus on ―belonging‖ at Orientation and Freshmen Studies

Development and growth of AiPStudent.com

Impact of improvements in the Student Lounge

New Student Council campaign – increased meeting attendance and
diverse student involvement

―Comments? Questions? Concerns?‖ button on AiPStudent.com

Results (―Satisfaction‖=gap
Factors to be assessed score of 1.0 or less)
 Provide move-in ready student • Factor 3: Achieved
housing (EBI Survey-Factor 3) Satisfaction
EDMC mean: 4.70
AiP Mean: 4.30
 Provide safe and secure housing
option for students (EBI Survey-
• Factor 8: Achieved
Factor 8)
Satisfaction
EDMC Mean: 5.00
 Provide caring and competent AiP Mean: 4.30
housing staff and policies that
reflect student needs (EBI Survey - • Factor 5: Achieved
Factor 5) Satisfaction
EDMC Mean: 4.58
AiP Mean: 4.41

Lessons Learned / Changes to be instituted
-For Factor 3, Question 50: floor coverings; we re-stained and sealed all of
the corridor floors throughout Shannon Hall as well as re-staining and
sealing apartment floors when units are renovated

-For Factor 8, we remain committed to adhering to a schedule of safety
inspections and building rounds as well as keeping strong lines of
communication open with security staff to ensure that safety issues are
addressed in a timely manner

-For Factor 5, focus groups conducted after the administration of the EBI
survey showed that students were unhappy with the number if overnight
guests they were permitted; the policy was revised to allow students in
good academic and housing standing to have more overnight guests

• Assessment: students, faculty and staff will be satisfied with the
customer service and materials available in the store.

• Results: Noel-Levitz survey results #54 – bookstore staff are helpful:
FY11 - 0.63 gap – FY12 - 0.44 gap.

• Key learning: the store is a key resource for students, faculty and
staff.
– It’s important for the store to be relevant and timely with key items and
information to support student and faculty success.

– It’s important for the store to listen, offer solutions and respond.

– It’s important for store staff to be knowledgeable about services campus-wide.

*Quarterly IDEA surveys aggregate scores

*Noel Levitz Survey - #3, 25,39,41,47 and 65

*Faculty Development Plans

*Yearly In Service Sessions

*Faculty Participation in Professional Events
and Memberships

 Aggregate scores for FY 2012 were 3.8 per quarter with individual
scores averaging 3.5 over 4 quarters within the dimension regarding
stated outcomes for the course.
 Noel Levitz satisfaction scores fell within the gap score of 1.0 or less
(from -0.16 to -0.03) as compared to FY 11.
 Faculty Development Plans are individualized according to the
strategic plan with all full time faculty attending on average 24
hours of faculty development and professional development training
both on campus and through professional associations.
 In service hours provided for faculty during FY12 exceeded the
required 10 hours per year of training. We offered 94.5 hours of
training, 57.5 of those hours focused on student learning outcomes.
 Faculty files reveal that in FY 12, 66 full time faculty had
memberships to professional organizations and of the 94.5 hours of
training, 33 hours were specific to professional, departmental and
program development.

Lee Ann Johnston
Katie Speers

• The success rate of graduates seeking
employment within six months of graduation.
• The employment services and professional
development for in-school students.
• Services provided to promote and sustain
continuous relationships with alumni.

• The Art Institute of Pittsburgh FY12 statistics closed at
74.6% and salaries of $25,886. Student satisfaction was
79% and employer satisfaction showed that 92% felt that
our graduates had the needed skills and 89% felt they
had the needed professional skills.
• In FY12 we averaged nine employers at part-time job
fairs and we did not have any internship fairs.
Presentations were conducted in Freshman Studies,
Advanced Communications and through industry
related career workshops.
• Since the elimination of the Alumni Coordinator
position, the majority of alumni outreach is handled at
the corporate level through the alumni website.

• Career Services was impacted in 2012 by staff
reductions and changes, a weaker than normal
economy, and challenges in several programs to
locate enough field-related openings.
• We need to increase our exposure to students
earlier on in their academic Career to promote the
need for field-related experience while in school
and enhance their professionalism.
• We need to have a more strategic method of
working with alumni on a school level, particularly
because AIP has the largest alumni network in the
system.

 Faculty, staff, and students will have
appropriate technology resources available to
support working and learning environments.

 Technology systems will function in an
efficient manner.

 Faculty, staff, and students will be satisfied
with the services provided by the technology
department.

 By tracking survey results and system logs we
have learned to better utilize our time and
resources in areas that meet the need of the
school and our students.

 Our results have helped us to identify other
areas for assessment in FY 13

• What Did we Assess?
– Students success based on at-risk debt difficulties
– Persistence of students
– Timeliness of financial information to students/ families to aid in decision
making
• Summary:
– We regularly identify and counsel students in an at-risk status financially
– All students complete their FA paperwork on time each year and our total
drops decreased compared to last year.
– The perception based on Noel-Levitz survey is that students are not happy
with our level of customer service from SFS
• What we learned:
– We increased scholarship awards from last year and educated students
and families on things we can do to help with FA, we need to continue to
monitor and work towards moving students/families through the FA
process more quickly, continue to provide customer service trainings to
the staff to ensure we are providing the highest level of service to change
overall perceptions.

 What was assessed?
◦ Employee Communication
◦ Staffing
◦ Employee Relations
◦ Benefits (401k)
 Summary of Assessment
◦ Low employee morale
◦ Need for more communication regarding training,
development, benefits, general employee information.
◦ Decrease in Employee Relations issues and legal costs year
over year.
◦ Strong staffing numbers. Focus on retaining talent moving
forward.
◦ Emphasis on auditing HR files to ensure compliance and
better use of results next fiscal year

 Each must assess each degree areas,
Bachelors, Associates, and Diploma/
Certificate
 Today we will highlight one area from
program.
 Each must assess 7 areas, some of these
areas overlap non-program area reports.

 1.Basic Skills – Graduates will demonstrate knowledge
of apparel production processes from concept
development through finished product.

 2.Process Skills - Graduates will demonstrate
advanced skills in construction, draping, fitting, and
patternmaking as well as in specialty textile design,
and product and concept development

 3.Technology and Production – Graduates will
demonstrate the ability to use industry software to
create, grade, and mark patterns; use software to
develop specification sheets; and develop advanced
knowledge of technical sketching and computer
design.

 4.Communication – Graduates can explain
connections between world events and design, color,
and forecasting trends in the apparel industry, and
apply current events to business trends

 5.Professional Practice - Graduates will demonstrate
ability to apply professional standards and business
concepts related to apparel design, including the
ability to work collaboratively and to present and
articulate concepts.

 6. Operational Goals
 Persistence: Maintain student persistence at or above
the Ai system desired targets.


 Graduate Placement: Maintain graduate placement at
or above Ai target levels.


 Student Satisfaction Level: Maintain student,
employer, and graduate satisfaction at appropriate
levels.

Learning Outcomes/Dept. Objectives Assessed:
• Sanitation
• Moral and Ethical codes
• Cost control
• Cooking
• Standardization of recipes
• Human Resource Principles
• Ability to create a Business Plan
• Student Persistence Goals

Learning Outcome 1 AI Pittsburgh will continue its efforts to assist students to complete the
TAPS program and to successfully pass the Serve Safe test

Learning Outcome 2 AI Pittsburgh does not collect information from internship employers.
Information has been gathered from the individual interns’ final grades which are prepared
by their supervising employer.

Learning Outcome 2a Faculty members who teach American Regional will separate the
Culinarian’s Code assignment for the collection of statistical information

Learning Outcome 3a AI Pittsburgh portfolio instructor has added the final project from
CUL305, which is a complete accounting business plan to the portfolio rubric for graduation.

Learning Outcome 3b AI Pittsburgh portfolio instructor has added the requirement of 5
costing sheets and a menu analysis to the portfolio rubric for graduation.

Learning Objective 7 Information for this fiscal year has not been available upon the date
that this report was due.

The Faculty and Staff of the Culinary Arts Department of AI Pittsburgh will continue to assess
and revise all materials for delivery to the students in this program by offering a variety of
schedule options, assistance in learning style, and by offering a variety of teaching/learning
options through technology and e-companion.

15 Full Time Faculty from campus and
online reviewed and score 80 randomly
selected and unmarked final projects (10
/online – 10/campus for a total of 20
per quarter for 4 quarters) from PH135
Photographic Design for FY’12 Summer,
Fall, Winter and Spring quarters.
The Faculty used a Program
Chair/Online Program Director
Photography

developed rubric in relationship to the
first column of the IE annual report -
Learning Outcomes .

The same Faculty assessed the campus
Bachelors of Science (BS) graduating
portfolios using a higher level Program
Chair/Online Program Director
developed rubric in relationship to the
first column of the IE annual report -
Learning Outcomes.

Note that during this assessment period
there were no online graduates to
assess. FY’13 will have online graduates
in the BS program.

Using assessment data collected through a
Google document (used like Survey Monkey) and
a portfolio share point, the Photography program
came to the following points:
• Technical Skills – we continue to be a solid
program
• Critical Thinking is an area for improvement
• The Online faculty see application of grading
criteria and rubrics in core courses across the
Photography

program lacking consistency as well as
inconsistency of grading from Adjunct faculty
• Campus faculty question assessing a 2nd
quarter course as they believe it is too early in
the students educational career
• Online PH135 portfolios scored slightly better
than campus portfolios in certain categories
of assessment

Assessment initiatives for FY’13:
• BS portfolios for campus and online
• Not assessed for FY’12, AS portfolios for
campus and online
• Though there are concerns, PH135
Photographic Design assessment will continue
• Assess business/marketing skills in the BS
and AS portfolio packages
• Employ additional initiatives developed as a

 What did I assess?

 -*Gateway 3-6-9(3rd quarter assessment)

 -16 fashion design students were assessed. How will students use the
design process to analyze and adopt construction methodology to industry
standards?

 -Evaluation: An exam, part written and part demonstration was given to
student’s week 11 of the quarter. The written portion of the exam included
information from the following courses: FD100-Clothing Construction,
FRM110-Introduction to Retailing and FD110-Intermediate Construction.
following courses: FD100-Clothing Construction, FRM110-Introduction to
Retailing. The demonstration portion of the exam was based on the following
courses; FD100-Clothing Construction and FD110-Intermediate
Construction.

 -The rubric was developed by the department chair and two faculty.

 Summary of my assessment

 -The fashion design student’s final assessment results were surprising. For the
final assessment, less than half of the students scored above a C+ on the rubric
scale. The majority of the students were at a solid C range. This was not good
news, but very useful information for me to have moving forward in making
curriculum and department revisions.

 What did I learn from this process?

 -I learned two important things;

 Students needed more time to practice in the sewing lab to improve their sewing
skills. I quickly added another tutor and extended lab hours.

 Students needed to experience learning from more than one faculty member.
Two additional faculty members were added to staff.

To provide an education in design,
media arts and culinary that prepares
students for individual and professional
success.

The Art Institute of Pittsburgh Departmental MSCHE
ASSESSMENT CHALLENGE 2012

CIDA COURSE
STANDARD COMPETENCY


COLLECTED STUDENT
ASSIGNMENT
WORK


ASSESSMENT
media arts and culinary that
prepares students for individual and
professional success.

Value-added Assessment Results = Measuring student knowledge and growth

PRE-TESTING POST-TESTING

CONCRETE DATA
This collected data is easily scored and analyzed
using statistical procedures.

kspewock

ASSESSMENT

Regularly
assessed
CONCRETE academic criteria
that leads to
DATA constant
academic
improvement
and change

Next Steps: Incorporate similar pre/post testing methods that address course
competencies as broad learning objectives that support the mission of the
foundations department enabling students to advance and succeed in their
respective major area.
kspewock


FORMAL
Peer Critique ASSESSMENT:

•USE OF FORMAL
RUBRICS

•INCREASED
STUDENT
IMPROVEMENT
Faculty Critique
Professional Jurors •PROJECT
PARAMETERS

•INDUSTRY
EXPECTATIONS

ASSESSMENT

FORMAL CONSTANT
ASSESSEMENT ACADEMIC
IMPROVEMENT AND
POSITIVE CHANGE

Next Steps: Critical review of the academic curriculum in both industrial design
and entertainment design will lead to a revised curriculum that meets current
industry expectations where program objectives are realized through course
competencies and assignment objectives.

 Spring 2012 the Graphic Design Portfolio class review
process and final portfolio requirements were revised
and updated.

 During the graduates final portfolio class:
 Week 3 students had their work evaluated by GD
faculty
for overall effective components of their books.
 Week 5 professional designers were brought in to
work with
each student and reviewed all the assessment
outcomes.
 Final portfolio reviews are now in a presentation
format
to a panel of instructors.
Mission: To provide an education in
design,
media arts and culinary that prepares
students
for individual and professional success.
In-Betweener

 Bachelors: Technical expertise - Graduates will demonstrate and
apply competencies in industry-specific computer software
programs within the
context of producing concrete projects. This includes technical
aspects of
prepress, output, and quality reproduction as well as web design.

 Associates & Diploma: Technical expertise is demonstrated through
proficiency
of software, manual skills, and craftsmanship for preparation and
presentation
of work.

 Bachelors Review:
 3rd quarter - seventy percent of the students will score 70 or higher
out of 100
 7th quarter - the overall average score on appropriate components
will be no less than 3.5 on a scale of 1.0-5.0

 Associates and Diploma Review:
out of 100

Fact: The Design Studio Students swept
The Pittsburgh Aids Task Force poster
design competition! http://www.patf.org/

 Bachelors: Design - graduates will analyze and incorporate
aesthetics and formal concepts of layout and design. This includes
spatial relationships; communication legibility and effectiveness;
interrelationships among imagery and text; balance; typography;
and color theory. This includes technical aspects of prepress,
output, and quality reproduction as well as web design.

 Associates & Diploma: Visual and verbal communication is
demonstrated through inexpressive visual voice, control of form,
and relevance to the needs of the client and values of the target
audience

out of 100.
will be no less than 3.5 on a scale of 1.0-5.0

out of 100

Fact: WORD PLAY is a ―while you wait‖ game
developed by The Fred Rogers Company for
parents and children to play together while
waiting for the bus with help from the Art
Institute Design Studio students.

 Bachelors: Conceptual - Graduates will demonstrate design concepts
and relate these to historical and contemporary trends and social
context by producing successful visual solutions to assigned
problems.

 Associates & Diploma: Design professionalism is displayed in the
student’s work through demonstration of insightful background
research, and credibility. The work should be error-free with a
meaningful hierarchy of visual elements.

 7th quarter - seventy percent of the students will score 70 or higher
out of 100.
will be no less than 3.5 on a scale of 1-5.0

 4th quarter - seventy percent of the students will score 70 or higher
out of 100

Fact: A GD grad, Mike Legacy
Is Art Director at a top Pittsburgh Agency,
Brunner. mlegacy@brunnerworks.com

 Bachelors: Visual Communication - graduates will model the
interdependence of content and visual expression and be able to
evaluate and critique their ideas. Graduates must be able to verbally
articulate the vision behind their creative work and explain and
promote their solutions.

 Associates & Diploma: Theory and Concept - student’s work
demonstrates striking originality in their work. Evidence of
knowledge of design trends is thorough and the solving of the
design problem is insightful.


out of 100

Fact: Art Institute of Pittsburgh Graphic Design
student Tyler Kozar entered a contest to win 1
million Pop-Tarts—and won.
http://www.pittsburghmagazine.com/Best-of-
the-Burgh-Blogs/Pitt-Girl/April-2011/Pop-
Tart-Pop-Art/

 Bachelors: Professional Presentation - graduates will demonstrate
professional presentation; articulation of knowledge of graphic
design and visual problem solving; and mastery of industry
standards, professional practices and ethics.

 Associates & Diploma: Form - student’s work demonstrates applied
mastery of application of the formal aspects of graphic design,
image making, and typography to the creation of design objects
(form) and page compositions. Graduates will have a basic
marketing package and basic portfolio of work, demonstrating the
competencies of the curriculum.

will
be no less than 3.5 on a scale of 1-5.0


Fact: A Solar Thermal Domestic Water
Panel mural on the playground side of
Pittsburgh Phillips School was designed by
Art Institute of Pittsburgh Graphic Design
students.

 Summer 2012 the Web Portfolio class review process
and final portfolio requirements were revised and
updated.

 During the graduates final portfolio class:
 Week 3 students will have their work evaluated by
faculty
for overall effective components of their books.
 Week 5 professional designers are brought in to work
with
each student and reviewed all the assessment
outcomes.
 Final portfolio reviews are now in a presentation
format
to a panel of instructors.
Mission: The Web Design & Interactive Media
program prepares students to work effectively
both individually and in a team creating
aesthetically compelling and effective content to
be presented and interacted with on the Web and
In-Betweener multiple interactive media platforms.

 What did you assess?
 The means of assessment by the MAA and GAD
faculty has been data collected from program
student reviews that occur during the 4th, 6th,
9th and 11th quarters. This year, these reviews
were used to assess to what degree the
programs meet the expectations as defined by
6 Learning Outcome ―statements‖ in the
Institutional Effectiveness report.
 Campus and online both provided data from
summer, fall, winter and spring quarters during
FY12.

A summary of your assessment
The general guidelines listed under ―criteria for success‖ (column 3)
were that 80% of the students whose skills were being reviewed
would score at least 70% out of 100%, which translates to a grade of
3.5 or better (5 being the highest).
The measured rate of success ranged between 36.6% at its lowest
and 94.4% at its highest across all six primary Learning Outcome
statements.
GAD based their assessment on the final grades earned as data in
classes where an assessment review was attached.
MAA based their assessment on data collected through independent
surveys for classes with assessment reviews attached.
What you learned from the process
Based on the results from the reviews conducted, both departments
conclude that the student numbers excelling in program disciplines
vary, depending on the specific skills being assessed.

 Academic Quality.

 Observing specific courses with established
data collection parameters on an average 5pt
scale measurement.

 DFV214 Intermediate Editing, DFV332 Multi-
Camera Production, DFV430 Senior Portfolio
and Defense, DFV420 Senior Project Post-
Production –Final Project, DFV311
Scriptwriting II, DFV410 Senior Project
Production, DFV321 Sound Design.

 Variables Observed:
 Student work in the form of class projects and a final project were analyzed in these areas:
 Conceptualization, Plan and Execution
 Leadership and Collaborative Responsibilities
 Written Communication
 Creative Writing
 Camera
 Lighting
 Technical Sound
 Creative Sound
 Technical Editing
 Creative Editing
 Editing & Post-Production
 Presentation
 Knowledge of Field

 Other Variables observed:
 PAC Committee Feed-back
 Program Student Persistence
 Career Services Employment Statistics: Graduate Rates
 Noel-Levitz Survey
 IDEA Survey of Key Courses in Program
 Employer Survey of Graduates Employed
 Graduate Survey


 The importance of keeping our academic processes in close observation in order
to act accordingly towards improvement and/or further implementation in
connection with our academic budget planning.

 In the DFV214 course we are not teaching editing software only, we are expecting
a holistic connection of all the previous courses taken by our students at this
academic level to be reflected in their final intermediate editing project.
 We need to bring more faculty and pertinent guests to this editing course
critique. We have not been very successful at programming this review due to 9th
week critique activity being parallel to senior projects critiques.
 Although the average results are positive students in DFV410 Senior Project
Production will benefit from a preliminary course with emphasis on pre-
production and conceptualization. Such course will exists in a new DFVP flowchart
where it is called Senior Project Pre-Production.
 The sound quality of our student work needs to be emphasized in DFV214
Intermediate Editing. The final result of the class in the form of an editing video
should reflect all audiovisual skills learned until then. Sometimes students focus
mainly on the editing process overseeing other production values.

 We will continue our efforts to improve our student persistence rates through
individual dedication from the faculty; relevant courses and education; and
providing significant outlets for our student’s creativity and sense of identity. One
example of these kinds of initiatives is our quarterly screenings recognizing the
best of our DFVP students work at the Harris Theater.
 We will work closely with Career Services to improve our professional’s network
and alumni data base in the field.
 One of the constant variables on the Noel-Levitz survey is the low rated
satisfaction average on the question: There is a commitment to academic
excellence on this campus. This is a perception that we must change through a
major commitment at all levels of this academic institution. More visible efforts of
academic excellent need to be provided through rigor, visiting lecturers, open
critiques and always placing our academic concerns in the forefront of our
leadership.
 Being 100% at or above the Idea and Database average is encouraging and
reaffirms our academic efforts
 At this point our employer survey needs to be more robust to be considered
relevant. We must work closely with Career Services to improve the amount of
data.
 At this point our graduate survey needs to be more robust to be considered
relevant. We must work closely with Career Services to improve the amount of
data.

Maria Boada
Katie Talerico
David Kokales

Behavioral and
Arts and Humanities
Social Sciences

Math and Natural
Oral Communication
Sciences

Develop
students
professionally
and
Written Information Fluency
Communication personally:
well rounded
academic
experience

Use of Results and
revision of future plans

Faculty Review and •Data
• Means of Planning collection
assessment •Recommendations
takes place
•Faculty in each are implemented in
and criteria throughout
discipline curriculum and
for success review and
the
pedagogy
provided by academic
plan •Changes to
Ai System year
implementatio assessment plan are
Assessment n of •Analysis of
Assessment Model recommended for
assessment data takes
Office
provided by Ai System
next cycle
instrument(s) place during
Data Collection and

in their courses in-service
Analysis

faculty
•Data collection
meeting
planned

Courses Assessed: Means of Assessment:
Written
Graduates
Communication
will ENG095 (Transitional a. Writing Portfolio is
demonstrate effective English), ENG1010 (English evaluated according to
Composition I) and ENG1020
written (English Composition II)
a set five point rubric

communication by
Means of Assessment:
providing their
b. ETS (Educational
audience sufficient Testing Service)
information on a topic Proficiency Profile
that is organized and Success Criteria:

employs the generally a. 80% of students will
score a 3 or higher
accepted conventions
b. 75% of graduates will
of English Language
score at or above the
national ETS average on
the ETS writing component

Assessment Cycle: ENG courses
Quarter of Artifact (s) Additional
Data Collected Data
Collection Collected
ENG095 FA2011 25% random FY2012
sample of all cumulative
Writing Portfolios passing rates.
Passing rates
in ENG1010
ENG1010 WI12 30% random Qualitative
sample of all focus on
writing reflective
portfolios. essay
ETS Proficiency
Profile Data
ENG1020 Collected NA NA
during FY2011.
Next data
collection:

ENG095: Transitional English

Criteria: 80% of
96% of students achieved an
students will earn
a 3 or higher overall score of 3.0 or better
(5 point rubric) – Far exceeded set criteria
74% of all student who
Criteria: 70% of took ENG095 in SU11,
students will
earn a C or
FA11, WI12 and SP12
better earned a C or better –
Exceeded set criteria

Additional Data will be presented in the
Transitional Curriculum Assessment
report

ENG095 Results: results by Competencies
Competency Average score
Writing process—prewriting, drafting, revising,
proofreading and editing 3.6
Structure 3.4
Use of detail 3.8
Use of standard rules of English grammar, mechanics 3.3
and sentence construction
Understanding of reading and writing as interconnected 3.4
processes
Use of critical thinking through argument or narrative 3.4

•Results exceeded set criteria
•Students did particularly well in the competency related to
understanding the writing process.
•Future assessment will focus on reading and writing as
interconnected processes.

ENG1010: Composition and Language

Criteria: 80% of students 70% of students earned a 3
or better – did not meet
will earn a 3 or higher (5 criteria (see analysis)
point rubric)

Writing Level 1: 55%
Proficient (National
Average is 69%)
Criteria: 75% of Writing Level 2: 15%
graduates will score at Proficient (National
Average is 41 %)
or above national Writing Level 3: 6 %
average in ETS Proficient (National
Average is 8%)
Proficiency Profile
Average Scores were
writing component below the ETS
National Averages for
College Seniors

ENG1010: Breakdown of Average Scores by Competency
Writing process—prewriting, drafting, revising,
proofreading and editing 3.0
Structure 3.4
Use of detail 3.0
Use of standard rules of English grammar,
mechanics and sentence construction 3.5
Understanding of reading and writing as
interconnected processes 3.1
Use of critical thinking through argument or
narrative 3.4

 While the 80% criteria was not met, the total
Analysis andscoresResults
average Use of was 3.25.= and no average for
each competency was below 3.0. Further analysis
showed two sections of ENG1010 with a high
percentage of left-outliers.
 The discussions centered on emphasizing data-
generating exercises and a higher focus on
intertextuality.
 The ETS data showed writing proficiency levels
below the national average. We recommend
additional data collection and comparison with
institutions that have programs similar to AiP.

Graduates will
Course evaluated: a. Capstone assignment
demonstrate effective in COM3010: Analytical
oral communication by COM3010 oral presentation with
providing their Advanced visual aids.
Communication
audience sufficient
information on a topic Means of Assessment:
that is organized and b. Graduate Surveys
that employed the
generally accepted Success Criteria:
conventions of English a. 80% of students will
score a 3 or higher
Language.
b. 90% of students agree
that they have good
communication skills

Assessment Cycle: Communications

Quarter of Data Artifact (s) Collected Additional Data
Collection Collected

COM3010 FA2011, A random 30% sample Results from
Advanced SP2012 of all final oral graduate surveys
Communications presentations, related to
centered around the communication
SWOT analysis) were skills
evaluated by a in-
class panel of faculty

COM3010 Advanced Communication - Results

Criteria: 80% of 90% of students
students will earn a scored a 3 or higher
score of 3 or higher in in the oral
the oral presentation presentation – Far
(5 point rubric) exceeded set criteria

Criteria: 90% of
graduate students
Data no yet received
agree/strongly agree
from corporate office
that they have good
communication skills

Average Scores: Overall and breakdown by Competency
Overall Average Average Average Average
% of 3 Score: Score: Score: Score:
and Structur Non- Visual Support
above e verbal Aids
Skills
FA11 90% FA11: FA11: SP12: FA11:
and 3.19 3.54 2.59 3.47
SP122
SP12:
2.35

Analysis and Use of Results
 The overall results met the stated criteria but some of the competencies
highlighted areas of improvement.
 Scores for the ―Support‖ category were lower in SP12 than in the FA11
cycle. Oral communication instructors recommend a higher emphasis in
research-based information and will collaborate with Writers’ Center to
incorporate MLA workshops and other information literacy initiatives.
 Average scores for the ―Visual Aids‖ were also
lower than expected. Added emphasis will be
placed on visual aids requirements for final oral
presentations.
 Support and Visual Aids will be evaluated in the
next assessment cycle.

Graduates will Course Assessed:
a. Comparative
demonstrate the ART3010 Analysis essay
ability to interpret American Art
holistically scored with
a five-point rubric
and analyze artistic History
work, historical Means of Assessment:
events, or b. ETS (Educational
philosophical Testing Service)
Proficiency Profile
thought and describe
the relevance of the Success Criteria:

work to themselves, a. 70% of students will
score a 3 or higher
society and cultures.
b. 75% of graduates will
score at or above the
national ETS average
subscore for humanities

 The results present a sample of the holistic
assessment picture in our general education
department
 Additional results for all outcomes can be found in
the FY12 Assessment Report
 Assessment also takes place in the online modality
and results are compared to the outcomes from
campus students
 Results from assessments continuously feed into
curriculum modifications and improvements in
pedagogy

KMT Oct 31 2012 Assessment

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (8)

Similar to KMT Oct 31 2012 Assessment

Similar to KMT Oct 31 2012 Assessment (20)

KMT Oct 31 2012 Assessment

Editor's Notes