Needs Assessments and Program Planning Lynda Milne 2009 CTL Leadership Development Program

Basic Questions of Needs Assessment

Needs Assessment Methods Method Kinds of Data Good for ? Features Specific Uses Surveys Opinions, Preferences Self-report of behavior Easily analyzed as quantitative information Aggregate information Big picture Comparison of sub-groups (if sufficient #s) Cheap Challenging to design well Difficult to get high enough response rate to ensure validity Identify: Interest in topics Prioritization Levels of satisfaction Stakeholder information Interviews Opinions, Preferences Informed, thoughtful Qualitative Expert input Details, rationale Triangulation Labor, time intensive Permit interactive probing Identify: Explanatory details, esp. post-survey Vested interests of stakeholders

Opinions,

Preferences

Self-report of behavior

Easily analyzed as quantitative information

Aggregate information

Big picture

Comparison of sub-groups (if sufficient #s)

Cheap

Challenging to design well

Difficult to get high enough response rate to ensure validity

Identify:

Interest in topics

Prioritization

Levels of satisfaction

Stakeholder information

Opinions,

Preferences

Informed, thoughtful

Qualitative

Expert input

Details, rationale

Triangulation

Labor, time intensive

Permit interactive probing

Identify:

Explanatory details, esp. post-survey

Vested interests of stakeholders

Needs Assessment Methods Method Kinds of Data Good for ? Features Specific Uses Focus Groups Opinions Preferences Qualitative Test concept Brainstorming Pre-design or pre-survey Labor, time intensive Permit interactive probing Open-ended Identify: Key ideas Response to new proposals New solutions Institutional data Performance indicators Background information, key documents Quantitative and qualitative Evidence of trends, patterns Explanation of context Comparison of sub-groups Availability dependent on resources Interpretation requires follow-up (interviews, e.g.) Identify: Actual aggregate data both before and after intervention.

Opinions

Preferences

Qualitative

Test concept

Brainstorming

Pre-design or pre-survey

Labor, time intensive

Permit interactive probing

Open-ended

Identify:

Key ideas

Response to new proposals

New solutions

Performance indicators

Background information, key documents

Quantitative and qualitative

Evidence of trends, patterns

Explanation of context

Comparison of sub-groups

Availability dependent on resources

Interpretation requires follow-up (interviews, e.g.)

Identify:

Actual aggregate data both before and after intervention.

Which source? Mixed sources, usually Triangulation : multiple forms of overlapping, diverse pieces of evidence and perspectives. By using multiple forms of evidence and perspectives, a truer portrait of the student can be developed (Wiggins, 1998). While the same biases in evidence collection still come into play, because more types of evidence are being used to form one's opinion about the student, there are more cross checks on the accuracy of the decision. Multiple data sources (survey, observation, focus groups, interview Ensure non-overlapping areas of weakness and strength

Triangulation : multiple forms of overlapping, diverse pieces of evidence and perspectives.

By using multiple forms of evidence and perspectives, a truer portrait of the student can be developed (Wiggins, 1998). While the same biases in evidence collection still come into play, because more types of evidence are being used to form one's opinion about the student, there are more cross checks on the accuracy of the decision.

Multiple data sources (survey, observation, focus groups, interview

Ensure non-overlapping areas of weakness and strength

NOAA’s 12 Steps for Needs Assessment http://www.csc.noaa.gov/needs/ Let’s try a few steps.

Let’s try a few steps.

Examples and Cases CTL Biennial Faculty Development Surveys 2003 triangulated with survey of CAOs, survey of presidents, interviews with deans, focus groups of CTL Campus Leaders. Yours?

CTL Biennial Faculty Development Surveys

2003 triangulated with survey of CAOs, survey of presidents, interviews with deans, focus groups of CTL Campus Leaders.

Yours?

Reporting…and Planning a Program Reporting data so that stakeholders and decision-makers feel they have enough information is key. What changes in behavior, attitudes and performance are expected? [How] will we get them? What are the expected economic costs and benefits of any projected solutions?

Reporting data so that stakeholders and decision-makers feel they have enough information is key.

What changes in behavior, attitudes and performance are expected?

[How] will we get them?

What are the expected economic costs and benefits of any projected solutions?

Go forth and inquire! Needs evaluation: University of Texas http://www.utexas.edu/academic/diia/assessment/iar/programs/plan/types/needs.php Needs Assessment: George Mason University http://classweb.gmu.edu/ndabbagh/Resources/Resources2/Needs.htm ScienceDirect - Evaluation and Program Planning : Problems in needs assessment data: Discrepancy analysis http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V7V-4NP9839-2&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=939124035&_rerunOrigin=google&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=db332240579260baf47 Who Needs What Training? A Look Askance at Training Needs Assessment http://home.att.net/~nickols/who_needs.htm Needs Assessment http://alumnus.caltech.edu/~rouda/T2_NA.html Instructional Design Models http://carbon.cudenver.edu/~mryder/itc_data/idmodels.html Needs Assessment Surveys: Do They Predict Attendance at Continuing Education Workshops? http://www.joe.org/joe/2002august/a4.php This set of links (and more?!) at http://www.diigo.com/list/lmilne/needs-assessment

Needs evaluation: University of Texas

http://www.utexas.edu/academic/diia/assessment/iar/programs/plan/types/needs.php

Needs Assessment: George Mason University

http://classweb.gmu.edu/ndabbagh/Resources/Resources2/Needs.htm

ScienceDirect - Evaluation and Program Planning : Problems in needs assessment data: Discrepancy analysis

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V7V-4NP9839-2&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=939124035&_rerunOrigin=google&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=db332240579260baf47

Who Needs What Training? A Look Askance at Training Needs Assessment

http://home.att.net/~nickols/who_needs.htm

Needs Assessment

http://alumnus.caltech.edu/~rouda/T2_NA.html

Instructional Design Models

http://carbon.cudenver.edu/~mryder/itc_data/idmodels.html

Needs Assessment Surveys: Do They Predict Attendance at Continuing Education Workshops?

http://www.joe.org/joe/2002august/a4.php

This set of links (and more?!) at

http://www.diigo.com/list/lmilne/needs-assessment