SlideShare a Scribd company logo
1 of 60
Download to read offline
Personal Finance
Outcome Assessments:
Methods and Metrics for
Face-to-Face Classes and
Social Media Outreach
Barbara O’Neill, Ph.D., CFP®, CRPC®
Distinguished Professor, Rutgers University
oneill@aesop.rutgers.edu
Are You Evaluating
Your Financial
Education Programs?
If so, how?
The Million Dollar “So
What?” Question…
At the end of the day…did your financial
education program make a difference?
How do you know?
The Current State of Financial
Education Program Evaluation
• Current evaluation efforts are still far from
satisfactory
• General lack of evaluation capacity
• Evaluation is often treated as an after-thought
• Outcomes (e.g., measures of changed behavior) are
often confused with program outputs (e.g., number
of participants)
• What gets measured gets funded
• With evaluation results, you can…
– assess impact of programs on learners
– see if you accomplished what you planned
– know if a program was “worth it”
– celebrate success and learn from failure
– make informed decisions to improve, hold, or fold programs
– promote your program and win public support
We are in an “Accountability Era”
Consider a Third Party
Program Evaluator
• Some funders require this (e.g., FINRA Investor
Education Foundation and IMLS)
• No potential for conflict of interest by one entity
performing both education and evaluation role
• Takes evaluation “monkey” off busy educators’ back
• Build into grant funding requests
• See http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
• Begins with the end in mind
• Explains what a program is and what it will accomplish
• Shows relationships between inputs, outputs, and outcomes
Introducing the Logic Model
Building a Strong
Financial Education Program
INPUTS OUTPUTS OUTCOMES
Program
investments
Activities Participation Short Medium
What
We
Invest
What
We Do
Who We
Reach
What Results
SO WHAT??
What is the VALUE?
Long-
term
Staff
Money
Time
Volunteers
Partners
Equipment/Technology
Policies
Research
INPUTS
OUTPUTS
What We Do Who We Reach
ACTIVITIES
Assess needs and assets
Design curriculum
Educate students
Conduct workshops
Facilitate learning groups
Sponsor conferences
Work with the media
Partner – collaborate
PARTICIPATION
Participants
Clients
Customers
Users
Groups
Reactions - Satisfaction
OUTCOMES
What Results for Individuals, Organizations, Communities..…
SHORT
Learning
Awareness
Knowledge
Attitudes
Skills
Opinion
Aspirations
Motivation
MEDIUM
Action
Behavior
Practice
Decisions
Policies
Social action
LONG-TERM
Conditions
Human
Economic
Civic
Environment
OUTCOMES
Commonly Measured Items
That Are Not Outcomes
• Participant satisfaction
• Number of people taught
• Units of education completed
• Number of events held
• Time and money spent
• Level of effort
Activity #1
What are some outcomes for
your financial education
programs?
Impact Evaluation Data
Collection Methods
• Surveys (Paper or Online)
– Post-evaluation only (short programs)
– Pre- and post-evaluation
– Follow-up (e.g., 3 months later)
• Focus groups
• Interviews
• Observations
• Tests of knowledge/ability
• RARE: Control groups and longitudinal studies
Typical Survey Questions
• General reactions to the program
• Changes in knowledge
• Changes in motivation, confidence, and abilities
• Intended changes in behavior
• Actual changes in behavior
• Future programming needs and preferences
• Demographics of participants
• Qualitative/open-ended responses
Post-Then-Pre
(Retrospective) Evaluation
Method
• Compares knowledge and attitudes before and
after a financial education intervention
• Administered once at the end of the intervention
• Helps avoid biases due to people thinking they
know more than they know
Case Example:
Humpty Dumpty
“Humpty Dumpty sat on a wall,
Humpty Dumpty had a great fall,
All the king’s horses and all the king’s men
Couldn’t put Humpty together again”
So Who/What Was
Humpty Dumpty?
Probably
NOT what
you think!
Post-Then-Pre (Retrospective)
Evaluation Method
• Why? Because people don’t know what they don’t know!
• Who knew what Humpty Dumpty really was?
• Five point rating scale
– 1 = Strongly Disagree
– 5 = Strongly Agree
• Post: After listening to Barb, I know the history of the
Humpty Dumpty nursery rhyme
• Pre: Before listening to Barb, I knew the history of the
Humpty Dumpty nursery rhyme
Post-Then-Pre Evaluation
• Also known as a “Retrospective” evaluation
• Helps identify changes in knowledge, attitudes, and
behavior
Another Post-Then-Pre
Evaluation Example
Big Advantage: Avoids the problem of learners under-
estimating what they don’t know before a program
Critical Incident Technique
• Qualitative evaluation research method where
program participants tell personal stories
• Often used with “Train the Trainer” programs for
professionals (e.g., teachers, librarians)
• Incidents are categorized and deemed successful
(positive results) or unsuccessful (negative results)
http://www.joe.org/joe/2013june/tt2.php (Journal of Extension article)
More About the Critical
Incident Technique
http://www.adb.org/documents/information/knowledge-
solutions/the-critical-incident-technique.pdf
• Ask subjects to describe- through interviews-incidents that
they handled well or poorly (need not be spectacular events)
– Example: How did library staff handle patrons’ personal finance
questions?
• Provides rich personal perspectives
• Pre-intervention and Post-intervention comparison of
number and type of incidents
Critical Incident Technique
Example: NYPL Grant
Program Title: Money Matters
CIT Questions in NYPL
Financial Education
Evaluation
• Think about your experiences helping patrons with personal finance
questions. Remember a time when you had a successful experience
helping someone with these types of questions. Please write down what
happened.
• What made this a successful, positive experience?
• Think about your experiences helping patrons with personal finance
questions. Remember a time when you had an unsuccessful experience
helping someone with these types of questions. Please write down what
happened.
• What made this an unsuccessful or challenging experience?
CIT Summary in Final NYPL
Project Evaluation Report
“The change in the percentages of the types of incidents is indicative of
change in attitudes and abilities. Also, a new category appeared in the
successful incidents, Increased Knowledge/Confidence/Satisfaction,
which was not present in the Pre-training Survey. Furthermore, for the
unsuccessful critical incidents, the category Lack of Training/
Knowledge which was the most frequently seen theme for the Pre-
training Survey was not found at all in the Post-training Survey.
These findings resonate with quantitative findings and provide confirming
evidence that the Money Matters training has been successful in
improving participants’ knowledge of personal finance and their
ability to be successful in handling patron inquiries. Additionally, staff
members provided eloquent testament in their qualitative responses that
their skills have been enhanced and their attitudes and behaviors became
more positive for ably handling personal finance queries” (Radford, 2013).
Social Media
Evaluation
Methods
America Saves Week (ASW)
Social Media Campaign
• Conducted by eXtension (online program of the
Cooperative Extension System), 2011-2015
• Prepared “cut and paste” tweets and FB messages on
saving, debt/expense reduction, America Saves, ASW
• Project cooperators from eXtension, America Saves
campaigns, military, others
Triangulation (Multiple
Methods) Evaluation
Approach
• Unique Twitter hashtag: #eXasw
• Follow-up follower/friend survey
• Follow-up project participant survey
• bit.ly analytics to determine number of clicks on unique
embedded links
• Pre- and post-ASW Twitter influence metrics (Klout score)
Quantitative Impact Data:
2011-2014
• #eXasw tweets recorded (1,091- 1,503)
• Facebook & Twitter message clicks (602- 2,814)
• Followers/friends reached (3,056- 8,163)
• Followers/friends reached (average): (192-254)
Follower/Friend Survey
Results: 2012 Example
• Response: 50% “very helpful” and 50% “helpful”
• 54% visited ASW links; 38% planned to
• 8% joined AS; 38% planned to; 16% already joined AS;
38% had not joined AS
• Mostly positive comments about SM messages
• One negative comment about seeing repeated messages
and the need for more action steps
Professional Collaborator
Survey Results: 2012 Example
• 22/22 Klout scores increased
– Average from 20.3 to 29.3 (vs. 11.22 to 19.68 in 2011)
• Positive comments (e.g., wanted more “cut and
paste” messages, improved SM skills, ease of use,
continue in 2013)
• State Governor and Senator started following one
participant
• Reported likes, retweets, and direct messages
Klout Score Progression
• Klout is a measure of a person’s “influence” on twitter
• Based on an algorithm with factors such as number of
followers and number of retweets
• Go to www.klout.com and log in with Twitter or Facebook
– 2011: 11.22 to 19.68
– 2012: 20.3 to 29.3
– 2013: 32.6 to 39.6
– 2014: 29.76 to 38.84
Project Take-Aways
• Social media messages are great for increasing
awareness but do not appear to prompt much action
• Project participants like having “cut and paste” social
media messages
• Twitter influence (Klout score) of project participants
became noticeably higher over time
• Social media is an effective outreach tool
Twitter Hashtags
• A key component of Twitter use is the hashtag (#)
symbol followed by a series of unique characters
• The hashtag serves like a “magnet” to link together the
tweets of Twitter users who are interested in the same
topic (e.g., credit) or who are linked together in some
way (e.g., professional conference attendees)
• By using a hashtag, users can effectively hold a
“conversation” with one another about a topic of
interest
Twitter Chats
• Online users gather at a specified time (e.g., every
Friday at 3 pm ET) to “discuss” (read: tweet about)
certain topics, questions, or issues.
• Synchronous learning environment where participants
interact in real time using their Twitter user names
(a.k.a.,”handles”).
• As with tweets at conferences, the “glue” that holds
Twitter chats together is the designated hashtag
• Twitter chat hashtag creates a running stream of
tweets so that everyone can follow the “conversation.”
More About Twitter Chats
• Chat hashtags are often derived from a chat sponsor’s
name (example: #wbchat for WiseBread); industry
(example: #creditchat); or program title (#SSHWchat
for the Small Steps to Health and Wealth™ program)
• A Twitter chat hashtag creates a running stream of
tweets so that everyone can follow the “conversation”
• The formatting convention used to follow Twitter chat
threads is Q1 for Question 1 and A1 for participant
responses to that question
Twitter Chat
Evaluation Methods
• Online survey (e.g., Qualtrics) link embedded into final tweets
(with prizes as an incentive to complete)
Dr. Barbara O'Neill @moneytalk1Apr 29
Please let us know if U found #SSHWchat helpful & take this brief survey:
https://rutgers.qualtrics.com/SE/?SID=SV_9X0V77kUKhOqlWB … Will pull winners at 1:30 pm #sshwchat
• TweetReach statistics: http://tweetreach.com/ (type in chat
hashtag to pull a report)
• Follow-up contact from participants (Twitter direct messages,
e-mail, etc.)
• Traffic to Web site during and after a chat
• Other?
Sample TweetReach
Report
#eXASchat
Evaluation Results
• Four consecutive Monday evenings in February 2014 to
promote America saves and saving money
• Hashtag: #eXASchat
• 69 participants
• Estimated reach: 42,838 (estimated overall reach of tweets in report)
• Exposure impressions: 249,382 (based on participants’
followers; tells how many tweets were sent by users to their followers)
• Number of tweets sent by users with > 1,000 followers
(96th percentile of all Twitter users): 38
#SSHWchat
Evaluation Results
• Held at noon ET on 4/29/14 for Financial Literacy Month
• Topic: Strategies to improve health and finances
• 34 participants
• Estimated reach: 25,294 (estimated overall reach of tweets in report)
• Exposure impressions: 50,553
• Six-question Qualtrics survey: 29% response rate
– 100% of respondents reported learning something new
– 100% of respondents reported planned behavior changes
• Comment: “Please have one monthly, if possible”
Archive a Twitter Chat
With Storify and Track Use
https://storify.com/ Storify is an organized collection of tweets
with links, photos, etc. to “tell a story” from Twitter content
Video: https://www.youtube.com/watch?v=l9iHniFjiVc
Storify Samples:
https://storify.com/RutgersNJAES/small-steps-to-health-and-wealth
https://storify.com/RutgersSEBS/cook-douglass-community-day-2014-at-rutgers
https://storify.com/wisebread/how-are-you-saving-for-retirement
https://storify.com/JerryBuchko/mcpd2014-twitter-chat-archive-1
One Last Step:
Share Your Program
Evaluation Results
WHAT?
SO
WHAT?
NOW
WHAT?
Who Needs to Know
About the Results of
Your Financial Education
Programs? Why?
Tips for Telling Your Story
• Use simple descriptive statistics (e.g., counts,
percentages, and averages)
• Don’t use jargon
• Don’t overstate your results (e.g., causality)
• Blend quantitative and qualitative data
• Clearly describe who the results represent (i.e.,
demographic characteristics of participants)
• Be honest about your program’s strengths and
weaknesses, while highlighting the positive
Impact Statements:
Intentions
As a result of participating in this financial education
program, X% of participants reported that they…
• plan to do/use/adopt…
• are more knowledgeable about…
• are more confident in their ability to…
• are more likely than before to do/use/adopt…
• will do/use/adopt…
…a particular attitude, piece of information, or behavior.
Impact Statements:
Actions
As a result of participating in this financial education
program, X% of participants reported that they…
• are now doing…
• did…
• used…
• increased their knowledge of…
• adopted…
• changed…
… a particular attitude, piece of information, or behavior.
Social Media
Impact Reports
• Number of participants
• TweetReach report outreach numbers
• Participant survey data
• Data from other feedback methods
• Sample Evaluation Report:
http://www.slideshare.net/BarbaraONeill/ssh
w-twitter-chat-impact-statement-0414
Public Value
Approach
Private benefits accrue to program participants
Public benefits are benefits to those who did not
participate in a program
It is possible to exclude non-payers from
participating in programs—and enjoying private
benefits—but it is not possible to exclude non-payers
from receiving public benefits of programs
How Do Financial
Education Programs
Create Public Value?
 Does it narrow an information gap?
 Does it address a crucial concern about fairness?
 Does one person’s participation benefit people who do
not participate in the program?
 Does one person’s participation reduce costs for
others?
 Does the program improve upon market outcomes?
Program
participants…
have been shown to change their
behavior in specific ways…
that have been shown to lead to
specific outcomes…
that directly benefit
the participants.
that generate public
value.
Participants in a nutrition program…
have been shown to increase their
consumption of fruits and
vegetables…
which has been shown to reduce the
incidence of diabetes…
which directly improves
quality of life for
participants.
which reduces public
health costs for all
citizens.
Participants
Changes
Outcome
Private benefits Public value
Public Value Statements
• Focus on an outcome that matters to
stakeholder(s)
• Use stakeholder’s language
• Avoid jargon and empty words
• Should be short and believable
Public Value
Statement Template
When you support _______________________program,
participants will ______________________________,
(changes)
which leads to _______________________________,
(outcomes)
which will benefit other community members by
_____________________________________________.
(public value)
Activity #2
Write a public value
statement for your financial
education program?
Program Evaluation
Resources
NEFE Evaluation Toolkit
http://toolkit.nefe.org
Downloadable NEFE
Financial Education
Evaluation Manual
http://toolkit.nefe.org/Portals/0/Toolkit-Manual.pdf
Questions and
Comments?
Barbara O'Neill, Ph.D., CFP®, CRPC
Extension Specialist in Financial Resource Management
and Distinguished Professor, Rutgers University
Phone: 848-932-9126
E-mail: oneill@aesop.rutgers.edu
Internet: http://njaes.rutgers.edu/money/
Twitter: http://twitter.com/moneytalk1

More Related Content

What's hot

Word cloud slideshow
Word cloud slideshowWord cloud slideshow
Word cloud slideshowstymiegreen
 
VLStudent Slide Deck
VLStudent Slide DeckVLStudent Slide Deck
VLStudent Slide Deckmmcdowell13
 
Personalization vs. Differentiation vs. Individualization Report-v3
Personalization vs. Differentiation vs. Individualization Report-v3Personalization vs. Differentiation vs. Individualization Report-v3
Personalization vs. Differentiation vs. Individualization Report-v3Barbara Bray
 
Rethinking Student Engagement in the Digital Classroom
Rethinking Student Engagement in the Digital ClassroomRethinking Student Engagement in the Digital Classroom
Rethinking Student Engagement in the Digital ClassroomD2L Barry
 
Educational Leadership Platform
Educational Leadership PlatformEducational Leadership Platform
Educational Leadership PlatformCourtney Huff
 
Teach the teacher motivation (unit one)
Teach the teacher motivation (unit one)Teach the teacher motivation (unit one)
Teach the teacher motivation (unit one)LouisCabuhat
 
Visiblelearningcollingwood23 11-09-120818042750-phpapp02
Visiblelearningcollingwood23 11-09-120818042750-phpapp02Visiblelearningcollingwood23 11-09-120818042750-phpapp02
Visiblelearningcollingwood23 11-09-120818042750-phpapp02msweducational
 
Engaging our Online Students
Engaging our Online StudentsEngaging our Online Students
Engaging our Online StudentsJim Marteney
 
Cas summer leadership retreat ii post
Cas summer leadership retreat ii postCas summer leadership retreat ii post
Cas summer leadership retreat ii postEdAdvance
 
ISB ES Vision - The ROAD MAP
ISB ES Vision - The ROAD MAPISB ES Vision - The ROAD MAP
ISB ES Vision - The ROAD MAPJustin Medved
 
BOCES intro to Coherence Summer 15 Share
BOCES intro to Coherence Summer 15 ShareBOCES intro to Coherence Summer 15 Share
BOCES intro to Coherence Summer 15 ShareEdAdvance
 
6 c's of motivation
6 c's of motivation6 c's of motivation
6 c's of motivationwaltmendez
 
Advisory Presentation.FINAL
Advisory Presentation.FINALAdvisory Presentation.FINAL
Advisory Presentation.FINALRhetta Sabean
 
Combining Technology with the 6 "C's" of Motivation
Combining Technology with the 6 "C's" of MotivationCombining Technology with the 6 "C's" of Motivation
Combining Technology with the 6 "C's" of Motivationsmileha33
 
Formative assessment: an important teaching tool for any subject
Formative assessment: an important teaching tool for any subject Formative assessment: an important teaching tool for any subject
Formative assessment: an important teaching tool for any subject billhutchison
 
So, you want to do work-based learning at your school?
So, you want to do work-based learning at your school?So, you want to do work-based learning at your school?
So, you want to do work-based learning at your school?Andrew Steinman
 
TEACHER-STUDENT RELATIONSHIP
TEACHER-STUDENT RELATIONSHIPTEACHER-STUDENT RELATIONSHIP
TEACHER-STUDENT RELATIONSHIPTeenugirl
 
Building student motivation
Building student motivationBuilding student motivation
Building student motivationbrandymeetze
 

What's hot (20)

Word cloud slideshow
Word cloud slideshowWord cloud slideshow
Word cloud slideshow
 
VLStudent Slide Deck
VLStudent Slide DeckVLStudent Slide Deck
VLStudent Slide Deck
 
Personalization vs. Differentiation vs. Individualization Report-v3
Personalization vs. Differentiation vs. Individualization Report-v3Personalization vs. Differentiation vs. Individualization Report-v3
Personalization vs. Differentiation vs. Individualization Report-v3
 
Rethinking Student Engagement in the Digital Classroom
Rethinking Student Engagement in the Digital ClassroomRethinking Student Engagement in the Digital Classroom
Rethinking Student Engagement in the Digital Classroom
 
Educational Leadership Platform
Educational Leadership PlatformEducational Leadership Platform
Educational Leadership Platform
 
Teach the teacher motivation (unit one)
Teach the teacher motivation (unit one)Teach the teacher motivation (unit one)
Teach the teacher motivation (unit one)
 
Visiblelearningcollingwood23 11-09-120818042750-phpapp02
Visiblelearningcollingwood23 11-09-120818042750-phpapp02Visiblelearningcollingwood23 11-09-120818042750-phpapp02
Visiblelearningcollingwood23 11-09-120818042750-phpapp02
 
Engaging our Online Students
Engaging our Online StudentsEngaging our Online Students
Engaging our Online Students
 
Cas summer leadership retreat ii post
Cas summer leadership retreat ii postCas summer leadership retreat ii post
Cas summer leadership retreat ii post
 
Trauma-Informed SEL
Trauma-Informed SELTrauma-Informed SEL
Trauma-Informed SEL
 
ISB ES Vision - The ROAD MAP
ISB ES Vision - The ROAD MAPISB ES Vision - The ROAD MAP
ISB ES Vision - The ROAD MAP
 
BOCES intro to Coherence Summer 15 Share
BOCES intro to Coherence Summer 15 ShareBOCES intro to Coherence Summer 15 Share
BOCES intro to Coherence Summer 15 Share
 
6 c's of motivation
6 c's of motivation6 c's of motivation
6 c's of motivation
 
Creative teaching-and-learning-strategies
Creative teaching-and-learning-strategiesCreative teaching-and-learning-strategies
Creative teaching-and-learning-strategies
 
Advisory Presentation.FINAL
Advisory Presentation.FINALAdvisory Presentation.FINAL
Advisory Presentation.FINAL
 
Combining Technology with the 6 "C's" of Motivation
Combining Technology with the 6 "C's" of MotivationCombining Technology with the 6 "C's" of Motivation
Combining Technology with the 6 "C's" of Motivation
 
Formative assessment: an important teaching tool for any subject
Formative assessment: an important teaching tool for any subject Formative assessment: an important teaching tool for any subject
Formative assessment: an important teaching tool for any subject
 
So, you want to do work-based learning at your school?
So, you want to do work-based learning at your school?So, you want to do work-based learning at your school?
So, you want to do work-based learning at your school?
 
TEACHER-STUDENT RELATIONSHIP
TEACHER-STUDENT RELATIONSHIPTEACHER-STUDENT RELATIONSHIP
TEACHER-STUDENT RELATIONSHIP
 
Building student motivation
Building student motivationBuilding student motivation
Building student motivation
 

Similar to IFL Workshop-Personal Finance Outcome Assessments-04-15

Measuring Impact-Methods and Metrics
Measuring Impact-Methods and MetricsMeasuring Impact-Methods and Metrics
Measuring Impact-Methods and MetricsBarbara O'Neill
 
AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...
AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...
AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...Barbara O'Neill
 
NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12
NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12
NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12Barbara O'Neill
 
Program Evaluation In the Non-Profit Sector
Program Evaluation In the Non-Profit SectorProgram Evaluation In the Non-Profit Sector
Program Evaluation In the Non-Profit Sectorwishart5
 
Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2Brent MacKinnon
 
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...OECD CFE
 
Organizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program EvaluationOrganizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program EvaluationINGENAES
 
From Reporting to Insight to Action
From Reporting to Insight to ActionFrom Reporting to Insight to Action
From Reporting to Insight to ActionEllen Wagner
 
Community and Public Health (Week 8)
Community and Public Health (Week 8)Community and Public Health (Week 8)
Community and Public Health (Week 8)Ana Anastacio
 
Grant Writing: Summary Concepts 1 4
Grant Writing: Summary Concepts 1 4Grant Writing: Summary Concepts 1 4
Grant Writing: Summary Concepts 1 4hberline
 
PROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat muftiPROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat muftiNUDRAT MUFTI
 
Full Program Design
Full Program DesignFull Program Design
Full Program Designforeman
 
Ellen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to WorkEllen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to WorkAlexandra M. Pickett
 
Salesforce Foundation HESUMMIT 2014 7Summits Social Strategies for Successf...
Salesforce Foundation HESUMMIT 2014   7Summits Social Strategies for Successf...Salesforce Foundation HESUMMIT 2014   7Summits Social Strategies for Successf...
Salesforce Foundation HESUMMIT 2014 7Summits Social Strategies for Successf...7Summits
 
Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11
Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11
Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11Barbara O'Neill
 
June 20 2010 bsi christie
June 20 2010 bsi christieJune 20 2010 bsi christie
June 20 2010 bsi christieharrindl
 
Lightning talks: the student digital experience
Lightning talks: the student digital experienceLightning talks: the student digital experience
Lightning talks: the student digital experienceJisc
 
Meaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matterMeaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matterRebecca Jones
 

Similar to IFL Workshop-Personal Finance Outcome Assessments-04-15 (20)

Measuring Impact-Methods and Metrics
Measuring Impact-Methods and MetricsMeasuring Impact-Methods and Metrics
Measuring Impact-Methods and Metrics
 
AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...
AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...
AFCPE 2016 Symposium workshop-Measuring & Reporting Impact of Financial Educa...
 
NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12
NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12
NEAFCS 2012 Financial Ed eval-assessment-impact-o'neill-09-12
 
Program Evaluation In the Non-Profit Sector
Program Evaluation In the Non-Profit SectorProgram Evaluation In the Non-Profit Sector
Program Evaluation In the Non-Profit Sector
 
Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2
 
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...
Evaluation of SME and entreprenuership programme - Jonathan Potter & Stuart T...
 
Organizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program EvaluationOrganizational Capacity-Building Series - Session 6: Program Evaluation
Organizational Capacity-Building Series - Session 6: Program Evaluation
 
From Reporting to Insight to Action
From Reporting to Insight to ActionFrom Reporting to Insight to Action
From Reporting to Insight to Action
 
Introduction to logic models
Introduction to logic modelsIntroduction to logic models
Introduction to logic models
 
Community and Public Health (Week 8)
Community and Public Health (Week 8)Community and Public Health (Week 8)
Community and Public Health (Week 8)
 
Street Jibe Evaluation
Street Jibe EvaluationStreet Jibe Evaluation
Street Jibe Evaluation
 
Grant Writing: Summary Concepts 1 4
Grant Writing: Summary Concepts 1 4Grant Writing: Summary Concepts 1 4
Grant Writing: Summary Concepts 1 4
 
PROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat muftiPROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat mufti
 
Full Program Design
Full Program DesignFull Program Design
Full Program Design
 
Ellen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to WorkEllen Wagner: Putting Data to Work
Ellen Wagner: Putting Data to Work
 
Salesforce Foundation HESUMMIT 2014 7Summits Social Strategies for Successf...
Salesforce Foundation HESUMMIT 2014   7Summits Social Strategies for Successf...Salesforce Foundation HESUMMIT 2014   7Summits Social Strategies for Successf...
Salesforce Foundation HESUMMIT 2014 7Summits Social Strategies for Successf...
 
Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11
Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11
Citi grant teacher training-fin ed evaluation-assessment tools-o'neill-06-11
 
June 20 2010 bsi christie
June 20 2010 bsi christieJune 20 2010 bsi christie
June 20 2010 bsi christie
 
Lightning talks: the student digital experience
Lightning talks: the student digital experienceLightning talks: the student digital experience
Lightning talks: the student digital experience
 
Meaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matterMeaningful Measures: Why size (and volume) doesn't matter
Meaningful Measures: Why size (and volume) doesn't matter
 

More from Barbara O'Neill

Final Rutgers Syllabus-Fall 2019
Final Rutgers Syllabus-Fall 2019Final Rutgers Syllabus-Fall 2019
Final Rutgers Syllabus-Fall 2019Barbara O'Neill
 
Inflation Causes, Impacts, Mitigation Strategies, and Benefits
 Inflation Causes, Impacts, Mitigation Strategies, and Benefits Inflation Causes, Impacts, Mitigation Strategies, and Benefits
Inflation Causes, Impacts, Mitigation Strategies, and BenefitsBarbara O'Neill
 
Income Tax Issues for Older Adults-02-23-UPDATED.pptx
Income Tax Issues for Older Adults-02-23-UPDATED.pptxIncome Tax Issues for Older Adults-02-23-UPDATED.pptx
Income Tax Issues for Older Adults-02-23-UPDATED.pptxBarbara O'Neill
 
Dr. Barbara O'Neill-Short CV
Dr. Barbara O'Neill-Short CVDr. Barbara O'Neill-Short CV
Dr. Barbara O'Neill-Short CVBarbara O'Neill
 
Inflation Fighting Strategies Class-01-23.pdf
Inflation Fighting Strategies Class-01-23.pdfInflation Fighting Strategies Class-01-23.pdf
Inflation Fighting Strategies Class-01-23.pdfBarbara O'Neill
 
Can Farmers Ever Afford to Retire-01-23.pdf
Can Farmers Ever Afford to Retire-01-23.pdfCan Farmers Ever Afford to Retire-01-23.pdf
Can Farmers Ever Afford to Retire-01-23.pdfBarbara O'Neill
 
Carney Clan Cookie Contest Entries
Carney Clan Cookie Contest Entries Carney Clan Cookie Contest Entries
Carney Clan Cookie Contest Entries Barbara O'Neill
 
RMD Planning Worksheet-12-22.pdf
RMD Planning Worksheet-12-22.pdfRMD Planning Worksheet-12-22.pdf
RMD Planning Worksheet-12-22.pdfBarbara O'Neill
 
2022 Personal Finance Trends-References
2022 Personal Finance Trends-References2022 Personal Finance Trends-References
2022 Personal Finance Trends-ReferencesBarbara O'Neill
 
Income Tax Issues for Older Adults-MTP-02-22.pdf
Income Tax Issues for Older Adults-MTP-02-22.pdfIncome Tax Issues for Older Adults-MTP-02-22.pdf
Income Tax Issues for Older Adults-MTP-02-22.pdfBarbara O'Neill
 
Beyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdf
Beyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdfBeyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdf
Beyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdfBarbara O'Neill
 
Give Yourself a Financial Check-Up-Older Adult Version
Give Yourself a Financial Check-Up-Older Adult VersionGive Yourself a Financial Check-Up-Older Adult Version
Give Yourself a Financial Check-Up-Older Adult VersionBarbara O'Neill
 
Five Tips to Flip Your Retirement Switch-06-22.pdf
Five Tips to Flip Your Retirement Switch-06-22.pdfFive Tips to Flip Your Retirement Switch-06-22.pdf
Five Tips to Flip Your Retirement Switch-06-22.pdfBarbara O'Neill
 
Leveraging LinkedIn Presentation
Leveraging LinkedIn PresentationLeveraging LinkedIn Presentation
Leveraging LinkedIn PresentationBarbara O'Neill
 
Digital Assets Inventory Presentation-BMO-04-22.pdf
Digital Assets Inventory Presentation-BMO-04-22.pdfDigital Assets Inventory Presentation-BMO-04-22.pdf
Digital Assets Inventory Presentation-BMO-04-22.pdfBarbara O'Neill
 
Creating Attractive Graphics Presentation-MTP-03-22.pdf
Creating Attractive Graphics Presentation-MTP-03-22.pdfCreating Attractive Graphics Presentation-MTP-03-22.pdf
Creating Attractive Graphics Presentation-MTP-03-22.pdfBarbara O'Neill
 
35 Useful Personal Finance Web Sites
35 Useful Personal Finance Web Sites35 Useful Personal Finance Web Sites
35 Useful Personal Finance Web SitesBarbara O'Neill
 
Social Media 101 Handouts
Social Media 101 HandoutsSocial Media 101 Handouts
Social Media 101 HandoutsBarbara O'Neill
 
Social Media 101 Twitter-Facebook-LinkedIn-02-22
Social Media 101 Twitter-Facebook-LinkedIn-02-22Social Media 101 Twitter-Facebook-LinkedIn-02-22
Social Media 101 Twitter-Facebook-LinkedIn-02-22Barbara O'Neill
 

More from Barbara O'Neill (20)

Final Rutgers Syllabus-Fall 2019
Final Rutgers Syllabus-Fall 2019Final Rutgers Syllabus-Fall 2019
Final Rutgers Syllabus-Fall 2019
 
Inflation Causes, Impacts, Mitigation Strategies, and Benefits
 Inflation Causes, Impacts, Mitigation Strategies, and Benefits Inflation Causes, Impacts, Mitigation Strategies, and Benefits
Inflation Causes, Impacts, Mitigation Strategies, and Benefits
 
Income Tax Issues for Older Adults-02-23-UPDATED.pptx
Income Tax Issues for Older Adults-02-23-UPDATED.pptxIncome Tax Issues for Older Adults-02-23-UPDATED.pptx
Income Tax Issues for Older Adults-02-23-UPDATED.pptx
 
Dr. Barbara O'Neill-Short CV
Dr. Barbara O'Neill-Short CVDr. Barbara O'Neill-Short CV
Dr. Barbara O'Neill-Short CV
 
Inflation Fighting Strategies Class-01-23.pdf
Inflation Fighting Strategies Class-01-23.pdfInflation Fighting Strategies Class-01-23.pdf
Inflation Fighting Strategies Class-01-23.pdf
 
Can Farmers Ever Afford to Retire-01-23.pdf
Can Farmers Ever Afford to Retire-01-23.pdfCan Farmers Ever Afford to Retire-01-23.pdf
Can Farmers Ever Afford to Retire-01-23.pdf
 
Carney Clan Cookie Contest Entries
Carney Clan Cookie Contest Entries Carney Clan Cookie Contest Entries
Carney Clan Cookie Contest Entries
 
RMD Planning Worksheet-12-22.pdf
RMD Planning Worksheet-12-22.pdfRMD Planning Worksheet-12-22.pdf
RMD Planning Worksheet-12-22.pdf
 
2022 Reference List
2022 Reference List2022 Reference List
2022 Reference List
 
2022 Personal Finance Trends-References
2022 Personal Finance Trends-References2022 Personal Finance Trends-References
2022 Personal Finance Trends-References
 
Income Tax Issues for Older Adults-MTP-02-22.pdf
Income Tax Issues for Older Adults-MTP-02-22.pdfIncome Tax Issues for Older Adults-MTP-02-22.pdf
Income Tax Issues for Older Adults-MTP-02-22.pdf
 
Beyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdf
Beyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdfBeyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdf
Beyond Legal Documents-12 Planning Strategies-Good Ending-08-22.pdf
 
Give Yourself a Financial Check-Up-Older Adult Version
Give Yourself a Financial Check-Up-Older Adult VersionGive Yourself a Financial Check-Up-Older Adult Version
Give Yourself a Financial Check-Up-Older Adult Version
 
Five Tips to Flip Your Retirement Switch-06-22.pdf
Five Tips to Flip Your Retirement Switch-06-22.pdfFive Tips to Flip Your Retirement Switch-06-22.pdf
Five Tips to Flip Your Retirement Switch-06-22.pdf
 
Leveraging LinkedIn Presentation
Leveraging LinkedIn PresentationLeveraging LinkedIn Presentation
Leveraging LinkedIn Presentation
 
Digital Assets Inventory Presentation-BMO-04-22.pdf
Digital Assets Inventory Presentation-BMO-04-22.pdfDigital Assets Inventory Presentation-BMO-04-22.pdf
Digital Assets Inventory Presentation-BMO-04-22.pdf
 
Creating Attractive Graphics Presentation-MTP-03-22.pdf
Creating Attractive Graphics Presentation-MTP-03-22.pdfCreating Attractive Graphics Presentation-MTP-03-22.pdf
Creating Attractive Graphics Presentation-MTP-03-22.pdf
 
35 Useful Personal Finance Web Sites
35 Useful Personal Finance Web Sites35 Useful Personal Finance Web Sites
35 Useful Personal Finance Web Sites
 
Social Media 101 Handouts
Social Media 101 HandoutsSocial Media 101 Handouts
Social Media 101 Handouts
 
Social Media 101 Twitter-Facebook-LinkedIn-02-22
Social Media 101 Twitter-Facebook-LinkedIn-02-22Social Media 101 Twitter-Facebook-LinkedIn-02-22
Social Media 101 Twitter-Facebook-LinkedIn-02-22
 

Recently uploaded

India Economic Survey Complete for the year of 2022 to 2023
India Economic Survey Complete for the year of 2022 to 2023India Economic Survey Complete for the year of 2022 to 2023
India Economic Survey Complete for the year of 2022 to 2023SkillCircle
 
Taipei, A Hidden Jewel in East Asia - PR Strategy for Tourism
Taipei, A Hidden Jewel in East Asia - PR Strategy for TourismTaipei, A Hidden Jewel in East Asia - PR Strategy for Tourism
Taipei, A Hidden Jewel in East Asia - PR Strategy for TourismBrian Lin
 
What Key Factors Should Risk Officers Consider When Using Generative AI
What Key Factors Should Risk Officers Consider When Using Generative AIWhat Key Factors Should Risk Officers Consider When Using Generative AI
What Key Factors Should Risk Officers Consider When Using Generative AI360factors
 
RWA Report 2024: Rise of Real-World Assets in Crypto | CoinGecko
RWA Report 2024: Rise of Real-World Assets in Crypto | CoinGeckoRWA Report 2024: Rise of Real-World Assets in Crypto | CoinGecko
RWA Report 2024: Rise of Real-World Assets in Crypto | CoinGeckoCoinGecko
 
Introduction to Entrepreneurship and Characteristics of an Entrepreneur
Introduction to Entrepreneurship and Characteristics of an EntrepreneurIntroduction to Entrepreneurship and Characteristics of an Entrepreneur
Introduction to Entrepreneurship and Characteristics of an Entrepreneurabcisahunter
 
Stock Market Brief Deck for 3/22/2024.pdf
Stock Market Brief Deck for 3/22/2024.pdfStock Market Brief Deck for 3/22/2024.pdf
Stock Market Brief Deck for 3/22/2024.pdfMichael Silva
 
MARKET FAILURE SITUATION IN THE ECONOMY.
MARKET FAILURE SITUATION IN THE ECONOMY.MARKET FAILURE SITUATION IN THE ECONOMY.
MARKET FAILURE SITUATION IN THE ECONOMY.Arifa Saeed
 
20240315 _E-Invoicing Digiteal. .pptx
20240315 _E-Invoicing Digiteal.    .pptx20240315 _E-Invoicing Digiteal.    .pptx
20240315 _E-Invoicing Digiteal. .pptxFinTech Belgium
 
ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.
ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.
ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.KumarJayaraman3
 
Monthly Market Risk Update: March 2024 [SlideShare]
Monthly Market Risk Update: March 2024 [SlideShare]Monthly Market Risk Update: March 2024 [SlideShare]
Monthly Market Risk Update: March 2024 [SlideShare]Commonwealth
 
20240314 Calibre March 2024 Investor Presentation (FINAL).pdf
20240314 Calibre March 2024 Investor Presentation (FINAL).pdf20240314 Calibre March 2024 Investor Presentation (FINAL).pdf
20240314 Calibre March 2024 Investor Presentation (FINAL).pdfAdnet Communications
 
Lundin Gold March 2024 Corporate Presentation - PDAC v1.pdf
Lundin Gold March 2024 Corporate Presentation - PDAC v1.pdfLundin Gold March 2024 Corporate Presentation - PDAC v1.pdf
Lundin Gold March 2024 Corporate Presentation - PDAC v1.pdfAdnet Communications
 
Buy and Sell Urban Tots unlisted shares.pptx
Buy and Sell Urban Tots unlisted shares.pptxBuy and Sell Urban Tots unlisted shares.pptx
Buy and Sell Urban Tots unlisted shares.pptxPrecize Formely Leadoff
 
Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...
Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...
Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...Matthews Bantsijang
 
LIC PRIVATISATION its a bane or boon.pptx
LIC PRIVATISATION its a bane or boon.pptxLIC PRIVATISATION its a bane or boon.pptx
LIC PRIVATISATION its a bane or boon.pptxsonamyadav7097
 
Stock Market Brief Deck for March 19 2024.pdf
Stock Market Brief Deck for March 19 2024.pdfStock Market Brief Deck for March 19 2024.pdf
Stock Market Brief Deck for March 19 2024.pdfMichael Silva
 
Contracts with Interdependent Preferences
Contracts with Interdependent PreferencesContracts with Interdependent Preferences
Contracts with Interdependent PreferencesGRAPE
 
Slideshare - ONS Economic Forum Slidepack - 18 March 2024.pptx
Slideshare - ONS Economic Forum Slidepack - 18 March 2024.pptxSlideshare - ONS Economic Forum Slidepack - 18 March 2024.pptx
Slideshare - ONS Economic Forum Slidepack - 18 March 2024.pptxOffice for National Statistics
 
Solution manual for Intermediate Accounting, 11th Edition by David Spiceland...
Solution manual for  Intermediate Accounting, 11th Edition by David Spiceland...Solution manual for  Intermediate Accounting, 11th Edition by David Spiceland...
Solution manual for Intermediate Accounting, 11th Edition by David Spiceland...mwangimwangi222
 

Recently uploaded (20)

India Economic Survey Complete for the year of 2022 to 2023
India Economic Survey Complete for the year of 2022 to 2023India Economic Survey Complete for the year of 2022 to 2023
India Economic Survey Complete for the year of 2022 to 2023
 
Taipei, A Hidden Jewel in East Asia - PR Strategy for Tourism
Taipei, A Hidden Jewel in East Asia - PR Strategy for TourismTaipei, A Hidden Jewel in East Asia - PR Strategy for Tourism
Taipei, A Hidden Jewel in East Asia - PR Strategy for Tourism
 
What Key Factors Should Risk Officers Consider When Using Generative AI
What Key Factors Should Risk Officers Consider When Using Generative AIWhat Key Factors Should Risk Officers Consider When Using Generative AI
What Key Factors Should Risk Officers Consider When Using Generative AI
 
RWA Report 2024: Rise of Real-World Assets in Crypto | CoinGecko
RWA Report 2024: Rise of Real-World Assets in Crypto | CoinGeckoRWA Report 2024: Rise of Real-World Assets in Crypto | CoinGecko
RWA Report 2024: Rise of Real-World Assets in Crypto | CoinGecko
 
Introduction to Entrepreneurship and Characteristics of an Entrepreneur
Introduction to Entrepreneurship and Characteristics of an EntrepreneurIntroduction to Entrepreneurship and Characteristics of an Entrepreneur
Introduction to Entrepreneurship and Characteristics of an Entrepreneur
 
Stock Market Brief Deck for 3/22/2024.pdf
Stock Market Brief Deck for 3/22/2024.pdfStock Market Brief Deck for 3/22/2024.pdf
Stock Market Brief Deck for 3/22/2024.pdf
 
MARKET FAILURE SITUATION IN THE ECONOMY.
MARKET FAILURE SITUATION IN THE ECONOMY.MARKET FAILURE SITUATION IN THE ECONOMY.
MARKET FAILURE SITUATION IN THE ECONOMY.
 
Monthly Economic Monitoring of Ukraine No.230, March 2024
Monthly Economic Monitoring of Ukraine No.230, March 2024Monthly Economic Monitoring of Ukraine No.230, March 2024
Monthly Economic Monitoring of Ukraine No.230, March 2024
 
20240315 _E-Invoicing Digiteal. .pptx
20240315 _E-Invoicing Digiteal.    .pptx20240315 _E-Invoicing Digiteal.    .pptx
20240315 _E-Invoicing Digiteal. .pptx
 
ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.
ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.
ACCOUNTING FOR BUSINESS.II DEPARTMENTAL ACCOUNTS.
 
Monthly Market Risk Update: March 2024 [SlideShare]
Monthly Market Risk Update: March 2024 [SlideShare]Monthly Market Risk Update: March 2024 [SlideShare]
Monthly Market Risk Update: March 2024 [SlideShare]
 
20240314 Calibre March 2024 Investor Presentation (FINAL).pdf
20240314 Calibre March 2024 Investor Presentation (FINAL).pdf20240314 Calibre March 2024 Investor Presentation (FINAL).pdf
20240314 Calibre March 2024 Investor Presentation (FINAL).pdf
 
Lundin Gold March 2024 Corporate Presentation - PDAC v1.pdf
Lundin Gold March 2024 Corporate Presentation - PDAC v1.pdfLundin Gold March 2024 Corporate Presentation - PDAC v1.pdf
Lundin Gold March 2024 Corporate Presentation - PDAC v1.pdf
 
Buy and Sell Urban Tots unlisted shares.pptx
Buy and Sell Urban Tots unlisted shares.pptxBuy and Sell Urban Tots unlisted shares.pptx
Buy and Sell Urban Tots unlisted shares.pptx
 
Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...
Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...
Remembering my Totem _Unity is Strength_ growing in Bophuthatswana_Matthews B...
 
LIC PRIVATISATION its a bane or boon.pptx
LIC PRIVATISATION its a bane or boon.pptxLIC PRIVATISATION its a bane or boon.pptx
LIC PRIVATISATION its a bane or boon.pptx
 
Stock Market Brief Deck for March 19 2024.pdf
Stock Market Brief Deck for March 19 2024.pdfStock Market Brief Deck for March 19 2024.pdf
Stock Market Brief Deck for March 19 2024.pdf
 
Contracts with Interdependent Preferences
Contracts with Interdependent PreferencesContracts with Interdependent Preferences
Contracts with Interdependent Preferences
 
Slideshare - ONS Economic Forum Slidepack - 18 March 2024.pptx
Slideshare - ONS Economic Forum Slidepack - 18 March 2024.pptxSlideshare - ONS Economic Forum Slidepack - 18 March 2024.pptx
Slideshare - ONS Economic Forum Slidepack - 18 March 2024.pptx
 
Solution manual for Intermediate Accounting, 11th Edition by David Spiceland...
Solution manual for  Intermediate Accounting, 11th Edition by David Spiceland...Solution manual for  Intermediate Accounting, 11th Edition by David Spiceland...
Solution manual for Intermediate Accounting, 11th Edition by David Spiceland...
 

IFL Workshop-Personal Finance Outcome Assessments-04-15

  • 1. Personal Finance Outcome Assessments: Methods and Metrics for Face-to-Face Classes and Social Media Outreach Barbara O’Neill, Ph.D., CFP®, CRPC® Distinguished Professor, Rutgers University oneill@aesop.rutgers.edu
  • 2. Are You Evaluating Your Financial Education Programs? If so, how?
  • 3. The Million Dollar “So What?” Question… At the end of the day…did your financial education program make a difference? How do you know?
  • 4. The Current State of Financial Education Program Evaluation • Current evaluation efforts are still far from satisfactory • General lack of evaluation capacity • Evaluation is often treated as an after-thought • Outcomes (e.g., measures of changed behavior) are often confused with program outputs (e.g., number of participants)
  • 5. • What gets measured gets funded • With evaluation results, you can… – assess impact of programs on learners – see if you accomplished what you planned – know if a program was “worth it” – celebrate success and learn from failure – make informed decisions to improve, hold, or fold programs – promote your program and win public support We are in an “Accountability Era”
  • 6. Consider a Third Party Program Evaluator • Some funders require this (e.g., FINRA Investor Education Foundation and IMLS) • No potential for conflict of interest by one entity performing both education and evaluation role • Takes evaluation “monkey” off busy educators’ back • Build into grant funding requests
  • 7. • See http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html • Begins with the end in mind • Explains what a program is and what it will accomplish • Shows relationships between inputs, outputs, and outcomes Introducing the Logic Model
  • 8. Building a Strong Financial Education Program INPUTS OUTPUTS OUTCOMES Program investments Activities Participation Short Medium What We Invest What We Do Who We Reach What Results SO WHAT?? What is the VALUE? Long- term
  • 10. OUTPUTS What We Do Who We Reach ACTIVITIES Assess needs and assets Design curriculum Educate students Conduct workshops Facilitate learning groups Sponsor conferences Work with the media Partner – collaborate PARTICIPATION Participants Clients Customers Users Groups Reactions - Satisfaction
  • 11. OUTCOMES What Results for Individuals, Organizations, Communities..… SHORT Learning Awareness Knowledge Attitudes Skills Opinion Aspirations Motivation MEDIUM Action Behavior Practice Decisions Policies Social action LONG-TERM Conditions Human Economic Civic Environment
  • 12. OUTCOMES Commonly Measured Items That Are Not Outcomes • Participant satisfaction • Number of people taught • Units of education completed • Number of events held • Time and money spent • Level of effort
  • 13. Activity #1 What are some outcomes for your financial education programs?
  • 14. Impact Evaluation Data Collection Methods • Surveys (Paper or Online) – Post-evaluation only (short programs) – Pre- and post-evaluation – Follow-up (e.g., 3 months later) • Focus groups • Interviews • Observations • Tests of knowledge/ability • RARE: Control groups and longitudinal studies
  • 15. Typical Survey Questions • General reactions to the program • Changes in knowledge • Changes in motivation, confidence, and abilities • Intended changes in behavior • Actual changes in behavior • Future programming needs and preferences • Demographics of participants • Qualitative/open-ended responses
  • 16. Post-Then-Pre (Retrospective) Evaluation Method • Compares knowledge and attitudes before and after a financial education intervention • Administered once at the end of the intervention • Helps avoid biases due to people thinking they know more than they know
  • 17. Case Example: Humpty Dumpty “Humpty Dumpty sat on a wall, Humpty Dumpty had a great fall, All the king’s horses and all the king’s men Couldn’t put Humpty together again”
  • 18. So Who/What Was Humpty Dumpty? Probably NOT what you think!
  • 19. Post-Then-Pre (Retrospective) Evaluation Method • Why? Because people don’t know what they don’t know! • Who knew what Humpty Dumpty really was? • Five point rating scale – 1 = Strongly Disagree – 5 = Strongly Agree • Post: After listening to Barb, I know the history of the Humpty Dumpty nursery rhyme • Pre: Before listening to Barb, I knew the history of the Humpty Dumpty nursery rhyme
  • 20. Post-Then-Pre Evaluation • Also known as a “Retrospective” evaluation • Helps identify changes in knowledge, attitudes, and behavior
  • 21. Another Post-Then-Pre Evaluation Example Big Advantage: Avoids the problem of learners under- estimating what they don’t know before a program
  • 22. Critical Incident Technique • Qualitative evaluation research method where program participants tell personal stories • Often used with “Train the Trainer” programs for professionals (e.g., teachers, librarians) • Incidents are categorized and deemed successful (positive results) or unsuccessful (negative results) http://www.joe.org/joe/2013june/tt2.php (Journal of Extension article)
  • 23. More About the Critical Incident Technique http://www.adb.org/documents/information/knowledge- solutions/the-critical-incident-technique.pdf • Ask subjects to describe- through interviews-incidents that they handled well or poorly (need not be spectacular events) – Example: How did library staff handle patrons’ personal finance questions? • Provides rich personal perspectives • Pre-intervention and Post-intervention comparison of number and type of incidents
  • 24. Critical Incident Technique Example: NYPL Grant Program Title: Money Matters
  • 25. CIT Questions in NYPL Financial Education Evaluation • Think about your experiences helping patrons with personal finance questions. Remember a time when you had a successful experience helping someone with these types of questions. Please write down what happened. • What made this a successful, positive experience? • Think about your experiences helping patrons with personal finance questions. Remember a time when you had an unsuccessful experience helping someone with these types of questions. Please write down what happened. • What made this an unsuccessful or challenging experience?
  • 26. CIT Summary in Final NYPL Project Evaluation Report “The change in the percentages of the types of incidents is indicative of change in attitudes and abilities. Also, a new category appeared in the successful incidents, Increased Knowledge/Confidence/Satisfaction, which was not present in the Pre-training Survey. Furthermore, for the unsuccessful critical incidents, the category Lack of Training/ Knowledge which was the most frequently seen theme for the Pre- training Survey was not found at all in the Post-training Survey. These findings resonate with quantitative findings and provide confirming evidence that the Money Matters training has been successful in improving participants’ knowledge of personal finance and their ability to be successful in handling patron inquiries. Additionally, staff members provided eloquent testament in their qualitative responses that their skills have been enhanced and their attitudes and behaviors became more positive for ably handling personal finance queries” (Radford, 2013).
  • 28. America Saves Week (ASW) Social Media Campaign • Conducted by eXtension (online program of the Cooperative Extension System), 2011-2015 • Prepared “cut and paste” tweets and FB messages on saving, debt/expense reduction, America Saves, ASW • Project cooperators from eXtension, America Saves campaigns, military, others
  • 29. Triangulation (Multiple Methods) Evaluation Approach • Unique Twitter hashtag: #eXasw • Follow-up follower/friend survey • Follow-up project participant survey • bit.ly analytics to determine number of clicks on unique embedded links • Pre- and post-ASW Twitter influence metrics (Klout score)
  • 30. Quantitative Impact Data: 2011-2014 • #eXasw tweets recorded (1,091- 1,503) • Facebook & Twitter message clicks (602- 2,814) • Followers/friends reached (3,056- 8,163) • Followers/friends reached (average): (192-254)
  • 31. Follower/Friend Survey Results: 2012 Example • Response: 50% “very helpful” and 50% “helpful” • 54% visited ASW links; 38% planned to • 8% joined AS; 38% planned to; 16% already joined AS; 38% had not joined AS • Mostly positive comments about SM messages • One negative comment about seeing repeated messages and the need for more action steps
  • 32. Professional Collaborator Survey Results: 2012 Example • 22/22 Klout scores increased – Average from 20.3 to 29.3 (vs. 11.22 to 19.68 in 2011) • Positive comments (e.g., wanted more “cut and paste” messages, improved SM skills, ease of use, continue in 2013) • State Governor and Senator started following one participant • Reported likes, retweets, and direct messages
  • 33. Klout Score Progression • Klout is a measure of a person’s “influence” on twitter • Based on an algorithm with factors such as number of followers and number of retweets • Go to www.klout.com and log in with Twitter or Facebook – 2011: 11.22 to 19.68 – 2012: 20.3 to 29.3 – 2013: 32.6 to 39.6 – 2014: 29.76 to 38.84
  • 34. Project Take-Aways • Social media messages are great for increasing awareness but do not appear to prompt much action • Project participants like having “cut and paste” social media messages • Twitter influence (Klout score) of project participants became noticeably higher over time • Social media is an effective outreach tool
  • 35. Twitter Hashtags • A key component of Twitter use is the hashtag (#) symbol followed by a series of unique characters • The hashtag serves like a “magnet” to link together the tweets of Twitter users who are interested in the same topic (e.g., credit) or who are linked together in some way (e.g., professional conference attendees) • By using a hashtag, users can effectively hold a “conversation” with one another about a topic of interest
  • 36. Twitter Chats • Online users gather at a specified time (e.g., every Friday at 3 pm ET) to “discuss” (read: tweet about) certain topics, questions, or issues. • Synchronous learning environment where participants interact in real time using their Twitter user names (a.k.a.,”handles”). • As with tweets at conferences, the “glue” that holds Twitter chats together is the designated hashtag • Twitter chat hashtag creates a running stream of tweets so that everyone can follow the “conversation.”
  • 37. More About Twitter Chats • Chat hashtags are often derived from a chat sponsor’s name (example: #wbchat for WiseBread); industry (example: #creditchat); or program title (#SSHWchat for the Small Steps to Health and Wealth™ program) • A Twitter chat hashtag creates a running stream of tweets so that everyone can follow the “conversation” • The formatting convention used to follow Twitter chat threads is Q1 for Question 1 and A1 for participant responses to that question
  • 38. Twitter Chat Evaluation Methods • Online survey (e.g., Qualtrics) link embedded into final tweets (with prizes as an incentive to complete) Dr. Barbara O'Neill @moneytalk1Apr 29 Please let us know if U found #SSHWchat helpful & take this brief survey: https://rutgers.qualtrics.com/SE/?SID=SV_9X0V77kUKhOqlWB … Will pull winners at 1:30 pm #sshwchat • TweetReach statistics: http://tweetreach.com/ (type in chat hashtag to pull a report) • Follow-up contact from participants (Twitter direct messages, e-mail, etc.) • Traffic to Web site during and after a chat • Other?
  • 40. #eXASchat Evaluation Results • Four consecutive Monday evenings in February 2014 to promote America saves and saving money • Hashtag: #eXASchat • 69 participants • Estimated reach: 42,838 (estimated overall reach of tweets in report) • Exposure impressions: 249,382 (based on participants’ followers; tells how many tweets were sent by users to their followers) • Number of tweets sent by users with > 1,000 followers (96th percentile of all Twitter users): 38
  • 41. #SSHWchat Evaluation Results • Held at noon ET on 4/29/14 for Financial Literacy Month • Topic: Strategies to improve health and finances • 34 participants • Estimated reach: 25,294 (estimated overall reach of tweets in report) • Exposure impressions: 50,553 • Six-question Qualtrics survey: 29% response rate – 100% of respondents reported learning something new – 100% of respondents reported planned behavior changes • Comment: “Please have one monthly, if possible”
  • 42. Archive a Twitter Chat With Storify and Track Use https://storify.com/ Storify is an organized collection of tweets with links, photos, etc. to “tell a story” from Twitter content Video: https://www.youtube.com/watch?v=l9iHniFjiVc Storify Samples: https://storify.com/RutgersNJAES/small-steps-to-health-and-wealth https://storify.com/RutgersSEBS/cook-douglass-community-day-2014-at-rutgers https://storify.com/wisebread/how-are-you-saving-for-retirement https://storify.com/JerryBuchko/mcpd2014-twitter-chat-archive-1
  • 43. One Last Step: Share Your Program Evaluation Results WHAT? SO WHAT? NOW WHAT?
  • 44. Who Needs to Know About the Results of Your Financial Education Programs? Why?
  • 45. Tips for Telling Your Story • Use simple descriptive statistics (e.g., counts, percentages, and averages) • Don’t use jargon • Don’t overstate your results (e.g., causality) • Blend quantitative and qualitative data • Clearly describe who the results represent (i.e., demographic characteristics of participants) • Be honest about your program’s strengths and weaknesses, while highlighting the positive
  • 46. Impact Statements: Intentions As a result of participating in this financial education program, X% of participants reported that they… • plan to do/use/adopt… • are more knowledgeable about… • are more confident in their ability to… • are more likely than before to do/use/adopt… • will do/use/adopt… …a particular attitude, piece of information, or behavior.
  • 47. Impact Statements: Actions As a result of participating in this financial education program, X% of participants reported that they… • are now doing… • did… • used… • increased their knowledge of… • adopted… • changed… … a particular attitude, piece of information, or behavior.
  • 48. Social Media Impact Reports • Number of participants • TweetReach report outreach numbers • Participant survey data • Data from other feedback methods • Sample Evaluation Report: http://www.slideshare.net/BarbaraONeill/ssh w-twitter-chat-impact-statement-0414
  • 49. Public Value Approach Private benefits accrue to program participants Public benefits are benefits to those who did not participate in a program It is possible to exclude non-payers from participating in programs—and enjoying private benefits—but it is not possible to exclude non-payers from receiving public benefits of programs
  • 50. How Do Financial Education Programs Create Public Value?  Does it narrow an information gap?  Does it address a crucial concern about fairness?  Does one person’s participation benefit people who do not participate in the program?  Does one person’s participation reduce costs for others?  Does the program improve upon market outcomes?
  • 51. Program participants… have been shown to change their behavior in specific ways… that have been shown to lead to specific outcomes… that directly benefit the participants. that generate public value.
  • 52. Participants in a nutrition program… have been shown to increase their consumption of fruits and vegetables… which has been shown to reduce the incidence of diabetes… which directly improves quality of life for participants. which reduces public health costs for all citizens.
  • 54. Public Value Statements • Focus on an outcome that matters to stakeholder(s) • Use stakeholder’s language • Avoid jargon and empty words • Should be short and believable
  • 55. Public Value Statement Template When you support _______________________program, participants will ______________________________, (changes) which leads to _______________________________, (outcomes) which will benefit other community members by _____________________________________________. (public value)
  • 56. Activity #2 Write a public value statement for your financial education program?
  • 59. Downloadable NEFE Financial Education Evaluation Manual http://toolkit.nefe.org/Portals/0/Toolkit-Manual.pdf
  • 60. Questions and Comments? Barbara O'Neill, Ph.D., CFP®, CRPC Extension Specialist in Financial Resource Management and Distinguished Professor, Rutgers University Phone: 848-932-9126 E-mail: oneill@aesop.rutgers.edu Internet: http://njaes.rutgers.edu/money/ Twitter: http://twitter.com/moneytalk1