1. Welcome to the Session on Quality Assurance,
Program Evaluation and Performance Management
in Settlement Services – OCASI Conference June 2012
Faed Hendry
Manager – Training and Outreach
Findhelp Information Services
543 Richmond Street West Suite 125
Toronto, Ontario
fhendry@findhelp.ca
416-392-4544

2. Session Learning Objectives
 To identify and apply resources and tools to
effectively enable settlement services to evaluate the
quality of both their services and staff.
 To briefly discuss the Modernized Approach to
Settlement work and how to measure outcomes.
 To identify strategies for performance management
including setting objectives, assessing progress and
providing on-going coaching and feedback.

3. What is all of this?
Accountability
Quality
Assurance
Evaluation
Program
Modernized Client Satisfaction
Service Levels
Approach
Logic Models
Metrics
SMART Objectives Standards
Key Performance Indicators
C o a c h in
g
Outcome Measurement
Performance Management

4. What we Know
The Settlement Program is an outcome-based
program. The logic model identifies both:
outcomes to achieve, and settlement activities
(streams) to achieve outcomes.
Results in terms of outcomes, outputs, and financial
resources will be gathered and monitored to
ensure activities continue to achieve expected
results and link services to specific settlement
outcomes.

5. …using a suite of services that can
From a suite of programs… … to a single program…
be combined to achieve results
The Settlement Program
An outcome-based program
Language Instruction for Newcomers to Canada
(LINC)
Needs Assessment and Referrals
A. Orientation – Newcomers make informed decisions about their
settlement and understand life in Canada
Information & Awareness Services
B. Language/Skills – Newcomers have language/skills needed to
function in Canada
Language Learning & Skills Development
Immigrant Settlement and Adaptation Program C. Labour Market Access – Newcomers obtain the required
(ISAP) assistance to find employment commensurate with their skills and
Employment-Related Services
education
D. Welcoming Communities – Newcomers receive help to establish
Community Connections
social and professional networks so they are engaged and feel
welcomed in their communities
Support Services
E. Policy and Program Development - To ensure effective delivery
and achieve comparable settlement outcomes across Canada
Host Program (Host)

6. The Modernized Approach
The Modernized Approach is outcome-based aimed at
supporting newcomers in their settlement by providing:
 the information they need to better understand life in
Canada and make informed decisions about their
settlement experience;
 language training so they have the language skills to
function in Canada;
 the required assistance to find employment that
corresponds with their skills and education; and
 the help to establish networks and contacts so they are
engaged and feel welcomed in their communities.

7. What is a Customer Service Culture?
• A customer service culture in a settlement agency exists
where services are planned and delivered in ways that
maximize positive impressions and outcomes for clients
• Customer satisfaction is a measure of the degree to which
a product or a service meets the customer’s expectation.
• It involves responding promptly and accurately to client
requests in such a way that each client feels valued,
respected, and understood.
• A settlement service should be conceived and executed
from the outside-in – not inside-out approach.

8. Customer service culture exists where:
• staff care to know what results they produce
• those results relate to client expectations
• A strong customer service culture should be
reflected in your mission, vision, values, strategic
plan, policies and procedures, performance
management system and your training and
orientation

9. Five Drivers of Satisfaction
The five “drivers of satisfaction” are the elements
that most strongly influence citizens’ perceptions
of service quality across the many services
provided by government and SPO’s including
settlement agencies.
3) Timeliness
4) Knowledge/Competence
5) Courtesy/Comfort
6) Fairness
7) Outcome

10. Mission, Vision and Values in a Customer Service Culture
• Client service cultures exist where organizational
mission, vision, values and systems support
behaviours that improve their capability to serve
clients. Effective organizations identify and develop
clear, concise work values that inform and impact
every aspect of their organization.
• All agencies have a mission, vision and values and a
charter of expectations, all of which contribute to its
organizational culture

11. What’s Yours?
Write down your organizations mission statement?
Write down your organizations values?

12. Examples
Our Mission
We provide learning and training opportunities for
immigrants and refugees
to access and fully participate in the workplace and
wider community.
Our Vision
We envision a Canada where every immigrant
succeeds.
Values
Cultural Diversity, Integrity, Compassion, Solidarity

13. So what now?
How well are your settlement staff performing?
How do you know if your staff are performing well?
How do you measure outcomes?
How do you evaluate the knowledge, skills and
attitudes of your settlement workers?
What systems do you have in place to assess
performance?

14. Examples of Outcomes for Settlement Programs
Clients have timely, useful and accurate information
needed to make informed settlement decisions
Clients understand life in Canada including laws, rights,
responsibilities and how to access community
resources
Clients have the official language skills needed to function
in Canadian society
Clients have knowledge of the Canadian work
environment and are connected to local labour markets
Clients have the skills to find and apply for employment
Clients are connected to the broader community and
social networks

15. Examples of Possible Indicators
% of clients who report that they received information which
helped them learn more about Canadian laws,
community resources, life and culture.
% of clients who report that they have the language ability
and skills needed to participate socially, culturally and
economically in Canada.
% of clients should report that they are connected to local
labour markets, have knowledge of the Canadian
workplace, and have the skills to find and apply for
employment
% of clients who report that they feel connected to the
broader community and social networks

16. Quality Assurance
A system of procedures, checks, audits and
corrective actions that are undertaken to ensure
that an organization’s products and services meet
the expectations and needs of the people they
serve.
For information and referral programs, quality
assurance relates to compliance with the
Standards for Professional Information and
Referral.

17. Program Evaluation
The systematic process of reviewing services provided by
an organization in relation to its objectives and
standards to assess how well the program is working,
and to identify ways to improve overall operation of the
individual I&R or settlement service.
Program evaluations may conducted by experts external to
the program, either inside or outside the agency, or by
program managers.

18. Performance Management
Performance management is the process of creating a
work environment or setting in which people are
enabled to perform to the best of their abilities.
The Performance Management process provides
employees and managers with the tools and support
to plan, manage and evaluate performance.

19. Discussion
Working in small groups with staff from different
organizations, discuss what systems you have in
place at your settlement agency to monitor:
• Quality Assurance
• Program Evaluation
• Performance Management

20. Performance Management
Before you embark on the development of an effective
performance management system, you should take a
moment to consider whether or not your organization has
HR management practices in place to support the
performance management process. These include:
* Well designed jobs and written job descriptions
* Effective supervision
* Comprehensive employee orientation and training
* A positive and supportive work environment

21. An Effective Performance Management System will:
 Be job specific, covering a broad range of jobs in the
organization
 Align with your organization’s strategic direction and
culture
 Be practical and easy to understand and use
 Provide an accurate picture of each employee’s
performance
 Include a collaborative process for setting goals and
reviewing performance based on two-way
communication between the employee and manager

22. An Effective Performance Management System will:
 Monitor and measure results (what) and behaviors (how)
 Provide constructive and continuous feedback on performance
 Provide training and development opportunities for improving
performance
 Establish clear communication between managers and
employees about what they are expected to accomplish

23. Critical Considerations
 Effective performance management system requires
dedicated time and resources.
 It is critical that you communicate the purpose and the
steps in the performance management process to
employees before it is implemented.
 Management support to act upon the outcomes of the
performance management process is also necessary to
ensure that good performance is recognized and
inadequate performance is effectively handled.

24. Performance Management and Program Integrity

25. Step 1: Plan
The planning phase is a collaborative effort involving
both managers and employees during which they
will:
 Identify, clarify and agree upon expectations.
 Agree upon how results will be measured
 Agree on monitoring process
 Document the plan

26. Setting Objectives and Measurements
Often the most difficult part of the planning phase is
finding appropriate and clear language to describe
the performance objectives and measures or
indicators of success.
Settlement Program Managers need to ensure that
the objectives are a good representation of the full
range of duties carried out by settlement workers.

27. Objectives should be SMART
 Specific
 Measurable
 Attainable
 Realistic
 Time-Bound

28. SMART Objectives
Specific – Be clear and unambiguous, avoid general
statements, focus on specific accomplishments that you
want to achieve and focus on end result.
Measurable – Agree on how objective will measured
Achievable – Does the employee have the necessary skills,
abilities, tools and resources?
Relevant – Does the goal move the organization towards
the vision/mission?
Timely – When do results need to be achieved?

29. Exercise: Writing SMART Objectives
Please break up into small groups, 2 to 4 people.
Please discuss and write down a couple of SMART
objectives for some of your specific programs.
Examples: Increase the number of JSW by 25% in 2012 with
a satisfaction rate of 90%.
To answer 85% of our calls within 30 seconds and achieve
an abandonment rate of less than 10% in the month of
June.

30. Set Goals and Define Expectations
• Goals must be determined before expectations can be set
and expectations have to be set before a settlement worker
can be held accountable for meeting them.
• In performance management terms, the expectation is the
minimal acceptable performance level and a goal is some
point beyond the expectation that the manager and the
settlement worker have agreed to target.
• These goals should be defined with corporate and business
unit objectives in mind and then be defined down to the
individual behaviours that you want to see demonstrated by
the settlement worker.

31. Guidelines for Defining Performance Measures
Quantity (how many clients served, number of JSW
workshops)
Quality (accuracy or error rate, level of client satisfaction)
Timeliness (completion dates, time spent)
Cost (how are finances affected by work performed)
Behaviours (do behaviours reflect organizational values)

32. What are Performance Expectations?
Performance Expectations = Results + Actions &
Behaviours
 What goods and services should the settlement
workers job produce?
 What impact should the work have on the organization?
 How do you expect settlement staff to act with clients,
colleagues, and supervisors?
 What are the organizational values the employee must
demonstrate?

33. Performance Dimensions
Performance dimensions are the range of behaviours
employees must exhibit to successfully meet or exceed
job expectations and help answer the question: “How
does someone act and/or behave when s/he does the
job well?” Examples include:
• Customer Service Orientation
• Teamwork
• Valuing Diversity
• Assessment and Problem-Solving Abilities
• Interpersonal Skills
• Fostering a Safe and Secure Environment

34. What is a valid measure of good client service?
If the measure used only considers the number of
clients served (i.e. quantity), then the quality of
service or how well it was done is not captured.
To assess quality of information provided, the
supervisor could do spot checks to listen to or
look at the information that the employee provides
to clients. The supervisor would then assess
accuracy and completeness of the information.

35. Step 2: Monitor
For a performance management system to be effective,
employee progress and performance must be
continuously monitored.
 Assess progress towards meeting performance
objectives
 Identify any barriers that may prevent the employee from
accomplishing performance objectives and what needs to
be done to overcome them
 Share feedback on progress relative to the goals

36. Effective Monitoring
Monitoring includes coaching settlement staff to
address concerns and issues related to
performance.
The main purpose of constructive feedback is to help
people understand where they stand in relation to
expected and/or productive job and workplace
behaviour.
Feedback should be provided frequently and not just at
performance appraisal time.

37. Performance Coaching
• Performance coaching is a role and expectation of the
Settlement Program Manager.
• Coaching is a process that enables learning and
development to occur and thus performance to
improve. It should be done with all settlement
workers, not just the ones that may be struggling or
require additional support.
• Performance coaching involves setting clear
expectations for the settlement worker.

38. Some Suggestions for Providing Feedback
• Effective feedback always focuses on a specific
behaviour, not on a person or their intentions.
• Successful feedback describes actions or behaviour
that the individual can do something about.
• Use a soft entry
• The best feedback is straightforward and simple
• Effective feedback needs to be well-timed
• Reach agreement about what the individual will do to
change his or her behaviour

39. Step 3: Review
The performance assessment or appraisal meeting is an
opportunity to review, summarize and highlight the
employee’s performance over the course of the review
period.
Self-assessment is a standard part of most performance
appraisals.
Managers should review their performance management
notes and documentation generated throughout the year
in order to more effectively assess the employee’s
performance.

40. Important – An Appeal Process
Even with a well-designed and implemented performance
management process, there may be situations when an
employee has a serious difference of opinion with the
manager about his or her performance assessment.
A procedure for the employee to discuss disagreement with
the process should be established. This can include:
 Step Review System
 Peer Review System
 Ombudsperson

41. Common Errors and Rating Bias
When a person evaluates someone else, his or her
evaluation reflects both the person being assessed and
the evaluator's built in biases. Managers should be aware
of their possible evaluation biases, so they can try to
eliminate them from the assessment process. Common
biases may include:
 Halo
 Horns
 Central Tendency
 Leniency bias / Strictness bias
 Same-as-me

42. Questions?
Thank you for attending this session!
Faed Hendry
Manager – Training and Outreach
Findhelp Information Services
543 Richmond Street West Ste 125
Toronto, Ontario M5V 1Y6
416-392-4544
fhendry@findhelp.ca
Enjoy the rest of the Conference!