1. +EOIS-CaMS
DATA MIA
EOIS-CaMS Data Management, Integrity and Analysis (Data MIA)
Prepared by: Robyn Cook-Ritchie, RCR Consulting
Management Integrity
Analysis
2. +
Focus for todayâĻData:
īŽ Management
īŽ Processes to collect good data
īŽ Integrity
īŽ Processes to input good data
īŽ Timely data
īŽ Analysis
īŽ Using the data to ask questions and make good decisions
related to the five core measures for
2014-2015
3. +
Currently Available Reports-SDS
īŽ LBS - All Data â IR
īŽ LBS- All Data-
Outcomes/Follow-ups
īŽ LBS âAll Data- Service Plan
/Profile
īŽ LBS - Case Activity
īŽ DF - LBS - Participation
īŽ DF - LBS - Information Session
īŽ DF - LBS - Sub Goals And Plan
Items
īŽ DF - LBS - Service Plans
īŽ LBS - Detailed Service Quality
īŽ DF - Clients (Participants)
īŽ DF - Education History
īŽ DF - Employment History
īŽ DF - Follow-up Events
īŽ Service Plan Case Follow Up
īŽ Inactive Cases
īŽ Service Provider User
Management
4. +
Currently Available Reports-Rollup
īŽ LBS - All Data - IR (Rollup)
īŽ LBS - All Data- Outcomes/Follow-ups (Rollup)
īŽ LBS - All Data- Service Plan/Profile (Rollup)
īŽ LBS - Service Delivery Site Mailing Labels
īŽ LBS - Service Provider Mailing Labels
īŽ LBS - Detailed Service Quality (Rollup)
īŽ DF - Codes And Descriptions
īŽ DF - Employers
Available for the
province or by region:
Central
Eastern
Northern
Western
5. +
Useful Reports
Report 60D
īŽ LBS-All Data Outcomes/Follow-
ups
Summarizes all outcome
information
īŽ Gives you summary data on
completions and outcomes at
exit, 3, 6 and 12 months
Report 60B
īŽ LBS-All Data Service Plan
Profile
Summarizes all profile information
īŽ Gives you summary data for 4
of the 5 core measures in 2014-
2015
Report 61
īŽ LBS Case Activity
Shows all data related to service
plans that are open, approved,
active or have been closed in the
fiscal year (weekly)
Report 19A
īŽ Service Plan Case Follow-up
Shows follow-up reviews that are
overdue or due within the next 30
days (weekly)
6. +
Most Important Report
Report 64
īŽ Detailed Service Quality
The MOST IMPORTANT report.
The only performance measure report.
īŽ Shows performance commitments and actual results related
to:
īŽ Customer Service
īŽ Effectiveness
īŽ Efficiency
This is the report you use to complete your Quarterly Status and
Adjustment Report (QSAR).
7. +
Report 64- Where did that number come
from?
Phase 11-A: Five measures in 2014-2016
Dimension Measure Weight Standard SQS
Value
Customer
Service (40%)
1. Customer
Satisfaction
15% 90% 1.35
2. Service
Coordination
25% 50% 1.25
Effectiveness
(50%)
3.
Suitability/Learner
Profile (all 12
indicators)
20% 30% 0.6
5. Learner
Progress
30% 60% 1.8
Efficiency (10%) 7. Learners
Served
10% 90% 0.9
SQS
Standard
5.9
8. + Five Core Measures
2014-2015
Customer
Satisfaction
Service
Coordination
Suitability
Learner
Progress
Learners
Served
What is
different
next fiscal
year?
9. +
Dimension-Customer Service (40%)
Measure #1: Customer Satisfaction (15%)
īŽ This number is based on closed service plans
īŽ Data comes from the answer to the question âOn a scale of 1-5,
how likely are you to recommend the LBS Program to someone
looking for similar services as those you receivedâ -learners who
respond to the question with a rating of 4 or 5 are considered
satisfied
īŽ This information is captured on the Participant Exit Form and is
entered when a service plan is closed
īŽ Found on the service plan home page
īŽ 2014-2015 target: 90%
10. + Customer Satisfaction
Integrity
īŽ Prior to closing a file in CaMS do
we have a process in place to
make sure we have collected the
customer satisfaction data so it
can be entered? (e.g. closing file
checklist)
Management
īŽ Is this information collected
from the learner at exit in a
formal way? (e.g. using the old
Learner Satisfaction Survey or
another survey)?
īŽ Is this information kept in the
learner file?
īŽ Is customer satisfaction data
collected at points other than
exit?
11. +
Analysis
Are 90% of our learners satisfied with our service? Where do I find
this information?
Review report 64 on a monthly basis
If not:
īŽ Ask learners why not and make changes based on that information
īŽ Survey learners more frequently
īŽ Implement an anonymous suggestion box
Possible changes:
īŽ Hours of operation
īŽ Offer short-term focused programming options
Customer Satisfaction
12. +Dimension-Customer Service (40%)
Measure #2: Service Coordination (25%)
īŽ Based on closed service plans
Three sources:
īŽ Referred in (when a service plan is opened âcaptured on the
Participant Registration Form)
īŽ Referral during service (as a service plan sub-goal)
īŽ Referred at exit (as a service plan sub-goal)
īŽ Referrals in from EO-Literacy and Basic Skills Providers, or
Informal or Word or Mouth/Media Referral are not included
īŽ 2014-2015 target: 50%
13. + Service Coordination
Integrity
īŽ Do we make sure that a âreferral inâ is
recorded in CaMS when a service plan is
opened?
īŽ Are referrals during service and at exit
that are recorded on the learner plan
entered on the service plan in CaMS?
īŽ Do we check to make sure all referrals
have been entered prior to closing a
service plan in CaMS?
Management
īŽ At intake, do we record a
âreferral inâ on the PRF if
there is one?
īŽ Do we keep track of any
referrals made during
service and record them on
the learner plan?
īŽ Do we record referrals made
at exit and record it on the
learner plan?
īŽ Do we keep additional
documentation related to
referrals in the learner file?
14. +
Analysis
Are we meeting the service coordination target of 50%? Where
do I find this information?
If not:
īŽ Determine where the gaps are- use report 60B to give you a
summary of âreferral inâ and âreferral outâ information
īŽ Make sure you are capturing all referrals that take place
īŽ Review the report 64 guide to see what referrals âcountâ
īŽ Discuss it at your service planning meeting with other
community partners
Service Coordination
15. +
Dimension-Effectiveness (50%)
Measure #3: Suitability/Learner Profile (20%)
īŽ Based on closed service plans
īŽ Phase II-A includes all suitability indicators (see page 31 of the 2014-2015 Service
Provider Guidelines):
âĸ Less than grade 12
âĸ Income source (OW/ODSP/No source of income/crown ward)
âĸ Out of education six years or more
âĸ Out of training for six years or more
âĸ Over 45 and under 64
âĸ History of interrupted education
âĸ Person with a disability
âĸ Aboriginal person
âĸ Deaf/Deaf blind
âĸ Francophone
īŽ This information is captured on the Participant Registration Form (PRF) and
recorded when creating the case in CaMS, and when entering information on the
Person Home Page and in the Client Summary in CaMS
īŽ 2014-2015 target: 30%
16. + Suitability/Learner Profile
Management
īŽ Is the PRF filled out completely?
īŽ Does intake staff understand
which items on the the PRF
relate to suitability/learner profile
in CaMS?
īŽ Is the PRF and CaMS person
home page updated if a client
self-identifies after the intake
process (e.g. aboriginal
person)?
Integrity
īŽ Is the information on the
person home page in CaMS
correct and up to date?
īŽ Do all learners have a client
summary in CaMS? â Use
report 60B to determine if any
are missing
īŽ Is client summary completed
and up to date in CaMS?
17. + Suitability/Learner Profile
Analysis
Are we meeting our suitability criteria?
If not:
īŽ What suitability indicators are not represented? -Use report 60B for a
summary of the suitability indicators represented or use the sort
feature in Report 61
īŽ Think about ways your agency can target underrepresented groups.
īŽ Do your referral partners understand the suitability criteria for the LBS
Program?
18. + Dimension-Effectiveness (W-50%)
Measure #5: Learner Progress (W-30%)
īŽ Based on both active and closed service plans
īŽ It is the percentage of learners who have completed at least
one milestone
īŽ Data comes from âattainedâ competency plan items
īŽ 2014-2015 target: 60%
19. + Learner Progress
Integrity
īŽ Is there a process in place to
make sure completed
milestones are recorded in
CaMS?
īŽ Are completed milestones
stored in the learner file?
īŽ When milestones are
completed are they updated in
CaMS in a timely fashion?
īŽ Do we check to make sure all
milestones have been entered
prior to closing a service plan
in CaMS?
Management
īŽ Have practitioners had adequate
training related to the administration
of milestones?
īŽ Do learners understand that they will
need to complete milestones as part
of the program?
īŽ Do instructors have easy yet secure
access to milestones?
īŽ Is there at least one milestone
related to the learner goal path
identified on each learner plan?
20. + Learner Progress
Analysis
Are at least 60% of the learners attaining one milestone?
If not:
īŽ Ensure there are clear processes in place so that learners and
practitioners understand the expectations around completion of
milestones.
īŽ Make sure ongoing training delivery includes a variety of task-based
activities that will help practitioners and learners know when they are
ready to attempt milestones.
21. + Dimension-Efficiency (10%)
Measure #7: Learners Served (10%)
īŽ The number includes all learners with an active learner plan
served
īŽ Only one service plan per learner is counted
īŽ Only service plans with at least one milestone are included
īŽ The milestone must be âin progressâ or âcompletedâ
īŽ 2014-2015 target: 90% butâĻ.. each service delivery agency
has a performance commitment target of 100%
22. + Learners Served
Management and Integrity
īŽ Do we know how many learners we have to serve? (See
Schedule E of your service delivery agreement)
īŽ Are learner plans developed in a timely fashion for each
learner?
īŽ Are all service plans entered in the system in a timely
fashion?
īŽ Are all our service plans âactiveâ in the system? Use
Report 61
īŽ Do all service plans in CaMS have a âcompletedâ or âin
progressâ milestone?
23. + Learners Served
Analysis
Are we meeting our numbers?
If not:
īŽ What client group could we target that we are not serving?- Use Report 60B
īŽ Who are our current referral partners? Which other partners could we focus on?
īŽ Are our services accessible?
īŽ Could we offer itinerant or mobile service?
īŽ Do our operating hours meet the needs of the learners?
īŽ Is our agency contact information and service description up-to-date on the Web, in
social media and in any print material?
īŽ Do we offer programming that meets the needs of clients being served by other
partners such as Employment Services of Ontario Works?
Mention the handout that we will send after the webinar.
3 main dimensions
Two of the seven measures will not feed into your service quality score for phase 11- learner gains and completion
Completions is 4
Learner gains is 6
Suitability measure will have all 12 indicatorsâĻâĻ
What is different from the current
Service Coordination counts now
Learner progress counts now
Point out that this number is higher this year
Ask participants to brainstorm other ways to know this.
Where can I find this information? Report 64âĻand ???
Categories
Referred in
Referred out- registered in education
Referred out- registered in training
Referred out- registered in ES
Referred out- Registered or Confirmed receiving services with Community Resources that Support Learning (Referred out to other community resources)
It is a measure of those being served. It examines and quantifies client barriers to learning. The measure ensures that the service providers are providing services to learners who can benefit the most from the LBS program.
SPs must serve clients who, on average, are experiencing at least 30% of identified suitability indicators (proposed standard at maturity). This does not mean that every single client must have 30% of the identified indicators (some clients may more and some others may have fewer and the combination does not matter.)
Talk about places to find task-based activities
The number of learners you have to serve is indicated in your LBS Service Delivery Agreement Schedule E