October 20, 2008 Competency Assessment: Designing Timely, Practical, and Holistic Assessments at Ontario Colleges
CIITE Overview Colleges Integrating Immigrants to Employment (CIITE) Vision: Internationally trained immigrants have access to programs and services in the Ontario college system that build on their qualifications and expedite their securing employment commensurate with their skills and knowledge
CIITE Overview Objective: To improve the support structures and services colleges provide to skilled immigrants. Focus: The development and implementation of systemic solutions to address barriers to immigrants’ rapid entry into the workplace. Result: Improved services for skilled immigrants Supported by the Ministry of Citizenship and Immigration of the Government of Ontario Developed, led, and implemented by Ontario colleges A multi-phased strategy for modifying and refining college systems and programs
CIITE History Phase 1: December 2003 – December 2004: Identify barriers ITIs face in the Ontario college system Phase 2: September 2005 – December 2007: Develop and test selected immigrant focused practices and services in the college system Phase 3.1: April 2008 - March 2009: Implement immigrant focused practices and services in the Ontario college system
Phase 3.1 Projects April 2008 - March 2009: Project Implementation
CIITE Phase 2 identified the need for a consistent and standardized approach in order to maximize the recognition of education and experience and competency assessment was consequently funded as an additional project in Phase 2 (2007).
What does CIITE mean by Competency Assessment? Competency assessment is a process whereby the competencies of an individual are evaluated in relation to a particular occupation, task, or process (i.e., “what does a person know and can they apply their knowledge and skills to a particular standard?”). Competency assessment may be used for two purposes. The first is the assessment of prior learning to determine which components of a particular college program are lacking for graduation (competency assessment for education). The second potential application is the provision of a credential to accompany formal degrees, diplomas, and certificates which attests to practical abilities (competency assessment for employment).
What does CIITE mean by Competency Assessment? (cont’d) The CIITE Competency Assessment Project design was based on two key assumptions of what is required to conduct viable, sustainable assessments of skills and experience in Ontario colleges. First, college competency assessments should be replicable across the Ontario college system so that ITIs can access assessment accordingly, and colleges can affordably offer appropriate assessment. Second, competency assessments should be based on program and professional requirements.
Competency Assessment Project The Competency Assessment Project was funded near the end of CIITE Phase 2 (2007). CIITE is currently in Phase 3, which began in April 2008. Phase 2 Competency Assessment : Project Timeline: February 2007 – December 2007 Participants: Algonquin, Boreal, CAPLA, Centennial, Conestoga, CRTO, Fanshawe, George Brown, Humber, La Cité, Niagara, OACETT, Seneca, St Clair, and Sheridan Key deliverable: Develop outlines of models that will assess the knowledge and skills of ITIs by program rather than on a course-by-course basis in Respiratory Therapy (regulated) and Mechanical Engineering Technologist (unregulated)
Results of Phase 2 Competency Assessment Results: A Competency Assessment model (see handout) and recommendations resulting from project work and research. The recommendations are listed below: 1. Pilot the design of two competency assessment models in a regulated and unregulated profession. Competency assessment models have been designed for both a regulated (Respiratory Therapy) and unregulated (Mechanical Engineering Technologist) profession. In order to prepare to pilot these models, colleges need to validate a List of Competencies for two pilots and develop a project plan for implementation of the pilots in selected colleges. Preliminary work has also been done to establish the List of Competencies to be tested in a pilot .
2. Increase the ability of colleges to deliver flexible programs. Competency assessment allows an ITI to demonstrate their skills and knowledge in a holistic way, in order to recognize these skills and knowledge and to make informed decisions about areas that require gap training. If competency assessment is to be successful in Ontario colleges, colleges must be able to deliver their programs in a flexible manner that allows an ITI to take only the courses s/he needs to meet program and/or professional standards.
Phase 2 Results Cont’d . 3. Develop college capacity to provide methods of recognition that correspond to flexible programming. Colleges currently have residency requirements that require a certain percentage of credits be taken at a college in order for that institution to grant a credential. These requirements are a barrier to ITIs who only require specific areas of development and may not wish to enrol in a college program. Colleges should re-examine residency requirements in order to provide a credential to ITIs who successfully complete a competency assessment. 4. Examine the opportunity to develop regional centres of assessment. Competency assessment is an expensive process and needs to be sustainable. Rather than preparing all colleges to complete competency assessments in each profession, regional centres of assessment should be designated. These could be coordinated through a central competency assessment office that would provide both colleges and candidates with a central point of administration. Assessment should be available in both official languages.
5. Colleges undergo discussions at all levels (faculty, management, etc.) with regard to the issues, implications, and opportunities for competency assessment in Ontario colleges. The implementation of competency assessment in the Ontario college system provides se. Colleges need to engage in discussion in order to make informed decisions on the adoption of competency assessment.
Phase 3.1 Overview The competency assessment activity in Phase 3.1 is developing, validating and piloting the model in two occupations: Respiratory Therapy (RT) and Mechanical Engineering Technology (MET). Teams of experts in each profession have developed the professions’ ‘List of Competencies’ to be tested, as well as assessment tools. These will be validated by employers, regulatory bodies, and/or professional associations before being piloted in the colleges in the last three months of Phase 3.1.
Phase 3.1 Participants MET – Algonquin, Conestoga, Fanshawe, George Brown, Georgian, Humber, St. Clair, Sheridan and OACETT in an advisory capacity RT – Algonquin, La Cité, Fanshawe and the CRTO in an advisory capacity
Validate standards for assessment in Respiratory Therapy (RT) and Mechanical Engineering Technology (MET)
Create assessment tools for competency assessment in RT and MET
Develop workplan and prepare for pilots at 3 colleges in RT and 8 in MET
Establish a Working Group that will ultimately develop an implementation strategy.
In the last 3 months:
Launch on a pilot basis the competency assessments in up to 6 colleges for each of RT and MET.
Implement strong measurement and evaluation processes and systems, including full tracking of activity, costs and outcomes.
Overview of RT Process
Update on Respiratory Therapy Competency Assessment The Respiratory Therapy project has worked closely with the regulatory body, the Council of Respiratory Therapists of Ontario (CRTO) in ensuring that the checklist of competencies meets the most up-to-date regulations and policies. Currently, the team is developing the assessment tools. The assessment will have three levels of testing: written, simulated, and clinical. A candidate has to be successful at each level of simulation in order to move on to the next. The pilot is set to begin in January 2009.
Challenge:Success of candidates - As the competency assessment aims to test the qualifications necessary to practice Respiratory Therapy(RT) in Ontario there is concern about the success of candidates. Respiratory Therapy is primarily a North American profession and many of the internationally trained applicants will be individuals who are switching professions (e.g. Anesthesiology Assistants, Intensive Care specialist, etc.). There is concern that there won’t be many individuals in the pilot who pass the written test. Action: Searching for pilot participants with relevant experience
Challenge: Cost of assessment for candidate – A general concern that should be kept in mind when developing the competency assessment is the direct financial costs of taking the assessment, the cost of travel and having to take time off work, securing child care, etc.
Update on Mechanical Engineering Technology
The team has determined 14 competencies which cover the skills and knowledge that are fundamental for an MET. OACETT has recognized the competencies as valid.
The team is currently in the process of developing the assessment tools to be piloted in January 2009.
Stage 1 Information/Advising Stage 3 Submission of Required Documents Stage 6 Competency Assessment Stage 2 Preparation of Required Documents Stage 4 Analysis of Education Stage 5 Develop a Plan
Candidate acquires information on applying for credit for previous learning and/or college competency assessment through websites, information packages, or advising services
Candidate prepares all relevant evidence requirements Candidate submits all necessary documents to the college, which may include: resume, supporting documents (evaluated transcripts, course outlines, certificates, licenses, etc.) and language test scores. The information is placed into the Record of Education and Experience (REE).
A knowledgeable Assessor is appointed by the college.
Outcomes of Competency Assessment MET Competency Assessment Letter of Achievement Referral Statement of Equivalence
The LOA will detail which skills were demonstrated and how and refer the candidate to further training to fill gaps
(Depending on the gaps, ITIs may be eligible to earn a diploma at the college. For example, if ITIs have to take enough courses to meet the residency requirement at the college, they may be eligible for a diploma.)
In the LoA it will state that the candidate’s skills and knowledge are equivalent to the competencies a student would acquire in a college Mechanical Engineering Technologist (MET) program.
CIITE is currently in discussion with OACETT to ensure the individuals who successfully complete the competency assessment are eligible for OACETT certification.
Challenges in MET Length of the assessment – The assessment should be as timely as possible. If an individual has to take the entire competency assessment( such as in the case of a refugee unable to produce transcripts or course outlines) they would currently have to undergo approximately 40 hours of assessment. Action: The team is currently revising the assessments to make this time shorter, while maintaining the quality of the assessment. Cost of assessment for candidate – A general concern that should be kept in mind when developing the competency assessment is the direct financial costs of taking the assessment, the cost of travel and having to take time off work, securing child care, etc. Success of the candidate – There is a concern that METs who have been in the workforce for many years won’t be able to remember some of the competencies learned in a college program. Action: The team is developing the assessment questions to be as “high level” as possible to help alleviate this issue. College Program Learning Outcomes verses On-the-Job Acquired Competencies – The competency assessment will test outcomes of the college program/ competencies needed for certification from OACETT. It will not test all of the skills and knowledge someone would develop in the workforce over a number of years. Should we test mid-career and senior-career skills? The next phase of development could include direct involvement with industry to develop job competencies.
Should there be separate “list of competencies” for ITIs with mid/senior level experience?
What are the implications for roll-out of competency assessment in Ontario colleges (cost, access to qualified Assessors, tools)? Is there opportunity to collaborate with other organizations to improve access to competency assessment for ITIs?