To assign the right person to the right job … … who masters the requiredskills… … in orderthatone’swork output meets the organization’sneeds for operations and sustainablegrowth Isn’titcritical ? Claude Trépanier Professional in measurement, assesssment and development of skills Extended version of a scientific communication deliveredat the 2008 Annualconference of the Canadian EducationalResearcher’s Association (http://www.csse.ca/CERA/home.htm) Copyright 2010. All rightsreserved.
Content Practicalproblemaddressed Study of researchproblem Selected a skills typologies Action-researchundertook Applications of skills typologies Furtherscientificresearchesrequired Conclusion References
1. Practicalproblemaddressed Researchobject : Skills set of a position Problem observed: Teachers and workplace trainers cannot easily reuse job skills set, as defined by employers, to design courses aimed at developing skills in a valid and effective fashion. Drawbacks : The skillsdevelopeddon’tyield optimal correlationwithskillsexpected in the job, theydon’tyield optimal performance (effectiveness = results) and competence(efficiency = use of resources).
Cost of the problem “The World Bank finances about US$720 million in training every year, through both its lending projects and its in-house World Bank Institute (WBI). The evaluation found that while most of the training reviewed resulted in demonstrable participant learning, this learning frequently did not lead to real change in participants' workplace performance. Poor training outcomes most often resulted from training content that wasn't relevant to the needs and goals of the target institutions, or from the trainees' lack of incentives or resources to apply learning in their workplaces. These findings highlight how important it is for training to be embedded in broader capacity-building programs that identify and address organizational and institutional capacity constraints alongside human ones.” Source : www.worldbank.org/ieg/training, Cost = billions of dollars $wastedeachyearworldwide
How to define the skills set of a position in order to measure, assess and develop them in a valid and reliable fashion ?
So … :
teachers and workplace trainers can easily reuse job skills set, as defined by employers, to design courses aimed at developing skills in a valid and effective fashion.
Hence enabling us as educational researchers, to become better at: Identifiyng skills to develop; Teaching skills; Measuring ans assessing skills.
2. Study of researchproblem Methodology 1. Problem observed within two Delphi sessions on ICT skills set definition www.ictc-ctic.ca. 2. Studied occupational skills Profile Models (frameworks). 3. Studied various skills typologies. 4. Draw specifications for a universal skills typology deemed universal (workplace and classroom).
What happened A facilitator asked experts of a workplace position (Intellectual Property Manager), to validate proposed set of accountabilities and skills, to precise them and to propose additional ones. Occupational levels Accountabilities Skills 2.1 Problemobservedwithin Delphi sessions The expert discuss until a consensus is reached
Observed accountabilities and skills through practical experience alone Job Intuitive observer Scope of essential accountabilities and skills for a job Accountabilities and skills which cannot be observed by practical experience alone 2.1 Problemobservedwithin Delphi sessions Job experts recalled their experience of the job activities and competencies only through their memory, hence inducing biases : interpretations, unbalanced weightings, emotive recalls, etc..
About the validity of the sessions It is fine for experts to recall past experiences and memories of a position to define accountabilities and skills. But : 1. How can they be confident they have covered all aspects? 2. How do they know they have not been too focused on certain aspects and omitted some? 3. Can they ensure their wording of competencies and skills can be reused by teachers in order to design courses developing and assessing exactly the competencies sought? How valid were the accountabilities and skills hence defined ? There are hence serious limitations stemming from the absence of an underlying conceptual framework in teaching and learning
ICTC Ancillary competencies Secondary competencies Core competencies 2.2 StudiedOccupationalSkills Profile Models (frameworks) Logical representation of factors contributing to understanding, developing and managing occupational skills. DACUM (Developing a Curriculum) Tasks, knowledge, behaviour, skills, tools and equipment Competency A Prepare meals Sub-competency A1 Buy food Sub-competency A2 Wash food Sub-competency A3 Cut food CSTD Training Competency Architecture Analyzing performance/training needs Designing training Facilitating/instructing Evaluating training Coaching the application of training
2.2 StudiedOccupationalSkills Profile Models (frameworks) Findings : 1. Mainly facts, information and tasks-based. 2. Most are groupings of competencies in homogeneous categories, sometimes in a sequence of visible tasks. 3. None based on any theories (educational, organizational, psychological). 4. Equivocal relation between activity, competency and skill : « a skill is the capacity to undertake an activity ». Circular definitions. 5. Wording of skills using a language’s syntax, no framework used. How one can be sure a skill is properly worded ? 6. Non-systemic, loosely structured frameworks. 7. None enable alignement of workplace and human capital resources.
2.3 Studiedvariousskills typologies Typology of skills typologies 1. Based on groups of skills applications (areas in which they are used)
Very loosely defined. Not grounded much in theories
No clear statement guidelines. Proned to interpretations.
Proned to mainly state visible behaviors (tasks)
Not integrated within a framework preventing inclusion of common skills traits
Is … similar to looking through capped binoculars … hence yielding researchers and educators to … Drawbacks of typologies of skills studied Educational researchers beware ! If skills typologies lack : 1. Grounds in psychological and educational theories; 2. An underlying framework/model depicting the functioning of a person; 3. Universal, objective, observable and measurable criteria to define specific skills …
Characteristics sought from an ideal skills framework 1. Skills are observable and measurable behaviors; 2. Skills cover all of a person’s set of observable and measurable behaviors (including internal dimensions affecting demonstration of behaviors) 3. Grounded on an integration of theoriespertaining to personal functioning and development; 4. Skills of the same type are grouped within functions; 5. Prescribes a clear, unequivocal structure to state skills; 6. Prescribes a clear, unequivocal structure to link skills with competencies, teaching models and measurement and assessment instruments; 7. Universal uses : personal development, interpersonal relationships, teaching learning and coaching within a classroom as well in workplaces with persons of all age ranges, allowing to align personal and professional development.
3. Selected a skillstypology Selected the Synergistic Person’s typology of skills (Racine, 2003)
Based on how a person functions (behave) and learn
Paradigm and theories underneath a proposed skills typologies Positivist paradigm putting a person’s functioning model at the core of the teaching and learning system (observable and measurable behaviors)
Functions-based process transforming inputs in outputs Inputs Outputs Imaginative Resources
Definition: Any learned human process deemed with a certain value acting on an object and generating a product (effect). Skill : Definition, structure and formula Structure A skill is composed of three elements: An object A process A product Formula Product (output is an observable and measurable behavior or deliverable generated by the act) Process (activities processing resources in outputs) Object (resource) Skill = + +
Typologies of skills of the Synergistic person Racine, S. (2003)
Skills : Examples I4 To solve (process) a problem (object) to generate a solution (product) A4 To organize (process) self around a value (object) in order to live by those values (product) P5 To convey a message (object)through speaking in public (process) in order to inform, entertain, convince or mobilize (product) C3 To apply (process) rules (object) in order to generate a deliverable expected by our role/position (product) S3 To help (process) customers satisfy their needs (object) in order they take decisions in their best interest (product)
A thoroughly validated typology of skills Validated with over 5,000 persons who completed a personal functioning test (PFT) on 7 occasions. (Racine, 1999, 2000, 2002, 2003, 2005, 2006, 2007) Validity based on : How well a respondent recognizes one’s PFT (over 97%) Content validity :
Skills’experts through consensus within a Delphi session
Measure of how well items are linked to a function
Construct validity : Through structural equation modeling, confirmatory factor analysis Diverging validity : Divergence between negative and positive items (forced choice items) Predictive validity : Regression indice Internal coherence = Internal fidelity : Homogeneity of test with sub-test Theoretical validity : link between items and taxons
Competence : Definition, structure and formula Definition
Socially acceptable human activity;
integrating various skills of various functions (imaginative, affective, physical, cognitive and social);
realised beyond a measurable performance threshold.
Skill of the A function = object –> process –> product Skill of the I function = object –> process –> product Skill of the P function = object –> process –> product Skill of the C function = object –> process –> product Skill of the S function = object –> process –> product Skill #6 Skill #5 Skill #4 Skill #1 Skill #3 Skill #2 Competence structure integration of skills of various functions in an action:
Competence formula Comp. = Σ weighted skills Comp. = (Iskill*weightI)+ (Askill * weightA) + (Pskill * weightP) + (Iskill * weightC) + (Iskill * weightS) Examples: I To know thyself (A2) in order to give a meaning to one’s life (I5) and a direction A To organize self (A4) and to explore (C4) with others means of living in harmony (S4) P To express verbally (P5) one’s emotions (A2) in order to willingly cooperate with others in solving relationship issues (S4) C To write a technical report according to industry’s guidelines (C3), which help target readers make sound decisions based on the findings (S3) S To help people (S3) find what they are looking for, through applying bureaucratic rules (C3) and showing them a hand-made picture (I3)
A definition of learning Learning is : A change in a person’s imaginative, affective, physical, cognitive and social functions; as aimed by the objectives (operational formulation of skills and competencies based on typologies); amplified by the teaching (application of the appropriate teaching models) and; noticed through using appropriate measurement and assessment instruments.
4. Action-researchundertook Goal: To design three competency-based university courses according to the competency profiles of the sought positions : R&D Manager, IP manager, Events organizer. Methodology followed 1. Identified experts for each position. 2. Devised questionnaires to draw competency profiles for the positions. 3. Analysed results, gathered experts for a Delphi session until consensus was reached. 4. Used the skills position profiles as exit profiles in the design of three university courses. 5. Delivered the courses. 6. Measured and assessed the developed skills.
5. Delphi sessions for three workplace positions
Process Corresponding job functions Person’s functions To conceive Imaginative To choose Affective To execute tasks Physical To assess :to process information Cognitive To communicate Social Feedback loops Synergistic Workplace System Model Every job transforms inputs into outputs through a process using 5 functions : Inputs
Defined a position competency profile Sent a survey questionnaire to experts of the positions. Synthesizedresults. Set-up a Delphi session to reach a consensus. Used a job-personmatching test to weigh I, A, P, C and S functionsdeemedmostdesirable in the position. Author:Serge Racine Available for free on : www.synergimax.com
… and those of an individual … skillsdeemeddesirable in the job… Results to the skills test Once skills set required are known for a job, skills of a person (candidate to the job or workeractuallyfilling the job), have to beassessed. Then, one has to assess gaps between… Coloredcellssuggest the level of skillsrequired in eachfunction, (based on the typology of skills) Result of the Job-Person Matching Test (JPMT) (consensus of experts)
Identification of skillsrequiringdevelopment Results of the JPMT : Skillssought in the job=> Skillsrequiringdevelopment=> Results to a skills test ; Skillsmastered by an individual=>
Tailoring of skills selected from typologies, to a role or a job Upon having identified, within the skills’typologies, the generic skills within the I, A, P, C and S functions which are deemed desirable for a particular role or job, one has to tailor them to a particular role or job. Application for a trainer’s role :
Functioning types deemedmostdesirable for the three positions surveyed
3 1 2 Skills typologies Models of teaching Measurement and assessment instruments Imaginative skills Affective skills Physical skills Cognitive skills Social skills Used the skills position profiles as exit profiles in the design of three university courses
Delivered the courses, measured and assessed the skills Allows to grade change in skills as a scorecard in Excel Kirkpatrick’s level #2
5. Applications of skills typologies Framework for any task aimed at measuring, assessing and developing any human organizational element: Personal Functioning Test Job-Person Matching Test Course design (choice of objectives, teaching models and measurement and assessment instruments) Course satisfaction assessment Competence and skills measurement and assessment Assessment of educational research Diagnostic of an organization Achitecturing information/content/template tools within organization’s databases
The use of the sameframeworkinsures congruent links Organization’s diagnostic Measurement and assessment ok keyperformance indicators: Functions : Direction, Management, Production, Assessmentand Sales Management dashboard, Prescription of enhancementactivities Job situationalassessment Competencies and skills profiles : Imaginative, Affective, Physical, Cognitive and Socialskillssought in a role or a job Competenciesdevelopmentprogram : Program, Course outlines, Pre-test, Educationalmaterial, Measurement and assessmentinstruments, post-test
Production of a formative and sommative evaluation
6. Furtherscientificresearchesrequired Furthervalidateskills by functionwithskills experts (Delphi) Furtherdefinestatement of skills by type (I, A, P, C and S) and provide application examples for variousroles and jobs Validatecorrespondingteachingmodels and measurement and assessment instruments Validatewith lots of positions (hundredsubjects per position)
7. Conclusion Skills typologies are an invaluable science-basedtool, an enablingtechnology, usedat the heart of humans and organization’sfunctioning. Theyallow us to: 1) Know more precisely how we behave; 2) Significantly learn better through more adapted teaching, thus becoming empowered; 3) Assign the right person in the right job in order to deliver the expected performance, in alignment with the organizational‘s objectives.
8. References BANDLER, R. GRINDER, J. (1982), Les secrets de la communication, Actualisation edh. D'HAINAUT, L. (1988), Des fins aux objectifs de l'éducation, Labor Nathan, Coll 2000. GORDON, W. J. (1987) The New Art of the Possible: The Basic Course in Synectics, Cambridge: Porpoise Books. HARROW, A. (1972), a Taxonomy of the Psychomotor Domain. McKay. KOHLBERG, L. (1969), Stage and sequence: The cognitive-developmental approach to socialization. KRATHWOHL, D.R. & al. (1964), Taxonomy of Educational Objectives. Handbook II: Affective Domain, McKay. RACINE, S., (2003), La personne synergique : théorie et applications, Ste-Foy : Septembre. RACINE, S., (1996), A dynamic and holistic model for formulating competence, choosing teaching strategies and evaluating synergic effects. In H.M. Kandarakis (Ed.) New Horizons in learning assessment: Conference Proceeding of the 21st annual meeting of the International Association for Educational Assessment. Montréal, Que: Université de Montréal). RACINE, S., (2007) Étude de validation du test de fonctionnement personnel, www.synergimax-international.com. TORRANCE, E. P. (1966), Torrance Tests of Creative Thinking. Normal Annual, Research Edition, Personnal Press.
About Claude Trépanier Professional services consultant in Human Capital Management, educational researcher, educational advisor, university lecturer in management and free-lance corporate trainer. Undergoing theoric and field researches along the organizational competencies development chain, from needs assessment to the assessment of competencies developed. Develops practical in-class and workplace practices and tools for the measurement, assessment and development of skills and competencies. Has delivered since 1998, more than 1 000 hours of courses with syllabus. Has developed and delivered since 2003 more than 15 competency-based courses, developed and used numerous performance assessment tools (skills assessment, teamwork’s diagnostic, project’s diagnostic, organization’s diagnostic). Electronic portfolio : http://eduportfolio.com/22421 Claude.firstname.lastname@example.orgPh. : 450-674-5904 (Canada)