The « Concept mapping » methodology: a review of users’ evaluative comments. A Case of Misconceived Mapping ?
The « Concept mapping »methodology: a review of users’ evaluative comments Christian Dagenais, Ph.D. Université de Montréal Valéry Ridde, Ph.D. Université de Montréal Normand Péladeau, Ph.D. Provalis Research AEA annual meeting 2012 Minneapolis, October 26
The « Concept mapping » methodology• Developed by W. M. K. Trochim• Based on the active participation of stakeholders• Process qualitative data using multivariate statistical analysis (MDS & HCA)• Presents results in graph format
Objective Review advantages and limitations of the“Concept Mapping” methodology identified in published studies
Search strategy (in short)• Studies published between 1989 and 2012• Publication about Concept Mapping methodology developped by W. Trochim• General boolean query: "concept mapping" AND ("multidimensional scaling" OR "cluster analysis")• Sources – Citation index: Web of Science – Databases: ERIC, PsycINFO, MEDLINE, Social Work Abstracts, Sociological Abstracts – Search engine : Google Scholar• 190 articles included• Exclusion criteria: Trochim, Rosas or Kane as authors
Intended use of Concept Mapping 60 50 40 30Number of articles 20 10 0 Logic models Planification Outcome Needs assessment Concept definition Theory creation Instruments Other development evaluation development
Evaluative comments formulated by users About DefinitionRessources needed Includes necessary time, money, equipment or human resources necessaryBenefits for participants eg: sense of cohesion, belonging, discussion and sharing of opinions and ideasProcess Understanding or appreciation of the participation process, logistics and use of softwareResults Interpretation, relevance, and usefulness of the resultsParticipants’ perspective Results reflect participants’ reality and are presented in their own wordsMethodological The usefulness of the method application, its flexibility, the choice of research topic and/or participantsStatistical The validity or limitations of statistical procedures
Evaluative comments made by the users About Definition Positive NegativeBenefits for participants eg: sense of cohesion, belonging, discussion and 70 (82) 0 (0) sharing of opinions and ideasParticipants’ perspective Results reflect participants reality and are presented 33 (44) 0 (0) in their own wordsRessources needed Includes time, money, equipment or human 20 (32) 12 (6) resources necessaryProcess Includes problems related to participation (fatigue, 28 (43) 27 (35) understanding or appreciation of the process), logistics and use of softwareResults About the interpretation, relevance, usefulness of the 35 (54) 16 (19) results and the ability to propose concrete actionsGeneralizability The extension of results and conclusions from a CM 1 (1) 57 (64) project to other population or the population at largeMethodological The ease of application of the method, its flexibility, 30 (46) 28 (29) the choice of research topic or participantsStatistical The validity or limitations of statistical procedures 3 (5) 9 (11)
The Mathematical ArgumentTraditional CM (Concept Systems) Computation of clustering solutions for 2 to 30 clusters.Alternate Computation of Clusters4. Computation of two indicators of “goodness of fit” • Percentage of pairings represented by clusters • Number of potentially misclassified statements
The Social Validity ArgumentSocial Validation ProcedureThree studies Dagenais & Hackett (2008) – Literacy: 12 items in 9 clusters Jean et al (2007) – Rural Living: 8 items in 8 clusters Kane & Trochim (2007) – Non profit: 17 items in 8 clustersSubjects 34 graduate students – University of Montreal (Psychology & Social Medicine)
Alternate Graphical Approaches Carter, Chicca Enyedy, Goodyear & Arcinue (2009). Concept mapping of the events supervisees find helpful in group supervision. Training and Education in Professional Psychology, 3 (1), 1-9.