Presentation done at the AEA annual meeting 2012. Minneapolis, October 26.

1. The « Concept mapping »
methodology: a review of users’
evaluative comments
Christian Dagenais, Ph.D.
Université de Montréal
Valéry Ridde, Ph.D.
Université de Montréal
Normand Péladeau, Ph.D.
Provalis Research
AEA annual meeting 2012
Minneapolis, October 26

2. The « Concept mapping »
methodology
• Developed by W. M. K. Trochim
• Based on the active participation of
stakeholders
• Process qualitative data using multivariate
statistical analysis (MDS & HCA)
• Presents results in graph format

3. Objective
Review advantages and limitations of the
“Concept Mapping” methodology identified in
published studies

4. Search strategy (in short)
• Studies published between 1989 and 2012
• Publication about Concept Mapping methodology
developped by W. Trochim
• General boolean query: "concept mapping" AND
("multidimensional scaling" OR "cluster
analysis")
• Sources
– Citation index: Web of Science
– Databases: ERIC, PsycINFO, MEDLINE, Social Work
Abstracts, Sociological Abstracts
– Search engine : Google Scholar
• 190 articles included
• Exclusion criteria: Trochim, Rosas or Kane as
authors

5. Intended use of Concept Mapping
60
50
40
30
Number of articles
20
10
0
Logic models Planification Outcome Needs assessment Concept definition Theory creation Instruments Other
development evaluation development

6. Evaluative comments formulated by
users
About Definition
Ressources needed Includes necessary time, money, equipment or human resources
necessary
Benefits for participants eg: sense of cohesion, belonging, discussion and sharing of opinions
and ideas
Process Understanding or appreciation of the participation process, logistics
and use of software
Results Interpretation, relevance, and usefulness of the results
Participants’ perspective Results reflect participants’ reality and are presented in their own
words
Methodological The usefulness of the method application, its flexibility, the choice of
research topic and/or participants
Statistical The validity or limitations of statistical procedures

7. Evaluative comments (a)
100
90
80
70
60
Positive
50
Negative
40
30
20
10
n=20 n=12 n=70 n=0 n=28 n=27 n=35 n=16 n=33 n=0
0
Ressources needed Benefits for participants Process Results Participants perspective

8. Evaluative comments (b)
Evaluative Comments (b)
100
90
80
70
60
50 Positive
Negative
40
30
20
10
0 n=30 n=28 n=3 n=9
Methodological Statistical

9. Evaluative comments made by the
users
About Definition Positive Negative
Benefits for participants eg: sense of cohesion, belonging, discussion and 70 (82) 0 (0)
sharing of opinions and ideas
Participants’ perspective Results reflect participants reality and are presented 33 (44) 0 (0)
in their own words
Ressources needed Includes time, money, equipment or human 20 (32) 12 (6)
resources necessary
Process Includes problems related to participation (fatigue, 28 (43) 27 (35)
understanding or appreciation of the process),
logistics and use of software
Results About the interpretation, relevance, usefulness of the 35 (54) 16 (19)
results and the ability to propose concrete actions
Generalizability The extension of results and conclusions from a CM 1 (1) 57 (64)
project to other population or the population at large
Methodological The ease of application of the method, its flexibility, 30 (46) 28 (29)
the choice of research topic or participants
Statistical The validity or limitations of statistical procedures 3 (5) 9 (11)

11. The CM Statistical Procedure

13. The CM Statistical Procedure

14. The CM Statistical Procedure

15. The CM Statistical Procedure

16. The CM Statistical Procedure

17. The CM Statistical Procedure

18. The The Statistical Procedure
CM CM Statistical Procedure

19. The Logical & Expert Argument

20. The Logical & Expert Argument

21. The Logical & Expert Argument

22. The Logical & Expert Argument

23. The Logical & Expert Argument

24. The Logical & Expert Argument

25. The Logical & Expert Argument
•
•

26. Are we really talking about
slight movements?

27. The Logical & Expert Argument

28. The Logical & Expert Argument

29. The Logical & Expert Argument

30. The Logical & Expert Argument

31. The Logical & Expert Argument

32. The Logical & Expert Argument

33. The Logical & Expert Argument

34. The Logical & Expert Argument

35. The Logical & Expert Argument

36. The Logical & Expert Argument

37. The Logical & Expert Argument
What Kruskal & Wish really suggest

38. The Mathematical Argument

39. The Mathematical Argument
Traditional CM (Concept Systems)
Computation of clustering solutions for 2 to 30 clusters.
Alternate Computation of Clusters
4. Computation of two indicators of “goodness of fit”
• Percentage of pairings represented by clusters
• Number of potentially misclassified statements

40. The Mathematical Argument

41. The Mathematical Argument

42. The Mathematical Argument

43. The Mathematical Argument

44. The Mathematical Argument

45. The Mathematical Argument
Defining misclassified statements

46. The Mathematical Argument

47. The Mathematical Argument

48. The Mathematical Argument

49. The Mathematical Argument

50. The Mathematical Argument

51. The Mathematical Argument

52. The Social Validity Argument

53. The Social Validity Argument

54. The Social Validity Argument

55. The Social Validity Argument

56. The Social Validity Argument

57. The Social Validity Argument

58. The Social Validity Argument
Social Validation Procedure
Three studies
Dagenais & Hackett (2008) – Literacy: 12 items in 9 clusters
Jean et al (2007) – Rural Living: 8 items in 8 clusters
Kane & Trochim (2007) – Non profit: 17 items in 8 clusters
Subjects
34 graduate students – University of Montreal
(Psychology & Social Medicine)

59. The Social Validity Argument

60. The Social Validity Argument

61. •
•
•
•

62. Alternate Graphical Approaches
Carter, Chicca Enyedy, Goodyear & Arcinue (2009).
Concept mapping of the events supervisees find
helpful in group supervision. Training and Education
in Professional Psychology, 3 (1), 1-9.

63. Alternate Graphical Approaches

64. Alternate Graphical Approaches

65. Alternate Graphical Approaches

66. Alternate Graphical Approaches

67. Alternate Graphical Approaches
Alternate Graphical Approaches

68. Thank you!