Apresentação oral de trabalho desenvolvido pelo Grupo de Pesquisa Mapas Conceituais durante a 7th International Conference on Concept Mapping (cmc2016) em Tallinn/Estônia no dia 06/09/2016

1. Cmaps with Errors: Why not?
Comparing Two Cmap-Based Assessment
Tasks to Evaluate Conceptual
Understanding
Paulo Correia, Gisele Cabral & Joana Aguiar
prmc@usp.br
06.09.16

2. Teaching & learning
Assessment plays a key role
Information exchange between teacher & students
Checkpoint to regulate the process
Alignment of expectations and duties
Special moment requires preparation

3. Cmaps are a great alternative
Knowledge structures become visible
Feedback can easily address learning obstacles
Continuous support to meaningful learning
Constructivist atmosphere reaches the classroom
Pedagogic resonance (the bridge between teacher
knowledge and student learning) is fostered

4. Reality
Requirements for systemic use of Cmaps
The preparedness of teachers
The motivation of students
Conditions in which teachers and students
communicate effectively as partners
… and the development of straightforward tasks
If concept mapping is so helpful to learning
biology, why aren’t we all doing it?
Kinchin, I. M. (2011). International Journal of Science Education, 23(2), 1257-1269.

5. Who is the mapper?

6. Aims
Explore Cmap with errors under classroom
conditions
Two assessment tasks
Find the errors
Judge the selected propositions
Two topics
Different level of difficulty (students’ background)

7. Context
Natural Science Course
Period of study: class #11 to #14 (100 min each)
Topics
Creationism vs Evolutionism |  background
Molecular biology |  background
Formal assessment: class #15
Questionnaire
Cmap with errors

8. Participants & Groups
110 1st-year undergraduate students
2 X 2 (Assessment task format X Topic)
quasi-experimental design

9. C=19 | P=24 | E=8 | J=6E+4C

10. C=26 | P=32 | E=9 | J=7E+4C

11. Data collection
Answer the questionnaire
10 statements abour Creationism vs Evolutionism
10 statements about Molecular Biology
10 min | 1pt for each correct answer (max. 10pt)

12. Answer the Cmap task (Find or Judge)
20 min | students marked the time on task (max. 10pt)
Data collection

13. Student’s performance
Find the errors
𝑷 =
𝑰𝒅𝒆𝒏𝒕𝒊𝒇𝒊𝒆𝒅 𝒆𝒓𝒓𝒐𝒓𝒔
𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒆𝒓𝒓𝒐𝒓𝒔
−
𝑾𝒓𝒐𝒏𝒈 𝒊𝒅𝒆𝒏𝒕𝒊𝒇𝒊𝒄𝒂𝒕𝒊𝒐𝒏𝒔
𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒄𝒐𝒓𝒓𝒆𝒄𝒕 𝒑𝒓𝒐𝒑
Example
20 propositions | 5 errors | 4 found | 2 misjudgments
𝑷 =
𝟒
𝟓
−
𝟐
𝟏𝟓
× 𝟏𝟎 = 𝟔. 𝟔𝟕

14. Student’s performance
Judge the selected propositions
𝑷 =
𝑪𝒐𝒓𝒓𝒆𝒄𝒕 𝒋𝒖𝒅𝒈𝒆𝒎𝒆𝒏𝒕𝒔
𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒑𝒓𝒐𝒑 𝒕𝒐 𝒃𝒆 𝒋𝒖𝒅𝒈𝒆
× 𝟏𝟎
Example
7 propositions to be judge | 4 correct judgements
𝑷 =
𝟒
𝟕
× 𝟏𝟎 = 𝟓. 𝟕𝟏

15. Data analysis
Two-way ANOVA considering
Questionnaire performance (max. 10pt)
Time on Cmap task (max. 20min)
Cmap task performance (max. 10pt)
Explored comparisons
Different Cmap tasks within the same topic
Different topics within the same Cmap task

16. Results
Main effect for the topic
Students’ background are different (CrEv > MoBi)
CrEv: revise & refine their previous schemas
MoBi: conceptual assimilation & schema construction
No effect for the Cmap task format

17. Results
No effect for the topic
No effect for the Cmap task format
Average time in task: 13min
Lower than the time to create & revise a Cmap
(~60min)
Difficult conceptual relationships were not avoided

18. Results
Main effect for the Cmap task format
Judge groups presented better performances
Only Find task penalized type 1 error (false positive)
No effect for the topic

19. Task X Topic interaction
BG
CrEv
BG
MoBi

20. Level of Difficulty
Adjustments from students’ right answers (RA)
Very easy 𝑹𝑨 ≥ 𝟖𝟎%
Easy 𝟔𝟎% ≤ 𝑹𝑨 ≥ 𝟖𝟎%
Moderate 𝟒𝟎% ≤ 𝑹𝑨 ≥ 𝟔𝟎%
Difficult 𝟐𝟎% ≤ 𝑹𝑨 ≥ 𝟒𝟎%
Very difficult 𝑹𝑨 < 𝟐𝟎%

21. Examples from CrEv

22. Learning Obstacle
M
D
D
E

23. Learning Obstacle
M
D
E

24. Conclusions
Cmap with errors is useful for learning
assessment
1. Short time to administer
2. Inclusion of difficult conceptual relationships
3. At least two task formats
4. Ease adjustment of difficult level
5. Short time to correct and provide precise feedback

25. Automated process
cmc2018?