Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Assessment

48 views

Published on

Peer-assessment & Data for CFRIDiL

Published in: Education
  • Be the first to comment

  • Be the first to like this

Assessment

  1. 1. Common Framework of Reference for Intercultural Digital Literacies ▶ DATA: › Baseline surveys › Evaluation surveys › Students’ productions › Students’ course evaluations › Summative assessment and peer-assessment
  2. 2. Quantitative analysis Baseline survey Evaluation survey Teacher Assessment Form Peer Assessment Form Search for correlations between items of different questionnaires Analysis per item/ per text-type/ per course
  3. 3. Number of Students per Course 46, 21% 87, 41% 30, 14% 21, 10% 30, 14% EU_Rome EU_Aarhus EU_Florence EU_Messina EU_Leeds
  4. 4. Number of Students per Text- Type 17, 8% 49, 23% 32, 15% 21, 10% 95, 44% Corporate Videos About us page Fanvids Video mediated interaction Weblogs
  5. 5. Ss’ evaluation of the module 5% 15% 28% 38% 15% 0% 5% 10% 15% 20% 25% 30% 35% 40% 1 2 3 4 5 1. Rate the module overall from 1 (poor) to 5 (excellent)
  6. 6. Ss’ evaluation of the material 3% 10% 36% 38% 12% 1 2 3 4 5 2. Rate the usefulness of handouts, references and readings from 1 (poor) to 5 (excellent)
  7. 7. Ss’ evaluation of the core part of the module 3% 16% 31% 34% 16% 1 2 3 4 5 3. Rate the usefulness of the core part of the module from 1 (not useful) to 5 (very useful)
  8. 8. Ss’ evaluation of the core part of the module 3% 12% 29% 40% 16% 1 2 3 4 5 4. Rate the quality of the core part of the module from 1 (poor quality) to 5 (high quality)
  9. 9. Rating how much one’s ability in designing/producing a digital text has improved 5.30% 5.10% 0.00% 0.00% 15.00% 10.50% 7.70% 10.70% 7.10% 0.00% 30.30% 30.80% 25.00% 42.90% 20.00% 36.80% 46.20% 50.00% 28.60% 35.00% 17.10% 10.30% 14.30% 21.40% 30.00% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Weblogs About us page Fanvids Promotional Videos Video mediated interactions no improvement little improvement good improvement very good improvement outstanding improvement
  10. 10. Rating how much one’s ability in interpreting/analysing a digital text has improved 1.30% 0.00% 3.60% 0.00% 5.00% 6.60% 12.80% 7.10% 0.00% 5.00% 22.40% 33.30% 14.30% 21.40% 10.00% 51.30% 30.80% 53.60% 57.10% 40.00% 18.40% 23.10% 21.40% 21.40% 40.00% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Weblogs About us page Fanvids Promotional Videos Video mediated interaction no improvement little improvement good improvement very good improvement outstanding improvement
  11. 11. Assignment 1: Production of Digital Text GENERAL CRITERIA 1.1 MULTIMODAL ORCHESTRATION 1.2 DIGITAL TECHNOLOGIES 1.3 INTERCULTURAL COMMUNICATION
  12. 12. Text-type Specific Assessment Criteria WEBLOGS ABOUT US PAGE PROMOTIONAL VIDEO FAN VIDEO VIDEO MEDIATED INTERACTIONS 1.4 Layout 1.4 Web writing techniques 1.4 Structure 1.4 Structure 1.4 Transcription 1.5 Colour 1.5 Visual resources 1.5 Image 1.5 Image 1.5 Linearization 1.6 Font 1.6 Structure 1.6 Written text and/or speech 1.6 Written text 1.6 Annotation 1.7 Image 1.7 Self-branding process 1.7 Sounds and/or music 1.7 Sounds and/or music 1.7 Balance 1.8 Writing 1.8 Informativity 1.8 Informing, advertising and entertaining 1.8 Resemiotization 1.8 Choice of segment to transcribe 1.9 Interactivity 1.9 Usability 1.9 Design 1.9 Spontaneity
  13. 13. Correlation between Teacher Assessment and Peer Assessment Marks (1.1-1.3 on the production of a digital text, 2.1-2.6 on the analysis of the digital text) PA 1.1 – T.A. 1.1 (ChiSquare = 65.118, df = 15, sig. (2-sided) = .000) TA_1_1 Total A B C D PA_1_1 A 36 21 5 0 62 B 34 42 7 1 84 C 14 21 11 0 46 D 2 6 1 2 11 E 0 4 2 0 6 F 0 1 0 1 2 Total 86 95 26 4 211
  14. 14. Peer-Assessment vs Teacher-Assessment Item 1.1 : Assessing Multimodal Orchestration F 1% E 3% D 5% C 22% B 40% A 29% Peer Assessment Item 1.1 F 0% E 0% D 2% C 12% B 45% A 41% Teacher Assessment Item 1.1
  15. 15. Correlation between PA and TA items 1.4-1.9 for Weblogs Example: PA 1.4 – T.A. 1.4 (ChiSquare = 53.812, df = 20, sig. (2- sided) = .000). TA_1_4 Total A B C D PA_1_4 A 12 15 5 0 32 B 8 18 9 1 36 C 4 9 8 0 21 D 1 1 1 0 3 E 1 1 0 0 2 F 0 0 0 1 1 Total 26 44 23 2 95
  16. 16. Peer-Assessment vs Teacher-Assessment Item 1.4 for Weblogs: Assessing Layout 1.10% 2.10% 3.20% 22.10% 37.90% 33.70% 0.00% 0.00% 2.10% 24.20% 46.30% 27.40% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% 50.00% F E D C B A Peer Assessment Teacher Assessment
  17. 17. ASSIGNMENT 1 Production of Digital Text 0% 0% 1%2% 3% 2% 12% 15% 7% 45% 42% 39% 41% 39% 51% 0% 10% 20% 30% 40% 50% 60% 1.1 Multimodal Orchestration 1.2 Digital literacy 1.3 Intercultural communication E D C B A
  18. 18. Assignment 2: Analysis of Digital Text 2.1 STRUCTURE 2.2 RESOURCES 2.3 TERMINOLOGY 2.4 COMMAND OF ENGLISH 2.5 REFERENCES 2.6 ANALYTICAL SKILLS
  19. 19. Correlation between Peer Assessment and Teacher Assessment items 2.1-2.6 Example: PA 2.1 – T.A. 2.1 (ChiSquare = 83.907, df = 20, sig. (2-sided) = .000). TA_2_1 Total A B C D E PA_2_1 A 35 23 11 1 0 70 B 24 26 11 0 0 61 C 11 13 23 1 0 48 D 5 4 7 2 0 18 E 1 2 7 0 1 11 F 1 0 1 0 1 3 Total 77 68 60 4 2 211
  20. 20. Peer-Assessment vs Teacher-Assessment Item 2.1 : Assessing Structure of Digital Text Analysis 1.40% 5.20% 8.50% 22.70% 28.90% 33.20% 0.00% 0.90% 1.90% 28.00% 32.70% 36.40% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% F E D C B A Peer Assessment Teacher Assessment
  21. 21. ASSIGNMENT 2 Analysis of Digital Text 0% 0% 0% 0% 0% 0%1% 1% 1% 0% 7% 0% 2% 1% 3% 6% 9% 2% 28% 19% 25% 29% 32% 25% 33% 44% 43% 45% 32% 44% 36% 34% 28% 21% 20% 28% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 2.1 Structure 2.2 Resources 2.3 Terminology2.4 Command of English 2.5 References 2.6 Analytical skills F E D C B A
  22. 22. ASSIGNMENT 3 Peer-Assessment 0% 0% 0%0% 0% 1% 4% 2% 7% 21% 19% 20% 36% 43% 35% 38% 35% 38% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 3.1 Argumentation 3.2 Consistency 3.3 Constructive feedback AxisTitle F E D C B A
  23. 23. Ss’ evaluation of the usefulness of the project 55% 67% 44% 44% 25% 3% theoretical understanding of multimodality practical digital skills use and design of texts outside university/ college ability in assessing other people's work improving chances to get a job other 27. For which purpose among the following do you think that this project is, if any, useful?

×