Digital competence assessment


Published on

Published in: Education, Technology

Digital competence assessment

  1. 1. Digital Competence in K-12.Theoretical Models, Assessment Tools and Empirical Research by Maria Ranieri, Antonio Calvani, Antonio Fini Department of Education University of Florence Italy Lisboa, 28th June 2011
  2. 2. TOC• Context and objectives• Definitions of digital literacy/competence• Assessment tools• Applications and results• Conclusion
  3. 3. Context and aimsOver the last years the theme of the digitalcompetence in its different aspects has been objectof a growing interest. In a number of officialdocuments and communications, internationalbodies underlined the significance of thiscompetence for lifelong learning and to participate inthe so-called ‘information society’. See for example the new framework of key competences of European Union (December, 2006)
  4. 4. Context and aimsUp to now, there exist various internationalcertifications for basic IT skills, but they emphasize theassessment of the very technical knowledge andskills, overlooking the cognitive and educationalaspects of the use of ICT. The aim of DCA project was to elaborate a conceptual model for digital literacy emphasizing cognitive and educational aspects, and develop relatve assessment tools easy-to-use for teachers and educators.
  5. 5. Up to now, there exist various internationalcertifications for basic IT skills, but they emphasize theassessment of the very technical knowledge andskills, overlooking the cognitive and educationalaspects of the use of ICT. The aim of DCA project was to elaborate a conceptual model for digital literacy emphasizing cognitive and educational aspects, and develop relatve assessment tools easy-to-use for teachers and educators.
  6. 6. What does DigLit orDigCom mean?
  7. 7. Literature Review• Digital literacy is «the ability to understand and useinformation in multiple formats from a wide range of sourceswhen it is presented via computers» (Gilster, 1997)• «Digital literacy merges capabilities: purely technical aspects,intellectual competences and also competences related toresponsible citizenship. They all allow individuals to developthemselves completely in information society»(Tornero, 2004)
  8. 8. Literature Review• Digital literacy as «the awareness, attitude and ability ofindividuals to appropriately use digital tools and facilities toidentify, access, manage, integrate, evaluate, analyze andsynthesize digital resources, construct new knowledge, createmedia expressions, and communicate with others, in thecontext of specific life situations, in order to enable constructivesocial action; and to reflect upon this process» (Martin, 2005).
  9. 9. Literature Review• Digital and media literacy is defined as the ability to:(1) make responsible choices and access information […];(2) analyze messages in a variety of forms by identifying the author, purpose and point of view and evaluating the quality and credibility of the content;(3) create content in a variety of forms for authentic purposes […];(4) reflect on one’s own conduct and communication behavior by applying social responsibility and ethical principles;(5) take social action by working individually and collaboratively to share knowledge and solve problems in a variety of contexts (Hobbs, 2010).
  10. 10. Literature ReviewThere is agreement among researchers that different types ofliteracies related to ICTs and more generally to the media, allare connected to the concept of digital literacy, together withother life skills. IT Literacy Inquiry skills Visual LiteracyCritical thinking skills Digital Literacy Collaborative skills Information Literacy Problem solving skills Media Literacy
  11. 11. The concept of Digital Competence is:Multidimensional - Implies the integration between abilities andskills of cognitive, relational and social nature.Complex - Is not completely quantifiable with single tests and notalways verifiable in the short run. At least for some aspects itrequires more time and different contexts before coming to surface.Interconnected - Is not independent from other keycompetences like reading, numeracy, problem solving, inferentialskills etc.Sensitive to the socio-cultural context – The meaning of thisconcept could change over time, according to the context and thevarious educational settings.
  12. 12. Defining DigCompDigital competence consists in being able• to explore and face new technological situations in aflexible way,• to analyze, select and critically evaluate data andinformation, to exploit technological potentials in order torepresent and solve problems and build shared andcollaborative knowledge,• while fostering awareness of one’s own personalresponsibilities and the respect of reciprocalrights/obligations.
  13. 13. Exploring new technological contexts in a flexible way Access, selectionInteracting through ICTs TECHNOLOGICAL and critical evaluation in a responsible way of information INTEGRATED ETHICAL COGNITIVE Understanding the potential of networking technologies for collaborative knowledge building
  14. 14. Assessing digital competence:What instruments?
  15. 15. As a DCA research group, we developed a set of instrumentsenabling teachers to assess digital competence at differentlevels and in the short/medium run. Situated DCA Instant DCA
  16. 16. Instant DCAIt is conceived as a rapid means of assessment and consistsin quantitative tests.The tests are made up of 35 items, with multiple-choiceanswers or matching items, with a small group of shortquestions and answers.
  17. 17. Situated DCAIt is intended to be used as an instrument for a more meaningfulassessment in the short/medium run and consists in fourtypologies of tests.• The first type (Technological exploration) presents anunknown technological interface which has to be learned andmastered.• The second type (Simulation) requires the experimentalelaboration of data by formulating hypotheses on the possiblerelationships.
  18. 18. Situated DCA• The third type (Inquiry) requires the critical selection andgathering of relevant and reliable information about a predefinedsubject.• The fourth type (Collaboration) requires the participation in thecollaborative production of a document, while respecting criteriaregulating collaborative activities.
  19. 19. Let’s focus on the Instant DCA
  20. 20. The Instant DCA is made up of three sections: technological, cognitive and ethical.The technological section does not focus on computer structure,hardware and software, but on the understanding of common situations,like the ones people can meet in everyday life and/or working directlyat the computer. Technologies in everyday life
  21. 21. The cognitive section can be identified with the Information Literacy(i.e., an ability in processing, managing and reading, together with thecritical evaluation of texts, data, information and sources).The items in this section need general cognitive skills from the students(i.e., mostly linguistic or logic-linguistic competences), to be solved. Critical evaluation of information
  22. 22. The ethical section concerns adequate behaviours needed in the use oftechnologies and more generally on the Internet. It relates to theethical and social consequences of technological behaviours.The section is made up of three sub-sections focused on: protectingpersonal data, respecting other Net-users and being aware of thedigital gap. Netiquette, Social Consequences of Digital Gap, Protecting Personal Data
  23. 23. iDCA applications in school
  24. 24. iDCA SurveysThe validation test and the Italy-China Survey • The first version of the tests, designed for adolescents (14-16 year-old high school students), was made up of 87 items. • It was administered in Italy (220 students) and in China (330). • The results of the item analysis was satisfactory: -Cronbach’s alpha coefficient Chinese DCA = 0.77 -Cronbach’s alpha coefficient Italian DCA = 0.79 This led to the identification of the 35 items used in the third stage of the experimentation, intended for Italian high schools and which took place between September 2009 and January 2010.
  25. 25. iDCA SurveysThe wide Italian survey• In the period September 2009-January 2010 the iDCA testwas administered to a sample of students from Italiansecondary schools with the aim at assessing the “state” ofdigital competence among Italian students.• Total number of respondents was 1056 from 34 schools andthe average age of the students was 15.• Examinining the overall scores, the results are on the wholelower than we expected. The average score was in fact 62,5(DEV.ST=15,8), which is less than the 75% threshold, andonly one fourth of the students reaches the 75% threshold
  26. 26. Conclusions• Although there is great emphasis today on teens’ digitalskills, evidence seems to say something different…• School should play a fundamental role in supporting a morecritical and significant use of technologies• School should guarantee the integration of the notions andtechnological abilities which adolescents could acquirespontaneously through their own practices, within a morearticulate cognitive framework, adequately related to othersignificant competences.
  27. 27. Related publications• Calvani, A., Fini, A. & Ranieri, M. (2010), La competenza digitale nella scuolaModelli e strumenti per valutarla e svilupparla, Trento, Erickson.• Calvani, A., Fini, A. & Ranieri, M. (2009), Assessing Digital Competence inSecondary Education. Issues, Models and Instruments, in M. Leaning (Ed.),Issues in Information and Media Literacy: Education, Practice and Pedagogy,Santa Rosa, California: Informing Science Press, 153-172.• Y. Li, M. Ranieri, Are ‘digital natives’ really digitally competent? A Study onChinese Teenagers, Paper accepted by the British Journal of EducationalTechnology, UK.
  28. 28. The get more information on the project please visit the web site: Thank you
  29. 29. CreditsThe DCA research project was funded bythe Italian Ministry of Education withinthe PRIN Research Programme 2006-2007and is led by Prof. Antonio Calvani(University of Florence).