Fluency with Information Technology: From rubric to Assessment


Published on

Presentation given by Gail Matthews-DeNatale and Bruce Tis to the faculty of Simmons College.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Fluency with Information Technology: From rubric to Assessment

  1. 1. From Rubric to Assessment Fluency with Information Technology 2/18/2005 Gail Matthews-DeNatale, Ph.D. Senior Instructional Designer, Simmons College Bruce Tis, Ph.D. Computer Science and Information Technology FIT Task Force Chair, Simmons College
  2. 2. A Liberal Arts Approach to FIT <ul><li>TECHNOLOGY </li></ul><ul><li>Computer Science </li></ul><ul><li>Telecommunications </li></ul>Original FIT domain as defined by NAS * * National Academy of Sciences
  3. 3. A Liberal Arts Approach to FIT <ul><li>TECHNOLOGY </li></ul><ul><li>Computer Science </li></ul><ul><li>Telecommunications </li></ul><ul><li>INFORMATION LITERACY </li></ul><ul><li>Library Science </li></ul><ul><li>Research </li></ul>Yet more and more information is “born/found digital”
  4. 4. A Liberal Arts Approach to FIT <ul><li>TECHNOLOGY </li></ul><ul><li>Computer Science </li></ul><ul><li>Telecommunications </li></ul><ul><li>INFORMATION LITERACY </li></ul><ul><li>Library Science </li></ul><ul><li>Research </li></ul><ul><li>MEDIA LITERACY </li></ul><ul><li>Semiotics / Linguistics </li></ul><ul><li>Art / Media Studies </li></ul><ul><li>Communications </li></ul>Another layer: understanding multimedia
  5. 5. A Liberal Arts Approach to FIT <ul><li>TECHNOLOGY </li></ul><ul><li>Computer Science </li></ul><ul><li>Telecommunications </li></ul><ul><li>INFORMATION LITERACY </li></ul><ul><li>Library Science </li></ul><ul><li>Research </li></ul><ul><li>MEDIA LITERACY </li></ul><ul><li>Semiotics / Linguistics </li></ul><ul><li>Art / Media Studies </li></ul><ul><li>Communications </li></ul><ul><li>Philosophy / Ethics </li></ul><ul><li>Cultural Studies </li></ul><ul><li>Anthropology </li></ul><ul><li>Sociology </li></ul><ul><li>Education </li></ul>… and much more
  6. 6. A Liberal Arts Approach to FIT <ul><li>TECHNOLOGY </li></ul><ul><li>Computer Science </li></ul><ul><li>Telecommunications </li></ul><ul><li>INFORMATION LITERACY </li></ul><ul><li>Library Science </li></ul><ul><li>Research </li></ul><ul><li>MEDIA LITERACY </li></ul><ul><li>Semiotics / Linguistics </li></ul><ul><li>Art / Media Studies </li></ul><ul><li>Communications </li></ul>FIT <ul><li>Philosophy / Ethics </li></ul><ul><li>Cultural Studies </li></ul><ul><li>Anthropology </li></ul><ul><li>Sociology </li></ul><ul><li>Education </li></ul>
  7. 7. Implications for Assessment <ul><li>Need to assess three dimensions (technology, information, and media literacy) </li></ul><ul><li>No off-the-shelf assessment filled the bill </li></ul><ul><li>Composed questions and drew from: </li></ul><ul><ul><li>IT Fluency: NAS / Washington State University </li></ul></ul><ul><ul><li>Information Literacy: California State University </li></ul></ul><ul><ul><li>Media Literacy: Media Awareness Network (CA) </li></ul></ul>
  8. 8. The Baseline Assessment <ul><li>Part I: Background Information </li></ul><ul><ul><li>Demographics </li></ul></ul><ul><ul><li>Access to computers </li></ul></ul><ul><ul><li>Experience with computers </li></ul></ul><ul><ul><li>Technology skills (self-assessed, scale of 1-10) </li></ul></ul><ul><li>Part II: Fluency (Developed out of Rubric) </li></ul><ul><ul><li>Emphasis on critical thinking, problem-solving, and ethical reasoning </li></ul></ul>
  9. 9. <ul><li>The Results </li></ul>
  10. 10. Ample Access at Home … Mean = 2.5 SD = 1.5
  11. 11. … and at / for School 96% of incoming students own a computer that can be used for schoolwork This includes those who live off campus
  12. 12. 97% of students have internet access at home
  13. 13. 98% of incoming students had email before Simmons
  14. 14. Access and Familiarity ≠ Fluency <ul><li>Despite considerable access to computers and the Internet, most incoming students are poorly prepared to live, learn, and work in a technology-rich, multimedia saturated society. </li></ul><ul><li>Why? </li></ul>
  15. 15. Social vs. Intellectual / Academic <ul><li>Prior use of technology </li></ul><ul><li>Prior experience with technical problem-solving </li></ul><ul><li>Has been largely social and/or informal </li></ul><ul><li>Example: </li></ul><ul><ul><li>Q: What do you do when you have a computer Problem? </li></ul></ul><ul><ul><li>A: 96% “Ask a family member or friend for help” </li></ul></ul>
  16. 16. What’s wrong with that? <ul><li>Basic technical understanding Example: How email works </li></ul><ul><li>Only 41% the students surveyed knew how email messages get sent </li></ul><ul><li>Lays a foundation for more sophisticated thinking Example: Distinguishing between valid and fraudulent email </li></ul><ul><li>Only 13% of the students surveyed knew how to best deal with a fraudulent email message requesting personal bank account information </li></ul>
  17. 17. Discrepancy Between Self-Assessment ... Mean = 5.6 SD = 1.8
  18. 18. and Actual Insight, Understanding, <ul><li>Example: </li></ul><ul><li>Students who believed they were highly proficient were less likely to say that there were &quot;legal and ethical issues to consider&quot; in: </li></ul><ul><ul><li>Copying and pasting extended passages from a web site into a term paper; and </li></ul></ul><ul><ul><li>Buying software and letting your friends install it on their computers. </li></ul></ul>
  19. 19. Ability, and … <ul><li>Example: </li></ul><ul><li>Q: &quot;You’re interested in career opportunities in nursing. What would you type into the Web search engine to find the most information?“ </li></ul><ul><li>A: Only 11% selected the correct response </li></ul>
  20. 20. Awareness <ul><li>Example: </li></ul><ul><li>33% of the students surveyed </li></ul><ul><li>did not know that computer viruses </li></ul><ul><li>spread through files on disk and on CDs </li></ul>
  21. 21. Their Greatest Difficulty?
  22. 22. Integrating Multiple Dimensions of Analysis Only 24% selected the most credible site for appropriate reasons 24% 22% 49% 29%
  23. 23. Incoming Student Misconceptions <ul><li>has a .org domain name (more credible than .gov) </li></ul><ul><li>has lots of “information” / links </li></ul><ul><li>has a “shorter,” “easy,” “straightforward” URL </li></ul><ul><li>has a “professional “look” </li></ul><ul><li>includes information on how to “send feedback” and/or contact the webmaster/designer </li></ul><ul><li>“ comes from the United States” </li></ul><ul><li>“ claims to be unbiased” </li></ul><ul><li>is “official,” or has a name that “sounds official” (such as “Centers” and “Organizations”) </li></ul><ul><li>has an online store </li></ul><ul><li>links to other “well-known organizations” </li></ul><ul><li>includes advertising </li></ul><ul><li>includes a search option </li></ul><ul><li>includes press releases </li></ul>A web site is more likely to be credible if it
  24. 24. In a Nutshell <ul><li>Incoming students need the most help with: </li></ul><ul><li>problem diagnosis and solving; </li></ul><ul><li>media literacy (as consumers, researchers, and authors); and </li></ul><ul><li>grappling with ethical challenges. </li></ul><ul><li>In the same way that incoming students need to </li></ul><ul><li>learn systematic processes for writing and research, </li></ul><ul><li>they also need to learn how to work, think, create, </li></ul><ul><li>and problem-solve systematically with technology. </li></ul>
  25. 25. FIT Breakout Groups <ul><li>Divide into groups (4-5 faculty members per group). Please take notes – you will report back to all. Each group has the FIT rubric, baseline assessment with explanations, and tabulated results. </li></ul><ul><li>Given the volume of information before you, you may decide that you need to focus on one issue or theme. That’s okay. Considering the rubric and assessment information, discuss the following questions: </li></ul><ul><li>What do you think are the most critical learning needs of first year students? </li></ul><ul><li>What are you already doing to address these learning needs? How could you be more explicit in connecting what you're already doing with the rubric? </li></ul><ul><li>What's one thing you'd like to add or change in your courses? </li></ul>