What is the  iSkills™  assessment? Bill Wynne ETS Higher Education Assessment Specialist Dr. Irvin Katz ETS Senior Researc...
Overview Spotlight on Digital Fluency A Framework for Digital Fluency Assessing Digital Fluency The Myths about Digital Fl...
Spotlight on Digital Fluency <ul><li>“ I’m calling on our nation’s governors and state education chiefs to develop standar...
Spotlight on Digital Fluency <ul><li>“ Information and communication literacy is redefining what we mean by basic literacy...
Spotlight on Digital Fluency <ul><li>“ Increasingly, the concept of information literacy is considered as crucially import...
Spotlight on Digital Fluency <ul><li>&quot;The  nature of the economy </li></ul><ul><li>is changing . It‘s putting more an...
U.S. Labor Secretary  “What Work Requires of Schools” (1991) <ul><li>An effective worker… </li></ul><ul><ul><li>Identifies...
Digital Fluency is  More  than Technology Skill Technology Judge Adapt Apply Design Organize Retrieve Summarize Compare Un...
Homework Copycats Prosper on the Net — San Francisco Chronicle Professors Vie with Web for Class’s Attention — New York Ti...
Popularity of Information Sources Source : De Rosa, Cathy, and others.  Perceptions of Libraries and Information Resources...
Trustworthiness of Information Source : De Rosa, Cathy, and others.  Perceptions of Libraries and Information Resources.  ...
International ICT Literacy Panel (2002) <ul><li>ICT literacy is not simply a mastery of technical skills, but is the appli...
Digital Fluency Framework A bridge between… <ul><li>Technical Literacy </li></ul><ul><li>Information Literacy </li></ul>Di...
Communicate Create Integrate Manage Evaluate Access Define Digital Fluency:   Skillful use of information via technology ...
Define:   using digital tools to identify and represent an information need. <ul><li>Activities include: </li></ul><ul><li...
Evaluate:   determining the degree to which digital information satisfies the need. <ul><li>Activities include:   </li></u...
Integrate:   interpreting and representing digital information from multiple sources. <ul><li>Activities include: </li></u...
Communicate:   communicating digital information properly in its context of use for a particular audience in a particular ...
Digital Fluency <ul><li>Information Literacy : “To be information literate, a person must be able to recognize when inform...
Assessment of Digital Fluency How do we assure students are ready for the digital fluency demands of college and the workp...
Challenges in developing a  believable measure of Digital Fluency <ul><li>Design an assessment with face validity </li></u...
The ETS  iSkills™  Assessment <ul><li>Interactive tasks –  not  multiple-choice </li></ul><ul><li>Organized around real-wo...
The ETS  iSkills™  Assessment <ul><li>Features </li></ul><ul><li>Interactive simulations –  not  multiple choice </li></ul...
iSkills Sample Tasks An inside look at the assessment
Measuring How We  Create <ul><li>Task Length: 4-minute duration </li></ul><ul><li>Task Type  Create :  Generate informatio...
In this task, examinees create a visual representation of data to answer two research questions.
Examinees select which variables to display on each of the axes.
Identifying the correct time span involves considering the implicit requirements of the information need.
Identifying the correct dependent variable (y-axis) involves thinking about how best to reflect “popularity.”
Examinees have the opportunity to try out different graphs before settling on their response, and this process is factored...
Answering the research questions involves correctly interpreting the graph.
The two research questions require different degrees of analytic skill.
<ul><li>Task Length: 4-minute duration </li></ul><ul><li>Task Type  Access :  Collect and/or retrieve information in digit...
This task requires examinees to construct an advanced search based on a complex information need.
… and must also select other appropriate parameters for the search. A successful examinee must enter the correct search te...
Examinees are scored on the characteristics of their searches as well as their ability to modify their search strategy in ...
The feedback describes characteristics of likely search results.
The feedback becomes increasingly specific with each unsuccessful search. In light of the initial feedback, the examinee p...
Examinees are allowed three attempts to formulate a correct search.
<ul><li>Task Length: 4-minute duration </li></ul><ul><li>Task Type  Integrate : Interpret and represent information in an ...
In this task, examinees summarize information from a variety of sources and then draw conclusions from their summary.
Examinees are presented with an information need and three different types of information sources.
1.  Webpage Examinees are presented with an information need and three different types of information sources:
2.  Email Examinees are presented with an information need and three different types of information sources:
3.  Advertisement The three sources present information according to different conventions.  A successful candidate must b...
Deciding how to compare the sources involves identifying the requirements of the stated information need.
After filling in the table, examinees must interpret the summary to rank the three sources correctly.
Preliminary Research Results The reality about your students’ digital fluency
Ongoing iSkills Data Collection <ul><li>Students from 76 institutions (N = 12,000)* </li></ul><ul><ul><ul><li>56 Colleges/...
Juniors and Seniors at 4-Year Colleges Only 27 % meet digital fluency expectations
iSkills – Some Early Findings… <ul><li>When selecting a research statement for a class assignment, 25% of test takers pick...
Overall Results <ul><li>Performance aligns with self-assessments </li></ul><ul><li>Doing a lot of digital fluency does not...
Prediction of Business Writing (CSU Los Angeles, n=193)  <ul><li>iSkills assessment given before instruction </li></ul><ul...
iSkills Scores vs. Business Writing Grade Source : Katz, I. R., Haras, C. M., & Blaszczynski, C. (2010). Does business wri...
iSkills Reports The various uses of the data that iSkills generates
Why Should We Assess Digital Fluency? <ul><li>Accreditation </li></ul><ul><ul><li>Institutional (General Education) </li><...
Institutional Data and Reports Offered <ul><li>Data Download  -  Allows you to determine whether or not your school needs ...
How do institutions use iSkills Institutional Data and Reports? <ul><li>Test results are a  portion  of the data that migh...
Individual Student Score Report  Confidential and Proprietary. Copyright ©2006 Educational Testing Service.
 Confidential and Proprietary. Copyright ©2006 Educational Testing Service. © 2007, Educational Testing Service.  All rig...
Institutional Skill Area Report <ul><li>Aggregates results from the test takers at an institution  </li></ul><ul><li>Compa...
Institutional Skill Area Report
Aggregate Task Performance Feedback Report <ul><li>Summarizes the performance feedback of the individual student reports  ...
Aggregate Task Performance Feedback Report  Confidential and Proprietary. Copyright ©2006 Educational Testing Service.
Institutional Data Download <ul><li>Help faculty determine relative performance of student subgroups </li></ul><ul><li>Del...
Institutional Data Download    Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo an...
Institutional Data Download  Confidential and Proprietary. Copyright ©2006 Educational Testing Service. Download results ...
Institutional Data Download  Confidential and Proprietary. Copyright ©2006 Educational Testing Service. <ul><li>Using ins...
Why Use a  Standardized  Assessment to Measure Digital Fluency? <ul><li>Broad comparison group </li></ul><ul><li>Confidenc...
Questions and Comments ? <ul><li>Visit the ETS Higher Education Assessment web pages at  www.ets.org/iSkills  to learn mor...
Upcoming SlideShare
Loading in …5
×

I skills webinar october 2011 (version 1.3)

3,576 views

Published on

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,576
On SlideShare
0
From Embeds
0
Number of Embeds
3,061
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

I skills webinar october 2011 (version 1.3)

  1. 1. What is the iSkills™ assessment? Bill Wynne ETS Higher Education Assessment Specialist Dr. Irvin Katz ETS Senior Research Scientist
  2. 2. Overview Spotlight on Digital Fluency A Framework for Digital Fluency Assessing Digital Fluency The Myths about Digital Fluency
  3. 3. Spotlight on Digital Fluency <ul><li>“ I’m calling on our nation’s governors and state education chiefs to develop standards and assessments…. [that measure]….21 st century skills like problem-solving and critical thinking …” Barack Obama President of the United States of America, Address to the U.S. Hispanic Chamber of Commerce </li></ul>
  4. 4. Spotlight on Digital Fluency <ul><li>“ Information and communication literacy is redefining what we mean by basic literacy beyond reading and math skills to include critical thinking, problem solving and creativity.” </li></ul><ul><li>Michael Dell Chairman and CEO of Dell, Inc. </li></ul>
  5. 5. Spotlight on Digital Fluency <ul><li>“ Increasingly, the concept of information literacy is considered as crucially important to enable people to deal with the challenge of making good use of information and communications technology.” </li></ul><ul><li>Abdul Waheed Khan UNESCO’s Assistant Director General for Communications and Information </li></ul>
  6. 6. Spotlight on Digital Fluency <ul><li>&quot;The nature of the economy </li></ul><ul><li>is changing . It‘s putting more and </li></ul><ul><li>more premium upon intellectual </li></ul><ul><li>skills, analytical skills, creative </li></ul><ul><li>skills, which are in short supply.” </li></ul>
  7. 7. U.S. Labor Secretary “What Work Requires of Schools” (1991) <ul><li>An effective worker… </li></ul><ul><ul><li>Identifies, finds, and selects necessary information </li></ul></ul><ul><ul><li>Assimilates and integrates information from multiple sources </li></ul></ul><ul><ul><li>Represents, conveys, and communicates information to others effectively </li></ul></ul><ul><ul><li>Converts information from one form to another </li></ul></ul><ul><ul><li>Prepares, interprets, and maintains quantitative and non-quantitative records and information, including visual displays </li></ul></ul>
  8. 8. Digital Fluency is More than Technology Skill Technology Judge Adapt Apply Design Organize Retrieve Summarize Compare Understand Define Disseminate
  9. 9. Homework Copycats Prosper on the Net — San Francisco Chronicle Professors Vie with Web for Class’s Attention — New York Times Abe Lincoln and the Truth Get Mugged at the Click of a Mouse -Los Angeles Times How to Handle Cyber-Sloth in Academe -Chronicle of Higher Education College Notes on the Web: Learning, or Laziness? -New York Times Americans Give Mixed Reviews to Use of Web for Schoolwork — San Diego Union-Tribune The Tyranny of Information — Los Angeles Times Information Inundation Imperils Our Children — Los Angeles Times Point, Click, Plagiarize - San Francisco Chronicle Lessons in the School of Cut and Paste — New York Times
  10. 10. Popularity of Information Sources Source : De Rosa, Cathy, and others. Perceptions of Libraries and Information Resources. (OCLC, November 2005) http://www.oclc.org/reports/2005perceptions.htm Google 71% Yahoo 64% MSN Search 48% Ask Jeeves 46% AltaVista, Lycos (tie) 28% Netscape 26% Library Web Site 21% Online librarian/“Ask a Librarian” 5%
  11. 11. Trustworthiness of Information Source : De Rosa, Cathy, and others. Perceptions of Libraries and Information Resources. (OCLC, November 2005) http://www.oclc.org/reports/2005perceptions.htm Information from libraries is… Less trustworthy than information from search engines 9% More trustworthy than information from search engines 21% NO DIFFERENCE in trustworthiness 70%
  12. 12. International ICT Literacy Panel (2002) <ul><li>ICT literacy is not simply a mastery of technical skills, but is the application of technical skills in an information society. </li></ul><ul><li>There is a lack of information about the ICT literacy of students and debate about how best to address this issue in academic curriculum </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  13. 13. Digital Fluency Framework A bridge between… <ul><li>Technical Literacy </li></ul><ul><li>Information Literacy </li></ul>Digital Fluency Can I find information in a library database? Can I create a persuasive presentation? Can I identify conflicts of interest in a web site? Database Word Processing Presentation <ul><li>Can I bold a word? </li></ul><ul><li>Can I open a database? </li></ul>Access Evaluate Use <ul><li>Can I find information? </li></ul><ul><li>Can I evaluate authority? </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  14. 14. Communicate Create Integrate Manage Evaluate Access Define Digital Fluency: Skillful use of information via technology  Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  15. 15. Define: using digital tools to identify and represent an information need. <ul><li>Activities include: </li></ul><ul><li>Developing a research topic to fit a particular information need </li></ul><ul><li>Asking questions to clarify the information need </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  16. 16. Evaluate: determining the degree to which digital information satisfies the need. <ul><li>Activities include: </li></ul><ul><li>Selecting the best database for an information need </li></ul><ul><li>Ranking web pages in terms of authority, relevance, objectivity </li></ul><ul><li>Determining whether collected information is sufficient </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  17. 17. Integrate: interpreting and representing digital information from multiple sources. <ul><li>Activities include: </li></ul><ul><li>Synthesizing information from IMs into word processing document </li></ul><ul><li>Comparing and contrasting information from web pages in a spreadsheet </li></ul><ul><li>Draw conclusions from information presented in emails, web pages, and print ads </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  18. 18. Communicate: communicating digital information properly in its context of use for a particular audience in a particular venue. <ul><li>Activities include: </li></ul><ul><li>Recasting an email into a presentation </li></ul><ul><li>Adapting presentation slides to a new audience </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  19. 19. Digital Fluency <ul><li>Information Literacy : “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate and use effectively the needed information …” (ACRL, 1989) </li></ul><ul><li>Digital Fluency = information literacy + digital environments </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  20. 20. Assessment of Digital Fluency How do we assure students are ready for the digital fluency demands of college and the workplace?
  21. 21. Challenges in developing a believable measure of Digital Fluency <ul><li>Design an assessment with face validity </li></ul><ul><ul><li>Authentic, relevant, performance-based </li></ul></ul><ul><ul><li>Measure the application of knowledge rather than facts or standard procedures </li></ul></ul><ul><ul><li>Aligned to nationally-recognized standards </li></ul></ul><ul><li>Ensure that it is based on solid measurement principles </li></ul><ul><ul><li>Quality and Fairness </li></ul></ul><ul><ul><li>Validity </li></ul></ul><ul><ul><li>Reliability </li></ul></ul><ul><li>Provide test-takers, instructors, and institutions with useful data and feedback </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  22. 22. The ETS iSkills™ Assessment <ul><li>Interactive tasks – not multiple-choice </li></ul><ul><li>Organized around real-world scenarios </li></ul><ul><li>Designed to measure higher-order problem-solving and critical thinking skills </li></ul><ul><li>Seven digital fluency content areas </li></ul><ul><li>14 short (4 minute) tasks </li></ul><ul><li>Administered in two 30-minute sections </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  23. 23. The ETS iSkills™ Assessment <ul><li>Features </li></ul><ul><li>Interactive simulations – not multiple choice </li></ul><ul><li>Vendor neutral </li></ul><ul><li>14 real-world, problem-solving scenarios </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  24. 24. iSkills Sample Tasks An inside look at the assessment
  25. 25. Measuring How We Create <ul><li>Task Length: 4-minute duration </li></ul><ul><li>Task Type Create : Generate information by adapting, applying, designing, or inventing information in ICT environments. </li></ul><ul><li>Within the task, the test taker is asked to… </li></ul><ul><ul><li>Visually represent data in a graph </li></ul></ul><ul><ul><li>Interpret the graph to answer research questions </li></ul></ul>
  26. 26. In this task, examinees create a visual representation of data to answer two research questions.
  27. 27. Examinees select which variables to display on each of the axes.
  28. 28. Identifying the correct time span involves considering the implicit requirements of the information need.
  29. 29. Identifying the correct dependent variable (y-axis) involves thinking about how best to reflect “popularity.”
  30. 30. Examinees have the opportunity to try out different graphs before settling on their response, and this process is factored into their score.
  31. 31. Answering the research questions involves correctly interpreting the graph.
  32. 32. The two research questions require different degrees of analytic skill.
  33. 33. <ul><li>Task Length: 4-minute duration </li></ul><ul><li>Task Type Access : Collect and/or retrieve information in digital environments. </li></ul><ul><li>Within the task, the test taker is asked to… </li></ul><ul><ul><li>Formulate an advanced search, utilizing Boolean operators, syntactic elements (parentheses, quotes), and search parameters. </li></ul></ul><ul><ul><li>Use feedback on likely results to improve search strategy </li></ul></ul>Measuring How We Access
  34. 34. This task requires examinees to construct an advanced search based on a complex information need.
  35. 35. … and must also select other appropriate parameters for the search. A successful examinee must enter the correct search terms and combine them using Boolean operators and syntax…
  36. 36. Examinees are scored on the characteristics of their searches as well as their ability to modify their search strategy in response to feedback.
  37. 37. The feedback describes characteristics of likely search results.
  38. 38. The feedback becomes increasingly specific with each unsuccessful search. In light of the initial feedback, the examinee performs a second search that… ·  includes the key content words · properly limits and expands the search … but does not have all of the correct syntactic elements ( e.g. , quotes, asterisk) .
  39. 39. Examinees are allowed three attempts to formulate a correct search.
  40. 40. <ul><li>Task Length: 4-minute duration </li></ul><ul><li>Task Type Integrate : Interpret and represent information in an ICT context, including synthesizing, summarizing, comparing, and contrasting information from multiple sources. </li></ul><ul><li>Within the task, the test taker is asked to… </li></ul><ul><ul><li>Summarize information from three different types of sources </li></ul></ul><ul><ul><li>Compare the information to reach a conclusion </li></ul></ul>Measuring How We Integrate
  41. 41. In this task, examinees summarize information from a variety of sources and then draw conclusions from their summary.
  42. 42. Examinees are presented with an information need and three different types of information sources.
  43. 43. 1. Webpage Examinees are presented with an information need and three different types of information sources:
  44. 44. 2. Email Examinees are presented with an information need and three different types of information sources:
  45. 45. 3. Advertisement The three sources present information according to different conventions. A successful candidate must be able to locate the relevant information in each source. Examinees are presented with an information need and three different types of information sources:
  46. 46. Deciding how to compare the sources involves identifying the requirements of the stated information need.
  47. 47. After filling in the table, examinees must interpret the summary to rank the three sources correctly.
  48. 48. Preliminary Research Results The reality about your students’ digital fluency
  49. 49. Ongoing iSkills Data Collection <ul><li>Students from 76 institutions (N = 12,000)* </li></ul><ul><ul><ul><li>56 Colleges/Universities </li></ul></ul></ul><ul><ul><ul><li>11 Community/Technical Colleges </li></ul></ul></ul><ul><ul><ul><li>9 High schools </li></ul></ul></ul><ul><li>* Results not necessarily representative of all college students </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  50. 50. Juniors and Seniors at 4-Year Colleges Only 27 % meet digital fluency expectations
  51. 51. iSkills – Some Early Findings… <ul><li>When selecting a research statement for a class assignment, 25% of test takers picked statements that did not address the assignment. </li></ul><ul><li>When asked to narrow an overly broad search, more than 80% of students could not correctly describe a specific problem with their initial search. </li></ul><ul><li>When asked to evaluate a set of websites, only 48% of test takers identified the website that met the criteria of currency, authority and objectivity. </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  52. 52. Overall Results <ul><li>Performance aligns with self-assessments </li></ul><ul><li>Doing a lot of digital fluency does not necessarily lead to good digital fluency skills </li></ul><ul><li>Digital fluency more strongly associated with verbal skills than math skills </li></ul><ul><li>Digital fluency more strongly associated with business writing than humanities writing </li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  53. 53. Prediction of Business Writing (CSU Los Angeles, n=193) <ul><li>iSkills assessment given before instruction </li></ul><ul><li>Grade assigned at end of academic quarter </li></ul>Source : Katz, I. R., Haras, C. M., & Blaszczynski, C. (2010). Does business writing require information literacy? Business Communication Quarterly, 73 (2), 135-149. GPA iSkills Pretest Business Writing Grade .32 .34 GPA - .12
  54. 54. iSkills Scores vs. Business Writing Grade Source : Katz, I. R., Haras, C. M., & Blaszczynski, C. (2010). Does business writing require information literacy? Business Communication Quarterly, 73 (2), 135-149.
  55. 55. iSkills Reports The various uses of the data that iSkills generates
  56. 56. Why Should We Assess Digital Fluency? <ul><li>Accreditation </li></ul><ul><ul><li>Institutional (General Education) </li></ul></ul><ul><ul><li>Program (Discipline-specific) </li></ul></ul><ul><li>Strategic planning </li></ul><ul><li>Renewed focus on institutional/program effectiveness and continuous improvement </li></ul><ul><ul><li>Trend analysis (longitudinal/cross-sectional studies) </li></ul></ul><ul><ul><li>Benchmarking </li></ul></ul><ul><ul><li>Comparison with other institutions/programs </li></ul></ul><ul><li>Measure instructional effectiveness </li></ul><ul><ul><li>Curriculum alignment </li></ul></ul><ul><ul><li>Teaching methods </li></ul></ul><ul><li>Facilitate student feedback/counseling/advising </li></ul><ul><li>Performance funding/making effective resource allocation decisions </li></ul><ul><li>Dispel the myths about what we think students/employees can do </li></ul>Copyright © 2010 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS). 14372
  57. 57. Institutional Data and Reports Offered <ul><li>Data Download - Allows you to determine whether or not your school needs to change elements of its curricula to better prepare your students for 21 st Century jobs </li></ul><ul><li>- Provides credible information that you can analyze to determine whether to implement, continue, or change an intervention program </li></ul>- Allows you to quickly assess your student performance on a comparative basis for the seven skill areas - Provides critical benchmark information to support your accreditation reporting needs <ul><li>Allows you to pinpoint specific skills and associated tasks that you need to change to better prepare your student </li></ul><ul><li>- Provides information that you can use to identify specific remediation to focus on </li></ul>Aggregate Task Performance Feedback Report Institutional Skill Area Report  Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  58. 58. How do institutions use iSkills Institutional Data and Reports? <ul><li>Test results are a portion of the data that might be used for different decision making </li></ul><ul><li>This is how some institutions are using the results: </li></ul><ul><ul><li>Helping to determine placement of transfer students </li></ul></ul><ul><ul><li>Pre- and post-testing in conjunction with intervention </li></ul></ul><ul><ul><li>End-of-course exams </li></ul></ul><ul><ul><li>Measuring outcomes </li></ul></ul><ul><ul><li>Meet accreditation requirements </li></ul></ul><ul><ul><li>Guide student in their academic careers </li></ul></ul><ul><ul><li>Collect evidence used for accreditation </li></ul></ul><ul><ul><li>Perform curriculum evaluation </li></ul></ul>
  59. 59. Individual Student Score Report  Confidential and Proprietary. Copyright ©2006 Educational Testing Service.
  60. 60.  Confidential and Proprietary. Copyright ©2006 Educational Testing Service. © 2007, Educational Testing Service. All rights reserved Individual Student Score Report
  61. 61. Institutional Skill Area Report <ul><li>Aggregates results from the test takers at an institution </li></ul><ul><li>Compares institution performance with a reference group </li></ul><ul><li>Report performance in each ICT literacy skill area </li></ul><ul><li>Includes two graphic representations of skill area performance relative to the reference group </li></ul><ul><ul><li>Score distribution </li></ul></ul><ul><ul><li>Median point </li></ul></ul>Reference group information can be used for external benchmarking and other needs related to accreditation and accountability
  62. 62. Institutional Skill Area Report
  63. 63. Aggregate Task Performance Feedback Report <ul><li>Summarizes the performance feedback of the individual student reports </li></ul><ul><li>Provides numbers and percentages of students who receive the highest score on each type of performance within reporting group </li></ul><ul><li>Compares reporting group versus reference group optimal response percentage </li></ul>Detailed performance feedback suggests areas for remediation, contributes data to the continuous improvement process, and provides evidence of achieving learning objectives for accreditation
  64. 64. Aggregate Task Performance Feedback Report  Confidential and Proprietary. Copyright ©2006 Educational Testing Service.
  65. 65. Institutional Data Download <ul><li>Help faculty determine relative performance of student subgroups </li></ul><ul><li>Delivered as a comma separated value (CSV) file </li></ul><ul><li>Results can be saved into an Excel spreadsheet </li></ul>Allows you to aggregate data according to your own analysis needs
  66. 66. Institutional Data Download  Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  67. 67. Institutional Data Download  Confidential and Proprietary. Copyright ©2006 Educational Testing Service. Download results can be saved into an Excel spreadsheet and analyzed
  68. 68. Institutional Data Download  Confidential and Proprietary. Copyright ©2006 Educational Testing Service. <ul><li>Using institutional data download you can perform program evaluations </li></ul><ul><li>Provides empirical results suggesting whether to implement, improve, continue, or expand a specific intervention program </li></ul>This analysis identifies a gap in transfer student skills – possibly indicating a need to implement a specific intervention program for transfer students Test Mean n = 149 n = 267
  69. 69. Why Use a Standardized Assessment to Measure Digital Fluency? <ul><li>Broad comparison group </li></ul><ul><li>Confidence in longitudinal data </li></ul><ul><li>Performance feedback </li></ul><ul><li>Test design </li></ul><ul><ul><li>Validity </li></ul></ul><ul><ul><li>Reliability </li></ul></ul><ul><ul><li>Fairness </li></ul></ul><ul><ul><li>Representativeness </li></ul></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
  70. 70. Questions and Comments ? <ul><li>Visit the ETS Higher Education Assessment web pages at www.ets.org/iSkills to learn more about the iSkills assessment </li></ul><ul><li>Contact an iSkills assessment expert toll free at </li></ul><ul><ul><li>1-800-745-0269 </li></ul></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).

×