Joanna Wrzesinska: Ideas for Enhancing the Assessment of Speaking Skills Through Technology1. Ideas for Enhancing the
Assessment of Speaking
Skills Through Technology
Joanna Wrzesinska
Academic Relations & ELT Manager
ETS Global (TOEFL® and GRE® Tests)
2. Session Objectives
• Reflect on the current use of technology to faciliate
assessment of speaking skills at your institution.
• See how technology is used in the TOEFL iBT® Test
Speaking Section to ensure reliabilty and fairness of
scoring.
• Consider ways in which technology can be useful for
capturing, storing and rating spoken responses at your
institution and what the benefits might be.
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
3. Warming Up…
Turn to the person next to you
and find out if s/he currently:
• Uses voice capture technology
for conducting various forms of
assessment of speaking skills in
the classroom. How?
• Uses technology to facilitate
rating/scoring processes (by
human raters) of formal
assessments of speaking skills at
her/his institution?
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
4. Assessment of Speaking Skills
in the TOEFL iBT® Test
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
5. The TOEFL iBT® Test Basics
• Measures the ability to communicate by combining all
four language skills — Reading, Listening, Speaking and
Writing.
• Is academically focused, measuring the proficiency in
English as it is used in academic settings.
• Provides fair and objective scoring.
• Provides valid and reliable information to support score
users.
• TOEFL iBT® scores are accepted by more than 10,000
institutions, universities and agencies in more than 150
countries.
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
6. The Speaking Section
• Test takers respond to 6 tasks
• 2 independent tasks about familiar topics
• Responses based on personal experience
• 4 integrated tasks
• Responses based on what was read and heard
(a combination of academic and campus-based
tasks)
• Time: 20 minutes
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
7. Three to six different raters are involved in
scoring the six responses of a single test taker
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
8. Speaking Scores
• Holistic score
• Test takers are rated on:
• Topic development
• Delivery
• Language use
• Ratings are 0 ─ 4
• Converted to a scaled score of 0 ─ 30
• Raters use Benchmark
Responses
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
9. Considerations for Your
Institution
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
10. Ideas to Consider:
• Could I ask students to simultaneously record their
spoken responses and have them rated later?
• Could I record and store some responses for student
practice/ self-assessment / peer-assessment/ self-
monitoring purposes?
• Could digitally-captured student responses be useful to
identify banchmark responses for each level?
• Would such benchmark responses be useful at my
institution?
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
11. Ideas to Consider:
• Could I record and store student responses for rater
training purposes?
• Could I involve a second rater if I’m in doubt about how
to rate a specific student response?
• Could some of the responses that have already been
rated be cross-rated as a quality control measure?
• ... (Your idea here) ☺
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
12. To Learn More
The Propell® Workshops for the TOEFL iBT® test:
• A FREE one-day professional development program
• Uses examples from all the sections of the TOEFL iBT® test
• Provides instructional techniques and strategies for using the
integrated-skills approach to teaching language
• Led in an interactive, hands-on classroom setting
• Can be organized your institution free of charge!
www.ets.org/toefl/propell
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.
13. Thank You!
jwrzesinska@etsglobal.org
Copyright© 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, PROPELL, TOEFL and TOEFL ITP are registered
trademarks of Educational Testing Service (ETS) in the United States and other countries. All other trademarks are the property of their
respective owners.