Presented by Paul Nisbet and Abi James at the BDAN International Conference, March 2014. Since 2012 the JCQ Access Arrangements have acknowledged that candidates using a Computer Reader or text-to-speech technology are reading independently making such provision available to candidates in exams that test reading skills for the first time. While use of digital exams with text-to-speech has been widely supported in Scotland through the work of SQA and CALL Scotland for a number of years, the rest of the UK has not had equivalent access. From 2013/14 exam boards in England, Wales and Northern Ireland propose to provide digital versions of exam papers to schools for text-to-speech users. This paper will draw on experiences in Scotland and the work of the BDA New Technology Committee to identify processes and best practices within schools for using these digital papers and to identify the best text-to-speech technology to maximise the benefits for students.

1. Paul Nisbet, CALL Scotland
University of Edinburgh
Abi James, BDA
USING TEXT-TO-SPEECH IN EXAMS -
PITFALLS AND PRACTICAL SOLUTIONS

2. Reading difficulties in secondary
• “20% of 11 year olds have poor reading
comprehension” (data from SAT results)
• “by 14 years of age 33% of pupils have
unsatisfactory reading comprehension”
Chris Singleton, Special Children 182 April/May 2008

3. “While four-fifths of pupils at Key Stage 2
reached national expectations over the last
three years, one in five primary pupils did not
achieve the expected standard in English.”
Moving English forward: action to raise standards in
English, Ofsted 2012
Reading difficulties in secondary

4. SQA Assessment Arrangements
“Assessment arrangements allow candidates
who are disabled and/or who have been
identified as having additional support needs
appropriate arrangements to access the
assessment without compromising its
integrity.”
Introduction to Assessment Arrangements
for Schools and Colleges
SQA January 2010, revised June 2013

5. Assessment Arrangements 2013
• Requests for Assessment Arrangements
made for
– Scotland: 17,263 candidates (11.3% of
candidates)
– Rest of UK: 145,430 candidates (6.9%)
• Requests made for 61,680 entries (exams)
(8.3% of all entries in Scotland)

6. Assessment Arrangements 2013
Scotland Rest of UK
Requests for Access
Arrangements
17,263
candidates
(11.3%)
145,430 candidates
(6.9% - many arrangements
delegated to schools)
Extra Time 76% 59%
Reader 30% 27%
Scribe 24% 15%
Use of ICT including
digital papers
19% n/a - use of word processing
delegated to schools
Computer readers 2% 0.4%
Coloured paper
(excludes coloured overlays)
3% 2%

7. Computer Reader (TTS) vs. Human
Reader
• TTS offers an independent means to decoding text.
• TTS provides greater consistency.
• Human readers can be more flexible (can read anything).
• Some learners prefer to use TTS.
• Studies have shown candidates are more likely to check
questions with TTS than using a human reader, resulting
in higher scores (Dolan et al, 2005)
• Computer Readers are allowed in ALL exams, including all
GCSE, A-level & Functional Skills exams including those
testing reading skills “since it allows the candidate to
independently meet the requirements of the reading
standards”

8. A 6th year pupil at Denny High in Falkirk was assessed using the Neale Analysis, reading with
and without text-to-speech. Reading herself, her comprehension age was 6 years 9 months.
With text-to-speech, it was over 13 years.

9. 2002 need for Digital Question Papers with
TTS identified
2003-04 research into specification;
development of papers
2004-05 evaluation in 6 schools
2005-06 pilot trials #1 - 31 students used digital
papers in 105 examinations
2006-07 pilot trials #2 - 80 candidates used 490
digital papers in 200 examinations
2014 Rest of UK – PDF papers will be provided
by exam boards (not interactive);
UKAAF guidance
The Scottish Experience:
Research, Development and Trial

10. Type your answers
Click to tick
Use on-screen drawing
tools
Read questions with
text-to-speech
SQA Digital Question Papers

11. Software and Apps
Windows
• Acrobat Reader (free)
• Text-to-speech:
– ClaroRead, Co:Writer, Ivona MiniReader ,
Penfriend, Read and Write Gold, Adobe Read Out
Loud, etc
iPad
• ClaroPDF (£0.69 + £1.49 for extra voices)
• PDF Expert (£6.99)
• Adobe Reader (free)
www.bdatech.org/what-technology/text-to-
speech/exams/computer-readers

12. Digital Paper Requests 2008-13
2008 2009 2010 2011 2012 2013
Number of requests 514 1,167 2,000 2,832 3,694 4,291
Number of centres 46 73 101 149 173 188
Number of
candidates
204 422 675 1,069 1,327 1,677
Mean number of
requests per centre
11.17 15.99 19.80 19.01 21.35 22.82
Mean number of
candidates per
centre
4.43 5.78 6.68 7.17 7.67 8.92
Mean number of
requests per
candidate
2.52 2.77 2.96 2.65 2.78 2.56

13. Digital Question Papers 2008-2013
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
2008 2009 2010 2011 2012 2013
Number of requests
Number of candidates

14. Scottish assessment:
Costs of readers/scribes
• 19,058 reader requests; 14,905 scribe
requests
• ~ 36,000 individual exams
• ~ 54,000 hours
• Say average £20/hour for reader/scribe?
= £1,080,000
• Say £10/hour for invigilator?
= £540,000
• TOTAL = £1.62min Scotland last year

15. 1. Become confident with the technology and text-to-
speech tool they will be using in the exam.
- Use PDFs of past papers to gain experience on how to
navigate the papers
- Use TTS in class
2. Centre applies for digital paper
1. In Scotland via the online AAR system.
2. In the rest of the UK through the Modified Paper Route
3. Digital exam paper arrives.
1. In Scotland on CD with papers
2. In the rest of the UK exam boards differ but most allow
for download 1 hour before exam from secure website
How do schools and candidates “use” a
computer reader?

16. Resolving issues with use of digital
papers & text-to-speech

17. 1. Voice quality
Problem Solution
“many of the students
commented that the
synthetic voice was of poor
quality and was difficult to
understand”
Nisbet, P.D., Aitken, S., Shearer, N.
(2004) Trial of External Papers in
Accessible PDF for Candidates with
Additional Support Needs.
http://www.adapteddigitalexams.org.uk/
Downloads/Reports/
1. Scottish ‘Heather’ voice
licenced for use by
Scottish Schools (2007).
2. Scottish ‘Stuart’ voice
developed and licenced
for Scottish schools
(2011).

18. 2. Pronunciation
Problem Solution
“Text to speech doesn’t read every
word accurately, 1928 would be
read as one thousand nine hundred
and twenty-eight. Words are
mispronounced, in particular
names and places, which can
affect the smooth reading of longer
passages. Words containing an
apostrophe are not read correctly.”
Nisbet, P.D. (2010) SQA Digital Papers 2010
Report.
1. 2011 and 2012 papers (154
papers; 2,044 pages;
235,205 words!) analysed
for mispronunciations
2. 308 terms identified.
3. CereProc updated the
voices to fix the
pronunciation.
4. Or use a TTS tool with a
pronunciation editor.

19. 3. Unreadable image text
Problem Solution
Some text is an image and cannot be read by
the computer:
1. SQA desktop publishers
now replace image text
with text boxes with
selectable, readable text.
2. By 2012, analysis showed
that almost all image text
elements were readable.
3. 2014 – UKAAF guidance
on accessible PDF exam
papers

20. Impact on practice

21. Reliant on Readers?
Stuck with Scribes?
or
Independent with ICT?