ICCE 2019 Taiwan

1. Diagnostic Language Assessment:
Lessons Learned from Rapid
Prototyping
Allan Nicholas
and
John Blake
University of Aizu

2. Overview
• Diagnostic language assessment
– Rationale
– Pragmatic competence
– Niche
– Three-step process
• Rapid Prototyping:
– Low- to high-fidelity prototypes
– Axure RP
• Lessons learned
– User needs
– Usability issues
– Interactive visual aid
02

3. Diagnostic language
assessment

4. Rationale for
Diagnostic Language Assesment
04
• Email writing is hybrid genre
• A key task (Kaneko et al., 2018)
• Helps students draft better emails:
–Identifies weak points
–Mediates improvement/development
Kaneko, E., Park, M., Wilson, I., Roy, D., Heo, Y., Yasuta, T., Blake, J., Nicholas, A. (2018). English
curriculum innovation for computer science majors in the Japanese EFL context: From needs to
tasks. Proceedings of the 2018 IEEE International Professional Communication Conference, 84-89.

5. Pragmatic competence
05
Errors related to contextual variables (Brown &
Levinson, 1987)
• power, distance
• and imposition
and
• directness
• and formality
negative interpretations
(Biesenbach-Lucas, 2007)
Biesenbach-Lucas, S. (2007). Students writing e-mails to faculty: an examination of e-politeness
among native and non-native speakers of English. Language Learning and Technology, 11(2), 59–81.
Brown, P. & Levinson, S. (1987). Politeness: Some Universals in Language Use. Cambridge University
Press.

6. Niche
06
• Paucity of research on pragmatic errors in
emails
• Little extant research on pragmatic errors in
– emails in Japan
– emails to faculty
• Diagnostic assessment gaining prominence

7. Three-step process
07
Development
and
administration
Efficacy
evaluation
Instructional
material creation
and integration

8. Rapid prototyping

9. Rapid Prototyping
09
• Back-of-napkin rough sketches
• Viewport cartoon strip
• Hierarchical diagrams
• PowerPoint simulation
• Prototyping software, e.g. Axure RP

10. Back-of-napkin sketch
Good for concepts, but little use when explaining to
others
10

11. Cartoon strip
Good to explain to collaborators familiar with project, but
difficult to envisage how the app or site should work
11

12. Hierarchical diagram
top
Level 2 Level 2 Level 2
Offshoot
Can see the overall web design, but not interactions
(server-client) or behaviours (DOM manipulation)
12

13. PowerPoint
simulation
Input text to translate 
13

14. PowerPoint
simulation
Translation appears here 
14

15. PowerPoint
simulation
New feature
Suggested response 
15

16. Axure RP
16

17. Axure RP
17

18. Axure RP
18

19. Lessons learned

20. Rapid ≠ Fast
20
• No need to set up
server, but still very
time-consuming
• Speed is proportional
to fidelity
e.g. how to mimic ML
algorithm discovering
errors in submitted text

21. User needs
21
• Early discovery of omissions in software
requirements specification when prototype
creator is forced to make unplanned decisions
• Can save time/money especially when
outsourcing fully-fledged version

22. Usability issues
22
User interface (UI)
• Users can try interface and so problems can
be identified before full code version is
developed
User experience (UX)
• High-fidelity version can accurately predict UX
since all behaviours and interactions are
simulated for chosen use cases

23. Communication aid
23
• Provides focus for all stakeholders
• Acts are model to ensure teachers, designers
and software developers understand how the
tool should operate
• “living specifications”
If
a picture = 1000 words;
Then
a prototype = 1000 pictures.

24. Any questions, comments or suggestions?
John Blake
jblake@u-aizu.ac.jp
JSPS KAKENHI 19K00826, “Developing a
diagnostic language assessment of Japanese
EFL learners' English language email writing”.