Doctoral Consortium presentation in the 12th Web for All Conference

1. Semiotics contributions to accessible interface design.
María Inés Laitano. Laboratoire Paragraphe, Université Paris 8.
1

2. This study was motivated by an observation: that the "communicative wishes" of
users with disabilities are still not met by accessible interface design.
An enlightening example comes up in the answers to the WebAIM Survey about
Preferences in Screen Readers. One question focused on images that enhance the
mood or feel of a web page and whether the screen reader should describe them.
Out of one thousand one hundred twenty-one respondents, fifty-nine percent
wanted these images described, while thirty-one percent said they should be ignored
and ten percent had no opinion.
However, significantly, the vast majority of respondents wanting images ignored were
non-disabled users, i.e. users who only evaluate web accessibility. This result is in
clear contradiction with the success criterion 1.1.1 of the Web Content Accessibility
Guidelines 2.0 and highlights that the "communicative wishes" of users with visual
impairments are still not satisfied.
2

3. The guidelines limitations has been addressed in the literature in both empirical and
theoretical studies.
In an article titled "Guidelines are only half of the story" Power et al. (2012)
demonstrate through user evaluations, that many of the problems encountered by
blind users were not covered by the WCAG. Similar results were obtained by Rømen
and Svanæs (2011) through usability testing with disabled users. In brief, these
authors propose expanding WCAG with usability principles.
In Web for All 2012, Cooper et al. argued that there was a need to address the user
and usage context requirements which are not accounted for by metrics which only
address factors associated with the digital resource. They argue for a relational
approach to accessibility. That is, Accessibility is not a property of the resource, but a
property of the relation between the user and the resource in the context of
mediation. This approach underlies for instance the British Standard 8878.
3

4. In the accessibility guidelines approach, content is the main component. Not
coincidentally, it is located above all in the scheme. Developers design, code, edit,
and create content. End-users get the content and interact with it. Content is defined
as the code and markup that determine the structure, presentation, and interaction,
as well as text, images, and sounds that convey information to end-users.
Since content is a technical component in this approach, the guidelines address
formal aspects of the same like format, language, structure, sequence, etc. Some of
these formal aspects can be checked automatically, which is an advantage of this
approach.
4

5. Semiotics, in its more social dimension, is the study of communication.
Communication understood not as a transmission of content but as a process of
influence. When applied in the field of Human Computer Interaction, Semiotics
argues that designers communicate with a target audience; they have a
communicative strategy towards this target audience. For example, in an e-commerce
website, designers can opt for a strategy of persuasion
in order to push users-customers to buy.
The communicative strategy is expressed not only in the interface content but in the
layout as well. Our term for communication through interface layout is modal
enunciation.
Significantly, communication between designers and users is not a face-to-face
communication. As designers and users do not share the same space-time situation,
communication between them is deferred. Designers cannot accurately predict the
interface interpretation that will make the user, they can only hypothesize it. In
consequence, communicative strategy always targets an implicit user, i.e. a
hypothesis of user behavior, a presumption about user experience, competences and
expectations. This communicative strategy will be efficient only if the empirical user
(i.e. a real user in a use situation) recognizes himself in this implicit user.
5

6. What are the differences between the two approaches?
We already said that the guidelines approach is content-centric while the Semiotic
approach is Communication-centric.
The notion of designers in Semiotic approach is broader than that of developers
because it also includes institutions that create websites and usually produce the
communicative strategy.
In the Semiotic approach, users are not just people using content, they are also the
target of communication (customers, students, tourists, etc.) as well as the designers’
hypothesis (how the user carries out its task, what are their skills, etc.).
Finally, while the Guidelines approach deals with formal aspects of the layout, the
Semiotic approach deals with communication through layout, in other words, with
modal enunciation.
6

7. Let's see a concrete example in order to illustrate these theoretical notions. A
research institute wants to design its accessible institutional website. The purpose of
the institute is to emphasize the regional identity of the whole organization as well as
the individual identity of each research team. For this, the communicative strategy lie
in to invite the user to explore artworks of a regional artist. Artworks integrate
autochthonous elements and each one is chosen to refer to a research team in a
symbolic way. The slide presents a first visual mockup of the home page. The header
has the institute logo, a login link and a search field. The main area is occupied by six
artworks.
7

8. In the source code, each artwork is an image within the text of a link that takes the
team name and opens the team page. According to the WCAG Images Tutorial, each
of these images would be a functional image because it’s used to initiate an action
rather than to convey information. The image does not represent different
functionality or convey other information than that already provided by the link text,
so a null ALT value is applied to avoid redundancy and repetition. It’s clear that, with
this solution, the communicative strategy of designers is completely overridden for
people who cannot see images.
In contrast, an accessible design from a semiotic approach must prioritize and ensure
the communicative strategy. Thus, advances in the field of auditory interfaces can be
used to provide a similar experience in the auditory modality. So, a short sound can
be associated with each research team, evoking by causal or semantic listening, the
identity of each team. The sound could be associated to the link through the cue-
after propertyof CSS3 Speech Module. Remember that causal listening refers to
listening for the clues in sounds that point to the sound’s cause. This is the mode of
listening used in auditory icons. Semantic listening is the mode of listening in contexts
involving coded audio signals where the listener focuses on decoding that signal to
arrive at the message.
For instance, the artwork chosen for the team called “Children's cognitive
8

9. development” look like a pull toy. If we hear the associated sound, through causal
listening we recognize the timbre of a little girl's voice. Through semantic listening we
recognize that the little girl is learning to talk because she babbles some words in
Spanish. Thus, interpretation through semantic listening not only refers to children's
cognitive development but also to the regional identity of the institute.
8

10. Another point to address in accessible design is the reading order of webpage
elements for sequential navigation.
The success criterion 1.3.2 suggests providing a reading order that follows sequences
and relationships within the content —which follow text directionality (left to right or
right to left) and markup structures such as tables or lists. In our example, this means
navigation from left to right and top to bottom: starting with header elements and
continuing through main content elements.
While this ordering criterion is not invalid, the semiotic approach provides a higher
level criterion founded on the communicative strategy. So the reading order must
follow the hierarchy established by modal enunciation. In our example, modal
enunciation prioritizes main content over the header: the size of the artworks aims to
draw the users’ attention. In fact, the communicative strategy is revealed in the main
content. Consequently, following the semiotic approach, it would be more
appropriate that sequential reading begins with main content elements and only then
continues to header elements.
It could be argued that this problem is resolved by success criterion 2.4.1 that
proposes providing a mechanism to navigate directly to the main content. However, if
our implicit user is a novice, it’s not evident that he would understand the phrase
"Skip to main content" at the beginning of a web page. Direct reading of the first
9

11. research team will be more understandable for such a user. In fact, the WebAIM
survey shows that the use of skip links is not widespread.
9

12. Another important issue in accessible design is the name given to textual elements of
the interface. For instance, the success criterion 2.4.4 suggests that the text of each
link should describe the purpose of the link, i.e. the nature of the result obtained by
activating the link. From this perspective, a name like "Log in" is sufficient for the link
found on the home header. "Log in" indicates that by activating this link we will find a
login webpage.
However, this name can be improved by adopting a semiotic approach. The institute
communicative strategy targets two audiences: the general public and
institute members. Only institute members can log on the website to access
information that is not public. Consequently, naming the link as "Members login"
instead of "Log in" marks the difference between the two types of implicit user.
Institute members will probably recognize himself in this link while non-members will
avoid wasting time activating it.
10

13. The Semiotic approach is based on human communication, a human capacity that
cannot be automated in an evaluation tool.
However, a kind of semiotic checklist could be added as a further human step in
existing methodologies such as Conformance Evaluation. Just like we involve users to
ensure usability, we could involve designers in order to check communicative issues.
For now, we can summarize this in three general questions:
1. Is the communicative strategy available for everyone in the target audience?
2. Is modal enunciation preserved in all modalities?
3. Do interactive elements take their implicit user into consideration?
11

14. By way of conclusion we can say that complementing the guidelines approach with a
semiotic approach means not only thinking of accessible interface design as content
legibility or as a usability problem but rather as more complete communication with
the user.
12

15. References:
Bootz, P. (2007). Éléments d’analyse de l’interface sémiotique des sites Web. In I.
Saleh, K. Ghdira, B. Badreddine, N. Bouhai, & B. Rieder (Eds.), Collaborer, Echanger,
Inventer : Expériences de réseaux (pp. 107–121). Paris: Hermès Lavoisier.
Chisholm, W. A., & Henry, S. L. (2005). Interdependent components of web
accessibility. Proceedings of the 2005 InternationalCross-Disciplinary Workshop on
Web Accessibility (W4A), 31–37.
Cooper, M., Sloan, D., Kelly, B., & Lewthwaite, S. (2012). A Challenge to Web
Accessibility Metrics and Guidelines: Putting People and Processes First. Proceedings
of the International Cross-Disciplinary Conference on Web Accessibility, 201–204.
De Souza, C. S. (2005). The semiotic engineering of human-computer interaction.
Cambridge, Mass.: MIT Press.
Leiserson, A. B. (2014). Functional Images.
http://www.w3.org/WAI/tutorials/images/functional/.
Pignier, N., & Drouillat, B. (2004). Penser le webdesign: modèles sémiotiques pour les
projets multimédias. Paris: Editions L’Harmattan.
13

16. Power, C., Freire, A., Petrie, H., & Swallow, D. (2012). Guidelines Are Only Half of the
Story: Accessibility Problems Encountered by Blind Users on the Web. Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems, 433–442.
Rømen, D., & Svanæs, D. (2011). Validating WCAG versions 1.0 and 2.0 through
usability testing with disabled users. Universal Access in the Information Society,
11(4), 375–385.
Schaeffer, P. (1967). Traité Des Objets Musicaux. Paris: Seuil.
Scolari, C. (2009). The sense of the interface: Applying semiotics to HCI research.
Semiotica, 2009(177), 1–27.
Souchier, E. (1998). L’image du texte pour une théorie de l’énonciation éditoriale. Les
cahiers de médiologie, 6(2), 137–145.
W3C. (2008). Web Content Accessibility Guidelines (WCAG) 2.0.
http://www.w3.org/TR/WCAG20/
WebAIM. (2009). Survey of Preferences of Screen Readers Users.
http://webaim.org/projects/screenreadersurvey/
** Slide 8 copyright notice: The author of the visual artwork is Adolfo Nigro and the
authors of the sound composition are Marisa Cenacchi and Patricia San Martín.
13