An overview of and introduction to the concept of data sonification as a potential resources for creating accessible views of data online. A presentation for the University of Michigan Web Accessibility Working Group.

1. Sight, Sound, Numbers, & Us:
Data Visualization + Data Sonification = Data Accessibility?
PF Anderson, Emerging Technologies Informationist
August 2, 2016; for the UM Web Accessibility Working
Group
PF Anderson, Emerging Technologies Informationist
August 2, 2016; for the UM Web Accessibility Working
Group

2. Data

3. Data:
Example

4. Data Accessibility Example, Pt. 1
Locations of concrete curb ramps installed to date for fiscal year 2014.
Street Cross Street Council District
NE Quantity
Constructed
NE Date
Completed
NW Quantity
Constructed
COUNTRY CLUB
DR LUCERNE BL 10 0 0
COLYTON ST 4TH ST 14 0 0
PALMETTO ST MATEO ST 14 0 0
SEATON ST 4TH ST 14 0 0
GRIFFIN AV AVENUE 26 1 1 7/1/2013 0

5. Data Accessibility Example, Pt. 2
Locations of concrete curb ramps installed to date for fiscal year 2014.
Street
COUNTRY CLUB
DR
COLYTON ST PALMETTO ST SEATON ST GRIFFIN AV
Cross Street
LUCERNE BL 4TH ST MATEO ST 4TH ST AVENUE 26
Council District 10 14 14 14 1
NE Quantity
Constructed 0 0 0 0 1
NE Date
Completed
7/1/2013
NW Quantity
Constructed 0 0 0 0 0

6. Data: Science
Chemistry and the visually
impaired: Available teaching
aids, by Dorothy Tombaugh.
Journal of Chemical
Education 1981
58(3):222-226.
<http://www.catea.gatech.ed
u/scitrain/kb/FullText_Article
s/chem_visuallyimpaired-tea
ching%20aids.pdf>

7. Data: Science
Maguvhe, M., 2015, ‘Teaching
science and mathematics to
students with visual impairments:
Reflections of a visually impaired
technician’, African Journal of
Disability 4(1), Art. #194, 6
pages.
http://dx.doi.org/10.4102/ajod.v4i
1.194
“This study reports on factors
that limit the participation of blind
and partially sighted learners in
mathematics and science
education.”

8. Data: Science
“In high school, teachers were open to me being in their classes until I got to
my senior year and wanted to take my calculus course. I was told that no
blind person had ever done calculus before and that if I wanted to do it I
could, but if I wanted to proceed down that path that my itinerant teacher and
my school district couldn't support my wishes at that level -- it was just too
complex. Hence, I took that as my first real barrier to wanting to be a
mathematician or scientist. At that moment I didn't know what I was going to
do and it was very hard for me to accept this perceived barrier in my life. I
didn't know any other blind people who could help me get around that type of
ordeal. So I accepted it and finished out high school under the mindset that I
was not going to be a scientist or engineer, even though I wanted to be one.”
Cary Supalo, Chemistry Doctoral Student. National Center for Blind Youth in
Science. <http://www.blindscience.org/supalo-interview>

9. Data Accessibility Means Different Things to
Different People
● Data Inventory Is Key to Data Sharing and Accessibility, From the Management Team, April 14,
2016
○ <https://financialresearch.gov/from-the-management-team/2016/04/14/data-inventory-is-key-to
-data-sharing-and-accessibility/>
○ NOTE: Actually about access in general for data sharing/archiving/collection/distribution etc.,
not accessibility for persons with access challenges such as vision loss.
● Data.gov: Accessibility: <https://www.data.gov/privacy-policy#accessibility>
○ “Images on the site contain “alt tags”, which aid users who listen to the content of the site by
using a screen reader, rather than reading the site. Likewise, a “skip to” link provides these
users with a method for bypassing the header and going directly to the main content each time
a page is accessed. Text transcripts accompany audio clips, and closed captioning is available
on videos.”

10. Data Accessibility (Other Resources)
● Teaching students with visual impairments: Measurement and data collection
○ <http://www.teachingvisuallyimpaired.com/measurement--data.html>
○ Mostly devices and tools for manipulation and data generation/collection
● Making your HTML accessible for the visually impaired (E-Consultancy, 2016):
<https://econsultancy.com/blog/67625-making-your-html-accessible-for-the-visually-impaired/>
○ In comments: Matt Terrington “One area I'm personally struggling with is how to marry up
fancy data visualisations (https://econsultancy.com/blog/tags/data-visualisation/) with
accessibility for visually impaired users.”

11. About Data Visualization

12. Data Visualization
“As the saying goes, ‘a picture is worth a thousand words’ - often more - but only
when the story is best told graphically rather than verbally and the picture is well
designed. You could stare at a table of numbers all day and never see what would
be immediately obvious when looking at a good picture of those same numbers.”
Stephen Few. 35. Data Visualization for Human Perception. Encyclopedia of
Human Computer Interaction, 2nd Ed.
<https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-co
mputer-interaction-2nd-ed/data-visualization-for-human-perception>

13. Data Visualization & Accessibility
“Information graphics such as plots, maps, plans, charts, tables and diagrams form an
integral part of the student learning experience in many disciplines. However, for a vision
impaired student accessing such graphical materials can be problematic. … We found that
difficulty in accessing graphical materials was a barrier to many vision-impaired students
and that there were systemic problems with current processes for accessible graphics
provision. Recommendations are made on ways to address these concerns in order to
provide a more equitable higher education experience.” Butler M, Holloway L, Mariott K,
Goncu C. Understanding the graphical challenges faced by vision-impaired students
in Australian universities. Higher Education Research & Development. Published
online: 16 May 2016 DOI:10.1080/07294360.2016.1177001
<http://www.tandfonline.com/doi/abs/10.1080/07294360.2016.1177001>

14. Data Visualization: Examples 1
Examples shown (simple):
1) A simple black and white bar chart showing four bars with counts against the days of the week:
approximate values are Friday 22, Saturday 79, Sunday 75, and Thursday 60).
2) A pie chart for the European Parliament election, 2004, with 8 percentage values shown in 8 bright
colors, tagged with acronyms which may not be clear, such as PES, EUL, UEN, etc.
3) A scatter plot in black and white showing an arc of dots against numeric values which are not
described other than as the X axis and the Y axis.
IMAGE SOURCES: Wikipedia: Bar chart https://en.wikipedia.org/wiki/File:Tips-day-barchart.pdf Pie chart
https://en.wikipedia.org/wiki/File:Pie_chart_EP_election_2004.svg Scatterplot https://en.wikipedia.org/wiki/File:Scatterplot5.pdf

15. Data Visualization: Historical Examples
Three data visualization examples from the early to mid-1800s.
1) Florence Nightingale. “This ‘Diagram of the causes of mortality in the army in the East’ was
published in Notes on Matters Affecting the Health, Efficiency, and Hospital Administration of the
British Army and sent to Queen Victoria in 1858. This graphic indicates the number of deaths that
occured from preventable diseases (in blue), those that were the results of wounds (in red), and
those due to other causes (in black).” The graph appears in wedges from a central point moving
outward, with size variations based on the numberes represented.
2) The famous diagram from Minard showing the death rate on the march of the French Army during
Napolean’s invasion of Russia, and how the deaths correlated inversely with the termperature
during the Russian winter. . The chart resembles two tree branches, overlaid, with the wide end at
opposite ends.
3) Another chart from Minard, illustrating an early use of pie-charts combined with mapping.
IMAGE SOURCES: Wikipedia: polar area diagram by Florence Nightingale https://en.wikipedia.org/wiki/File:Nightingale-mortality.jpg Charles Joseph
Minard's vectorized map (1869) displaying the movements and the number of Napoleonic troops during the Russian campaign (1812-1813), as well as the
temperature on the return path https://en.wikipedia.org/wiki/File:Minard.png | http://www.martingrandjean.ch/historical-data-visualization-minard-map/
File:Minard-carte-viande-1858.png https://en.wikipedia.org/wiki/File:Minard-carte-viande-1858.png

16. Data Visualization: Examples 2
Examples shown (complex);
1) Social network analysis visualization of dots
of varying sizes in shades of blue on white
background, revealing a central cluster of
roughly 20 large dots and showing how
those connect to other clusters of
medium-sized dots among a variety of
scattered isolated small dots.
2) A three-dimensional scatterplot showing a
dense cloud of dots describing energy
sources (gas and solar) colored in a rainbow.
IMAGE SOURCES: Wikipedia: Social Network Analysis
https://en.wikipedia.org/wiki/File:Social_Network_Analysis_Visualization.png
3D Scatterplot https://en.wikipedia.org/wiki/File:Scatter_plot.jpg

17. Data Visualization: Examples 3
1) Screenshot of the Data Visualization Gallery
from Census.Gov. Screenshot includes
images of a variety of data visualization
techniques, including mapping, bubble
displays, tree structures, social network
maps, and more.
IMAGE SOURCES: Census: Data
Visualization Gallery
<https://www.census.gov/dataviz/>

18. Data Visualization Resources
● CreativeBloq: The 38 best tools for data visualization
<http://www.creativebloq.com/design-tools/data-visualization-712402>
● DataPine: The Ultimate Guide to Data Visualization Resources
<http://www.datapine.com/blog/data-visualization-resources/>
● Dummies: TOP 10 DATA VISUALIZATION RESOURCES
<http://www.dummies.com/programming/big-data/big-data-visualization/top-10-data-visualization-resources/>
● Honkiat: Data Visualization: 20+ Useful Tools and Resources
<http://www.hongkiat.com/blog/data-visualization-tools-resources/>
● Mu Lin: Teaching data visualization: Recommended readings and resources
<http://www.mulinblog.com/teaching-data-visualization-recommended-readings-and-resources/>
● Quora: What are the best resources for learning data visualization?
<https://www.quora.com/What-are-the-best-resources-for-learning-data-visualization>
● Visualizing data <http://www.visualisingdata.com/resources/>

19. About Data Sonification

20. Data Sonification: Definitions
“Sonification is the use of non-speech audio to convey information.” Goddard Space
Flight Center. <http://spdf.sci.gsfc.nasa.gov/research/sonification/>
“Sonification is the data-dependent generation of sound, if the transformation is
systematic, objective and reproducible, so that it can be used as scientific method.”
Thomas Hermann <http://sonification.de/son/definition>

21. Data Sonification: What Is It?
Data sonification -- from
deep space research to
improving lives through
cancer research
<https://www.youtube.co
m/watch?v=ndLkP-bNL1s>

22. Data Sonification Examples
● Listening to data from the Large Hadron Collider: Lily
Asquith at TEDxZurich
<https://www.youtube.com/watch?v=iQiPytKHEwY>
● Cellevision EP.23: Sonification of Space
<https://www.youtube.com/watch?v=scnG96ylrJ8>
● Full Transit of Venus (All Wavelengths + Sonification
<https://www.youtube.com/watch?v=urbTgl9jeNs>
● The Higgs Boson Song : A Sonification of the ATLAS Data
by Domenico Vicinanza
<https://www.youtube.com/watch?v=KjHvGyPlcT4>

23. Data Sonification Examples 2
● Sonification of Tohoku Earthquake / Sendai
Coast, Japan, 2011/03/11
<https://www.youtube.com/watch?v=3PJxUPvz9
Oo>
● Sonification of Fish Movement Using Pitch Mesh
Pairs (NIME 2015)
<https://www.youtube.com/watch?v=HzsFGQyIp
uc>
● Interactive Sonification Using Pure Data (cc)
<https://www.youtube.com/watch?v=iSmMaPT9
VI8>

24. Data Sonification Examples 3
● Musical sonification of brain activity
<https://www.youtube.com/watch?v=_nAzcyVP_
ZQ>
● Hearing brainwaves - epilepsy EEG sonified
<https://www.youtube.com/watch?v=I5JKOjZ60P
0>
● An instrument for the sonification of everyday
things (“non-contact laser triangulation sensor”)
<https://www.youtube.com/watch?v=Lyc7VNsxjo
A>
● Sonification of Twitter - Scott Lindroth
<https://www.youtube.com/watch?v=MUsBeJoB
Rxw>

25. Data Sonification at UMich
● Using the Sun to Make Music (Robert Alexander)
<https://www.youtube.com/watch?v=kcqiLvHiACQ>
● Two turn tables and a satellite: Robert Alexander at
TEDxUofM
<https://www.youtube.com/watch?v=55dIfA7C038>
● Now hear this: Scientific data might be filled with
important things waiting to be discovered. Just listen to
them
<http://www.economist.com/news/science-and-technol
ogy/21694992-scientific-data-might-be-filled-important-
things-waiting-be-discovered>
● User Story: Robert Alexander and Sonification of Data
<http://um3d.dc.umich.edu/user-story_robert-alexande
r/>
● Stereoscopic Solar Sonification
<https://www.youtube.com/watch?v=j-Dtk8fBKZo>
● Justin Joque: Sonification of the Dow:
<http://continentcontinent.cc/index.php/continent/article
/viewArticle/63>

26. Data Sonification: Who, How, Where, & Why
Audio Engineering Society (AES) <http://www.aes.org>
International Community for Auditory Display (ICAD)
<http://www.icad.org>
Georgia Tech Sonification Lab (Bruce Waker)
<http://sonify.psych.gatech.edu>
Thomas Hermann's research on Sonification, Data Mining and
Ambient Intelligence <http://sonification.de>
Berkeley Sonification Links
<http://cse.ssl.berkeley.edu/stereo_solarwind/sounds_links.html

27. Data Sonification: Selected Terms
● Acoustic technics / acoustic diagnosis / acoustic analysis
● Data composer
● Data sound engineer
● Acoustic ecology
● Audification (audify, audified, audificate, etc.)
● Auditory Display
● Non-visual overview of data
● Sonic information design
● Sonification (sonify, sonified, sonificate, etc.)
● Sound design

28. Data Sonification Tools
● EarSketch <http://earsketch.gatech.edu/learning/sonification-images-as-data>
● Flocking <http://flockingjs.org>
● iSonic: Interactive Data Sonification for Blind Users <http://www.cs.umd.edu/hcil/audiomap/>
● Listen: A Data Sonification Toolkit
<http://www.icad.org/websiteV2.0/Conferences/ICAD96/proc96/lodha.htm>
● Sonification sandbox <http://sonify.psych.gatech.edu/research/sonification_sandbox/>
○ A graphical toolkit for auditory graphs (2003)
<https://smartech.gatech.edu/handle/1853/50490>
● Sonify <http://sonify.org/home/release.html>
● Sonifyer <http://www.sonifyer.org>
● Sonic Pi <http://sonic-pi.net>
● SoniPy <https://sourceforge.net/projects/sonipy/>
● xSonify <https://sourceforge.net/projects/xsonify/>
● DIASS (Digital Instrument for Additive Sound Synthesis)
<http://www.mcs.anl.gov/~kaper/Sonification/DIASS/index.html>

29. Data Sonification Resources
● What does data sound like? An overview of sonification
<http://www.open-shelf.ca/160201-data-sonification/>
● Data sonification <http://datasonification.tumblr.com>
● Scientific sonification project (Argonne National Laboratory)
<http://www.mcs.anl.gov/~kaper/Sonification/>

30. Data Visualization, Sonification, &
Access

31. Sonification for Accessibility
“An obvious strength of sonification is its utility for people with visual impairments.
Sonification has been used by many researchers to make information about objects,
colours, and even maps perceivable to visually-impaired users. Representing
information in multiple ways that can be understood by those with different
perceptual abilities is vital for accessibility.” Jack Jamieson, WHAT DOES DATA
SOUND LIKE? AN OVERVIEW OF DATA SONIFICATION, February 1, 2016.
<http://www.open-shelf.ca/160201-data-sonification/>

32. Sonification for Accessibility
“Sonification's most obvious audience would seem to be the learner with visual
impairments, but it can also be appreciated by those with cognitive impairments,
learning disabilities, or a preference for audio presentation. It can help supplement
data visualizations or open new areas for discussion, research and learning.” The
Case for Sonification in Open Educational Resources, FLOE Project
<http://handbook.floeproject.org/Sonification.html>

33. Sonification for Access
Wanda Diaz Merced: How a blind
astronomer found a way to hear the
stars
<https://www.ted.com/talks/wanda_dia
z_merced_how_a_blind_astronomer_f
ound_a_way_to_hear_the_stars?langu
age=en> |
<https://www.youtube.com/watch?v=-h
Y9QSdaReY> Transcript:
<https://www.ted.com/talks/wanda_dia
z_merced_how_a_blind_astronomer_f
ound_a_way_to_hear_the_stars/transc
ript?language=en>

34. Sonification for Access: Examples
NY Sound Map
<http://www.nysoundmap.org>
Sound Seeker:
<http://www.soundseeker.org>
Sonification of guidance data during road
crossing for people with visual impairments
or blindness
<http://www.sciencedirect.com/science/articl
e/pii/S1071581915001275>
Wearable Auditory Biofeedback Device for
Blind and Sighted Individuals
<http://slis.tsukuba.ac.jp/~terasawa.hiroko.k
a/Papers/Matsubara2015_IEEEMM.pdf>

35. Sonification for Access: Examples 2
Follow that sound: using sonification and corrective verbal feedback to teach
touchscreen gestures <http://dl.acm.org/citation.cfm?id=2513455>
Headlock: a wearable navigation aid that helps blind cane users traverse large
open spaces <http://dl.acm.org/citation.cfm?id=2661453>
Categorization of seismic sources by auditory display: A blind test
<http://www.sciencedirect.com/science/article/pii/S1071581915001263>
Sounds of Seismic - Earth System Soundscape <http://sos.allshookup.org>
Designing Accessible Visualizations: The Case of Designing a Weather Map
for Blind Users
<http://link.springer.com/chapter/10.1007/978-3-642-39188-0_47>
REVOLUTION OF MULTI-SENSORY SCIENCE EDUCATION APPROACHES
FOR STUDENTS WHO ARE BLIND
<http://openjournals.library.usyd.edu.au/index.php/IISME/article/viewFile/1034
4/10273>

36. Sonification for Access: Examples 3
● Dataffect: Numerical Epistemology and the Art of Data Sonification
<http://slis.tsukuba.ac.jp/~terasawa.hiroko.ka/Papers/Matsubara2015_IEEE
MM.pdf>
○ “Concluding with a discussion of his installation Seismology as
Metaphor for Empathy (2012), the author suggests that representing
data through sound can engender powerful affective responses to the
cold abstraction of information.”
● Broadening Telematic Electroacoustic Music by Affective Rendering and
Embodied Real-time Data Sonification
<http://www.smc-conference.net/smc-icmc-2014/papers/images/VOL_1/030
1.pdf>
● See-Through-Sound: Transforming Images into Sonic Representations to
Help the Blind
<http://www.igi-global.com/article/see-through-sound/111252>

37. Data Sonification for Cognitive Differences
Computer-Based Cognitive Prosthetics: Assistive technology
for the Treatment of Cognitive Disabilities
<http://sonify.psych.gatech.edu/~walkerb/classes/assisttech/pdf
/Cole(1998).pdf>
Sonification as a possible stroke rehabilitation strategy
<http://journal.frontiersin.org/article/10.3389/fnins.2014.00332/f
ull>
Hear the Rainbow: An Interactive and Emotive Sonification of a
3-dimensional Color Space
<http://munsell.com/color-blog/hear-the-rainbow-color-sound/>

38. Data Sonification for Access
● Data sonification from the desktop: Should sound be part of standard data analysis software?
<http://dl.acm.org/citation.cfm?id=1101544>
● FLOE Project: Inclusive Learning Design Handbook <http://handbook.floeproject.org/index.html>
○ Sonification: What is Sonification? <http://handbook.floeproject.org/Sonification.html>
○ FLOE: Sonification Wiki: <https://wiki.fluidproject.org/display/fluid/%28Floe%29+Sonification>
○ Sonificaton Literature Review
<https://docs.google.com/document/d/1tnQyScm86Vj9rFKEogxPeP3x8A79A2dg6yLZ_ARuZtg/edit>
● The Sonification Handbook <http://sonification.de/handbook/>
● Sonification Special Issue <http://www.open-shelf.ca/columns/sonification-special-issue/>
● Quora: How can we make big data accessible to blind people?
<https://www.quora.com/How-can-we-make-big-data-accessible-to-blind-people>
● Universal Usability: Designing Computer Interfaces for Diverse User Populations, Section 5.2.5 Data Sonification
<https://books.google.com/books?id=BeHo0XPylDAC&dq=data+sonification+blind&source=gbs_navlinks_s>

39. And What About Touch?

40. And what about touch?
Constructing sonified haptic line graphs for the blind student: first steps <http://dl.acm.org/citation.cfm?id=354330>
Supporting blind navigation using depth sensing and sonification <http://dl.acm.org/citation.cfm?id=2494173> [Kinect]
Sonification of guidance data during road crossing for people with visual impairments or blindness
<http://www.sciencedirect.com/science/article/pii/S1071581915001275>
Open Touch/Sound Maps: A system to convey street data through haptic and auditory feedback
<http://www.sciencedirect.com/science/article/pii/S0098300413000666>