1. Zachary Dalton Miller ~ Virginia Tech ~ zackm12@vt.edu
Engineering Science and Mechanics
ICAT Advisor Ivica Ico Bukvic
Sonification – the
transformation of data
into audio signals
ABSTRACT
This project explores the possibilities of
brainwave sonification and pursues answers to
questions such as: How much practice is
required to consistently manipulate one’s
wave spectrum? How well can complex results
be communicated and understood? How can
sonification influence the world of art? What
other applications can come from this
technology?
• Mass of 90g
• Defined EEG wave spectrum (Alpha, Beta..)
• 250 kbit/s RF* data rate
• 10m RF range
• 512 Hertz sampling rate
• 12 bit ADC* resolution (.0244% of full)
• 57,600 Baud UART*
• Mobile version iOS compatible
*RF = radio frequency
*ADC = analog-to-digital converter
*UART = universal asynchronous
receiver/transmitter
RESOURCES
BrainWaveOSC
https://github.com/trentbrooks/BrainWaveOSC
L2Ork
http://l2ork.music.vt.edu/
Pure Data
http://puredata.info/
NeuroSky
http://store.neurosky.com/
Lisa Park
http://thelisapark.com/#/
NeuroSky MindWave
Park sent EEG data from the headset through
Processing using Max/MSP that was linked to
Reaktor to create sound. I created a patch in Pure
Data (PD) which aimed to mimic her software. PD
was used so the data can eventually be run
through the K12 version of PD-L2Ork, created by
Ivica Ico Bukvic at Virginia Tech.
For her performance, Park broke down her EEG readings
into five emotions: happiness, hatred, sadness, anger,
and desire. Each emotion has a corresponding tray of
water upon a speaker for a multimedia display of visuals
and sound.
PD-L2Ork K12 is user-
friendly software designed
for the user to manipulate
raw data with a focus on
audio signals.
With the help of BrainWaveOSC (above) by
Laurent Bédubourg, my patch provides a basic
visualizer for eight types of MindWave EEG
signals that are defined by neuroscience: low
alpha, high alpha, low beta, high beta, low
gamma, mid-gamma, delta, and theta. The
headset also provides readings of attention and
meditation which register as values from 0 to
100. The visualizer for six of the data streams
can by seen in a section of my patch shown
below.
BrainwaveOSC receives
EEG data signals from
the headset via
bluetooth. Compatible
with Windows and OSX
The visualizer can be used to practice the art of
manipulating brainwaves. Park shared that she believes
months of practice are required before performing with
acuity.
An incredibly useful characteristic of the headset is to
reliably respond to intentional blinks of the wearer. In my
patch, eye blinks are designed to activate a bass hit when
registering a value above zero. A different program could
activate a snare drum, turn on/off a light, or select options
when prompted, among infinitely many other possibilities.
RESULTS FUTURE
The current objective is to modify existing
software for compatibility with Linux so PD-L2Ork
K12 can be used for the sonification process. With
that done, MindWave headsets can be used in
ICAT’s Maker Camps, which teach elementary and
secondary school children how to take initiative
and be resourceful to create beautiful and useful
things, such as their own instruments.
As technology develops, more applications will
arise, such as the control of a toy helicopter as
seen below. Now that generations are raised
alongside information technology, new capabilities
like EEG observation will offer committed
researchers an opportunity to develop novel
technologies, be they useful, beautiful, or both.
Device Specs
RESEARCH
The introduction of an affordable, research-
grade EEG device like NeuroSky’s Mindwave
presents an opportunity to further explore the
human-computer relationship. One particular
project that offers incredible insight and
creativity is Lisa Park’s Eunoia (shown below).