This presentation is based on an assignment I’ve been running in a senior visual literacy class, in which students are assigned a Shakespeare sonnet and then tasked with producing multiple “versions” of the same poem using different concepts and materials. I started running that assignment because that section of the course had mostly been used to teach Photoshop, and it just wasn’t working out. Students learned how to use the tool, but they really didn’t think much about what they were producing or what they were trying to communicate. And it just seemed like it was time to remind them that the world was a much more interesting and complicated place in which to compose.
My first understanding is that we are living in a material world when we interact with expressive objects all the time. That is, we don’t just occupy the same spaces as objects; they are always communicating with us. So when Carla Diana says that objects are becoming more “expressive” a large part of that is that they have been programmed to respond to our presence and our input. My second understanding is that learning to live in a world of expressive objects doesn’t just mean learning how to program a device – it means learning how to communicate with it – how to read and write, more or less, using the physical world. This is what Cathy Davidson is talking about when she references the “multimedia imagination.”
So, given those two understandings, what can we do to move from a static piece of text to one that invites us to want to puzzle it out, or play with it, or disassemble/reassemble it?
Two concepts via Ian Bogost seem relevant: procedural literacy and procedural rhetoric. Procedural literacy, the reconfiguring of concepts & rules to understand and solve problems, encourages us to try breaking down and building back up objects and texts to see how they work – the primary goal of any good literary scholar. Procedural rhetoric goes even further: the use of core concepts in computing to create new texts: use their grammars and affordances to compose new works of literature and argumentation.
In setting up this assignment I decided to work with a poem. Poems, more so than most literary forms, are useful for composition exercises because they’re economical, highly structured, and are possessed of their own technical language and affordances that give the beginning student something to hang onto. William Carlos Williams describes this economy as part of a poem’s “machinery.”
For those of you who teach literature you’ll know that poetry is a tricky machine, though. Students usually are more familiar with prose, and it’s usual to encounter resistance to looking at a more highly structured, carefully managed format. They are often distracted by the easier, more surface features of prose such as plot and imagery (let’s leave out experimental prose for the moment, which is its own kind of animal).
So, what’s my plan here? It’s specifically to show, in Williams’ words, how a poem is like a machine. But it’s also a way of showing how poetic concepts might have similar analogs in computing. In this way I’m trying to inculcate a little of Cathy Davidson’s “digital literacy” and “multimedia imagination”, and maybe even sneak in a programming concept or two. Not to teach programming, but just to show that the fundamentals of programming are not incomprehensible or arbitrary – they involves familiar concepts that we can manipulate to compose works of literature in multimedia format.
So, here’s the sonnet sequence. As you can see, it starts out simply, and then gets more complex as the semester goes on, with students learning a new concept in each round. The problem in this sequence of four assignments is to find alternative ways to translate a structured textual object like a poem into a sonic, visual or haptic object. We start out fairly simply, with the first assignment asking students to create a version of the sonnet that captures the visual imagery of the poem in some way, organizing it using a conceptual metaphor gleaned from the sonnet. The second assignment asks students to find a way to express the actual structure of the poem, breaking it down into iambs, lines, quatrains, the volta, and the final couplet. After that we take a turn into procedural literacy, asking how we can take advantage of the kinds of procedural tools available to us (applying concepts of encoding and algorithm respectively in parts three and four). Thus, part three asks students to create a version of the poem that can only be read using a “decoder” (and along the way we discuss such concepts as lossy and lossless compression). And part four asks students to come up with a kitset that can be used, in conjunction with a series of instructions, to recreate the poem, not unlike a Fluxus box or a maker kit.
Here’s the sonnet I’m using. It’s got all the usual features of a Shakespearean sonnet – ABAB rhyme scheme, three quatrains and a couplet, a volta, etc.
What are we going to do in these assignments? We’re going to create new versions of that sonnet, using crafty materials, in class. Rather usefully given our departmental learning outcomes, the sonnet sequence also gives students an opportunity to re-engage with concepts in poetics and scansion that they learned at the beginning of their career as English majors, and allows them to think about poetic forms as extending beyond the written word. All work is done in pairs or threes, and done in-class so that students can observe the way other people are coming up with solutions to the same problem. This tends to encourage friendly competition in the classroom, resulting in more imaginative work, as well as cooperative trading of materials and supplies. It also introduces an extra constraint – time. One cannot play without constraints.
The first exercise is the most traditional, but I also used it to introduce a concept that led many other discussions and assignments in the same class: dimensionality.
Here’s one finished piece. Each “planet” represents a couplet, and the planets are stuck with an image that is relevant to that couplet – e.g. judgement, luck, reproduction etc. The planet Earth is stuck with a set of eyes, which the students identified as the main conceptual metaphor they wanted to work with, i.e. looking at the beloved, seeing into the future, etc.
Here’s another one. This solution features a stack of images on heavy posterboard cards. Each card has a “death” image on one side and a “life” image on the other. The cards are weighted with magnets so that you can stand them on their edges and they will fall one way or the other. This version plays on the hypertextual nature of the poem: if the addressee has a child, all will be truth and beauty. If not, death and despair.
So, onto exercise 2. This time, we’re moving away from the “what” of the poem, and into the “how” – revealing the technical details. This is designed to remind students that poems (and programs) are possessed of specific structural features, no matter what it is they’re about.
A simple version: 6 pairs of atoms, and a central much larger couplet. It is the couplet that looms largest, given that it spells out the consequences of not reproducing. Rather nicely, it turns out that #12 on the periodic table is magnesium, which burns a bright flame. Like love, I guess.
Showing the rhyme scheme: abab, cdcd, etc. The four strings on the right represent the quatrains and couplet, and each one is strung with an image that is central to that piece of the poem. The whole thing is conceived as a musical instrument to reflect the musicality inherent in poetry.
Parts of speech. Verbs, nouns, etc.
Now we’re going to move from more traditionally literary tools to looking at the poem as something else: something that can be encoded, transmitted, and decoded.
This solution is a braided cord using color-coded thread to represent key imagery, along with a cord/card decoder.
A Braille poem using beads and thread, with color-coded beads for the words and thread to mark out different parts of the poem.
This one is an envelope system in which each line was translated from letters to a 5-digit number. The number became a zip code, and each envelope was addressed to a real-life address that represented a word in the poem (e.g. “doom” in the last line means the envelope is addressed to a cemetery).
The fourth assignment is perhaps most closely aligned with the procedures and affordances of electronic literature, but in an analog framework.
A burger assembly box, in which the user assembled the poem using burger buns to represent each quatrain, and a patty and toppings to represent various parts of speech.
A Jenga tower, in which users build a tower according to a set of instructions (requiring them to write key words and concepts on the blocks), and then knock it down at the end (to signify “doom”).
Here’s the turning point:
I posted one of the solutions (@SonnetOneFour, by Jones, Robbins & Downey) to Facebook & it was picked up as a “first encounter” piece for this conference. SonnetOneFour was conceived as a solution to the “encoding/decoding” assignment, and it was based on Jaci Jones’ recent experience at the summer DefCon conference, where they run competitions to solve puzzles. So what makes it electronic literature? Is it because it’s on Twitter?
I’d like to argue for an expansive definition of electronic literature that includes Bogost’s procedural rhetoric and procedural literacy. For one thing, it creates a larger tent for students who want to work with pieces that are not “made” of computers but are deeply beholden to the kinds of conceptual frameworks that have allowed for computers. Works by Mez, for example, don’t “execute” – they have a presentation layer on top of an authoring layer, which could be separated. Similarly, pseudocode may not execute, but it could easily be read as part of a critical code study.
FOURTEEN RECIPES FOR
“ . . . we ’ r e e n te r in g a t ime wh e n s o u n d , l i g h t a n d moveme n t a r e
equally impor tant par ts of the creative palette. Everyday
objects whose expressive elements have long been static wi l l
now glow, sing, vibrate and change position at the drop of a
h a t . ”
(NY T, C a r l a Di a n a , “ Ta lki n g , Wa l kin g Ob j e c t s, ” J a n . 2 6 2 01 3 .)
“Di g it al l i te rac y me a ns not rote l e a rni ng b u t ex p er imentat io n,
process, creativity, not just technology but multimedia
imagination, expression–and principles too. ”
- Cathy Davidson
THE STARTING QUESTION
What does a
poem do in a
Can we bui ld
a poem using
Do we need to
P ro c e d ur al l i te ra cy: “ e n t a ils the abi lity to reconfigure basic
concepts and rules to understand and solve problems, not just
on the computer, but in general. ”
p ro c e d ural rh eto r ic : “ a type of rhetoric tied to the core
af fordances of computers: running processes and executing
rule-based symbol ic ma ni p ulat io n. ”
Ian Bogost , Procedural Literacy; intro to Persuasive Games
MACHINES AND POEMS
“ To ma ke two b o l d s t a teme n t s: T h e re 's n ot h i n g s e n t ime n t al
about a machine, and: A poem is a small (or large) machine
made out of words. When I say there's nothing sentimental
about a poem, I mean that there can be no par t that is
r e d u n dan t . …
Prose may carry a load of i l l -defined matter l ike a ship. But
poetry is a machine which drives it, pruned to a per fect
economy. As in al l machines, its movement is intrinsic,
undulant, a physical more than a literary character . ”
(Wi lliam Carlos Wi l liams)
LOOKING AT/LOOKING THROUGH
A p o em may b e a ma c h in e , b u t i t ’ s e a s y to g e t d i s t r ac te d by
meaning and not see the mechanisms working on us
Students of ten resist looking at the formal attributes of
poetry, preferring to stick with more fami l iar representational
aspects: imagery, metaphor, emotional resonance.
“ Pe o ple l o o k fo r me s s ag e s i n p o ems ; c e r t a in ly mo s t o f my
s t u d ent s d o , no ma t ter h ow mu c h I t r y to d i s c o u r age t h em. ” –
Students are easi ly flummoxed by the economy of a poem,
preferring free expression to af fordances and constraints.
THE PLAN: USE A POEM TO BETTER
UNDERSTAND DIGITAL CONCEPTS
…a n d u s e digital l iteracy to better understand the
mechanisms of a poem.
Try working with a some key digital concepts that l iterature
students tend to avoid (of ten on purpose) :
Encoding: writing is a code, not a conveyor of transparent
Algorithm: understanding a poem as programmatic, i .e.
constructed according to a set of procedures
SONNET SEQUENCE: TRANSLATION &
A four-par t assignment sequence in which a Shakespearean
sonnet is reinterpreted in various forms according to the
1) Visual imagery
2) Sonnet structure & scansion
3) Encoding and decoding with a key
4) A “ k i t” fo r a s s embling a ve r s io n o f t h e s o nnet wi t h a
program, recipe or other kind of instructional document.
Q1 NOT from the stars do I my judgment pluck,
And yet methinks I have astronomy;
But not to tel l of good or evi l luck,
Of plagues, of dear ths, or seasons' qual i ty;
Q2 NOR can I for tune to brief minutes tel l ,
Pointing to each his thunder, rain, and wind,
Or say wi th princes i f i t shal l go wel l
By of t predict that I in heaven find.
Q3, volta BUT from thine eyes my knowledge I derive,
And, constant stars, in them I read such ar t
As truth and beauty shal l together thrive
If from thysel f to store thou wouldst conver t:
couplet OR else of thee this I prognosticate,
Thy end is truth's and beauty's doom and date.
EXERCISE ONE: IMAGERY AND
Key c o nc e p t: “ d imens io ns. ”
In this context, dimensions are qual ities that can be used to
structure a piece of information. (Concept from Edward Tuf te)
Dimensions could include anything that can be quanti fied or
g ro u p e d, fo r example , “ d u r at io n ”, “ d i st an c e , ” “we at h e r, ”
“ v i s io n. ”
A dimension is useful for identifying structure and pattern in
poems – fo r exampl e, t h e “ p rimar y d ime ns io n” wo u l d b e
a n a l o g o u s to t h e “ c o n c e p tual me t a ph o r ” t h a t h e l ps s t r u c t ur e
Students identified and came up with sample visual
representations for each dimension (e.g. clock=time,
eye=vision), and then created an object that expressed the
poem in some way.
#1: IMAGERY. ALTERNATE FORTUNES
INSIDE A CREEPY FACE BOX.
EXERCISE TWO: STRUCTURE AND
Go through the project l ine-by- line, and come up with an
object made of precisely 14 par ts, that shows how the poem
Could represent the rhyme scheme, or other features of
grammar or scansion ( leads to a discussion of what a unit is,
& how one might measure or represent it) .
Designed to show how a sonnet is constructed as a form,
irrespective of its par ticular content/message. .
But at the same time, to show how that content might be
integrated into the structure (e.g. where is the volta, and what
i s t h e “ t u r n ” i n t h e me a n i n g o f t h e p o em?)
EXERCISE 3: ENCODE/DECODE
Key concepts: encoding schema, lossy/lossless
A schema: a set of rules or agreed-upon language that is used
to encode a piece of text
Lossy & lossless: Are you going to encode the whole poem, or
just key par ts of it?
The assignment: choose an encoding schema, use it to
translate the poem into another format, and then provide a
“ d e c o de r. ”
The encoding could be of the structure of the poem (ie
quatrains, iambs etc) or of the words themselves
EXERCISE 4: ALGORITHM
C r e a te a “ p ro g ram” t h a t wi l l “ b u i ld” t h e p o em wh e n exe c ute d .
T h e p ro g r am i s ex p l ain e d a s a k i n d o f “ r e c ipe , ” wh i c h h a s t h e
benefit of several key computing concepts:
Procedure: series of instructions
Function (smal l procedure that can be repeated over when
(maybe, stretching the analogy a bit) : objects, smal l
p r e a ss emble d “ i n g r e die n t s” t h a t c a n b e c omb i n e d
(VOLTA): IS THIS ELECTRONIC
LITERATURE OR NOT?
HOLDING THE LIGHT
T h i s c o nfe r enc e’ s c a l l wa s a b o u t “ h o l d ing t h e l i g h t” o f
electronic l iterature, and brings up a number of questions:
Wh a t’ s e l e c t ro n i c l i te r atur e a nyway ?
Is it necessarily beholden to a computer?
I f n ot , wh a t ma ke s i t d i f fe r e n t f rom “ n o n- e l e c tro n i c l i te r atur e ”
( i f there is such a thing)?
S h o u l d we b e “ h o l din g t h e l i g h t” a n d d r awi n g c l e a r b o u n d a r ie s
a ro u n d wh a t we c o n s i d e r “ e l e c t ro n i c”?
Wh a t h a p p e n s wh e n “ e l e c t ro n i c” i s i n te g ra te d i n to u s a n d o u r
environment, so that we are no longer working with screens
and input devices? Wi ll it sti l l be electronic l iterature, or just