encoding for the imagination Douglas Goodwin 03/04/2013Monday, March 4, 13encoding for the imagination,doug goodwin, march fourth, 2013
encoding strategy: Parrhasius’ grapes Zeuxis’ grapes fooled nature Parrhasius’ grapes fooled Zeuxis!Monday, March 4, 13According to the Naturalis Historia of Pliny the Elder, Zeuxis and his contemporary Parrhasius (of Ephesus and later Athens) staged a contest to determine the greater artist. When Zeuxis unveiled his painting of grapes,they appeared so real birds ﬂew down to peck at them. But when Zeuxis asked Parrhasius to pull aside the curtain from his painting, the curtain itself turned out to be a painted illusion. Parrhasius won, and Zeuxisadmitted, I have deceived the birds, but Parrhasius has deceived Zeuxis.
overview 1. how does digital video represent an image? 2. how can we tap into the system that lies between stimuli and sensation? 3. how does video take advantage of imagination and memory? 4. the grapes will soon be forgotten. will we starve? 5. Lossless #3 & #4 6. q&aMonday, March 4, 13overview
how does digital video encode an image? 1. cull what cannot be seen (ex: everything below a certain luminance value) 2. divide the x/y frame into a matrix of macroblocks 3. analyze temporal diﬀerence between current frame and the next frame for each block (motion analysis) 4. if temporal diﬀerence < n then retain the current block 5. split the x/y frame luminance and chrominance 6. throw out a lot of chrominance (4:2:2) 7. reorder the frames for eﬃcient storage 8. transform the frame into waveforms and then digitize them.Monday, March 4, 13MPEG Compression & how it works in IPTV and other digital media delivery systems.http://youtu.be/P7abyWT4dss
we call this “lossy” compression because so much of the image is discarded in the encoding process.Monday, March 4, 13overview
are you missing that image data?Monday, March 4, 13overview
what can you oﬀer to replace it?Monday, March 4, 13overview
what do I mean by that? we may be enticed to forget about the high-resolution image. how? 1. there may be images that suggest the image without showing it, 2. images that please us more than the grapes, 3. and/or images that go around the problem altogether.Monday, March 4, 13overview
Monday, March 4, 13here is an example. This image contains a symbol within it. once seen it is not easily forgotten. and more to our point: once seen it does not need to be shownagain.ready?
Monday, March 4, 13once seen, it can never beunseen
Monday, March 4, 13do I ever need to show you the arrowagain?
what is this? how do you know?Monday, March 4, 13how do you interpret this image? After you find an anchor point your eyes track to one eye, then to the other, then to the mouth, back to the eye, then up to theforehead, along the hairline to the side of the cheek, down to the chin, etc.
your eyes travel in a predictable pattern when you look at faces. what could you do with that?Monday, March 4, 13you have done this before. Your saccades have travelled down this road. That’s how you know it’s a face.
Aoccdrnig to rscheearch at Cmabrigde Uinervtisy, it deosnt mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.Monday, March 4, 13can you read this? did you need to think about it?could a machine?where is the processing that makes this possible?does it suggest ways you might hack the perceptual pipeline?
placing the viewer with representational styleMonday, March 4, 13
Monday, March 4, 13example from scott mccloudalso: mcluhan’s hot vs. cold media
The bedroom window—the one nearest the hallway— opens outward. The upper part of A .. ,s body is framed within it. She says "Hello" in the playful tone of someone who has slept well and awakened in a good mood; or of someone who prefers not to show what she is thinking about—if anything—and always ﬂashes the same smile, on principle; the same smile, which can be interpreted as derision just as well as aﬀection, or the total absence of any feeling whatever. Besides, she has not awakened just now. It is obvious she has already taken her shower. She is still wearing her dressing gown, but her lips are freshly made up—the lipstick color the same as their natural color, a triﬂe deeper, and her carefully brushed hair gleams in the light from the window when she turns her head, shifting the soft, heavy curls whose black mass falls over the white silk of her shoulder. She goes to the heavy chest against the rear partition. She opens the top drawer to take out a small object and turns back toward the light. On the log bridge the crouching native has disappeared. There is no one visible around the house. No cutting crew is working in this sector, for the moment.Monday, March 4, 13robbe-grillet, jealousy“objective” language
This room was too big, the ceiling was too high, the doors were too tall, and the white carpet that went from wall to wall looked like a fresh fall of snow at Lake Arrowhead. There were full- length mirrors and crystal doodads all over the place. The ivory furniture had chromium on it, and the enormous ivory drapes lay tumbled on the white carpet a yard from the windows. The white made the ivory look dirty and the ivory made the white look bled out. The windows stared towards the darkening foothills. It was going to rain soon. There was pressure in the air already. I sat down on the edge of a deep soft chair and looked at Mrs. Regan. She was worth a stare. She was trouble. She was stretched out on a modernistic chaise-longue with her slippers oﬀ, so I stared at her legs in the sheerest silk stockings. They seemed to be arranged to stare at. They were visible to the knee and one of them well beyond. The knees were dimpled, not bony and sharp. The calves were beautiful, the ankles long and slim and with enough melodic line for a tone poem. She was tall and rangy and strong-looking. Her head was against an ivory satin cushion. Her hair was black and wiry and parted in the middle and she had the hot black eyes of the portrait in the hall. She had a good mouth and a good chin. There was a sulky droop to her lips and the lower lip was full. She had a drink. She took a swallow from it and gave me a cool level stare over the rim of the glass.Monday, March 4, 13raymond chandler, the big sleep