Living with Tableau
Machine: A longitudinal
investigation of a curious
domestic intelligence
Zachary Pousman, Georgia Tech
Mario Romero, Georgia Tech
Adam Smith, UC Santa Cruz
Michael Mateas, UC Santa Cruz

Living with Tableau
Machine: A longitudinal
investigation of a curious
domestic intelligence
Zachary Pousman, Georgia Tech
Mario Romero, Georgia Tech
Adam Smith, UC Santa Cruz
Michael Mateas, UC Santa Cruz

Everyday domestic life
Capture it (Computer vision)
Interpret it (Artificial Intelligence)
Represent it (Generative Graphic Design, HCI)

Conversation, reflection

A mirror of life
&
A social ‘actor’

Say hello to Tableau Machine

Tableau Machine
System Implementation Overview

Tableau Machine
System Implementation Overview
Energy
Density
Flow

Tableau Machine
System Implementation Overview

Quiet times
(watching TV, reading...)

Active times alone
(vacuuming, exercising...)

Social times
(family meals, homework, games...)

Wild times
(play, fighting, parties...)

Investigating

Tableau Machine
Longitudinal Investigation
Household A
• Three households in Atlanta GA
• 6-8 weeks
• Elicitation techniques:
• Interviews
• Feltboard Household B
• Wordgame
• Print outs
• Broken (misconfigured) TM
Household C

The Broken TM Household A

The Broken TM Household A
Trajectory of Appreciation, Gaver et al. 2005

The Broken TM Household A
Trajectory of Appreciation, Gaver et al. 2005
“Natural Experiment”

Experiments
Andrew: No pictures are ever two the Household A
same. I’ve already tried to do that.
Waving at the cameras (kids)
Interviewer: Is that right? You tried to do
that? How did that work? Turning off and on lights
Andrew: ... So yeah, no, there was no one
else in the house. So I just sat right here
and I was just [he mimes holding still],
and I was looking at it and looking at it
and... Household C
But I could never get it, the same exact
image. ... Well that’s what I’m trying to Waving at the cameras (kids)
figure out, if it’s taking numbers, images, Wailing on the floor while mom
from [the cameras], then why is it not the watched from the dining room
same picture every time? You know?
Household A

Household C
‘NIKO’

Printing
Household B

Printing
Household A Average 1.46 / day
Household B Average 3.16 / day
Household C Average 12.64 / day

Deepening of
Reflection
Household C

(Hints of) Personality
Household C

Feelings of being watched
- All 3 Mothers changed behaviors
- Aloneness as privacy
Household A
Household B
Charlie: [pointing up] his spies...
Household C
Carla: Yeah, he calls it Niko and his spies.
Interviewer: Really, ‘Niko and his spies’?
Carla: Yeah.
Household C

Implications for Ubicomp design
Co-interpretation

Implications for Ubicomp design
Activity Recognition + Activity Characterization
Household Management Household life
Indoor Activities Outdoor Activities
Attribute you care about
Cleaning Yard work
Cooking Exterior maintenance
Attribute you care about

Implications for Ubicomp design
Activity Recognition + Activity Characterization
Household Management Household life
Indoor Activities Outdoor Activities
Attribute you care about
Cleaning Yard work
Cooking Exterior maintenance


Attribute you care about

Implications for Ubicomp design
Cameras in the home

Thanks! Questions?
We acknowledge the support of NSF
IIS-0752684. We also want to thank our
participant families, our advisors (John
Stasko and Gregory Abowd), draft readers,
reviewers and shepherd.
Zach Pousman : zach@cc.gatech.edu
Mario Romero : mromero@cc.gatech.edu