Data types for the
Real World ™
Eric Torreborre
YOW LambdaJam 2015
Yaron Minsky Make illegal state
unrepresentable!
Make illegal state
unrepresentable!
Make illegal state
unrepresentable!
Make illegal state
unrepresentable!
Real W
orld?
Constraints
CS
Data
structures
“File line”
Errors
interpret
Errors
Data
structures
Non-empty list
CS
Parser
Real World?
S3 EC2 EMR
sync
sync
sync
storage
execution execution
sync
storage
execution
“Local”
storage
execution
“Local - test”
sync
storage
“Hadoop”
execution
sync
storage
execution
“Hadoop - test”
sync
storage
“Hadoop – half sync”
execution
main
Location
Run application
“sync” files
download?
where?
how?
upload?
“sync” files
sync dir valid?
sync
sync
sync
storage
execution execution
Execution location
“sync” directory
unrepresentable!
“sync” locations
download?
where?
location execution
local local
hdfs
S3 local
hdfs
hdfs local
hdfs
Synchronized location
createSynchronizedLocation
9 cases
unrepresentable!
synchronize
efficient!
Locations
Location
ExecutionLocation
SynchronizedLocation
Datatypes are
cheap!
Make illegal state
unrepresentable!
Make illegal state
unrepresentable!
For the Real World ™