Presentation by Gordon Bell, Principal Researcher and Microsoft Research Silicon Valley Laboratory. The title is "Where's all that data? What's it good for?". This was presented at our Fujitsu North America Technology Forum 2012, held in Santa Clara, CA on Jan. 25th, 2012. The theme of the event was "From Sensor Networks to Human Networks: Turning Big Data into Actionable Wisdom"

1. Where’s all that data? What’s it good for?
Gordon Bell
Microsoft Research
Silicon Valley Laboratory
Fujitsu 5th Technology Forum
From Sensor Networks to Human Networks:
Turning Big Data into Actionable Wisdom
25 January 2012

2. Where do you get all those bits?
Some Stories…
• World’s commercial transactions
• The Cloud
• Personal lives from recording everything (MyLifeBits)
– Individuals
– Social sites
– Libraries e.g. Mormon Library for preserving member archives
• Fourth Paradigm of Science based on data
– Our world is being instrumented for observing everything
• Monitoring the earth and water for energy, food, and
pleasure

3. Courtesy of Barabba (or Steve Haeckel, IBM) or someone else

4. Commercial People & Science & Real time,
Transactions All Their Bits 4th Paradigm Real World
Sense & Effect
Courtesy of Gordon Bell, Barabba, Steve Haeckel, IBM and probably someone else

5. Lifelogging
With extreme lifelogging, all of us will have the
ability to recall or have recalled everything
we’ve ever said, saw, and did
… just like today’s Political candidates
Are people basically narcissistic?

6. My five lifelogging epiphanies
1. Its capture and digitization (1998)
2. It’s organization and recall (2001)
3. It’s a transaction processor for everything in and
about your life (2005)
4. It’s your true e-memory(2007)
Bio-memory is just the meta-data and URL for e-Memory
5. Your e-memory is everywhere and beyond your
control (2011)

7. The challenge now
With extreme lifelogging, all of us will have the ability
to recall or have recalled everything we’ve ever said,
saw, and did
… just like today’s Political candidates
The Challenge:
Collecting the bits from the individuals.
• Where are the bits?
• Can they be recalled?
• Who owns them?
• How much does it cost to store forever?

8. Let’s look at individuals
& their e-memories

9. I’m losing my mind

10. THE ULTIMATE DIARY
WHAT IF YOU COULD REMEMBER
EVERYTHING?
“Dalgliesh, he knew, had almost total recall.”
-P.D. James, Death of an expert witness

11. Everything you’ve ever read

12. Everything you’ve ever seen

13. Everything you’ve ever heard

14. And much more… your very state
• Location, bio, temperature, light level… sensors galore
• Your heart beat, blood pressure, stress level, etc.

15. As little or as much as you like
Certainly much more than ever before
IF YOU WANT, YOU CAN HAVE
TOTAL RECALL

16. Recording, Storage, Recall
THREE STREAMS OF
TECHNOLOGY
LEAD TO TOTAL RECALL

17. Storage: cheap and abundant
10000
1000
Gigabytes (GB)
PC (3.5")
Notebook (2.5")
100
PDA (2")
Cellphone (Flash)
10
1
2000 2005 2010 2015

19. Recall: search, analyze & present

20. Next 10 years will see revolution of life and society
TOTAL RECALL IS INEVITABLE
A TOOL OR THE ULTIMATE DIARY?

21. We think life-Blogging is nuts!
NOT LIFE-BLOGGING

22. Personal and private
LIFE-LOGGING NOT LIFE-BLOGGING

23. Recording/Sensors

24. Why just paper?

25. Memex
As We May Think, Vannevar Bush, 1945
“A memex is a device in which an individual stores all his
books, records, and communications, and which is
mechanized so that it may be consulted with exceeding
speed and flexibility”
• Full-text search, text & audio annotations, and hyperlinks

26. Recording Everything

27. Bits per person…
A One Terabyte, Low Resolution Life
• 2000 VL res life can be stored in a TB (GB/month)
• 2005 MyLifeBits captured about 1 GB per month…
– Very little audio, video, lower resolution photos
– Web page, photos, and video takes up the space
• 2010 10-20 Terabytes is more realistic
– SenseCam 3 GB/month 3 samples/minute
– Audio 17 GB/month

28. Special Persons Archives
• Charles Vest: former President of MIT
• Einstein
• National Lib. Of Medicine: Lederberg
• Salman Rushdie
• LDS Church (Mormon)

29. Public 21st century figure legacies
• Charles Vest, president of MIT from 1990 to 2004,
delivered a hard drive with nearly all of the files of
his 14 year tenure to the MIT Archivist. It including
speeches and letters (drafts), presentations,
planning documents, meeting minutes, e-mails
and a few photos. The only items Vest had deleted
were a few files about his personal finances.
• Nothing had been scanned, so no incoming input
such as letters, web page views, articles, unless
they were attachments.

30. www.alberteinstein.info

31. www.alberteinstein.info

32. National Library of Medicine
• Top 30: 265 GB; 181K Files; 1.5 MB/file
– 99.99% tiff images
– less than .01% plain text
– less than .01% html files
– less than .001% AVI video
• Web derivative files 36 GB; 75K files; 0.5 MB/file
• Items Pages Video Who
18,615 49,951 8 Lederberg
1,738 37,110 25 RMP
1,054 10,811 5 Koop
580 4,374 1 Avery
469 1,833 - Crick
279 1160 1 Pauling
302 893 - Varmus
…
27.5 K 143 K

33. Lederberg Finder page

34. Lederberg papers official
reports
Number of document segments

35. Emory University: Rushdie Archives
Snooping Through Salman Rushdie's Computer
http://www.youtube.com/watch?v=pBtFNpgzlsg
• 200 cardboard boxes;
• Four emulated MACs; 18 Gbytes; 40,000 files

36. SenseCam: Recording everything seen

37. GB with
SenseCam &
Voice
Recorder
Camera now
available:
Viconreview.com

38. Capturing
every step

40. The “killer app”… Health!

41. Capturing every heartbeat
• 72.6 beats/min; 38.16 Million beats/year
• 3.13 billion beats per life
• Battery life: the expected time to next surgery!
– St. Jude battery was 4-4.5 years, or ETS
– Medtronic current, 8 years.

43. Audiometric
Test 050117

44. Sensors with IP On Everything

45. HR, weight, BP, distance,

46. Health monitoring devices

47. In-body health sensing
pillcam Nanobot in the bloodstream
EndoSure Wireless Pressure Sensor
in an aneurysm sac

48. Navigenics
Report
2006-03

49. Health
Monitoring:
“Your
husband
just died,
… here’s
his black
box”

50. 100
75
50
25
0
Work: email, im, social sites
Work: Who & When
Work: Legacy documents
Work: Web pages
>Work: T&M (VIBE)
>Work: Meetings
>Work: Telephone…
Home: Finance, Legal
Learning: Books, journals, etc.
Health: PHR
>>Health: Diet & Exercise
Health: On & inbody metrics
Life: Music (CDs, cassettes,…
Life: Photos
Life: Memorabilia, ephemera
>Life: Tracked Days
Life: Video Productions
Life: SenseCam Days

51. Bits per person…
A One Terabyte, Low Resolution Life
• 2000 VL res life can be stored in a TB (GB/month)
• 2005 MyLifeBits captured about 1 GB per month…
– Very little audio, video, lower resolution photos
– Web page, photos, and video takes up the space
• 2010 10-20 Terabytes is more realistic
– SenseCam 3 GB/month 3 samples/minute
– Audio 17 GB/month

52. A View of Preserving Digital Lives
• Preserving the analog life of a 20th century person:
10-100 GB. 2 Mpgs, 100Kimages, 100-1,000 hrs. video
– Won’t analog people need to be converted to digital,
… if not they’re really gone and forgotten history?
– Gresham’s Law: digital lives drive out analog lives
• How will a 21st century, digital person be preserved?
– Which “lives” of a person e.g. personal, professional?
– Depth of each life?
– Size. Who’s in a library’s digital lifeboat?
• Preserving Everybody?
– Role of public institutions vs. the cloud for “all of us”

53. Fire in the Library
Technology Review January 2012

54. How far do we trust our institutions to save lives?
• Re a comment on NPR in late January
http://www.npr.org/templates/story/story.php?storyId=99372779
about people saving recordings of early pre-bluegrass American folk
music:
• "He considered giving his collection to the Library of Congress, …
Alden says he worried that they'd be hard for musicians … to access,
and that they'd gather dust lying …, what librarian … would let
someone into the stacks with a banjo or a fiddle …?"
• … they're burning CDs and shipping them all over, which is the "lots
of copies keeps stuff safe" philosophy (www.lockss.com). They
haven't taken the next step and put them online, and anyway don't
have a virtual place to put them that has a good chance of surviving
and caring for them in perpetuity.

55. Scientific Data Deluge
• CERN detectors
• Radio telescopes
• New telescopes and observatories
• Gene sequencers
• Global weather sensors
• Earth science sensors

56. Science Paradigms
1. Thousand years ago:
science was empirical
describing natural phenomena
2. Last few hundred years:
theoretical branch
using models, generalizations .
2
a 4G c2
3. Last few decades (FORTRAN):  a   3  2
  a
 
a computational branch
simulating complex phenomena
4. Today Data-intensive science :
data exploration (eScience)
unify theory, experiment, and simulation
– Data captured by instruments
Or generated by simulation
– Processed by software
– Information/Knowledge stored in computer
– Scientist analyzes database / files
using data management and statistics
Jim Gray NRC-CSTB 2007-01

57.  Make sure the scientists have a data
problem – otherwise they won’t take the
time to talk with you
 Define 20 questions/plots – this drives
the technical design, but also helps the
cross-disciplines communication
 Spread the 20 questions/plots across
“easy”, “tricky”, “too hard to do now”
 Ask about sharing and security and get
to shared pragmatic consensus
 Don’t forget to write the papers on both
sides - they help drive adoption
Courtesy Catharine van Ingen

58. Synthesizing Imagery, Sensors, Models
and Field Data
Climate classification FLUXNET
~1MB (1file) Curated
sensor
dataset
30GB
(960 files)
Vegetative clumping
NASA MODIS imagery archives FLUXNET
~5MB (1file)
5 TB (600K files) curated field
dataset
2 KB (1 file)
Sizes given are 1 US year
20 US year ~ 1 global land
surface year
NCEP/NCAR ~100MB
(4K files)

64. Global Scale Global Scale Archive
Continental US Reprojection Reduction Download

66. By the numbers….
• 22 months • 1.3 M re-projected tiles
• 2 CS interns; 1 architect; • 25 M reduction files
1 science intern; 1 • (TBD) VM
senior scientist; 3 scaleup/scaledown
hangers-on operations
• 522 K cpu hours • (TBD) Lines of
• 14 TB upload (nonMatLab) code
• 10 TB max storage • $79K external billing
• 5 TB download
• 2.3 B storage operations

67. The South Esk Hydrological Sensor Web:
Next-Generation Catchment Management
Water for a Healthy Country
Andrew Terhorst
Tasmanian ICT Centre (Hobart WSM real time) award
winner
9 September 2011

68. The sustainability challenge …
• Australia is the driest inhabited continent
• River flows can be extremely fickle/unreliable
• Sustainable management of freshwater resources
FLOOD EARLY WARNING
requires good situation awareness
WATER HYDRO-POWER
REGULATIONS GENERATION
REQUIRES
RESERVOIR GOOD WATER
MANAGEMENT SITUATION QUALITY
AWARENESS
WATER ENVIRONMENTAL
TRADING FLOWS
IRRIGATION PLANNING
2011 iAwards - Sustainability and Green IT

69. South Esk River, Tasmania
• Catchment receives variable
rainfall - river flows are very
erratic
• Water resource managers
require better situation
awareness for managing
water restrictions
• Sustainability goal is to
maximise water harvesting
opportunities without
compromising environmental
flows
2011 iAwards - Sustainability and Green IT

70. Hydro-meteorological sensor network
2011 iAwards - Sustainability and Green IT

71. Integrating sensor data from multiple agencies
2011 iAwards - Sustainability and Green IT

72. Project goal
Develop a prototype
water information
system made up of two
linked sub-systems:
• Continuous flow
forecast system
- Based on emerging
Sensor Web standards
• Provenance
management system
- Provides information
on how flow forecasts
are produced
2011 iAwards - Sustainability and Green IT

73. Current practice
Decision
Numeric Application Layer
Support
Models
Tools
Physical Sensors, Observation
Sensor Layer
Archives
2011 iAwards - Sustainability and Green IT

74. Paradigm shift
Decision
Numeric Semantic Application Layer
Support
Models Broker
Tools
Sensor Web Services Layer
Physical Sensors, Observation Archives Sensor Layer
2011 iAwards - Sustainability and Green IT

75. Architectural framework
Network Management
And Provenance
Scientific
workflow
Sensor data Atmospheric
feeds Clients
models
Flow forecast
models
2011 iAwards - Sustainability and Green IT

76. 2011 iAwards - Sustainability and Green IT

77. Key system features
Interoperable
Provenance
Highly
management Re-locatable
scalable
First hydrological
sensor web built in Redundancy Rapid
Australia integration of
Uses near Open sensor assets
real-time data
feeds from
Unique Architecture
Standards-based
multiple
agencies Improved
Key understandin
Reusable software of natural
Features components system
Published behaviour
research articles
Value
Quality Enables
Proposition
sustainable
Included in the management of Generic
Global Earth scarce water applications
Described as next- Serves regulators
Observing System of resources
generation water and community
Systems Provides economic
information system in Serve other purposes benefit to irrigators
implementation pilot
ITU technology briefing e.g. flood warning,
fire-danger risk
2011 iAwards - Sustainability and Green IT
assessment

78. The end