Topic: Secret Life of a Weather Datum project - presentation of research findings related to the Old Weather citizen science project. A lecture given to PGT students of the Archives and Records Management module at the Information School, University of Sheffield, April 2015.

1. From Archives to Climate Science:
a story of pirates, old weather, serendipity,
data rescue, lovelorn sailors, perseverance,
community values, and clearing the fog…
Paula Goodale
Research Associate, Information School
The Secret Life of a Weather Datum
@PaulaGoodale, @lifeofdata
with thanks to
Jo Bates (iSchool), Joan Arthur (Old Weather)

2. The Secret
Life of a
Weather
Datum
www.lifeofdata.org.uk

3. Romilly Close – Pi weather station designer (Aerospace Engineering UG student)
Fred Sonnenwald - Pi weather station project co-supervisor (Civil Engineering)
Sophie Rutter – Usability testing (Information School)
Jo Bates,
Information
School,
University
of Sheffield
Paula
Goodale,
Information
School,
University
of Sheffield
YuWei Lin,
University for
the Creative
Arts
Dave Mee
Tandot/The
Garden
#lifeofdata

4. The Secret Life of Data?
Big data, open data, managing data, data = oil
….Lots of talk about data!
Critical Data Studies
Dalton and Thatcher (2014): “in both its
production and interpretation, all data – ‘big’
included – is always the result of contingent
and contested social practices that afford and
obfuscate specific understandings of the
world”
How to capture the complex socio-cultural shaping of (‘big’) data infrastructures?
Where to begin? Where to end?
How to talk to people about it?

5. The Secret Life of a Weather Datum
Research Questions
1. What is the ‘journey’ that weather data produced by the UK’s Met Office
takes from its production through to its collation and re-use as ‘big’
weather data in different contexts?
2. What socio-cultural values and practices are articulated in the
transformation of this data on its journey from production through to
various contexts of collation, distribution and re-use, and how do these
socio-cultural values and practices themselves transform as they interact
with the data over the course of its journey?
3. What institutional policies and practices, and government policies and
legislation, shape the distribution and licensing of weather data for re-use
in different contexts?
4. How can the complexity of the socio-cultural dynamics shaping the
production, collation, distribution and re-use of ‘big’ weather data be
communicated to a wider audience?

6. Methods
Case studies and
participants
Semi-structured
interviews and
Observations
Documentary evidence
and Digital ethnography
Thematic analysis
Data production
• Weston Park weather station
• Met Office
• Amateur observers
• Old Weather
• Policy maker
Climate Science
• Met Office Hadley Centre
• Climatic Research Unit, UEA
• Intergovernmental Panel on Climate Change (IPCC)
• Met Office projects – ACRE and Old Weather
Financial Markets
• Weather Market Data supplier
• [Financial markets]
• Policy maker
Citizen Science
• Amateur observers
• Old Weather / Archives
• Met Office projects
- WOW and Old Weather

7. Methods
Case studies and
participants
Semi-structured
interviews and
Observations
Documentary evidence
and Digital ethnography
Thematic analysis
Towards ethnography
• Site visits
• Multiple interviews
• Observation
• Field notes
• Photographs
• Video
• Interviews - core themes
• Own role, data practices
• Motivations and challenges
• Relationships with other people and
organisations
• External environment – e.g. economic and policy
issues
• Tailored to individuals
• Observations – working environment
• Communal and personal space
• Branding and artefacts
• Personal interactions
• Technology

8. Methods
Case studies and
participants
Semi-structured
interviews and
Observations
Documentary evidence
and Digital
ethnography
Thematic analysis
Extending coverage and filling in gaps
• Official records
• Access restrictions
• Discourse
• Community
Documentary evidence
• Reports
• Policy documents
• Research papers
• Web sites
• Corporate publications
Digital ethnography
• Twitter
• IPCC conference
• Online forums
• Citizen scientists

9. Methods
Case studies and
participants
Semi-structured
interviews and
Observations
Documentary evidence
and Digital ethnography
Thematic analysis
Coding
• Manual
• 2-3 coders
Coding themes
• Attitudes, values and beliefs
• Valuable activities and practices
• Social relationships
• The data journey
Working papers
• One per case
• Analysis by coding themes
• By organisation
Cross-case analysis
• Web site
• Research papers

10. lifeofdata.org.uk
#lifeofdata

11. Old Weather Case Study: Data collection
• 1x interview ACRE project
• 1x interview Met Office climate scientist
• 1x interview Climate historian
• 2x interviews Old Weather members
• Content / discourse analysis of Old Weather forum
• Content analysis of documents and web pages

12. Old Weather: Climate Science Challenges
• To improve climate models you need more data, over
longer time periods, from more locations
“I usually say to people if you’ve got a weather
observation, if it’s south of the equator, we’ll take it.
Right. If it’s north of the equator before about 1950 —
we’ll take it. If it’s north of the equator, outside 1950,
and it’s not in North America, Europe — Japan, or the
North Atlantic — we’re probably still interested.“
[MO_03]

13. Pirates
• ACRE project – Atmospheric Reconstructions of the Earth
• A loose network of like-minded climate scientists engaged in data
recovery projects
• I often think about you know, the organisations work a bit like the
Royal Navy, we work like Pirates of the Caribbean. [MO_06]
• “It’s very difficult to get funding for data recovery because people
want scientific results. A lot of the funding bodies or people who
make these decisions don’t realise that unless you’ve got the data,
you’re not going to do any science, but they don’t want to fund
the recovery. As soon as you say data rescue, data recovery, they
switch off. “ [CRU03]

14. A Mission
• Challenge: knowing what’s available and
where it’s located
• “I compiled an inventory of every major Royal Navy
vessel from 1800 to 1950, where it was, what its
movements were, where its logbook is, and I also
compiled reports for them, which are available,
concerning logbooks generally, where they are in
the archives, what they consist of, what some of the
problems are in using them, this sort of
thing.” [CRU_03]

15. Serendipity
“My favourite research method is serendipity”
[CRU_03]

16. “When it says miscellaneous in my view it’s because it’s got lots of
stuff with numbers on and they didn’t know what to do with it.”
[CRU_03]

17. Data Rescue and Recovery
• Challenge – knowing the data
• Well, my expertise is finding the data, knowing how it was
collected, knowing some of the problems of the–, what I’ll do
is I’ll look through the logbooks, and I’ve got a lot of
experience, I’ve looked at thousands of them, and I
anticipate the problems they’re going to have with the
data. And sometimes I say, “You’re going to have this
problem. This is how you overcome it.” [CRU_03]

18. Data Rescue and Recovery
• Challenge - records may not be kept in ideal
conditions
• “it shows the stuff in archive is just piled up and it’s
all mouldering.”
• “I didn’t actually go there but it was--, it’s
horrendous. Things are just falling to pieces and it’s
just piled up on the shelves and it’s all… It’s
irreplaceable.” [CRU03]

19. Digitisation and Preservation
• …and the archivist looks at them and says, “How
good condition is this record?” You know, “Can
we just photograph it, or do we need to repair it
first? Is it going to fall to pieces on us? Does it
need curating?” So they do the curating as
necessary, and then we try and photograph it,
and that requires a bit of human expertise
because the quality of the ink after 150 years is
not always great. Okay. But usually we get
fairly legible photographs out of it. [MO_03]

22. Old Weather: Transcribing the logs
• Process
– Select a ship
– Transcribe the log book
• Weather data
• Notes
– Each one transcribed at
least 3 times
– Cross-check for errors
– Data sent to Met Office
for processing
• Challenges
– Hand-writing!
– Place names
– Can be dull / repetitive

23. There are some things computers
cannot do well…
From 4 to 6 light breezes from the S x W and cloudy - latter part
light rain – Air 80. Water 82

24. TWYS –
Type What
You See
Typed records are a rare treat!

25. Lovelorn Sailors
• Not just numbers… inventories, events, stories
• Attachments formed to historical people
• A rich source of community engagement
• Edited notes are captured for future reference
• Sharing with historians – www.naval-history.net

26. “Our interest in our subject, our own humanity for
our scribes and for each other. Produces a powerful
community, a family.” [Joan Arthur, Old Weather]

27. Community
• I think the sense of contributing to something that I
care about, but also definitely the forum. The forum is
massively important. [OW_02]
• “One of the other threads is ‘Signs of Old Weather
Addiction’” [OW1]
• Helping with transcriptions
• Sharing stories, fun and games
• Personal interactions, keeping up morale
• Recognition and information from the scientists
is also valued

28. Motivations
• “I think it is, yeah, you know it's the weather, it's the
history, and it's the forum I think for me are the sort
of the three key important things that have sort of
kept me interested in it really.” [OW_02]
• “I think initially it was because of the whole thing to
do with climate change. Because obviously, I've got a
concern for the planet, you know, and the way that
we treat it and so on. And so actually doing
something that was in some way going to contribute
to that kind of research attracted me.” [OW_01]

29. Reasons for Transcribing
Eveleigh et al, 2012

30. Serious Leisure: Volunteers
http://www.seriousleisure.net/slp-diagrams.html

31. Data Processing
• Transcribed data is sent to the Met Office
• Transformed into a standard IMMA data format
• Checked for errors
• Uploaded to ICOADS data centre for access by
climate scientists across the world
• Primarily used in ‘reanalysis’ projects
Reanalysis is a scientific method for developing a comprehensive
record of how weather and climate are changing over time. In it,
observations and a numerical model that simulates one or more
aspects of the Earth system are combined objectively to generate a
synthesized estimate of the state of the system. A reanalysis typically
extends over several decades or longer, and covers the entire globe
from the Earth’s surface to well above the stratosphere.

32. Old Weather Data in Climate Science
Video: National Maritime Museum, www.oldweather.org
http://player.vimeo.com/video/15388983

33. “My job is fog removal”
“But the yellow region… that's where because we did our citizen science Old
Weather Project, and we got new weather observations, there now isn't as
much fog as there used to be. So the yellow area, I call that the ‘glow of
discovery. That's where there isn't any fog any longer.” [MO3]
Video: https://vimeo.com/99917882

34. Other Archival Data
Climatic weather stations
- Weston Park, Museums
Sheffield
- More than 130 years of weather
data
- Collected and preserved by
generations of museum curators
- Archived by the Met Office and
by the Museum
- Used in climate science
- Available for research
- Public good – data and weather
station belongs to the people of
Sheffield

35. Other Archival Crowdsourcing Projects
• Transcribe Bentham
• BBC World Service archive
• TNA – Operation War Diary
• http://helpinghistory.com/
• Crowd-sourcing 101

36. References
• Dalton, C. & Thatcher, J. (2014). What Does a Critical Data Studies Look Like and Why Do
We Care? Seven Points for a Critical Approach to ‘Big Data’. Society & Space [Online].
Available: http://societyandspace.com/material/commentaries/craig-dalton-and-jim-
thatcher-what-does-a-critical-data-studies-look-like-and-why-do-we-care-seven-points-
for-a-critical-approach-to-big-data/
• Eveleigh, A. M. M., Jennett, C., Blandford, A., Brohan, P., Cox, A. L. (2014). Designing for
dabblers and deterring drop-outs in citizen science. CHI'14: Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems. ( pp.2985-2994). New York, NY,
USA: ACM
• Eveleigh, A. (2014). Crowding out the archivist? Locating crowdsourcingwithin the
broader landscape of participatory archives. In Ridge, M. (Ed.).Crowdsourcing our
cultural heritage ( ). Ashgate.
• Eveleigh, A., Jennett, C., Lynn, S., Cox, A. L. (2013). "I want to be a captain! I want to be a
captain!": Gamification in the Old Weather citizen science project.Gamification'13:
Proceedings of the First International Conference on Gameful Design, Research, and
Applications. ( pp.79-82). New York, NY, USA: ACM Press
• Stebbins, R. A. (1996). Volunteering: A serious leisure perspective. Nonprofit and
Voluntary Sector Quarterly, 25(2), 211-224.

37. Web sites
• Life of Data: http://www.lifeofdata.org.uk/
• Life of Data research data archive:
https://archive.org/search.php?query=secret%20life%20of%20
a%20weather%20datum
• Old Weather: http://www.oldweather.org/
• Old Weather Forum: http://forum.oldweather.org/
• 130 years of Weston Park Weather Station:
http://www.museums-sheffield.org.uk/blog/2012/9/130-years-
of-weston-park-weather-station

38. Thanks!
Paula Goodale
@PaulaGoodale, p.goodale@sheffield.ac.uk
The Secret Life of a Weather Datum
@lifeofdata, www.lifeofdata.org.uk