I’m going to be straight with you. The talk you’re about to hear differs substantially from the one advertised in your programme.
I’m not from around here. Until to a few months ago, I had never worked in a gallery, library, archive or museum. I was actually trained as a human geographer. My graduate studies involved designing data structures to handle vaguely defined, dynamic geographic institute. After University I worked at a CRI building tools for scientists to visualise all manner of environmental information… and of course I got into the web. These are the sorts of things I have done up until now…
I raise all of this in part to let you know who am I and where I come from, but also because I want to give you a sense of the strange process I’ve gone through in preparing for this talk. As you’ll see later in this presentation, preparing for the National Digital Forum has totally turned my thinking upside down. So this is what I did…
Now I work for DigitalNZ.
DigitalNZ is an initiative that aims to make New Zealand digital content easier to find, share and use. We harvest metadata describing digital items from around about 150 partner organisations. This includes things like photographs, paintings, newspaper clippings, old maps, aerial photographs and publications. We store this diverse metadata in a common form and make it available through a variety of search tools and also through a developer API. My role within DigitalNZ involves looking after the development and day-to-day operations of our technical infrastructure as well as assisting the NZ developer community make use of our API.
My original plan was to create a set of DigitalNZ visualisations . I wanted to understand what exactly are the 27 million pieces of New Zealand digital content we link to. So, I started asking myself questions like &quot; How are various images, articles, videos, manuscripts and so on that DigitalNZ indexes distributed over time? “. “Are things evenly distributed?, “Do things start slow with a steady ramp up to the modern day and then an explosion of contemporary digital imagery and video”? Being a geographer, I am naturally curious about space. “ Where are the items located in geographic space? Are they clustered together? Where are the holes? “ And so I started building these things.
This histogram, for example. It displays the volume of items we index organised by the year the content was originally published. I've restricted the figure to only show items that were created between 1800 and 2011. You should keep in mind that not every item of content that we index has a publication date in its metadata. If we don't know when the item was originally published then it is not represented in this chart. So, this is what I see. DigitalNZ has metadata describing a large volume of digital content that builds steadily from the 1850s, reaches a peak in the early 20th Century, before dropping away in the mid-1940s. We can see another rise in content beginning at the start of the 21st Century. I was a bit confused as to what the mountain of data represents but when I showed this chart to a few DigitalNZ colleagues and they each said, &quot;Oh, that data mountain is obviously Papers Past&quot;.
Papers Past is a collection of digitised articles and images from newspapers and periodicals published between 1839 and 1945. It is an amazing treasure and by far the largest set of items indexed by DigitalNZ. This second chart visually distinguishes between Papers Past and all other items DigitalNZ collects metadata on that has a publication date.
Here’s the DigitalNZ dataset visualised in a different way. Only 45,000 items indexed by DigitalNZ have geographic coordinates. If we exclude Flickr images, the number drops to 9000. Those 9000 records are displayed on this map. The large majority of these geo-tagged items are from Te Papa, Christchurch Art Gallery, Ministry of Culture and Heritage and Ross Becker and Moira Fraser. I expected to see a tight cluster of records around New Zealand. However, I was surprised the large cluster of items in Europe. These represent the amazing efforts of the Christchurch Art Gallery to geocode their many artworks. I’ll take you in a little closer in a moment, but it’s worth mentioning that this view of the data is already proving useful. See that cluster of dots in the North pacific that looks suspiciously like an upside down New Zealand. Those are items where the latitude has been accidentally reported as a positive number instead of a negative value. Like 38 degrees north instead of 38 degrees south. Creating a cartographic view of the data makes it simple to
Here’s the distribution of items around New Zealand. I’m actually pretty impressed by the coverage. I had expected to see strong concentrations around the urban centres Initially I was dismayed to see so many points in the water, assuming they were geo-coding errors. However, once I hooked up pop boxes to the visualisation it became clear that they are specimens from Te Papa’s various natural history collections – like the Grey Petrel in the top left. Clicking around more reveals a wealth of corals, fish, shells, seaweeds, crabs and other organisms collected from the seas around Aotearoa.
Zooming in further still reveals even more detail. All the cities have dense clusters of geo-tagged items, but to my eyes Christchurch is particularly impressive. This is largely due to the collection of Christchurch Earthquake Photographs taken by Ross Becker and Moira Fraser. It’s an incredible set of imagery. These interactive maps are extremely rudimentary. Obviously there is a great deal that can be done to improve their legibility, to differentiate between different types of content and collections, and to pull different patterns out. I would be curious to hear from people after this talk whether they would be interested in exploring any of these ideas further.
… so I got that far and, well to tell you truth I got a little dismayed. There is certainly some interesting stuff in there, and the DigitalNZ team genuinely learnt some things about the nature of the data that we index.
The good news is that I could do everything with just the API. Although I have special privileges to tap directly into our metadata database, this proved unnecessary. It makes me happy that the DigitalNZ developer API is flexible enough to support these sorts of query.
Here’s what got me down. I was uncovering useful information about the scale and structure of the metadata we index. There was certainly more that I could do too. However, I started getting concerned that I was just giving the view from a 60,000 feet and not saying anything about what things look like on the ground. This is rich stuff. These are the stories of our lives. These digital artifacts describe people’s dreams and hopes and desires. They tell of civilizations meeting for the first time, of conflicts won and lost, of cities rising and falling. To boil all that stuff down to lines on a histogram or dots on a map… well it seems like a bit of a lost opportunity.
… and then I saw something that knocked my socks off. As many of you are aware, for the last couple of years DigitalNZ, with the help of many sponsors, have organised a remix and mash-up competition called Mix & Mash. The competition entrants create amazing things. This year there was one entry that blew me away. Candy Elsmore created a video called “A Grand Mother”.
She wanted to convey the feelings of pride and delight that her family felt when they discovered that her great grandmother, Mary Edith, had signed the Suffrage Petition at the age of 21. The video begins with an empty table laid out for tea. Over four minutes, Elsmore tells her ancestor’s story by placing printed images on the tablecloth. It begins with a portrait of Mary Edith.
She adds an 1885 image of her birthplace Grove Town, Blenheim from the Auckland Art Gallery and then a portrait of her father from The Cyclopedia of New Zealand, now made available through the New Zealand Electronic Text Center.
Over time the table accumulates more images that describe Mary Edith’s world. Two and a half minutes in, Elsmore highlights her grandmother’s name on a digitised copy of the 1893 Suffrage Petition, made available by Archives New Zealand for the Mix & Mash competition. She notes that Marry Edith was one of 192 women to sign the petition in the Blenheim area. She was the only member of her family to do so – neither her mother or older sister signed.
The video ends with images of Mary Edith’s many descendents and a hand written note… “how cool is that?” In case you can’t tell, I think that you should definitely check out this video. It’s remarkable. So *I* watch this video and I can’t stop thinking about it. …and then I remember, of all things, a tweet a saw a while back.
Back in August, Max Ogden, an American developer and open web advocate a tweeted this: “I would love for ‘small data’ to be the ‘slow food’ of the internet. At the time I thought, “Right on. That’s a pretty cool thing to say.” … and then I did what most of us do on Twitter when we see something poignant. I retweeted, I even favourited it… and then I forgot all about it. Until I saw Candy’s video. The two seemed to be related somehow. Slowly, Ogden’s tweet began to make a strange sort of sense to me… Please indulge me in a small diversion. I promise you, it’s going somewhere.
Let’s consider for a moment some of the things that slow food is about.
One thing that slow food is about is forming and sustaining seed banks to preserve heirloom varieties.
There is something special about old varieties of fruits and vegetables that makes people want to continue to grow and share them.
Slow food is about preserving and promoting traditional food products.
There are ways of preparing food that have been handed down for generations. To take part in these practices is to touch history.
Doing it yourself is empowering. Just you, your tools and a vision.
So all this was sloshing around in my brain and I started thinking about this talk and the act of creating all these different visualisations. You know… what it says I was going to do in your programmes.
… and I thought, “What if we turned the problem on its head?” Instead of me coming in and creating these massive visualisations of content that I can’t make nuanced statements about, what if we instead we give people tools to create small personal visual stories? So I am going to do something that I am very nervous about. I am going to show a live demonstration of web tool that I built a couple of weekends back. This is pretty raw, but I am want to get people’s thoughts on it.
Visual explorations of New Zealand’s digital heritage
Visual explorations of New Zealand’s digital heritage <ul><li>Chris McDowall, DigitalNZ </li></ul><ul><li>National Digital Forum </li></ul><ul><li>Te Papa, Wellington </li></ul><ul><li>November 29, 2011 </li></ul>
Stall advertising Taradale Wines at an A & P Show, Stratford McAllister, James, 1869-1952 :Negatives of Stratford and Taranaki district. Ref: 1/1-009787-G. Alexander Turnbull Library, Wellington, New Zealand. http://beta.natlib.govt.nz/records/22778857