Managing and exploiting the digital deluge: issues, challenges and opportunities Michael Day
The digital deluge - outline <ul><li>Understanding the scope of the problem </li></ul><ul><li>Some challenges </li></ul><u...
The digital deluge - what is it? (1) <ul><li>A phrase applicable in more than one context: </li></ul><ul><ul><li>The netwo...
 
 
The digital deluge - what is it? (2) <ul><li>International Data Corporation (IDC) White Paper: </li></ul><ul><ul><li>Estim...
The digital deluge - challenges (1) <ul><li>Problems of scale: </li></ul><ul><ul><li>Can our infrastructures begin to cope...
The digital deluge - challenges (2) <ul><li>Problems of complexity: </li></ul><ul><ul><li>Many different types of digital ...
The digital deluge - challenges (3) <ul><li>Organisational problems: </li></ul><ul><ul><li>Lack of co-ordination between s...
The digital deluge - opportunities <ul><li>Some opportunities: </li></ul><ul><ul><li>While many curation challenges remain...
Thank you for your attention
Upcoming SlideShare
Loading in …5
×

Managing and exploiting the digital deluge: issues, challenges and opportunities

1,330 views

Published on

Presentation, University of Bath, 5 June 2008

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,330
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Managing and exploiting the digital deluge: issues, challenges and opportunities

    1. 1. Managing and exploiting the digital deluge: issues, challenges and opportunities Michael Day
    2. 2. The digital deluge - outline <ul><li>Understanding the scope of the problem </li></ul><ul><li>Some challenges </li></ul><ul><li>Opportunities for researchers </li></ul>
    3. 3. The digital deluge - what is it? (1) <ul><li>A phrase applicable in more than one context: </li></ul><ul><ul><li>The network infrastructure (the 'Exaflood'): </li></ul></ul><ul><ul><ul><li>The rapid expansion of Internet traffic, e.g. from the streaming of movies or TV (BBC iPlayer) </li></ul></ul></ul><ul><ul><li>Managing a rapidly growing and diverse range of digital content, e.g. </li></ul></ul><ul><ul><ul><li>Personal content, e.g. from digital cameras, e-mail </li></ul></ul></ul><ul><ul><ul><li>Digitised content, e.g. sound and video reformatting, e-texts generated by mass-digitisation programmes </li></ul></ul></ul><ul><ul><ul><li>The &quot;Data Deluge&quot; - curating the vast amounts of research data being generated by experiments, observational instruments and computer simulation </li></ul></ul></ul>
    4. 6. The digital deluge - what is it? (2) <ul><li>International Data Corporation (IDC) White Paper: </li></ul><ul><ul><li>Estimates the digital universe in 2007 as 281 exabytes (281 billion gigabytes) and still growing fast </li></ul></ul><ul><ul><ul><li>But these estimates include outputs from surveillance cameras, financial transaction journals, Web search logs, as well as more directly user-generated forms of content </li></ul></ul></ul><ul><ul><li>Notes a growing environmental impact: </li></ul></ul><ul><ul><ul><li>Increased power consumption, electronic waste </li></ul></ul></ul><ul><ul><li>Key areas of recent growth identified include: </li></ul></ul><ul><ul><ul><li>Healthcare data, e.g. medical imaging </li></ul></ul></ul><ul><ul><ul><li>User-generated content, e.g. YouTube videos </li></ul></ul></ul><ul><ul><ul><li>Scientific experiments, e.g. LHC (300 exabytes a year) </li></ul></ul></ul>
    5. 7. The digital deluge - challenges (1) <ul><li>Problems of scale: </li></ul><ul><ul><li>Can our infrastructures begin to cope with dealing with petabytes or exabytes of data? </li></ul></ul><ul><ul><li>Technology has been quite good at keeping pace with data growth in the past (although Moore's Law will not rescue us for ever) </li></ul></ul><ul><ul><li>Dealing with Organisational change is more problematic </li></ul></ul><ul><ul><ul><li>The need for better co-ordination of effort is compromised by: </li></ul></ul></ul><ul><ul><ul><ul><li>Professional and disciplinary differences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Fragmented funding structures </li></ul></ul></ul></ul>
    6. 8. The digital deluge - challenges (2) <ul><li>Problems of complexity: </li></ul><ul><ul><li>Many different types of digital content: </li></ul></ul><ul><ul><ul><li>Structured, semi-structured, completely unstructured </li></ul></ul></ul><ul><ul><ul><li>Mediated, non-mediated </li></ul></ul></ul><ul><ul><ul><li>Interactivity and contextual links </li></ul></ul></ul><ul><ul><ul><li>Sometimes key supporting information (documentation, metadata, representation information, etc.) is missing </li></ul></ul></ul><ul><ul><li>Content is stored in many different places: </li></ul></ul><ul><ul><ul><li>Active environments </li></ul></ul></ul><ul><ul><ul><li>'Repositories' of various kinds (new forms of silos?) </li></ul></ul></ul><ul><ul><li>Ownership and privacy issues </li></ul></ul>
    7. 9. The digital deluge - challenges (3) <ul><li>Organisational problems: </li></ul><ul><ul><li>Lack of co-ordination between sectors, institutions, funding bodies, etc. </li></ul></ul><ul><ul><li>Still little consensus on: </li></ul></ul><ul><ul><ul><li>Deciding what needs to be kept (selection and appraisal) </li></ul></ul></ul><ul><ul><ul><li>Deciding who should ultimately be responsible for looking after content, i.e. who pays? </li></ul></ul></ul><ul><li>Infrastructures for preservation </li></ul><ul><ul><li>These are still emerging from R&D projects and the commercial sector (rapid progress in last five years) </li></ul></ul><ul><ul><li>In HE, still questions on exactly where institutional repositories fit within the digital preservation landscape </li></ul></ul>
    8. 10. The digital deluge - opportunities <ul><li>Some opportunities: </li></ul><ul><ul><li>While many curation challenges remain to be solved, the growing availability of digital content means that researchers: </li></ul></ul><ul><ul><ul><li>Will find new and innovative ways of combining data to develop and test new research hypotheses </li></ul></ul></ul><ul><ul><ul><li>Will develop methodologies for mining and analysing vast amounts of data </li></ul></ul></ul><ul><ul><li>It could also foster new and innovative ways of doing research, e.g. 'Science 2.0' </li></ul></ul>
    9. 11. Thank you for your attention

    ×