• Like
  • Save
The Developing Needs for e-infrastructures
Upcoming SlideShare
Loading in...5
×
 

The Developing Needs for e-infrastructures

on

  • 2,130 views

Presentation by Prof John Wood to the e-Infrastructure programme meeting in February 2008

Presentation by Prof John Wood to the e-Infrastructure programme meeting in February 2008

Statistics

Views

Total Views
2,130
Views on SlideShare
2,127
Embed Views
3

Actions

Likes
0
Downloads
40
Comments
0

1 Embed 3

http://www.slideshare.net 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The Developing Needs for e-infrastructures The Developing Needs for e-infrastructures Presentation Transcript

  • The Developing Needs for e-infrastructures Professor John Wood, Chair, JISC Committee for the Support of Research
  • E-infrastructure - definitions
    • The term “e-infrastructure” usually refers to ICT based infrastructure to support the research process. This includes:
    • Networks;
    • Access management and other “middleware” to manage the use of networked resources;
    • Computer facilities and peripherals including High Performance Computing (HPC);
    • Online content (research data, papers and journals, bibliometric data etc.)
  • JISC’s Role in Developing a UK e-infrastructure – current activities
    • JISC already provides the research community with:
    • (i) Physical network to support research – SuperJANET5, UKLight and a new dark fibre testbed for photonics research, to inform SuperJanet6
    • (ii) Access Management for researchers – the UK Access Management Federation meets the needs of most researchers, but JISC recognises the specialised requirements of Grid/e-Science community and is funding pilot projects to enable seamless inter-working between Grid and JISC’s information environment
    • (iii) Advice and guidance through initiatives such as the Digital Curation Centre and the National Centre for Text Mining
  • OSI Report – Key recommendations
    • The UK’s e-infrastructure should provide researchers with:
    • > Access to the systems, services, networks and resources that they need at the point that they need them
    • > Facilities to discover resources easily and use them appropriately
    • > Confidence in the integrity, authenticity and quality of the services and resources they use
    • > Assurance that their outputs will be accessible now and in the future
    • > A location-independent physical infrastructure for combining computation and information from multiple data sources
    • > Advanced technologies to support collaborative research
    • > The training and skills needed to exploit the services and resources available to them
  • OSI Report – Key recommendations
    • The e-infrastructure should allow researchers to:
    • > Exploit the power of advanced information technologies and applications to continuously enhance the process of research itself
    • > Collaborate and communicate securely with others, across disciplines, institutions and sectors
    • > Maximise the potential of advanced technologies to support innovation and experimentation
    • > Share their research outputs with others and re-use them in the future
    • > Engage with industry in support of wider economic goals
  • OSI Report – Key recommendations
    • The e-infrastructure must enable:
    • > The growth of knowledge transfer and the development of the commercial applications of research outputs
    • > Research funders to track the outputs from the research they fund
    • > The protection of individuals’ privacy and work, within regulatory, legal and ethical constraints
    • > The protection of intellectual property and rights management
    • > The preservation of digital information output as a vital part of the nation’s cultural and intellectual heritage
  • The Future?
    • How much supercomputing power do we need. The requirement to balance capacity and capability
    • Enabling data sets from many different sources and disciplines to be mined effectively. “Just how did the scientists and engineers work together across boundaries in the construction and running of ATLAS at CERN?”– a possible history of science Ph.D in the future!
    • Matching the pull of computer scientists with the needs of the academic community. Raising aspirations and integrating e-science with the development of large networks and facilities
    • How to cope with massive data sets and to protect them
    • How will students and teachers know something is true? – the need for strong measures to track provenance.
  • JISC’s Role in Developing a UK e-infrastructure
    • The JISC Support of Research committee is developing a new strategy to reflect the changing needs of the research community and to address the priorities identified in the OSI report
    • The strategy will inform the committee’s funding recommendations for 2008/09 and subsequent Academic Years
    • We will continue to work with the research councils to ensure that there are no gaps in provision and to eliminate overlaps
  • No longer one technique in one place !
    • The problems facing society demand a multi-technique approach.
    • Users are not expert in these techniques
    • E.g. Biologists will send samples and remotely access data.
    • Access Grid will enable several scientists to control in real time
    • Interoperability between equipment and data sets becomes imperative.
    • In real time who can drive the experiment.
  • Rutherford Appleton Laboratory
  •  
  • SNS target stations and beamlines
  • Science driver:- Integration of Data (and publications) Neutron diffraction X-ray diffraction NMR } High-quality structure refinement }
  • The Information Infrastructure The Body of Knowledge Creation Archival Access Storage Compute Network Services Curation the researcher acts through ingest and access Virtual Research Environment the researcher shouldn’t have to worry about the information infrastructure Information Infrastructure
  • Current View Raw Data Data Analysis Analysed Data Publication Data Publications Facility 1 Raw Data Data Analysis Analysed Data Publication Data Publications Facility 2 Raw Data Data Analysis Analysed Data Publication Data Publications Facility 3 Distinct Infrastructures / Distinct User Experiences
  • Future View Raw Data Data Analysis Analysed Data Publication Data Publications Facility 1 Raw Data Data Analysis Analysed Data Publication Data Publications Facility 2 Raw Data Data Analysis Analysed Data Publication Data Publications Facility 3 Capacity Storage Publications Repositories Standards/ Converters Data Repositories Raw Data Catalogue Data Analysis Analysed Data Catalogue Publication Data Catalogue Publications Catalogue Common Infrastructure / Common User Experience
  • In practice: The web has changed everything!
    • Scientists increasingly expect:
    • Access to everything:
      • distributed, interoperating information sources
    • Interlinking of everything :
      • Revalidation of results ‘repeat experiment’
    • Discovery across everything :
      • new knowledge from old
    • Archiving of everything :
      • Recording unique events
        • Antarctic environmental data
    The challenge is to keep pace with increasing expectations
  • Attracting new research communities: a proposed way forward
    • e-infrastructures should move from computing grids toward knowledge grids- the use of semantic webs and data mining
      • Many research communities have limited computing needs
      • Need to achieve deployment of large scale data oriented scientific applications
      • And beyond: data integration and knowledge management
    • Requirement: development and deployment of services
      • Middleware services must be extended
        • Data management
        • Security
      • User access must be eased
        • In terms of user friendliness
        • In terms of user support
  • JISC’s Role in Developing a UK e-infrastructure
    • Needs to take account of relevant international developments, including work by:
    • e-IRG (e-Infrastructure Reflection Group)
    • ESFRI ( European Strategy Forum on Research Infrastructures)
    • Proposed developments in GEANT2
    • US initiatives to develop a ‘cyberinfrastructure’
  • Excellence and Research Infrastructures  
    • Europe has a long-standing tradition of excellence in research and innovation;
    • European teams continue to lead progress in many fields of science and technology;
    • However our centres of excellence often fail to reach critical mass in the absence of adequate networking and cooperation;
    • Need to bring resources together and build a research and innovation equivalent of the " common market".  
    • The European Research Area
  • The European Roadmap for Large Research Facilities
    • European Strategy Forum on Research Infrastructures (ESFRI)
    • Launched in April 2002
    • Commissioned by the Council in 2004 to produce a forward look Roadmap akin to the DoE Large Facilities Roadmap but including all disciplines
    • First edition published in October 2006
    • Many of the projects are now being funded for drawing up preliminary proposals including the requirements for e-infrastructure, remote access etc.
  • What is ESFRI?
    • The European Strategy Forum on Research Infrastructures
    • Brings together representatives of the 27 Member States, 5 Associated States, and one representative of the European Commission (EC)
    • Projects must be “open access” and genuinely Pan-European or Global
    ESFRI E urope an S trate gy F orum o n Research Infrastructures
  • Social Science and Humanities CESSDA EROHS ESS DARIAH SHARE CLARIN 6 Projects
  • CLARIN Social Science and Humanities
    • Common Language Resources and Technology Infrastructure
    • language resources and technology available and useful to scholars of all disciplines, in particular the humanities and social sciences
    • harmonise structural and terminological differences
    • based on a Grid-type of infrastructure and by using Semantic Web technology
    www.mpi.nl/clarin 2008 First open access foreseen 106 M€ Estimated construction cost
  • CESSDA www.nsd.uib.no/cessda
    • Council of European Social Science Data Archives
    • distributed RI that provides and facilitates access of researchers to high quality data and supports their use
    • now 21 countries in Europe
    • 15,000 data collections
    • access to over 20,000 researchers
    Social Science and Humanities 2008 First open access foreseen 30 M€ Estimated construction cost
  • DARIAH Social Science and Humanities
    • Digital Research Infrastructure for the Arts and Humanities)
    • based upon an existing network of Data Centres and Services based in Germany (Max Planck Society), France (CNRS), the Netherlands (DANS) and the United Kingdom (AHDS)
    • bring essential cultural heritage online.
    www.dariah.eu 2008 First open access foreseen 10 M€ Estimated construction cost
  • SHARE Social Science and Humanities
    • Survey of Health, Ageing and Retirement in Europe
    • fact‑based economic and social science analyses of the on-going changes in Europe due to population ageing
    • will be expanded to all 27 member States of the EU.
    www.share-project.org 2007 First open access foreseen 50 M€ Estimated construction cost
  • Environmental Sciences AURORA BOREALIS EURO-ARGO EUFAR LIFEWATCH ICOS EMSO IAGOS-ERI 7 Projects
  • EMSO Environmental Sciences
    • deep sea-floor observatories deployed on specific sites offshore European coastline
    • allow continuous monitoring for environment and security.
    • part of a global endeavour in sea-floor observatories
    • long term monitoring of environmental processes related to ecosystem life and evolution, global changes and geo-hazards
    • key component of GMES and GEOSS.
    www.ifremer.fr/esonet/emso 2011 First open access foreseen 150 M€ Estimated construction cost
  • EURO-ARGO Environmental Sciences
    • European component of a world wide in situ global ocean observing system
    • based on autonomous profiling floats throughout the ice-free areas of the deep ocean
    • data are transmitted in real time by satellite to data centres for processing, management, and distribution
    www.coriolis.eu.org 2010 First open access foreseen 76 M€ Estimated construction cost
  • LIFEWATCH Environmental Sciences
    • protection, management and sustainable use of biodiversity
    • network of observatories, facilities for data integration and interoperability
    • virtual laboratories offering a range of analytical and modelling tools
    • a Service Centre providing special services for scientific and policy users, including training and research opportunities for young scientists
    www.lifewatch.eu/ 2014 First open access foreseen 370 M€ Estimated construction cost
  • ICOS Environmental Sciences
    • Integrated Carbon Observation System
    • co-ordinated, integrated, long-term high‑quality observational data of the greenhouse balance of Europe and of adjacent key regions of Siberia and Africa
    www.carboeurope.org 2010 First open access foreseen 255 M€ Estimated construction cost
  • Energy HiPER JHR 3 Projects IFMIF Need to nucleate further work
  • HiPER Energy
    • large scale laser system designed to demonstrate significant energy production from inertial fusion
    • supporting a broad base of high power laser interaction science
    • revolutionary approach to laser‑driven fusion known as “Fast Ignition”
    www.hiper-laser.org 2015 First open access foreseen 850 M€ Estimated construction cost
  • IFMIF Energy
    • International Fusion Materials Irradiation Facility
    • accelerator‑based very high flux neutron source to provide a suitable data base on irradiation effects on material needed for the construction of a fusion reactor
    www-dapnia.cea.fr 2017 First open access foreseen 855 M€ Estimated construction cost
  • Biomedical and Life Sciences STRUCTURAL BIOLOGY BIOBANKS CLINICAL TRIALS EATRIS INFRAFRONTIER Upgrade of EBI 6 Projects
  • European Biobanking and Biomolecular Resources Biomedical and Life Sciences
    • network of existing and de novo biobanks and biomolecular resources
    • samples from patients and healthy persons, molecular genomic resources and bioinformatics tools
    www.biobanks.eu 2009 First open access foreseen 170 M€ Estimated construction cost
  • INFRAFRONTIER Biomedical and Life Sciences
    • “ Phenomefrontier ”: in vivo imaging and data management tools, for the phenotyping of medically relevant mouse models
    • “ Archivefrontier ” state-of-the-art archiving and dissemination of mouse models (major upgrade of the European Mouse Mutant Archive (EMMA))
    www.eumorphia.org | www.emma.rm.cnr.it 2007 First open access foreseen 320 M€ Estimated construction cost
  • The European Spallation Source Material Sciences
    • world’s most powerful source of neutrons.
    • built-in upgradeability
    • initial 20 instruments
    • will serve 4,000 users annually across many areas of science and technology.
    http://neutron.neutron-eu.net/n_ess 2017 First open access foreseen 1050 M€ Estimated construction cost
  • ESRF Upgrade Material Sciences
    • European Synchrotron Radiation Facility (ESRF)
    • supported and shared by 17 European countries and Israel.
    • wide range of disciplines including physics, chemistry and materials science as well as biology, medicine, geophysics and archaeology
    • many industrial applications, including pharmaceuticals, cosmetics, petrochemicals and microelectronics.
    www.esrf.fr 2007-2014 First open access foreseen 230 M€ Estimated construction cost
  • Astronomy, Astrophysics and Nuclear Physics European ELT SKA SPIRAL2 FAIR KM3NeT 5 Projects
  • The European ELT Astronomy Astrophisics and Nuclear Physics
    • highest priorities in ground-based astronomy
    • detailed studies of inter alia planets around other stars, the first objects in the Universe, super-massive Black Holes, and the nature and distribution of the Dark Matter and Dark Energy which dominate the Universe
    • maintain and reinforce Europe’s position at the forefront of astrophysical research.
    www.eso.org/projects/e-elt 2018 First open access foreseen 850 M€ Estimated construction cost
  • FAIR Astronomy Astrophisics and Nuclear Physics
    • high energy primary and secondary beams of ions of highest intensity and quality
    • including an “antimatter beam” of antiprotons allowing forefront research
    • experiments with primary beams of ion masses up to Uranium and the production of a broad range of radioactive ion beams.
    www.gsi.de/fair/index_e.html 2014 First open access foreseen 1186 M€ Estimated construction cost
  • KM3NET Astronomy Astrophisics and Nuclear Physics
    • deep-sea research infrastructure in the Mediterranean Sea
    • cubic-kilometre sized deep‑sea neutrino telescope for astronomy
    • detection of high-energy cosmic neutrinos
    • long-term deep-sea measurements.
    www.km3net.org 2015 First open access foreseen 220-250 M€ Estimated construction cost
  • SKA Astronomy Astrophisics and Nuclear Physics
    • Square Kilometre Array
    • next generation radio telescope
    • 50 times more sensitive than current facilities
    • survey the sky more than 10,000 times faster than any existing radio telescope.
    www.skatelescope.org 2014-2020 First open access foreseen 1150 M€ Estimated construction cost
  • ESFRI and e-IRG: EU-HPC
    • New generation of Capability (high-performance) and Capacity Computing (high‑throughput) top-level machines
    • Scientific computing network to be set-up at European level associated with national, regional and local centres
    • Different machine architectures will fulfil the requirements of different scientific domains and applications
    www.hpcineuropetaskforce.eu 2007 First open access foreseen 550 M€ Estimated construction cost
  • Global Dimension
    • Several of the projects on the Roadmap require a global approach. 
    • Discussions are taking place on how the EU can act with one voice
    • A Forum for decision making is urgently needed. Carnegie meeting agreed to regular meeting of science ministers
    • Major player are Australia, Japan, Russia, South Africa, USA, China, India
  • Lessons learnt from first ESFRI Roadmap
    • Many countries were not ready for the Roadmap
      • No national strategy for infrastructures in respective fields
      • Lack of information
    • Remarkable differences between research fields
      • Higher level of coordination in Physics
      • Pan-European coordination of infrastructures in other fields not common
    • Triggering effect of the ESFRI Roadmap
      • ESFRI stimulated many countries to start the process of national prioritisation
      • High demand for Europe-wide accessible infrastructure
  • Implementation
    • Where are we with the implementation of the
    • Roadmap 2006:
    • Preparatory phase from FP7
    • Member States discussion on all the projects
    • Some project in advanced state of implementation:
    • the example of XFEL
    • The update of the Roadmap has started since not all fields
    • were covered.
      • Update and addendum of first Roadmap
      • Assessment of maturity of Emerging Proposals
      • Identification of further important research infrastructures
    The ESFRI Roadmap is an ongoing process
  • ESFRI – The Forum Chair: John Wood (Carlo Rizzuto, March 2008) (+ 60 Representatives) Executive Board Chair + EC + 3 elected ESFRI-Members ESFRI Secretariat (EC) Hervé Pero ( Executive Secretary) RWG-PSE Chair: Jørgen Kristian Kjems RWG-BMS Chair: Eckhart Curtius RWG-SSH Chair: Bjorn Henrichsen ESFRI Structure RWG-ENV Chair: Eeva Ikonen e-IWG Chair: Dany Vandromme
  • The necessity for e-science
    • e-science is about inventing and exploiting new advanced computational methods to:
      • create a new approach to shared research between groups and facilities
      • generate, curate and analyze data
      • link publications to data
      • develop and explore models and simulations at an unprecedented scale and to use simulations to run experiments
      • help the set-up of distributed virtual organizations to ease collaboration and sharing of resources and information and the remote operation of facilities
  • Who are the users today?
    • Research communities in urgent need for new advanced methods because they face unprecedented computational challenges
      • Example High Energy Physics
        • LHC
        • Neutrino Mass
        • Gravitational Waves
    • Research communities foreseeing the need for new advanced computational methods because of new major projects
        • Example: fusion (ITER)
    • Other research communities - a hollistic approach
        • Geophysics
        • Condensed Matter
        • Meteorology
        • Energy
  • The early adopters: HEP
    • The High Energy Physics was the first research community to adopt globally the grid paradigm for data collection and analysis
      • High Energy Physics adopted grids for LHC to handle the unprecedented volume of data produced
      • Highly structured community acting as “Guinea pig”
      • High Energy Physics is the n°1 user of e-infrastructures around the world
      • 99.9% of the data from Atlas has to be removed in the first few microseconds to avoid web overload!!
    • Progress towards the LHC at CERN - first beam this year !
    Particle Physics
  • Looking forward - the LHC at CERN ATLAS tracker at RAL CMS CMS calorimeter crystal LHC computing at RAL Physics data in 2008! LHC analysis inititative with Southampton Invented at RAL ATLAS
  • Achievements in High Energy Physics
    • the example of EGEE
    • ( Enabling Grids for E-sciencE)
      • ~50K jobs/day
      • > 10K simultaneous jobs during prolonged periods
      • Reliable data distribution service demonstrated at 1.6 GB/sec from CERN to LHC Computing Grid national nodes
  • Meteorology (1)
  • Reaching the critical mass
    • Many research communities are trying the Grid
      • Very positive experience within EGEE application sector
      • Slow transition from pilot application to scientific production
    • A critical mass is needed to move from pilot applications to scientific production
      • Critical mass of scientists: dissemination in the research community to reach beyond pioneers
      • Critical mass of resources: large enough virtual organization
      • Critical mass of grid expertise
    • Beside geographic extensions of the infrastructures, need for community oriented infrastructure projects:
  • Big Issues
    • Data Deluge
    • Curation and Provenance
    • Interoperability
    • Multi-disciplinarity of research
    • Linking of publications to data
    • What is coming up – “Tower of Babel” or “Nations Speaking unto Nations.” – the need for International Strategies
  • The sun imaged with neutrinos (by the SuperK experiment) SNO detector Oscillations Confirmed by MINOS in 2006 SuperKamiokande and SNO open a new world of neutrino oscillations: discovery that neutrinos have tiny masses and mix Neutrino discovery timeline  The new world of neutrino physics
  • Looking forward - Neutrino Physics
    • MICE at RAL - installing now
    • Demonstrate cooling a muon beam
    Explore CP violation: origin of matter in the universe?
    • Neutrino Factory
    • international scoping study  design study
    • RAL is one credible site
    • T2K at J-PARC - starts 2009
    • A strong role in detector and accelerator development and in physics analysis
    Technology demonstration Learn more about neutrino mixing angles
  • Complex detectors with advanced electronics
    • e-technologies (e-science)
    • bringing the central facilities into the universities
    • detectors & advanced data acquisition
    • unique synergy within CCLRC
    Neutron powder diffraction data rates (1950-2010) (4 orders of magnitude gain with ILL/ISIS alone) D20(ILL) GEM(ISIS)  D1a(ILL) NEUTRON MOORE’S LAW 4 orders of magnitude in 30 years 1983 1998 efficient large solid angle detectors... … fast electronics
  • In practice: Its all about scale
    • Creation:
    • Examining the detector arrays on the MAPs spectrometer at ISIS
  • A One off experiment
    • Collection:
    • An ATSR image of Sicily with Mount Etna eruption; taken 24 July 2001
  • Data Deluge!
    • Capacity:
    • eg at RAL
    • 20PB by 2010
    • 1PB = 10 15 Bytes
    • Billions of Floppys
    • Millions of CDs
    • Thousands of PCs (today’s)
  • Curation – who is responsible?
    • Curation :
    • Some STFC based Repositories
      • The Atlas Datastore
      • The British Atmospheric Data centre
      • The CCLRC Data Portal
      • The CCLRC Publications Archive
      • The CCPs (Collaborative Computational Projects)
      • The Chemical Database Service
      • The Digital Curation Centre
      • The EUROPRACTICE Software service
      • The HPCx Supercomputer
      • The JISCmail service
      • The NERC Datagrid
      • The NERC Earth Observation Data Centre
      • The Starlink Software suite
      • The UK Grid Support Centre
      • The UK Grid for Particle Physics Tier 1A
      • The World Data Centre for Solar-Terrestrial Physics
    Atlas Datastore Tape Robot
  • The problem will grow
    • New large scale facilities are being planned and built around the world.
    • They will be run remotely and have to interact in real time with HPC simulations, each informing the other. What will be the role of the researcher once the experiment starts?
    • Data storage etc needs to be planned right at the start.
    • An example: XFEL in Hamburg
  • Schematic layout of a single pass XFEL A new X-ray source is needed for studies of new , of non-equilibrium states of matter at atomic resolution in space and time
  • 3 rd Gen. SR 2 nd Gen. SR Laser Slicing SPPS Initial H.-D. Nuhn, H. Winick Peak Brightness [Phot./(s · mrad 2 · mm 2 · 0.1%bandw.)] FWHM X-Ray Pulse Duration [ps] Ultrafast x-ray sources will probe space and time with atomic resolution . Peak brightness of pulsed X-ray sources what do we do today and what tomorrow? Future Future ERLs X-Ray FELs Initial
  • The X-ray free-electron lasers will provide coherent radiation of the proper wavelength and the proper time structure , so that materials and the changes of their properties can be portrayed at atomic resolution in four dimensions, in space and time . Diffraction pattern of 10 x 10 x 10 Au cluster Fascination - FELs for hard X-rays
  • Schematic presentation of transition states in a chemical reaction Take a movie of chemical reactions
  • single molecule crystal with atomic resolution Lysozym Oversampling: J. Miao, K.O. Hodgson and D. Sayre , PNAS 98 (2001) 6641-6645 Imaging of a single bio-molecule
  • R. Neutze, R. Wouts, D. van der Spoerl, E . Weckert, J. Hajdu: Nature 406 (2000) 752-757 t=0 t=50 fsec t=100 fsec Coulomb Explosion von Lyzosym
  • The VUV-FEL user facility at DESY
  • VUV-FEL RF gun FEL experimental area bypass 4 MeV 150 MeV 450 MeV 1000 MeV undulators collimator bunch compressor Laser M1 M2 M3 M4 M5 M6 M7 250 m bunch compressor
  • XFEL Length ca. 3.3 km PETRA HERA European XFEL Facility in Hamburg phase I phase II
  • XFEL: Office and Laboratory Building
  • 1k x 1k system with 4 x 4 super modules
  • DAQ Challenge : 2D X-Ray Detector Systems
    •  10 6 pixels per frame for one detector
    • O(400-500) frames per train (goal, likely will start with less)
    • 10 trains per second (machine allows up to 30 Hz…)
    • With 2 Byte/pixel  average rate  10 Gbyte/sec for one 2D detector!
    • Time between frames as short as 200ns  buffering needed
    600  s 99.4 ms 100 ms 100 ms 200 ns LPD
  • Data Storage Issues
    • Assume:
    • 3 x 1 Megapixel 2D Detector Systems
    • 2 Byte/pixel
    • 500 frames per train are read taken and read out
    • 10 trains per second
    • Running year: 200 days = 4800 hours
    • Running efficiency: 10%
    • Good-frame efficiency/compression: 25% (is this realistic?)
    •  ~ 13 Pbyte/year (1 Petabyte = 10 6 Gigabyte =10 15 Byte)
  • Today’ Situation at DESY
      • SUN SL8500 Tape Robot
      • Installed at DESY in Jan 07
      • Up to 10,000 Cartridges
      • Multi library capability
      • Lifetime of about 5 to 10 years (matter of running costs)
      • Up to 64 drives possible
      • Currently 30 drives LTO3 24 data/6 backup
      • LTO3 400 GB/Cart, 120 MB/s
    10,000 LTO3 Cartridges: 4 Petabyte V.Gülzow et al.
  • DESY Storage Capacity Planning w/o XFEL V.Gülzow et al. Terabyte Year
  • Technology Forecast – Storage at DESY
    • not a technology problem
    • money and manpower issues
    • to be determined:
      • user behaviour
      • compression and accept/reject algorithms
    • potentially critical: access to data!
    26 5 2012 200 40 2016 3 1 2009 Storage Space [Petabyte] Rate Capability [Gbyte/sec] Year
  • Exciting times ahead for JSR!
    • Raising awareness
    • Training
    • New interrogation tools and approaches
    • Infrastructure
    • Integrating information from many sources
    • Waking up decision makers
    • A new way of doing research for many communities
    • Who is going to be responsible for the long term storage, curation and authentication of data?