Sinnott Paper


Published on

An Introduction to eScience and the Grid by Prof. Richard Sinnott.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This shows the National Centre, 8 regional centres and 2 laboratories in blue That is the original set up at the start of UK e-Science August 2001 NeSC is jointly run by Edinburgh & Glasgow Universities In 2003 several smaller centres were added (vermilion) (1 st call out) The e-Science Institute is run by the National e-Science Centre. It runs a programme of events and hosts visiting international researchers. It was established in 2001. The Open Middleware Infrastructure Institute was established in 2004, to provide support and direction for Grid middleware developed in the UK. It is based at the University of Southampton. The Grid Operations Centre will be established in 2004. (2 nd call out) The Digital Curation Centre was established in 2004 by the Universities of Edinburgh and Glasgow, the UK Online Library Network at the University of Bath, and the Central Laboratories at Daresbury and Rutherford. It’s job is to provide advice on curating scientific data and on preserving digital media, formats, and access software. (3 rd call out) Edinburgh is one of the 4 founders of the Globus Alliance (Sept 2003) which will take responsibility for the future of the Globus Toolkit. The other founders are: Chicago University (Argonne National Lab), University of Southern California, Los Angeles (Information Sciences Institute) and the PDC, Stockholm, Sweden (4 th call out) The EU EGEE project (Enabling Grids for E-Science in Europe) is establishing a common framework for Grids in Europe. The UK e-Science programme has several connections with EGEE. NeSC leads the training component for the whole of Europe.
  • Sinnott Paper

    1. 1. The e-Context of ENROLLER Prof Richard O. Sinnott Technical Director National e-Science Centre [email_address] 16 th April 2010
    2. 2. e-Science and e-Research <ul><li>Goal: to enable better research in all disciplines </li></ul><ul><li>Method: Develop collaboration supported by advanced distributed computation </li></ul><ul><ul><li>to generate, curate and analyse rich data resources </li></ul></ul><ul><ul><ul><li>From experiments, observations and simulations </li></ul></ul></ul><ul><ul><ul><li>Quality management, preservation and reliable evidence </li></ul></ul></ul><ul><ul><li>to develop and explore models and simulations </li></ul></ul><ul><ul><ul><li>Computation and data at all scales </li></ul></ul></ul><ul><ul><ul><li>Trustworthy, economic, timely and relevant results </li></ul></ul></ul><ul><ul><li>to enable dynamic distributed collaboration </li></ul></ul><ul><ul><ul><li>Facilitating collaboration with information and resource sharing </li></ul></ul></ul><ul><ul><ul><li>Security, trust, reliability, accountability, manageability and agility </li></ul></ul></ul>The challenge is to develop an integrated approach to all three Often realised through Grids and Grid infrastructures
    3. 3. The Grid Context <ul><li>There are many Grids </li></ul><ul><ul><li>Data Grids, Compute Grids, Information Grids, Enterprise Grids, … </li></ul></ul><ul><li>There are many ways to build Grids </li></ul><ul><ul><li>Grid middleware (many flavours), </li></ul></ul><ul><ul><li>Web services, </li></ul></ul><ul><ul><li>Clouds, </li></ul></ul><ul><ul><li>Web2.0, </li></ul></ul><ul><ul><li>internet computing, … </li></ul></ul><ul><li>There are many moving targets </li></ul><ul><ul><li>changing middleware, changing standards, changing sciences, changing resources, new questions, new funding streams… </li></ul></ul><ul><li>There has been a lot of hype </li></ul><ul><li>There has been a lot of money invested </li></ul><ul><li>There are lots of projects and big scientific challenges </li></ul><ul><li>There is an urgent need to build user communities </li></ul><ul><li>There needs to have much more research pull than middleware push </li></ul><ul><ul><li>… there are many more things that could go here! </li></ul></ul>
    4. 4. UK e-Science Core Programme <ul><li>Major cross council initiative </li></ul><ul><ul><li>AHRC, BBSRC, EPSRC, ESRC, MRC, NERC, PPARC/STFC, … </li></ul></ul><ul><li>Over £250m funding over 7-8 years from 2001 </li></ul><ul><ul><li>Does not include industry monies from </li></ul></ul><ul><ul><ul><li>Department of Trade and Industry </li></ul></ul></ul><ul><ul><ul><li>Technology Strategy Board </li></ul></ul></ul><ul><ul><ul><li>Europe </li></ul></ul></ul><ul><ul><ul><li>JISC </li></ul></ul></ul><ul><ul><ul><li>Regional development agencies </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><li>Programme now completed and reviews/planning for future government spending in this area on-going </li></ul>
    5. 5. e-Science in the UK CeSC (Cambridge) e-Science Institute Grid Operations Support Centre National Institute for Environmental e-Science Core NGS Nodes +HECTOR +partners/affiliates (HECTOR investment £113m) Digital Curation Centre Digital Curation Centre Digital Curation Centre Digital Curation Centre OMII-UK OMII-UK OMII-UK NERC e-Science Centre National Centre for Text Mining National Centre for e-Social Science Software Sustainability Institute Core NGS Nodes +HECTOR +partners/affiliates (HECTOR investment £113m) Core NGS Nodes +HECTOR +partners/affiliates (HECTOR investment £113m) National Centre for e-Social Science National Centre for e-Social Science National Data Centres + UK Federation + International dimension including EGEE/EGI + SuperJanet + Training/Education +… NeSC 4 th Phase Platform Grant
    6. 6. NeSC Background <ul><li>E-Science Hub </li></ul><ul><ul><li>Externally </li></ul></ul><ul><ul><ul><li>Glasgow end of NeSC </li></ul></ul></ul><ul><ul><ul><ul><li>Involved in numerous UK wide activities/projects </li></ul></ul></ul></ul><ul><ul><li>Internally </li></ul></ul><ul><ul><ul><li>Focal point for e-Science research/activities at Glasgow </li></ul></ul></ul><ul><ul><ul><li>Work closely with foundation departments </li></ul></ul></ul><ul><ul><ul><ul><li>Department of Computing Science </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Established first UK Grid Computing course </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Department of Physics & Astronomy </li></ul></ul></ul></ul><ul><ul><ul><li>Also working with other groups including </li></ul></ul></ul><ul><ul><ul><ul><li>Bioinformatics Research Centre, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Biostatistics </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Electronics and Electrical Engineering </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Dept of Public Health, Dept. of Pathology, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Dept. of English, Arts & Humanities, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>University Services, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Clinicians & numerous hospitals across Scotland, </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Yorkhill, Royal Infirmary, Western General, Southern General … </li></ul></ul></ul></ul></ul><ul><ul><li>NeSC GU now part of University IT Services </li></ul></ul>J. Jiang Chris Bayliss C. Millar Gordon Stewart J.Mohammad (PhD) T.Doherty VPman M. Sarwar (ENROLLER) Nurazian Mior Dahalan (PhD) Camera Shy David Martin (ScotGrid sys-admin) S. Hussain (PhD)
    7. 7. NeSC Glasgow Projects <ul><li>National e-Science Centre (NeSC-I, NeSC-II, NeSC-III) </li></ul><ul><li>Dynamic Virtual Organisations for e-Science Education (DyVOSE) </li></ul><ul><li>Biomedical Research Informatics Delivered by Grid Enabled Services (BRIDGES) </li></ul><ul><li>Grid Enabled Microarray Expression Profile Search (GEMEPS) </li></ul><ul><li>GridNet </li></ul><ul><li>Glasgow early adoption of Shibboleth (GLASS) </li></ul><ul><li>Joint Data Standards Survey (JDSS) </li></ul><ul><li>ESP-Grid </li></ul><ul><li>GridNet-2 </li></ul><ul><li>HPC Compute cluster award </li></ul><ul><li>Sun industrial sponsorship </li></ul><ul><li>OGC Collision </li></ul><ul><li>OMII-Security Portlets </li></ul><ul><li>OMII-RAVE </li></ul><ul><li>Integrating VOMS and PERMIS for Superior Grid Authorization (VPman) </li></ul><ul><li>NCeSS Technical Management </li></ul><ul><li>CESSDA PPP </li></ul><ul><li>Pharming of Therapeutic RNA </li></ul><ul><li>Grid Enabled Occupational Data Environment (GEODE) </li></ul><ul><li>Towards an e-Infrastructure for e-Science Digital Repositories </li></ul><ul><li>Grid enabled Biochemical Pathway Simulator </li></ul><ul><li>Virtual Organisations for Trials and Epidemiological Studies (VOTES) </li></ul><ul><li>Towards a European e-Infrastructure for e-Science Repositories </li></ul><ul><li>Modelling, Inference and Analysis for Biological Systems up to the Cellular Level </li></ul><ul><li>Drug Discovery Portal </li></ul><ul><li>Advanced Grid Authorisation through Semantic Technologies (AGAST) </li></ul><ul><li>ShinTau (Supporting Multiple Shibboleth Attribute Authorities) </li></ul><ul><li>Grid-enabled Virtual Safe Settings – Security & the State of the Nation </li></ul><ul><li>Scottish Bioinformatics Research Network (SBRN) </li></ul><ul><li>Generation Scotland Scottish Family Health Study </li></ul><ul><li>Meeting the Design Challenges of nanoCMOS Electronics (nanoCMOS) </li></ul><ul><li>EU FW7 Avert-IT </li></ul><ul><li>EU FW7 EuroDSD </li></ul><ul><li>Breast Cancer Tissue Biobank </li></ul><ul><li>Data Management through e-Social Science (DAMES) </li></ul><ul><li>NeSC Research Platform (NRP) </li></ul><ul><li>NeSC Information Network (NIN) </li></ul><ul><li>European Network for Study of Adrenal Tumors </li></ul><ul><li>Scottish Health Informatics Platform for Research (SHIP) </li></ul><ul><li>National E-Infrastructure for Social Simulation (NeISS) </li></ul><ul><li>Enhancing Repositories for Language and Literature Researchers (ENROLLER) </li></ul><ul><li>Proxy Credential Auditing Infrastructure for the NGS </li></ul><ul><li>European Network for Study of Adrenal Tumors Cancer Research Platform </li></ul><ul><li>Diagnostic Identification of Parkinsons (DiPAR) </li></ul>Completed Running Applications oriented Security!
    8. 8. Data Grids for High Energy Physics Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute Institute Institute ~0.25TIPS Physicist workstations ~100 MBytes/sec ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size Physics data cache ~PBytes/sec ~622 Mbits/sec ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 Tier2 Centre ~1 TIPS Online System Pentium II 300 MHz Pentium II 300 MHz Pentium II 300 MHz Pentium II 300 MHz HPSS HPSS Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server HPSS HPSS HPSS Tier2 Centre ~1 TIPS Tier2 Centre ~1 TIPS Tier2 Centre ~1 TIPS Caltech ~1 TIPS 1 TIPS is approximately 25,000 SpecInt95 equivalents
    9. 9. Next Generation Transistor Design 3D + Statistical
    10. 10. Inter-disciplinary e-Health Example Nucleotide sequences Nucleotide structures Gene expressions Protein Structures Protein functions Protein-protein interaction (pathways) Cell Cell signalling Tissues Organs Physiology Organisms Populations Security!!! biologists, bioinformaticians, statisticians, clinicians, pharmacists, physicists, epidemiologists, chemists, geospatial modellers, public health ... + environmental, social, geographic …
    11. 11. Bridges Project SyntenyService Information Integrator OGSA-DAI Magna Vista Service VO Authorisation blast + + +
    12. 12. Grid Blast Interface <ul><li>Allows ‘genome scale’ blasting </li></ul><ul><li>Transparently uses NGS, ScotGrid, other GU clusters, Condor pools </li></ul><ul><li>Many databases already deployed across nodes </li></ul><ul><li>No user certificates </li></ul><ul><ul><li>Fine grained security at </li></ul></ul><ul><ul><li>back-end </li></ul></ul>
    13. 13. MagnaVista
    14. 14. MagnaVista
    15. 15. GeneVista
    16. 16. E-Security <ul><li>Security </li></ul><ul><ul><li>Key is that should support </li></ul></ul><ul><ul><ul><li>seamless access to a heterogeneous variety of “ distributed ” compute and data (and other) resources </li></ul></ul></ul><ul><ul><ul><ul><li>Often domain specific – especially data! </li></ul></ul></ul></ul><ul><ul><ul><li>single sign-on </li></ul></ul></ul><ul><ul><ul><ul><li>Authenticate once and access numerous distributed resources </li></ul></ul></ul></ul><ul><ul><li>A A A A ( +privacy, confidentiality, integrity… ) </li></ul></ul><ul><ul><ul><ul><li>Authentication </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>(know who “they” are) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Authorisation </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>(decide what “they” can do and enforce it) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Auditing/accounting </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>(keeping track of who did what/when for security checks/charging etc) </li></ul></ul></ul></ul></ul>
    17. 17. Ease of Use <ul><li>For Grids/e-Research to be truly successful </li></ul><ul><ul><li>have to be made as seamless to access and use as the internet </li></ul></ul><ul><ul><ul><li>Forget training, education for some (most?) users! </li></ul></ul></ul><ul><ul><li>have to be based on research pull and not middleware push </li></ul></ul><ul><ul><li>experiences in various projects and across whole e-Science programme have shown that users don’t like digital certificates </li></ul></ul>
    18. 18. User Oriented Security <ul><li>A _ _ _ </li></ul><ul><ul><li>Federated Authentication, e.g. through Shibboleth </li></ul></ul>Service provider 5. User accesses resource Web site/e-Journal Identity Provider Home Institution W.A.Y.F. Federation User AuthN Log-in once and roam <ul><li>User points browser at Grid resource/portal (or non-Grid resource) </li></ul>2. Shibboleth redirects user to W.A.Y.F . service 3. User selects their home institution 4. Home site authenticates user LDAP
    19. 19. _ A _ _ <ul><li>Authorisation </li></ul><ul><ul><li>Defining what they can do and define and enforce rules </li></ul></ul><ul><ul><ul><li>Each site will have different rules/regulations </li></ul></ul></ul><ul><ul><li>Also known as Virtual Organisations (VO) </li></ul></ul><ul><ul><ul><li>Collection of distributed resources shared by collection of users from one or more organizations typically to work on common research goal </li></ul></ul></ul><ul><ul><ul><ul><li>Provides conceptual framework for rules and regulations for resources to be offered/shared between VO institutions/members </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Different domains place greater/lesser emphasis on expression and enforcement of rules and regulations (policies) </li></ul></ul></ul></ul>. . . {Resources} {Users} Org 1 {Resources} {Users} Org n VO VO specific agreements
    20. 20. Privileges, Resources, Access Control and Trust Service provider Shib Frontend 5. Pass authentication info and attributes to auth Z function Grid Portal 6. M ake final AuthZ de cision Grid Application Identity Provider Home Institution W.A.Y.F. Federation User AuthN AuthZ ? ? ? ? ? ? ? 1. User points browser at Grid resource/portal 2. Shibboleth redirects user to W.A.Y.F . service 3. User selects their home institution 4. Home site authenticates user and pushes attributes to the servic e provider LDAP LDAP