BeSTGRID: Developing NZ research infrastructurewww.bestgrid.orgNick JonesDirector, BeSTGRIDCo-DirectoreResearch, Centre for eResearchUniversity of Aucklandn.jones@auckland.ac.nz

... was a Pilot$2.5million: Sep 2006 – March 2008“BeSTGRID aims to enhance e-Research capability in New Zealand by providing the skill base to help the various research disciplines engage with new eResearch services”“BeSTGRID aims to kick start centralised infrastructurewith some capital investment at key institutions”BeSTGRID:intial three years

Collaboration ServicesAdvanced Video Conferencing (Access GRID, EVO - Enabling Virtual Organizations, HD, H.323)Telemedicine – APAN 26

Advanced Video Conferencing (Access GRID, EVO - Enabling Virtual Organizations, HD, H.323)

Telemedicine – APAN 26

Collaboration ServicesSakai VRE (Virtual Research Environment) – collaboration space (wikis, mailing lists, event calendars, file sharing)“VRE” development by UK e-Science group at Hartree Centre, DaresburyBeSTGRID IAM FederationPilot integration with Australian AAFAuckland, Canterbury, Lincoln, OpenWikis, SLCS (Grid Certificates)

Sakai VRE (Virtual Research Environment) – collaboration space (wikis, mailing lists, event calendars, file sharing)

“VRE” development by UK e-Science group at Hartree Centre, Daresbury

BeSTGRID IAM Federation

Pilot integration with Australian AAF

Auckland, Canterbury, Lincoln, Open

Wikis, SLCS (Grid Certificates)

Computation Grid ToolsGrid Computing interfacesGrisu GUI, Grisu CLIBioportal (Phylogenetics)x.509 CertificatesGrix certificate issuing toolsSLCS Shibboleth issued certificates (Canterbury)

BioinformaticsSteven Stones-Havas1 and Allen Rodrigo21Biomatters Ltd 2The Bioinformatics Institute (New Zealand)

Data Services30TB storage at each foundation BeSTGRIDsiteAuckland, Canterbury, MasseyLocal mirrors of research databasesNZ BioMirrorEcology and Human BehaviourWhole Genome Association StudiesNZ Social Sciences Data ServiceOcean Biogeographic Information SystemAustronesian Basic Vocabulary and Bantu Language DatabaseData services (GridFTP, NFS, Samba, FTP)

30TB storage at each foundation BeSTGRIDsite

Auckland, Canterbury, Massey

Local mirrors of research databases

NZ BioMirror

Ecology and Human Behaviour

Whole Genome Association Studies

NZ Social Sciences Data Service

Ocean Biogeographic Information System

Austronesian Basic Vocabulary and Bantu Language Database

Data services (GridFTP, NFS, Samba, FTP)

Successes: within NZFirst grid infrastructure in NZ, focused on BioInformatics, with Industry collaboration with BioMatters and New Zealand Supercomputing CentreImplementation of first federated identity management framework in NZcontribution to a multi-sector working group to further evolve identity management across research and education (IMAGER, http://www.morst.govt.nz/current-work/science-infrastructure/imager/) Established the foundations of a shared infrastructure service delivery model across institutionsRecognition from MoRST of need for eResearch support, with $4.2M in ‘08/’09 budget for eResearch infrastructure

What did we learn?Change difficult within InstitutionsMedium to Long term timeframes required with researchers, ITS, SMT, research support units“Final Report: A Workshop on Effective Approaches to Campus Research Computing Cyberinfrastructure” Klingenstein, Morooney, Olshansky. 2006Collaboration seen as critical yet complex and expensiveWorking with user communities essential to mutual understand of needs and opportunities, and developing capabilitiesNeed significant capability before international collaborations (and related learning) possibleNew forms of organisation and support required“Beyond Being There: A Blueprint for Advancing the Design, Development, and Evaluation of Virtual Organization” Cummings, Finholt, Foster, Kesselman, Lawrence. 2008“The Importance of Long-Term Science and Engineering Infrastructure for Digital Discovery” Wilkins-Diehr, Alameda, Droegemeier, Gannon. 2008Government and Institutional Investment well below international levelsAustralia NCRIS: $540M ++ over 5 years alongside State and Institutional investmentsEU FP7 e-Infrastructure call + National Grid Initiatives + Identity + HPC + Networks in member statesUS Office of Cyberinfrastructure, NSF, NIH, DoE all funding major Cyberinfrastructure initiativesEducation programs lackingWho is training the next generation?

BeSTGRID 2009 - 2010MoRST GRID Computing Middleware Initiative

MoRSTeResearchprogrammeReview:In Vote RS&T 2008/09, funding was appropriated to develop capability within the Kiwi Advanced Research and Education Network user group to make effective use of KAREN: Advanced Video Conferencing Collaboration and Support Centre Federated Identity and Access ManagementTowards a Federated Approach – strategy development Technical Support and Resources Semantic Data and Public Access to Research Grid Middleware Initiative Hardship fund for remote site connection to KAREN

Advanced Video Conferencing Collaboration and Support Centre

Federated Identity and Access Management

Towards a Federated Approach – strategy development

Technical Support and Resources

Semantic Data and Public Access to Research

Grid Middleware Initiative

Hardship fund for remote site connection to KAREN

Advanced Video Collaboration CentreAdvanced Video Collaboration CentreHelpdesk 0800 666 889http://avcc.karen.net.nz/Event calendarQuality Assurance for end pointsNathan Gardiner

Advanced Video Collaboration Centre

Helpdesk 0800 666 889

http://avcc.karen.net.nz/

Event calendar

Quality Assurance for end points

Nathan Gardiner

NZ Identity FederationTowards a Federated Identityhttps://nzfed.auckland.ac.nz/Strategy consultation – Q3 – Q4 2009Resources, Toolkit, and examplar services – 2010Governance and Ownership discussion underway – NZVCC IT, CRI IT, supported by MoRSTIMAGER – http://www.morst.govt.nz/current-work/science-infrastructure/imager/Matthew Cocker

Semantic public research dataPublic research data – semantic interoperabilityIntegration of Geoscience related data holdingsUniversity of Auckland, Landcare ResearchMark Gahegan, Robert Gibb

BeSTGRID2009 – 2010MoRST Dec 2008 funding request forMiddleware development for eResearch in NZ $840k approved 24 Dec 2008, initiated April 2009Provides ongoing support for BeSTGRIDWidened from founders: Auckland, Canterbury & Masseyby active participation of new Universities and CRIsVictoria, Otago, Lincoln, Landcare Research, AUTAim: coordinate national grid middleware; use best-practice tools & services; focus on sustainability

MoRST Dec 2008 funding request forMiddleware development for eResearch in NZ

Provides ongoing support for BeSTGRID

Widened from founders: Auckland, Canterbury & Massey

by active participation of new Universities and CRIsVictoria, Otago, Lincoln, Landcare Research, AUT

Aim: coordinate national grid middleware; use best-practice tools & services; focus on sustainability

BeSTGRID2009 – 2010Form collaborations with related Science Infrastructure programmesSCENZ GridNew Zealand Genomics LimitedVirtual Institute of Statistical Genetics and MapNetSocial Sciences Data Archive and ServicesAuckland Cancer Society Research CentreSupport Science Gateways, applications and services infrastructure supports science communities with own specialised needsBio – Extension of scope from phylogenetics to other areas of Bioinformatics and Biomolecular sciencesGeo – collaboration with Canadian geoscience infrastructureAdd in new Data and HPC sites and services:Otago UniversityVictoria University WellingtonLandcare ResearchAuckland University Technology

SCENZ Grid

New Zealand Genomics Limited

Virtual Institute of Statistical Genetics and MapNet

Social Sciences Data Archive and Services

Auckland Cancer Society Research Centre

Bio – Extension of scope from phylogenetics to other areas of Bioinformatics and Biomolecular sciences

Geo – collaboration with Canadian geoscience infrastructure

Otago University

Victoria University Wellington

Landcare Research

Auckland University Technology

AucklandMassey, AlbanyAUTLandcare ResearchVictoriaLincolnCanterburyOtagoComputationClusterIdentity providerPlannedStorageVirtual ApplicationsCurrent

Computation Grid Resources

NZ eResearch Infrastructure

Key MilestonesWinter Retreat 2009Geo, Bio, Grid strategy workshops – Auckland29 June – 1 JulySpring Retreat 2009Technical Working Group working session – Canterbury14 SeptemberStrategies and plansGeo and Bio strategy drafts – near completionInitial projects kicking off in parallelNational workshop – planning underway for 2010Coordinating with KAREN, Federated IAM, and othersGrid Middleware strategy – Q3 2010

BeSTGRID Winter Retreat 2009Geoscience WGMiddleware WG

Bioscience themeLeadershipDr. Mik Black, Biochemistry Department, University of OtagoThe TeamMikBlack, AnetteBecher, Cris Print, Jack Flanagan, Mike Cooling, Mike Eccles, NeilGemmell, Phil Wilcox, Robert Hickson, George Slim, Mark Gahegan, Nick Jones, Richard Li Research CommunitiesVirtual Institute of Statistical Genetics, MapNet, NZ Genomics Ltd, Centre for Reproduction and Genomics, Auckland Bioengineering Institute, Auckland Cancer Society Research Centre, Bioinformatics InstituteInitial Goals – Genetics/Genomics Storage and management of high-throughput data (e.g., microarray and sequence data). Distributed analysis of large data sets (analysis via GenePattern).

Storage and management of high-throughput data (e.g., microarray and sequence data).

Distributed analysis of large data sets (analysis via GenePattern).

GenePattern: a platform for integrative genomicsGenePattern provides access to a broad array of computational methods used to analyze genomic and proteomic data. It is designed to be easy for computational biologists to add their own analysis and visualizations, which enables access to new computational methods on a regular basis.ImagingClinical ResearchPathologyMolecular BiologySlide adapted from Ted LiefeldThe Broad Institute of MIT and Harvard

Geoscience themeLeadershipProf. Mark Gahegan, Centre for eResearch, University of Auckland The TeamMark Gahegan, Kevin Buckley, Isabella Cawthorn,Robert Gibb, Niels Hoffmann,FemkeReitsma,Nick Jones, Richard Li, Mik Black Research CommunitiesInvasive species modelling, surface and ground water modelling, ecosystem services modelling, carbon emissions, landcover and landuse databases, drill core integration, taxonomy reconcilliation, nationally significant databases development Initial Goals Invasive Species Modelling (R on the grid with workflows, mapping) Surface and Ground Water modelling (Lidar processing and workflows, mapping) Taxonomy Reconcilliation (Data and Collections Management)

Invasive Species Modelling (R on the grid with workflows, mapping)

Surface and Ground Water modelling (Lidar processing and workflows, mapping)

Taxonomy Reconcilliation (Data and Collections Management)

WSDataPaul Grimwood & colleagues, GNS

User Interface: PortalRobert Gibb, Niels Hoffmann, Landcare

Technology (future)WorkflowKeplercurrently issue with proxiesTrianaTavernaNo spatial componentsRobert Gibb, Niels Hoffmann, Landcare

Grid Middleware Planning

Technical Working GroupDeliver on needs of Bio and Geo themesMeet monthly, to present and discuss proposals, progress reports, and services and applications deployedMembershipInstitutions formally contracted to BeSTGRID + collaboratorsTerms of Reference, Meeting notes, etc on BeSTGRID websitehttp://www.bestgrid.org/index.php/Grid_Technical_Working_Group

SummaryWhat do we have, where are we going?

What do we have?A good high-speed network with capacity for growthThe foundations of a sophisticated infrastructure, some HPC and massive storage silos, Authentication, some grid middleware and basic services (but not nationally coordinated)Good links to e-research development communities: e.g. Australian National Collaborative Research Infrastructure Committee (NCRIS), Argonne (Globus), San Diego Supercomputer Center (SDSC), eScienceUK, UK NGS, OMII-UK (Edinburgh, Manchester)

BeSTGRID2009 – 2010MoRST Dec 2008 funding request forMiddleware development for eResearch in NZ $840k approved 24 Dec 2008, initiated April 2009Provides ongoing support for BeSTGRIDWidened from founders: Auckland, Canterbury & Masseyby active participation of new Universities and CRIsVictoria, Otago, Lincoln, Landcare Research, AUTAim: coordinate national grid middleware; use best-practice tools & services; focus on sustainability

MoRST Dec 2008 funding request forMiddleware development for eResearch in NZ

Provides ongoing support for BeSTGRID

Widened from founders: Auckland, Canterbury & Massey

by active participation of new Universities and CRIsVictoria, Otago, Lincoln, Landcare Research, AUT

Aim: coordinate national grid middleware; use best-practice tools & services; focus on sustainability

BeSTGRID2009 – 2010Form collaborations with related Science Infrastructure programmesSCENZ GridNew Zealand Genomics LimitedVirtual Institute of Statistical Genetics and MapNetSocial Sciences Data Archive and ServicesAuckland Cancer Society Research CentreSupport Science Gateways, applications and services infrastructure supports science communities with own specialised needsBio – Extension of scope from phylogenetics to other areas of Bioinformatics and Biomolecular sciencesGeo – collaboration with Canadian geoscience infrastructureAdd in new Data and HPC sites and services:Otago UniversityVictoria University WellingtonLandcare ResearchAuckland University Technology

SCENZ Grid

New Zealand Genomics Limited

Virtual Institute of Statistical Genetics and MapNet

Social Sciences Data Archive and Services

Auckland Cancer Society Research Centre

Bio – Extension of scope from phylogenetics to other areas of Bioinformatics and Biomolecular sciences

Geo – collaboration with Canadian geoscience infrastructure

Otago University

Victoria University Wellington

Landcare Research

Auckland University Technology

What next?Aligning with the NZ science communityDeveloping a RIAG bid for a long-term integrated landscape of eResearch including Federated IAM and HPC, as a National eResearch InfrastructureWhat are the research needs, tools, applications, environments, computing capabilities that we will need in New Zealand, over the next 10 - 20 years?Please get in touch if you would like to include your ideas and needs:m.gahegan@auckland.ac.nzn.jones@auckland.ac.nztim.david@canterbury.ac.nz

Thanks !!!

EndQuestions, comments