Slides from Jisc panel session at HPC & Big Data 2016 with contributions from the Francis Crick Institute, QMUL and King's College London covering their use of the Jisc shared data centre and the eMedLab project
Reading lists as open data - Meeting the Reading List Challenge 2016Martin Hamilton
My session at MTRLC 2016 describes how Jisc has been working with Lancaster University, the Open Data Institute and Talis to explore the potential of reading lists as open data. The immediate goal of this work is to use open reading list data to power an app for students, BookMart, which will help them to find affordable textbooks. This talk also covers other potential uses of open reading list data such as national deals and identifying areas where effort might usefully be devoted to open educational resources.
Supercomputing and the cloud - the next big paradigm shift?Martin Hamilton
How can cloud technologies help us to address the challenges of re-use of research data and software and reproducibility of experiments? My slides from the University of Birmingham BEARcloud launch event, October 2016
UK e-Infrastructure for Research - UK/USA HPC Workshop, Oxford, July 2015Martin Hamilton
A briefing on UK e-Infrastructure for research from Jisc and the UK research councils, presented at the UK/USA HPC workshop in July 2015, organized by HPC-SIG (UK) and CASC (USA).
The research data spring project "DataVault" slides for the third sandpit workshop. Project led by University of Manchester and University of Edinburgh.
Reading lists as open data - Meeting the Reading List Challenge 2016Martin Hamilton
My session at MTRLC 2016 describes how Jisc has been working with Lancaster University, the Open Data Institute and Talis to explore the potential of reading lists as open data. The immediate goal of this work is to use open reading list data to power an app for students, BookMart, which will help them to find affordable textbooks. This talk also covers other potential uses of open reading list data such as national deals and identifying areas where effort might usefully be devoted to open educational resources.
Supercomputing and the cloud - the next big paradigm shift?Martin Hamilton
How can cloud technologies help us to address the challenges of re-use of research data and software and reproducibility of experiments? My slides from the University of Birmingham BEARcloud launch event, October 2016
UK e-Infrastructure for Research - UK/USA HPC Workshop, Oxford, July 2015Martin Hamilton
A briefing on UK e-Infrastructure for research from Jisc and the UK research councils, presented at the UK/USA HPC workshop in July 2015, organized by HPC-SIG (UK) and CASC (USA).
The research data spring project "DataVault" slides for the third sandpit workshop. Project led by University of Manchester and University of Edinburgh.
The slides for my talk on "HPC as a service" at the 25th anniversary Machine Evaluation Workshop in December 2014. I cover Jisc's HPC brokerage and related initiatives including our shared data centre, industry connectivity to Janet, our VAT cost sharing group, and our pilot of the Kit-Catalogue equipment sharing database.
The user -driven evolution of Janet - Jisc Digifest 2016Jisc
Janet is one of the world’s most advanced networks built to support research and education across the UK, and through participation in GÉANT provides global reach, supporting key activities such as transnational education and access to global research facilities.
The latest version of the network – Janet6 – came into operation in November 2013. This talk will take a look at user requirements and how these are shaping the continued evolution of Janet to ensure that a flexible, reliable and secure network service is provided.
Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205Martin Hamilton
How can public cloud and technologies like Docker and OpenStack help to deliver next generation scientific computing infrastructure? My talk for the UK/USA HPC workshop in July 2015, organized by HPC-SIG (UK) and CASC (USA).
Presentation at the Open Knowledge Festival: Open Research and Education Stream, 20 September 2012, Helsinki; also
Presentation at the DINI-Jahrestagung - Bausteine für Open Science, 24 September 2012, Karlsruhe;
also Belgian Open Access Week: Open Access to Excellence in Research, 22 October 2012, Brussels.
Research data spring - Jisc Digital Festival 2015Jisc
This demonstration explored a few ideas and the collborative process implemented by Jisc R&D to select ideas and gather feedback for technical tools, software and service solutions to support the management of research data.
Jisc - Rebooting a National Innovation Agency (EUNIS 2014)Martin Hamilton
This is my presentation on "Rebooting" Jisc, from the EUNIS 2014 Congress at Umeå, Sweden. I begin by introducing Jisc, for anyone not already familiar with who we are and what we do. I highlight a few of our success stories that the EUNIS audience might not be familiar with, talk about some current projects - and how our focus and structure has changed following the Wilson Review. I close with our mission statement and vision for 2020.
Health and clinical research - data futures, NIHR accelerating digital programmeMartin Hamilton
My slides from the National Institute for Health Research's "Visioning the Future Clinical Research Network" event in London on May 3rd 2016. I look at Jisc initiatives supporting health and clinical research, consumer led technology trends, digital capability and digital leadership, and areas where we can come together as a community
Big data and the dark arts - Jisc Digital Media 2015Jisc
There still remains a certain misunderstanding by the very definition of "big data" and the perceived hype around the term. This workshop clarified the concepts and give examples of relevant big data projects.
From Jisc's campus network engineering for data-intensive science workshop on 19 October 2016.
https://www.jisc.ac.uk/events/campus-network-engineering-for-data-intensive-science-workshop-19-oct-2016
The Climate Tagger - a tagging and recommender service for climate informatio...Martin Kaltenböck
The Climate Tagger - a tagging- and recommender service for climate information based on PoolParty Semantic Suite - slides of the talk by Sukaina Bharwani (Stockholm Environment Institute, SEI Oxford) and Martin Kaltenböck (Semantic Web Company, SWC Vienna) at the Taxonomy Boot Camp London 2016 (TBC London) taking place on 19.10.2016
Bob Jones, CERN & HNSciCloud Coordinator gives an update on the HNSciCloud Pre-Commercial Procurement which is now in its Solution Prototyping phase. The presentation includes also an overview of the prototypes under development.
This presentation, given by Bob Jones, CERN & HNSciCloud Coordinator, at the ESA-ESPI Workshop on “Space Data & Cloud Computing Infrastructures: Policies and Regulations”, describes what are the challenges and needs of the cloud users and explains how an hybrid cloud model can support them.
Slides from my panel session at Science & Innovation 2015 with STFC DiRAC, HPC Midlands, Francis Crick Institute and UCL. As we move into the expected post-election comprehensive spending review, it is a good time to take stock of some of the innovations that have helped the UK’s institutions and industry to work together to accelerate innovation whilst achieving operating efficiencies over the last few years.
In this session we hear about trend setting initiatives such as Jisc’s shared data centre and equipment sharing initiative, which makes over £200m of capital equipment available for sharing between institutions and with industry, and industrial connectivity to the UK’s Janet network.
PoolParty Semantic Suite - LT-Innovate Industry Summit-2016 - BrusselsMartin Kaltenböck
Slides of the 'Technology Spotlight: PoolParty Semantic Suite (http://www.poolparty.biz)' by Martin Kaltenböck (Managing Partner of Semantic Web Company) on 17 May 2016 at the LT-Innovate Industry Summit 2016 at the International Auditorium in Brussels (http://www.lt-innovate.org/).
Slides from my talk on R&D innovation projects around the Janet network for the HEAnet / Juniper Innovation Day, September 2015. I talk about some recent Janet R&D initiatives such as our Reach scheme for connecting industry to the network, our end to end performance initiative, and our Safe Share project for secure access to sensitive data by researchers - e.g. medical records. There is also a recap of some of our recent activity around equipment sharing, our shared data centre, connectivity and deals with major cloud providers.
In this deck from the 2019 Stanford HPC Conference, Nik Nystrom from the Pittsburgh Supercomputing Center presents: Pioneering and Democratizing Scalable HPC+AI.
"PSC's Bridges was the first system to successfully converge HPC, AI, and Big Data. Designed for the U.S. national research community and supported by NSF, Bridges now serves approximately 1800 projects and 7500 users at 380 institutions, and it is the foundation around which new HPC+AI projects have launched. Bridges emphasizes "nontraditional" uses that span the life, physical, and social sciences, computer science, engineering, business, and humanities. Scalable HPC+AI is driving many of those applications, which span diverse topics such as learning root causes of cancer, strategic reasoning, designing new materials, predicting severe storms, recognizing speech including contextual information, and detecting objects in 4k streaming video. To address the demand for scalable AI, PSC recently introduced Bridges-AI, which adds transformative new AI capability. In this presentation, we share our vision in designing HPC+AI systems at PSC and highlight some of the exciting research breakthroughs they are enabling."
Nick Nystrom is Interim Director and Sr. Director of Research at the Pittsburgh Supercomputing Center (PSC). Nick is architect and PI for Bridges, PSC's flagship system that successfully pioneered the convergence HPC, AI, and Big Data. He is also PI for the NIH Human Biomolecular Atlas Program’s HIVE Infrastructure Component and co-PI for projects that bring emerging AI technologies to research (Open Compass), apply machine learning to biomedical data for breast and lung cancer (Big Data for Better Health), and identify causal relationships in biomedical big data (the Center for Causal Discovery, an NIH Big Data to Knowledge Center of Excellence). His current research interests include hardware and software architecture, applications of machine learning to multimodal data (particularly for the life sciences) and to enhance simulation, and graph analytics.
Watch the video: https://youtu.be/ucRs4A_afus
Learn more: https://www.psc.edu/bridges
and
http://hpcadvisorycouncil.com/events/2019/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
The slides for my talk on "HPC as a service" at the 25th anniversary Machine Evaluation Workshop in December 2014. I cover Jisc's HPC brokerage and related initiatives including our shared data centre, industry connectivity to Janet, our VAT cost sharing group, and our pilot of the Kit-Catalogue equipment sharing database.
The user -driven evolution of Janet - Jisc Digifest 2016Jisc
Janet is one of the world’s most advanced networks built to support research and education across the UK, and through participation in GÉANT provides global reach, supporting key activities such as transnational education and access to global research facilities.
The latest version of the network – Janet6 – came into operation in November 2013. This talk will take a look at user requirements and how these are shaping the continued evolution of Janet to ensure that a flexible, reliable and secure network service is provided.
Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205Martin Hamilton
How can public cloud and technologies like Docker and OpenStack help to deliver next generation scientific computing infrastructure? My talk for the UK/USA HPC workshop in July 2015, organized by HPC-SIG (UK) and CASC (USA).
Presentation at the Open Knowledge Festival: Open Research and Education Stream, 20 September 2012, Helsinki; also
Presentation at the DINI-Jahrestagung - Bausteine für Open Science, 24 September 2012, Karlsruhe;
also Belgian Open Access Week: Open Access to Excellence in Research, 22 October 2012, Brussels.
Research data spring - Jisc Digital Festival 2015Jisc
This demonstration explored a few ideas and the collborative process implemented by Jisc R&D to select ideas and gather feedback for technical tools, software and service solutions to support the management of research data.
Jisc - Rebooting a National Innovation Agency (EUNIS 2014)Martin Hamilton
This is my presentation on "Rebooting" Jisc, from the EUNIS 2014 Congress at Umeå, Sweden. I begin by introducing Jisc, for anyone not already familiar with who we are and what we do. I highlight a few of our success stories that the EUNIS audience might not be familiar with, talk about some current projects - and how our focus and structure has changed following the Wilson Review. I close with our mission statement and vision for 2020.
Health and clinical research - data futures, NIHR accelerating digital programmeMartin Hamilton
My slides from the National Institute for Health Research's "Visioning the Future Clinical Research Network" event in London on May 3rd 2016. I look at Jisc initiatives supporting health and clinical research, consumer led technology trends, digital capability and digital leadership, and areas where we can come together as a community
Big data and the dark arts - Jisc Digital Media 2015Jisc
There still remains a certain misunderstanding by the very definition of "big data" and the perceived hype around the term. This workshop clarified the concepts and give examples of relevant big data projects.
From Jisc's campus network engineering for data-intensive science workshop on 19 October 2016.
https://www.jisc.ac.uk/events/campus-network-engineering-for-data-intensive-science-workshop-19-oct-2016
The Climate Tagger - a tagging and recommender service for climate informatio...Martin Kaltenböck
The Climate Tagger - a tagging- and recommender service for climate information based on PoolParty Semantic Suite - slides of the talk by Sukaina Bharwani (Stockholm Environment Institute, SEI Oxford) and Martin Kaltenböck (Semantic Web Company, SWC Vienna) at the Taxonomy Boot Camp London 2016 (TBC London) taking place on 19.10.2016
Bob Jones, CERN & HNSciCloud Coordinator gives an update on the HNSciCloud Pre-Commercial Procurement which is now in its Solution Prototyping phase. The presentation includes also an overview of the prototypes under development.
This presentation, given by Bob Jones, CERN & HNSciCloud Coordinator, at the ESA-ESPI Workshop on “Space Data & Cloud Computing Infrastructures: Policies and Regulations”, describes what are the challenges and needs of the cloud users and explains how an hybrid cloud model can support them.
Slides from my panel session at Science & Innovation 2015 with STFC DiRAC, HPC Midlands, Francis Crick Institute and UCL. As we move into the expected post-election comprehensive spending review, it is a good time to take stock of some of the innovations that have helped the UK’s institutions and industry to work together to accelerate innovation whilst achieving operating efficiencies over the last few years.
In this session we hear about trend setting initiatives such as Jisc’s shared data centre and equipment sharing initiative, which makes over £200m of capital equipment available for sharing between institutions and with industry, and industrial connectivity to the UK’s Janet network.
PoolParty Semantic Suite - LT-Innovate Industry Summit-2016 - BrusselsMartin Kaltenböck
Slides of the 'Technology Spotlight: PoolParty Semantic Suite (http://www.poolparty.biz)' by Martin Kaltenböck (Managing Partner of Semantic Web Company) on 17 May 2016 at the LT-Innovate Industry Summit 2016 at the International Auditorium in Brussels (http://www.lt-innovate.org/).
Slides from my talk on R&D innovation projects around the Janet network for the HEAnet / Juniper Innovation Day, September 2015. I talk about some recent Janet R&D initiatives such as our Reach scheme for connecting industry to the network, our end to end performance initiative, and our Safe Share project for secure access to sensitive data by researchers - e.g. medical records. There is also a recap of some of our recent activity around equipment sharing, our shared data centre, connectivity and deals with major cloud providers.
In this deck from the 2019 Stanford HPC Conference, Nik Nystrom from the Pittsburgh Supercomputing Center presents: Pioneering and Democratizing Scalable HPC+AI.
"PSC's Bridges was the first system to successfully converge HPC, AI, and Big Data. Designed for the U.S. national research community and supported by NSF, Bridges now serves approximately 1800 projects and 7500 users at 380 institutions, and it is the foundation around which new HPC+AI projects have launched. Bridges emphasizes "nontraditional" uses that span the life, physical, and social sciences, computer science, engineering, business, and humanities. Scalable HPC+AI is driving many of those applications, which span diverse topics such as learning root causes of cancer, strategic reasoning, designing new materials, predicting severe storms, recognizing speech including contextual information, and detecting objects in 4k streaming video. To address the demand for scalable AI, PSC recently introduced Bridges-AI, which adds transformative new AI capability. In this presentation, we share our vision in designing HPC+AI systems at PSC and highlight some of the exciting research breakthroughs they are enabling."
Nick Nystrom is Interim Director and Sr. Director of Research at the Pittsburgh Supercomputing Center (PSC). Nick is architect and PI for Bridges, PSC's flagship system that successfully pioneered the convergence HPC, AI, and Big Data. He is also PI for the NIH Human Biomolecular Atlas Program’s HIVE Infrastructure Component and co-PI for projects that bring emerging AI technologies to research (Open Compass), apply machine learning to biomedical data for breast and lung cancer (Big Data for Better Health), and identify causal relationships in biomedical big data (the Center for Causal Discovery, an NIH Big Data to Knowledge Center of Excellence). His current research interests include hardware and software architecture, applications of machine learning to multimodal data (particularly for the life sciences) and to enhance simulation, and graph analytics.
Watch the video: https://youtu.be/ucRs4A_afus
Learn more: https://www.psc.edu/bridges
and
http://hpcadvisorycouncil.com/events/2019/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this presentation from the Dell booth at SC13, Joseph Antony from NCI describes how they are using HPC Virtualization to meet user needs.
Watch the video presentation: http://insidehpc.com/2013/12/05/panel-discussion-thought-hpc-virtualization-never-going-happen/
RD shared services and research data springJisc RDM
Daniela Duca's presentation at the DataVault workshop on 29 June. An overview of research at risk, research data shared service and research data spring.
The Jisc Research Data Shared Service (RDSS) is a
project that will integrate the offerings of a number of
repository, preservation, storage, reporting and information
management providers in order to allow UK universities to
easily deposit data for publication, discovery, safe storage,
and long-term archiving and preservation. This is a pilot
project until April 2018 working with 17 UK universities with
the overarching aim to ensure the long-term accessibility of
valuable research data, allowing it to be reused and shared.
To foster greater and more consistent use of the new 100 Gbps connections that is being deployed in the national RNP backbone, the e-Cyber project aims at delivering high-performing services to the most infrastructure-demanding research centers in Brazil. To do this, the project is getting inspired by the “superfacility” concept, which is adopted by initiatives like GRP (Global Research Platform) and EOSC (European Open Science Cloud). However, one of our biggest challenges is to engage the client institutions and bring them to co-create solutions and participate in the project governance.
Big Data HPC Convergence and a bunch of other thingsGeoffrey Fox
This talk supports the Ph.D. in Computational & Data Enabled Science & Engineering at Jackson State University. It describes related educational activities at Indiana University, the Big Data phenomena, jobs and HPC and Big Data computations. It then describes how HPC and Big Data can be converged into a single theme.
If Big Data is data that exceeds the processing capacity of conventional systems, thereby necessitating alternative processing measures, we are looking at an essentially technological challenge that IT managers are best equipped to address.
The DCC is currently working with 18 HEIs to support and develop their capabilities in the management of research data and, whilst the aforementioned challenge is not usually core to their expressed concerns, are there particular issues of curation inherent to Big Data that might force a different perspective?
We have some understanding of Big Data from our contacts in the Astronomy and High Energy Physics domains, and the scale and speed of development in Genomics data generation is well known, but the inability to provide sufficient processing capacity is not one of their more frequent complaints.
That’s not to say that Big Science and its Big Data are free of challenges in data curation; only that they are shared with their lesser cousins, where one might say that the real challenge is less one of size than diversity and complexity.
This brief presentation explores those aspects of data curation that go beyond the challenges of processing power but which may lend a broader perspective to the technology selection process.
Introduction of streaming data, difference between batch processing and stream processing, Research issues in streaming data processing, Performance evaluation metrics , tools for stream processing.
A talk at the RPI-NSF Workshop on Multiscale Modeling of Complex Data, September 12, 2011, Troy NY, USA.
We have made much progress over the past decade toward effectively
harnessing the collective power of IT resources distributed across the
globe. In fields such as high-energy physics, astronomy, and climate,
thousands benefit daily from tools that manage and analyze large
quantities of data produced and consumed by large collaborative teams.
But we now face a far greater challenge: Exploding data volumes and powerful simulation tools mean that far more--ultimately
most?--researchers will soon require capabilities not so different from those used by these big-science teams. How is the general population of researchers and institutions to meet these needs? Must every lab be filled
with computers loaded with sophisticated software, and every researcher become an information technology (IT) specialist? Can we possibly afford to equip our labs in this way, and where would we find the experts to operate them?
Consumers and businesses face similar challenges, and industry has
responded by moving IT out of homes and offices to so-called cloud providers (e.g., GMail, Google Docs, Salesforce), slashing costs and complexity. I suggest that by similarly moving research IT out of the lab, we can realize comparable economies of scale and reductions in complexity. More importantly, we can free researchers from the burden of managing IT, giving them back their time to focus on research and empowering them to go beyond the scope of what was previously possible.
I describe work we are doing at the Computation Institute to realize this approach, focusing initially on research data lifecycle management. I present promising results obtained to date and suggest a path towards
large-scale delivery of these capabilities.
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013SALCTG
An overview of Research Data Management: the research process from developing ideas to preservation of data; funder perspectives, the impact on the wider service, Data Asset Frameworks, preservation and access, and cost implications.
A brief overview of the development and current workflows for Research Data Management at Imperial College London, presented to colleagues at the University of Copenhagen and Roskilde University in Denmark.
Similar to Shared services - the future of HPC and big data facilities for UK research (20)
Keep taking the tablets? The graduation of the iPad generationMartin Hamilton
Over the last ten years, smartphones and tablets and pervasive Internet connectivity have taken over our lives, and more importantly the lives of our children. In this talk for Loughborough University's 2018 Learning and Teaching conference I reflect on how higher education could respond to this trend. I invite the audience to consider what children who have always had access to the sum total of human knowledge will will expect and need from universities in the future, and how we might best support learners from disadvantaged backgrounds to thrive
The Intelligent Campus - Where the Internet of Things meets AI - HESCA June 2018Martin Hamilton
In this talk for the Higher Education Smart Card Assocation (HESCA) I introduce Jisc's Intelligent Campus initiative, which is looking at how we can connect data from Internet of Things sensors to Learning Analytics services to improve learning outcomes and attainment whilst also delivering institutional efficiency savings and exploring new delivery models for higher education
The Digital Book Thief has a Napster Moment - Edinburgh Near Future Library S...Martin Hamilton
We’ve grown accustomed to being able to call up any information, anywhere, any time - but what happens when our digital landlord forecloses? When that service we entrusted our data to goes to the Startup Graveyard? In this talk for the University of Edinburgh's Near Future Library Symposium I highlight some of the risks to our culture and the scientific record, and what the near future library can do to help
Martin Hamilton - The wind from nowhere - Horizon scanning in an uncertain ag...Martin Hamilton
Does it feel like the world has recently become much less certain and predictable? We tend to expect that each day will be like the last, more or less, but incremental change is increasingly feeling like a thing of the past. In this talk for the 2018 Scientia EMEA User Conference I look at how we can read the signals around us to better predict the future, and consider the impact of technologies and trends like blockchain, artificial intelligence and Brexit on further and higher education
From Blockchain to Brexit - edtech trends for 2018 - BETT 2018Martin Hamilton
In this talk for BETT 2018 I take a look at a few of the socio-technical trends that are set to have a big impact on universities and colleges in 2018 from blockchain to Brexit, and data vandalism to UK spaceports. I look at some approaches that institutions can take to help plan for an uncertain future, and consider how the community can mobilise to protect the progressive values that now often seem to be under threat.
Martin Hamilton - Digital skills: You won't believe what happened next!Martin Hamilton
We're often told how it's vitally important for everyone to develop their digital skills, but what digital skills will we need for near future careers, and how can we go about developing them? In this talk for the Manchester Digital Skills Festival 2018 I take a look at a few examples, including some that are literally out of this world
Martin Hamilton - Librarians in Outer Space - CILIP invited talkMartin Hamilton
We are becoming used to living in an interconnected world, with vast amounts of data at our fingertips, but what happens when our preconceptions are challenged? What happens when the things that we take for granted simply don’t work any more? How can librarians rise to the challenge? In this invited talk for CILIP, I reflect on the impact for libraries and librarians of some of the defining narratives of the late Anthropocene era: from climate change and failed states to cheap space travel and artificial intelligence
Martin Hamilton - The impact of technology on the higher education sector - L...Martin Hamilton
How digital is your university? In this talk for the Leadership Foundation for Higher Education (LFHE) I invite viewers to think about the impact of technology on the sector, with some examples of disruptive technologies and trends, like Udacity's self-driving car engineering nanodegree
Martin Hamilton - Robots and AI, the calm before the Singularity? - BCS invit...Martin Hamilton
In this invited talk for the BCS I look at the state of the art in robots and artificial intelligence, and what this tells us about the near-term future that our children and grandchildren will live in. Will the imminent arrival of AI powered self-driving vehicles mean redundancies for truck and taxi drivers? What will these people do next? I also show how robots and AI are already becoming commonplace, working in places like shops, restaurants and distribution centres
Martin Hamilton - What did your AI make today? - BCS invited talkMartin Hamilton
Artificial intelligence and machine learning are used extensively by Internet services to help us find and classify information. This is how products like Google Photos can "magically" find all your photos of cats or trees - or cats up trees! But there's a new wave of AI called generative adversarial neural networks that's all about using AI to make things. In this invited talk for the British Computer Society (BCS) I reflect on what we could use this technology for and the implications for people and society.
Blockchain in research and education - UKSG Webinar - September 2017Martin Hamilton
There’s a lot of hype right now about blockchain, the technology that underpins the Bitcoin virtual currency, with speculation that it could transform just about every aspect of our lives. In this talk for UKSG I consider possible blockchain applications in research and education, and do a little myth-busting about when and where it makes sense to use blockchain.
HPC in the cloud comes of age - Red Oak HPC SeminarMartin Hamilton
What does HPC in the cloud look like in 2017, and how did we get there? In this talk for Red Oak's HPC Seminar, I look at the origins of cloud HPC, and how it has become mainstream through technologies like Amazon Web Services and OpenStack. I also offer a sneak preview of the 2017 UK national e-Infrastructure survey results, and some thoughts about what's next in cloud HPC, from hyperscale providers to the momentum behind container technologies from Docker and the Open Container Initiative.
Imagining Mars University - Universities UK 2017 conferenceMartin Hamilton
In this talk from the Universities UK 2017 annual conference I update delegates on the latest developments in brain computer interfaces, artificial intelligence, robotics and affordable space travel, and encourage them to consider what Mars University might look like.
In this talk for the University of Glasgow's Future Proof IT event I explore a few near future careers and technologies that will impact learners and institutions, such as self-driving vehicles, and how we might respond to them.
Tech in exams - SQA Assessment Expert Group - June 2017Martin Hamilton
Is it time to start looking at how we can embrace technology in exams, instead of banning it? My elevator pitch to the Scottish Qualifications Authority assessment expert group meeting in June 2017.
Through the Overton Window - Health Education England horizon scanning worksh...Martin Hamilton
Video of my talk for Health Education England's horizon scanning workshop in June 2017, looking at how we can break out of our comfort zone and move our own personal Overton Window to think about teaching and learning in healthcare in the near future.
A new life awaits you in the off world colonies - UCISA Spotlight on Digital ...Martin Hamilton
The chance to begin again in a golden land of opportunity and adventure? In this talk for the 2017 UCISA Spotlight on Digital Capabilities event I invite the audience to consider what and how the DNA editors and asteroid miners of tomorrow will be learning, and what digital capability and digital skills will mean for them.
Help! My robot is a teacher! - Future Edtech 2017Martin Hamilton
Robots and AI powered digital assistants are increasingly becoming part of our lives.But one person's utopia could be another's dypstopia. This talk for Future Edtech 2017 looks at how could we use these new technologies to help teachers and learners, and what to do if it all goes horribly wrong.
Towards a UK Edtech Strategy - Edtech Vision 2020Martin Hamilton
Do we need an Edtech Strategy for the UK, and what would that look like? Some thoughts on blockers and potential enablers from myself and Jisc CIO Phil Richards, presented at our joint June Edtech Vision 2020 event with EdtechUK
Bridging the digital divide - Digital Skills Summit 2017Martin Hamilton
My talk on Jisc support for digital skills, digitally enabled apprenticeships and building digital capability in organisations, from the 2017 Sunderland Software City Digital Skills Summit #DSS17
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.Sérgio Sacani
The return of a sample of near-surface atmosphere from Mars would facilitate answers to several first-order science questions surrounding the formation and evolution of the planet. One of the important aspects of terrestrial planet formation in general is the role that primary atmospheres played in influencing the chemistry and structure of the planets and their antecedents. Studies of the martian atmosphere can be used to investigate the role of a primary atmosphere in its history. Atmosphere samples would also inform our understanding of the near-surface chemistry of the planet, and ultimately the prospects for life. High-precision isotopic analyses of constituent gases are needed to address these questions, requiring that the analyses are made on returned samples rather than in situ.
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
Richard's entangled aventures in wonderlandRichard Gill
Since the loophole-free Bell experiments of 2020 and the Nobel prizes in physics of 2022, critics of Bell's work have retreated to the fortress of super-determinism. Now, super-determinism is a derogatory word - it just means "determinism". Palmer, Hance and Hossenfelder argue that quantum mechanics and determinism are not incompatible, using a sophisticated mathematical construction based on a subtle thinning of allowed states and measurements in quantum mechanics, such that what is left appears to make Bell's argument fail, without altering the empirical predictions of quantum mechanics. I think however that it is a smoke screen, and the slogan "lost in math" comes to my mind. I will discuss some other recent disproofs of Bell's theorem using the language of causality based on causal graphs. Causal thinking is also central to law and justice. I will mention surprising connections to my work on serial killer nurse cases, in particular the Dutch case of Lucia de Berk and the current UK case of Lucy Letby.
A brief information about the SCOP protein database used in bioinformatics.
The Structural Classification of Proteins (SCOP) database is a comprehensive and authoritative resource for the structural and evolutionary relationships of proteins. It provides a detailed and curated classification of protein structures, grouping them into families, superfamilies, and folds based on their structural and sequence similarities.
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Sérgio Sacani
Since volcanic activity was first discovered on Io from Voyager images in 1979, changes
on Io’s surface have been monitored from both spacecraft and ground-based telescopes.
Here, we present the highest spatial resolution images of Io ever obtained from a groundbased telescope. These images, acquired by the SHARK-VIS instrument on the Large
Binocular Telescope, show evidence of a major resurfacing event on Io’s trailing hemisphere. When compared to the most recent spacecraft images, the SHARK-VIS images
show that a plume deposit from a powerful eruption at Pillan Patera has covered part
of the long-lived Pele plume deposit. Although this type of resurfacing event may be common on Io, few have been detected due to the rarity of spacecraft visits and the previously low spatial resolution available from Earth-based telescopes. The SHARK-VIS instrument ushers in a new era of high resolution imaging of Io’s surface using adaptive
optics at visible wavelengths.
Slide 1: Title Slide
Extrachromosomal Inheritance
Slide 2: Introduction to Extrachromosomal Inheritance
Definition: Extrachromosomal inheritance refers to the transmission of genetic material that is not found within the nucleus.
Key Components: Involves genes located in mitochondria, chloroplasts, and plasmids.
Slide 3: Mitochondrial Inheritance
Mitochondria: Organelles responsible for energy production.
Mitochondrial DNA (mtDNA): Circular DNA molecule found in mitochondria.
Inheritance Pattern: Maternally inherited, meaning it is passed from mothers to all their offspring.
Diseases: Examples include Leber’s hereditary optic neuropathy (LHON) and mitochondrial myopathy.
Slide 4: Chloroplast Inheritance
Chloroplasts: Organelles responsible for photosynthesis in plants.
Chloroplast DNA (cpDNA): Circular DNA molecule found in chloroplasts.
Inheritance Pattern: Often maternally inherited in most plants, but can vary in some species.
Examples: Variegation in plants, where leaf color patterns are determined by chloroplast DNA.
Slide 5: Plasmid Inheritance
Plasmids: Small, circular DNA molecules found in bacteria and some eukaryotes.
Features: Can carry antibiotic resistance genes and can be transferred between cells through processes like conjugation.
Significance: Important in biotechnology for gene cloning and genetic engineering.
Slide 6: Mechanisms of Extrachromosomal Inheritance
Non-Mendelian Patterns: Do not follow Mendel’s laws of inheritance.
Cytoplasmic Segregation: During cell division, organelles like mitochondria and chloroplasts are randomly distributed to daughter cells.
Heteroplasmy: Presence of more than one type of organellar genome within a cell, leading to variation in expression.
Slide 7: Examples of Extrachromosomal Inheritance
Four O’clock Plant (Mirabilis jalapa): Shows variegated leaves due to different cpDNA in leaf cells.
Petite Mutants in Yeast: Result from mutations in mitochondrial DNA affecting respiration.
Slide 8: Importance of Extrachromosomal Inheritance
Evolution: Provides insight into the evolution of eukaryotic cells.
Medicine: Understanding mitochondrial inheritance helps in diagnosing and treating mitochondrial diseases.
Agriculture: Chloroplast inheritance can be used in plant breeding and genetic modification.
Slide 9: Recent Research and Advances
Gene Editing: Techniques like CRISPR-Cas9 are being used to edit mitochondrial and chloroplast DNA.
Therapies: Development of mitochondrial replacement therapy (MRT) for preventing mitochondrial diseases.
Slide 10: Conclusion
Summary: Extrachromosomal inheritance involves the transmission of genetic material outside the nucleus and plays a crucial role in genetics, medicine, and biotechnology.
Future Directions: Continued research and technological advancements hold promise for new treatments and applications.
Slide 11: Questions and Discussion
Invite Audience: Open the floor for any questions or further discussion on the topic.
This pdf is about the Schizophrenia.
For more details visit on YouTube; @SELF-EXPLANATORY;
https://www.youtube.com/channel/UCAiarMZDNhe1A3Rnpr_WkzA/videos
Thanks...!
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Ana Luísa Pinho
Functional Magnetic Resonance Imaging (fMRI) provides means to characterize brain activations in response to behavior. However, cognitive neuroscience has been limited to group-level effects referring to the performance of specific tasks. To obtain the functional profile of elementary cognitive mechanisms, the combination of brain responses to many tasks is required. Yet, to date, both structural atlases and parcellation-based activations do not fully account for cognitive function and still present several limitations. Further, they do not adapt overall to individual characteristics. In this talk, I will give an account of deep-behavioral phenotyping strategies, namely data-driven methods in large task-fMRI datasets, to optimize functional brain-data collection and improve inference of effects-of-interest related to mental processes. Key to this approach is the employment of fast multi-functional paradigms rich on features that can be well parametrized and, consequently, facilitate the creation of psycho-physiological constructs to be modelled with imaging data. Particular emphasis will be given to music stimuli when studying high-order cognitive mechanisms, due to their ecological nature and quality to enable complex behavior compounded by discrete entities. I will also discuss how deep-behavioral phenotyping and individualized models applied to neuroimaging data can better account for the subject-specific organization of domain-general cognitive systems in the human brain. Finally, the accumulation of functional brain signatures brings the possibility to clarify relationships among tasks and create a univocal link between brain systems and mental functions through: (1) the development of ontologies proposing an organization of cognitive processes; and (2) brain-network taxonomies describing functional specialization. To this end, tools to improve commensurability in cognitive science are necessary, such as public repositories, ontology-based platforms and automated meta-analysis tools. I will thus discuss some brain-atlasing resources currently under development, and their applicability in cognitive as well as clinical neuroscience.
Multi-source connectivity as the driver of solar wind variability in the heli...Sérgio Sacani
The ambient solar wind that flls the heliosphere originates from multiple
sources in the solar corona and is highly structured. It is often described
as high-speed, relatively homogeneous, plasma streams from coronal
holes and slow-speed, highly variable, streams whose source regions are
under debate. A key goal of ESA/NASA’s Solar Orbiter mission is to identify
solar wind sources and understand what drives the complexity seen in the
heliosphere. By combining magnetic feld modelling and spectroscopic
techniques with high-resolution observations and measurements, we show
that the solar wind variability detected in situ by Solar Orbiter in March
2022 is driven by spatio-temporal changes in the magnetic connectivity to
multiple sources in the solar atmosphere. The magnetic feld footpoints
connected to the spacecraft moved from the boundaries of a coronal hole
to one active region (12961) and then across to another region (12957). This
is refected in the in situ measurements, which show the transition from fast
to highly Alfvénic then to slow solar wind that is disrupted by the arrival of
a coronal mass ejection. Our results describe solar wind variability at 0.5 au
but are applicable to near-Earth observatories.
Introduction:
RNA interference (RNAi) or Post-Transcriptional Gene Silencing (PTGS) is an important biological process for modulating eukaryotic gene expression.
It is highly conserved process of posttranscriptional gene silencing by which double stranded RNA (dsRNA) causes sequence-specific degradation of mRNA sequences.
dsRNA-induced gene silencing (RNAi) is reported in a wide range of eukaryotes ranging from worms, insects, mammals and plants.
This process mediates resistance to both endogenous parasitic and exogenous pathogenic nucleic acids, and regulates the expression of protein-coding genes.
What are small ncRNAs?
micro RNA (miRNA)
short interfering RNA (siRNA)
Properties of small non-coding RNA:
Involved in silencing mRNA transcripts.
Called “small” because they are usually only about 21-24 nucleotides long.
Synthesized by first cutting up longer precursor sequences (like the 61nt one that Lee discovered).
Silence an mRNA by base pairing with some sequence on the mRNA.
Discovery of siRNA?
The first small RNA:
In 1993 Rosalind Lee (Victor Ambros lab) was studying a non- coding gene in C. elegans, lin-4, that was involved in silencing of another gene, lin-14, at the appropriate time in the
development of the worm C. elegans.
Two small transcripts of lin-4 (22nt and 61nt) were found to be complementary to a sequence in the 3' UTR of lin-14.
Because lin-4 encoded no protein, she deduced that it must be these transcripts that are causing the silencing by RNA-RNA interactions.
Types of RNAi ( non coding RNA)
MiRNA
Length (23-25 nt)
Trans acting
Binds with target MRNA in mismatch
Translation inhibition
Si RNA
Length 21 nt.
Cis acting
Bind with target Mrna in perfect complementary sequence
Piwi-RNA
Length ; 25 to 36 nt.
Expressed in Germ Cells
Regulates trnasposomes activity
MECHANISM OF RNAI:
First the double-stranded RNA teams up with a protein complex named Dicer, which cuts the long RNA into short pieces.
Then another protein complex called RISC (RNA-induced silencing complex) discards one of the two RNA strands.
The RISC-docked, single-stranded RNA then pairs with the homologous mRNA and destroys it.
THE RISC COMPLEX:
RISC is large(>500kD) RNA multi- protein Binding complex which triggers MRNA degradation in response to MRNA
Unwinding of double stranded Si RNA by ATP independent Helicase
Active component of RISC is Ago proteins( ENDONUCLEASE) which cleave target MRNA.
DICER: endonuclease (RNase Family III)
Argonaute: Central Component of the RNA-Induced Silencing Complex (RISC)
One strand of the dsRNA produced by Dicer is retained in the RISC complex in association with Argonaute
ARGONAUTE PROTEIN :
1.PAZ(PIWI/Argonaute/ Zwille)- Recognition of target MRNA
2.PIWI (p-element induced wimpy Testis)- breaks Phosphodiester bond of mRNA.)RNAse H activity.
MiRNA:
The Double-stranded RNAs are naturally produced in eukaryotic cells during development, and they have a key role in regulating gene expression .
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
Shared services - the future of HPC and big data facilities for UK research
1. Shared services – the future of HPC & big data facilities for UK research?
Martin Hamilton, Jisc
David Fergusson & Bruno Silva, Francis Crick Institute
Andreas Biternas, King’s College london
Thomas King, Queen Mary University of London
Photo credit: CC-BY-NC-ND JiscHPC & Big Data 2016
2. Shared services for HPC & big data
1. About Jisc
– Who, why and what?
– Success stories
2. Recent developments
3. Personal perspectives & panel discussion
– David Fergusson & Bruno Silva, Francis Crick Institute
– Andreas Biternas, King’sCollege London
– Thomas King, Queen Mary University of London
4. 1. About Jisc
Jisc is the UK higher education, further education and
skills sectors’ not-for-profit organisation for digital
services and solutions.This is what we do:
› Operate shared digital infrastructure and services
for universities and colleges
› Negotiate sector-wide deals, e.g. with IT vendors
and commercial publishers
› Provide trusted advice and practical assistance
8. 1. About Jisc
Netflix
Voicenet
Akamai
Virgin Radio
Bogons
Logicalis UK
Pipex / GXN
BBC
Datahop
InTechnology
INUK
Simplecall
LINX multicast
Gamma
Google
Simplecall
Redstone
Updata
aql
Voicenet
Google
Limelight
Limelight
Akamai
BTnet
Init7
Amazon
Microsoft EU (viaTN)
Telekom Malaysia
Globelynx
10Gbit/s
1Gbit/s
100Gbit/s
GÉANT
GÉANT+
LINX
Microsoft EU (viaTW)
Total external connectivity ≈ 1Tbit/s
Leeds
Akamai
Google
VM for LGfLInTechnology
NHS N3
Exa Networks
Synetrix BBC (HD 4K pilots)
One Connect
Glasgow
&
Edinburgh
HEAnet
BBC (Pacific Quay)
Gamma
BBC (HD 4K pilots)
NHS N3
SWAN (Glas)
SWAN (Edin)
Manchester
Telecity
Harbour
Exch.
Telehouse
North &
West
VM for LGfL
RM for Schools
VM for LGfL
RM for Schools
GlobalTransit
Tata
IXManchester
IXLeeds
GlobalTransit
Level3
GlobalTransit
Level3
12. 2. Recent developments
www.jisc.ac.uk/financial-x-ray
Financial X-Ray
› Easily understand and compare
overall costs for services
› Develop business cases for
changes to IT infrastructure
› Mechanism for dialogue
between finance and IT
departments
› Highlight comparative cost of
shared and commercial third
party services
13. 2. Recent developments
Assent (formerly Project
Moonshot)
› Single, unifying technology that enables
you to effectively manage and control
access to a wide range of web and non-
web services and applications.
› These include cloud infrastructures, High
Performance Computing, Grid Computing
and commonly deployed services such as
email, file store, remote access and
instant messaging
www.jisc.ac.uk/assent
14. 2. Recent developments
Equipment sharing
› Brokered industry access to £60m
public investment in HPC
› Piloting the Kit-Catalogue software,
helping institutions to share details
of high value equipment
› Newcastle University alone is
sharing £16m+ of >£20K value
equipment via Kit-Catalogue
Photo credit: HPC Midlands
http://bit.ly/jiscsharing
15. 2. Recent developments
http://bit.ly/jiscsharing
Equipment sharing
› Working with EPSRC and University of
Southampton to operationalise
equipment.data as a national service
› 45 organisations sharing details of over
12,000 items of equipment
› Conservative estimate: £240m value
› Evidencing utilisation & sharing?
16. 2. Recent developments
Janet Reach:
› £4M funding from BIS to work
towards a Janet which is "open and
accessible" to industry
› Provides industry access to university
e-infrastructure facilities to facilitate
further investment in science,
engineering and technology with the
active participation of business and
industry
› Modelled on Innovate UK
competition process
bit.ly/janetreach
17. 2. Recent developments
Janet Reach:
› £4M funding from BIS to work
towards a Janet which is "open and
accessible" to industry
› Provides industry access to university
e-infrastructure facilities to facilitate
further investment in science,
engineering and technology with the
active participation of business and
industry
› Modelled on Innovate UK
competition process
bit.ly/jisc-hpc
18. 2. Recent developments
Research Data
Management Shared
Service
› Procurement under way
› Aiming to pilot for 24 months
starting this summer
› 13 pilot institutions
› Research Data Network
› Find out more:
researchdata.jiscinvolve.org
19. 2. Recent developments
Research Data
Discovery Service
› Alpha!
› Uses CKAN to aggregate
research data from institutions
› Test system has 16.7K datasets
from 14 organisations so far
› Search and browse:
ckan.data.alpha.jisc.ac.uk
23. 3. Personal perspectives
› David Fergusson
› Head of Scientific Computing
› Bruno Silva
› HPC Lead
› Francis Crick Institute
24. eMedLab:
Merging HPC and Cloud for
Biomedical Research
Dr Bruno Silva
eMedLab Service Operations Manager
HPC Lead - The Francis Crick Institute
bruno.silva@crick.ac.uk 01/12/2015
30. Winning bid
• 6048 cores (E5-2695v2)
• 252 IBM Flex servers, each with
• 24 cores
• 512GB RAM per compute server
• 240GB SSD (2x120GB RAID0)
• 2x10Gb Ethernet
• 3:1 Mellanox Ethernet fabric
• IBM GSS26 – Scratch 1.2PB
• IBM GSS24 – General Purpose (Bulk) 4.3PB
• Cloud OS – OpenStack
31.
32. Benchmark results
preliminary
• Aggregate HPL (one run per server – embarrassingly parallel)
• Peak 460Gflops*252 = 116Tflops
• Max – 94%
• Min – 84%
• VM ≈ Bare metal HPL runs (16 core)
33. Benchmark results
preliminary – bare metal only
• Storage throughput
Bulk File System (gpfsperf GB/s) Scratch File System (gpfsperf GB/s)
Create Read Write Create Read Write
Sequential Sequential Random Sequential Random Sequential Sequential Random Sequential Random
16M 16M 512K 16M 512K 16M 512K 16M 512K 16M 16M 512K 16M 512K 16M 512K 16M 512K
100 88 86 131 22 96 97 89 60 141 84 83 107 20 137 137 125 28
37. Projects
• Principal Investigator / Project lead
• Reports to eMedLab governance
• Controls who has access to project resources
• Project Systems Administrator
• Institutional resource and / or
• Specialised research team member(s)
• Works closely with eMedLab support
• Researchers
• Those who utilise the software and data available in eMedLab for the project
39. Federated Institutional support
Operations Team Support
(Support to facilitators and Systems Administrators)
Institutional Support
(direct support to research)
Tickets
Training
Documentation
elasticluster
44. Pilot Projects
• Chela James - Gene discovery, rapid genome sequencing,
somatic mutation analysis and high-definition phenotyping
VM
Image
Installing OS
CPU RAM Disk
“Flavours”
VM
Instanc
e
1
VM
Instanc
e
N
Network
Start/Stop/Hold/Checkpoint
Instance
Horizon Console
SSH - External IP
SSH – Tunnel
Web interface, etc…
45. Pilot Projects
• Peter Van Loo – Scalable, Collaborative, Cancer Genomics
Cluster
elasticluster
50. Challenges - Support
• High Barrier to entry
• Provide environments that resemble HPC or Desktop, or more intuitive interfaces
• Engender new thinking about workflows
• Promote Planning and Resource management
• Train support staff as well as researchers
• Resource-intensive support
• Promote community-based support and documentation
• Provide basic common tools and templates
• Upskill and mobilise local IT staff in departments
• Move IT support closer to the research project – Research Technologist
52. Challenges - Integration
• Suitability of POSIX Parallel file systems for Cloud Storage
• Working closely with IBM
• Copy-on-write feature of SS (GPFS) is quite useful for fast instance creation
• SS has actually quite a lot of the scaffolding required for a good object store
• Presentation SS or NAS to VMs requires additional AAAI layer
• Working closely with Red Hat and OCF to deliver IdM
• Presentation of SS to VMs introduces stability problems that could be worked-
around with additional SS licenses and some bespoke scripting
• Non-standard Network and Storage architecture
• Additional effort by vendors to ensure stable and performant infrastructure up-to-
date infrastructure – great efforts by everyone involved!
• Network re-design
54. Challenges - Performance
• File System Block Re-Mapping
• SS performs extremely well with 16MB blocks – we want to leverage this
• Hypervisor overhead (not all cores used for compute)
• Minimise number of cores “wasted” on cloud management
• On the other hand fewer cores means more memory bandwidth
• VM IO performance potentially affected by virtual network stack
• Leverage features available in the Mellanox NICs such as RoCE, SR-IOV, and
offload capabilities
55. Challenges – Performance
Block Re-Mapping
• SS (GPFS) is very good at handling many small files – by design
• VMs perform random IO reads and a few writes with their storage
• VM storage (and Cinder storage pools) are very large files on top of GPFS
• VM block size does not match SS (GPFS) block size
Bulk File System (gpfsperf GB/s) Scratch File System (gpfsperf GB/s)
Create Read Write Create Read Write
Sequential Sequential Random Sequential Random Sequential Sequential Random Sequential Random
16M 16M 512K 16M 512K 16M 512K 16M 512K 16M 16M 512K 16M 512K 16M 512K 16M 512K
100 88 86 131 22 96 97 89 60 141 84 83 107 20 137 137 125 28
56. Challenges – Performance
Block Re-Mapping
• Idea: turn random into sequential IO
• Have a GPFS standing
Bulk File System (gpfsperf GB/s) Scratch File System (gpfsperf GB/s)
Create Read Write Create Read Write
Sequential Sequential Random Sequential Random Sequential Sequential Random Sequential Random
16M 16M 512K 16M 512K 16M 512K 16M 512K 16M 16M 512K 16M 512K 16M 512K 16M 512K
100 88 86 131 22 96 97 89 60 141 84 83 107 20 137 137 125 28
58. Challenges - Presentation
• Access to eMedLab through VPN only
• Increases security
• Limits upload throughput
• Rigid, non-standard networking
• Immediately provides a secure environment with complete separation
• Projects only need to add VMs to the existing network
• Very inflexible, limits the possibility of a shared ecosystem of “public”
services
• Introduces great administration overheads when creating new projects –
space for improvement
62. Challenges - Security
• Presentation of SS shared storage to VMs raises security concerns
• VMs will have root access – even with squash, user can sidestep identity
• Re-export SS with a server-side authentication NAS protocol
• Alternatively, abstract shared storage with another service such as iRODS
• Ability of OpenStack users to maintain security of VMs
• Particularly problematic when deploying “from scratch” systems
• A competent, dedicated PSA mitigates this
64. Challenges - Allocation
• Politics and Economics of “unscheduled” cloud
• Resource allocation in rigid portions of infrastructure (large, medium, small)
• Onus of resource utilisation is with Project team
• A charging model may have to be introduced to promote good behaviour
• The infrastructure supplier does not care about efficiency, as long as cost is recovered
• Scheduling over unallocated portions of infrastructure may help maximise utilisation
• Benefits applications that function as Direct Acyclid Graphs (DAGs)
• Private cloud is finite and limited
• Once it is fully allocated, projects will be on a waiting list, rather than a queue
• Cloud bursting can “de-limit” the cloud, if funding permits it
• This would be a talk on its own.
66. Future Developments
• VM and Storage performance analysis
• Create optimal settings recommendations for Project Systems Administrators and Ops team
• Revisit Network configuration
• Provide a simpler, more standard OpenStack environment
• Simplify service delivery, account creation, other administrative tasks
• Research Data Management for Shared Data
• Could be a service within the VM services ecosystem
• IRODS is a possibility
• Explore potential of Scratch
• Integration with Assent (Moonshot tech)
• Access to infrastructure through remote credentials and local authorisation
• First step to securely sharing data across sites (Safe Share project)
67. Conclusions
• eMedLab is ground breaking in terms
• Institutional collaboration around a shared infrastructure
• Federated support model
• Large scale High Performance Computing Cloud (it can be done!)
• Enabling a large scale highly customisable workloads for Biomedical research
• Linux cluster still required (POSIX legacy applications)
• SS guarantees this flexibility at very high performance
• We can introduce Bare Metal (Ironic) if needed for a highly versatile platform
• Automated scheduling of granular workloads
• Can be done inside the Cloud
• True Parnership - OCF, Red Hat, IBM, Lenovo, and Mellanox
• Partnership working very well
• All vendors highly invested in eMedLab’s success
68. The Technical Design Group
• Mike Atkins – UCL (Project Manager)
• Andy Cafferkey – EBI
• Richard Christie – QMUL (Chair)
• Pete Clapham – Sanger
• David Fergusson – the Crick
• Thomas King – QMUL
• Richard Passey – UCL
• Bruno Silva – the Crick
69. Institutional Support Teams
UCL:
Facilitator: David Wong
PSA: Faruque Sarker
Crick:
Facilitator: David Fergusson/Bruno Silva
PSA: Adam Huffman, Luke Raimbach, John Bouquiere
LSHTM:
Facilitator: Jackie Stewart
PSA: Steve Whitbread, Kuba Purebski
70. Institutional Support Teams
Sanger:
Facilitator: Tim Cutts, Josh Randall
PSA: Peter Clapham, James Beal
EMBL-EBI:
Facilitator: Steven Newhouse/Andy Cafferkey
PSA: Gianni Dalla Torre
QMUL:
Tom King
71. Operations Team
Thomas Jones (UCL) Pete Clapham (Sanger)
William Hay (UCL) James Beale (Sanger)
Luke Sudbery (UCL)
Tom King (QMUL)
Bruno Silva (Ops Manager, Crick)
Adam Huffman (Crick) Andy Cafferkey (EMBL-EBI)
Luke Raimbach (Crick) Rich Boyce (EMBL-EBI)
Stefan Boeing (Data Manager, Crick) David Ocana (EMBL-EBI)
73. VM
Image
Installing OS
CPU RAM Disk
“Flavours”
VM
Instanc
e
1
VM
Instanc
e
N
Network
Start/Stop/Hold/Checkpoint
Instance
Horizon Console
SSH - External IP
SSH – Tunnel
Web interface, etc…
76. example research themes to be studied in the Academy Labs; by exploiting the commonalities underl
the datasets, we shall build tools and algorithms that cut across the spectrum of diseases.
Storage,)Compute,)Security,)Networking)
Access)to)Infrastructure)
Tools)&)analy<cs)
Genomic,)imaging,)clinical)datasets)
Cancer,)rare)and)cardiovascular)diseases)
GSK,)Saran)Cannon,)
DDN,)Intel,)IBM,)
Aridhia))
Farr)Ins<tute,)
Genomics)England,)
UCLH)BRC)
Informa<on)flow)
links)
ELIXIR,)ENCODE,)
1000)Genomes,)
Ensembl)
Proposed)funding)
External)funding)
Fig#1.
77. users to sh
within mul
will also
services a
developed
systems.
programme
infrastructu
resources
technologie
parts of th
others. Ea
guaranteed
Private(
Secure(
Collabora0ve(
Space(
Partner(
projects(
eMedLab(
Partner(
projects(
Partner(
projects(
EBI(
Partner(
projects(
FARR@UCLP(
Kings((
Health(
Partners(
Fig#3.!The!co
shared!data
resources!al
78. Winning bid
• Standard Compute cluster
• Ethernet network fabric
• Spectrum Scale storage
• Cloud OS
79. Initial requirements
• Hardware geared towards very high data throughput work – capable
for running an HPC cluster and a Cloud based on VMs
• Cloud OS (open source and commercial option)
• Tiered storage system for:
• High performance data processing
• Data Sharing
• Project storage
• VM storage
80. Bid responses – interesting facts
• Majority providing OpenStack as the Cloud OS
• Half included an HPC and a Cloud environment
• One provided a Vmware-based solution
• One provided a OpenStack-only solution
• Half tender responses offered Lustre
• One provided Ceph for VM storage
82. CHALLENGES HAVING A SERVER FARM IN THE CENTER OF
LONDON
A N D R EAS B I T ER NAS
F A CU LTY OF N A TU RAL A N D M A T HEMATICAL S C I ENCES
King’s College HPC
infrastructure in JISC
DC
83. • Cost of Space: Roughly £25k per square meter in Strand;
• Power:
• Expensive switches and UPS which require annual maintenance;
• Unreliable power supply due to high demand in center of London;
• Cooling:
• Expensive cooling system similar to one in Virtus DC;
• High cost for running and maintenance of the system;
• Weight: Due to the oldness of the building, there are strict weight restrictions
as an auditorium is below the server farm(!);
• Noise pollution: There is strong noise from the server farm up to 2 floors
below;
Problems and costs of having server farm in
Strand campus
84. King’s college infrastructure in Virtus DC
• Total 25 cabinets with ~200 racks in Data Hall 1:
• 16 cabinets HPC cluster ADA+Rosalind;
• Rest King’s Central IT infrastructure: fileservers, firewalls etc.;
• Rosalind, a consortium between Faculty of Natural and
Mathematical Sciences, South London and Maudsley NHS
Foundation Trust BRC( Biomedical Research Centre) and
Guy’s and St Thomas’ NHS Foundation Trust BRC;
• Rosalind has around 5000 cores, ~150 Teraflops,
HPC and Cloud part using OpenStack;
85. Features of Virtus Datacentre
• Power:
• Two Redundant central power connections;
• UPS & onsite power generator;
• Two redundant PSU in each rack ;
• Cooling:
• Chilled water system cooled via fresh air;
• Configures as hot and cold aisles;
• Services:
• Remote hands;
• Installation and maintenance;
• Office, storing spaces and wifi;
• Secure access control environment;
86. • Better internet connection;
• No “single” connections;
• Fully resilient network;
• The bandwidth requirements of
large data sets were being met;
Connectivity with Virtus Datacentre
87. • Due to the contract with JISC, tenants(Francis Crick Institute,
Queen Mary University, King’s College etc.) have special rates;
• Costs:
• Standard fee for each rack which includes costs of space, cooling,
connectivity etc.;
• Power consumed form each rack in normal market(education) prices;
Costs of Virtus Datacentre
88. 3. Personal perspectives
› Thomas King
› Head of Research Infrastructure
› Queen Mary University of London
90. Who are we?
20,000 students and 4,000 staff
5 campuses in London
3 faculties
Humanities & Social Sciences
Science & Engineering
Barts & the London School of Medicine and Dentistry
92. Old World IT
Small central provision
Lots of independent teams offering a lot of overlap in services and
bespoke solutions
21 machine rooms
93. IT Transformation Programme 2012-15
Centralisation of staff and services ~200 people
Consolidation into two data centres
On-site ~20 racks
Off-site facility within fibre channel latency distances
Highly virtualised environment
Enterprise services run in active-active
JISC Janet6 upgrades
94. Research IT
Services we support –
HPC
Research Storage
Hardware hosting
Clinical and secure systems
Enterprise virtualisation is not what we’re after
Five nines is not our issue – bang for buck
No room at the inn
Build our own on-site?
The OAP home
95. Benefits of shared data centre
Buying power and tenant’s association
Better PUE than smaller on-site DC
contribution to sustainability commitment
Transparent costing for power use
Network redundancy – L2 and L3 of JISC network
Collaboration – it’s all about the data
Cloudier projects
Emotional detachment from blinking LEDs
Direction of funding – GridPP, Environmental omics Cloud
96. That’s all folks…
Except where otherwise noted, this
work is licensed under CC-BY
Martin Hamilton
Futurist, Jisc, London
@martin_hamilton
martin.hamilton@jisc.ac.uk
HPC & Big Data 2016