An update on BeSTGRID activity and plans, in particular in preparation for the planned future developments of a unified approach to high performance and distributed computing in NZ.
ICT Centre of Excellence and Open Data –iCEODCIARD Movement
Prof Muliaro Wafula PhD. FCCS,FCSK (Director ICT Centre of Excellence and Open Data –iCEOD, JKUAT) at the Forum on Open Data and Open Science in Agriculture on 15th June 2015
Presentation on the work we've done within BeSTGRID as it relates to bioinformatics in NZ, for the 2010 Bioinformatics Symposium https://www.bestgrid.org/NZ-Bioinformatics-Symposium-2010
High-level Meeting & Workshop on Environmental and Scientific Open Data for Sustainable Development Goals in Developing Countries. Madagascar, 4-6 December 2017
An update on BeSTGRID activity and plans, in particular in preparation for the planned future developments of a unified approach to high performance and distributed computing in NZ.
ICT Centre of Excellence and Open Data –iCEODCIARD Movement
Prof Muliaro Wafula PhD. FCCS,FCSK (Director ICT Centre of Excellence and Open Data –iCEOD, JKUAT) at the Forum on Open Data and Open Science in Agriculture on 15th June 2015
Presentation on the work we've done within BeSTGRID as it relates to bioinformatics in NZ, for the 2010 Bioinformatics Symposium https://www.bestgrid.org/NZ-Bioinformatics-Symposium-2010
High-level Meeting & Workshop on Environmental and Scientific Open Data for Sustainable Development Goals in Developing Countries. Madagascar, 4-6 December 2017
Using geographic information systems for community and environmental responsibility. Public Participatory GIS provides a means for 'citizen scientists' to not only supply
information but to participate in the consultation process.
Geographic information technologies are used in participatory settings and to support information gathering.
The value of PPGIS:
* community inclusion and engagement are integrated
* conservation and development
* sustainable natural resource management
* discussion of customary property rights
The presentation shows examples of projects where information has been collected by community groups and how it made a difference.
SGCI Science Gateways: Harnessing Big Data and Open Data 03-19-2017Sandra Gesing
The importance of Big Data and Open Data to achieve scientific advancements in precision medicine is beyond doubt and evident in many different projects and initiatives such as the Precision Medicine Initiative (All of Us), ICTBioMed, NCIP Hub, 100K Genomics England Project, NIH Cancer Moonshot, and the Million Veterans Program. In April 2013, McKinsey & Company proclaimed that Big Data has the ability to revolutionize pharmaceutical research and development within clinical environments, by using data for better informed decision making and targeting the diverse user roles including physicians, consumers, insurers, and regulators. Companies from a wide spectrum such as Oracle Health Sciences, Google, and Data4Cure build solutions that help address efficient and secure data sharing with the patient or clinician in mind. Open data can be maintained and shared by patient communities such as PatientsLikeMe.com and build an invaluable resource for further data mining.
Even with all these advances there are still challenges to address including a recent Precision Medicine World Conference announcement in November 2016: “We are missing easy-to-use solutions to share patient data.” Science gateways are a solution to fill the gap and help form by definition end-to-end solutions – web-based, mobile or desktop applications - that provide intuitive access to advanced resources and allow researchers to focus on tackling today’s challenging science questions. Science Gateways abstract the complex underlying computing and data infrastructure as far as feasible and desired by the stakeholder and can be tailored to different target groups with diverse backgrounds, demands, and technical knowledge.
Science Gateways have existed for over a decade and a wide variety of frameworks and APIs have been developed to support the efficient creation of science gateways and ease the implementation of connections to Cloud infrastructures and distributed data on a large scale. The importance of science gateways has been recognized by NSF by funding the creation of a Science Gateways Community Institute (SGCI) to serve the community with free resources, services, experts, and ideas for creating and sustaining science gateways. To achieve this goal, the SGCI serves the community with five areas that have diverse foci and which also closely interact: Incubator, Extended Developer Support, Scientific Software Collaborative, Community Engagement and Exchange and Workforce Development.
The Institute is technology-agnostic and serves the community by offering a wide variety of services and using technologies that are the best fitting solution for the use case. Gateways allow for precision medicine to be more efficiently developed or adapted by lowering the barriers to data sharing and Big Data analysis.
Presentation of current challenges of upgrading the intrasturcture for access and preservation of social science research data and worklow in Slovene social science data archive
“Open Research Data: Implications for Science and Society”, Warsaw, Poland, May 28–29, 2015, conference organized by the Open Science Platform — an initiative of the Interdisciplinary Centre for Mathematical and Computational Modelling at the University of Warsaw. pon.edu.pl @OpenSciPlatform #ORD2015
Presentation during the 14th Association of African Universities (AAU) Conference and African Open Science Platform (AOSP)/Research Data Alliance (RDA) Workshop in Accra, Ghana, 7-8 June 2017.
Collaborating across institutional andjurisdictional boundar.docxpickersgillkayne
Collaborating across institutional and
jurisdictional boundaries: enabling the
emergence of a national innovation system
through public knowledge management
Richard Vines1,2
Michael Jones2 and
Gavan McCarthy2
1Department of Environment and Primary
Industries, Knoxfield, Victoria, Australia;
2eScholarship Research Centre, University of
Melbourne, Parkville, Victoria, Australia
Correspondence: Richard Vines, Department
of Environment and Primary Industries,
621 Burwood Highway, Knoxfield,
Victoria, VIC 3180.
E-mails: [email protected];
[email protected];
[email protected]
Abstract
Public institutions involved in research that aims to strengthen the productivity,
profitability and adaptiveness of industries face a multiplicity of challenges when
managing for the emergence of cost effective solutions to problems. We reflect
upon the learnings of a Government sponsored Visiting Fellow’s programme that
we describe as a knowledge management (KM) intervention within Australia’s
primary industries Research, Development and Extension (R, D and E) system. Our
central concern is to draw upon the learnings of an internet-based initiative in the
United States called eXtension to show how ‘traditional’ D and E activities can be
transformed. We argue that organisations and networks involved in such D and E
activities need to perceive themselves as belonging to systems that are socio-
technical in nature. That is, the development and deployment of cross-jurisdictional
and cross-institutional innovations are shaped by both the social interactions
between people and the systematic use of technology to support distributed
learning. We explain how the elements of an integrated model to support public
KM can be developed to create the conditions for enhanced innovation. Our
findings have relevance to a wide range of other industry sectors considering
contemporary service models involving public and private partnerships.
Knowledge Management Research & Practice (2015) 13(2), 187–197.
doi:10.1057/kmrp.2013.41; published online 19 August 2013
Keywords: agriculture; networks; knowledge management practice; explicit knowledge,
tacit knowledge; systems thinking
The online version of this article is available Open Access
A good sheep is a good sheep regardless of how you get there, but I don’t believe
in the figure world or picking a ram off a computer. … I’d rather put my trust in
looking at the sheep and seeing how it performs, than in some number dreamed
up by some scientists on a bit of paper. (Mr. Wal Merriman - Former President,
Australian Stud Merino breeders Association, cited in Neale, 2012)
Introduction
Organisations with responsibilities that mediate public and private interests
in Australian agriculture face a substantial knowledge challenge. Signifi-
cantly, a core element of this challenge is how to agree on, identify and
maintain ‘trusted knowledge’, including how knowledge is created, commu-
nicated and used to create and deploy innovations, solve.
A VIVO VIEW OF CANCER RESEARCH: Dream, Vision and RealityPaul Courtney
Presentation made by Paul Courtney (Dana-Farber Cancer Institute, Boston, MA and OHSL, MD) and Anil Srivastava (OHSL) at the 2013 VIVO conference in St. Louis, MO. Material contributed by Rubayi Srivastava (OHSL), Swati Mehta (Centre for Development of Advanced Computing, India), Juliusz Pukacki (Poznan Supercomputing and Network Center, Poland) and Devdatt Dubhashi (Chalmers Institute of Technology, Sweden).
Lecture for a course at NTNU, 27th January 2021
CC-BY 4.0 Dag Endresen https://orcid.org/0000-0002-2352-5497
See also http://bit.ly/biodiversityinformatics
https://www.gbif.no/events/2021/lecture-ntnu-gbif.html
ANDS health and medical data webinar 16 May. Storing and Publishing Health an...ARDC
Dr Jeff Christiansen (QCIF) introduced med.data.edu.au, a national facility to provide petabyte-scale research data storage, and related high-speed networked computational services, to Australian medical and health research organisations.
Webinar: https://www.youtube.com/watch?v=5jwBwDJrWAs
Jeff Christiansen Snippet: https://www.youtube.com/watch?v=PV_vuUKRm6w
Transcript: https://www.slideshare.net/AustralianNationalDataService/transcript-storing-and-publishing-health-and-medical-data-16052017
Pros and Cons of Open Data: A Global South PerspectiveMichelle Willmers
Presentation by ROER4D Curation & Dissemination Manager Michelle Willmers on open data practice in the Global South to the Committee of Plenipotentiary Representatives of the International Committee for Scientific and Technical Information (ICSTI).
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
More Related Content
Similar to Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management in New Zealand
Using geographic information systems for community and environmental responsibility. Public Participatory GIS provides a means for 'citizen scientists' to not only supply
information but to participate in the consultation process.
Geographic information technologies are used in participatory settings and to support information gathering.
The value of PPGIS:
* community inclusion and engagement are integrated
* conservation and development
* sustainable natural resource management
* discussion of customary property rights
The presentation shows examples of projects where information has been collected by community groups and how it made a difference.
SGCI Science Gateways: Harnessing Big Data and Open Data 03-19-2017Sandra Gesing
The importance of Big Data and Open Data to achieve scientific advancements in precision medicine is beyond doubt and evident in many different projects and initiatives such as the Precision Medicine Initiative (All of Us), ICTBioMed, NCIP Hub, 100K Genomics England Project, NIH Cancer Moonshot, and the Million Veterans Program. In April 2013, McKinsey & Company proclaimed that Big Data has the ability to revolutionize pharmaceutical research and development within clinical environments, by using data for better informed decision making and targeting the diverse user roles including physicians, consumers, insurers, and regulators. Companies from a wide spectrum such as Oracle Health Sciences, Google, and Data4Cure build solutions that help address efficient and secure data sharing with the patient or clinician in mind. Open data can be maintained and shared by patient communities such as PatientsLikeMe.com and build an invaluable resource for further data mining.
Even with all these advances there are still challenges to address including a recent Precision Medicine World Conference announcement in November 2016: “We are missing easy-to-use solutions to share patient data.” Science gateways are a solution to fill the gap and help form by definition end-to-end solutions – web-based, mobile or desktop applications - that provide intuitive access to advanced resources and allow researchers to focus on tackling today’s challenging science questions. Science Gateways abstract the complex underlying computing and data infrastructure as far as feasible and desired by the stakeholder and can be tailored to different target groups with diverse backgrounds, demands, and technical knowledge.
Science Gateways have existed for over a decade and a wide variety of frameworks and APIs have been developed to support the efficient creation of science gateways and ease the implementation of connections to Cloud infrastructures and distributed data on a large scale. The importance of science gateways has been recognized by NSF by funding the creation of a Science Gateways Community Institute (SGCI) to serve the community with free resources, services, experts, and ideas for creating and sustaining science gateways. To achieve this goal, the SGCI serves the community with five areas that have diverse foci and which also closely interact: Incubator, Extended Developer Support, Scientific Software Collaborative, Community Engagement and Exchange and Workforce Development.
The Institute is technology-agnostic and serves the community by offering a wide variety of services and using technologies that are the best fitting solution for the use case. Gateways allow for precision medicine to be more efficiently developed or adapted by lowering the barriers to data sharing and Big Data analysis.
Presentation of current challenges of upgrading the intrasturcture for access and preservation of social science research data and worklow in Slovene social science data archive
“Open Research Data: Implications for Science and Society”, Warsaw, Poland, May 28–29, 2015, conference organized by the Open Science Platform — an initiative of the Interdisciplinary Centre for Mathematical and Computational Modelling at the University of Warsaw. pon.edu.pl @OpenSciPlatform #ORD2015
Presentation during the 14th Association of African Universities (AAU) Conference and African Open Science Platform (AOSP)/Research Data Alliance (RDA) Workshop in Accra, Ghana, 7-8 June 2017.
Collaborating across institutional andjurisdictional boundar.docxpickersgillkayne
Collaborating across institutional and
jurisdictional boundaries: enabling the
emergence of a national innovation system
through public knowledge management
Richard Vines1,2
Michael Jones2 and
Gavan McCarthy2
1Department of Environment and Primary
Industries, Knoxfield, Victoria, Australia;
2eScholarship Research Centre, University of
Melbourne, Parkville, Victoria, Australia
Correspondence: Richard Vines, Department
of Environment and Primary Industries,
621 Burwood Highway, Knoxfield,
Victoria, VIC 3180.
E-mails: [email protected];
[email protected];
[email protected]
Abstract
Public institutions involved in research that aims to strengthen the productivity,
profitability and adaptiveness of industries face a multiplicity of challenges when
managing for the emergence of cost effective solutions to problems. We reflect
upon the learnings of a Government sponsored Visiting Fellow’s programme that
we describe as a knowledge management (KM) intervention within Australia’s
primary industries Research, Development and Extension (R, D and E) system. Our
central concern is to draw upon the learnings of an internet-based initiative in the
United States called eXtension to show how ‘traditional’ D and E activities can be
transformed. We argue that organisations and networks involved in such D and E
activities need to perceive themselves as belonging to systems that are socio-
technical in nature. That is, the development and deployment of cross-jurisdictional
and cross-institutional innovations are shaped by both the social interactions
between people and the systematic use of technology to support distributed
learning. We explain how the elements of an integrated model to support public
KM can be developed to create the conditions for enhanced innovation. Our
findings have relevance to a wide range of other industry sectors considering
contemporary service models involving public and private partnerships.
Knowledge Management Research & Practice (2015) 13(2), 187–197.
doi:10.1057/kmrp.2013.41; published online 19 August 2013
Keywords: agriculture; networks; knowledge management practice; explicit knowledge,
tacit knowledge; systems thinking
The online version of this article is available Open Access
A good sheep is a good sheep regardless of how you get there, but I don’t believe
in the figure world or picking a ram off a computer. … I’d rather put my trust in
looking at the sheep and seeing how it performs, than in some number dreamed
up by some scientists on a bit of paper. (Mr. Wal Merriman - Former President,
Australian Stud Merino breeders Association, cited in Neale, 2012)
Introduction
Organisations with responsibilities that mediate public and private interests
in Australian agriculture face a substantial knowledge challenge. Signifi-
cantly, a core element of this challenge is how to agree on, identify and
maintain ‘trusted knowledge’, including how knowledge is created, commu-
nicated and used to create and deploy innovations, solve.
A VIVO VIEW OF CANCER RESEARCH: Dream, Vision and RealityPaul Courtney
Presentation made by Paul Courtney (Dana-Farber Cancer Institute, Boston, MA and OHSL, MD) and Anil Srivastava (OHSL) at the 2013 VIVO conference in St. Louis, MO. Material contributed by Rubayi Srivastava (OHSL), Swati Mehta (Centre for Development of Advanced Computing, India), Juliusz Pukacki (Poznan Supercomputing and Network Center, Poland) and Devdatt Dubhashi (Chalmers Institute of Technology, Sweden).
Lecture for a course at NTNU, 27th January 2021
CC-BY 4.0 Dag Endresen https://orcid.org/0000-0002-2352-5497
See also http://bit.ly/biodiversityinformatics
https://www.gbif.no/events/2021/lecture-ntnu-gbif.html
ANDS health and medical data webinar 16 May. Storing and Publishing Health an...ARDC
Dr Jeff Christiansen (QCIF) introduced med.data.edu.au, a national facility to provide petabyte-scale research data storage, and related high-speed networked computational services, to Australian medical and health research organisations.
Webinar: https://www.youtube.com/watch?v=5jwBwDJrWAs
Jeff Christiansen Snippet: https://www.youtube.com/watch?v=PV_vuUKRm6w
Transcript: https://www.slideshare.net/AustralianNationalDataService/transcript-storing-and-publishing-health-and-medical-data-16052017
Pros and Cons of Open Data: A Global South PerspectiveMichelle Willmers
Presentation by ROER4D Curation & Dissemination Manager Michelle Willmers on open data practice in the Global South to the Committee of Plenipotentiary Representatives of the International Committee for Scientific and Technical Information (ICSTI).
Similar to Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management in New Zealand (20)
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
The Department of Energy's Integrated Research Infrastructure (IRI)Globus
We will provide an overview of DOE’s IRI initiative as it moves into early implementation, what drives the IRI vision, and the role of DOE in the larger national research ecosystem.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
Extending Globus into a Site-wide Automated Data Infrastructure.pdfGlobus
The Rosalind Franklin Institute hosts a variety of scientific instruments, which allow us to capture a multifaceted and multilevel view of biological systems, generating around 70 terabytes of data a month. Distributed solutions, such as Globus and Ceph, facilitates storage, access, and transfer of large amount of data. However, we still must deal with the heterogeneity of the file formats and directory structure at acquisition, which is optimised for fast recording, rather than for efficient storage and processing. Our data infrastructure includes local storage at the instruments and workstations, distributed object stores with POSIX and S3 access, remote storage on HPCs, and taped backup. This can pose a challenge in ensuring fast, secure, and efficient data transfer. Globus allows us to handle this heterogeneity, while its Python SDK allows us to automate our data infrastructure using Globus microservices integrated with our data access models. Our data management workflows are becoming increasingly complex and heterogenous, including desktop PCs, virtual machines, and offsite HPCs, as well as several open-source software tools with different computing and data structure requirements. This complexity commands that data is annotated with enough details about the experiments and the analysis to ensure efficient and reproducible workflows. This talk explores how we extend Globus into different parts of our data lifecycle to create a secure, scalable, and high performing automated data infrastructure that can provide FAIR[1,2] data for all our science.
1. https://doi.org/10.1038/sdata.2016.18
2. https://www.go-fair.org/fair-principles
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Globus Compute with Integrated Research Infrastructure (IRI) workflowsGlobus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and I will give a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Reactive Documents and Computational Pipelines - Bridging the GapGlobus
As scientific discovery and experimentation become increasingly reliant on computational methods, the static nature of traditional publications renders them progressively fragmented and unreproducible. How can workflow automation tools, such as Globus, be leveraged to address these issues and potentially create a new, higher-value form of publication? LivePublication leverages Globus’s custom Action Provider integrations and Compute nodes to capture semantic and provenance information during distributed flow executions. This information is then embedded within an RO-crate and interfaced with a programmatic document, creating a seamless pipeline from instruments, to computation, to publication.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Utilocate offers a comprehensive solution for locate ticket management by automating and streamlining the entire process. By integrating with Geospatial Information Systems (GIS), it provides accurate mapping and visualization of utility locations, enhancing decision-making and reducing the risk of errors. The system's advanced data analytics tools help identify trends, predict potential issues, and optimize resource allocation, making the locate ticket management process smarter and more efficient. Additionally, automated ticket management ensures consistency and reduces human error, while real-time notifications keep all relevant personnel informed and ready to respond promptly.
The system's ability to streamline workflows and automate ticket routing significantly reduces the time taken to process each ticket, making the process faster and more efficient. Mobile access allows field technicians to update ticket information on the go, ensuring that the latest information is always available and accelerating the locate process. Overall, Utilocate not only enhances the efficiency and accuracy of locate ticket management but also improves safety by minimizing the risk of utility damage through precise and timely locates.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management in New Zealand
1. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Moemoea nui Aotearoa:
Challenges and Strategies in Data Lifecycle Management in New
Zealand
Claire Rye, Product Manager Data Services, New Zealand eScience Infrastructure
2. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
2
Matariki in the night sky.
Photo courtesy of Fraser Gunn
www.astroimage.co.nz
Matariki
3. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Matariki touches us all
3
The stolen stars of Matariki
Published 2018, Scholastic New Zealand
4. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Matariki touches us all
4
Matariki in the night sky.
Photo courtesy Mark Russell
https://www.stuff.co.nz/travel/experiences/maori-culture/300361917/my-first-matariki-celebrating-mori-new-year-at-the-bay-of-islands-new-festival?rm=a
“I see this day as a chance to move the understanding of
mātauranga Māori beyond the idea that it’s mainly stories
about deities.
Our knowledge systems are still so often seen as “myths and
legends”, as if they’re devoid of proper science. But there is
empirical science that sits at the heart of mātauranga Māori.”
Professor Rangi Matamua
Quote from: How Matariki will connect us all
https://e-tangata.co.nz/comment-and-analysis/how-matariki-will-connect-us-all/
5. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
○
5
https://www.mbie.govt.nz/science-and-technology/science-and-innovation/research-and-data/te-ara-paerangi-future-pathways/
8. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
799 TB 72 million 5,597
Amount of data transferred Number of files transferred Number of transfers made
Globus benefits:
High-speed data transfer
● Move gigabits of data on a network 1,000 times faster than broadband
Internet. Powered by REANNZ (NZ’s national advanced network provider),
research data transfers can be done at 10Gbps.
Secure and easy data sharing
● Share your research data with collaborators locally, nationally, and around
the world. Control who has access using group management tools.
Research data delivery network
● Take advantage of our partnering institutions across NZ who have existing
data delivery network nodes. Transfer to and from any laptop or server
using a Globus connect endpoint.
National data transfer platform activities in 2021:
Live
Coming soon
Testing
9. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Our big numbers are small
CERN’s computer centre (Image: Maximilien Brice;
Claudia Marcelloni/CERN)
Globus global figures
10. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Aotearoa Genomic Data Repository
Data Repository for Taonga Species
Co-designing genomics projects with Māori and
ensuring Māori data sovereignty.
11. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
What does co-design mean?
Co-design is the act of creating with stakeholders, users, the community,
specifically within the design development process to ensure the results meet
their needs and are usable.
Genuine desire to take on board and incorporate the ideas and perspectives of
the stakeholders, users and community.
Why is it important?
● It’s the right thing to do
● Te Tiriti obligations
● Ensure culturally appropriate genomics studies
● Deliver benefits to Māori from genomics
● New ideas and perspectives bring innovation
● History of exploitation rather than collaboration
12. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
No repository for NZ genomic data of any kind.
● Researchers would host their own data or
would put it in overseas repositories e.g.
ENA, GenBank
○ No way to know what/where all the “NZ
genomic data” was
○ No metadata standard
○ Neither result allowed for any kind of
governance of the data
The situation:
Aotearoa
Genomic Data
Repository
13. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
What did co-design look like for us?
● Workshops
○ Indigenous governance, ethical frameworks
○ Metadata
○ End usability
● Weekly review with the working group
● Case studies
○ Learning about metadata requirements on different types of research
https://www.waikato.ac.nz/__data/assets/pdf_file/0018/321534/Te-Mata-Ira-Genome-Research-Guidelines.
https://www.gida-global.org/care
14. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Māori data sovereignty
Māori Data Sovereignty refers to the inherent rights and interests Māori,
whānau, hapū, iwi and Māori organisations have in relation to the creation,
collection, access, analysis, interpretation, management, dissemination, re-use,
and control of data relating to Māori, whānau, hapū, iwi and Māori organisations
as guaranteed in Article II of Te Tiriti/Treaty of Waitangi (Taiuru, K. 2020).
Taiuru, Karaitiana (2020. Māori Data Ethical Framework https://www.taiuru.maori.nz/tiritiethicalguide/Māori-data-ethical-framework/
15. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Whakapapa
Whakapapa is a key reference point for Māori when
talking about health and genetics. Whakapapa was
described as the connection between people and as
creating a responsibility for both future and past
generations. Whakapapa is often used as a framework
to describe genealogy, social and ecological
relationships, cultural histories, family traits and
ancestral inheritances. At a physical and spiritual level,
whakapapa is embodied within the DNA of a person;
therefore, the storage and use of human tissue for
research becomes a culturally significant activity.
Te Mata Ira—Faces of the
Gene
Developing a cultural
foundation for biobanking
and genomic research
involving Māori
M. Hudson et al.
16. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
○ AGDR
https://data.agdr.org.nz/
16
19. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Kaitiakitanga
Kaitiakitanga is a form of guardianship with
responsibility to look after specific resources. In the
context of genomic research it relates to the use of
samples, DNA and data (genome data, clinical data, and
information about whakapapa). It is important to clarify
who holds the role of kaitiaki within the research team
and the community, and that they are aware of the
responsibilities of that position to maintain ‘te hau o te
taonga’.
20. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Sharing genomic data via Globus
20
22. New Zealand eScience Infrastructure
New Zealand eScience Infrastructure
Aotearoa Genomic Data Repository
● More submissions
● Improvements based on feedback
● Finding Kaitaki for projects
without Māori engagement
Rakeiora Project
The Rakeiora Project a programme to improve genomic knowledge
that will impact on the wellbeing of New Zealanders is being
co-developed and co-governed with Māori. It is a “pathfinder” to
test options to acquire, protect, use and store genomic datasets for
use in healthcare research in Aotearoa New Zealand.
What’s next?
Dr Huti Watson’s slide, Rakeiora keynote,
eResearch conference 2022
23. New Zealand eScience Infrastructure
● Genomics Aotearoa data repository group
○ Mik Black - University of Otago
○ Ben Te Aika - GA Vision Mātauranga Coordinator
○ Rudi Brauning - AgResearch
○ Libby Liggins - Massey University
○ Miles Benton - ESR
○ Ben Curran - University of Auckland
● NeSI development team
○ Eiran Perkins
○ Jun Huh
○ Brian Flaherty
The AGDR Team