Title: Ansible, best practices.
Ansible has taken a prominent place in the configmanagement world. By now many people involved in DevOps have taken a look at it, or done a first project with it. Now it is time to step back and look at quality and craftmanship. Bas Meijer, Ansible ambassador, will talk about Ansible best practices, and will show tips, tricks and examples based on several projects.
About the speaker
Bas is a systems engineer and software developer and wasted decades on latenight hacking. He is currently helping out 2 enterprises with continuous delivery and devops.
Title: Ansible, best practices.
Ansible has taken a prominent place in the configmanagement world. By now many people involved in DevOps have taken a look at it, or done a first project with it. Now it is time to step back and look at quality and craftmanship. Bas Meijer, Ansible ambassador, will talk about Ansible best practices, and will show tips, tricks and examples based on several projects.
About the speaker
Bas is a systems engineer and software developer and wasted decades on latenight hacking. He is currently helping out 2 enterprises with continuous delivery and devops.
Application construction is great with Ansible, using it for docker helps fight complexity, improves maintainability. And playbooks are portable from docker to cloud.
Foreman Discovery is the Foreman project's solution to Metal as a Service. It can discover, register, and provision bare metal systems without knowing anything about them beforehand. Now, with the latest version of Discovery, you're able to boot hosts using the Discovery image directly instead of relying on DHCP and PXE, by mounting the ISO or burning to a CD-ROM or USB stick. In this talk, we'll do a quick introduction to Discovery, before going on to cover the new PXEless features.
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...SaltStack
Arnold gave this presentation at the Secure Linux Admin Conference in Berlin Germany. He provides an overview of what configuration management systems do, explains the fundamentals of SaltStack and provides a look inside.
Cobbler - Fast and reliable multi-OS provisioningRUDDER
In a lot of companies, machine deployment is a delicate subject: every administrator has his own recipe, using CD-ROMs, static binary images deployed via the network, peer delegation ...
However, one solution makes the consensus when it comes to automated mass deployments ( except in the Cloud ): PXE boot. The main cons are that the deployment and the management of such a service is a pain, and every OS has its own installation automation system.
This is where Cobbler saves the day: it enables a painless and reliably to create a PXE service, usable on either virtual or physical machines, while beeing the most agnostic possible towards the target OSes and its preconfiguration system (preseed, kickstart, sysprep, ...) while offering the possibility to handle lots of configuration parameters in a modular fashion (network, partitionning, user accounts, configuration management agent...)
This conference aims to introduce the audience to the general concepts of Cobbler, and some scenarios where it would be a useful solution.
Chef Provisioning a Chef Server Cluster - ChefConf 2015Chef
This talk is an exploration of how to build Chef Server 12 clusters with Chef Provisioning, including premium features like Chef Reporting and Chef Analytics. I will cover several new capabilities and tools in the Chef ecosystem that makes it easier than ever before to build and manage scalable Chef Server clusters.
https://youtu.be/HUQA1Ikm5Iw
SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStackSaltStack
This talk will demonstrate how to use Salt Mine leveraging Salt grains to create several environments (parallel universes) that decide how to run the same Salt formulas with different outcomes. "Roles” will be defined in an OpenStack Marconi (queuing as a service) deployment and a few formulas will be shared to demonstrate the concept.
SaltConf14 - Matthew Williams, Flowroute - Salt Virt for Linux contatiners an...SaltStack
This SaltConf14 talk by Matthew Williams of Flowroute shows the power of Salt Virt and Runner for creating and managing VMs and Linux containers. A demonstration of the Salt lxc module shows the simplicity with which containers and VMs can be created and configured.
Spot Trading - A case study in continuous delivery for mission critical finan...SaltStack
This is a presentation given by Jeremy Alons, Spot Trading, at the DevOps Summit Chicago in August 2014. Jeremy shares how Spot Trading does automated deployments for mission-critical financial services with a case study in continuous delivery.
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...SaltStack
Google is making the power of its datacenter, network, and technology innovations available to the world through its Cloud services. This presentation will provide an overview of the Google Cloud Platform and a deeper dive on Google Compute Engine. Google recently made an open source contribution to SaltStack and now you can now use Salt Cloud to manage your Compute Engine resources (IaaS virtual machine services). Come find out more about Google's Cloud Platform and how you can leverage Google scale with SaltStack.
Introduction to Stacki at Atlanta Meetup February 2016StackIQ
An introduction to Stacki-the fastest bare metal Linux server provisioning tool from the Stacki Atlanta kickoff meetup on 2/23/16 at the Microsoft Innovation Center. Greg Bruno is the VP Engineering at StackIQ.
Application construction is great with Ansible, using it for docker helps fight complexity, improves maintainability. And playbooks are portable from docker to cloud.
Foreman Discovery is the Foreman project's solution to Metal as a Service. It can discover, register, and provision bare metal systems without knowing anything about them beforehand. Now, with the latest version of Discovery, you're able to boot hosts using the Discovery image directly instead of relying on DHCP and PXE, by mounting the ISO or burning to a CD-ROM or USB stick. In this talk, we'll do a quick introduction to Discovery, before going on to cover the new PXEless features.
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...SaltStack
Arnold gave this presentation at the Secure Linux Admin Conference in Berlin Germany. He provides an overview of what configuration management systems do, explains the fundamentals of SaltStack and provides a look inside.
Cobbler - Fast and reliable multi-OS provisioningRUDDER
In a lot of companies, machine deployment is a delicate subject: every administrator has his own recipe, using CD-ROMs, static binary images deployed via the network, peer delegation ...
However, one solution makes the consensus when it comes to automated mass deployments ( except in the Cloud ): PXE boot. The main cons are that the deployment and the management of such a service is a pain, and every OS has its own installation automation system.
This is where Cobbler saves the day: it enables a painless and reliably to create a PXE service, usable on either virtual or physical machines, while beeing the most agnostic possible towards the target OSes and its preconfiguration system (preseed, kickstart, sysprep, ...) while offering the possibility to handle lots of configuration parameters in a modular fashion (network, partitionning, user accounts, configuration management agent...)
This conference aims to introduce the audience to the general concepts of Cobbler, and some scenarios where it would be a useful solution.
Chef Provisioning a Chef Server Cluster - ChefConf 2015Chef
This talk is an exploration of how to build Chef Server 12 clusters with Chef Provisioning, including premium features like Chef Reporting and Chef Analytics. I will cover several new capabilities and tools in the Chef ecosystem that makes it easier than ever before to build and manage scalable Chef Server clusters.
https://youtu.be/HUQA1Ikm5Iw
SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStackSaltStack
This talk will demonstrate how to use Salt Mine leveraging Salt grains to create several environments (parallel universes) that decide how to run the same Salt formulas with different outcomes. "Roles” will be defined in an OpenStack Marconi (queuing as a service) deployment and a few formulas will be shared to demonstrate the concept.
SaltConf14 - Matthew Williams, Flowroute - Salt Virt for Linux contatiners an...SaltStack
This SaltConf14 talk by Matthew Williams of Flowroute shows the power of Salt Virt and Runner for creating and managing VMs and Linux containers. A demonstration of the Salt lxc module shows the simplicity with which containers and VMs can be created and configured.
Spot Trading - A case study in continuous delivery for mission critical finan...SaltStack
This is a presentation given by Jeremy Alons, Spot Trading, at the DevOps Summit Chicago in August 2014. Jeremy shares how Spot Trading does automated deployments for mission-critical financial services with a case study in continuous delivery.
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...SaltStack
Google is making the power of its datacenter, network, and technology innovations available to the world through its Cloud services. This presentation will provide an overview of the Google Cloud Platform and a deeper dive on Google Compute Engine. Google recently made an open source contribution to SaltStack and now you can now use Salt Cloud to manage your Compute Engine resources (IaaS virtual machine services). Come find out more about Google's Cloud Platform and how you can leverage Google scale with SaltStack.
Introduction to Stacki at Atlanta Meetup February 2016StackIQ
An introduction to Stacki-the fastest bare metal Linux server provisioning tool from the Stacki Atlanta kickoff meetup on 2/23/16 at the Microsoft Innovation Center. Greg Bruno is the VP Engineering at StackIQ.
Joe Kaiser, System Engineer at StackIQ at the Seattle Scalability Meetup on April 27, 2016
This presentation was followed by a demo of Kubernetes on Stacki
Presentation by StackIQ's Director of Open Source Engineering, Joe Kaiser, at StackiFest 2017.
Step 1 of every Hadoop vendor’s documentation reads something like this: “First install a cluster.” Without a consistent group of installed machines, a Hadoop installation is prone to failure. Open source Stacki installs machines to a ping and a prompt enabling the consistency and configuration required for a functioning Hadoop installation.
StackIQ released a new open source Hortonworks bridge pallet to enable the installation of Hortonworks through the Ambari appliance at the beginning of 2017. In this presentation, Joe will show you how to set-up Stacki, the HDP Bridge pallet, Ambari, and then install Hadoop on a running cluster.
Deploying datacenters with Puppet - PuppetCamp Europe 2010Puppet
Rafael Brito at PuppetCamp Europe 2010
"Deploying datacenters with Puppet."
Follow along with the Video at: https://www.youtube.com/watch?v=3DaWrKQ82j4
Puppet Camp Europe 2010: Ghent, Belgium
May 27-28, 2010
Automating everything with PowerShell, Terraform, and AWSChris Brown
From the Melbourne PowerShell meetup in December 2016. At this event I presented a demonstration of how easy it is to build, test, and destroy infrastructure using combinations of technologies.
I will be giving a brief overview of the history of NGINX along with an overview of the features and functionality in the project as it stands today. I will give some real use case of example of how NGINX can be used to solve problems and eliminate complexity within infrastructure. I will then dive into the future of the modern web and how NGINX is monitoring and leveraging industry changes to enhance the product for individuals and companies in the industry.
openATTIC is the webUI for managing ceph storage in SUSE Linux and it is also accepted upstream as part of the default managing UI. Within openATTIC, it contains lots of different projects to provide all the functions including salt, Grafana and Prometheus.
This will not be a ceph or openATTIC focus talk, even we use it as the example of showing how everything work together. Instead we will look into openATTIC to see how each project work with each other. Mainly focus on Grafana and Prometheus, they are not only very useful for ceph and openATTIC but also equally powerful for monitoring your clusters, vm or cloud / container status.
Both Grafana and Prometheus are very easy to extend, which allow user or administrator to build the dashboard which fit their own need. Even the presentation example will be base on ceph / storage. Participants can use the same idea to monitor any system status they wanted after understand how they work.
Installing a Cluster of Raspberry Pis with Stacki AceStackIQ
To fully utilize the power of Raspberry Pis, StackIQ ported Stacki to support the inexpensive single-board computers, creating Stacki Ace: an open-source bare-metal installer for Raspberry Pis.
With the release of Stacki 4.0 comes more improvements in Stacki Ace. The avalanche installer has been added for parallel installation and 20MB images instead of 2GB images are now used to install the backend nodes increasing the installation speed immensely.
This is Greg Bruno's presentation from his webinar on how to create a cluster of Raspberry Pis with Stacki Ace.
Presentation by Teradata's Anthony Chen at StackiFest 2017.
Teradata is the global leader in large-scale Linux Data Warehouse and data analytics applications. Teradata’s challenges have always been how to quickly complete a massively parallel system installation from bare metal components.
For many years, Teradata has developed its own tools and methodology for the system installation but it has become obvious that we need a different approach to meet our customer’s demand and expectation. We have selected Stacki as Teradata’s tool to help us with bare metal provisioning. This presentation is to show the challenges Teradata faces today and how Teradata uses Stacki to address these challenges.
Installing a Cluster of Raspberry Pis with Stacki AceStackIQ
Presentation by StackIQ's VP Engineering/Co-Founder, Greg Bruno, at StackiFest 2017.
The Raspberry Pi was originally developed by the Raspberry Pi Foundation to promote the teaching of basic computer science in schools and developing countries. And although these little single-board computers have done just that, they have the ability to do so much more when paired with the correct tools. That’s why StackIQ ported Stacki (their original bare metal x86_64 server installer) to support Raspberry Pis, creating Stacki Ace: an open-source bare-metal installer for Raspberry Pis.
Automation of your OpenStack Infrastructure with StackiStackIQ
Presentation from StackiFest 2017 by Hugh Ma, Cloud Validation Developer at Flex Ciii.
One of the primary purposes of CloudLabs in Flex is to provide optimized rack level solutions while integrating third-party open source vendor products. With the fast growing adoption of cloud computing, many companies are looking to shift their workloads into the cloud. CloudLabs’ already established engineering and design services allow for an environment where validation and performance testing can be done in a controlled manner on a variety of cloud platforms, including OpenStack. This talk will look at how CloudLabs utilizes open source tools: a Linux provisioning software called Stacki, Ansible, and OpenStack to help the validation process of hardware and rack level solutions for cloud infrastructure.
Provisioning Heterogenous Bare Metal with StackiStackIQ
Justin Senseney of NIST's presentation from StackiFest 2017.
Stacki was used to upgrade a high-performance computing (HPC) cluster at the National Institute of Standards and Technology (NIST) in Gaithersburg, Maryland. NIST is the United States’ federal metrology institute, performing research and creating standards for measurements and technology, including materials, data, and cyber-security. A 1,200 node CentOS5 Maui/Torque cluster was upgraded to CentOS7 with a Slurm queuing system. At the same time, hundreds of servers were removed and added to this cluster. This presentation will show the application of Stacki to this HPC cluster and contrast previous methods used for provisioning. Stacki carts and pallets are used to provision role-based servers, including GPU, high-memory, and multiple login servers. Ideas are proposed to allow us to extend this application to managing multiple clusters. Any mention of commercial products within this presentation, including Stacki, is for information purposes; it does not imply recommendation or endorsement by NIST.
Stacki DC Meetup (11/30/16)
Presenter: Justin Senseney- Senior Computer Scientist, NIST
Description:Stacki was used to upgrade a high performance computing (HPC) cluster at the National Institute of Standards and Technology (NIST) in Gaithersburg, MD. A 1,200 node CentOS5 Maui/Torque cluster was upgraded to CentOS7 with a slurm queuing system. This presentation will show the application of Stacki to this HPC cluster and contrast previous methods used for provisioning. Stacki carts and pallets are used to provision role-based servers. Ideas are presented that would make it easier for multiple clusters to be managed. Any mention of commercial products, including Stacki, within this presentation is for information only; it does not imply recommendation or endorsement by NIST.
Public vs. Private Cloud Performance by FlexStackIQ
This is a presentation given by Hugh Ma and Michael O'Rourke from Flex at the Stacki San Jose Meetup on September 15, 2016. Learn about the differences between public and private cloud performance, their OpenStack-Ansible & FlexBench environment, and how they use Stacki.
Salesforce at Stacki Atlanta Meetup February 2016StackIQ
Dave Peterson's presentation on how Salesforce uses Stacki and Chef to provision and manage thousands of servers. Stacki Atlanta kickoff Meetup on 2/23/16 at the Microsoft Innovation Center. Dave is a Lead Systems Engineer at Salesforce.
The debate over whether a business should use a public or private
cloud will likely never end, but trends are showing that private
cloud-based solutions are gaining a lead.
There are some things in Stacki that you can only do with Remove commands. This tutorial takes you over the most common remove commands and offers an overview of how they work.
Download Stacki: www.stacki.com
Open source software has emerged as the driving force of technology innovation, from cloud and Big Data to social media and mobile.
Download Stacki, open source bare metal installer, at www.stacki.com
Stacki, our new open source bare metal installer, is really fast. So fast that it lets Sysadmins get to the more important tasks. Download it at www.stacki.com
Stacki: Automate with Spreadsheets (Tutorial)StackIQ
Stacki can use CSV-formatted spreadsheets to automate server configuration. These few slides give you a preview on how to do it. Stay tuned for the comprehensive tutorial coming soon. www.stacki.com
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Advanced Flow Concepts Every Developer Should KnowPeter Caitens
Tim Combridge from Sensible Giraffe and Salesforce Ben presents some important tips that all developers should know when dealing with Flows in Salesforce.
Your Digital Assistant.
Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data.
Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you.
Feasible Features
One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated
User Friendly – can be easily used on Android, iOS, and Web Interface
Multiple Accessibility – Log in through any device from any place at any time
One app for all industries – a Visitor Management System that works for any organisation.
Stress-free Sign-up
Visitor is registered and checked-in by the Receptionist
Host gets a notification, where they opt to Approve the meeting
Host notifies the Receptionist of the end of the meeting
Visitor is checked-out by the Receptionist
Host enters notes and remarks of the meeting
Customizable Components
Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings
Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors
VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information
Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments
Alerts & Notifications – Get notified on SMS, email, and application
Parking Management – Manage availability of parking space
Individual log-in – Every user has their own log-in id
Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system
Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization.
"Secure Your Premises with VizMan (VMS) – Get It Now"
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Why React Native as a Strategic Advantage for Startup Innovation.pdfayushiqss
Do you know that React Native is being increasingly adopted by startups as well as big companies in the mobile app development industry? Big names like Facebook, Instagram, and Pinterest have already integrated this robust open-source framework.
In fact, according to a report by Statista, the number of React Native developers has been steadily increasing over the years, reaching an estimated 1.9 million by the end of 2024. This means that the demand for this framework in the job market has been growing making it a valuable skill.
But what makes React Native so popular for mobile application development? It offers excellent cross-platform capabilities among other benefits. This way, with React Native, developers can write code once and run it on both iOS and Android devices thus saving time and resources leading to shorter development cycles hence faster time-to-market for your app.
Let’s take the example of a startup, which wanted to release their app on both iOS and Android at once. Through the use of React Native they managed to create an app and bring it into the market within a very short period. This helped them gain an advantage over their competitors because they had access to a large user base who were able to generate revenue quickly for them.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Modern design is crucial in today's digital environment, and this is especially true for SharePoint intranets. The design of these digital hubs is critical to user engagement and productivity enhancement. They are the cornerstone of internal collaboration and interaction within enterprises.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
6. Backend Installation
Save your Host Configuration spreadsheet as a CSV
Import CSV on frontend
◦ “stack load hostfile file=hosts.csv”
Tell backend nodes to install on their next PXE boot
◦ “stack set host boot backend action=install”
PXE boot all backend nodes
Done!
10. Custom Partitioning
We will substitute the “os” pallet with the full CentOS 6.7 distribution
◦ “stack list storage partition”
◦ “stack load storage partition file=partition.csv”
Prep the host for reinstall
◦ “stack list host partition”
◦ “stack remove host partition backend-0-0”
◦ “stack set host attr backend-0-0 attr=nukedisks value=true”
◦ “stack set host boot backend-0-0 action=install”
12. Adding CentOS to Default Box
We will substitute the “os” pallet with the full CentOS 6.7 distribution
◦ “stack list pallet”
◦ “stack add pallet CentOS*iso”
◦ “stack enable pallet CentOS”
◦ “stack disable pallet os”
13. Create a New Box with CentOS
Making a new box
◦ “stack list box”
◦ “stack add box centos”
◦ “stack enable pallet CentOS box=centos”
◦ “stack enable pallet stacki box=centos”
Assign a host to a new box
◦ “stack list host”
◦ “stack set host box backend-0-0 box=centos”
17. The “Step 0” Problem
Check namenodes are
empty
Format/start HDFS
Create all directories
Create all metastores
Start services (Hbase, Hive,
Oozie, Sqoop, Impala, etc)
Deploy client configuration Configure database
Setup/assign monitors
(activity, services, and host)
Test database connections
Validate/resolve hostnamesConsistent host timezones
No bad kernel versions
running
(CDH) version consistency
Java version consistency
Daemons versions
consistency
Mgmt Agents versions
consistency
Host specification/SSH
ports
MUCH MORE …
DHCP Server/Client setup TFTP/PXE configuration
Server OS installation
Node OS Install
RAID configuration
Boot configuration
System/data disk
partitioning
Monitoring system setup
and config
Lights Out/IPMI setup
User accounts added and
synced
SSH keys on all hosts
Network node configuration
Config Mgmt install and
configuration
Route configurationOS upgrades/updates
Site specific software and
configuration
Host specification/SSH
ports
Security
Firewall setupCluster Mgmt utility Database install and config
Multiple network configPackage installation MUCH MORE …
18. Clusters are Different
Adding new servers does require coordination
Newly added servers must:
• Have same software stack as original
servers
• Have same configuration as original
servers
• Know about original servers
And, original servers must:
• Know about new servers
Result: The management complexity added to the
Operations staff is “exponential”
20. The Pain Curve
Number of Servers
ManagementComplexity
General Data Center
Clusters
PAIN
21. The Pain Threshold
The pain threshold differs for every
organization
Function of:
• cluster(s) size
• number of people in Operations
• Operations staff cluster expertise
25. Time is Money
The clock starts ticking when hosts land on your
loading dock
Without your applications online, you have an
paper weight that consumes power, cooling, and
management’s attention