This document provides an overview of DevOps, including:
1) It discusses Garry, a product owner whose team had slow and manual software delivery processes until discovering DevOps.
2) DevOps is defined as a culture change involving shared responsibility between development and operations teams to deliver software rapidly through practices like continuous integration, delivery, infrastructure as code, and monitoring.
3) The benefits of DevOps are outlined as faster and more frequent software delivery, improved quality, security, and compliance through automation and visibility across the software lifecycle.
4) Examples of DevOps in action include deploying a web application directly from a development machine versus through an automated build and test process on a separate build box.
Understanding TDD - theory, practice, techniques and tips.Malinda Kapuruge
My talk at the Swinburne University on 09/10/2019. Presented to students as part of the series, Development Projects - Tools and Practices.
Level: Beginner
Software-as-a-Service has become a very popular software delivery method due to its inherent advantages to both the service provider and the consumer. Startups are emerging businesses that usually provide innovative products to win a market share. In the recent past there are many Information Technology startups adopt SaaS as a way to quickly deliver their products to customers.
This talk is discusses the software engineering challenges in a SaaS startup environment, so that software practitioners those who do not have experience in such an environment can foresee what to be expected.
Javascript Frameworks (and How to Learn Them)All Things Open
Presented at: All Things Open RTP Meetup
Presented by: Peter Elbaum, Praxent
Abstract:
There are countless blog posts and tweets given to the topic of whether to choose Vue, React or Angular. We spend a lot of time debating the difference between the frameworks, but we often overlook the reasons that front-end frameworks were created in the first place. This talk will address how front-end development was done before frameworks existed and discuss the main problem that frameworks solve. Through grasping the reason for frameworks, attendees of this talk will be able to accelerate the process of learning a new front-end framework. Specifically, this talk with address component-based architecture, application state management, and component interaction. We'll cover what to look for when learning a new framework and strategies for internalizing the nuances of the various framework ecosystems. Seeing the commonalities among these frameworks allows for grasping the bigger picture.
Improving WordPress Development and Deployments with DockerBrett Palmer
This presentation will discuss how we use Docker to improve our development and deployment of WordPress sites. The presentation describes how themes and plugins can be developed locally and then packaged into a Docker container. A Jenkins pipeline is used with Git to automatically run a build and deploy the new WordPress container onto a test server where automated Selenium scripts are executed. Based on the results of the tests the container is tagged as a production candidate. A deployment pipeline is then used to execute a Blue-Green deployment on the latest update to production. Once the deployment is proven to work through automated testing the router is switched to the new deployment. The overall process helps reduce the risk of production updates to WordPress, themes, and plugins as well as reduce security concerns caused by randomly installing plugins by users. The presentation will also discuss strategies for running a WordPress database both outside and inside of a Docker container. It will discuss the pros and cons of each approach.
Presentation given at Salt Lake City WordCamp 2019, Oct 12, 2019. (https://2019.slc.wordcamp.org/).
Understanding TDD - theory, practice, techniques and tips.Malinda Kapuruge
My talk at the Swinburne University on 09/10/2019. Presented to students as part of the series, Development Projects - Tools and Practices.
Level: Beginner
Software-as-a-Service has become a very popular software delivery method due to its inherent advantages to both the service provider and the consumer. Startups are emerging businesses that usually provide innovative products to win a market share. In the recent past there are many Information Technology startups adopt SaaS as a way to quickly deliver their products to customers.
This talk is discusses the software engineering challenges in a SaaS startup environment, so that software practitioners those who do not have experience in such an environment can foresee what to be expected.
Javascript Frameworks (and How to Learn Them)All Things Open
Presented at: All Things Open RTP Meetup
Presented by: Peter Elbaum, Praxent
Abstract:
There are countless blog posts and tweets given to the topic of whether to choose Vue, React or Angular. We spend a lot of time debating the difference between the frameworks, but we often overlook the reasons that front-end frameworks were created in the first place. This talk will address how front-end development was done before frameworks existed and discuss the main problem that frameworks solve. Through grasping the reason for frameworks, attendees of this talk will be able to accelerate the process of learning a new front-end framework. Specifically, this talk with address component-based architecture, application state management, and component interaction. We'll cover what to look for when learning a new framework and strategies for internalizing the nuances of the various framework ecosystems. Seeing the commonalities among these frameworks allows for grasping the bigger picture.
Improving WordPress Development and Deployments with DockerBrett Palmer
This presentation will discuss how we use Docker to improve our development and deployment of WordPress sites. The presentation describes how themes and plugins can be developed locally and then packaged into a Docker container. A Jenkins pipeline is used with Git to automatically run a build and deploy the new WordPress container onto a test server where automated Selenium scripts are executed. Based on the results of the tests the container is tagged as a production candidate. A deployment pipeline is then used to execute a Blue-Green deployment on the latest update to production. Once the deployment is proven to work through automated testing the router is switched to the new deployment. The overall process helps reduce the risk of production updates to WordPress, themes, and plugins as well as reduce security concerns caused by randomly installing plugins by users. The presentation will also discuss strategies for running a WordPress database both outside and inside of a Docker container. It will discuss the pros and cons of each approach.
Presentation given at Salt Lake City WordCamp 2019, Oct 12, 2019. (https://2019.slc.wordcamp.org/).
Dockerized containers are the current wave that promising to revolutionize IT. Everybody is talking about containers, but a lot of people remain confused on how they work and why they are different or better than virtual machines. In this session, Black Duck container and virtualization expert Tim Mackey will demystify containers, explain their core concepts, and compare and contrast them with the virtual machine architectures that have been the staple of IT for the last decade.
Microservices Practitioner Summit Jan '15 - Maximizing Developer Productivity...Ambassador Labs
Tom Petr, Hubspot
HubSpot is an all-in-one sales and marketing platform made up of over 350 RESTful APIs deployed hundreds of times a day. This session will discuss the decisions and tradeoffs we've made in our quest to maximize developer productivity in a fast moving environment.
Full video here: http://www.microservices.com/tom-petr-maximizing-developer-productivity-microservices-environment
Can you get away with that answer after crashing your production website with a change you just deployed? Usually you can’t, and instead you’re tasked with figuring out and fixing the problem. In this session, we will talk about typical architectural, coding and deployment problems you might recognize, show what data you need to quickly identify them, and how to catch them before impacting the business.
Everyone heard about Kubernetes. Everyone wants to use this tool. However, sometimes we forget about security, which is essential throughout the container lifecycle.
Therefore, our journey with Kubernetes security should begin in the build stage when writing the code becomes the container image.
Kubernetes provides innate security advantages, and together with solid container protection, it will be invincible.
During the sessions, we will review all those features and highlight which are mandatory to use. We will discuss the main vulnerabilities which may cause compromising your system.
Contacts:
LinkedIn - https://www.linkedin.com/in/vshynkar/
GitHub - https://github.com/sqerison
-------------------------------------------------------------------------------------
Materials from the video:
The policies and docker files examples:
https://gist.github.com/sqerison/43365e30ee62298d9757deeab7643a90
The repo with the helm chart used in a demo:
https://github.com/sqerison/argo-rollouts-demo
Tools that showed in the last section:
https://github.com/armosec/kubescape
https://github.com/aquasecurity/kube-bench
https://github.com/controlplaneio/kubectl-kubesec
https://github.com/Shopify/kubeaudit#installation
https://github.com/eldadru/ksniff
Further learning.
A book released by CISA (Cybersecurity and Infrastructure Security Agency):
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
O`REILLY Kubernetes Security:
https://kubernetes-security.info/
O`REILLY Container Security:
https://info.aquasec.com/container-security-book
Thanks for watching!
This workshop was given at Crikeycon 2019 in Brisbane. It introduces Velociraptor and explains some of the design goals and implementation.
Note - this slide deck is outdated but might still be useful. The tool has evolved significantly since Crikeycon.
Idi2018 - Serverless does not mean OpslessLinuxaria.com
Presentaion done at Devops Day Bologna 2018.
We talk about DevOps as Dev + Ops, and the evolution of this movement, mainly on the ops point of view.
We’ll arrive to today new paradigm “NoOps”, to try to answer a question: “Is this the end of the operations team ?”
This presentation showed how SQL Server in Docker containers can facilitate a DevOps CI/CD pipeline starting from production and moving data and binaries seamlessly through dev - test - pre-prod - production.
Introduction to deployment with Ruby on Rails presented at JAX09 in Mainz by Jonathan Weiss.
Learn about the deployment architectures and setups (web and app tier) and using Capistrano/Webistrano.
Cassandra Summit 2014: Deploying Cassandra for Call of DutyDataStax Academy
Presenters: Seán O Sullivan, Service Reliability Engineer & Tim Czerniak, Software Engineer at Demonware
This presentation covers the eight-month evaluation process we underwent to migrate some of Call of Duty’s core services from MySQL to Cassandra. We will outline our requirements, the process we followed for the evaluation, decisions we made around our schema, configuration and hardware, and some issues we encountered.
Dockerized containers are the current wave that promising to revolutionize IT. Everybody is talking about containers, but a lot of people remain confused on how they work and why they are different or better than virtual machines. In this session, Black Duck container and virtualization expert Tim Mackey will demystify containers, explain their core concepts, and compare and contrast them with the virtual machine architectures that have been the staple of IT for the last decade.
Microservices Practitioner Summit Jan '15 - Maximizing Developer Productivity...Ambassador Labs
Tom Petr, Hubspot
HubSpot is an all-in-one sales and marketing platform made up of over 350 RESTful APIs deployed hundreds of times a day. This session will discuss the decisions and tradeoffs we've made in our quest to maximize developer productivity in a fast moving environment.
Full video here: http://www.microservices.com/tom-petr-maximizing-developer-productivity-microservices-environment
Can you get away with that answer after crashing your production website with a change you just deployed? Usually you can’t, and instead you’re tasked with figuring out and fixing the problem. In this session, we will talk about typical architectural, coding and deployment problems you might recognize, show what data you need to quickly identify them, and how to catch them before impacting the business.
Everyone heard about Kubernetes. Everyone wants to use this tool. However, sometimes we forget about security, which is essential throughout the container lifecycle.
Therefore, our journey with Kubernetes security should begin in the build stage when writing the code becomes the container image.
Kubernetes provides innate security advantages, and together with solid container protection, it will be invincible.
During the sessions, we will review all those features and highlight which are mandatory to use. We will discuss the main vulnerabilities which may cause compromising your system.
Contacts:
LinkedIn - https://www.linkedin.com/in/vshynkar/
GitHub - https://github.com/sqerison
-------------------------------------------------------------------------------------
Materials from the video:
The policies and docker files examples:
https://gist.github.com/sqerison/43365e30ee62298d9757deeab7643a90
The repo with the helm chart used in a demo:
https://github.com/sqerison/argo-rollouts-demo
Tools that showed in the last section:
https://github.com/armosec/kubescape
https://github.com/aquasecurity/kube-bench
https://github.com/controlplaneio/kubectl-kubesec
https://github.com/Shopify/kubeaudit#installation
https://github.com/eldadru/ksniff
Further learning.
A book released by CISA (Cybersecurity and Infrastructure Security Agency):
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
O`REILLY Kubernetes Security:
https://kubernetes-security.info/
O`REILLY Container Security:
https://info.aquasec.com/container-security-book
Thanks for watching!
This workshop was given at Crikeycon 2019 in Brisbane. It introduces Velociraptor and explains some of the design goals and implementation.
Note - this slide deck is outdated but might still be useful. The tool has evolved significantly since Crikeycon.
Idi2018 - Serverless does not mean OpslessLinuxaria.com
Presentaion done at Devops Day Bologna 2018.
We talk about DevOps as Dev + Ops, and the evolution of this movement, mainly on the ops point of view.
We’ll arrive to today new paradigm “NoOps”, to try to answer a question: “Is this the end of the operations team ?”
This presentation showed how SQL Server in Docker containers can facilitate a DevOps CI/CD pipeline starting from production and moving data and binaries seamlessly through dev - test - pre-prod - production.
Introduction to deployment with Ruby on Rails presented at JAX09 in Mainz by Jonathan Weiss.
Learn about the deployment architectures and setups (web and app tier) and using Capistrano/Webistrano.
Cassandra Summit 2014: Deploying Cassandra for Call of DutyDataStax Academy
Presenters: Seán O Sullivan, Service Reliability Engineer & Tim Czerniak, Software Engineer at Demonware
This presentation covers the eight-month evaluation process we underwent to migrate some of Call of Duty’s core services from MySQL to Cassandra. We will outline our requirements, the process we followed for the evaluation, decisions we made around our schema, configuration and hardware, and some issues we encountered.
De facto DevOps, de facto Agile. Today DevOps is the Manufacturing Revolution of Our Age. There is no escape for us. When got a DevOps, you got a DevOps.
DevOps simply is the combination of cultural philosophies,practices,and tools that increase an organization’s ability to deliver applications and services at high velocity : evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes.
This session is designed to teach security engineers, developers, solutions architects, and other technical security practitioners how to use a DevSecOps approach to design and build robust security controls at cloud-scale. This session walks through the design considerations of operating high-assurance workloads on top of the AWS platform and provides examples of how to automate configuration management and generate audit evidence for your own workloads. We’ll discuss practical examples using real code for automating security tasks, then dive deeper to map the configurations against various industry frameworks. This advanced session showcases how continuous integration and deployment pipelines can accelerate the speed of security teams and improve collaboration with software development teams.
Make It Cooler: Using Decentralized Version Controlindiver
A commonly used version control system in the ColdFusion community is Subversion -- a centralized system that relies on being connected to a central server. The next generation version control systems are “decentralized”, in that version control tasks do not rely on a central server.
Decentralized version control systems are more efficient and offer a more practical way of software development.
In this session, Indy takes you through the considerations in moving from Subversion to Git, a decentralized version control system. You also get to understand the pros and cons of each and hear of the practical experience of migrating projects to decentralized version control.
Version control is often used in conjunction with a testing framework and continuous integration. To complete the picture, Indy walks you through how to integrate Git with a testing framework, MXUnit, and a continuous integration server, Hudson.
Your developers just walked into your cube and said: "Here's the new app, I built it with Docker, and it's ready to go live." What do you do next? In this session, we'll talk about what containers are and what they are not. And we'll step through a series of considerations that need to be examined when deploying containerized workloads - VMs or Container? Bare Metal or Cloud? What about capacity planning? Security? Disaster Recovery? How do I even get started?
This webinar discusses the gaps that prevent enterprises from fully automating the DevOps lifecycle and how technologies like Containers and Sandboxes can assist with crossing that chasm.
This webinar discusses the gaps that prevent enterprises from fully automating the DevOps lifecycle and how technologies like Containers and Sandboxes can assist with crossing that chasm.
Microservices at Scale: How to Reduce Overhead and Increase Developer Product...DevOps.com
As a cloud native application grows in size—more microservices, more dependencies, more teams—there’s a corresponding increase in…
Complexity: Over time, the application becomes a lot harder for a single developer to reason about and contribute to. Staying on top of READMEs and managing cross-team communication is practically a full-time job.
Scaling challenges: The reality of building, deploying, and testing a 100+ service distributed application means developers are going to spend a lot of time sitting around waiting.
But it doesn’t have to end up this way, and there are concrete steps that DevOps engineers can take to keep their developers moving quickly even as an application grows. In this webinar, we’ll show you how to use open source products to:
Make it easy for your developers to code and run on-demand tests against a production-like environment—without having to constantly deal with the complexity that comes with a large application
Codify the relationship between all your services and tests, making your system self-documented and easy to understand
Keep your integration tests running fast so that devs can more easily write and debug their tests and get the quick feedback loops they need
Facilitate remote, in-cluster development and give every developer their own isolated namespace—and never again ask a developer to deploy the application on their laptop
Moving faster with CI/CD: Best DevOps practices and lessons learntMalinda Kapuruge
Practicing CI/CD is an essential part of DevOps culture. In this talk we discuss what is CI/CD and best practices to follow in modern application development environments.
The audience is Undergrads at Swinburne University in Melbourne studying Software Engineering.
Startups face many challenges due to their limited budget, frequent changes, high competition and lack of visibility. A software practitioner can provide the required technical guidance for such startups to help them facing these challenges. However, software practitioners need to have a breadth of knowledge to find smart solutions within the limitations. The students who are expecting to join a startup as engineers or even create startups as entrepreneurs will benefit from this talk
Test Driven Development - a Practitioner’s PerspectiveMalinda Kapuruge
Guest lecture at Swinburne University of Technology, Melbourne. We introduced TDD concepts to students. We also did a live interactive demo with students to understand benefits of TDD.
Finally, we discussed benefits as well as pitfalls from a practitioner's point of view.
EPClets - A Lightweight and Flexible Textual Language to Augment EPC Process ...Malinda Kapuruge
Event-driven Process Chains (EPC) has been widely adopted as a business process modeling notation. The common practice is to use a software tool to build a directed graph consisting of EPC constructs to model a business process. If a change is required, the constructed graph is partially dismantled and reconstructed. While the graph-based representation is beneficial in providing visualization, we investigate in this paper the advantages of having a textual representation alongside the graph representation. We introduce a textual language, called EPClets, to represent an EPC graph as a set of declarative event-action rules. The EPC graph can be constructed incrementally and automatically from the textual representation, separating the business- process specification and graph (re)construction concerns. The advantages of our approach have been evaluated through a controlled experiment. The experimental results suggest that having a textual representation alongside the graph representation increases the efficiency compared to an entirely graph-based approach.
Enabling Ad-hoc Business Process Adaptations through Event-driven Task Decoup...Malinda Kapuruge
The ability to adapt running process instances is a key requirement to handle exceptions in service orchestrations. The design of the orchestration middleware and its underlying meta-model plays an important role to fulfill this requirement. However, current service orchestration middleware such as BPEL engines suffer from their imperative and tightly coupled task execution mechanisms making it difficult to adapt running process instances. In this paper we present a novel service orchestration middleware and its underlying meta-model to overcome this limitation. To achieve this, we combine the benefits of the models@runtime concept with the event-driven publish-subscribe mechanism. We evaluate our approach for its support to process instance adaptation and compare its performance to an existing orchestration runtime.
http://link.springer.com/chapter/10.1007%2F978-3-642-38709-8_25
Supporting Adaptation Patterns in the Event-driven Business Process Modell...Malinda Kapuruge
The event-driven business process modeling has been emerged as an alternative paradigm to the traditional workflow-based business process-modeling paradigm. One of the influencing factors for using this new paradigm is the flexibility and the agility it provides in supporting business process change. Weber et al. have proposed 14 high-level adaptation patterns that need to be supported by a flexible process-aware information system irrespective of the chosen paradigm. In this presentation we present a paper that investigate whether the event-driven paradigm can satisfactorily support these adaptation patterns; if so, how an event-driven approach should support these adaptation patterns. We also point to the future research directions that this work can lead to.
Paper available at: http://link.springer.com/chapter/10.1007/978-3-642-41154-0_22
Representing Service-Relationships as First Class Entities in Service Orchest...Malinda Kapuruge
This talk presents a novel approach to design and enact service orchestrations that explicitly capture the service relationships in service orchestrations. Presented at WISE - 2012 conference, Paphos, Cyprus.
Achieving Multi-tenanted Business Processes in SaaS Applications Malinda Kapuruge
With the emergence of Cloud Computing and maturity of Service Oriented Architecture (SOA), the Software-as-a-Service (SaaS) delivery model has gained popularity, due to advantages such as lower startup cost and reduced time to market. A SaaS vendor owns and takes the responsibility of maintaining a single application for multiple clients/tenants who may have similar but also varying requirements. Business process modeling (BPM) approaches can be used to package service offerings to meet these varying requirements on a shared basis. However the customizations in those business processes can be challenging. In this paper we discuss the challenges arising from single-instance multi-tenancy, and present our approach to defining customizable business processes in SaaS applications to address those challenges.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
3. Garry got a problem
• Software delivery takes too long
• Not frequent enough
• Poor biz responsiveness
!
We will deploy
a new version
to Prod
(hopefully !!!)
24. Dev Ops
How can I write better programs,
so that my application can be
easily and quickly deploy/scale ?
How can I provide better ops,
so that the programs can be
built, tested and deployed efficiently?
25. DevOps
How we get things
DONE! DONE !!
Thinking end-to-end …
Visibility end-to-end …
26. Don’t be mistaken.
You still need to do the same work, i.e.,
coding, testing, deploying and monitoring.
But now you doing it in a more integrated and
transparent manner.
27. Devops
• Rapid IT delivery
• Adopt agile and lean practices
• To improve collaboration between operations and development
• Programmable infrastructure
• Automate everything
• Feedback -> optimize
• Identify bottlenecks
• Usage patterns
38. What happened there?
• In Demo 1, we deployed a nodejs application to production server
directly from the dev machine.
• Very easy, but very risky.
• In Demo 2, a build box is used to automate the build and test our
change.
• Demo 2 setup shows more maturity compared demo 1
• The code build and deploy concerns are decoupled
• How can we further improve Demo 2 ?
43. Continuous Integration
• Build and test every time there is a code change
• Code commit / merge -> automated build and testing
• Clean master (or develop) branch
• Version control strategies – Git
• Small increments
• Long isolated development -> merge conflicts
Coding in
a branch
Create pull
request
Code
review
Merge
44. Automated Testing
• Build pipeline performs testing, e.g., before every merge
• Quality gates – SonarQube
• Confidence to deliver
• Test pyramid
• Unit tests (test a unit of code in isolation)
• Integration tests (modules tested as a group)
• Regression tests (whenever something is changed)
• Contract tests
• tests the interactions between modules
Acceptance tests
Integration tests
Unit tests
45. Automated Testing
• Test pyramid – a lot of unit tests
• Test coverage reports
• Microservices –> many interactions to test
• Reduce the amount of integration tests
• Slow. Hard to detect issues
• Contract testing as Unit tests (fast)
• Pact Framework
ms1
ms2
ms3
ms4
46. Continuous Delivery
• Software can be released to production at anytime
• A set of design principles
• Any version to any environment
• A deployment pipeline
• Speed and Quality
• Continuous deployment requires continuous delivery
47. Continuous Delivery - example
Code
change
Branch
build and
test
Merge
and test
Build
artifact Test 1
Prod
Staging 1
Staging 1
Test 1
v3
Artifact repo
v1 v2
v2
v2
v2
48. Infrastructure as code
• Versioned
• Repeatable
• Standardized
• Automating tests and configuration free up valuable time
• Deploy the complete system, any day, any time
• Self-recovery
• Idempotence
• Immutability
• Cattle vs Pets
• Reveal possibilities to optimize
53. KPI Description
Change lead time From coding to deploy in production
Change volume Features pushed to production per deploy
Deployment frequency Number of deployments per day
Success rate Successful deployments / total deployments
Time to detect defects Mean time taken to detect a bug in the
production
Time to recover Mean time to recover
Other application metrics For example, improvements in resource
consumption due to devops feedback.
54. Tools
• So many vendors
• So many alternatives
• Cost vs business requirements
• SaaS devops tools
• Vendor lockin
• Overlap different phases of Devops cycles
• Separate ops team. Champions
57. PaaS
• Challenging and helping at the same time
• Security concerns
• On demand infrastructure
• Wipe out and re-create the complete infrastructure within an hour
• Do that with your own server. Good luck!
• Demand fluctuations - Scalability
59. Container orchestration
• Kubernetes
• For container orchestration
• Distribution
• Scheduling
• Logical container units
• When there are hundreds of containers to manage
• Works with docker
64. Causes of failure
• Lack of willingness
• Lack of trust
• Resistance to change
• Lack of leadership vision
• Lack of knowledge
• Devops taken as ops
• Old baggage
• Lack of patience
65. I’m a student. Where should I start ?
• Learn Linux Fundamentals. Please !
• Learn the devops concepts (Here I covered few…)
• Learn Devops-y languages, e.g., Python, Perl, Gradle
• Learn about new devops tools and learn their pros and cons, e.g.,
Kubernetes vs Docker Swarm
• Join a meetup
• Improve soft skills
66. Conclusion
• Devops is a culture change
• There are
• business benefits, e.g., faster feature delivery
• technical benefits, e.g., faster defect detection and recovery
• Concepts
• Devops is not a silver bullet
• Continuously optimize