This document discusses issues related to operations support from a software engineering perspective. It begins by providing context on NICTA and an overview of traditional and broader views of software systems. It then discusses key activities of system operators, including monitoring systems, installing new applications, and ensuring business continuity. Common problems that can occur during these operator activities are outlined. The document also provides examples of NICTA research related to disaster recovery, managing upgrades, operator undo capabilities, and modeling installation processes. Throughout, it emphasizes the importance of considering operators in the development and management of software systems.
DevOps practices emphasize small, independent teams and continuous deployment. This leads to system structures with: 1) a deep service-oriented hierarchy to enable reuse across small teams; 2) versioning and backward compatibility to allow simultaneous deployment from multiple teams; and 3) packaging of multiple services per virtual machine while avoiding race conditions during deployment.
The document discusses the quality attribute of upgradability. It begins by providing background on NICTA and then discusses how upgrades to enterprise systems often fail around 10% of the time despite being thoroughly tested. It analyzes why upgrades fail through a Failure Modes and Effects Analysis that examines the potential faults at each stage of an upgrade process, including making the upgrade available, preparing the environment, configuration, deployment, and activation. It also provides research opportunities to improve success rates of upgrades.
The document discusses a rapid deployment methodology for BMC Remedy solutions developed by generationE Technologies. It outlines the issues affecting traditional BMC Remedy deployments like developer productivity and time to market. It then describes the benefits of a rapid application development approach and generationE's BMC Remedy rapid deployment methodology which uses iterative development, prototyping, and timeboxing to compress project timelines. A case study example is also provided.
Ibm innovate adoption of continuous delivery at scale at a large telco - pr...Mirco Hering
The document presents a maturity model for adopting continuous delivery at a large telecommunications company. It outlines various technical capabilities required for continuous delivery, arranged in a dependency tree showing the foundational capabilities required first. These capabilities include features like automated unit testing, configuration management, and deployment practices. The model is intended to help teams progressively improve processes and achieve continuous delivery over multiple years by establishing these foundation capabilities. It also provides definitions and metrics for measuring progress for each capability.
The document discusses several software development life cycle (SDLC) models, including Waterfall, Incremental, Spiral, Evolutionary Prototyping, Agile, and Rapid Application Development (RAD) models. It provides an overview of the key phases and characteristics of each model, as well as their strengths, limitations, and situations where they are best applied. The models differ in their structure, flexibility to change, emphasis on documentation or code, and ability to incorporate customer feedback throughout the development process.
Nilesh Kumar Demmita has over 14 years of experience managing IT projects involving enterprise applications, global infrastructure operations, and customer support across geographies. He has expertise in managing mainframe and legacy systems projects, complex upgrades, and large transitions with onsite and offshore models. Notable achievements include successfully implementing cost savings initiatives and transitions. He is currently the Mainframe Competency Manager at Wipro Technologies, overseeing demand, competency development, delivery assurance, and pre-sales support for global mainframe projects.
IBM's DevOps solution for CLM includes a full lifecycle suite of products for managing continuous business planning, Agile project management, continuous build, source code management, test management, and continuous application monitoring.
ITIL Best Practice for Software CompaniesDaniel Brody
Detailed outline of an Information Technology Infrastructure Library (ITIL) is a set of practices for IT service management (ITSM) that focuses on aligning IT services for Software Companies
DevOps practices emphasize small, independent teams and continuous deployment. This leads to system structures with: 1) a deep service-oriented hierarchy to enable reuse across small teams; 2) versioning and backward compatibility to allow simultaneous deployment from multiple teams; and 3) packaging of multiple services per virtual machine while avoiding race conditions during deployment.
The document discusses the quality attribute of upgradability. It begins by providing background on NICTA and then discusses how upgrades to enterprise systems often fail around 10% of the time despite being thoroughly tested. It analyzes why upgrades fail through a Failure Modes and Effects Analysis that examines the potential faults at each stage of an upgrade process, including making the upgrade available, preparing the environment, configuration, deployment, and activation. It also provides research opportunities to improve success rates of upgrades.
The document discusses a rapid deployment methodology for BMC Remedy solutions developed by generationE Technologies. It outlines the issues affecting traditional BMC Remedy deployments like developer productivity and time to market. It then describes the benefits of a rapid application development approach and generationE's BMC Remedy rapid deployment methodology which uses iterative development, prototyping, and timeboxing to compress project timelines. A case study example is also provided.
Ibm innovate adoption of continuous delivery at scale at a large telco - pr...Mirco Hering
The document presents a maturity model for adopting continuous delivery at a large telecommunications company. It outlines various technical capabilities required for continuous delivery, arranged in a dependency tree showing the foundational capabilities required first. These capabilities include features like automated unit testing, configuration management, and deployment practices. The model is intended to help teams progressively improve processes and achieve continuous delivery over multiple years by establishing these foundation capabilities. It also provides definitions and metrics for measuring progress for each capability.
The document discusses several software development life cycle (SDLC) models, including Waterfall, Incremental, Spiral, Evolutionary Prototyping, Agile, and Rapid Application Development (RAD) models. It provides an overview of the key phases and characteristics of each model, as well as their strengths, limitations, and situations where they are best applied. The models differ in their structure, flexibility to change, emphasis on documentation or code, and ability to incorporate customer feedback throughout the development process.
Nilesh Kumar Demmita has over 14 years of experience managing IT projects involving enterprise applications, global infrastructure operations, and customer support across geographies. He has expertise in managing mainframe and legacy systems projects, complex upgrades, and large transitions with onsite and offshore models. Notable achievements include successfully implementing cost savings initiatives and transitions. He is currently the Mainframe Competency Manager at Wipro Technologies, overseeing demand, competency development, delivery assurance, and pre-sales support for global mainframe projects.
IBM's DevOps solution for CLM includes a full lifecycle suite of products for managing continuous business planning, Agile project management, continuous build, source code management, test management, and continuous application monitoring.
ITIL Best Practice for Software CompaniesDaniel Brody
Detailed outline of an Information Technology Infrastructure Library (ITIL) is a set of practices for IT service management (ITSM) that focuses on aligning IT services for Software Companies
Rational Team Concert is IBM's tool for team collaboration powered by the Jazz platform. The Jazz platform provides middleware services that allow tools to communicate by listening to and sending standardized events, reducing complexity compared to direct tool integration. The high-level Jazz architecture includes Rational Team Concert for source control, work items, and building; and a Jazz server with extensions that provides team services like presence and chat. Jazz aims to improve team productivity by providing awareness of teams, artifacts, responsibilities and processes to help avoid issues like broken builds.
The document provides an overview of the Rational Unified Process (RUP) software engineering methodology. It discusses key RUP concepts and best practices such as developing software iteratively, managing requirements, using component architectures, visually modeling software, continuously verifying software quality, and controlling changes to software. The RUP is presented as a mature, disciplined process that guides development activities and provides standardized artifacts and deliverables. It incorporates industry best practices and can be adapted to individual project needs while providing a common development framework.
SDLC is a framework defining tasks performed at each step in the software or system development process. It aims to produce high quality system that meets or exceeds customer expectations, work effectively and efficiently in the current and planned information technology infrastructure, and is inexpensive to maintain and cost effective to enhance.
This presentation includes different stages of Software Deveolopment.
You wouldn't be surprised if i told you that we do live in interesting times. New business models are created at the same pace today at which older ones are being destroyed. Technology is no longer just an enabler for business, it has become the business for most organizations. In this session we will touch upon some of the challenges and opportunities that the cloud has to offer. The cloud (IaaS, PaaS, SaaS) as we know it offers organizations immense opportunity in terms of reducing time to market on delivering engaging customer experiences but with all of that agility a move to the cloud also brings numerous challenges, some obvious and some not very obvious. In this session we will go over challenges engineering systems for the cloud including a case study engineering a complex legacy application for the cloud.
IT Ops Mgmt in the New Virtualized, Software-defined WorldEMC
During this recorded webcast, you will hear from EMC and Enterprise Management Associates Inc. discuss leading approaches to manage your virtualized, software-defined environment. Learn how the EMC Service Assurance Suite for Virtual Data Center and Software-Defined environments, can help you deliver new services quickly and reliably, identify and resolve problems before service impact, and improve operational efficiency.
Virtual Private Data Center Solution OverviewAngela Chavez
With a Virtual Private Data Center (VPDC) from NewCloud, all the delivery and management of data center resources is handled for you in the cloud, while still giving you access to virtual machine management via a user-friendly portal. NewCloud’s solutions are based on industry-leading technologies from VMware including vCloud Director, vShpere and vApp, providing you with the most flexible, reliable and secure cloud infrastructure available today.
For more information, check us out at www.newcloudnetworks.com
The document discusses various software development life cycle (SDLC) models, including:
- The waterfall model, which uses sequential phases of requirements, design, coding, testing, and deployment. It is structured but rigid.
- Iterative development models, which allow for feedback loops and releasing partial software in iterations to get faster feedback.
- Agile methodologies like Scrum, which embrace changing requirements, focus on working software over documentation, and value customer collaboration over contracts. Key aspects are iterative development, regular refactoring, and communicating for learning.
- Pitfalls of agile include skill gaps, lack of traceability, poor communication, and not staying close enough to customers. Overall, agile aims to
This document contains Chuck Roden's resume. It summarizes his experience as a senior systems analyst and project manager with over 30 years of experience in information technology. He is seeking immediate availability for interviews and project engagements.
This lecture provides an overview of software engineering concepts including definitions of software, what software engineering is, and the major shifts in software development approaches over time. It discusses the differences between student projects and industrial strength software in terms of quality, cost, and development time. The lecture also covers problems commonly encountered in software projects such as not meeting user needs, going over budget, missing deadlines, and being unreliable or unusable. The goal of software engineering is to provide techniques to help resolve these problems and achieve high quality software delivery on time and within budget.
The Changing Role of IT:From Service Managers to AdvisorsJesse Stockall
The document discusses the changing role of IT from service managers to advisors. It notes the rise of hybrid IT environments with public cloud, private cloud, SaaS, edge computing and more. It outlines how IT roles are expanding to include DevOps, platform operations, security operations and site reliability engineering. IT teams now operate more like service providers but with less decision making power. The document advocates for automation and governance to help scale operations while controlling costs and security risks across complex hybrid environments.
Fifteen Years of DevOps -- LISA 2012 keynoteGeoff Halprin
There has been a lot of hullabaloo over the past few years around a concept called “DevOps.” The idea is that we need to break down the barriers between development and operations teams, and treat infrastructure as code, in order to move towards better software, more reliable and scalable systems, and continuous deployment.
For some of us who have been around a while, this is just a new label for something we’ve always done.
They say those that don’t learn from history are destined to repeat it. In this talk, we will look back at how the DevOps movement evolved, what it advocates, what it doesn’t address, and what you should take away from the movement that will help you in your professional life. We will also use this opportunity to look back over the past decade or two of system administration, and see how our challenges have changed, and how they have remained the same.
Jeff Reynolds is the Director of Enterprise Solutions Consulting at CollabNet. He has over 24 years of experience in software development. CollabNet provides an enterprise platform called TeamForge that allows organizations to securely manage development tools like Git and Subversion across distributed teams. TeamForge uses a community architecture approach with features like site organization, access controls, templates, and associating related intellectual property to address the needs of highly complex organizations.
L'any 2011, en l'entorn del grup de treball TF-NOC (Task Force for Network Operation Centres), es va fer una enquesta entre els centres d'operació de xarxa per obtenir informació sobre les eines utilitzades per a monitoratge, estadístiques, etc., fent un mapeig entre les funcionalitats i les eines que es fan servir als NOC per a aquestes funcionalitats.
Donat que han aparegut noves necessitats i noves eines, i també que altres han evolucionat o desaparegut, a la primera reunió del Grup d'Interès sobre Centres d'Operació de Xarxes (SIG-NOC, Special Interest Group - Network Operation Centers) de Géant es va decidir fer una nova edició de l'enquesta entre els seus membres. En aquesta presentació es fa una revisió dels resultats de l'anterior edició i es proposen alguns canvis per debatre quina serà l'enquesta definitiva, que es farà entre el desembre i el gener, i de la qual es presentaran els resultats al proper SIG-NOC Meeting.
This presentation discusses the benefits of merging NPM & APM together to better assist problem response teams in troubleshooting network and application problems.
The presentation highlights a new product offering called NetPod which is a joint solution developed between Emulex and Dynatrace.
The document discusses software security remediation and provides data on how long it takes to fix common vulnerabilities. It finds that setup, testing fixes, and deployment take significant time. Cross-site scripting fixes average 9.6 minutes for stored and 16.2 for reflected XSS. Confirming fixes and testing can take more time than the fixes. The data provides a starting point for planning but has limitations from being one company's projects. Understanding all phases is key to minimizng remediation costs.
Building The Agile Enterprise - LSSC '12Gil Irizarry
The document discusses how to scale Agile practices beyond individual teams to the enterprise level using Kanban and release management. It recommends prioritizing work, planning dependencies between teams, continuously integrating and testing code, and deploying features using a "release train" model even if not all work is complete by a deadline. Automating builds, tests and deployments is key to enabling small, frequent releases across multiple interdependent teams.
Chuck Roden has over 33 years of experience in roles such as senior systems analyst, SME, project manager, and architectural engineer. He has a proven track record of building team collaboration and meeting deliverable timelines. Roden has extensive experience supporting large corporate clients with over 18,000 servers and leading teams of over 45 people. He is seeking a new opportunity and is immediately available for an interview or new engagement.
Four Essential Steps for Removing Risk and Downtime from POWER9 MigrationPrecisely
The performance and scalability of IBM’s POWER9 servers is exciting, but the prospect of migrating to new systems isn’t. View this webinar on-demand as we explore the essential steps you need to take when planning and executing a POWER9 migration project to eliminate risk and downtime. We also share how real-time replication can be leveraged to support a painless cutover to your new machine.
During this webinar, we discuss:
• Assessing migration scope and planning your project
• Selecting a migration method
• Creating and executing a migration plan
• How Syncsort can help
We will run a scenario on three types of monolithic architectures and then focus on how it is done with microservices. Amazon EventBridge and the Event Carried State Transfer pattern to create loosely coupled and independent services. This eliminates synchronous calls between services to increase system availability.
Presented at: https://www.serverless-summit.io/
Architecting for the cloud scability-availabilityLen Bass
This document discusses architecting for scalability and availability in the cloud. It begins with an introduction to scalability, including definitions of scalability and why systems need to scale. It describes techniques for scaling such as load balancers, rule-based autoscaling, and scaling patterns like push and pull. It also covers CPU and I/O scaling. The document then discusses availability, including definitions and metrics. It describes different availability levels, calculating system availability, and how to define availability requirements based on understanding potential fault scenarios.
Architecting for the cloud cloud providersLen Bass
The document discusses cloud providers and services available on Amazon Web Services. It provides an overview of compute, storage, database, and other services and how they can provide redundancy across availability zones and regions. Examples are given of different outage scenarios that can occur at the zone, region, or provider level and strategies for architecting applications to mitigate risks from these outages.
Rational Team Concert is IBM's tool for team collaboration powered by the Jazz platform. The Jazz platform provides middleware services that allow tools to communicate by listening to and sending standardized events, reducing complexity compared to direct tool integration. The high-level Jazz architecture includes Rational Team Concert for source control, work items, and building; and a Jazz server with extensions that provides team services like presence and chat. Jazz aims to improve team productivity by providing awareness of teams, artifacts, responsibilities and processes to help avoid issues like broken builds.
The document provides an overview of the Rational Unified Process (RUP) software engineering methodology. It discusses key RUP concepts and best practices such as developing software iteratively, managing requirements, using component architectures, visually modeling software, continuously verifying software quality, and controlling changes to software. The RUP is presented as a mature, disciplined process that guides development activities and provides standardized artifacts and deliverables. It incorporates industry best practices and can be adapted to individual project needs while providing a common development framework.
SDLC is a framework defining tasks performed at each step in the software or system development process. It aims to produce high quality system that meets or exceeds customer expectations, work effectively and efficiently in the current and planned information technology infrastructure, and is inexpensive to maintain and cost effective to enhance.
This presentation includes different stages of Software Deveolopment.
You wouldn't be surprised if i told you that we do live in interesting times. New business models are created at the same pace today at which older ones are being destroyed. Technology is no longer just an enabler for business, it has become the business for most organizations. In this session we will touch upon some of the challenges and opportunities that the cloud has to offer. The cloud (IaaS, PaaS, SaaS) as we know it offers organizations immense opportunity in terms of reducing time to market on delivering engaging customer experiences but with all of that agility a move to the cloud also brings numerous challenges, some obvious and some not very obvious. In this session we will go over challenges engineering systems for the cloud including a case study engineering a complex legacy application for the cloud.
IT Ops Mgmt in the New Virtualized, Software-defined WorldEMC
During this recorded webcast, you will hear from EMC and Enterprise Management Associates Inc. discuss leading approaches to manage your virtualized, software-defined environment. Learn how the EMC Service Assurance Suite for Virtual Data Center and Software-Defined environments, can help you deliver new services quickly and reliably, identify and resolve problems before service impact, and improve operational efficiency.
Virtual Private Data Center Solution OverviewAngela Chavez
With a Virtual Private Data Center (VPDC) from NewCloud, all the delivery and management of data center resources is handled for you in the cloud, while still giving you access to virtual machine management via a user-friendly portal. NewCloud’s solutions are based on industry-leading technologies from VMware including vCloud Director, vShpere and vApp, providing you with the most flexible, reliable and secure cloud infrastructure available today.
For more information, check us out at www.newcloudnetworks.com
The document discusses various software development life cycle (SDLC) models, including:
- The waterfall model, which uses sequential phases of requirements, design, coding, testing, and deployment. It is structured but rigid.
- Iterative development models, which allow for feedback loops and releasing partial software in iterations to get faster feedback.
- Agile methodologies like Scrum, which embrace changing requirements, focus on working software over documentation, and value customer collaboration over contracts. Key aspects are iterative development, regular refactoring, and communicating for learning.
- Pitfalls of agile include skill gaps, lack of traceability, poor communication, and not staying close enough to customers. Overall, agile aims to
This document contains Chuck Roden's resume. It summarizes his experience as a senior systems analyst and project manager with over 30 years of experience in information technology. He is seeking immediate availability for interviews and project engagements.
This lecture provides an overview of software engineering concepts including definitions of software, what software engineering is, and the major shifts in software development approaches over time. It discusses the differences between student projects and industrial strength software in terms of quality, cost, and development time. The lecture also covers problems commonly encountered in software projects such as not meeting user needs, going over budget, missing deadlines, and being unreliable or unusable. The goal of software engineering is to provide techniques to help resolve these problems and achieve high quality software delivery on time and within budget.
The Changing Role of IT:From Service Managers to AdvisorsJesse Stockall
The document discusses the changing role of IT from service managers to advisors. It notes the rise of hybrid IT environments with public cloud, private cloud, SaaS, edge computing and more. It outlines how IT roles are expanding to include DevOps, platform operations, security operations and site reliability engineering. IT teams now operate more like service providers but with less decision making power. The document advocates for automation and governance to help scale operations while controlling costs and security risks across complex hybrid environments.
Fifteen Years of DevOps -- LISA 2012 keynoteGeoff Halprin
There has been a lot of hullabaloo over the past few years around a concept called “DevOps.” The idea is that we need to break down the barriers between development and operations teams, and treat infrastructure as code, in order to move towards better software, more reliable and scalable systems, and continuous deployment.
For some of us who have been around a while, this is just a new label for something we’ve always done.
They say those that don’t learn from history are destined to repeat it. In this talk, we will look back at how the DevOps movement evolved, what it advocates, what it doesn’t address, and what you should take away from the movement that will help you in your professional life. We will also use this opportunity to look back over the past decade or two of system administration, and see how our challenges have changed, and how they have remained the same.
Jeff Reynolds is the Director of Enterprise Solutions Consulting at CollabNet. He has over 24 years of experience in software development. CollabNet provides an enterprise platform called TeamForge that allows organizations to securely manage development tools like Git and Subversion across distributed teams. TeamForge uses a community architecture approach with features like site organization, access controls, templates, and associating related intellectual property to address the needs of highly complex organizations.
L'any 2011, en l'entorn del grup de treball TF-NOC (Task Force for Network Operation Centres), es va fer una enquesta entre els centres d'operació de xarxa per obtenir informació sobre les eines utilitzades per a monitoratge, estadístiques, etc., fent un mapeig entre les funcionalitats i les eines que es fan servir als NOC per a aquestes funcionalitats.
Donat que han aparegut noves necessitats i noves eines, i també que altres han evolucionat o desaparegut, a la primera reunió del Grup d'Interès sobre Centres d'Operació de Xarxes (SIG-NOC, Special Interest Group - Network Operation Centers) de Géant es va decidir fer una nova edició de l'enquesta entre els seus membres. En aquesta presentació es fa una revisió dels resultats de l'anterior edició i es proposen alguns canvis per debatre quina serà l'enquesta definitiva, que es farà entre el desembre i el gener, i de la qual es presentaran els resultats al proper SIG-NOC Meeting.
This presentation discusses the benefits of merging NPM & APM together to better assist problem response teams in troubleshooting network and application problems.
The presentation highlights a new product offering called NetPod which is a joint solution developed between Emulex and Dynatrace.
The document discusses software security remediation and provides data on how long it takes to fix common vulnerabilities. It finds that setup, testing fixes, and deployment take significant time. Cross-site scripting fixes average 9.6 minutes for stored and 16.2 for reflected XSS. Confirming fixes and testing can take more time than the fixes. The data provides a starting point for planning but has limitations from being one company's projects. Understanding all phases is key to minimizng remediation costs.
Building The Agile Enterprise - LSSC '12Gil Irizarry
The document discusses how to scale Agile practices beyond individual teams to the enterprise level using Kanban and release management. It recommends prioritizing work, planning dependencies between teams, continuously integrating and testing code, and deploying features using a "release train" model even if not all work is complete by a deadline. Automating builds, tests and deployments is key to enabling small, frequent releases across multiple interdependent teams.
Chuck Roden has over 33 years of experience in roles such as senior systems analyst, SME, project manager, and architectural engineer. He has a proven track record of building team collaboration and meeting deliverable timelines. Roden has extensive experience supporting large corporate clients with over 18,000 servers and leading teams of over 45 people. He is seeking a new opportunity and is immediately available for an interview or new engagement.
Four Essential Steps for Removing Risk and Downtime from POWER9 MigrationPrecisely
The performance and scalability of IBM’s POWER9 servers is exciting, but the prospect of migrating to new systems isn’t. View this webinar on-demand as we explore the essential steps you need to take when planning and executing a POWER9 migration project to eliminate risk and downtime. We also share how real-time replication can be leveraged to support a painless cutover to your new machine.
During this webinar, we discuss:
• Assessing migration scope and planning your project
• Selecting a migration method
• Creating and executing a migration plan
• How Syncsort can help
We will run a scenario on three types of monolithic architectures and then focus on how it is done with microservices. Amazon EventBridge and the Event Carried State Transfer pattern to create loosely coupled and independent services. This eliminates synchronous calls between services to increase system availability.
Presented at: https://www.serverless-summit.io/
Architecting for the cloud scability-availabilityLen Bass
This document discusses architecting for scalability and availability in the cloud. It begins with an introduction to scalability, including definitions of scalability and why systems need to scale. It describes techniques for scaling such as load balancers, rule-based autoscaling, and scaling patterns like push and pull. It also covers CPU and I/O scaling. The document then discusses availability, including definitions and metrics. It describes different availability levels, calculating system availability, and how to define availability requirements based on understanding potential fault scenarios.
Architecting for the cloud cloud providersLen Bass
The document discusses cloud providers and services available on Amazon Web Services. It provides an overview of compute, storage, database, and other services and how they can provide redundancy across availability zones and regions. Examples are given of different outage scenarios that can occur at the zone, region, or provider level and strategies for architecting applications to mitigate risks from these outages.
This paper discusses challenges in diagnosing errors when deploying Hadoop ecosystems. It provides 15 examples of specific errors that can occur with Hbase/Hadoop deployment on Amazon EC2, along with potential root causes. The paper also classifies errors as operational, configuration, software, or resource-related. It identifies inconsistencies across component logs, high signal-to-noise ratios, and uncertainty in correlating events as difficulties for error diagnosis. The paper contributes examples to a repository for mapping deployment symptoms to fault trees to determine root causes.
Architecture patterns for continuous deploymentLen Bass
This document discusses architectural patterns to support continuous deployment. It recommends a service-oriented architecture with many loosely coupled microservices to align with small, independent teams. Services can be packaged individually or together in virtual machines, balancing independence and reuse. Backward compatibility is key and can be achieved through translation layers. Services should be version aware to safely utilize dependencies as they are upgraded. Overall structure, packaging, backward compatibility, and version awareness are important considerations for continuous deployment.
The document discusses dependability in cloud computing applications. It defines dependability as the ability of a system to deliver a service that can be trusted. In cloud environments, dependability concerns include instance and data failures, performance issues like latency during provisioning, and security threats from shared infrastructure. The document outlines techniques for handling stateful and stateless application components during failures to help achieve high dependability in cloud applications. These include checkpointing state periodically and replaying logged messages during instance recovery.
A collection of exercises to build a simple deployment pipeline. This comes from the course I have taught in DevOps and is targeted at instructors or individuals who want to learn the basics of a pipeline.
DevOps refers to practices aimed at reducing the time between committing a code change and deploying it to production while maintaining quality. This document discusses various DevOps practices including treating operators as first class citizens, making developers responsible for incident handling, enforcing uniform deployment processes, using continuous deployment, and developing infrastructure code like application code. Microservice architectures support continuous deployment by decentralizing coordination needs into architectural decisions.
This document discusses deployability and continuous deployment in the context of microservice architectures. It begins by describing National ICT Australia (NICTA) and its work in information and communications technology research. It then discusses how microservice architectures support continuous deployment by allowing individual teams to deploy new versions of their services independently without coordination. Key aspects of microservice architectures that enable this include: each service having a single responsibility; services communicating asynchronously via messaging; and services registering themselves with a discovery service. The document also discusses how feature toggles and canary deployments can be used to maintain consistency when deploying new versions of services.
Architecting for the cloud elasticity securityLen Bass
Concurrency and state management are important considerations for achieving elasticity in cloud systems. There are three types of state: session state kept by clients, server-side state kept in processes, and persistent state stored externally. Server-side state makes scaling difficult, while stateless servers allow elasticity. Memcached provides a way to synchronize small amounts of in-memory state across servers to support stateless services running elastically in the cloud.
Architecting for the cloud intro, virtualization, iaa sLen Bass
- The document provides an introduction to virtual machines, virtual networks, and Infrastructure as a Service (IaaS) cloud computing.
- It demonstrates how to run a virtual machine using Oracle VirtualBox, showing that a virtual machine acts like a fully functional computer system isolated from the host machine.
- Key concepts discussed include virtual memory address translation, hypervisors managing virtual machine page tables and scheduling, and virtual machine images containing the files and settings used to boot virtual machines.
Architecting for the cloud storage build testLen Bass
This document discusses best practices for deploying applications to the cloud, including:
- Using a deployment pipeline with continuous integration, integration testing, and staging environments to minimize errors and delays.
- Managing versions and branches to prevent errors from multiple teams working simultaneously.
- Performing integration testing after each commit to catch errors early.
- Maintaining separate databases for different environments like test vs production.
- Using feature toggles to allow uncompleted code to be checked in without breaking builds.
- Performing staging tests using production data and load to thoroughly test before deployment.
Architectural Tactics for Large Scale SystemsLen Bass
The document discusses several challenges for large-scale systems operating in cloud environments, including failure, inconsistency, continuous deployment, and installation errors. It describes tactics used by companies like Google and Netflix to address issues like fault tolerance, eventual consistency, rolling upgrades of loosely coupled services, and error diagnosis across distributed systems. The document also outlines research at NICTA to build process models for installation, analyze configuration errors, and develop new tactics for managing changes in complex cloud applications.
The document discusses different packaging tool options for deploying code changes through a continuous integration and deployment pipeline. It describes how tools like Vagrant, Chef, Puppet, Ansible, and Docker handle various stages of the process like creating virtual machines, specifying configuration parameters, building machine images, and loading images into VMs. Containers are presented as an approach to speed up deployment by only loading updated components rather than entire virtual machine images each time.
Principles of software architecture designLen Bass
The document discusses principles of software architecture design. It states that quality attribute requirements have the strongest influence on architectural design. Quality attributes can be specified through concrete scenarios involving stimuli, sources, environments, artifacts, responses, and measures. Architectural tactics are techniques that improve specific quality attributes by modifying architectures. Tactics relate to quality models or expert experience and are important for designing and evaluating architectures.
Are your cloud applications performing? How Application Performance Managemen...DevOps.com
This document discusses application modernization and why application performance monitoring (APM) is important during the modernization process. It provides an overview of common business reasons for modernizing applications, such as increasing flexibility, availability, scalability and portability. The document then discusses common challenges of modernization and provides examples of how companies approach modernizing applications. It emphasizes the importance of APM throughout the modernization lifecycle to deliver applications with speed, quality and control. The document concludes with examples of client experiences modernizing applications and lessons learned regarding monitoring tools in containerized/cloud environments.
Presentazione dello speech tenuto da Carmine Spagnuolo (Postdoctoral Research Fellow - Università degli Studi di Salerno/ ACT OR) dal titolo "Technology insights: Decision Science Platform", durante il Decision Science Forum 2019, il più importante evento italiano sulla Scienza delle Decisioni.
The document discusses scalability and provides examples of scaling a weather forecast application from 1,000 users to over 1 million users. It describes scaling the application vertically by upgrading server hardware, horizontally by adding more servers, and through distributed architectures like clustering, caching, and moving to a cloud solution. The key aspects covered are performance testing, high availability, and optimizing the application infrastructure as user load increases substantially over time.
Developers are in constant search for greenfield projects because it allows them to be free of constraints. But over time, I started to appreciate the innovation found within working around constraints.
The goal of this presentation is use two experiences in my career which highlights constraints which begged for innovation. Only on my retrospective of these developments, one can appreciate the true novelty in constraints.
App Modernization with .NET Core: How Travelers Insurance is Going Cloud-NativeVMware Tanzu
At Travelers Insurance, a decade old rating engine was still on the mainframe, making it difficult to scale, expensive to run, and reliant on a shrinking pool of skilled engineers. Rating engines are fundamental to the insurance quoting process, and at the very heart of the business. It was time to modernize and take advantage of the scalability, stability, automation and fast iteration cycles made possible by cloud-native architecture.
In this webinar, Viraj Naik of Travelers Insurance and Rohit Kelapure of Pivotal will take us on their journey from mainframe to microservices. Viraj and Rohit will describe how they built a distributed, event-driven rating engine with .NET Core on Pivotal Platform using Steeltoe, running on Linux stem cells. The new rating engine exceeded SLAs and reduced time to production to under 60 minutes.
You’ll learn the keys to a successful mainframe rewrite-based modernization, including:
● A pragmatic, domain-driven approach and phased delivery, including implementing a strangler pattern and anti-corruption layers.
● How to port business objects and business rules from mainframe to .NET.
● New innovations developed and delivered during the migration process.
Speakers:
Viraj Naik, Lead Solutions Architect at Travelers Insurance
Rohit Kelapure, Principal Solution Architect at Pivotal
Aitp presentation ed holub - october 23 2010AITPHouston
This presentation from Gartner discusses 10 top IT infrastructure and operations trends for organizations to watch. The trends covered include virtualization, big data, energy efficiency, unified communications, staff retention, social networks, legacy migrations, compute density, cloud computing, and converged fabrics. For each trend, the presentation provides details on how the trend affects organizations and recommendations on how to prepare and respond. The overall message is that IT leaders need to be aware of these emerging trends and develop strategies to leverage and adapt to them.
The document discusses challenges with development and operations teams wanting different things and outlines how adopting DevOps practices can help. It provides an example of a utilities company that automated their application deployment, reducing time from 30 minutes with 5 people to 5 minutes. Adopting DevOps through practices like continuous integration, centralized tools, and automated deployments can improve productivity, compliance, and reduce costs.
Visualizing Your Network Health - Know your NetworkDellNMS
An old adage states that you cannot manage what you don’t know. Do you know what devices are on your network, where they are located, how they are configured, what they are connected to, and how they are affected by changes and failures?
Today’s network infrastructure is becoming more and more complex, while demands on the Network Administrator to ensure network availability and performance are higher than ever. Business critical systems depend upon you managing your entire network infrastructure and delivering high-quality service 24/7, 365 days a year. So how do you keep the pace?
Learn how real-time visibility into your entire network infrastructure provides the power to manage your assets with greater control.
This document discusses adopting a DevOps approach for 2-Speed IT. It presents value stream mapping as a way to identify bottlenecks in development and delivery pipelines. Addressing these bottlenecks through practices like continuous integration, deployment automation, and shifting security left can help organizations deliver hybrid applications across hybrid platforms and teams more quickly and with higher quality. Case studies are presented of organizations that improved delivery times, increased innovation, and gained competitive advantages by adopting DevOps.
Getting Started with ThousandEyes Proof of ConceptsThousandEyes
The document provides an overview and agenda for a ThousandEyes proof of concept. It discusses the ThousandEyes overview, identifying opportunities, defining success criteria, executing the proof of concept, and includes a demo. The agenda includes preparing for the proof of concept over two weeks, running the active trial for 4-6 weeks, and developing a go-forward plan over another two weeks. It also discusses best practices for executing the proof of concept and ensuring a focus on the defined success criteria.
join SolarWinds Federal Systems Engineers, Product Management, Head Geek, and other U.S. Federal Government users of SolarWinds software (Military, Civilian, and Contractors) tomorrow for an online Federal User Group. We hosted this online Federal User Group in an interactive Webcast format so that SolarWinds Federal customers can share feedback with the SolarWinds product team, hear the latest product updates, and federal features and tips.
This free online Federal User Group is an exciting opportunity for you to:
• Learn about the most recent updates to SolarWinds IT software
• Hear about the latest Federal features & tips and enterprise scalability of SolarWinds software
Speakers:
• Edward Bender, Head Federal Systems Engineer, SolarWinds
• Francois Caron, Product Management Director, SolarWinds
•Patrick Hubbard, Head Geek, SolarWinds
Agenda:
• Welcome
• SolarWinds Product Update - Francois Caron, Product Management Director, SolarWinds, Patrick Hubbard, Head Geek, SolarWinds
• Federal Features & Tips, Scaling to the Federal Enterprise - Edward Bender, Head Federal Systems Engineer, SolarWinds
•Q&A/Closing Remarks
Converting from a three tier or monolithic application to microservices can be daunting, and often comes at a non-trivial cost or effort. So why are organizations doing it, and how do they justify the expense? We will discuss some of the practices and migration strategies used by organizations who undergo this sort of transformation, such as extracting functions through refactoring and converting them to microservices. As the journey progresses, we learn that there is no one-size-fits-all approach to making applications cloud-native… so the real question needs to be ‘how do I find the right approach for me?’ We can help you begin to answer that question for yourself, by discussing the facets of consideration such as technical, procedural, and risk tolerance to name a few.
Cloud-Native Data: What data questions to ask when building cloud-native appsVMware Tanzu
While a number of patterns and architectural guidelines exist for cloud-native applications, a discussion about data often leads to more questions than answers. For example, what are some of the typical data problems encountered, why are they different, and how can they be overcome?
Join Prasad Radhakrishnan from Pivotal and Dave Nielsen from Redis Labs as they discuss:
- Expectations and requirements of cloud-native data
- Common faux pas and strategies on how you can avoid them
Presenters:
Prasad Radhakrishnan, Platform Architecture for Data at Pivotal
Dave Nielsen, Head of Ecosystem Programs at Redis Labs
Reactive applications tools of the trade huff poshinolajla
A description of what makes Reactive an important term in developing today's enterprise applications, and a discussion of various tools that support Reactive, including node.js, Go, RxJava, Futures and Actors.
This webinar will be presented by Greg Lowe (COO/Partner) of McKonly & Asbury and Mike Yeager (President) of Cargas Systems. The webinar will provide an overview of cloud accounting applications as well as a discussion on how a cloud accounting system can lower your total cost of ownership of accounting software. During the webinar, Greg and Mike will also talk about why fast growing companies are adopting cloud accounting and how it is benefiting their organization. In addition, they will provide a brief overview demonstration of Intacct, a best in class accounting application.
Enable business continuity and high availability through active active techno...Qian Li Jin
IBM provides an overview of an active-active solution implemented by China Everbright Bank for their credit card system. The solution uses WebSphere MQ for real-time data synchronization between active sites in Beijing and Shanghai. This allows workload and data to be distributed across both sites for continuous availability in case of an outage. Key components discussed include the messaging architecture, application design considerations for performance, and procedures for planned and unplanned site switches. The implementation provides business continuity for Everbright Bank's credit card processing.
Operating a Highly Available Cloud ServiceDepankar Neogi
Operating a highly available cloud service is not just about technology and architecture. It has a lot to do with people and processes. Everything fails all the time. So, how do you ensure you have the right people and the right processes in the right places to run a highly available web service. This talk covers people, processes and technology and tools required to run a highly available web service.
Raggiungere nuovi livelli di time-to market ed efficienza: dallo sviluppo, al test, alla produzone in un solo passo.
Gabriele Giacomelli, HP ALM Solution Consultant
Similar to Supporting operations personnel a software engineers perspective (20)
This document provides an overview of the DevOps: Engineering for Deployment and Operations course. The course focuses on DevOps technology and teaches infrastructure concepts like virtualization, networking, and security. Students will learn DevOps principles such as infrastructure as code, configuration management, microservices architecture, and deployment pipelines. The course involves weekly readings, videos, quizzes, discussions, and assignments using DevOps tools like Vagrant, Docker, Jenkins, Ansible, and Kubernetes. The final grade is based on daily quizzes, a final exam, assignments, and class participation.
This document outlines the syllabus for a DevOps engineering course covering deployment and operations. The course will introduce students to DevOps concepts and software engineering practices through lectures, assignments, and discussions. Students will complete assignments involving automation tools and reflect on their experiences. Assignments are graded based on automation, documentation, and reflections. The course aims to provide both theoretical knowledge through exams and practical skills through hands-on assignments.
The document discusses securing software supply chains and operations. It covers identifying critical data and resources to protect, such as credentials, sensitive data, and hardware. Techniques for protection include encryption, access controls, and input validation. OAuth and vaults are described for managing credentials for services, while RBAC and LDAP help control individual access. Regularly patching vulnerabilities is important, but the process can be complex. The OWASP Top 10 list catalogs common web application security risks and mitigations.
This document discusses disaster recovery and deployment operations for software engineers. It defines key terms like business continuity, disaster recovery, recovery time objective (RTO), and recovery point objective (RPO). Systems are divided into tiers based on their RTO and RPO. Tier 1 systems require data to be kept up to date in a mirrored secondary data center, while tiers 2-4 can use backups stored either online or offline. Software in secondary data centers must also be kept in alignment. The failover process involves triggering a switch, activating the secondary data center, and resuming operations from that site.
The document discusses three key aspects of deployment and operations for software engineers: telemetry, incident response, and live testing. Telemetry involves collecting various metrics and logs to monitor systems. When incidents occur, the response aims to restore service, analyze the cause, and prevent future occurrences. Companies differ in whether developers or dedicated teams handle incidents. Live testing after deployment identifies weaknesses by intentionally introducing failures or performing maintenance tasks.
1) The build environment creates an executable image of the system by compiling and linking all code and dependencies. It then performs functional tests on the integrated system using a test harness.
2) If the tests pass, the build is promoted to the staging environment for further testing. Otherwise, failures will stop the promotion process or just be noted.
3) The build environment aims to detect defects in module interfaces and interactions by testing the fully integrated system, using real data but in limited quantities to keep tests fast. External services may be directly used if read-only or have test versions if read-write.
Version control systems like Git allow software engineers to maintain textual information in a shared central or distributed repository with versioning capabilities. Configuration management tools like Chef and Puppet maintain consistency across machines by specifying configuration actions through scripts. Configuration parameters provide flexibility by allowing values to vary between environments, but credentials should be treated separately due to security risks of exposure.
Microservice architecture consists of independently deployable services that communicate through defined interfaces. It supports DevOps processes by reducing coordination needs and allowing independent technology choices. Microservices improve modifiability over performance and can achieve reuse through deep service hierarchies. Discovery services locate service instances by name, and communication occurs via RPC, REST, or protocol buffers with JSON or XML payloads. Microservices are packaged with dependencies as containers deployed in pods for scalability. Design considers forward/backward compatibility across versions.
This document discusses various aspects of infrastructure security. It begins with an overview of cryptography concepts like symmetric encryption, asymmetric encryption, hashing, and public key infrastructure (PKI). It then explains how protocols like TLS, SSH, and secure file transfer use these concepts to provide security. The document also summarizes intrusion detection systems and how they can monitor hosts or network traffic for anomalous activity.
Container images are stored in repositories similar to version control systems. Container orchestrators like Kubernetes allocate images to hosts in pods, which group related containers, and can automatically scale pods. Serverless architectures use containers that load quickly for individual requests from pools maintained by cloud providers, restricting usage but enabling fast load times.
The document discusses distributed systems and failures in cloud computing environments. It describes how cloud providers organize their data centers across regions and availability zones. When failures occur, they can impact the entire cloud or just parts of it. The long tail is discussed as a phenomenon where some operations take much longer than average to complete. Techniques for dealing with failures include retrying operations, instantiating redundant services, and implementing circuit breakers. Load balancers help distribute traffic across multiple servers. Autoscaling allows adding more servers when load increases. Achieving atomic operations is challenging in distributed systems due to latency and potential component failures. Consensus algorithms like Paxos can help synchronize data across servers in a consistent manner.
A container is a lightweight virtualization technology that isolates applications from other applications. Container images are structured in layers that allow for faster deployment of updates, as only changed layers need to be transferred. Loading a container across a network takes milliseconds compared to minutes for virtual machines, but containers have limitations in networking and file system capabilities compared to virtual machines.
IPv4 and IPv6 are protocols that assign numeric addresses to devices to enable communication and routing of information across the internet. DNS is a hierarchical system that maps human-friendly domain names to these numeric IP addresses. It uses caching and time-to-live values to improve efficiency. Local networks can be merged using bridges, which expose internal addresses, or NAT, which hides internal addresses behind a single external address.
This document discusses quantum computing. It explains that qubits, the basic unit of quantum computers, can exist in superposition and be entangled. Common qubit operations like CNOT gates are used to entangle qubits and generate Bell pairs. Quantum algorithms like Grover's, Shor's, and HHL are described, though current quantum computers have only demonstrated small problems compared to what the algorithms enable. Quantum computers are not expected to replace classical computers but will be useful for problems involving combinatorics.
This document discusses blockchain technology and designing systems with blockchain. It begins with an overview of the hype around blockchain and how interest has grown over time. It then covers the key elements of a blockchain, including the contract, immutable transaction history achieved through cryptography and consensus, and examples of how blockchain could be applied in areas like payments, identity management, and asset registry. The document dives deeper into specific blockchains like Bitcoin and Ethereum and the concepts of smart contracts. It also outlines CSIRO's research focus areas regarding blockchain.
This document discusses Len Bass's experience teaching a DevOps course. It begins with an overview of DevOps and what it aims to accomplish. It then describes the structure and content of the course, which combines lectures, readings, discussions, and hands-on assignments using open source DevOps tools. The document notes that students often lack fundamental knowledge in areas like networking, security, and operations. It proposes establishing an "infrastructure minor" to address gaps in students' undergraduate education.
Blockchains are distributed ledger technologies that are not well understood outside of cryptography experts. They consist of three main elements - a contract that specifies how users can interact, an immutable transaction history, and cryptography. Zerocash is a cryptocurrency that uses zero-knowledge proofs to allow transactions to occur while keeping private all details of the transaction such as sender, recipient, and amount. In conclusion, blockchains have significant potential but also hype, with security and privacy properties that can be configured through their technical design.
A blockchain has three key elements: 1) A contract that specifies how individuals can interact with the blockchain and their obligations, 2) An immutable history of all valid transactions within the contract, and 3) Cryptographic encoding of the contract and proofs of compliance that can be verified but keep details private.
DevOps and Safety Critical Systems discusses applying DevOps practices like continuous deployment to safety critical systems. It proposes "partial continuous deployment" which involves:
1. Identifying and isolating safety critical portions of a system's architecture.
2. Applying continuous deployment practices to non-safety critical portions.
3. Continuing traditional testing methods for safety critical portions.
It discusses past efforts in smart grid security controls and hardening deployment pipelines that provide foundations for this approach. Key steps include explicitly defining safety requirements, analyzing architectures to identify minimum required safe components, and refactoring to separate safe and non-safe concerns. Regulatory approval is viewed as a major gate to implementing partial continuous deployment for real safety
The document discusses securing the "last mile" of the software supply chain, which refers to getting software deployed from development to production. It presents a process for hardening the deployment pipeline that involves identifying security requirements, determining which components are trusted vs untrusted, analyzing for vulnerabilities, and refining the model by adding new trusted components until no vulnerabilities remain. Specifically, it applies this process to example deployment pipeline that uses Jenkins and Docker, finding vulnerabilities and addressing them by adding steps like encrypting files and verifying image checksums using small, independently verifiable components.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Supporting operations personnel a software engineers perspective
1. NICTA Copyright 2012 From imagination to impact
Supporting Operations
Personnel: A Software
Engineering
Perspective
Len Bass
2. NICTA Copyright 2012 From imagination to impact
2
About NICTA
National ICT Australia
• Federal and state funded research
company established in 2002
• Largest ICT research resource in
Australia
• National impact is an important
success metric
• ~700 staff/students working in 5 labs
across major capital cities
• 7 university partners
• Providing R&D services, knowledge
transfer to Australian (and global) ICT
industry
NICTA technology is
in over 1 billion mobile
phones
3. NICTA Copyright 2012 From imagination to impact
Traditional View from Software Engineers
3
Application
Cloud
Environment
Traditionally, the software engineering community
has viewed systems as being developed for users
and existing in an environment. The motivating
questions have been: With this world view: how can
development costs be reduced and run time quality
improved?
End users
Developers
4. NICTA Copyright 2012 From imagination to impact
A Broader View
4
Application
Cloud
Environment
Applications are not only affected by the behavior of the
end users but also by actions of operators who control
the environment for a consumer’s application.
Consumer
Operator
End users
Developers
5. NICTA Copyright 2012 From imagination to impact
My Message: Consider the Operator in this
Picture
5
Application
Cloud
Environment
Consumer
Operator
End users
Developers
Computer operations is a domain that impacts every
application that operates in an enterprise environment. As
such, Software Engineers need to be aware of how
actions of operators can affect their application and how
actions of their application can simplify life for operators..
6. NICTA Copyright 2012 From imagination to impact
Business Context
“Through 2015, 80% of outages impacting mission-critical
services will be caused by people and process issues, and
more than 50% of those outages will be caused by
change/configuration/release integration and hand-off
issues.”
Change/configuration/release integration and hand off are
all operations issues.
Gartner - http://www.rbiassets.com/getfile.ashx/42112626510
"I&O [Infrastructure and operations] represents
approximately 60 percent of total IT spending worldwide, "
http://www.gartner.com/it/page.jsp?id=1807615
6
7. NICTA Copyright 2012 From imagination to impact
Outline
• Overview of operations domain
– What do operators do?
– What can go wrong with what they do?
• Some results NICTA has achieved or activities
we have ongoing
7
8. NICTA Copyright 2012 From imagination to impact
What Do Operators Do?
8
Akamai’s NOC in Cambridge, Massachusetts
• Monitor and control data center/network/system
activity
– Install new/upgraded
applications/middleware/configurations/hardware
• Support business continuity through back ups
and disaster recovery
9. NICTA Copyright 2012 From imagination to impact
Monitor and Control
• Data Center
– Total number and type of resources (may be virtual)
• Processors
• Storage
• Network
• Network
– Intrusion detection
– Routing
– Loading
• System
– Allocation to resources
– Install/uninstall
– Configure 9
10. NICTA Copyright 2012 From imagination to impact
What can go Wrong with Monitor and
Control?
Everything that was on previous slide.
• Failure
• Installations can fail
• Resources fail and must be replaced
• Overload
– Resources are over/under loaded and must be
supplemented/removed
– Networks get overloaded and routing must be changed
• Error
– Routing may be incorrectly specified
– Allocation of systems to resources may be incorrect
– Configurations can be incorrectly specified
10
11. NICTA Copyright 2012 From imagination to impact
Install New/Upgraded Applications
• Specifying configuration for applications
• Synchronizing state for upgraded applications
• Testing new/upgraded applications in target
environment
• Allocating resources for new version
11
12. NICTA Copyright 2012 From imagination to impact
What Can go Wrong with Installation?
• Again its everything.
– Configuration can be misspecified
– Cut over to new version may leave inconsistent state
– Upgrade to level N of the stack may break software in
level >N of the stack
– Testing environment may not appropriately mirror real
environment
– Configuration of one level of the stack may be
inconsistent with requirements of another level.
12
13. NICTA Copyright 2012 From imagination to impact
Supporting Business Continuity
• Disasters happen – natural or human causes
• Backing up data provides recovery possibility
– Lag between last version backed up and when
disaster happens
– In the Cloud, backing up large amounts of data to
different geographic regions takes time.
13
14. NICTA Copyright 2012 From imagination to impact
Hand Offs
• Problems can arise when a shift changes
– What problems did old shift deal with?
– What problems were totally solved?
– What problems were partially solved?
– What operations activities are currently ongoing?
14
15. NICTA Copyright 2012 From imagination to impact
Operations is a Target Rich Environment
• There are many existing tools. Operation of data
centers would not work without tools
• Much room for improvement (see Gartner quote)
• Some general approaches for improvement
– Make software systems operations and tools process and
incident aware. E.g. make them aware of upgrade or shift
change
– Model operations processes and systems using a single model.
• Model analysis will provide opportunities for detecting trade offs between
human and automated activities.
• Model might also enable smoother error detection
15
16. NICTA Copyright 2012 From imagination to impact
Outline
• Overview of operations domain
• Some results we have achieved or activities
we have ongoing
– Disaster Recovery product
– Upgrade
– Operator undo
– Installation process.
16
17. NICTA Copyright 2012 From imagination to impact
Disaster Recovery
• Clouds fail – Amazon had three outages in 2011
that affected whole availability zones or regions.
• NICTA has a subsidiary (Yuruware) with a non-
intrusive disaster recovery product (Bolt).
• Bolt copies data periodically to a back up region.
• Bolt utilizes sophisticated data movement
techniques to reduce time required to back up
• This is an insurance policy.
17
18. NICTA Copyright 2012 From imagination to impact
Next Problem – Upgrade
• Upgrades are a very common occurrence
• Upgrade frequency of some common systems
• Some systems have multiple releases per day,
driven by developers – continuous deployment
18
Application Average release interval
Facebook (platform) < 7 days
Google Docs <50 days
Media Wiki 21 days
Joomla 30 days
19. NICTA Copyright 2012 From imagination to impact
Various Upgrade Strategies
• How many at once?
– One at a time (rolling upgrade)
– Groups at a time (staged upgrade, e.g. canaries. This
is using production environment for testing)
– All at once (big flip)
• How long are new versions tested to determine
correctness?
– Period based – for some period of time
– Load based – under some utilization assumptions
• What happens to old versions?
– Replaced en masse
– Maintained for some period for compatibility purposes
19
20. NICTA Copyright 2012 From imagination to impact
Having Multiple Versions Simulaneously
Active May Lead to Mixed Version Race
Condition
20
Server 2 (new
version
3
4
X ERROR
Initial request
Client (browser)
Server 1 (old
version
1
2
5
Start rolling
upgrade
HTTP reply with
embedded JavaScript
AJAX callback
21. NICTA Copyright 2012 From imagination to impact
One Method for Preventing Mixed Version Race Condition
is to Make Load Balancers Version Aware
Client may
request
particular version
of a service
External facing
Router (wrt to
cloud)
Internal Router
Server for
Version A
Server for
Version A
Server for
Version B
Internal Router
Server for
Version A
Server for
Version B
21
At each level of the routing hierarchy
there are two possibilities for each
request
• Request is neutral with respect to
version
• Request specifies version
Routing must
• Be fast to ensure rapid response
• Satisfy “goodness” criteria for
scheduling
• Conform to client request wrt
version.
In addition:
• Servers are being
upgraded to a later
version while servicing
client requests
• Load variation may
trigger elasticity rules
22. NICTA Copyright 2012 From imagination to impact
What is Criterion for Measuring Load
Balancer Scheduling?
• What is “goodness” with respect to routing
decisions within the constraints of scheduling
strategy and version awareness?
– Uniform distribution of requests?
– Keeping utilization within bounds?
– Utilizing wide variety of clients?
– Other?
• Main result so far. Version awareness is
incompatible with any of the above “goodness”
criteria for the staged upgrade strategy.
22
23. NICTA Copyright 2012 From imagination to impact
Canary or Staged Strategy
• Upgrade one or several servers to new version
and leave them for some time.
• Formulation:
– Staged upgrade
• M version A servers (constant number)
• N version B servers (constant number)
• Fixed number of clients
– Version aware
• Once a client has had a request serviced by a version B
server it cannot subsequently have any requests serviced by
any version A server.
23
24. NICTA Copyright 2012 From imagination to impact
Bifurcation of Clients
• Clients are bifurcated into version A clients and
version B clients after some time
– Intuitively, for each client, either it is serviced by a
server with version B and consequently never served
by any server with version A or never served by a
server with version B. So each client ends in up the
Server A class or the Server B class but not both.
• We call clients that end up being serviced by
services with version A, class A clients.
Similarly, for class B clients.
• Allowing additional clients does not
fundamentally change result.
24
25. NICTA Copyright 2012 From imagination to impact
Bifurcation of Clients Implies
• Cannot control for utilization unless create new instances
of version B in response to demand
– There are a fixed number of clients sending requests to a fixed
number of servers with version B. Cannot vary the number of
servers to reflect the load generated by the fixed set of clients.
Consequently cannot control the utilization by servers with
version B.
• Cannot control for uniform distribution.
– Uniform distribution means that every request has an equal
change of being sent to any server. If a client is in class A, then it
has 0% chance of being sent to a server with Version B.
• Difficult to control for wide variety of clients.
– Variations among the clients must be mirrored within class A and
class B clients since the classes are fixed after the bifurcation.
This is difficult to accomplish since types of variations that are
important are usually not known.
25
26. NICTA Copyright 2012 From imagination to impact
Questions to Answer.
– How long does it take to reach bifurcated state under
what assumptions?
– How can the goals of staged upgrade be achieved
within the constraints of version awareness?
26
27. NICTA Copyright 2012 From imagination to impact
Next Problem
• Operators use scripts to perform actions such as
update
• Scripts may fail
– May be result of API failure (more on this later)
– May be desire to set up testing environment
– May be result of failure of underlying virtual machine.
• When a script fails, the operator may wish to
return to a known state (undo several
operations)
27
28. NICTA Copyright 2012 From imagination to impact
Operator Undo
• Not always that straight-forward:
– Attaching volume is no problem while the instance is
running, detaching might be problematic
– Creating / changing auto-scaling rules has effect on
number of running instances
• Cannot terminate additional instances, as the rule would
create new ones!
– Deleted / terminated / released resources are gone!
28
29. NICTA Copyright 2012 From imagination to impact
Undo for System Operators
29
+ commit
+ pseudo-delete
begin-
transaction
rollback
do
do
do
Administrator
30. NICTA Copyright 2012 From imagination to impact
Approach
30
begin-
transaction
rollback
do
do
do
Sense cloud
resources states
Sense cloud
resources states
Administrator
Undo System
31. NICTA Copyright 2012 From imagination to impact
Approach
31
begin-
transaction
rollback
do
do
do
Sense cloud
resources states
Sense cloud
resources states
Administrator
Undo System
Goal
state
Goal
state
Initial
state
Initial
state
32. NICTA Copyright 2012 From imagination to impact
begin-
transaction
rollback
do
do
do
Sense cloud
resources states
Sense cloud
resources states
PlanGenerate codeExecute
Administrator
Undo System
Goal
state
Goal
state
Initial
state
Initial
state
Set of
actions
Set of
actions
Approach
32
33. NICTA Copyright 2012 From imagination to impact
What about API Failures?
• Operator scripts make heavy use of checking or
controlling state of resources
– Start/stop VM
– Is VM active?
• These scripts becomes calls to the cloud
provider’s API.
• Calls may fail
– Underlying VM has failed
– Eventual consistency.
33
34. NICTA Copyright 2012 From imagination to impact
We Have Performed an Empirical Study of
API Failures in EC2
• 922 cases out of 1109 reported API-related
cases in the EC2 forum from 2010 to 2012 are
API failures (rather than feature requests or
general inquiries).
• We classified the extracted API failures into four
types of failures:
– content failures,
– late timing failures,
– halt failures, and
– erratic failures.
34
35. NICTA Copyright 2012 From imagination to impact
Results
• A majority (60%) of the cases of API failure are related to
stuck API calls or unresponsive API calls.
• A large portion (12%) of the cases are about slow
responsive API calls.
• 19% of the cases are related to the output issues of API
calls, including failed calls with unclear error
messages, as well as missing output, wrong output, and
unexpected output of API calls.
• 9% of the cases reported that their calls were pending
for a certain time and then returned to the original state
without informing the caller properly or the calls were
reported to be successful first but failed later.
35
36. NICTA Copyright 2012 From imagination to impact
Next Problem - Operations Processes
• We are looking at the process of installing new
software
– Error Prone
– Potential process improvements.
36
37. NICTA Copyright 2012 From imagination to impact
Motivating Scenario
• You change the operating environment for an
application
– Configuration change
– Version change
– Hardware change
• Result is degraded performance
• When the software stack is deep with portions
from different suppliers, the result is frequently:
37
38. NICTA Copyright 2012 From imagination to impact
Why is Installation Error Prone?
• Installation is complicated.
– Installation guides for SAS 9.3 Intelligence, IBM i, Oracle 11g for
Linux are ~250 pages each
– Apache description of addresses and ports (one out of 16
descriptions) has following elements:
• Choosing and specifying ports for the server to listen to
• IPv4 and IPv6
• Protocols
• Virtual Hosts
– The number of configuration options that must be set can be
large
• Hadoop has 206 options
• HBase has 64
– Many dependencies are not visible until execution
38
39. NICTA Copyright 2012 From imagination to impact
Installation Processes
• Processes may be
– Undocumented
– Out of date
– Insufficiently detailed
• Our goal is to build process model including
error recovery mechanisms
39
40. NICTA Copyright 2012 From imagination to impact
Our Activities
40
• Create up to date process models for installation
processes. Information sources are
– Process discovery from logs
– Process formalization from existing written
descriptions.
• Process descriptions can be used to
– Make trade offs
– Make recommendations in real time to operations
staff
– Recommend setting checkpoints for potential later
undo, before a risky part of a process is entered
– Assist in the detection of errors
41. NICTA Copyright 2012 From imagination to impact
Hard Problems
41
• Creating accurate process models
– Exception handling mechanisms are not well
documented
– Labor intensive.
– Our approach
• Top down modeling using process modeling formalism
• Bottom up process mining from error logs
• Diagnosing errors
42. NICTA Copyright 2012 From imagination to impact
Why is Error Diagnosis Hard?
In a distributed computing
environment, when an error
occurs during operations, it is
difficult and time consuming to
diagnosis it.
Diagnosis involves correlating
messages from
• different distributed servers
• different portions of the
software stack
and determining the root
cause of the error.
The root cause, in turn, may
be within a portion of the stack
that is different from where the
error is observed.
43. NICTA Copyright 2012 From imagination to impact
Test Bed
43
Our current test bed is the Hbase stack
44. NICTA Copyright 2012 From imagination to impact
Currently Performing Analysis of
Configuration Errors
44
• Cross stack errors may take hours to diagnose
– Log files are inconsistent
– Error message may not give context necessary to
determine root cause.
45. NICTA Copyright 2012 From imagination to impact
Where to Find Information about Operations
Domain?
• Every open source program requires a variety of
configuration parameters.
• Every modern application depends on a variety
of middleware so cross domain examples should
be readily available.
• Most organizations have extensive processes for
their operations personnel. Use these processes
as a framework for investigating process/product
interactions.
45
46. NICTA Copyright 2012 From imagination to impact
Summary
• Operations problems will account for the majority
of outages and IT costs in the next several
years.
• The operations space is a rich source of
research problems that has been insufficiently
mined.
• Best way to determine what problems to attack
is to monitor or interview operators
46
47. NICTA Copyright 2012 From imagination to impact
NICTA Team
• Anna Liu
• Alan Fekete
• Min Fu
• Jim Zhanwen Li
• Qinghua Lu
• Hiroshi Wada
• Ingo Weber
• Xiwei Xu
• Liming Zhu
47