The document discusses making code more testable by addressing issues like dependency spaghetti, lack of unit tests, and no business layer. It recommends starting with small wins like adding new unit tests and an automated deployment process. This helps initiate change and saves significant time compared to manual deployments. However, the underlying quality problems still remained, like null pointer exceptions, showing quality cannot be fully tested in and the dependency/design issues need addressed.
Continuous Delivery refers to the process of releasing high quality software quickly and with confidence through the use of build, test and deployment automation. By applying Lean techniques to the development, test and deployment of software, waste is reduced and staff are freed up to work on more important tasks. By following a continuous delivery model, release cycles shift from a matter of months to weeks or days.
In this presentation, we will look at the key tools and processes involved in transitioning from a manual culture to one that embraces automation. We will look at real world examples, including the tools and architectural components. We will discuss organizational impacts, including the dramatic improvements in morale as team delivery commitments are met more easily through automation.
The document discusses introductory concepts and best practices for unit testing, including:
1. Why unit tests are important for reducing bugs, improving design, and making development faster.
2. The different types of tests like unit tests, integration tests, and functional tests.
3. What constitutes a unit test, including that it is small, repeatable, consistent, and fast.
4. An example of writing a "Hello World" unit test in C#.
1) The document summarizes a Jenkins meetup about the XTrigger, EnvInject, and DryRun plugins.
2) The presenter is Grégory BOISSINOT and discusses how the plugins can be used to trigger jobs, inject environment variables, and perform dry runs of builds.
3) Examples and demos of how to configure the plugins are provided.
Lasse Koskela presented on optimizing build times. He explained that delayed feedback is bad and can cost companies money when bugs are found later in the process. He demonstrated profiling builds to identify bottlenecks, such as tests that do unnecessary setup, build inputs slowly, or repeat work. Tests can also be slow if they use real objects instead of stubs. The presentation provided examples of optimizing test code and infrastructure. Infrastructure tweaks included using multiple CPUs, faster hardware, or the cloud to lift CPU or I/O bottlenecks.
Resilience Engineering: A field of study, a community, and some perspective s...John Allspaw
These are slides from my talk on March 28, 2018 at the LA SCALE tech Meetup, graciously hosted at TicketMaster's office. (https://www.meetup.com/scalela/events/248904126/)
Agile & DevOps - It's all about project successAdam Stephensen
The document provides information on DevOps practices and tools from Microsoft. It discusses how DevOps enables continuous delivery of value through integrating people, processes, and tools. Benefits of DevOps include more frequent and stable releases, lower change failure rates, and empowered development teams. The document provides examples of DevOps scenarios and recommends discussing solutions and migration plans with Microsoft.
The document discusses features of Nectar, CloudBees' proprietary version of Jenkins, including:
- Handling large numbers of jobs through folders and job templating.
- Scaling jobs using the VMware auto-scale plugin to leverage existing VM infrastructure.
- Security features like role-based access control and filtering.
- Backup using the backup plugin.
- Other plugins like wiki text formatting.
The webinar provides demonstrations of these features and discusses support offered through CloudBees.
The document summarizes a performance analysis study of a large enterprise application called Zanzibar. The researchers initially struggled to get the application to scale even on modest hardware. Through analyzing javacores, they identified and addressed multiple bottlenecks, including lock contention, disk throughput issues, and inefficient Java code. These changes resulted in performance improvements of 1.4-5x. The researchers also developed the WAIT performance tool to help identify primary bottlenecks in deployed applications with low overhead. The tool uses standard OS and JVM data and has seen widespread adoption due to its ease of use.
Continuous Delivery refers to the process of releasing high quality software quickly and with confidence through the use of build, test and deployment automation. By applying Lean techniques to the development, test and deployment of software, waste is reduced and staff are freed up to work on more important tasks. By following a continuous delivery model, release cycles shift from a matter of months to weeks or days.
In this presentation, we will look at the key tools and processes involved in transitioning from a manual culture to one that embraces automation. We will look at real world examples, including the tools and architectural components. We will discuss organizational impacts, including the dramatic improvements in morale as team delivery commitments are met more easily through automation.
The document discusses introductory concepts and best practices for unit testing, including:
1. Why unit tests are important for reducing bugs, improving design, and making development faster.
2. The different types of tests like unit tests, integration tests, and functional tests.
3. What constitutes a unit test, including that it is small, repeatable, consistent, and fast.
4. An example of writing a "Hello World" unit test in C#.
1) The document summarizes a Jenkins meetup about the XTrigger, EnvInject, and DryRun plugins.
2) The presenter is Grégory BOISSINOT and discusses how the plugins can be used to trigger jobs, inject environment variables, and perform dry runs of builds.
3) Examples and demos of how to configure the plugins are provided.
Lasse Koskela presented on optimizing build times. He explained that delayed feedback is bad and can cost companies money when bugs are found later in the process. He demonstrated profiling builds to identify bottlenecks, such as tests that do unnecessary setup, build inputs slowly, or repeat work. Tests can also be slow if they use real objects instead of stubs. The presentation provided examples of optimizing test code and infrastructure. Infrastructure tweaks included using multiple CPUs, faster hardware, or the cloud to lift CPU or I/O bottlenecks.
Resilience Engineering: A field of study, a community, and some perspective s...John Allspaw
These are slides from my talk on March 28, 2018 at the LA SCALE tech Meetup, graciously hosted at TicketMaster's office. (https://www.meetup.com/scalela/events/248904126/)
Agile & DevOps - It's all about project successAdam Stephensen
The document provides information on DevOps practices and tools from Microsoft. It discusses how DevOps enables continuous delivery of value through integrating people, processes, and tools. Benefits of DevOps include more frequent and stable releases, lower change failure rates, and empowered development teams. The document provides examples of DevOps scenarios and recommends discussing solutions and migration plans with Microsoft.
The document discusses features of Nectar, CloudBees' proprietary version of Jenkins, including:
- Handling large numbers of jobs through folders and job templating.
- Scaling jobs using the VMware auto-scale plugin to leverage existing VM infrastructure.
- Security features like role-based access control and filtering.
- Backup using the backup plugin.
- Other plugins like wiki text formatting.
The webinar provides demonstrations of these features and discusses support offered through CloudBees.
The document summarizes a performance analysis study of a large enterprise application called Zanzibar. The researchers initially struggled to get the application to scale even on modest hardware. Through analyzing javacores, they identified and addressed multiple bottlenecks, including lock contention, disk throughput issues, and inefficient Java code. These changes resulted in performance improvements of 1.4-5x. The researchers also developed the WAIT performance tool to help identify primary bottlenecks in deployed applications with low overhead. The tool uses standard OS and JVM data and has seen widespread adoption due to its ease of use.
The document discusses efficient verification methodology. It recommends defining a conceptual framework or methodology to standardize some aspects while allowing diversity. The methodology should define interfaces and transactions upfront using an interface definition language to generate verification components and reusable assertions. It also recommends modeling systems at the transaction level using executable specifications to frontload the verification schedule.
The document discusses research into what makes a good bug report based on a survey of over 150 developers. It finds that the most helpful items for fixing bugs are steps to reproduce, stack traces, and observed behavior. The biggest problems causing delays are incomplete information, wrong steps to reproduce, and wrong expected behavior. The research also measured bug report quality and found stack traces and readability correlated with shorter fix times.
This document discusses agile testing practices that help mitigate technical risk. It describes eight key testing practices in agile: acceptance test driven development, test driven development, automated system tests, automated unit tests, exploratory testing, continuous integration, collective test ownership, and rehearsing delivery. These practices like ATDD, TDD, and continuous integration help reduce ambiguity, dependencies, assumptions, and risks to capacity in agile software development.
This document discusses common primitives needed in Docker environments when deploying applications across multiple machines. It outlines several key primitives like persistence, service discovery, monitoring, logging, authentication and authorization. For each primitive, it describes different approaches, lists relevant open source projects, and provides recommendations on how to implement the primitive in a way that is standardized, scalable and works for both new and legacy applications. The goal is to abstract infrastructure and treat all machines similarly while achieving reliability, reproducibility and reducing manual labor.
Continuous Delivery in the real world - techniques to reduce the developers b...Nikolai Blackie
Do you suffer from elevated stress and high blood pressure from your current software release cycles? Lost count of the 2am troubleshooting sessions trying to working out why production didn’t deploy like test? In this session you will see real life continuous delivery strategies from the field, learn some implementation techniques and demonstrations of a few tools that will assist in reducing the headache of manual software delivery.
CloudBees has acquired Stax Networks to provide an integrated platform for the full application lifecycle in the cloud. The combined offering of Dev@Cloud and Run@Cloud will allow developers to develop, build, test, and deploy applications to production in the cloud. Stax's existing Platform as a Service will be integrated into Run@Cloud, which will be generally available in January 2011. This will provide an end-to-end cloud solution for Java applications from development through deployment.
Q con shanghai2013-[黄舒泉]-[intel it openstack practice]Michael Zhang
This document summarizes an Intel IT presentation on their OpenStack practice. It discusses Intel's contributions to OpenStack projects, their converged OpenStack and IT platform, and their solutions for continuous delivery, deployment, Tempest testing automation, and improving the speed of OpenStack snapshots.
Creator: Joel Champagne, President of CodeX Enterprises
This presentation covers various issues associated with SQL unit testing. We’ll look at end solutions in demo form using Visual Studio 2010 and other third party tools.
You'll learn:
* The value of pursuing SQL testing, early and continually in the development cycle
* Capabilities in Visual Studio 2010 to support SQL unit testing
* Capabilities in other tools to support SQL unit testing
Nuxeo on the Cloud discusses running the Nuxeo content management platform on cloud infrastructure. It describes how Nuxeo can be deployed as software-as-a-service (SaaS) on Amazon EC2, or on infrastructure-as-a-service (IaaS) using tools like CloudFormation scripts. It also explores deploying Nuxeo on platform-as-a-service (PaaS) options like CloudFoundry, and focuses on adapting Nuxeo features to run on this type of platform. Finally, it mentions plans to offer Nuxeo on more IaaS providers and private clouds like OpenStack, as well as developing a multi-tenant PaaS specifically for
This document discusses barriers to integration testing and introduces Arquillian and ShrinkWrap as tools to help address those barriers. It describes how Arquillian handles container lifecycles and test deployment, allowing tests to focus on logic. ShrinkWrap provides a fluent API for programmatically creating deployment archives. The presentation provides an overview of their capabilities and benefits, such as running tests directly in containers without full application builds. It also outlines future plans like additional container and framework support. Attendees are encouraged to get involved in the open source projects.
This document provides instructions for uploading different file types to Slide Share for Group 2. Users can upload PowerPoint presentations, documents, and videos to the site. The document lists the options to upload these common file formats for sharing content on Slide Share.
This document discusses best practices for running a production Jenkins installation, including preparing for failures, running efficiently, and monitoring performance. It covers backing up the Jenkins configuration, setting up high availability, optimizing plugins usage, and tools for thread dumps and heap analysis when issues arise. The overall goal is helping administrators eliminate downtime and improve efficiency.
Agile Open Source Performance Test Workshop for Developers, Testers, IT OpsClever Moe
Training For Selenium, soapUI, Sahi, TestMaker Performance Testing. Slide deck from the free Webinar titled "Technical Training On The Agile Open Source Way To Load Test, Scalability Test, and Stress Test." Learn the Agile Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
Webinar: Zing Vision: Answering your toughest production Java performance que...Azul Systems Inc.
Solving Java performance issues in production can be frustrating. You’re left in the dark about what could be causing the problems because standard Java tools have too much performance overhead for production use. They’re designed for development or pre-production testing and realistically can’t be used to monitor a business-critical application during peak loads, which is when the problems occur!
Zing Vision is your flashlight. Its low overhead metric collection is built into Zing, Azul’s high performance virtual machine, and designed to run in production with zero performance overhead. At last, you can see your applications’ operation at the thread level, track memory usage, find “hot” code and even save data for later analysis. In this webinar, Joseph Coha, Azul Systems Senior Staff Engineer, describes how Zing Vision works, shows sample data and discusses how you can use this information to find and fix your most stubborn production performance issues. He also tells you how you can download and try Zing and Zing Vision with your current applications to see for yourself how far you can take the performance of your existing apps.
JRebel is a tool that allows Java developers to see changes to code instantly without redeploying applications. It eliminates lengthy redeploys, allowing developers to test incremental changes quickly and spend more time on coding, debugging, and collaborating. JRebel supports over 90 frameworks and allows remote debugging and deploying code changes to servers from a local machine. By removing redeploys, JRebel helps developers work more efficiently and productively.
The document provides an overview of key concepts in Apache Maven, a build automation tool. It discusses that Maven uses conventions over configuration, relies on a pom.xml file to define a project, supports multi-module projects, and executes builds through a hierarchy of lifecycles, phases, plugins, and goals. It also describes common Maven concepts like dependencies, repositories, profiles, and plugins.
Haj 4344-java se 9 and the application server-1Kevin Sutter
Our presentation at InterConnect 2017 about Java SE 9 and our direct experiences with using it as our Java runtime for the WebSphere Liberty application server.
The document discusses strategies for continuous delivery through parallel development and continuous integration, including maintaining feature branches, release branches, and a main development branch. It also outlines the general development workflow and processes for building, deploying, acceptance testing, releasing, and pushing updates to production through automated deployment. The goal is to enable one-click software updates and releases at any time through establishing testing, integration, and deployment best practices.
This is a 90 min talk with some exercises and discussion that I gave at the DHS Agile Expo. It places DevOps as a series of feedback loops and emphasizes agile engineering practices being at the core.
Deliver Faster with BDD/TDD - Designing Automated Tests That Don't SuckKevin Brockhoff
Kevin Brockhoff presented on accelerating software delivery through test-driven development (TDD) and behavior-driven development (BDD). TDD and BDD help identify incorrect behavior early, reduce rework, increase code confidence through automated tests, and improve code flexibility and extensibility. Brockhoff discussed key software delivery metrics like deployment frequency and change failure rate. He also covered testing best practices like the testing pyramid, unit testing frameworks, test source layout, and refactoring for testability.
XebiaLabs, CloudBees, Puppet Labs Webinar Slides - IT Automation for the Mode...XebiaLabs
Learn how you can enhance and extend your existing infrastructure to create an automated, end-to-end IT platform supporting on-demand middleware and application environments, application release pipelines, Continuous Delivery, Private/ hybrid development platform and PaaS and more.
The document discusses efficient verification methodology. It recommends defining a conceptual framework or methodology to standardize some aspects while allowing diversity. The methodology should define interfaces and transactions upfront using an interface definition language to generate verification components and reusable assertions. It also recommends modeling systems at the transaction level using executable specifications to frontload the verification schedule.
The document discusses research into what makes a good bug report based on a survey of over 150 developers. It finds that the most helpful items for fixing bugs are steps to reproduce, stack traces, and observed behavior. The biggest problems causing delays are incomplete information, wrong steps to reproduce, and wrong expected behavior. The research also measured bug report quality and found stack traces and readability correlated with shorter fix times.
This document discusses agile testing practices that help mitigate technical risk. It describes eight key testing practices in agile: acceptance test driven development, test driven development, automated system tests, automated unit tests, exploratory testing, continuous integration, collective test ownership, and rehearsing delivery. These practices like ATDD, TDD, and continuous integration help reduce ambiguity, dependencies, assumptions, and risks to capacity in agile software development.
This document discusses common primitives needed in Docker environments when deploying applications across multiple machines. It outlines several key primitives like persistence, service discovery, monitoring, logging, authentication and authorization. For each primitive, it describes different approaches, lists relevant open source projects, and provides recommendations on how to implement the primitive in a way that is standardized, scalable and works for both new and legacy applications. The goal is to abstract infrastructure and treat all machines similarly while achieving reliability, reproducibility and reducing manual labor.
Continuous Delivery in the real world - techniques to reduce the developers b...Nikolai Blackie
Do you suffer from elevated stress and high blood pressure from your current software release cycles? Lost count of the 2am troubleshooting sessions trying to working out why production didn’t deploy like test? In this session you will see real life continuous delivery strategies from the field, learn some implementation techniques and demonstrations of a few tools that will assist in reducing the headache of manual software delivery.
CloudBees has acquired Stax Networks to provide an integrated platform for the full application lifecycle in the cloud. The combined offering of Dev@Cloud and Run@Cloud will allow developers to develop, build, test, and deploy applications to production in the cloud. Stax's existing Platform as a Service will be integrated into Run@Cloud, which will be generally available in January 2011. This will provide an end-to-end cloud solution for Java applications from development through deployment.
Q con shanghai2013-[黄舒泉]-[intel it openstack practice]Michael Zhang
This document summarizes an Intel IT presentation on their OpenStack practice. It discusses Intel's contributions to OpenStack projects, their converged OpenStack and IT platform, and their solutions for continuous delivery, deployment, Tempest testing automation, and improving the speed of OpenStack snapshots.
Creator: Joel Champagne, President of CodeX Enterprises
This presentation covers various issues associated with SQL unit testing. We’ll look at end solutions in demo form using Visual Studio 2010 and other third party tools.
You'll learn:
* The value of pursuing SQL testing, early and continually in the development cycle
* Capabilities in Visual Studio 2010 to support SQL unit testing
* Capabilities in other tools to support SQL unit testing
Nuxeo on the Cloud discusses running the Nuxeo content management platform on cloud infrastructure. It describes how Nuxeo can be deployed as software-as-a-service (SaaS) on Amazon EC2, or on infrastructure-as-a-service (IaaS) using tools like CloudFormation scripts. It also explores deploying Nuxeo on platform-as-a-service (PaaS) options like CloudFoundry, and focuses on adapting Nuxeo features to run on this type of platform. Finally, it mentions plans to offer Nuxeo on more IaaS providers and private clouds like OpenStack, as well as developing a multi-tenant PaaS specifically for
This document discusses barriers to integration testing and introduces Arquillian and ShrinkWrap as tools to help address those barriers. It describes how Arquillian handles container lifecycles and test deployment, allowing tests to focus on logic. ShrinkWrap provides a fluent API for programmatically creating deployment archives. The presentation provides an overview of their capabilities and benefits, such as running tests directly in containers without full application builds. It also outlines future plans like additional container and framework support. Attendees are encouraged to get involved in the open source projects.
This document provides instructions for uploading different file types to Slide Share for Group 2. Users can upload PowerPoint presentations, documents, and videos to the site. The document lists the options to upload these common file formats for sharing content on Slide Share.
This document discusses best practices for running a production Jenkins installation, including preparing for failures, running efficiently, and monitoring performance. It covers backing up the Jenkins configuration, setting up high availability, optimizing plugins usage, and tools for thread dumps and heap analysis when issues arise. The overall goal is helping administrators eliminate downtime and improve efficiency.
Agile Open Source Performance Test Workshop for Developers, Testers, IT OpsClever Moe
Training For Selenium, soapUI, Sahi, TestMaker Performance Testing. Slide deck from the free Webinar titled "Technical Training On The Agile Open Source Way To Load Test, Scalability Test, and Stress Test." Learn the Agile Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
Webinar: Zing Vision: Answering your toughest production Java performance que...Azul Systems Inc.
Solving Java performance issues in production can be frustrating. You’re left in the dark about what could be causing the problems because standard Java tools have too much performance overhead for production use. They’re designed for development or pre-production testing and realistically can’t be used to monitor a business-critical application during peak loads, which is when the problems occur!
Zing Vision is your flashlight. Its low overhead metric collection is built into Zing, Azul’s high performance virtual machine, and designed to run in production with zero performance overhead. At last, you can see your applications’ operation at the thread level, track memory usage, find “hot” code and even save data for later analysis. In this webinar, Joseph Coha, Azul Systems Senior Staff Engineer, describes how Zing Vision works, shows sample data and discusses how you can use this information to find and fix your most stubborn production performance issues. He also tells you how you can download and try Zing and Zing Vision with your current applications to see for yourself how far you can take the performance of your existing apps.
JRebel is a tool that allows Java developers to see changes to code instantly without redeploying applications. It eliminates lengthy redeploys, allowing developers to test incremental changes quickly and spend more time on coding, debugging, and collaborating. JRebel supports over 90 frameworks and allows remote debugging and deploying code changes to servers from a local machine. By removing redeploys, JRebel helps developers work more efficiently and productively.
The document provides an overview of key concepts in Apache Maven, a build automation tool. It discusses that Maven uses conventions over configuration, relies on a pom.xml file to define a project, supports multi-module projects, and executes builds through a hierarchy of lifecycles, phases, plugins, and goals. It also describes common Maven concepts like dependencies, repositories, profiles, and plugins.
Haj 4344-java se 9 and the application server-1Kevin Sutter
Our presentation at InterConnect 2017 about Java SE 9 and our direct experiences with using it as our Java runtime for the WebSphere Liberty application server.
The document discusses strategies for continuous delivery through parallel development and continuous integration, including maintaining feature branches, release branches, and a main development branch. It also outlines the general development workflow and processes for building, deploying, acceptance testing, releasing, and pushing updates to production through automated deployment. The goal is to enable one-click software updates and releases at any time through establishing testing, integration, and deployment best practices.
This is a 90 min talk with some exercises and discussion that I gave at the DHS Agile Expo. It places DevOps as a series of feedback loops and emphasizes agile engineering practices being at the core.
Deliver Faster with BDD/TDD - Designing Automated Tests That Don't SuckKevin Brockhoff
Kevin Brockhoff presented on accelerating software delivery through test-driven development (TDD) and behavior-driven development (BDD). TDD and BDD help identify incorrect behavior early, reduce rework, increase code confidence through automated tests, and improve code flexibility and extensibility. Brockhoff discussed key software delivery metrics like deployment frequency and change failure rate. He also covered testing best practices like the testing pyramid, unit testing frameworks, test source layout, and refactoring for testability.
XebiaLabs, CloudBees, Puppet Labs Webinar Slides - IT Automation for the Mode...XebiaLabs
Learn how you can enhance and extend your existing infrastructure to create an automated, end-to-end IT platform supporting on-demand middleware and application environments, application release pipelines, Continuous Delivery, Private/ hybrid development platform and PaaS and more.
Five Ways Automation Has Increased Application Deployment and Changed CultureXebiaLabs
Paychex, a recognized leader in the payroll, human resource, and benefits outsourcing industry, found that the demand for application deployments had increased beyond what could be supported by manual configuration. Keeping up with this demand required a shift from manually providing a service to developing an automated platform for self-service resulting in a culture change with new partnering across their DEV, OPS and Architecture teams.
David Jozis, Automation Engineer at Paychex, discusses the challenges they encountered when making these significant changes and how they were able to overcome them to accomplish 5x as many deployments as before.
The document discusses lessons learned in building robust QA infrastructure for commercial software. Key points include:
1) QA infrastructure is as important as the application itself and must be developed in parallel, not as an afterthought. QA and development teams should work closely together.
2) The build, source control, and test systems form an integrated distributed infrastructure that should allow development and testing from any location.
3) Tests should be written to easily grow over time, from simple unit tests to complex tests that export application states. An automated continuous testing system is critical.
Automated testing DrupalCamp in AshevillePromet Source
Promet Solutions specializes in developing complex web applications using open source technologies. They have over 30 solutions architects and have been in business for 10 years, contributing to over 32 open source projects. They have offices in Chicago and San Jose and have developed over 10 mobile apps in the past 12 months. They are presenting on their experience with automated testing for Drupal applications using tools like Selenium and PHPUnit.
Test-Driven Infrastructure with CloudFormation and Cucumber. Stelligent
The document discusses test-driven infrastructure using CloudFormation and Cucumber. It recommends writing automated tests with Cucumber to verify infrastructure provisioned by CloudFormation templates, versioning infrastructure scripts and configurations, and integrating infrastructure testing into continuous delivery pipelines. Examples are provided of using Cucumber features and step definitions to test CloudFormation templates for provisioning AWS resources.
Software functional testing can unveil a wide range of potential malfunctions in applications. However, there is a significant fraction of errors that will be hardly detected through a traditional testing process. Problems such as memory corruptions, memory leaks, performance bottlenecks, low-level system call failures and I/O errors might not surface any symptoms in a tester’s machine while causing disasters in production. On the other hand, many handy tools have been emerging in all popular platforms allowing a tester or an analyst to monitor the behavior of an application with respect to these dark areas in order to identify potential fatal problems that would go unnoticed otherwise. Unfortunately, these tools are not yet in widespread use due to few reasons. First, the usage of tools requires a certain amount of expertise on system internals. Furthermore, these monitoring tools generate a vast amount of data even with elegant filtering and thereby demand a significant amount of time for an analysis even from experts. As the end result, using monitoring tools to improve software quality becomes a costly operation. Another facet of this problem is the lack of infrastructure to automate recurring analysis patterns.
This paper describes the current state of an ongoing research in developing a framework that automates a significant part of the process of monitoring various quality aspects of a software application with the utilization of tools and deriving conclusions based on results. According to our knowledge this is the first framework to do this. It formulates infrastructure for analysts to extract relevant data from monitoring tool logs, process those data, make inferences and present analysis results to a wide range of stakeholders in a project.
The document discusses best practices for database development and deployment. It recommends having identical environments for development, testing, and production to enable easy comparisons. This allows issues to be detected and fixed before production deployment. It also suggests using tools that track database changes and compare schemas to simplify environments and ensure consistency across stages. Regular practice deployments in non-production environments are advised to work out any issues before changes reach production.
Test driven development for infrastructure as-a-code, the future trend_Gianfr...Katherine Golovinova
This document discusses testing infrastructure as code (IaaC) using test-driven development (TDD) principles. It recommends applying different types of automated tests for IaaC: unit tests to check for errors, integration tests to validate functionality, and acceptance/security tests on deployed infrastructure. Various tools are mentioned for testing IaaC written in languages like Terraform, Ansible, Chef, and Puppet at the unit, integration, and security levels. Adopting a testing mindset and tools can help catch errors and non-compliance early in development pipelines.
The document discusses using a service-oriented architecture and RESTful web services to build scalable systems. It describes a case study of a telecommunications company that needed to scale from handling 1 million to 1 billion messages per month. The initial enterprise architecture resulted in bottlenecks and reliability issues. By taking inspiration from web-scale companies and focusing on delivery of individual services, a new storage manager service and compute grid were implemented. This improved scalability, reduced costs, and delivered business value by keeping customers happy with a more reliable system.
DSL, Page Object и WebDriver – путь к надежным функциональным тестамSQALab
DSL, Page Object and WebDriver patterns help address issues with Selenium tests such as verbosity, lack of expressiveness, fragility, and lack of reuse. The presentation recommends a three step approach:
1. Introduce a DSL to create high-level, readable tests in the language of the application domain.
2. Implement the Page Object pattern to separate test logic from page details through page maps.
3. Divide pages into reusable elements to increase code reuse and simplify page objects.
This approach results in reliable, clear, and maintainable tests that better model the application under test.
Easy Java Integration Testing with TestcontainersPayara
Having problems creating a maintainable set of integration tests for your enterprise Java application? This talk will help you understand how to use Test containers to implement easy integration tests for your next project!
Simplifying Integration Tests for Enterprise Java
Integration testing is always a difficult area. You need to make sure that all system dependencies are prepared, data is correctly initialised for each run and test runs do not interfere with each other. Even with tools like the Arquillian Framework, writing integration tests can be a complicated task and the act of maintaining large sets of tests can become a nightmare if there's not enough knowledge of all dependencies involved.
With the Testcontainers project, convoluted and complicated integration tests can be a thing of the past. A test container allows you to create reliable integration tests covering a wide range of scenarios like using databases, microservices interactions and even GUI and user experience testing. One of the important factors for the success of this framework is the usage of Docker containers to create a reliable reproducible environment for each test, which separates the dependencies and the application code in a way that it is easy to maintain for developers.
Releasing fast code - The DevOps approachMichael Kopp
Agile makes you Develop faster, DevOps also makes you Deploy faster but how do you make your Application faster?
Many currently used Performance Management practices don’t work anymore as they are too time consuming. It takes a new approach to track performance in Continuous Integration, get more value out of Load Testing and leverage production data for performance optimization.
We will show you real world examples on how the new DevOps approach can work.
DSL, Page Object and WebDriver – the path to reliable functional tests.pptxMikalai Alimenkou
Presentation from 10th SQADays conference in Moscow (December 2011) about different test design approaches to make functional tests on WebDriver more flexible, reliable and stable.
SOASTA's tens-of-thousands of tests in customer labs and production environments have uncovered issues that range from code level bugs to issues in 3rd party services. Testing early, often and at real scale is the only way to be fully prepared.
Karthik Gaekwad is a member of the cloud team at National Instruments who owns the Canopy user management and licensing platform. He discusses National Instrument's approach to cloud development which includes short monthly iterations to incrementally develop and deploy new features. Key aspects of their approach include modeling the end-to-end system, designing features to be reusable across platforms, extensive testing and monitoring, and getting early user feedback through demos.
IT Agility: Rapid Application Development and Testing in the Cloud with CQ5 O...Day Software
This document discusses how to achieve IT agility through rapid application development and testing in the cloud using Adobe Experience Manager (AEM) 5.3. It promotes frequent iterations, early user validation through prototyping, and teamwork. It then describes features of AEM 5.3 like the Site Importer, CRXDE Lite content editor, Selenium test framework, and cloud deployment that help enable faster development and testing cycles.
Similar to Testability for developers – Fighting a mess by making it testable (20)
This presentation explains parameterized tests, theory tests, and generative testing. It also explains single mode faults and double mode faults and shows how to reduce the number of test cases when there's an combinatorial explosion. Lot's of JUnit examples.
Presentation given for the SweNug user group. Based on the contents of my book "Developer Testing". Covers a variety of test related stuff: defining testability, test techniques, anti-testability constructs, duplication, test-driven development.
Agile Testing – Is it for me and how are others doing it?Alexander Tarlinder
This is a 45 minute introduction to agile testing. The slides are "speaking slides", which means that they should be accompanied by an oral presentation and may not be all that easy to comprehend without context.
Being good at waiting - Using Selenium to test Ajax-intensive pages Alexander Tarlinder
This document discusses strategies for waiting for Ajax calls to complete when testing dynamic web pages using Selenium. It explains that the naive approach of hardcoded sleeps does not work well. Instead, it recommends using implicit waits to ignore Ajax timing inconsistencies or explicit waits via WebDriverWait when waiting for a specific element to appear. The key is to avoid sleeps and instead wait for an observable element or condition.
2. Additional note:
This presentation has been slightly adjusted after
the conference. Some images have been removed
and some notes have been added to provide
context.
3. Ongoing project
alexander.tarnowski@crisp.se
alexander_tar
Blog blog.crisp.se/author/alexandertarnowski
www www.techbookreader.com
http://leanpub.com/developer_testing
4. My system’s dependencies are like spaghetti
My system is ruled by a database
It’s a ritual to deploy my system
My system doesn’t have a business layer
My system has few and poor unit tests
My system’s end-to-end tests don’t work
5. Frontend Legend
EWC = Enterprise-wide componment
Front
= Depends on
end = Packaged into
WS developer pack from 2004
”Enterprise-wide” component Backend
EWC Back
core end
core
EWC Back
shared end
shared
Enterprise message bus library
Joker: flickr: GoShows
6. A number of
data sources
Alternative message queue
implementation Application’s
config files
?
Parts of the
application
Message bus
library
flickr: Andrew Coulter Enright
7. One Database to rule them all, One Database to find them,
One Database to bring them all and in the darkness bind them
8. Additional note:
A lengthy manual deployment
script that’s not up-to-date has a
striking resemblance to
hieroglyphs.
flickr: rfzappala
9. Good idea Bad idea
Database
Additional note:
Some frameworks try really hard
to provide persistence as close to
the view as possible. When is that
a good idea?
10.
11. Είχαν τεςςαρϊν μεταγλωτίςει ασ ςαν, αλφα εκτελείται δεν τι. Από κα χρόνου χαμθλόσ ποςοςτό. Γειτονιάσ εργαηόμενων να πεσ
επενδυτισ το. Χάοσ προςεκτικά δε ςτθ. Όρο πάρα ιδιαίτερα αν, όςο το πετοφν γνωρίηει.
Το όςο γραφικά δθμιουργίεσ χαρακτθριςτικό, όλα μθ ζργων περίπου. Διακοπι οζλεγχοσ χαρακτθριςτικϊν να όχι, μθ Ήδθ ζτςι
Thread.sleep(3000);
χρόνου κα. Στο με όταν λοιπόν ςθμαντικόσ, ζρκει ςχεδιαςτισ ιδθ τι. Ασ τθν γειτονιάσ αγοράηοντασ. Ασ πετοφν αποτελζςει κα μπουν
προςλάμβανεσ, πιο ωσ ςτθν προςπακοφν.
Thread.sleep(3000);
Πω δουλζψει προςεκτικά χαρακτθριςτικϊν ςου, ζςτελνε φακζλουσ ανακλφψεισ τι νζα. Ατόμου αποφάςιςε που τι, προςλάμβανεσ
των, ποςοςτό επιτίκενται το ναι. Πω ερωτιςεισ χρειάηονται λεσ, βιμα πάρεισ κρατάει να κλπ. Τθν ξεχειλίηει. Τθν πετάνε χαμθλόσ
χαρακτθριςτικϊν να, δε κρατιςουν προβλθματικι μθν. Τθ κφκλο κανείσ απίςτευτα ϊρα, αλγόρικμου προβλιματα αντιλιφκθκαν.
Υόρκθ καρζκλα λιγότερο να ότι, τότε μπουν κρατάει πιο οι. Τι ςου γραφικά αρπάηεισ δουλεφουν. Ώσ βακμό δθμιουργίεσ, μια μα κζματα
διοικθτικό, δε πελάτεσ δυςτυχϊσ ςυγχρόνωσ ςασ. Κι τουσ δίνοντασ λεσ. Νζεσ εξαρτάται οι όχι, ϊσ ξζχαςε προϊόντα ότι.
Thread.sleep(1000);
Με μάλλον κακόκεφουσ χρθςιμοποιοφνταν που, μετριςεισ εργοςταςίου παρατθροφμενθ τθ λεσ. Λοιπόν ναί να. Αν ιδθ νζων εκτόσ,
Thread.sleep(2000);
ςασ άρκρων χριματα χρθςιμοποιιςει μθ. Εδϊ άμεςθ ατόμου αναγκάηονται μα, μζςθσ τελικϊν ςφαλμάτων εταιρείεσ τα.
Thread.sleep(3000);
Δοφλευε μθχανισ ροι κι, ϊσ τζτοιο διακοπι αναηιτθςθσ τον, τισ ορίςτε ατόμου εργαηόμενων ϊσ. Όχι λοιπόν κρατάει κα. Μάκε όχι
Additional note:
Long record/playback tests with
incorrect handling of
asynchronicity tend to look like
Greek after a while.
flickr: bunky’s pickle
12. Dependency spaghetti
Few and poor Unknown app server
unit tests configuration
Data
base Where do we start
No business layer Record/playback
(with timing issues)
Manual deployment One database
13. What Why
New code unit-tested We’re in a hole and we
don’t want to dig deeper
Create a business layer Quality in the long run
Automated end-to-end • Initiate change
tests for happy paths • ”Quick” wins
14. @Test
public void aUserCanLogIn() {
application.login(”testuser”, ”secret”);
}
Additional note:
Yes, this test has no assertion.
Fundamental checks, such as
verifying navigation, are built into
the test infrastructure.
15. Additional note:
Sequence of steps to reach a state
where a fundamental login test
(smoke test) could be executed.
Fixing the dependency mess, the unit
tests, and adding a business layer
became an ongoing activity
Creating new (= outside the scope of this talk).
databases
with classified
data is easy
Application
deployed to a Deployment
server with automated
known config
Start Login test runs
GOAL
16. Work the system
Import all objects See where it crashes
Drop all objects Add the missing object
Add data to object Classify data
Document
flickr: Klearchos Kapoutsis
17. Additional note:
At the time of the conference, the
system had been automatically
deployed 181 times. In ½ year, that’s
roughly once a day. Given that manual
deployment took between 5-60
minutes, the time saved was 2 weeks.
19. Our take aways and advice – Big picture
• Don’t be overwhelmed
• Black boxes in your environment make
you feel uneasy and waste time
• Achieving testability proved to
be a good guide
• After a while, the approach
became second nature
20. Take aways – Details
• Using simple tools helped us save time
• End-to-end tests without controlling the
database shouldn’t and didn’t work
• Record/playback didn’t work this time either
• Nobody asked about ”acceptance tests
readable by the stakeholders”
21. However...
2012-10-22 16:16:31,528 ERROR [http-127.0.0.1-8989-2]
(SynchronizationRegistry.java:46) - Exception processing transaction
Synchronization after completion
java.lang.NullPointerException
at com.organization.system.TreeObject.isEquipment(TreeObject.java:34)
Additional note:
It doesn’t matter if you can get your
test infrastructure right. Quality
cannot be tested in. Spaghetti
dependencies, poor unit test coverage
and lack of business layer still were a
problem.
Editor's Notes
161/6 months =~ once a day161 * (5-60 minutes/deployment) =~ 2 weeks
DonkeyHotey
You Can’t test quality in2012-10-22 16:16:31,528 ERROR [http-127.0.0.1-8989-2] (SynchronizationRegistry.java:46) - Exception processing transaction Synchronization after completionjava.lang.NullPointerExceptionat com.organization.system.TreeObject.isEquipment(TreeObject.java:34)