Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How the DevOps Movement Can Impact your Company's Data

22 views

Published on

White paper on the impact of DevOps principles on the data delivery pipeline and continuous data testing. Read about:

- DevOps tools
- Terms in the DevOps world
- Test automation in the DataOps pipeline

Published in: Software
  • Be the first to comment

  • Be the first to like this

How the DevOps Movement Can Impact your Company's Data

  1. 1. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 1 How the DevOps Movement Can Impact your Company's Data Published on July 24, 2019 DevOps is one of the hottest trends in the software industry and is the goal of most progressive IT organizations (see chart below, courtesy of Google Trends). DevOps (short for development and operations) is a set of automated software practices that combine software development (Dev), testing and IT operations (Ops) to shorten the software development life cycle while delivering features, fixes, and updates frequently in alignment with the business’ objectives. Bill Hayduk Founder & CEO of RTTS
  2. 2. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 2 DevOps is typically cross-functional (people from different areas) and uses different software tools. These tools usually fit into one or more of the following categories: • Coding – code development and review, source code management tools, code merging • Building – continuous integration tools (like Jenkins), build status • Testing – continuous testing tools (like QuerySurge, Selenium, Cucumber, JMeter) that provide feedback on business risks • Packaging – artifact repository, application pre-deployment staging • Releasing – change management, release approvals, release automation • Configuring – infrastructure configuration and management, infrastructure as code tools • Monitoring – applications performance monitoring, end-user experience While we're at it, let's add a couple more terms to the DevOps movement: Continuous Integration (CI) Continuous Integration is about automating build and test processes to make sure the resulting software is in a good state, ideally every time a developer changes code. CI helps development teams avoid integration issues where the software works on individual developers’ machines, but it fails when all developers combine their code. Continuous Delivery (CD) Continuous Delivery goes one step further to automate a software release, which typically involves packaging the software for deployment in a production-like environment. The goal of CD is to make sure the software is always ready to go to production, even if the team decides not to do it for business reasons. Continuous Deployment (also CD) Continuous deployment goes one step further than continuous delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There's no human intervention, and only a failed test will prevent a new change to be deployed to production. Continuous Testing One of the hottest buzz terms in the testing world, continuous testing is the process of executing automated tests as part of the delivery pipeline to obtain immediate feedback on the business risks associated with a release candidate. Continuous testing cannot be implemented without test automation. DevOps principles demand strong interdepartmental communication and rely heavily on automation tools. And now the movement to incorporate a DevOps-type of automated process for data has grown stronger. These practices are often referred to as Data DevOps or DataOps and apply DevOps tools and techniques to data. Data is growing geometrically and applying automation to develop, deploy and validate/test the data is becoming more critical, as businesses are implementing BI & Analytics to make sense of their data and to leverage it in hopes of providing a competitive advantage.
  3. 3. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 3
  4. 4. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 4 Tests in DataOps have a role in both the Value and Innovation Pipelines. In the Value Pipeline, tests monitor the data values flowing through the data factory to catch anomalies or flag data values outside statistical norms. In the Innovation Pipeline, tests validate new analytics before deploying them. DataKitchen Comparing DevOps to DataOps process - from DataKitchen Many of our QuerySurge customers have been requesting that we provide more DevOps-type features for validating their data pipeline and fitting into their CD automation processes. So we just released our new DevOps solution, QuerySurge for DevOps, which now automates data validation of Big Data, Data Warehouses, Enterprise Apps and Business Intelligence reports with full DevOps functionality for continuous testing.
  5. 5. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 5 With our new QuerySurge for DevOps release: • Testers can now dynamically generate, execute, and update tests and data stores utilizing API calls • Teams have access to 60+ API calls with almost 100 different properties • QuerySurge integrates with virtually all DevOps solutions in the marketplace QuerySurge for DevOps' API features include the ability to create and modify: • source and target test queries • connections to data stores • tests associated with an execution suite • new staging tables from various data connections • custom flow controls based on run results Below are some sample Use Cases that can be utilized in a DataOps pipeline with QuerySurge for DevOps:
  6. 6. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 6
  7. 7. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 7 The use cases are practically endless as QuerySurge for DevOps provides the flexibility to integrate your data testing process into your existing DevOps implementation. We are now in a full Data DevOps or DataOps movement to validate and automate the testing of the data pipeline. We're just at the beginning of this movement but we are all-in on building the right solution into QuerySurge to automate the continuous testing process.
  8. 8. copyright Real-Time Technology Solutions, Inc. August 2019 www.rttsweb.com page 8 About RTTS - The Software Quality Experts RTTS, the developer of QuerySurge, is the premier pure-play QA & Testing organization that specializes in Test Automation. Headquartered in New York, RTTS has had 1,000+ successful engagements at over 700 corporations since 1996. RTTS focuses on improving software & data quality through test automation. We create QA strategies, provide software (QuerySurge) and implementation services targeted at the data space. To learn more about RTTS, click here>> About QuerySurge QuerySurge is the software division of RTTS. QuerySurge is the smart Data Testing solution that automates the data validation & testing of Big Data, Data Warehouses, and Business Intelligence Reports. QuerySurge will help you: • Improve your data quality at speed • Dramatically increase data validation coverage • Detect data issues early in the delivery cycle • Leverage analytics to optimize your critical data • Provide a huge ROI But don’t believe us (or our clients). Try it for yourself. Check out our free trials and great tutorial

×