QuerySurge, the smart data testing solution, QuerySurge, the smart data testing solution that automates data validation & testing of critical data, released the first-of-its-kind full DevOps solution for continuous data testing. The latest release, QuerySurge-for-DevOps, enables users to drive changes to their test components programmatically while interfacing with virtually all DevOps solutions in the marketplace. See how to implement a DevOps-for-Data solution in your delivery pipeline and improve your data quality at speed!
Testers will now have the capability to dynamically generate, execute, and update tests and data stores utilizing API calls. QuerySurge for DevOps has 60+ API calls with almost 100 different properties. This will enable a higher percentage of automation in your current data testing practice and a more robust DevOps for Data, or DataOps pipeline.
API Features Include:
- Create and modify source and target test queries
- Create and modify connections to data stores
- Create and modify the tests associated with an execution suite
- Create and modify new staging tables from various data connections
- Create custom flow controls based on run results
- Integration with virtually all build solutions in the market
QuerySurge for DevOps integrates with:
- Continuous integration/ETL solutions
- Automated build/release/deployment solutions
- Operations and DevOps monitoring solutions
- Test management/issue tracking solutions
- Scheduling and workload automation solutions
For more information on QuerySurge for DevOps, visit:
https://www.querysurge.com/solutions/querysurge-for-devops
2. QuerySurge™ a software division of
The DevOps Challenge
The key to a successful DevOps
implementation is:
• to automate as many of the
development and operational tasks as
possible
• to automate changes defined in the
Planning phase & propagate changes
to all phases of the DevOps process
The Issue
The Goal
Teams find it difficult for design changes to make
their way to all phases of the delivery process in a
continuous manner
3. Traditional DataTesting Validation methods
built by
1) Sampling
QuerySurge™ a software division of
3) HomegrownSolution
2) Minus Queries Source Target
4. built by
QuerySurge™
• Be able to validate large amounts of data quickly
• Be able to validate difficult transformation rules between the
various source and target systems being tested
• Be able to be integrated into the DevOps pipeline to achieve
continuous testing
• Be able to detect requirement and code changes and have the
tests either reflect or alert team members of those changes .
• Be able to inform team members of any failures as they occur.
• Be able to store historical results and provide analytics
The Ideal DevOpsTesting Solution
The ideal data testing solution should be able to perform the following:
5. is the smart testing solution for
automated validation & testing of Data
QuerySurge
QuerySurge™
Use Cases
a software division of
What is QuerySurge?
6. QuerySurge connects
to any 2 points
at one time
SQL
HQL
SQL
Comparison of every data set
Source
Data
Target
Data
Data Analytics Dashboard,
Data Intelligence Reports,
automated emails
Results – pass/fail
Target Data
Big Data
stores
• Hadoop
• NoSQL
Data
Warehouses
Business Intelligence
Reports
XML
Web Services
Source Data
Data Stores
• Databases
• Data Warehouses
• Data Marts
Flat Files
• Fixed Width
• Delimited
• Excel
• JSON
Mainframes
• DB2
• Various file types
7. Data Warehouse
ETL
Data Mart
ETL
Source Data Big Data lake BI & Analytics
ETL Developer: Codes data movement based on Mapping Requirements
Data Tester: Tests data movement based on Mapping Requirements
Testing Point #1 Testing Point #2 Testing Point #3 Testing Point #4
8. a software division ofQuerySurge™
Q uerySurge for DevOps
“With QuerySurge for DevOps, testers will
now have the capability to dynamically
generate, execute, and update tests and
data stores utilizing API calls”
• First full DevOps for Data solution
• improve their data quality at speed
QuerySurge for DevOps integrates with:
• Continuous integration/ETL solutions
• Automated build/release/deployment
solutions
• Operations and DevOps monitoring
solutions
• Test management/issue tracking solutions
• Scheduling and workload automation
solutions
60+ API calls with almost 100 different
properties that users can utilize to retrieve,
edit, update, or delete information.
9. a software division ofQuerySurge™
Q uerySurge for DevOps
Run Test Scenario
Kill Test Scenario
Execution
Test Suite Results
Individual Test Results
Source and Target Data
Failed Record Data
Test Suite Execution Status
Retrieve
QueryPairs
Create / Modify / Delete
Datastore Connections
Test Suites
Staging Tables
Query Snippets
Staging Queries
With the new expanded QuerySurge DevOps API, customers now have the ability to perform
design and analysis operations externally from QuerySurge, which allows QuerySurge to be
adopted and integrated into any DevOps process that focuses around data.
QuerySurge Server
10. a software division ofQuerySurge™
DevOps Marketplace
QuerySurge can
now be seamlessly
integrated with
other products in
the DevOps
marketplace.
11. a software division ofQuerySurge™
Q uerySurge for DevOps Upgrade
The QuerySurge for DevOps release has 9x more API calls
12. Q uerySurge for DevOps
Base API QuerySurge for DevOps full API
Create
• createExtensibleConnection
• createFolder
• createQueryPair
• createSnippet
• createStagingQuery
• createSuite
Execute
• executeDesigntimeRun
• executeScenario
• executeScenarioBySuiteId<product>*
• executeScenarioBySuiteName<product>*
Modify
• modifyExtensibleConnection
• modifyFolder
• modifyQuery
• modifyQueryConnection
• modifyQueryPair
• modifyReusableQueryTextById*
• modifyReusableQueryTextByName*
• modifySnippet
• modifySnippetText*
• modifyStagingQuery
• modifySuite
• modifySuiteQueryPairList
Stop/Kill
• stopDesignTimeRun
• killScenario
Delete
• deleteQueryPair
• deleteSnippet
• deleteStagingQuery
• deleteSuite
Retrieve
• getAgentStatus
• getDesignTimeRunOutcome
• getDesignTimeRunRawResults
• getDesignTimeRunStatus
• getFolder
• getQueryPairs
• getScenarioOutcome*
• getScenarioOutcome<product>*
• getScenarioResults<product>*
Run
• runTestSuiteById*
• runTestSuiteByName*
Retrieve
• getScenarioOutcome
Modify
• modifyReusableQueryTextById
• modifyReusableQueryTextByName
• modifySnippetText
Run
• runTestSuiteById
• runTestSuiteByName
Test Management API
= Base API + Test Management API + DevOps API
Execute
• executeScenarioBySuiteId<product>
• executeScenarioBySuiteName<product>
Retrieve
• getScenarioOutcome<product>
• getScenarioResults<product> *also in base & test managementAPIs
13. a software division ofQuerySurge™
DevOps Pipeline
Execute
Job
QuerySurge ™
Server
Run QuerySurge
Validation Tests
Execution
API
Rollback
Job
Fail
Automatic
Email Notification
Promote
Environment
Pass Continue…
Q uerySurge for DevOps Use Cases
Use Case Example #1
Have QuerySurge Test automatically initiated after ETL execution
completes and have conditional logic applied bases on specific
results of those executions
14. a software division ofQuerySurge™
Q uerySurge for DevOps Use Cases
Use Case Example #2
When new environments are deployed, new connections
automatically created in QuerySurge and tests duplicated to test
against this environment
QA
QA Connection
Pre-Production
New Folders created with tests copied over
Pre-ProductionConnection
All Test updated with the new connection
15. a software division of
Q uerySurge for DevOps Use Cases
Use Case Example #3
Have QuerySurge tests automatically analyze and detect changes to the
data store schemas and have those changes reflected in the tests
Original Target
Modified Target
Auto-detect
Multiple Sources
16. a software division of
Q uerySurge for DevOps Use Cases
Use Case Example #3 – continued…
Original QueryPair
Modified QueryPair
18. built by
QuerySurge™ a software division of
Key Links
QuerySurge home page
https://www.querysurge.com/
QuerySurge for DevOps
https://www.querysurge.com/solutions/querysurge-for-devops
Free Trials
https://www.querysurge.com/compare-trial-options
Licensing & Pricing
https://www.querysurge.com/product-tour/licensing-pricing-options
for