ACM-SAC-2016-Keynote (1)

•

0 likes•104 views

The document discusses extreme automation of testing distributed systems, services, and APIs. It proposes: 1) Automating all testing activities, including automated test input and oracle generation, intelligent test prioritization and scheduling, execution, arbitration and reporting. 2) Using models of the distributed system under test, including service models, architecture topology, and protocol state machines to generate test cases. 3) Intelligently scheduling and reactively planning tests by choosing the next test case to run based on past test verdicts and generating new test cases to diminish ignorance of system elements. 4) Providing these testing technologies and functional testing workflow as a cloud-native, on-demand testing as a service platform.

EXTREME AUTOMATION OF
DISTRIBUTED SYSTEMS, SERVICES, & API TESTING
simplyTestify.com
Service Functional Testing Automation with
Intelligent Scheduling and Planning
Lom M. Hillah,A-P. Maesano, L. Maesano, F. De Rosa, F. Kordon, and P-H.Wuillemin

Model and Inference Driven, Automated Testing
of Services Architectures (MIDAS)
midas-project.eu
simplyTestify.com
FOREGROUND OF EU FP7 MIDAS

Distributed systems, services &
APIs
Large scale, multi-owner
Synchronous & Asynchronous
Time constrained
Heterogeneous Protocols
(SOAP, REST,…)
WHAT IS UNDER TEST?
simplyTestify.com

HOW DO WE TEST IT?
Test extreme automation
Test automation as a service
Test automation methods PaaS
simplyTestify.com
send stimuli /
collect outcomes
direct connection
Internet
S1
Distributed System Under Test
S2
S3
S7
S6
S4
S5
Internet
Cloud
simplyTestify
Portal
directconnection
directconnection
Test
method
#1
Test
method
#1
Test
method
#1
Test
method
#1
User
Web
Client
DSUTadapter
invoke test
generation
& run
methods on
the DSUT
via
API
via
GUI
API gateway
User
Corp.
Server
Model-based, Black-box, Grey-box,
Troubleshooting

• AUTOMATED TEST INPUT & ORACLE
GENERATION (Formal Methods)
• INTELLIGENT DYNAMIC TEST
PRIORITISATION & SCHEDULING (A.I.)
• AUTOMATED TEST EXECUTION,
ARBITRATION & REPORTING
• INTELLIGENT EVIDENCE-BASED REACTIVE
PLANNING OF TEST SESSIONS (A.I.)
ALL (HARD) TESTING ACTIVITIES ARE AUTOMATED
OUR TESTING TECHNOLOGIES
simplyTestify.com

• Service Model (WSDL, XSD)
• Distributed Architecture under Test:
topology of components and services
(SCA)
• Protocol State Machine (SCXML) for
each component: conditions and
transfer functions in Javascript
INPUT MODELS FOR FUNCTIONAL TESTING
simplyTestify.com

TEST CASE GENERATION
simplyTestify.com
Preprocessing
Translation (PlusCal)
Compilation (TLA+)
Model Checking
Test Input Generation
Parallel PSM Model
Execution
Oracle
Generation
Models of the
DSUT
Test Generation
Directives
Test Suite
Deﬁnition
Test Suite
Samples

AUTOMATED TEST SCHEDULING
simplyTestify.com
[TC][TV]
[evidences] [probabilities]
STOP
STOP
Inference engine
Service interface
Scheduling
Policy module
Probabilities container
RequestTestScheduling([TC]) NotifyTestSchedOutcome(C)
C = [TC] C = ∅
• Cycle schedule/execute/arbitrate
• Choose next test case (TC) to run
on the basis of past tests verdicts
• Detect failures early
• Locate faulty elements
(troubleshooting)

AUTOMATED REACTIVE PLANNING
simplyTestify.com
• Scheduler not only drives the choice of
the next TC, but also the on-the-ﬂy
generation of new TCs
• Using evidences from past test runs:
• calculates the degree of ignorance of
DSUT elements, and recommend the
generation of TCs whose execution
would diminish this ignorance
[TC][TV]
[evidences] [probabilities] [probabilities]
STOP
STOP
Test
Generation
Directives
Inference engine
Service interface
Scheduling
Policy module
Generation
Policy module
Probabilities container
RequestTestScheduling([TC]) NotifyTestSchedOutcome(C)
C = [TC] C = ∅ C = TestGenDirectives.xml

FUNCTIONAL TESTING WORKFLOW OVERVIEW
simplyTestify.com
Request Init Scheduling
Test GenerationTest Scheduling
Test Execution
Models of the DSUT
Test Conﬁguration
Test Suites
Test Verdicts
Reports
Dashboard
Verdicts Test Suites
Test Suites
Test Generation Directives
Logs Workﬂow Manager is the actual broker
between the involved components

• Cloud native TaaS
• Highly elastic, available, reliable
& secure
• Programmable through APIs
• On-demand, self-provisioning,
pay as you go
• Looking for early adopters!
DISRUPTIVE APPROACH
simplyTestify.com
TEST MECHANIZATION, OPTIMIZATION & ROUTINIZATION

DEMO
simplyTestify.com
StartVideo at 2’2’’

Similar to ACM-SAC-2016-Keynote (1)

C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010

TEST Huddle

Test Automation is becomming a MUST in software development life cycle now. DevOps has been an emerging trend, and it's no longer new. Remebering the old days, when you have to stand-up the test servers, get the builds from developers, deploy it, start-up agent machines, run your tests, collect reports, shutdown all resources you have just started, and spend days to analyze the failures. Now it's time to bring DevOps into this game and let it streamline all of these processes then you can save your days for other greater jobs of software testing.

[DevDay2019] Power of Test Automation and DevOps combination - One click savi...

DevDay.org

OBIEE online training offered by Quontra Solutions with special features having Extensive Training will be in both OBIEE Online Training and Placement. We help you in resume preparation and conducting Mock Interviews. Emphasis is given on important topics that were required and mostly used in real time projects. Quontra Solutions is an Online Training Leader when it comes to high-end effective and efficient IT Training. We have always been and still are focusing on the key aspect which is providing utmost effective and competent training to both students and professionals who are eager to enrich their technical skills.

Performance Testing and OBIEE by QuontraSolutions

QUONTRASOLUTIONS

Automation Framework Presentation

Ben Ngo

SAP Testing Services

r_shanki

QTP ONLINE TRAINING

Santhosh Sap

Key Concepts: The structure of a Test Framework, its components, and the process to create one will be presented. A Test Framework provides a means to automate tests fast and guarantee their extended existence. It makes the automation process straightforward and systematic, simplifies the error debugging procedures, makes the testware tolerant to all kinds of object changes and reliable in an unstable test environment. The presentation is not an illustration of a specific test framework implementation, but rather a description of the features that should be mandatory attributes of any test framework. Learning Objectives: How to present the testware internally as a hierarchy of test sets (structural level), use cases (functional - scenario level), test cases (the scenario steps with verification), test actions (the interface "action" words). How to present the testware externally as a set of configuration files, scripts, libraries.How to separate the business action words from the interface action words and the respective drivers.How to create debug logs that allow to identify the source of a failure in minutes.

A Test Automation Framework

Gregory Solovey

FUNTASY is a state of the art Test Automation Management tool with capabilities of running GUI and non-GUI test automation. FUNTASY is based on the Keyword Driven Testing (KDT) approach and promotes cross-organization test design by bridging the gap between the test automation team and the subject matters experts. With FUNTASY, you can improve product quality while reducing test script development, maintenance and execution time. http://www.qualitestgroup.com

FUNTASY - Functional testing automated system

Qualitest

JMeter & ColdFusion

isummation

Cloud-based Test Microservices JavaOne 2014

Shelley Lambert

Combinatorial testing

Kedar Kumar

The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing. Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.

Neotys PAC 2018 - Ramya Ramalinga Moorthy

Neotys_Partner

Web Performance Testing

CSC Trusted Cloud Services

C010241521

IOSR Journals

Test Automation for QTP

Kiran Rayachoti

Test Automation

Kiran Rayachoti

Generally, the purpose of testing is to assess applications quality. Many approaches existing for traditional software systems can be adapted or even reused for service-oriented systems. Service-oriented testing has many similarities with component-based testing. The Oracle SOA Suite offers out of the box support for testing BPEL and Service Bus services. But is this enough? How can we test services which are dependent on external services not available at testing time? Is there an efficient way to mock these services? This session we will present solutions and best practices for testing service-oriented systems and show how testing can be improved by adding some 3rd party products to the Oracle SOA Suite, like SoapUI.

Best Practices for testing of SOA-based systems - with examples of SOA Suite 11g

Guido Schmutz

Performance Test Slideshow Recent

Future Simmons

Performance Test Slideshow R E C E N T

Future Simmons

Parasoft .TEST, Write better C# Code Using Data Flow Analysis

Engineering Software Lab

Similar to ACM-SAC-2016-Keynote (1) (20)

C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010

[DevDay2019] Power of Test Automation and DevOps combination - One click savi...

Performance Testing and OBIEE by QuontraSolutions

Automation Framework Presentation

SAP Testing Services

QTP ONLINE TRAINING

A Test Automation Framework

FUNTASY - Functional testing automated system

JMeter & ColdFusion

Cloud-based Test Microservices JavaOne 2014

Combinatorial testing

Neotys PAC 2018 - Ramya Ramalinga Moorthy

Web Performance Testing

C010241521

Test Automation for QTP

Test Automation

Best Practices for testing of SOA-based systems - with examples of SOA Suite 11g

Performance Test Slideshow Recent

Performance Test Slideshow R E C E N T

Parasoft .TEST, Write better C# Code Using Data Flow Analysis

More from Libero Maesano

HSSP-SOA_Forum-HL7IntParisWGM-MLettere_LMaesano-MIDAS-Testing HSSP Services-M...

Libero Maesano

Steps towards an industrial implementation of HSSP standards

Libero Maesano

SOA Testing As A Service

Libero Maesano

Service contracts between digital providers and consumers should be: (i) understood and verified by the business stakeholders, (ii) mechanically checked for consistency and (iii) automatically mapped to system implementations. This presentation sketches guidelines and templates for expressing service contract clauses as business rules, i.e. logical formulations in constrained natural language conforming to the OMG Semantics of Business Vocabulary and Business Rules (SBVR) specification. When supported by the SBVR abstract notation, natural language business rules can be mechanically checked for consistency, and eventually processed for automated generation of technical model elements, machine-readable components, executable code and tests. The service contract clauses are established through a method that is inspired by the Design by contract™ approach: the signature of the service operation is expressed by means of a set of business vocabulary entries and its pre/post-conditions are stated as operative business rules. The method and the corresponding rule templates are included in a methodological framework for model driven engineering of services architectures (simpleSOA™), in which the service contract, as a bundle of rights and duties for service providers and consumers, is represented by a layered collection of formal models that work as a body of requirements for implementations. In this model-driven framework, the service contract vocabulary and rule-book constitute the service Computation Independent Model (CIM). We introduce the method guidelines and templates through an example and discuss the assumptions on the resulting conceptual model as well as some recommendations on the contract building enterprise.

Service contract clauses as business rules

Libero Maesano

This presentation reports the progress of a project whose goal is to develop a model-driven, contract-based SOA testing environment in which grey-box testing strategies for large services architectures are driven by stochastic inference. In order to achieve this goal, a Bayesian network is built directly by mapping and model transformation from the services architecture design and test models. Because services architectures are complex and large systems, the generated BN and its associated probability tables may also be very large and the inference process may be excessively slow. Nevertheless, the structures of the services architecture design and test models let techniques of compilation improve the inference task, enabling the use of BN inference as a sustainable tool for driving failure discovering and troubleshooting strategies in large scale services architectures.

Steps towards model-based, inference-driven SOA Testing

Libero Maesano

This presentation introduces the simpleSOAD® methodological framework for design and implementation of services and services architectures, founded upon a contract-based, model-driven (MDA) approach and the OMG standards. Contract-based service orientation is presented first. The service contract is a first-class object, is independent from the systems that endorse it, and acts as a functional and behavioral specification for these systems. Furthermore, the principles of application of the model-driven engineering approach to service design are given. A service contract is a layered set of models: BMM (motivational), BM (conceptual), PIM (logical) and Interoperability PSM (physical). They are all based upon largely accepted OMG standards. The service contract is designed top-down with the help of methods and tools of model mapping and transformation. The Implementation PSM of a system that intends to provide the service can be partially generated by the contract model. Future work will concern phases and activities of the SOA life cycle till now poorly covered, such as Validation &Verification, Test and Governance.

simpleSOAD 2.0 Architecture and Governance

Libero Maesano

More from Libero Maesano (6)

HSSP-SOA_Forum-HL7IntParisWGM-MLettere_LMaesano-MIDAS-Testing HSSP Services-M...

Steps towards an industrial implementation of HSSP standards

SOA Testing As A Service

Service contract clauses as business rules

Steps towards model-based, inference-driven SOA Testing

simpleSOAD 2.0 Architecture and Governance

ACM-SAC-2016-Keynote (1)

1. EXTREME AUTOMATION OF DISTRIBUTED SYSTEMS, SERVICES, & API TESTING simplyTestify.com Service Functional Testing Automation with Intelligent Scheduling and Planning Lom M. Hillah,A-P. Maesano, L. Maesano, F. De Rosa, F. Kordon, and P-H.Wuillemin

2. Model and Inference Driven, Automated Testing of Services Architectures (MIDAS) midas-project.eu simplyTestify.com FOREGROUND OF EU FP7 MIDAS

3. Distributed systems, services & APIs Large scale, multi-owner Synchronous & Asynchronous Time constrained Heterogeneous Protocols (SOAP, REST,…) WHAT IS UNDER TEST? simplyTestify.com

4. HOW DO WE TEST IT? Test extreme automation Test automation as a service Test automation methods PaaS simplyTestify.com send stimuli / collect outcomes direct connection Internet S1 Distributed System Under Test S2 S3 S7 S6 S4 S5 Internet Cloud simplyTestify Portal directconnection directconnection Test method #1 Test method #1 Test method #1 Test method #1 User Web Client DSUTadapter invoke test generation & run methods on the DSUT via API via GUI API gateway User Corp. Server Model-based, Black-box, Grey-box, Troubleshooting

5. • AUTOMATED TEST INPUT & ORACLE GENERATION (Formal Methods) • INTELLIGENT DYNAMIC TEST PRIORITISATION & SCHEDULING (A.I.) • AUTOMATED TEST EXECUTION, ARBITRATION & REPORTING • INTELLIGENT EVIDENCE-BASED REACTIVE PLANNING OF TEST SESSIONS (A.I.) ALL (HARD) TESTING ACTIVITIES ARE AUTOMATED OUR TESTING TECHNOLOGIES simplyTestify.com

6. • Service Model (WSDL, XSD) • Distributed Architecture under Test: topology of components and services (SCA) • Protocol State Machine (SCXML) for each component: conditions and transfer functions in Javascript INPUT MODELS FOR FUNCTIONAL TESTING simplyTestify.com

7. TEST CASE GENERATION simplyTestify.com Preprocessing Translation (PlusCal) Compilation (TLA+) Model Checking Test Input Generation Parallel PSM Model Execution Oracle Generation Models of the DSUT Test Generation Directives Test Suite Deﬁnition Test Suite Samples

8. AUTOMATED TEST SCHEDULING simplyTestify.com [TC][TV] [evidences] [probabilities] STOP STOP Inference engine Service interface Scheduling Policy module Probabilities container RequestTestScheduling([TC]) NotifyTestSchedOutcome(C) C = [TC] C = ∅ • Cycle schedule/execute/arbitrate • Choose next test case (TC) to run on the basis of past tests verdicts • Detect failures early • Locate faulty elements (troubleshooting)

9. AUTOMATED REACTIVE PLANNING simplyTestify.com • Scheduler not only drives the choice of the next TC, but also the on-the-ﬂy generation of new TCs • Using evidences from past test runs: • calculates the degree of ignorance of DSUT elements, and recommend the generation of TCs whose execution would diminish this ignorance [TC][TV] [evidences] [probabilities] [probabilities] STOP STOP Test Generation Directives Inference engine Service interface Scheduling Policy module Generation Policy module Probabilities container RequestTestScheduling([TC]) NotifyTestSchedOutcome(C) C = [TC] C = ∅ C = TestGenDirectives.xml

10. FUNCTIONAL TESTING WORKFLOW OVERVIEW simplyTestify.com Request Init Scheduling Test GenerationTest Scheduling Test Execution Models of the DSUT Test Conﬁguration Test Suites Test Verdicts Reports Dashboard Verdicts Test Suites Test Suites Test Generation Directives Logs Workﬂow Manager is the actual broker between the involved components

11. • Cloud native TaaS • Highly elastic, available, reliable & secure • Programmable through APIs • On-demand, self-provisioning, pay as you go • Looking for early adopters! DISRUPTIVE APPROACH simplyTestify.com TEST MECHANIZATION, OPTIMIZATION & ROUTINIZATION

12. DEMO simplyTestify.com StartVideo at 2’2’’

ACM-SAC-2016-Keynote (1)

Recommended

Recommended

More Related Content

Similar to ACM-SAC-2016-Keynote (1)

Similar to ACM-SAC-2016-Keynote (1) (20)

More from Libero Maesano

More from Libero Maesano (6)

ACM-SAC-2016-Keynote (1)