Coherent Solutions, Inc. Confidential Page 1 of 3
Coherent Solutions – QA Automation Questionnaire
The purpose is to provi...
Coherent Solutions, Inc. QA Automation Questionnaire
Coherent Solutions, Inc. Confidential Page 2 of 3
 When backend Inte...
Coherent Solutions, Inc. QA Automation Questionnaire
Coherent Solutions, Inc. Confidential Page 3 of 3
Dictionary
 UI tes...
Upcoming SlideShare
Loading in …5
×

Solit 2014, Как эффективно продавать Automation Service, Семенченко Антон

568 views

Published on

Антон Семенченко, опыт в IT более 10 лет, работает в компании ISSoft, специализируется в разработке и автоматизированном тестировании ПО плюс менеджмент\продажи. C++ Architect, Automation Practice Lead, PM, Group Manager

«Agile ValueTeam, учимся понимать Scrum». IT секция. Agile отделение. Для всех уровней подготовки.
«Как эффективно продавать Automation Service». IT секция. Продажи.
«Как эффективно организовать Автоматизацию, если у вас недостаточно времени, ресурсов и денег». Development секция. Отделение тестирования.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Solit 2014, Как эффективно продавать Automation Service, Семенченко Антон

  1. 1. Coherent Solutions, Inc. Confidential Page 1 of 3 Coherent Solutions – QA Automation Questionnaire The purpose is to provide delivery manages, sales people and QA leads a structured tool to collect information about QA automation opportunities for the purpose of (a) qualifying the opportunity (i.e. is it real); (b) scoping the opportunity to determine parameters for proposals; (c) demonstrating Coherent’s automation expertise by asking intelligent questions; and (d) providing requirements/constraints used to staff and initiate the team’s activities. 1) What are your QA automation goals, vision, high level needs? Which factors influenced your decision to consider automation? 2) What type of benefits do you expect to get out of automation? Specific cost savings in reducing manual labor? Better quality by providing broader test coverage, faster releases by improving regression testing efficiency? 3) What types of Automation are you interested in pursuing?  UI o Smoke testing o Regression testing o Functional testing o Other testing  Performance testing  Domain specific (database testing)  Application development business process specific (like Behavior Driven Development and Testing)  Code driven testing o Web Services / APIs  Smoke testing  Regression testing  Functional testing  Other testing  Performance testing  Domain specific (database testing)  Application development business process specific (like Behavior Driven Development and Testing) 4) Is your application stable (how much do you change the application UI and APIs) enough for the efficient Automation? How often will your application be changed? Do you have an established process for managing changes to the existing application? If yes, is it well documented and closely followed? How often do you see exceptions to the process?”  UI changes often trigger changes in automation tests. The degree to which changes drive automation maintenance effort depends on the automation solution’s architecture, the automation tools used, and the implementation technology of the underlying application. Therefore, if an application’s UI will be changed frequently, this becomes an important consideration in selecting an appropriate automation technology that can be resilient to those changes
  2. 2. Coherent Solutions, Inc. QA Automation Questionnaire Coherent Solutions, Inc. Confidential Page 2 of 3  When backend Interfaces are changed, code driven automation tests will need to be modified appropriately.  An unstable application will increase the maintnenace effort required for the automation suite thereby increasing its costs and reducing ROI. 5) What Automation experience do you have internally in your organization? Do you have existing tools, standards, etc. in place or do you need a vendor to develop those? 6) How you envision execution of the Automation suite?  Do you envision running Automation manually on demand?  Do you envision running Automation as a part of Continuous Integration (CI)? If yes, do you have any requirements/constraints defined for how that would work?  Do you envision running Automation as a part of daily development flow? If yes, do you have any requirements/constraints defined for how that would work 7) Do you have requirements/constraints on the automation solution’s architecture? For example 7.1) Develop a separate Automation solution for each Application layer (UI, High level backend, Database Layer) and run (use) each layer separately? 7.2) Integrate all Automation layers via continuous integration? 7.3) Use or develop your own one entry point solution for all Automation layers and use it as a part of every-day development flow? Notes: 7.1) This is the first phase of Automation and it is the most effort intensive/time consuming phase. 7.2) This is the next phase of Automation solution’s maturation. It introduces and additional set of effort/costs: o Special tools o Additional hardware (CI servers) o Administration effort/labor (CI servers) o possibly Automation solution performance improvement 7.3) This is the highest level of Automation solution maturity, however, because it builts on the prior phases( 7.1 and 7.)2, it will cost much less than the first phase. If one can accomplish this level of maturity the automation solution will change the development process dramatically, significantly increasing development performance and decreasing new developers training time. 8) Can your database be used directly in the Automation solution to validate results for UI & database? 9) Do you have confirguration management practices and presense of development / test / QA / integration environments (for the full application stack including database) and recommendations around where the automation should be done. Do you have or need a test database for Automation? 10) What are your performance requirements for Automation (time, hardware)? Do you want to run Automation remotely? How frequently do you want to run All Automation tests? 11) Do you have any requirements/constraints for Automation tools (open source, vendor supported, etc.)?
  3. 3. Coherent Solutions, Inc. QA Automation Questionnaire Coherent Solutions, Inc. Confidential Page 3 of 3 Dictionary  UI testing (frontend) - access, identify, and manipulate UI elements in the same way as end user (the highest level of Automation).  Code driven/ Service / API testing (backend) o Web Servicies (backend) testing o Services /daemons (backend) testing o Shared libraries [dlls or shared objects] (backend) testing - the public (usually) interfaces to classes, modules or libraries (i.e. code internals) are tested with a variety of input arguments to validate that the results that are returned are correct.  Smoke testing (frontend or backend) - is any type of “quick and dirty” software testing that proves that the software will not crash outright (i.e. the device doesn’t start smoking when you simply plug it it). Passing smoke testing means the system is ready for more robust testing. Smoke tests are first tests made after assembly or repairs to a system, to provide some assurance that the system under test will not catastrophically fail.  Regression testing (frontend or backend) - is any type of software testing that seeks to uncover software bugs, or regressions, in the unchanged parts of a system after changes, (enhancements, patches, configuration changes, etc.), have been made to them or other parts of the system.  Functional testing (frontend or backend) - is a QA black box testing process that bases its test cases on the specifications of the software component under test.  CI (Continuous Integration) - the practice of merging all developer working copies of code with a shared mainline of code several times a day. CI was originally intended to be used in combination with automated tests (first strictly with unit tests, but best practice has evolved to ideally include the execution of all tests against all environments, at least periodically). There are a set of industry standard tools that support CI and we use some flavor of CI (often without corresponding automation) on almost every development project we do. Principles of CI o Maintain a code repository o Automate the build o Make the build self-testing o Everyone commits to the baseline every day o Every commit (to baseline) should be built o Keep the build fast o Test in a clone of the production environment o Make it easy to get the latest deliverables o Everyone can see the results of the latest build o Automate deployment

×