Artificial intelligence in qa


Published on

Published in: Technology
  • Be the first to comment

Artificial intelligence in qa

  1. 1. Taras Lytvyn © 2012GlobalLogic Inc. 1
  2. 2. Mathematical models and artificial intelligence in software testing Taras Lytvyn processes © 2012GlobalLogic Inc. 2
  3. 3. Contents- What is artificial intelligence?- Problem statement in general- Analysis of recent researches & technologies- Main idea of artificial intelligence approach- Math model- Test oracle based on AI- Result evaluation and classification.Comparison tool algorithm. © 2012GlobalLogic Inc. 3
  4. 4. What is artificial intelligence ? Semiotic Biologicalthinking, Neural networksjudgment,language,emotions,creativity, etc. Intellectual mental behavior processes © 2012GlobalLogic Inc. 4
  5. 5. Problem statement in general SIMPLE TEST PROCESSHow well an evaluated app. conforms to its specs. 3 stages: - test data generation, - testing, Regression - result evaluation testing PROBLEM OF HIDDEN ERRORS © 2012GlobalLogic Inc. 5
  6. 6. Analysis of recent researches & techsWHERE AI CAN BE USED ?- metric’s analysis- cost of testing- reliability of testing in general- optimization processes & data optimization © 2012GlobalLogic Inc. 6
  7. 7. Main idea Decisions during Regression Testing MANUAL AUTOMATION Assert functions (Actual is equal/not equal to expected)Intellectual automationDecision System during Regression Testing © 2012GlobalLogic Inc. 7
  8. 8. Main ideaDecision System is based on artificial intelligent approach and is build on algorithm with NEURAL NETWORK usage. Neural Network training process © 2012GlobalLogic Inc. 8
  9. 9. Main idea Testedinputs it works ? Software Test outputs cases Neural Trained neural networkinputs Network that will emulate our SOFTWARE workability © 2012GlobalLogic Inc. 9
  10. 10. Math Modeling usage The test case execution process should be presented as Complex Math Model - Neural Network needs digits (not text) - Input and output data should be normalized in specific way - Math Model will give some limitations Theorem & LimitationsV ji couldn’t be empty FunctionsThere is no intermediate step, TCE(t ji ) : Sl ji Vl ji , l k (k m)if a corresponding stage of verification exist TCP(Sl ji ) : Sl 1 ji Vl 1 ji ,...,S1 ji V1 ji , l 1.Intermediate stage of verification is possible TCP(S1 ji ) : S1 ji V1 ji . © 2012GlobalLogic Inc. 10
  11. 11. AI Test Oracle Test oracle – is a classifier that shows us whether test was passed or not Tested Software Comparison Test Tool ResultCases Algorithm classifications Trained Neural Network Test oracle classify the result of test case execution © 2012GlobalLogic Inc. 11
  12. 12. Result classification & Comparison ToolThe Comparison Tool is employed as an independent method of comparing results from neural network and the results of the tested versions of app. App. result Neural Network result Correct Incorrect Correct 1 True Positive 2 True Negative Incorrect 4 False Negative 3 False Positive Comparison of outputs Output type Same Different Both correct ANN correct Binary Both wrong APP correct ANN correct Both Correct Continuous APP correct Both wrong Both wrong © 2012GlobalLogic Inc. 12
  13. 13. ExampleInput Data (test case) should be NORMALIZED !!! TC Test Verification (test TC Title (test Test steps (test case id case verification) / case name) steps) / Action (№) Expected result 1. Input Regular 1. Regular Hours Hours should be 2. Input Age displayed 3. Input Rate of Pay 2. Age should be Gross Pay 1.2 4. Click Calculate displayed View 3. Rate of Pay should be displayed 4. Gross Pay should be displayed Structure of ANN is based Training phase with 50 – 5000 on Kolmogorov records Arnold approach © 2012GlobalLogic Inc. 13
  14. 14. - Introduced a mathematical model describing thetest case execution for software that is tested- Constructed a new algorithmic model of testoracle based on neural networks- This Model can be used in the processes ofregression testing software- Comparison Tool as result analyzer was provided © 2012GlobalLogic Inc. 14
  15. 15. <Q&A?> © 2012GlobalLogic Inc. 15
  16. 16. Taras Lytvynautomation QC in Global Logic : taras.lytvyn@globallogic.comweb: tasryk © 2012GlobalLogic Inc. 16