ARTIFICIENT INTELLIGENCE IN
SOFTWARE TESTING
VLADIMIR ARUTIN
Vladimir Arutin
QA Engineer
in HYS Enterprise
ISTQB® Certified Test Manager
WHAT IS ARTIFICIENT INTELLIGENCE?
village idiot Einstein
Einstein
village idiotmouse
chimp
recursively self-improved AI
The scale of intelligence
WHAT IS ARTIFICIENT INTELLIGENCE?
“Artificial intelligence (AI),
the ability of a digital
computer or computer-
controlled robot to perform
tasks commonly associated
with intelligent beings.”
The Encyclopedia
Britannica states
“We are on the edge of
change comparable to the
rise of human life on
Earth”
Vernor Vinge,
“TECHNOLOGICAL SINGULARITY”, 1993
Karel Capek “Rossum’s Universal Robots” (RUR), 1921
Turing Test in “Computing Machinery and Intelligence", 1950
Alan Turing
(1912 - 1954)
Dartmouth Conference: The Founding Fathers of AI, 1956
Electronic delay storage automatic calculator (EDSAC), 1952
Apple Computer 1, 1976
Neural Network-Based Autonomous Driving, 1992
Deep Blue win Kasparov, 1997
Watson' computer wins at 'Jeopardy', 2011
AlphaGo wins Lee Sedol, 2017
Credit: PWC
Credit: PWC
“The future is
scary
and very bad
for people” “AI is a ‘demon’ that
is potentially more
dangerous than
nuclear weapons ”
“I don’t
understand why
some people are
not concerned ”
“… full AI could
spell the end
of the human
race ”
WHAT PEOPLE SAY ABOUT AI
Bill Gates
Elon MuskSteve Wozniak
Stephen Hawking
Credit: BusinessInsider.com
Industries that robots will transform
by 2025
Automotive – 10% of cars will be fully autonomous and many will
drive themselves. Japan is testing “robot taxis” for transportation
during the 2020 Olympics in Tokyo.
Agriculture – Farm will increasingly use AI technology and big data
analytics to optimize crop output. More driverless tractors, drones
and milk bots.
Service – Personal robots will take on easy, dangerous or repetitive
jobs. Mowing your lawn, cleaning your windows, washing dishes.
Financial – Up to $2.2 trillion in investments will be made through AI-enabled
computers that can learn markets.
Healthcare – Robot assistance in critical surgery, elderly care, disabled patient
assistance. In 2000 there were 1, 000 robot-assisted surgeries performed, with 570, 000
in 2014.
Manufacturing – 10 % of worldwide manufacturing tasks are automated. In 10 years
that will increase to 45% as robots get cheaper.
Aerospace and Defense – 90 countries now operate drones, 1/3 are armed. The
number of commercial and military drones will triple over the next 5 years.
Autonomous military vehicles and land robots are under development.
Industries that robots will transform
by 2025
Credit: BusinessInsider.com
80 million jobs in the United States are at risk of being taken over by robots in the
next few decades, a Bank of England(BoE) official warned.
In a speech at the Trades Union Congress in
London, the bank’s chief economist,
Andy Haldane, said that up to 15 million jobs
in the U.K. were at risk of being lost to an age
of machines, which is around half of the
employed population.
ARE OUR JOBS AT RISK? YES!
Credit: Ray Kurzweil
HOW COMPANIES ARE CURRENTLY USING AI
Credit: MarketingCharts.com
HOW ANIMALS LEARN
• PAY ATTENTION
• REMEMBER WHAT
HAPPENED
• CREATE A PATTERN
IN MIND
WEAK AI STRONG AI
AI AROUND US
PEOPLE LAUGH AT AI
PEOPLE are trying to trick AI
WE ARE ALSO
NOT PERFECT
SIMPLE TEST
ONLY 1 SECOND AND ONLY 1 QUESTION:
SO, WHAT IS YOUR ANSWER?
23 PERSONS
WHAT’S THE PROBLEM?
WHO CARES IF MACHINES ARE SMARTER THAN PEOPLE?
CATS DO ☺
DO APPIUM AND SELENIUM HAVE BRAINS?
YES… YOURS ☺
AI TESTING TOOLS
Jason Arbon is the CEO at test.ai,
co-author of such books as
How Google Tests Software and
App Quality, Secrets for Agile App Teams,
20+ years of working experience in
Applause.com, Google, Microsoft
Hi Arbon… Hi Vladimir…
constwd = require( ‘wd’ );
constB = require( ‘bluebird’ );
constAPPIUM= “http://localhost:4723/wd/hub”;
constANDROID_CAPS= {
platformName: ‘Android’,
deviceName: ‘Android Emulator’,
automationName: ‘UiAutomator2’,
appActivity: ‘.app.main.HomeActivity’,
customFindModules: {ai: ‘test-ai-classifier’} ,
shouldUseCompactResponses: false,
};
describe( ‘Finding an Androidelement with machine learningmagic’ , function() {
let driver;
before( async function () {
driver = wd.promiseChainRemote(APPIUM);
await driver.init(ANDROID_CAPS);
await driver.setImplicitWaitTimeout(20000);
});
after(async function() {
If (driver) {
await driver.quit();
}
});
it( ‘should find the search button’,async function() {
await driver.elementByAccessibilityId(‘Opennavigationdrawer’);
await driver.elementByCustom(‘ai: cart ’ ).click();
await B.delay(6000);// for effect
});
});
ANY CART ICON IN ANY APPLICATION
Allows you to find an icon that’s visible in the application-under-test —as long as it
looks like one of 105 pre-trained labels like “cart”, “edit”, and “twitter
Founded in 2016
OLGA KISELEVA
www.testbase.ru
www.users.bugred.ru
DEMO
• Automatically detect whether elements of your application have
changed, and dynamically updates the tests to compensate for
those changes.
• Continuously compare test results to test history to quickly detect
changes and regressions, resulting in more stable releases.
• Helps identify and surface problems quickly, alerting you to
possible impacts before they impact your customers.
Founder & CEO @ Testim.io
OREN RUBIN
Founded in 2014
THANKS OREN ROSIE
O’BRIEN FOR HELP
Testim.io makes use of ML for the authoring, execution, and
maintenance of automated tests.
It emphasizes functional end-to-end testing and user interface
testing.
The tool becomes smarter with more runs and increases the
stability of test suites.
Testers can use JavaScript and HTML to write complex
programming logic. Credit: https://dzone.com
• Early completion of STLC
• Better coverage
• Less time and effort
• No breaks
• Increase Work Efficiency
• Reduce cost of training and operation
BENEFITS OF USING AI IN QA
RISKS OF USING AI IN QA
• High Cost
• Technology Addiction
• Loss of information
Credit: Jeremy Howard
WILL IT BE THE SAME? NO.
Credit: Jeremy Howard
how long to wait
CAST
SPEAKER
VLADIMIR ARUTIN
BRAND MANAGER
TATIANA GOLUBENKO
CREATIVE PRODUCER
VLADIMIR ARUTIN
GENERAL CAMERA OPERATOR
IVAN CHUMACHENKO
ORGANISED BY
HYS ENTERPRISE
PLACE SUPPLIER
HILLEL IT SCHOOL
LOCATION CO-ORDINATOR
VADYM DRUMOV
DIRECTED BY
VLADIMIR ARUTIN
WRITTEN BY
VLADIMIR ARUTIN
FINAL MUSIC BY
STARLIGHT ORCHESTRA
‘TERMINATOR II MAIN THEME’
SPECIAL THANKS TO
JASON ARBON
RAJ SUBRAMANIAN
MAURICE CONTI
JOE ROGAN
SAM HARRIS
OLGA KISELEVA
VADYM DRUMOV
USED VIDEO RESOURCES
https://www.youtube.com/watch?v=n-kMeXIngZE
https://www.youtube.com/watch?v=jLhOwkH5BYs&t=314s https://www.youtube.com/watch?v=ABrjdyavqkI&t=163s
https://www.youtube.com/watch?v=ycPr5-27vSI&t=162s
https://www.youtube.com/watch?v=ilP4aPDTBPE&t=81s
https://www.youtube.com/watch?v=3YLxZfZHnPg&t=522s
SOURCES
http://asimo.honda.com/
https://en.wikipedia.org/wiki/Autonomous_car
http://www.sciencemag.org/news/2017/08/how-google-making-music-artificial-intelligence
http://www.legaltechnews.com/id=1202795003827/Artificial-Intelligence-is-Molding-the-Attorney-ofthe-
Future?sqlreturn=20170709132104
http://www.newsweek.com/2017/08/18/ai-facebook-artificial-intelligence-machine-learning-robots-robotics-
646944.html
http://www.nasdaq.com/article/how-alibaba-is-using-artificial-intelligence-in-healthcare-cm815551
ANALYTIC SOURCES
www.relx.com
www.pwc.com
www.mckinsey.com
www.cdn.aiindex.org
www.crunchbase.com
www.ivc-online.com
www.engadget.com
www.sogeti.com
www.aitesting.org
www.deloitte.com
www.neurala.com
www.walkme.com
www.xponentialworks.com
www.domo.com
www.zoominsoftware.com
www.intel.com
www.trigovision.com
www.zuora.com
www.team8.vc
www.ge.com
www.outlier.ai
www.dominodatalab.com
www.emcap.com
www.nutrinohealth.com
www.auroralabs.com
www.duality.cloud
www.mprest.com
www.lending-express.com
www.vayavision.com
www.skyline.ai
www.tactilemobility.com
www.appsflyer.com
www.sweetch.com
www.mobility.here.com
www.taranis.ag
www.teradata.com
www.unbounce.com
www.globalwebindex.com
www.redwood.com
www.cognitivesystems.com
www.affectiva.com
www.pega.com
www.cloudian.com
www.qlik.com
www.gooddata.com
www.wallarm.com
www.linuxfoundation.org
www.facefirst.com
www.oxx.vc
www.techatbloomberg.com
www.finn.ai
www.ciena.com
www.logmein.com
www.miteksystems.com
www.avaamo.com
www.sap.com
www.forbes.com
ELON MUSK’S INTERVIEW WAS TAKEN FROM JOE ROGAN EXPERIENCE #1169
THIS IS A WORK OF KNOWLEDGE SHARING PRESENTATION. ANY SIMILARITY TO ORIDENTIFICATION WITH THE
LOCATIONS, CHARACTER OR HISTORY OF ANY PERSON, PRODUCT OR ENTITY IS ENTIRELY COINCIDENTAL AND
UNINTENTIONAL
NO ANIMALS WERE HARMED IN THE MAKING OF THIS PRESENTATION
QA MEETUP
ODESSA
2019
TODAY WE ARE EXPERIENCING A REBIRTH OF ARTIFICIENT INTELLIGENCE SINCE 1950s.
THEN AI HAD NO OPPORTUNITY TO GROW, LEARN, TRAIN AND BECOME STRONGER.
BUT EVERYTHING CHANGED AND WE CHANGED IT. HUMANS CREATED ZETABYTES OF INFORMATION –IMAGES, VIDEOS,
TEXTS, SOUNDS ETC. WE CREATED COUNTLESS LIBRARIES FOR AI AND GAVE IT THE OPPORTUNITY TO LEARN, TO LEARN
ITSELF, EVERYDAY, EVERY HOUR, WITHOUT ANY REST…
“…NOTHING REALLY MATTERS
ANYONE CAN SEE
NOTHING REALLY MATTERS
NOTHING REALLY MATTERS
TO ME…”
‘BOHEMIAM RHAPSODY’ , ‘QUEEN’, 1975

Artificial Intelligence In Software Testing

  • 1.
    ARTIFICIENT INTELLIGENCE IN SOFTWARETESTING VLADIMIR ARUTIN
  • 3.
    Vladimir Arutin QA Engineer inHYS Enterprise ISTQB® Certified Test Manager
  • 4.
    WHAT IS ARTIFICIENTINTELLIGENCE? village idiot Einstein Einstein village idiotmouse chimp recursively self-improved AI The scale of intelligence
  • 6.
    WHAT IS ARTIFICIENTINTELLIGENCE? “Artificial intelligence (AI), the ability of a digital computer or computer- controlled robot to perform tasks commonly associated with intelligent beings.” The Encyclopedia Britannica states
  • 7.
    “We are onthe edge of change comparable to the rise of human life on Earth” Vernor Vinge, “TECHNOLOGICAL SINGULARITY”, 1993
  • 15.
    Karel Capek “Rossum’sUniversal Robots” (RUR), 1921
  • 16.
    Turing Test in“Computing Machinery and Intelligence", 1950 Alan Turing (1912 - 1954)
  • 17.
    Dartmouth Conference: TheFounding Fathers of AI, 1956
  • 18.
    Electronic delay storageautomatic calculator (EDSAC), 1952
  • 19.
  • 20.
  • 21.
    Deep Blue winKasparov, 1997
  • 22.
    Watson' computer winsat 'Jeopardy', 2011
  • 23.
    AlphaGo wins LeeSedol, 2017
  • 25.
  • 26.
  • 27.
    “The future is scary andvery bad for people” “AI is a ‘demon’ that is potentially more dangerous than nuclear weapons ” “I don’t understand why some people are not concerned ” “… full AI could spell the end of the human race ” WHAT PEOPLE SAY ABOUT AI Bill Gates Elon MuskSteve Wozniak Stephen Hawking
  • 28.
    Credit: BusinessInsider.com Industries thatrobots will transform by 2025 Automotive – 10% of cars will be fully autonomous and many will drive themselves. Japan is testing “robot taxis” for transportation during the 2020 Olympics in Tokyo. Agriculture – Farm will increasingly use AI technology and big data analytics to optimize crop output. More driverless tractors, drones and milk bots. Service – Personal robots will take on easy, dangerous or repetitive jobs. Mowing your lawn, cleaning your windows, washing dishes.
  • 29.
    Financial – Upto $2.2 trillion in investments will be made through AI-enabled computers that can learn markets. Healthcare – Robot assistance in critical surgery, elderly care, disabled patient assistance. In 2000 there were 1, 000 robot-assisted surgeries performed, with 570, 000 in 2014. Manufacturing – 10 % of worldwide manufacturing tasks are automated. In 10 years that will increase to 45% as robots get cheaper. Aerospace and Defense – 90 countries now operate drones, 1/3 are armed. The number of commercial and military drones will triple over the next 5 years. Autonomous military vehicles and land robots are under development. Industries that robots will transform by 2025 Credit: BusinessInsider.com
  • 30.
    80 million jobsin the United States are at risk of being taken over by robots in the next few decades, a Bank of England(BoE) official warned. In a speech at the Trades Union Congress in London, the bank’s chief economist, Andy Haldane, said that up to 15 million jobs in the U.K. were at risk of being lost to an age of machines, which is around half of the employed population. ARE OUR JOBS AT RISK? YES!
  • 31.
  • 32.
    HOW COMPANIES ARECURRENTLY USING AI Credit: MarketingCharts.com
  • 33.
  • 34.
    • PAY ATTENTION •REMEMBER WHAT HAPPENED • CREATE A PATTERN IN MIND
  • 35.
  • 36.
  • 37.
  • 38.
    PEOPLE are tryingto trick AI
  • 45.
  • 46.
    SIMPLE TEST ONLY 1SECOND AND ONLY 1 QUESTION:
  • 48.
    SO, WHAT ISYOUR ANSWER? 23 PERSONS
  • 51.
    WHAT’S THE PROBLEM? WHOCARES IF MACHINES ARE SMARTER THAN PEOPLE? CATS DO ☺
  • 52.
    DO APPIUM ANDSELENIUM HAVE BRAINS? YES… YOURS ☺
  • 53.
  • 54.
    Jason Arbon isthe CEO at test.ai, co-author of such books as How Google Tests Software and App Quality, Secrets for Agile App Teams, 20+ years of working experience in Applause.com, Google, Microsoft Hi Arbon… Hi Vladimir…
  • 56.
    constwd = require(‘wd’ ); constB = require( ‘bluebird’ ); constAPPIUM= “http://localhost:4723/wd/hub”; constANDROID_CAPS= { platformName: ‘Android’, deviceName: ‘Android Emulator’, automationName: ‘UiAutomator2’, appActivity: ‘.app.main.HomeActivity’, customFindModules: {ai: ‘test-ai-classifier’} , shouldUseCompactResponses: false, }; describe( ‘Finding an Androidelement with machine learningmagic’ , function() { let driver; before( async function () { driver = wd.promiseChainRemote(APPIUM); await driver.init(ANDROID_CAPS); await driver.setImplicitWaitTimeout(20000); }); after(async function() { If (driver) { await driver.quit(); } }); it( ‘should find the search button’,async function() { await driver.elementByAccessibilityId(‘Opennavigationdrawer’); await driver.elementByCustom(‘ai: cart ’ ).click(); await B.delay(6000);// for effect }); });
  • 57.
    ANY CART ICONIN ANY APPLICATION
  • 58.
    Allows you tofind an icon that’s visible in the application-under-test —as long as it looks like one of 105 pre-trained labels like “cart”, “edit”, and “twitter
  • 59.
  • 60.
  • 61.
    • Automatically detectwhether elements of your application have changed, and dynamically updates the tests to compensate for those changes. • Continuously compare test results to test history to quickly detect changes and regressions, resulting in more stable releases. • Helps identify and surface problems quickly, alerting you to possible impacts before they impact your customers.
  • 62.
    Founder & CEO@ Testim.io OREN RUBIN Founded in 2014 THANKS OREN ROSIE O’BRIEN FOR HELP
  • 63.
    Testim.io makes useof ML for the authoring, execution, and maintenance of automated tests. It emphasizes functional end-to-end testing and user interface testing. The tool becomes smarter with more runs and increases the stability of test suites. Testers can use JavaScript and HTML to write complex programming logic. Credit: https://dzone.com
  • 64.
    • Early completionof STLC • Better coverage • Less time and effort • No breaks • Increase Work Efficiency • Reduce cost of training and operation BENEFITS OF USING AI IN QA
  • 65.
    RISKS OF USINGAI IN QA • High Cost • Technology Addiction • Loss of information
  • 66.
    Credit: Jeremy Howard WILLIT BE THE SAME? NO.
  • 67.
  • 71.
    CAST SPEAKER VLADIMIR ARUTIN BRAND MANAGER TATIANAGOLUBENKO CREATIVE PRODUCER VLADIMIR ARUTIN GENERAL CAMERA OPERATOR IVAN CHUMACHENKO ORGANISED BY HYS ENTERPRISE PLACE SUPPLIER HILLEL IT SCHOOL LOCATION CO-ORDINATOR VADYM DRUMOV DIRECTED BY VLADIMIR ARUTIN WRITTEN BY VLADIMIR ARUTIN FINAL MUSIC BY STARLIGHT ORCHESTRA ‘TERMINATOR II MAIN THEME’ SPECIAL THANKS TO JASON ARBON RAJ SUBRAMANIAN MAURICE CONTI JOE ROGAN SAM HARRIS OLGA KISELEVA VADYM DRUMOV
  • 72.
    USED VIDEO RESOURCES https://www.youtube.com/watch?v=n-kMeXIngZE https://www.youtube.com/watch?v=jLhOwkH5BYs&t=314shttps://www.youtube.com/watch?v=ABrjdyavqkI&t=163s https://www.youtube.com/watch?v=ycPr5-27vSI&t=162s https://www.youtube.com/watch?v=ilP4aPDTBPE&t=81s https://www.youtube.com/watch?v=3YLxZfZHnPg&t=522s SOURCES http://asimo.honda.com/ https://en.wikipedia.org/wiki/Autonomous_car http://www.sciencemag.org/news/2017/08/how-google-making-music-artificial-intelligence http://www.legaltechnews.com/id=1202795003827/Artificial-Intelligence-is-Molding-the-Attorney-ofthe- Future?sqlreturn=20170709132104 http://www.newsweek.com/2017/08/18/ai-facebook-artificial-intelligence-machine-learning-robots-robotics- 646944.html http://www.nasdaq.com/article/how-alibaba-is-using-artificial-intelligence-in-healthcare-cm815551 ANALYTIC SOURCES www.relx.com www.pwc.com www.mckinsey.com www.cdn.aiindex.org www.crunchbase.com www.ivc-online.com www.engadget.com www.sogeti.com www.aitesting.org www.deloitte.com www.neurala.com www.walkme.com www.xponentialworks.com www.domo.com www.zoominsoftware.com www.intel.com www.trigovision.com www.zuora.com www.team8.vc www.ge.com www.outlier.ai www.dominodatalab.com
  • 73.
  • 74.
    ELON MUSK’S INTERVIEWWAS TAKEN FROM JOE ROGAN EXPERIENCE #1169 THIS IS A WORK OF KNOWLEDGE SHARING PRESENTATION. ANY SIMILARITY TO ORIDENTIFICATION WITH THE LOCATIONS, CHARACTER OR HISTORY OF ANY PERSON, PRODUCT OR ENTITY IS ENTIRELY COINCIDENTAL AND UNINTENTIONAL NO ANIMALS WERE HARMED IN THE MAKING OF THIS PRESENTATION QA MEETUP ODESSA 2019 TODAY WE ARE EXPERIENCING A REBIRTH OF ARTIFICIENT INTELLIGENCE SINCE 1950s. THEN AI HAD NO OPPORTUNITY TO GROW, LEARN, TRAIN AND BECOME STRONGER. BUT EVERYTHING CHANGED AND WE CHANGED IT. HUMANS CREATED ZETABYTES OF INFORMATION –IMAGES, VIDEOS, TEXTS, SOUNDS ETC. WE CREATED COUNTLESS LIBRARIES FOR AI AND GAVE IT THE OPPORTUNITY TO LEARN, TO LEARN ITSELF, EVERYDAY, EVERY HOUR, WITHOUT ANY REST… “…NOTHING REALLY MATTERS ANYONE CAN SEE NOTHING REALLY MATTERS NOTHING REALLY MATTERS TO ME…” ‘BOHEMIAM RHAPSODY’ , ‘QUEEN’, 1975