Snapshot Regression Testースナップショットを用いたリグレッション自動テスト

Unity Technologies Japan K.K.

データのスナップショットを使って自動化の効率を改善した話

Snapshot Regression Test
ースナップショットを用いたリグレッション自動テスト
2020/6/30
Satoru Awasawa
Travel Agile Dept. QA Section
Rakuten, Inc.

2
Agenda
Introduction
自己紹介
Rakuten Travel 使ってみてください！
テスト自動化
自動テストの問題って？
じゃーこうすればどうかな？
アーキテクチャー紹介
デモ

3
Satoru Awasawa
Java Developer + System Architect
7 years in JAPAN AIRLINE system company
5 years in Rakuten Travel ‘Backend API’
Tech Manager
2 years in Rakuten Travel ‘Backend API’
QA Senior Manager
1 year before coming to QA!
As of now managing both manual testing and automation testing
My hobby
Splatoon2(Nintendo-switch)
Golf (Not virtual but real!)

4
What’s ‘Rakuten Travel’?
https://travel.rakuten.co.jp/ (2020/3/22)

5
What’s ‘Rakuten Travel’?
Domesti
c Hotels
Overseas
hotels
JAL DP
ANA DP
Overseas
DP
AIR
Bus
Rent-a-
car
https://travel.rakuten.com/ (2020/6/18)
https://travel.rakuten.co.jp/ (SP)
(2020/6/18)

6
What’s ‘Rakuten Travel’?
https://www.mlit.go.jp/report/press/content/001339698.pdf (国土交通省HPより) (2020/6/18)
©️いらすとや

7
What’s the problem of automation test?
SCENARIO EXPECTED
VALUE
TEST DATA

9
Effort Comparison for Test Execution (1000 Test Cases)
0
83
833
1666
500 525
750
1000
0
200
400
600
800
1000
1200
1400
1600
1800
Initial 1time 10times 20times
Man*Day（人日）
Manual Test
Automation Test

10
with Getting the snapshot
New automation test
framework “SRT”
https://publicdomainq.net/camera-taking-picture-0019733/

11
New Automation Testing ‘SRT’
Manual Tester
Data
Req•
Res•
API
Req•
Res•
APIHTTP
©️いらすとや

12
New Automation Testing ‘SRT’
Manual Tester
Data
Req•
Res•
API
Req•
Res•
APIHTTP
Data
Req•
Res•
Req•
Res•
HTTP
Snapshot
©️いらすとや

13
New Automation Testing ‘SRT’
Manual Tester
Data
Req•
Res•
API
• Req
• Res
APIHTTP
Data
Req•
Res•
Req•
Res•
HTTP
Snapshot
Get the RQ+RS set snapshot with
traceID and stacked it as regression
test cases
At the same time, Get the Data set
snapshot.
©️いらすとや

14
New Automation Testing ‘SRT’
Developer
Data
Req•
Res•
API
Req•
Res•
APIHTTP
Data
Req•
Res•
Req•
Res•
HTTP
Snapshot
©️いらすとや

15
New Automation Testing ‘SRT’
Developer
Data
Req•
Res•
API
• Req
• Res
APIHTTP Data
Req•
Res•
Req•
Res•
HTTP
Snapshot©️いらすとや

16
New Automation Testing ‘SRT’
Developer
Data
Req•
Res•
API
Req•
Res•
APIHTTP Data
Req•
Res•
Req•
Res•
HTTP
Snapshot©️いらすとや

17
SRT Architecture (TestCase Creation)
Manual Tester
Capture
Start
Take Data
Snapshot
Mongo Oracle
Mongo Oracle
Log Analyzer
Preparation
Execution <API RQ/RS Data>
<Test Id> <Transaction Id>
Test DB
<Test Specification>
TestID
RQ+RS
DataSet
App Client
STG
Proxy
©️いらすとや

18
SRT Architecture (Test Execution)
Developer
Restore
Data
Snapshot
Mongo Oracle
Commit
Test DB
<Test Specification>
TestID
RQ+RS
DataSet
QA
Test
Executer
Build/Deploy
CICD©️いらすとや

19
Demo
Recordingthe scenario which is tested by manual tester
Execution with updated data
Execution with wrong code

21
Effort Comparison for Test Execution (1000 Test Cases)
0
83
833
1666
500 525
750
1000
83 84 93 103
0
200
400
600
800
1000
1200
1400
1600
1800
Initial 1time 10times 20times
Man*Day（人日）
Manual Test
Automation Test
New AT 'SRT'

Snapshot Regression Testースナップショットを用いたリグレッション自動テスト

講演アーカイブ：https://www.youtube.com/playlist?list=PLr_Cbd4sUDTzxG9a4TrU0Rg6-ApCJBCsI 講演内容：ふつうのSEがVRとUnreal Engineと出会い、どのように仕事にしていったのか。市場でのXR + UE4活用事例を紹介するとともに、開発中のVRワークプレイスについて紹介します講演者：前鼻毅 (株式会社リコー AR-PT RxRチームリーダー) UNREAL FEST EXTREME 2020 WINTER 再生リスト https://www.youtube.com/playlist?list=PLr_Cbd4sUDTzxG9a4TrU0Rg6-ApCJBCsI https://unrealengine.jp/unrealfest/timetable/ #ue4fest

Bpstudy ajnreviewKazunori Sato

ASP.NET MVC - Javascript Overview

Dimitar Danailov

【Unite 2017 Tokyo】みんなのスマートフォンでここまでできる〜Kudan ARを使ったモバイルAR/MRの世界

Visual Validation - The Missing Tip of the Automation Pyramid

Anand Bagmar

The Test Automation Pyramid is not a new concept. The top of the pyramid is our UI / end-2-end functional tests - which should cover the breadth of the product. What the functional tests cannot capture though, is the aspects of UX validations that can only be seen and in some cases, captured by the human eye. This is where the new buzzwords of AI & ML can truly help. In this session, we will explore why Visual Validation is an important cog in the wheel of Test Automation and also different tools and techniques that can help achieve this. We will also see a demo of Applitools Eyes - and how it can be a good option to close this gap in automation!

Computer Vision Based Driver Monitoring, Assisting and Grading System for Dri...

Full webinar recording here: https://youtu.be/vx34FJYdp9o ** In this talk, Sr. Architect Gil Tayar gives a comprehensive overview of visual testing, discuss why pixel comparison is not enough, detail some of the challenges of visual testing, and review how current visual testing tools offer a solution to these challenges ** Unit tests, component tests, automation tests, acceptance tests: all the various kinds of tests check that an application works as it should, and functions as expected. But until now, testing how an app looks like was delegated to manual testing. When all you had was a desktop or web app, that was almost acceptable: there was a lot of application to check, but only one form factor. But in these days of multiple form factors -- desktops of various widths, mobile, tablet -- automation is a necessity. But how does one go about checking how an app looks? Comparing pixels against a baseline yields too many false positives, and any small change in your app will drown you in failed tests that you need to fix manually. Visual Testing offers a solution to the problem. Using online services and algorithms taken from the world of AI, Visual Testing offers a platform that enables you to finally ensure that your application looks like you expect it to. Key takeaway: * Why functional testing is not enough * Challenges of testing apps in a multiple-form-factor world * How current visual testing tools overcome the challenges * The place of Visual Testing in your tests * How to write visual tests

From Selenium to Appium: How Hard Can It Be? by Sergio Neves Barros

Sauce Labs

Testers have all heard of Selenium and have used it to test web sites. It’s the de facto standard in writing automated tests and most browsers have either incorporated drivers into their builds or provided separate drivers that allow Selenium to interact with the browser. But what about Appium? It’s using the Selenium JSON wire protocol, so users should just be able to point their tests at an Appium server, right? During this SauceCon 2018 talk, Sergio Neves Barros will discuss the (historical) challenges of mobile web testing, platform differences between Appium and its drivers and Selenium, some of the additional endpoints/features Appium provides, the common features between Appium and Selenium, and discuss the future of mobile testing.

Designing Autonomous Car using OpenCV and Machine Learning

Test automationatrakutentravel

ClintCayetano

製作 Unity Plugin for Android

Johnny Sung

Bootiful Development with Spring Boot and Angular - RWX 2018

To simplify development and deployment, you want everything in the same artifact, so you put on your Angular app “inside” and your Spring Boot app, right? But what if you could create your Angular app as a standalone app and make cross-origin requests to your API? This session shows how to develop with Java 11, Spring Boot, Angular 7, and TypeScript. You'll learn how to create REST endpoints with Spring MVC, Spring Data REST, configure Spring Boot to allow CORS, and create an Angular app to display its data. Blog: https://developer.okta.com/blog/2017/04/26/bootiful-development-with-spring-boot-and-angular GitHub: https://github.com/oktadeveloper/spring-boot-angular-example Screencast: https://www.youtube.com/watch?v=GhBwKT7EJsY

Appium workshop technopark trivandrum

Syam Sasi

http://www.techforum.prathidhwani.org/appium/index.html Appium Basics Appium Desktop Android Locator Types and Inspector Tools Appium Capabilities Mobile Interactions - Swipe and Scroll Page Object Model and Dependency Injection Real Device v/s Emulators / Simulators iOS Automation Cloud Automation Appium Tips and Tricks Please follow the steps in the below link to configure your system for the workshop https://medium.com/@syamsasi/setting-up-appium-on-windows-and-ubuntu-ea9a73ab989

[Azure Council Experts (ACE) 第37回定例会] Microsoft Azureアップデート情報 (2019/08/22-201...

Naoki (Neo) SATO

Microsoft Azure 最新アップデート情報 http://a-c-e.biz/activity/update.html https://satonaoki.wordpress.com/tag/azureupdate/ http://www.slideshare.net/search/slideshow?q=azureupdate [Azure Council Experts (ACE) 第37回定例会] Microsoft Azureアップデート情報 (2019/08/22-2019/10/18) https://satonaoki.wordpress.com/2019/10/21/ace-azure-update-37/ https://www.slideshare.net/satonaoki/20191018-ace-azure-update https://speakerdeck.com/satonaoki/18-7f13f446-fe8f-400c-a0db-7da5fb943305

20200613_勝手にLT_オンラインのSAP関連情報

MasashiOtsuka1

Встреча Google Post IO ( Владимир Иванов, Катерина Заворотченко и Сергей Комлач)

Alina Vilk

Sharada_ResumeSharada Nandikolmat

IRJET - Obstacle Detection using a Stereo Vision of a Car

Microservices for the Masses with Spring Boot, JHipster, and OAuth - Switzerl...

Microservices are being deployed by many Java Hipsters. If you're working with a large team that needs different release cycles for product components, microservices can be a blessing. If you're working at your VW Restoration Shop and running its online store with your own software, having five services to manage and deploy can be a real pain. This presentation will show you how to use JHipster to create Angular + Spring Boot apps with a unified front-end. You will leave with the know-how to create your own excellent apps! Related blog posts: * Java Microservices with Spring Boot and Spring Cloud: https://developer.okta.com/blog/2019/05/22/java-microservices-spring-boot-spring-cloud * Java Microservices with Spring Cloud Config and JHipster: https://developer.okta.com/blog/2019/05/23/java-microservices-spring-cloud-config * Secure Reactive Microservices with Spring Cloud Gateway: https://developer.okta.com/blog/2019/08/28/reactive-microservices-spring-cloud-gateway

Driver Drowsiness Detection System using Google ML Kit Face Detection API and...

Resume subbu addanki

Subbu Addanki

IRJET- Driver Monitoring System and Smart Vehicle

[Rakuten TechConf2014] [B-6] Rakuten Travel Architecture and Development Process

Motion capture for Animation

コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話

CI/CDツールであるJenkinsとIDEであるIntelliJ IDEA。両者の共通点は、Javaで書かれている点と、プラグインによって機能を拡張できる点です。つまり、Javaを用いてプラグインを実装することで、自分が欲しい機能を追加することができます。本セッションでは、コードレビュープロセスを改善するためにJenkinsとIntelliJ IDEAのプラグインを自作した経験を踏まえて、実装方法に触れつつ、得られた結果や苦労した点について話します。

楽天における安全な秘匿情報管理への道のり

Similar to Snapshot Regression Testースナップショットを用いたリグレッション自動テスト

[RakutenTechConf2013] [E-2] HTML5 in Rakuten

Selenium to Appium - how hard can it be (SauceCon).

snevesbarros

Visual Testing: The Missing Piece of the Puzzle -- presentation by Gil Tayar

Applitools

From Selenium to Appium: How Hard Can It Be? by Sergio Neves Barros

Sauce Labs

Designing Autonomous Car using OpenCV and Machine Learning

Test automationatrakutentravel

ClintCayetano

製作 Unity Plugin for Android

Johnny Sung

Bootiful Development with Spring Boot and Angular - RWX 2018

Appium workshop technopark trivandrum

Syam Sasi

[Azure Council Experts (ACE) 第37回定例会] Microsoft Azureアップデート情報 (2019/08/22-201...

Naoki (Neo) SATO

20200613_勝手にLT_オンラインのSAP関連情報

MasashiOtsuka1

Встреча Google Post IO ( Владимир Иванов, Катерина Заворотченко и Сергей Комлач)

Alina Vilk

Sharada_ResumeSharada Nandikolmat

IRJET - Obstacle Detection using a Stereo Vision of a Car

Microservices for the Masses with Spring Boot, JHipster, and OAuth - Switzerl...

Driver Drowsiness Detection System using Google ML Kit Face Detection API and...

Resume subbu addanki

Subbu Addanki

IRJET- Driver Monitoring System and Smart Vehicle

[Rakuten TechConf2014] [B-6] Rakuten Travel Architecture and Development Process

Motion capture for Animation