2014-04-22 Ques #4 Automation Testing of Mobage Platform
1. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
1
2014/04/22
!
Masaki Nakagawa
Manager, SWET(Software Engineer in Test) Gr.
Quality Management Dept.
System Management Unit
DeNA Co., Ltd.
Mobageオープンプラットフォーム
でのテスト自動化
2. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
About Me
• Masaki Nakagawa
• @ikasam_a
• github.com/masaki
• CPAN Author (metacpan.org/author/MASAKI)
• Testing Casual
3. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://gihyo.jp/magazine/wdpress/archive/2013/vol75
4. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://gihyo.jp/magazine/wdpress/archive/2013/vol77
5. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://atnd.org/events/40914
6. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage統合事業本部
Japanリージョン事業本部
プラットフォーム本部
システム部
SWETグループ
6
7. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
システム本部
品質管理部
SWETグループ
(2014/04 )
7
8. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
今日話すこと
• Mobageオープンプラットフォーム概要
• SWET って何?
• プラットフォームのテスト自動化戦略
8
9. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobageオープンプラットフォーム概要
9
10. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage Open Platform とは?
• http://developer.dena.jp/mbga/
• Mobageでゲームを公開できる仕組み
• 色々なデバイスに対応
• Feature Phone
• Smartphone
• PC
10
11. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobage Open Platform の歴史
• 2009/08 Open Platform Project スタート
• 2010/01 JP Platform (Feature Phone)
• 2010/10 JP Platform (Y!Mobage / PC)
• 2011/02 JP Platform (Smartphone Web)
• 2011/06 JP Platform (Smartphone App)
• 2012/12 JP Platform (ShellApp)
11
19. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://en.wikipedia.org/wiki/SET
20. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
a
Quality Assurance
job title
21. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Names
• SET (Software Engineer in Test) by @Google
• SDET (Software Design/Development Engineer in Test) by @Microsoft
• SWET by @DeNA
22. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Definition
•Google Testing Blog
•http://googletesting.blogspot.com
•How Google Tests Software
•http://googletesting.blogspot.jp/2011/01/how-google-tests-
software.html
23. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://googletesting.blogspot.jp/2011/02/how-google-
tests-software-part-two.html
24. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
the SET
•a developer role focused on testability
•review designs
•refactor code to testable
•write testing frameworks
25. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://googletesting.blogspot.jp/2011/02/how-google-
tests-software-part-two.html
26. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
the TE
•a role that puts testing first
•write automation scripts
•interpret test results
•drive test execution
27. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://googletesting.blogspot.jp/2011/02/how-google-
tests-software-part-two.html
28. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Focus
•SETs primary focus is on the developer
•Individual feature quality
•Enabling developers to easily test
29. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Developer
Productivity
30. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
our SWET = SET + TE
• Developer Productivity
• Quality Assurance
31. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
SWET Group
Mission Statement
•Keep the quality of Services and
Products
•Improve the quality and
productivity of Services and
Products
32. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
As an Engineer
•can be developed test target
•NOT a tester but a test engineer
33. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
プラットフォームのテスト自動化戦略
33
34. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Our Works
• Server Side Testing
• Client Side Testing
• …
35. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Open Platform Architecture
API
Mobage Developers
Databases / Middlewares
Mobage
RESTful API
API
ProxyServer
API
mbga.jp
Mobile
Browser
PC
Browser
Mobile
App
Mobage
SDK
Game
Server
36. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Targets
1. Mobage RESTful API (Web API)
2. Mobage Developers (Web Application)
3. mbga.jp & ProxyServer (Mobile Web)
4. Mobage SDKs (Client SDK)
37. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Open Platform Architecture
API
Mobage Developers
Databases / Middlewares
Mobage
RESTful API
API
ProxyServer
API
mbga.jp
Mobile
Browser
PC
Browser
Mobile
App
Mobage
SDK
Game
Server
38. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
プラットフォームのテスト戦略
• 適切なシステム分割をする
• システムビッグバンを避ける
• サーバで完結できるところはサーバで
• WebAPIの機能テストはクライアント無しで
• ブラウザを使う部分は極力実機レスで
38
39. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
スマートフォンテスト自動化
• テスト対象によってテスト方法も異なる
• システム分類
• WebAPI, Webアプリ
• ネイティブアプリ, SDK(ライブラリ)
39
40. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
スマートフォンブラウザ
• ブラウザ自動化の技術をそのまま使う
• Selenium WebDriver
• スマートフォンブラウザを偽装
• テストコードを流用して実機でも動かす
• Appium (w/ JSON Wire Protocol)
40
41. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Selenium WebDriver
• Capability でドライバを選択
41
FirefoxProfile fp = new FirefoxProfile();
DesiredCapabilities dc = DesiredCapabilities.firefox();
dc.setCapability(FirefoxDriver.PROFILE, fp);
WebDriver driver = new RemoteWebDriver(dc);
ChromeOptions options = new ChromeOptions();
DesiredCapabilities dc = DesiredCapabilities.chrome();
dc.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new RemoteWebDriver(dc);
42. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
ネイティブアプリ
• アプリの操作を自動化する
• 外部からプロセスにアタッチして操作
• Appium を使う
• テスト用ライブラリを埋め込んで操作
• Calabash を使う
• ただしアプリ改変になる
42
43. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
クライアントライブラリ(SDK)
• ライブラリを組み込んだテストアプリを作成
• ライブラリの機能を網羅できるように作る
• テストアプリの操作を自動化
• あとはネイティブアプリと同様
43
44. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
44
API Server
module module module
API ServerAPI Server
DB
API ServerAPI Server
Cache
Test Client
A) WebAPI
45. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Conditions
• HTTP Request to API Server
• JSON Response from API Server
• no UI
46. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
API Server
module module module
API ServerAPI Server
DB
API ServerAPI Server
Cache
Test Client
Gray Box Fixture
47. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Gray Box Fixture
• DB / Cache Manipulation
• insert test-case specific data
• delete cache for continuous tests
48. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
API ServerAPI ServerAPI Server
module module module
API ServerAPI Server
DB
API ServerAPI Server
Cache
Test Client
Specific Targeting
49. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Specific Targeting
• programmable local DNS (MyDNS)
• emulates “inet_aton”
50. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Domain Specific Client
• Mobage::API::Client (Perl)
• mobage-api-client (Ruby)
51. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Message Dumping
• keep previous request in response
• interactive debugging
52. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Web Application
M V C
API ServerAPI Server
DB
API ServerAPI Server
Cache
Test Browser
B) WebApp / Mobile Web
53. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Conditions
• HTTP Request to Web App
• HTML Response from Web App
• with JavaScript
54. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Basic Strategy
• Selenium WebDriver
• Ruby, RSpec, Capybara
• Headless Browser
55. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Browser Emulator
Web Application
M V C
API ServerAPI Server
DB
API ServerAPI Server
Cache
Agent Switching
56. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Agent Switching
• Browser can’t touch HTTP Header
• Emulator can’t parse JavaScript
• Save/Restore pseudo agent state
57. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Domain Specific Client
• called “mobage-browser”
• We don’t use “Page Object Pattern”
• “Domain Agent Pattern”
58. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Mobile Emulation
• FireMobileSimulator
• Mobile WebDrivers (DEPRECATED)
• AndroidDriver, IPhoneDriver
• with Real Devices
• Selendroid, ios-driver, Appium
59. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Test Scripts
C) Client SDKs
Test
Application
Mobage
SDK
60. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
Conditions
• create test apps using specific SDK
• use SDK as a developer
• run test scripts to automate test apps
• check SDK-API behaviors
61. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
UI Automation
• Multiple Devices Support
• Android, iOS
• Test Cases Consistency
• Calabash
• Appium
62. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
http://gihyo.jp/magazine/wdpress/archive/2013/vol77
63. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.
今日話したこと
• Mobageオープンプラットフォーム概要
• SWET って何?
• プラットフォームのテスト自動化戦略
63