2014-04-22 Ques #4 Automation Testing of Mobage Platform

3,595 views
3,511 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,595
On SlideShare
0
From Embeds
0
Number of Embeds
1,967
Actions
Shares
0
Downloads
17
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

2014-04-22 Ques #4 Automation Testing of Mobage Platform

  1. 1. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. 1 2014/04/22 ! Masaki Nakagawa Manager, SWET(Software Engineer in Test) Gr. Quality Management Dept. System Management Unit DeNA Co., Ltd. Mobageオープンプラットフォーム でのテスト自動化
  2. 2. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. About Me • Masaki Nakagawa • @ikasam_a • github.com/masaki • CPAN Author (metacpan.org/author/MASAKI) • Testing Casual
  3. 3. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://gihyo.jp/magazine/wdpress/archive/2013/vol75
  4. 4. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://gihyo.jp/magazine/wdpress/archive/2013/vol77
  5. 5. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://atnd.org/events/40914
  6. 6. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Mobage統合事業本部 Japanリージョン事業本部 プラットフォーム本部 システム部 SWETグループ 6
  7. 7. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. システム本部 品質管理部 SWETグループ (2014/04 ) 7
  8. 8. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. 今日話すこと • Mobageオープンプラットフォーム概要 • SWET って何? • プラットフォームのテスト自動化戦略 8
  9. 9. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Mobageオープンプラットフォーム概要 9
  10. 10. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Mobage Open Platform とは? • http://developer.dena.jp/mbga/ • Mobageでゲームを公開できる仕組み • 色々なデバイスに対応 • Feature Phone • Smartphone • PC 10
  11. 11. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Mobage Open Platform の歴史 • 2009/08 Open Platform Project スタート • 2010/01 JP Platform (Feature Phone) • 2010/10 JP Platform (Y!Mobage / PC) • 2011/02 JP Platform (Smartphone Web) • 2011/06 JP Platform (Smartphone App) • 2012/12 JP Platform (ShellApp) 11
  12. 12. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Mobage Open Platform の歴史 • 2011/07 WW Platform (Smartphone App) • 2011/07 CN Platform (Smartphone App) • 2012/02 KR Platform (Smartphone App) • 2012/11 KR Platform (ShellApp) • 2012/11 TC Platform (Smartphone App) • 2013/01 CN Platform (ShellApp) • 2013/02 KR Platform (Smartphone Web) 12
  13. 13. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. 13 プラットフォームのグローバル展開
  14. 14. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. SWETグループ立ち上げの背景 • プラットフォームのグローバル展開 • 大規模システムの拡張とリファクタリング • デリバリーのスピードを落とさない • 検証属人性の解消 14
  15. 15. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. SWETグループ立ち上げの方針 • End-to-End テストを確立する • テストを徹底的に自動化する • テストしやすい環境を提供する 15
  16. 16. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. ∼テストしやすい環境∼ • 単体テストのREDが消えない問題 • リリース頻度・速度・影響範囲のバランス • テスト時間のコスト問題 • CIの必要性 16
  17. 17. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. チーム立ち上げ • 当初はQAチームとして出発 • 2012/01 当時は3人でスタート • 現在はSWETグループ(正式に組織化) • 2014/04 現在は20名弱で活動中 • なぜ独立したチームにしたか? • 横串チームによる「戦略的横展開」を狙う 17
  18. 18. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. SWET って何? 18
  19. 19. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://en.wikipedia.org/wiki/SET
  20. 20. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. a Quality Assurance job title
  21. 21. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Names • SET (Software Engineer in Test) by @Google • SDET (Software Design/Development Engineer in Test) by @Microsoft • SWET by @DeNA
  22. 22. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Definition •Google Testing Blog •http://googletesting.blogspot.com •How Google Tests Software •http://googletesting.blogspot.jp/2011/01/how-google-tests- software.html
  23. 23. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://googletesting.blogspot.jp/2011/02/how-google- tests-software-part-two.html
  24. 24. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. the SET •a developer role focused on testability •review designs •refactor code to testable •write testing frameworks
  25. 25. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://googletesting.blogspot.jp/2011/02/how-google- tests-software-part-two.html
  26. 26. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. the TE •a role that puts testing first •write automation scripts •interpret test results •drive test execution
  27. 27. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://googletesting.blogspot.jp/2011/02/how-google- tests-software-part-two.html
  28. 28. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Focus •SETs primary focus is on the developer •Individual feature quality •Enabling developers to easily test
  29. 29. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Developer Productivity
  30. 30. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. our SWET = SET + TE • Developer Productivity • Quality Assurance
  31. 31. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. SWET Group Mission Statement •Keep the quality of Services and Products •Improve the quality and productivity of Services and Products
  32. 32. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. As an Engineer •can be developed test target •NOT a tester but a test engineer
  33. 33. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. プラットフォームのテスト自動化戦略 33
  34. 34. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Our Works • Server Side Testing • Client Side Testing • …
  35. 35. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Open Platform Architecture API Mobage Developers Databases / Middlewares Mobage RESTful API API ProxyServer API mbga.jp Mobile Browser PC Browser Mobile App Mobage SDK Game Server
  36. 36. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Targets 1. Mobage RESTful API (Web API) 2. Mobage Developers (Web Application) 3. mbga.jp & ProxyServer (Mobile Web) 4. Mobage SDKs (Client SDK)
  37. 37. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Open Platform Architecture API Mobage Developers Databases / Middlewares Mobage RESTful API API ProxyServer API mbga.jp Mobile Browser PC Browser Mobile App Mobage SDK Game Server
  38. 38. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. プラットフォームのテスト戦略 • 適切なシステム分割をする • システムビッグバンを避ける • サーバで完結できるところはサーバで • WebAPIの機能テストはクライアント無しで • ブラウザを使う部分は極力実機レスで 38
  39. 39. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. スマートフォンテスト自動化 • テスト対象によってテスト方法も異なる • システム分類 • WebAPI, Webアプリ • ネイティブアプリ, SDK(ライブラリ) 39
  40. 40. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. スマートフォンブラウザ • ブラウザ自動化の技術をそのまま使う • Selenium WebDriver • スマートフォンブラウザを偽装 • テストコードを流用して実機でも動かす • Appium (w/ JSON Wire Protocol) 40
  41. 41. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Selenium WebDriver • Capability でドライバを選択 41 FirefoxProfile fp = new FirefoxProfile(); DesiredCapabilities dc = DesiredCapabilities.firefox(); dc.setCapability(FirefoxDriver.PROFILE, fp); WebDriver driver = new RemoteWebDriver(dc); ChromeOptions options = new ChromeOptions(); DesiredCapabilities dc = DesiredCapabilities.chrome(); dc.setCapability(ChromeOptions.CAPABILITY, options); WebDriver driver = new RemoteWebDriver(dc);
  42. 42. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. ネイティブアプリ • アプリの操作を自動化する • 外部からプロセスにアタッチして操作 • Appium を使う • テスト用ライブラリを埋め込んで操作 • Calabash を使う • ただしアプリ改変になる 42
  43. 43. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. クライアントライブラリ(SDK) • ライブラリを組み込んだテストアプリを作成 • ライブラリの機能を網羅できるように作る • テストアプリの操作を自動化 • あとはネイティブアプリと同様 43
  44. 44. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. 44 API Server module module module API ServerAPI Server DB API ServerAPI Server Cache Test Client A) WebAPI
  45. 45. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Conditions • HTTP Request to API Server • JSON Response from API Server • no UI
  46. 46. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. API Server module module module API ServerAPI Server DB API ServerAPI Server Cache Test Client Gray Box Fixture
  47. 47. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Gray Box Fixture • DB / Cache Manipulation • insert test-case specific data • delete cache for continuous tests
  48. 48. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. API ServerAPI ServerAPI Server module module module API ServerAPI Server DB API ServerAPI Server Cache Test Client Specific Targeting
  49. 49. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Specific Targeting • programmable local DNS (MyDNS) • emulates “inet_aton”
  50. 50. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Domain Specific Client • Mobage::API::Client (Perl) • mobage-api-client (Ruby)
  51. 51. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Message Dumping • keep previous request in response • interactive debugging
  52. 52. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Web Application M V C API ServerAPI Server DB API ServerAPI Server Cache Test Browser B) WebApp / Mobile Web
  53. 53. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Conditions • HTTP Request to Web App • HTML Response from Web App • with JavaScript
  54. 54. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Basic Strategy • Selenium WebDriver • Ruby, RSpec, Capybara • Headless Browser
  55. 55. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Browser Emulator Web Application M V C API ServerAPI Server DB API ServerAPI Server Cache Agent Switching
  56. 56. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Agent Switching • Browser can’t touch HTTP Header • Emulator can’t parse JavaScript • Save/Restore pseudo agent state
  57. 57. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Domain Specific Client • called “mobage-browser” • We don’t use “Page Object Pattern” • “Domain Agent Pattern”
  58. 58. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Mobile Emulation • FireMobileSimulator • Mobile WebDrivers (DEPRECATED) • AndroidDriver, IPhoneDriver • with Real Devices • Selendroid, ios-driver, Appium
  59. 59. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Test Scripts C) Client SDKs Test Application Mobage SDK
  60. 60. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. Conditions • create test apps using specific SDK • use SDK as a developer • run test scripts to automate test apps • check SDK-API behaviors
  61. 61. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. UI Automation • Multiple Devices Support • Android, iOS • Test Cases Consistency • Calabash • Appium
  62. 62. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. http://gihyo.jp/magazine/wdpress/archive/2013/vol77
  63. 63. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. 今日話したこと • Mobageオープンプラットフォーム概要 • SWET って何? • プラットフォームのテスト自動化戦略 63
  64. 64. Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved. ご清聴ありがとうございました 64

×