Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Uncovering breaking changes behind UI on mobile applications

255 views

Published on

A talk about http request capture at AppiumConf 2019

Published in: Software
  • Be the first to comment

  • Be the first to like this

Uncovering breaking changes behind UI on mobile applications

  1. 1. Uncovering breaking changes behind UI on mobile applications HeadSpin Senior Software Engineer, Device Automation
  2. 2. Hi✋ • Kazuaki Matsuo • from ", GitHub: KazuCocoa, Twitter: @Kazu_cocoa • HeadSpin • Senior Software Engineer now, former was a test/QA engineer • Maintain Appium project
  3. 3. Topics • Case study: • Combination of capturing HTTP requests over Appium scenarios • Tips: • Some tips for Appium
  4. 4. Test Target
  5. 5. App: Long running app • Many historical code • Multiple teams developed one app • Automated test environment was not matured enough • Both Android and iOS
  6. 6. Situation: Collecting user flow • Tracking user behaviour is important • evaluate efforts by features, campaigns, etc • Should be reliable • The data was wrong case happened after releasing a new version • Unreliable data • Monitoring it, get rid of them from the analytics
  7. 7. User events, flow Open the app Click the button Go to the view
  8. 8. Events to track actions { “events”: [ { “name”: “OpenAnAppFromLauncher”, “time”: ”2019-06-09 13:39:04 +0900” # time series to track user actions }, { …. } ] }
  9. 9. “Open an app” Open an app
  10. 10. Detect the action Open an app
  11. 11. Store the log on the app Store the log { “events”: [ { “name”: “OpenAnAppFromLauncher”, “time”: ”2019-06-09 13:39:04 +0900” }
 ] }
  12. 12. Send the log to the server Store the log Send to the stored log to server
  13. 13. UI and non-UI • UI layer • User actions • non-UI layer • Store the data into local • Send it to servers UI no UI
  14. 14. “behind UI” • Behind UI in this title means such non- UI layer • Event logs • HTTP requests • Too many event logs/few event logs unexpectedly
  15. 15. Might break non-UI layer cases
  16. 16. Android • Android has two main component to build UIs • Activity • Base view • Fragment • Re-useable view component
  17. 17. Lifecycle: Activity & Fragment Activity Fragment
  18. 18. Before they are active Activity Fragment
  19. 19. Where we set logs? Activity Fragment
  20. 20. Easy to mistake • “Open”, from where? • Open the app from icon • Open the app from app history • Open the app from notification bar • Back button and go to the app view from another app • They would like to define user flow • We must define them correctly
  21. 21. From App Icon? • “Open”, from where? • Open the app from icon • Open the app from app history • Open the app from notification bar • Back button and go to the app view from another app • They would like to define user flow • We must define them correctly
  22. 22. From App History? • “Open”, from where? • Open the app from icon • Open the app from app history • Open the app from notification bar • Back button and go to the app view from another app • They would like to define user flow • We must define them correctly
  23. 23. From App Notification Bar? • “Open”, from where? • Open the app from icon • Open the app from app history • Open the app from notification bar • Back button and go to the app view from another app • They would like to define user flow • We must define them correctly
  24. 24. From Back Button? • “Open”, from where? • Open the app from icon • Open the app from app history • Open the app from notification bar • Back button and go to the app view from another app • They would like to define user flow • We must define them correctly
  25. 25. Where we set logs, again Activity Fragment
  26. 26. Fragment • Fragment is re-usable component • Unexpected usage: • A developer put a log into a fragment, for view A • Another developer uses the fragment in another view, for view B, while a log defined in the fragment should not be reused on view B • What happen if a user open the view A and B? • The log event happens on both view A and view B • The data will break since the log should be called only on view A View A View B
  27. 27. iOS
  28. 28. Change their implemented languages • Developers change their implementation languages • Objective-C -> Swift • Re-write to Swift, the behaviour should not change
  29. 29. Both
  30. 30. Prototype, Implement more • prototype/minimal implementation • Evaluate the value • It is worth • Re-write/refactor/add new features • Do not change logs to make the data comparable
  31. 31. Various path to move views View A View B View D View C
  32. 32. User flow View A View B View D View C
  33. 33. Wrap up the issue behind UI • Logging user flows • Android has view lifecycle which is complicated • New programming language has been published • Developers rewrite/refactor their product code • Long running code has history • It is not easy to introduce new testable architecture
  34. 34. What I did
  35. 35. Solution • Capture HTTP requests during test running by automation • Proxy server tools: WireMock, http_proxy • Can we prevent such breaks only human communication to make sure the behaviour? • Probably, sometimes we can, but we cannot almost time • Against release modules (or the same configuration)
  36. 36. HTTP proxy server • Need a proxy server to capture the requests • We have some kind of ways to capture it
  37. 37. How to capture the network? WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on the device Reverse proxy No proxy server Application Servers Change app endpoint and connect directly
  38. 38. How to capture the network? WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on the device Reverse proxy No proxy server Application Servers Change app endpoint and connect directly
  39. 39. How to capture the network? WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on the device Reverse proxy No proxy server Application Servers Change app endpoint and connect directly
  40. 40. How to capture the network? WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on the device Reverse proxy No proxy server Application Servers Change app endpoint and connect directly
  41. 41. How to capture the network? WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on the device Reverse proxy No proxy server Application Servers Change app endpoint and connect directly
  42. 42. Where? WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on the device Reverse proxy Application Servers No proxy server Change app endpoint and connect directly
  43. 43. Which way? • We can choose various points to capture the request • Which way was the most fit for us?
  44. 44. Some points • Wanted to handle test environment in the test script • Would have like to start from small implementation • Less configuration in server side
  45. 45. Direct connect WiFi Proxy serverNo proxy settings on the device Change Proxy Setting on Android device Reverse proxy No proxy server Application Servers Change app endpoint and connect directly
  46. 46. How to connect to the proxy server? Proxy server Change app endpoint and connect directly Reverse proxy Application Servers • Change the endpoint with: 1.Build configuration (need rebuild) 2.Environment variables, process arguments, intent arguments 3.Change the preference via helper app like io.appium.settings
  47. 47. 2nd way Proxy server Direct connect Reverse proxy Application Servers • Change the endpoint with: 1.Build configuration (need rebuild) 2.Environment variables, process arguments, intent arguments 3.Change the preference via helper app like io.appium.settings
  48. 48. Test Communication flow between the Appium client to servers Proxy server Server Driver App Test target
  49. 49. Which parts are handled by Ruby Proxy server Server Driver App Test target
  50. 50. Capture the http requests in proxy server Proxy server Server Driver App Test target Capture data
  51. 51. With Appium • Appium can control building test environment outside the app with the same script • Appium’s test cycle is before running the app under test • Leverage the system to iOS and Android • Espresso is after onCreate, XCTest is after launching the app • XCTest was not released when I started to try this issue (2016) • Espresso did not work on our app then (test not found happened frequently, then)
  52. 52. Basic Appium flow Configure the device/the test app Run tests on the test device Remain apps/delete apps
  53. 53. Appium’s test lifecycle Configure the device/the test app Run tests on the test device Remain apps/delete apps def setup() // call create session end def test_some_cases() // processes end def teardown() // quit end Can conducts tests Keeping the session
  54. 54. Espresso/XCTest Configure the device/the test app Run tests on the test device Remain apps/delete apps Setup Test cases Teardown Runner install the test target before test scripts Espresso XCTest
  55. 55. XCTest/Espresso • Below setup and test cases are called after finishing to install the test app class appiumConfTest: XCTestCase { override func setUp() { // simulator is already running XCUIApplication().launch() } func testExample() { } } @RunWith(AndroidJUnit4::class) class ExampleMainActivityTest { private val testWatcher = MyTestWatcher() private val scenarioRule = ActivityScenarioRule<MainActivity>( MainActivity::class.java) // AndroidTestRunner launch onCreate (for example) @get:Rule val ruleChain = RuleChain.outerRule(testWatcher) .around(scenarioRule) @Test fun test() { //... } } XCTest Espresso
  56. 56. HTTP proxies • WireMock: https://github.com/tomakehurst/wiremock • Java • http_proxy: https://github.com/KazuCocoa/http_proxy • Elixir
  57. 57. WireMock/http_proxy • Can control responses/requests as JSON format • Very handy to launch them as standalone { “request”: { “path”: “/request/path”, “port”: 8080, “method”: “GET” }, “response”: { “body”: “<html>hello world</html>“, “cookies”: {}, “headers”: { “Content-Type”: “text/html; charset=UTF-8", “Server”: “GFE/2.0" }, “status_code”: 200 } }
  58. 58. Integrate with Appium • Launch proxy server before launching the test app • Can configure test environment from test scripts • Test lifecycle is greater than XCTest/Espresso • Before installing test apps ~ after uninstalling them
  59. 59. Test suite Configure the device the test app Run tests on the test device Remain apps delete apps def setup() // call create session end def test_some_cases() // processes end def teardown() // quit end Start Stop Analyse the data Summarise test results Proxy
  60. 60. Assert the captured data • Count the number of HTTP requests • Parse the JSON body and make sure the logs • If something happens, the Ruby script raises failure
  61. 61. Summary • Test lifecycle on Appium • Getting HTTP requests via a proxy server • How I implemented the capturing data behind UI
  62. 62. Tips
  63. 63. 2nd way Proxy server Direct connect Reverse proxy Application Servers • Change the endpoint with: 1.Build configuration (need rebuild) 2.Environment variables, process arguments, intent arguments 3.Change the preference via helper app like io.appium.settings
  64. 64. Process arguments • XCTest framework provides environment arguments for xcodebuild argument • An app can handle it as env[“PROCESS_VALUE"] in the app • The environment works only for the process { platformName: :ios, automationName: 'XCUITest', ... processArguments: { env: { "PROCESS_VALUE": "process value" } } }
  65. 65. Packages, instrument args • Android case • We can make the release app debug mode with: • Launch an activity with the arguments (adb command) • Install a particular package name { platformName: :android, automationName: 'uiautomator2', ... optionalIntentArguments: '--ez key_name true’, } Package name: com.example.randome_package_names
  66. 66. Simulating push notification, launch via history • When we would like to emulate background to foreground • Launch the all without auto launch • Launch the app with broadcast receiver • partially, we can emulate • suspend/back • Start activity with process arguments
  67. 67. One more thing
  68. 68. One more • HTTP request is not the only thing which happens behind the UI • CPU power etc • Recently, we can see “performance measurement feature” by Google and Apple to capture them in particular period • https://developers.google.com/web/fundamentals/performance/rail • This is mainly for Web, but this kind of performance should be same as mobile • XCTMetric, benchmark etc • https://github.com/KazuCocoa/droid-monitor • Easy to collect data by Android via add/linux commands • Nowadays, HeadSpin can do more rich thing tho… • HeadSpin (company, service)
  69. 69. Takeaways
  70. 70. Takeaways • Happening behind UI is not easy to uncover, but they are also important • The benefit of Appium to configure test environment • Some tips to control the test app condition via Appium capability
  71. 71. Thanks email: kazu@headspin.io Twitter: @Kazu_cocoa

×