Мы вместе проговорить следующие вопросы 1. Запуск тестов в облаке: server-side VS client side execution 2. Парк устройств и поддерживаемые фреймворки в TestDroid, TestObject 3. Доступные действия и ограничения в работе с устройствами в облаке 4. Интеграция проекта на базе Appium-а используя client-side execution & API облачного сервиса 5. Интеграция с hockeyApp. 6. Как мы тестировали и что из этого получилось – личная история

1. mobile
Automation
in the cloud

2. Hello!Dmitry Lemeshko
QA Automation @ Ciklum Minsk / eBuilder Sweden

3. PLAN for TODAY’s EVENING:
× Automation without a cloud: any problems?
× Execution in cloud: server-side vs client-side
× Device labs, frameworks and integrations
× What may go wrong a.k.a limitations
× Tool set and Use case on real project
× Your Questions

4. LOCAL
RUN

5. Who took the device?
UPDATE device os VERSION?APPIUM Server errors?
Device cleaning? Xcode 7 or xcode 8?
Who is responsible
for maintenance?Parallel run?

6. WITH
CLOUD

7. Who took the device?
UPDATE device os version?APPIUM Server errors?
DEVice Cleaning? Xcode 7 or xcode 8?
Who is responsible
for maintenance?Parallel run?

8. LET’S START the JOURNEY

9. POPULAR Mobile Device Clouds

10. Client-side vs Server side cloud execution

11. SERVER-side CLOUD execution
PROS
× Tests execution self-managed
× Parallel run out-of-box
× Add extra data and apps
× Automatic results aggregation
CONS
× Limited language support
× Follow code structure
× No control on test execution
× Limited integration

12. CLient-side cloud execution
PROS
× Language flexibility
× Drive your own framework
× Code privacy
× More ways for integration
CONS
× Lots to manage yourself:
device availability, test
execution, parallel run,
results

13. Device labs
Android/iOS 143 / 36 357 / 42 176 / 176
Android 4.4+/6.x/7.x 67 / 31 / 7 137 / 35 / 5 150 /16 / 8
iOS 9.x/10.x 23 / 3 5 / 14 100 / 10
Manufactures 20 32 22
Tablets 18% 30% 30%

14. Framework support
Android Appium
Espresso
Robotium
Appium Calabash
Espresso UI Automator
Robotium
iOS Appium Appium Calabash
UI Automation XCTest / XCUITest

15. Integration support
CLOUD API JAVA, RUBY CLIENTS JAVA, RUBY, PYTHON
CLIENTS
AWS API / AWS CLI
PLUGINS JENkiNS, GRaDLE JENkiNS, GRaDLE
App release tools Hockey App
Bug Trackers JIRA, GITHUB JIRA

16. URL
http://appium.testobject.com/wd/hub
Desired capabilities
testobject_api_key = 73A**********12
Testobject_device = Samsung_Galaxy_S7_real
TEST OBJECT Appium Setup

17. testobject_appium_version = 1.6.3
testobject_app_id = 92
testobject_suite_name = Smoke Test
testobject_test_name = Navigation via menu
testobject_cache_device = true
testobject_session_creation_timeout = 30
Return
testobject_test_live_view_url
testobject_test_report_url
TEST OBJECT OPTIONAL CAPABILITIES

18. test execution output in cloud
1. Device logs
2. Appium logs
3. Screenshots
4. Test run recording
5. Cpu/Memory usage

19. What may go wrong a.k.a limitations
1. Appium server update frequency
2. test runs fail on device setup stage
3. No login with custom google account
4. No device settings change
5. No wifi turn on/off

20. Moved to testobject.. still have limitations
1. No device storage access
2. Sms, contacts & call history cleaning
3. No push notifications for iOS
4. No Geo location setup
5. Camera usage

21. Tool set and Use case on real project
× Appium
× Java/TestNG
× HockeyApp API
× TestObject API
× Team City CI
× Smoke: random in group
× Regression: top 10 devices
(covers 70% Android users)
× Feature test (on request):
50+ devices

22. BACKUP & RESTORE FUNCTIONALity TESTING

23. Can you
try it?

24. GO AHEAD!
250 min trial
5 free devices
3 free devices

25. THANKS!Any questions?
You can find me at dima.inspired@gmail.com