Automated Testing on iOS
•Download as PPTX, PDF•
0 likes•293 views
Slides from an Xcake Dublin meetup given on the 12th of October 2016. A walkthrough of some of the reasons why we do automated testing on iOS. Some discussion on unit testing with XCTest as well as some of the UI Testing features of Xcode 7. There's some discussion of the benefits of mocking HTTP endpoints as part of the development of your connected apps.
Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Automated Testing on iOS
- 1. Automated Testing for iOS Robert Collins @collinsrj IBM Watson Health Please refer to speaker notes if reading this! Xcake Dublin - 12th October 2016
- 2. Battery Life Localisation Memory Usage Bandwidth Consumption Network Conditions SecurityStorage App Upgrades Function! Accessible OS Versions Size Classes Usability Device Versions Device Orientation Core Data
- 5. The Network
- 6. Mock the Network for Tests
- 7. Reuse
- 8. Text Soak Test If your users do stuff all day, maybe you should test that.
- 10. Xcode Server
- 11. Text Monkey Test
- 12. Balance
- 13. You automate to: give you more time give you the confidence to change features keep quality high help understand what’s going on!
Editor's Notes
- Note for people reading this: These slides are my own opinions and experiences. They do not reflect those of my employer! There are no guarantees about the information provided. It may be wrong or bad. Please take your own time to understand the material presented and form your own opinions. The included speaker notes were my thoughts at time of writing these slides. They are not exactly what I said - I forgot some stuff and included other stuff. Hi my name is Robert Collins. I’m a mobile architect at IBM Watson Health. I’m going to talk today about a few testing topics we’re working through. These are based on our recent experience. Our group is pretty new. We only actively started developing for iOS in April this year. Our app is written in Swift, but I don’t think anything I refer to here, excludes the Objective-C guys.
- I noted down some of the things we need to test. Realistically, we barely have time to check all of these. It’s funny, at times it’s easy to forget the app has to function in addition to all of these things. Users have a really high expectation for the quality of mobile apps. The app has to look good as well as function. In one of the first weeks I attended Xcake, Karl asked where we were from and I said IBM. Karl mentioned this could be quite different to some of the usual Xcake attendees who might have quite a different customer base. I guess my own experience having worked in both enterprise and small shops is that traditional enterprise apps can be slow and clunky. Take for example IBM Lotus Notes. It’s is an app that brings me daily frustration. (I can say that now given the release of IBM Verse.) Consumers are used to interacting with a small number of likely high quality apps. By quality I mean, stable, performant, great user experience. I think the enterprise is moving there too. Initiatives like IBM Design are really changing things for the better. They’re bringing the great consumer app experiences to the enterprise. To deliver the great experience, we have got to test. We test not only to make our code more performant, but also to make our code more maintainable. To ensure the coupling of our classes was correct. Writing tests is about ensuring we really have thought about all the scenarios we can encounter. Let’s automate what we can. We’re programmers, one of the premises of computing is to avoid us doing repetitive tasks. We do automated testing to allow us more time to work on the hard things.
- There isn’t too much say about Unit Testing. I think it’s something all professional developers should be doing already. If you’re not, you really need to get with it. I think the reasons should be obvious in most cases. We’re aiming for about 70% unit test coverage. This number was suggested as a reasonable balance from another team. It might be a little low, it might be a little high. At the moment it’s our goal. We’ll refine. Don’t ignore the performance piece of unit testing. It’s not something our team has been using, but I think it will help you to understand how iOS will perform. One example I have of this is around JSON parsing. Parsing might not be linear. If you only test with a couple of items coming from a test server, how will you know what happens in the real world? Protocols definitely make decoupling your components easier. I guess this is a bit of my Java background creeping through. You should be coding to an interface.
- I won’t go into how to UI test. Our goal here is test that the View and Controller code we’ve written functions as expected. It’s not going to let you test that a button is blue. Don’t try to do too much here. Consider this in a sort of testing pyramid way - lots of unit tests, less UI Tests, even less integration tests and then a small amount of manual testing.
- Our app connects to a REST API. We have never managed to structure our projects so that the APIs are stable and complete before that feature is required. This has led to great frustration! If the dev server is unavailable you’re unable to develop or test. We looked at some solutions to this. One I proposed which kinda worked was to mocking up the REST APIs in simple Node.JS app. This started us off interacting with a real system quickly while we were waiting for the full API to be completed. At the start, this was really useful. It got the app team writing code, appreciating some of the challenges of networks. There were lots of headaches. We ended up changing the schema quite a few times! This would typically happen in such a way to cause a major headache for the app team. The mock server would go down as well. This approach did not let us test some of the more challenging aspects of networking e.g. how to manage slow downloads, how would our app work function if a very large data set was returned. How would the app deal with the network being unavailable.
- The approach we’re currently using is to mock in the app code using OHHTTPStubs. This library proved to be really useful for unit testing. We could achieve a high coverage level. We’ve been able to test lots of states we were not able to run through before. What happens if connections die, what happens if slow, what messaging is presented to users if the app times out. Mocking can cause real problems. Late integration is a real risk in any project. If you take the approach we’ve taken of mocking for UI Tests, you need to look at where you’re doing your integration tests!!
- Reusability - tests require as much care an maintenance as your code. This becomes quite apparent as you start to write your UI tests. You’ve got to look at ways to simplify your tests. They will require care and refactoring. A nice step we’ve made recently is to move all mocked network JSON responses into a folder structure. For the UI tests, this reuse will be really important. You’ll have the same steps repeated over and over. If I take an app I’m working on at the moment, we have a number of common setup steps e.g. login and search for a client. You’ll reuse these in many of your tests. Factor them out.
- This is something we haven’t yet done. It’s definitely on our list though. Our expectation is that our app will be used throughout a working day. We’re building apps which must stay responsive throughout the day. There is some work to be done in the setup. I’d recommend looking at a separate build target for Soak Tests. You want these to run, but not all the time. There is some tweaking to be done for config etc. here. Be conscious that you’re testing a ‘release’ versions here and not debug. Will the app be compiled with different settings to your release?
- Physical devices will always be a challenge. The simulator will set unrealistic expectations on performance. There will be issues you can only experience on real devices. You can’t ignore testing on physical devices, but they can really help you expand your testing. - Again, this is a confidence thing!
- Well worth the €20. It’s not entirely reliable. Take all the issues you have with Xcode in general and now make another system out of it. I haven’t found it to be the most performant system in the world. I’m not sure if this is added overhead or if it’s just my build server is too slow. It’s unclear to me if I can parallelise builds effectively. Something we code do with Jenkins. I really like sticking with the Apple way. In this instance I think it will be worth the pain. Just be prepared for some :-) It has been a big driver for us to move to Git.
- Monkey testing is something I found from Android. The goal here is to simulate the in-pocket user. There’s a handy Github project which uses the Automation instrument to simulate random user interactions - swipe, tap, pinch… these combined with some predefined user interactions allow you to simulate what would happen with annoying users who just start tapping everything. In my Android testing I got the odd crash out of it. It’s hard to say it’s a must do. If you’ve got some spare cycles it’s definitely worth playing with. More so if you’ve an ‘interesting’ user interface with many gestures.
- Having really high code coverage and full coverage of all the possible scenarios in your user interface is nice. But if you’re not shipping product it’s no good. There’s definitely a balance to be struck here. Where the balance lies is up to you and your team. If you’re a super experienced team maybe you don’t need tests (or maybe you’ve already written them).
- Useful links: https://github.com/jonathanpenn/ui-auto-monkey https://github.com/AliSoftware/OHHTTPStubs