Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quality Tracking/Crash Reporting Services for Mobile Apps

14,174 views

Published on

This weekend I did some fairly extensive research on different quality tracking/crash reporting services available for mobile applications. Since I'm a iOS developer, I mainly focussed on this platform. Yet, for Android developers this post can be handy too although I believe crash reports handling is much better on the Android platform compared to Apple (iTunes Connect).
I tested the following services:

iTunes Connect (https://itunesconnect.apple.com)
BugSense (http://www.bugsense.com)
Crittercism (http://www.crittercism.com)
Crashlytics (http://try.crashlytics.com) (not tested it as I'm still on their invitation list)
HockeyApp (http://www.hockeyapp.net)
Apphance (http://www.apphance.com)
TestFlight (https://testflightapp.com)

Published in: Technology

Quality Tracking/Crash Reporting Services for Mobile Apps

  1. 1. Quality Tracking/Crash reportersfor Mobile AppsNiels Mouthaan, October 13th, 2012http://nielsmouthaan.nl/quality-trackingcrash-reporters-for-mobile-apps
  2. 2. SaaS solutions iTunes Connect (https://itunesconnect.apple.com) BugSense (http://www.bugsense.com) Crittercism (http://www.crittercism.com) Crashlytics (http://try.crashlytics.com) HockeyApp (http://www.hockeyapp.net) TestFlight (https://testflightapp.com) Apphance (http://www.apphance.com)
  3. 3. iTunes Connect Default crash reporting service provided by Apple Pricing: free (included in the iOS developer account) Pros:  No need to install separate library  No need to create separate account elsewhere Cons:  Never seen a crash log appearing for any of our apps (e.g. HockeyApp detects 20 times more crashes than iTunes Connect)  Only sends crash logs if user agreed at the first setup of their device to send anonymous data to Apple  No server symbolication provided  Does not show all crash logs (only top 5 is provided)
  4. 4. BugSense Crash reporting service founded in 2011 located in CA Platforms: iOS, Android (1.5+) & WP7 Pricing: $99/month (up to 100k errors/month) Pros:  Server symbolication (dSYM for iOS, supports Proguard for Android)  Libraries are open-source (https://github.com/bugsense)  Crash fixed notification (prevents 1-star ratings)  Good UI (e.g. sorting, filtering, graphs)  Notifications per email when crash occurs Cons:  Does not ask the user to send its crash logs  No end-user communication (e.g. feedback forum)
  5. 5. Crittercism Crash reporting service founded in 2010 in SA, funded by 5 VC’s ($5.5m in total). More than 4.5b app loads. Platforms: iOS and Android (HTML5, WP7 & BB are in beta) Pricing: $24/month per 100k monthly active users Pros:  Server symbolication (dSYM for iOS, supports Proguard for Android)  Notifications via email  Libraries are open-source (https://github.com/crittercism)  Breadcrumbs (checkpoints, enterprise account is needed)  End-user communication (rate my app, notification if crash is due old version)  In-App Feedback Forum (handy for beta testing)  Excellent & initiative UI (e.g. crash sorting & crash info.) Cons:  Does not ask the user to send its crash logs
  6. 6. Crashlytics Crash reporting service founded in 2011 (Cambridge, US) Platforms: iOS and Android Pricing: freemium (no details available) No public access yet, have not tried it (I’m on their invitation list) Pros:  Server symbolication  More advanced reporting & crash log analysis (not just the stack trace)  Remote logging  Comprehensive information about the state of the device (type, battery status, location, orientation, etc.)  Offers a Mac OS X Desktop application  Supports user notification when sending crash log  Integration with multiple issue management systems (JIRA 5, etc.) Cons:  Still in beta but production ready
  7. 7. HockeyApp Crash reporting service and beta distribution founded in 2011 and located in Germany Platforms: iOS, Android & Windows Phone Pricing: $79 (100 apps, 10GB storage & 5 managers) Pros:  Server symbolication (dSYM for iOS, supports Proguard for Android)  Notifications via email  Libraries are open-source (see QuincyKit and HockeyKit)  Asks the user to send crash logs (can be configured to automatic send logs)  Issue status management (mark an issue as closed, open, etc.)  Integration with external issue trackers (JIRA, Mantis, etc.)  Extensive & detailed crash reports  Easy app & dSYM file upload via Mac OS X Desktop utility Cons:  No end-user communication (e.g. feedback forum)  No graphs illustrating crash history
  8. 8. TestFlight Crash reporting service, beta distribution & app statistics provider founded in 2011 and located in CA Platforms: iOS Pricing: free (premium plans in the future) Pros:  Server symbolication (dSYM)  Notifications via email  Checkpoints, remote logging  Easy app & dSYM file upload via Mac OS X Desktop utility (better than HockeyApp)  TestFlight Live gives a nice overview of the app’s usage Cons:  Portal can be slow sometimes  Crash logs sorting & filtering is not efficient  Does not ask the user to send its crash logs  In the past, we had crashes due issues in their SDK  Sometimes crashes and usage info do not appear in their portal
  9. 9. Apphance Crash reporting service & beta distribution as part of uTest (worlds largest marketplace for software testing services) Platforms: iOS, Android & Windows Phone Pricing: free in 2012 (expensive premium plans following) Pros:  Server symbolication (not tested, see below)  In-app bug reporting and in-app user feedback  Remote logging Cons:  Plans are expensive compared to competitors  No desktop utility available (IPA upload via site)  Service feels and looks a bit too new for production (I could not test this service due installation errors… I tried installing via iTunes and via their website).
  10. 10. Other solutions PLCrashReporter (http://code.google.com/p/plcrashreporter/) QuincyKit (http://github.com/TheRealKerni/QuincyKit) & HockeyKit (http://hockeykit.net) CrashReporter (https://github.com/amazingsyco/CrashReporter) Airbrake (http://airbrake.io)
  11. 11. PLCrashReporter Open source iOS & Mac crash handler Used by different solutions (HockeyApp, BugSense, etc.) No armv7s (iPhone 5) support in the current codebase No server integration provided (QuickyKit offers a comprehensive solution)
  12. 12. QuincyKit Comprehensive solution  iOS library  Server application including web application  Needs maintenance and no support is offered  Mac OS X is needed for symbolication Based on PLCrashReporter Still being maintained (armv7s/iPhone 5 support) Open source/free Supports push notifications Can be used together with/is part of HockeyKit: a self-hosted over-the-air beta distribution system
  13. 13. CrashReporter iOS only Sends crash reports via email (no server needed) Outdated (2 years old, probably no armv7s (iPhone 5) support
  14. 14. Airbrake Haven’t test it myself Expensive No server symbolication Does not work with App Store apps (Ad Hoc and beta only) Possibility to host Airbrake in your own datacenter
  15. 15. (extra information) http://www.jeremyfuller.net/2011/06/the-battle-of-the-ios- crash-reporters/ http://www.jeremyfuller.net/2012/09/hockeyapp-still-the-best- thing-since-sliced-pizza/ http://support.hockeyapp.net/discussions/questions/206-why- hockey-over-testflight http://digitalflapjack.com/blog/2012/jun/19/hockeyapp

×