Quality Tracking/Crash Reporting Services for Mobile Apps

13,936 views

Published on

This weekend I did some fairly extensive research on different quality tracking/crash reporting services available for mobile applications. Since I'm a iOS developer, I mainly focussed on this platform. Yet, for Android developers this post can be handy too although I believe crash reports handling is much better on the Android platform compared to Apple (iTunes Connect).
I tested the following services:

iTunes Connect (https://itunesconnect.apple.com)
BugSense (http://www.bugsense.com)
Crittercism (http://www.crittercism.com)
Crashlytics (http://try.crashlytics.com) (not tested it as I'm still on their invitation list)
HockeyApp (http://www.hockeyapp.net)
Apphance (http://www.apphance.com)
TestFlight (https://testflightapp.com)

Published in: Technology
2 Comments
11 Likes
Statistics
Notes
No Downloads
Views
Total views
13,936
On SlideShare
0
From Embeds
0
Number of Embeds
3,165
Actions
Shares
0
Downloads
133
Comments
2
Likes
11
Embeds 0
No embeds

No notes for slide

Quality Tracking/Crash Reporting Services for Mobile Apps

  1. 1. Quality Tracking/Crash reportersfor Mobile AppsNiels Mouthaan, October 13th, 2012http://nielsmouthaan.nl/quality-trackingcrash-reporters-for-mobile-apps
  2. 2. SaaS solutions iTunes Connect (https://itunesconnect.apple.com) BugSense (http://www.bugsense.com) Crittercism (http://www.crittercism.com) Crashlytics (http://try.crashlytics.com) HockeyApp (http://www.hockeyapp.net) TestFlight (https://testflightapp.com) Apphance (http://www.apphance.com)
  3. 3. iTunes Connect Default crash reporting service provided by Apple Pricing: free (included in the iOS developer account) Pros:  No need to install separate library  No need to create separate account elsewhere Cons:  Never seen a crash log appearing for any of our apps (e.g. HockeyApp detects 20 times more crashes than iTunes Connect)  Only sends crash logs if user agreed at the first setup of their device to send anonymous data to Apple  No server symbolication provided  Does not show all crash logs (only top 5 is provided)
  4. 4. BugSense Crash reporting service founded in 2011 located in CA Platforms: iOS, Android (1.5+) & WP7 Pricing: $99/month (up to 100k errors/month) Pros:  Server symbolication (dSYM for iOS, supports Proguard for Android)  Libraries are open-source (https://github.com/bugsense)  Crash fixed notification (prevents 1-star ratings)  Good UI (e.g. sorting, filtering, graphs)  Notifications per email when crash occurs Cons:  Does not ask the user to send its crash logs  No end-user communication (e.g. feedback forum)
  5. 5. Crittercism Crash reporting service founded in 2010 in SA, funded by 5 VC’s ($5.5m in total). More than 4.5b app loads. Platforms: iOS and Android (HTML5, WP7 & BB are in beta) Pricing: $24/month per 100k monthly active users Pros:  Server symbolication (dSYM for iOS, supports Proguard for Android)  Notifications via email  Libraries are open-source (https://github.com/crittercism)  Breadcrumbs (checkpoints, enterprise account is needed)  End-user communication (rate my app, notification if crash is due old version)  In-App Feedback Forum (handy for beta testing)  Excellent & initiative UI (e.g. crash sorting & crash info.) Cons:  Does not ask the user to send its crash logs
  6. 6. Crashlytics Crash reporting service founded in 2011 (Cambridge, US) Platforms: iOS and Android Pricing: freemium (no details available) No public access yet, have not tried it (I’m on their invitation list) Pros:  Server symbolication  More advanced reporting & crash log analysis (not just the stack trace)  Remote logging  Comprehensive information about the state of the device (type, battery status, location, orientation, etc.)  Offers a Mac OS X Desktop application  Supports user notification when sending crash log  Integration with multiple issue management systems (JIRA 5, etc.) Cons:  Still in beta but production ready
  7. 7. HockeyApp Crash reporting service and beta distribution founded in 2011 and located in Germany Platforms: iOS, Android & Windows Phone Pricing: $79 (100 apps, 10GB storage & 5 managers) Pros:  Server symbolication (dSYM for iOS, supports Proguard for Android)  Notifications via email  Libraries are open-source (see QuincyKit and HockeyKit)  Asks the user to send crash logs (can be configured to automatic send logs)  Issue status management (mark an issue as closed, open, etc.)  Integration with external issue trackers (JIRA, Mantis, etc.)  Extensive & detailed crash reports  Easy app & dSYM file upload via Mac OS X Desktop utility Cons:  No end-user communication (e.g. feedback forum)  No graphs illustrating crash history
  8. 8. TestFlight Crash reporting service, beta distribution & app statistics provider founded in 2011 and located in CA Platforms: iOS Pricing: free (premium plans in the future) Pros:  Server symbolication (dSYM)  Notifications via email  Checkpoints, remote logging  Easy app & dSYM file upload via Mac OS X Desktop utility (better than HockeyApp)  TestFlight Live gives a nice overview of the app’s usage Cons:  Portal can be slow sometimes  Crash logs sorting & filtering is not efficient  Does not ask the user to send its crash logs  In the past, we had crashes due issues in their SDK  Sometimes crashes and usage info do not appear in their portal
  9. 9. Apphance Crash reporting service & beta distribution as part of uTest (worlds largest marketplace for software testing services) Platforms: iOS, Android & Windows Phone Pricing: free in 2012 (expensive premium plans following) Pros:  Server symbolication (not tested, see below)  In-app bug reporting and in-app user feedback  Remote logging Cons:  Plans are expensive compared to competitors  No desktop utility available (IPA upload via site)  Service feels and looks a bit too new for production (I could not test this service due installation errors… I tried installing via iTunes and via their website).
  10. 10. Other solutions PLCrashReporter (http://code.google.com/p/plcrashreporter/) QuincyKit (http://github.com/TheRealKerni/QuincyKit) & HockeyKit (http://hockeykit.net) CrashReporter (https://github.com/amazingsyco/CrashReporter) Airbrake (http://airbrake.io)
  11. 11. PLCrashReporter Open source iOS & Mac crash handler Used by different solutions (HockeyApp, BugSense, etc.) No armv7s (iPhone 5) support in the current codebase No server integration provided (QuickyKit offers a comprehensive solution)
  12. 12. QuincyKit Comprehensive solution  iOS library  Server application including web application  Needs maintenance and no support is offered  Mac OS X is needed for symbolication Based on PLCrashReporter Still being maintained (armv7s/iPhone 5 support) Open source/free Supports push notifications Can be used together with/is part of HockeyKit: a self-hosted over-the-air beta distribution system
  13. 13. CrashReporter iOS only Sends crash reports via email (no server needed) Outdated (2 years old, probably no armv7s (iPhone 5) support
  14. 14. Airbrake Haven’t test it myself Expensive No server symbolication Does not work with App Store apps (Ad Hoc and beta only) Possibility to host Airbrake in your own datacenter
  15. 15. (extra information) http://www.jeremyfuller.net/2011/06/the-battle-of-the-ios- crash-reporters/ http://www.jeremyfuller.net/2012/09/hockeyapp-still-the-best- thing-since-sliced-pizza/ http://support.hockeyapp.net/discussions/questions/206-why- hockey-over-testflight http://digitalflapjack.com/blog/2012/jun/19/hockeyapp

×