This document discusses how to optimize app delivery and quality through the right tools and processes. It emphasizes maintaining quality in a fast-paced environment through visibility into usage and crashes, controlled staged releases, and regression prevention with testing. Keys to success include designing analytics for visibility, periodic staged releases, using feature flags, testing at multiple stages including nightly builds, and ongoing post-release monitoring.

1. Optimize delivery and
quality with the right
combination of tools

2. MyHeritage
6,600,000,000
Historical Records
2,500,000,000
Profiles
85,000,000
Users
Who are we?

3. • Personal goals - We should all want to produce and
release a high quality app
• Poor quality impacts your bottom line
• User satisfaction leads to high retention rates
• Brand reputation
• Pride
Why Quality Matters?

4. • Keeping the foot on the gas paddle when it comes
to releasing new features and versions while
maintaining high quality.
• Quality isn’t determined just by crashes, need a
way to determine that your users get the
experience you want them to.
Challenges In a Fast
Pace Environment

5. • High device fragmentation that leads to greater
complexity
Challenges In a Fast
Pace Environment

6. • The need to support a high number of OS versions
Challenges In a Fast
Pace Environment

7. • Keeping track of your app versions and their
contents over long periods of time.
• Preserving your API’s integrity over time.
Challenges In a Fast
Pace Environment

8. • Visibility
• Controlled releases
• Regression Prevention
Keys to Success

9. Visibility
• Design and implement your analytics strategy in a way that provides
you the most visibility and information.
• Set a unique user ID so that you can monitor the same user over
multiple analytic platforms

10. Visibility

11. Visibility
• Monitoring crash reports
• Monitoring version adoption

12. Controlled App Releases
• Try to stick to a pre-defined version release schedule
• Periodic releases - New version every 2-3 weeks
• Feature flags
• Staged Rollout

13. Regression Prevention
• Beta/Alpha testing
• Unit tests – Build machine
• End to end Automations
• API automations

14. Regression prevention – Nightly Builds
Nightly Unit Tests End to End UI Test
Nightly Coverage

15. How it all fits together
Version
Release
New Feature
Development
Testing
Alpha/Beta
Phase

16. How it all fits together
• Writing unit tests
• Writing E2E/Integration/Automation tests
• Implement feature flags
• Fixing the top crashes that were collected from previous version
New Feature
Development Phase

17. How it all fits together
• Running Unit Test from nightly build machine
• Running E2E/Integration from nightly build machine
• Manual QA
• Monitor test usage to see if the user experience is what was intended
• Analyze user feedback
Testing and Beta
analysis

18. How it all fits together
• Set feature flags to their production values
• Final QA sanity check
• Staged rollout over about 2 days – Bug fixes if needed
Version
Release Phase

19. How it all fits together
• Crash report monitoring – Collect top crashes
• Version adoption monitoring
• Key app event monitoring (e.g. in-app purchases)
• Monitoring new feature analytics for quality and usage insights
• Monitoring Google play reviews and support tickets
Post Version
Release Analysis

20. • Visibility
• Controlled releases
• Regression Prevention
Summary

21. Thank you
Guy Tsype
guy.tsype@myheritage.com