The document discusses how AppDynamics helped a healthcare software company successfully integrate two different codebases and architectures during a major project. AppDynamics identified performance bottlenecks that were addressed, improving response times. It also increased trust between engineering, QA and operations by providing a shared view of metrics. The company plans to implement additional monitoring tools like AppDynamics EUM and Sumologic going forward.

1. AppDynamics: Making
Healthcare Perform

2. Who the hell is this guy?
Mike Ruangutai
Director, Platform Engineering
Cornerstone Ondemand

3. We’re hiring
15 folks in some of the following roles:
● API Architect
● Lead engineer
● Sr. Engineer, Platform
● Sr. Engineer, Data

4. In my prior lives...
Director of Engineering
Technical Director
We’re going to talk about this one
primarily.

5. At Kareo...
Director of Engineering
● Responsible for…
○ Technology architecture, management, strategy, and culture.
○ Scaling team
■ Built out Java team, and stood up offshore teams.

6. Philosophy
Goals is to get around this circle as quickly
as possible.

7. Philosophy
Which means deploying more quickly and
frequently.

8. Diametric opposition
Humans are at the root of all mistakes that
happen in software.

9. Diametric opposition
Humans are the root of the creative process.

10. Diametric opposition
...So let’s put in some gates to increase trust.

11. Diametric opposition
● QA
o Paid to question changes
● Operations
o Paid to prevent changes
● Software Engineers
o Paid to make as many
changes as possible.

12. Diametric opposition
Oh wait, and we want to
do this Agile thing too.

13. Diametric opposition
Agile or Safe
is easy.

14. Diametric opposition
Agile or Safe
is easy.

15. Diametric opposition
Agile and Safe
is hard.

16. Invest in tools
How do we employ tools at each step so we
can be both agile and safe and ultimately
increase trust?

17. Invest in tools
Development operations
● Team City, Jenkins
● Chef, Puppet
● Selenium
Technology metrics
● Gomez
● Pingdom
● AppDynamics
● Splunk, Sumologic
Use Product + Technology metrics to
get high up Maslow’s Hierarchy
of needs.
Product metrics
● Mixpanel
● Flurry
● Google Analytics
● AppDynamics EUM

18. Hierarchy of Needs
Hell yeah! Like a boss!
At minimum, get to this point.

19. The Kareo circumstance
What is Kareo?
● Multi-tenant SaaS for small office healthcare providers, .NET / SQL Server
● .NET desktop client for practice management, aka billing, scheduling
● Growing like gangbusters in all measures:
○ Organization
○ Revenue
○ Users
○ Etc.

20. Bet the farm
● Move from thick desktop client to web platform, aka new architecture.
● Evolve the engineering culture
● Hire, hire, hire!
….And one more thing.

21. Bet the farm
While you’re doing all that, let’s go ahead and acquire an EHR (electronic health records) system
that:
● Puts us in direct competition with our partners, aka significant source of revenue
● Is already browser-based and iOS
● Is completely on Java/Oracle stack and a foreign architecture
● ...Needs to work seamlessly with our .NET application in < 9 months with no team in place.
● Oh and hire them in secret.

22. .NET Architecture

23. Java Architecture

24. Integration Architecture

25. So why AppDynamics?
AppDynamics was the only solution that gave us the confidence
needed to launch such an ambitious project.
No other solution supported an architecture predicated on message
queues.

26. Us vs. Them
vs.

27. Crappy response times
● EHR system was notoriously slow; sometimes 1o seconds to login
● Significant barrier to adoption and usage

28. Never climbing the mountain.
We couldn’t even get here.

29. Improved response times
Using a combination of Gomez, Apache log analyzer, and AppDynamics…
● Quickly identified clusters of slow business transactions
● We reduce a Gomez six step workflow from > 20 seconds to ~ 6
seconds
● Sales calls were no longer painful

30. Identify and triage
● Lots of chatty calls
o Several thousand JDBC calls
o Superfluous JMS calls
● Application areas loading unnecessary data
● Correlating product use to performance metrics, removing functionality
● Identified JRuby bottlenecks
● Uncovered masked exceptions, HTTP errors
o Third party API integration response times and service failures

31. Ultimately led to...
Architecture redesign.
Called Project Galvatron

32. Trust
● Engineering, QA, and Operations using same tools, looking at the
same data.
● Less guessing, less finger pointing

33. Next...
● Implement AppDynamics End User Monitoring
● OpsCode Chef
● Sumologic

34. Learnings
● Implementing an APM like AppDynamics is a large commitment
o Application configurations
o Business transaction analysis
 Which are most important?
 What are the thresholds? Standard deviation? Absolute value?
o Internal training and SOP
o Identifying appropriate actions, i.e. Pager Duty, running scripts
o Know your architecture

35. Recap
Development operations
● Team City, Jenkins
● Chef, Puppet
● Selenium automation
Technology metrics
● Gomez
● Pingdom
● AppDynamics
● Splunk, Sumologic
Use Product + Technology metrics to
get high up Maslow’s Hierarchy
of needs.
Product metrics
● Mixpanel
● Flurry
● Google Analytics
● AppDynamics EUM

36. Repeat buyer
Could use some AppDynamics at Cornerstone Ondemand.

37. Thanks!
Q&A