The document discusses how New Relic launched its Metric Explorer initiative in 3 months by adopting an agile approach focused on failing better. It emphasizes incremental development, feature flagging, dark launches, data-driven scoping, and empowering self-reliant teams with shared goals and real-time visibility into product usage and deployments provided by New Relic tools. This allowed for rapid iteration, integration of feedback, and reliability improvements to achieve a successful launch.

1. ©2008–18 New Relic, Inc. All rights reserved
Fail Better
Henry Shapiro
VP of Solutions Strategy, New Relic

2. ©2008–18 New Relic, Inc. All rights reserved 2
Safe Harbor
This presentation and the information herein (including any information that may be incorporated by reference) is provided for informational
purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell
securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and
may not be replicated or disclosed without New Relic’s express written permission.
Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or
refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking
statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as
“believes,” “anticipates,” “expects” or words of similar import.
Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are
subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New
Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such
forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks,
uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially
from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements
is included in the filings New Relic makes with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s
Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov.
New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no
warranties, expressed or implied, in this presentation or otherwise, with respect to the information provided.

3. ©2008–18 New Relic, Inc. All rights reserved
Going Faster Requires
Technology + Culture That Is
Agile and Highly Resistant to
Failure
3

4. ©2008–18 New Relic, Inc. All rights reserved 4
Okay, but really, what?
Fast feedback and directionally
correct data to make changes
and decisions quickly
Autonomous
T-shaped
Integrated with the
business
Cloud for experimentation and POC
Containers for deploying fast
Microservices for rapid iteration
Feature flags and incremental development
Agreement on key metrics
between PM and Engineering
A software team
that is:
Technology (and observability for
those technologies) that enable
agility and pace of development

5. ©2008–18 New Relic, Inc. All rights reserved 5
Release
Something
Repeat Measure It
Find that it doesn’t
work (or it does)
React by rolling
back or iterating

6. ©2008–18 New Relic, Inc. All rights reserved 6
This isn’t
failing fast.
It’s failing
better.
Repeat

7. ©2008–18 New Relic, Inc. All rights reserved
Storytime!
7

8. ©2008–18 New Relic, Inc. All rights reserved 8
Metric Explorer: A Spooky Initiative
Had to index
and ingest
MILLIONS
of metrics per
minute across
all accounts
Provide interface for
querying timeslices
across
HUNDREDS
OF
THOUSANDS of
users
Enable customers to
chart and modify
METRICS
on dashboards
Fundamentally
changed
SIGNIFICANT
parts of the
Insights UX

9. ©2008–18 New Relic, Inc. All rights reserved
How Did We Get This Built
and Launched in 3 Months?
9

10. ©2008–18 New Relic, Inc. All rights reserved 10
We Dark Launched and Feature Flagged Aggressively
Launched a backend service (which fell over!) and
indexed staging traffic before we moved to prod
Measured performance as we up-ticked ingest
and query traffic
Isolated launched features to beta customers for fast
feedback and iteration mid-sprint (lots of failure here)
Failed at scaling to build confidence before customers
could have a bad experience

11. ©2008–18 New Relic, Inc. All rights reserved 11
We Did a LOT of Data-driven Scope Cutting
and Incremental Development
We cut stuff that was going to destroy time to market (and
that didn’t have enough value)
ex. “Suggested Searches” feature in last week of development,
timeouts, decided to cut based on user research and data
Split our work into over 10 micro-features, each
of which was shipped incrementally
Consistent and efficient strategies for retrospectives
and demos
Lightweight and low-process planning

12. ©2008–18 New Relic, Inc. All rights reserved 12
Engineering and PM Shared a Brain Cell
Shared success metrics (user adoption,
reliability, performance)
Shared understanding of key MMFs and
prioritization
Shared understanding of MOST
IMPORTANT customer problems to solve
Shared understanding of WHY we cut
something or WHY we didn’t

13. ©2008–18 New Relic, Inc. All rights reserved 13
We Created Positive Redundancy
Team was on-call
Team literally wrote code together (mob programming)
Team was composed of T-shaped individuals
(and we didn’t start with T-shaped ones!)
… all of this created trust and empathy
Team managed their own services and deployment strategy

14. ©2008–18 New Relic, Inc. All rights reserved 14

15. ©2008–18 New Relic, Inc. All rights reserved 15
What Was Special About This Team?
T-Shaped
Shared skills
Highly
Redundant
(In a good way)
Self-Reliant
Deployed and
managed our
own svcs
Containerized
everything
AWS
Used AWS to
experiment
with indexing +
optimization
Incremental &
Integrated
Mob
programming
Single-threading
MMFs
“PM is part
of the team”
Standups
Demos
Retrospectives
Reliability
and DRI work

16. ©2008–18 New Relic, Inc. All rights reserved 16
We Could Only Do This Because Of New Relic
Dashboards that we all used for
understanding usage patterns
Design • PM • Engineering
Visibility and instrumentation for
things we were working on
Java services that our agents instrumented
Client-side experience
Custom services that required us to do some of
our own instrumentation
Real-time feedback when we
launched new stuff
Alerts • Real-time data + deploy markers

17. ©2008–18 New Relic, Inc. All rights reserved 17
Outcome of Launching the Metric Explorer

18. ©2008–18 New Relic, Inc. All rights reserved 18
Do I:
Have the right data to move fast?
Have the right team and practices to move fast?
Have the right technology to move fast?

19. ©2008–18 New Relic, Inc. All rights reserved
Because You Won’t Win
(or Have Much Fun)
Doing It Any Other Way
19

20. Thank You
©2008–18 New Relic, Inc. All rights reserved
linkedin.com/in/shapirohenry
@nah_serious