1.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Avoiding Availability Disasters in Service-based Applications
Lee Atchison
Senior Director Strategic Architecture at New Relic, Inc.
leeatchison@leeatchison

2.
©2008–18 New Relic, Inc. All rights reserved
What is a Modern Application
Responsive to
Customer Needs
Scale
Dynamically
Maintain High
Availability

3.
©2008–18 New Relic, Inc. All rights reserved 3
Building and Maintaining a Modern Application
Flexible
Architecture
Instrumentation
And
Monitoring
Dynamic
Infrastructure
DevOps
Culture

4.
©2008–18 New Relic, Inc. All rights reserved
Service Based Application
Service
A
Service
B
Service
C
Service
D
Service
E
Service
F

5.
©2008–18 New Relic, Inc. All rights reserved
Service Failure
Service
A
Service
B
Service
C
Service
D
Service
E
Service
F
X
X XXX
What if Service D is not critical to
the success of Service A running?
?

6.
©2008–18 New Relic, Inc. All rights reserved
Inter-Service Failures
Common Cause for Availability Issues
How do you decide where to invest your time and energy?
Add resiliency
everywhere
expensive,
complex,
inefficient
Ignore some
dependencies
can
be
disastrous
Waste time/money or make risky decisions?

7.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1
Tier 2
Tier 3
Tier 4
Mostcriticalservices
Leastcriticalservices
A label applied to every service

8.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1
Tier 2
Tier 3
Tier 4
Mostcriticalservices
Leastcriticalservices

9.
©2008–18 New Relic, Inc. All rights reserved
Tier 1 Service
A service where failure will
result in a significant impact to
customers or company bottom
line

10.
©2008–18 New Relic, Inc. All rights reserved 10
Example Tier 1 Services
Login Service
Order
Accepting
Service
Credit Card
Processing
Service

11.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1
Tier 2
Tier 3
Tier 4
Mostcriticalservices
Leastcriticalservices

12.
©2008–18 New Relic, Inc. All rights reserved
Tier 2 Service
A service where failure will
result in degraded customer
experience, but doesn’t
prevent its use

13.
©2008–18 New Relic, Inc. All rights reserved 13
Example Tier 2 Services
Search
Service
Order
Fulfillment
Service

14.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1
Tier 2
Tier 3
Tier 4
Mostcriticalservices
Leastcriticalservices

15.
©2008–18 New Relic, Inc. All rights reserved
Tier 3 Service
A service where failure will
result in minimal or difficult to
notice impact to
customers/business

16.
©2008–18 New Relic, Inc. All rights reserved 16
Example Tier 3 Services
Message of
the Day
Service
Recommendations
Service

17.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1
Tier 2
Tier 3
Tier 4
Mostcriticalservices
Leastcriticalservices

18.
©2008–18 New Relic, Inc. All rights reserved
Tier 4 Service
A service where failure will
result in no or minor impact to
customer/business

19.
©2008–18 New Relic, Inc. All rights reserved 19
Example Tier 4 Services
Sales Report
Generation
Service
Marketing
Email
Delivery
Service

20.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1 – Significant Impact
Tier 2 – Degraded Experience
Tier 3 – Limited Impact
Tier 4 – No or Minor Impact
A label applied to every service based on impact

21.
©2008–18 New Relic, Inc. All rights reserved
Example Application

22.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store

23.
©2008–18 New Relic, Inc. All rights reserved
Assign Tiers

24.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store

25.
©2008–18 New Relic, Inc. All rights reserved
View Catalog
Customers need to see products to shop for them
• Website frontend
• Catalog View
• Catalog Database
 These are all Tier 1 Services
(company can’t operate without it)

26.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1

27.
©2008–18 New Relic, Inc. All rights reserved
Search Catalog
Customers want to search for product
• Catalog Search
 This is a Tier 2 Service
(Useful for shopping)

28.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2

29.
©2008–18 New Relic, Inc. All rights reserved
Catalog Editing
Merchandisers need to add/update products and set prices
• Catalog Editing
• Price Change
 These are Tier 3 Services
(Not immediately impactful to customers)

30.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2
3
3

31.
©2008–18 New Relic, Inc. All rights reserved
Checkout
Customers need to create orders
• Checkout Service
• Order Management
• Price Calculator
 These are Tier 1 Services
(company cannot operate without them)

32.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2
3
3
1
1
1

33.
©2008–18 New Relic, Inc. All rights reserved
Order Processing and Shipping
Fulfillment Center Agents want to
• Process Orders
• Ship Orders
 These are Tier 3 Services
(some delays have no impact on customers)

34.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2
3
3
1
1
1
3
3

35.
©2008–18 New Relic, Inc. All rights reserved
Order Status
Customer wants to
• See status of order
 This is a Tier 2 Service
(useful but not essential for customers)

36.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2
3
3
1
1
1
3
3
2

37.
©2008–18 New Relic, Inc. All rights reserved
Order Report
Managements wants to
• See weekly status of all orders
 This is a Tier 4 Service
(useful internally, but delay in delivery is not serious)

38.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2
3
3
1
1
1
3
3
2
4

39.
©2008–18 New Relic, Inc. All rights reserved
Using Tiers

40.
©2008–18 New Relic, Inc. All rights reserved
How do I Use Service Tiers?
Responsive Dependencies Expectations

41.
©2008–18 New Relic, Inc. All rights reserved
Responsiveness
1. Severity of the issue
2. Tier of service having the issue
When an issue with a service occurs, responsiveness differs depending
on the needs of the service. Your responsiveness depends on:

42.
ProblemSeverity
Service Tier of service with problem
Low Severity
High Severity
Tier 1 Tier 2 Tier 3 Tier 4
Noncritical
Perhaps next day
followup acceptable
Critical
Immediate response
needed
Service Tier
Helps Determine Required Responsiveness
Issue #1
Issue #2

43.
©2008–18 New Relic, Inc. All rights reserved
How do I Use Service Tiers?
Responsive Dependencies Expectations

44.
©2008–18 New Relic, Inc. All rights reserved
Dependencies
Compare Service Tier of a service and a dependency determines the
criticalness of the dependency.
Service Dependency
1 3
How you handle dependencies is determined by the comparison of Service Tiers.

45.
©2008–18 New Relic, Inc. All rights reserved
Critical Dependency
Service Dependency
1 3
Service needs to handle dependency failures gracefully.
Service is higher tier than dependency
Critical Dependency:
Dependency failure should not cause service to fail!

46.
©2008–18 New Relic, Inc. All rights reserved
Noncritical Dependency
Service Dependency
13
Service can generally ignore dependency failures.
Service is lower tier than dependency
Noncritical Dependency:
Usually acceptable for service to fail if dependency fails.

47.
©2008–18 New Relic, Inc. All rights reserved
Dependencies
Tier 1
Tier 2
Tier 3
Tier 4
Tier 1 Tier 2 Tier 3 Tier 4
Serviceyoudependon
Your service
Critical
Dependency
Noncritical
Dependency
Service Tier
Helps Determine Criticalness of Dependencies

48.
©2008–18 New Relic, Inc. All rights reserved
How do I Use Service Tiers?
Responsive Dependencies Expectations

49.
©2008–18 New Relic, Inc. All rights reserved
Higher ExpectationsCritical Dependencies
Expectations
Well defined SLAs
• Performance
• Availability
Higher the service’s Service Tier, the greater the expectations for that service.
Faster, higher availability, more responsive support…
More Generally:
Service Tier
Helps Establish Appropriate Expectations of a
Service

50.
©2008–18 New Relic, Inc. All rights reserved
How do I Use Service Tiers?
Responsive Dependencies Expectations

51.
Website
frontend
Checkout
View
Order
C
U
S
T
O
M
E
R
S
Cat View
Search
Price
Calculator
Order
Management
Catalog
Database
Catalog
Editing
Price
Change
Merchandiser
Order
Processing
Order
Shipping
Fulfillment Center
Agent
Weekly Order Report Management
T-Shirt Store
1
1
1 2
3
3
1
1
1
3
3
2
4
Critical Dependency
Noncritical Dependency
Website frontend needs to handle
dependency failures of Search
service gracefully.
View Order service can generally
ignore dependency failures of Order
Management service.

52.
©2008–18 New Relic, Inc. All rights reserved
Classic, True Story Example

53.
©2008–18 New Relic, Inc. All rights reserved
True Story Avatar Example
My Web Application

54.
©2008–18 New Relic, Inc. All rights reserved
True Story Avatar Example
My Web Application ?

55.
©2008–18 New Relic, Inc. All rights reserved
True Story Avatar Example
My Web Application X

56.
©2008–18 New Relic, Inc. All rights reserved
True Story Avatar Example
My Web Application
Tier 1 Service(s)
Tier 2 Service(s)
Tier 4 Service

57.
©2008–18 New Relic, Inc. All rights reserved
Service Tiers
Tier 1
Tier 2
Tier 3
Tier 4
Mostcriticalservices
Leastcriticalservices
A label applied to every service
Simple, basic, but critical component in
performance contracts between services.

58.
Thank You
©2008–18 New Relic, Inc. All rights reserved
Lee Atchison
Senior Director Strategic Architecture at New Relic, Inc.
leeatchison@leeatchison
Architecting for Scale
By: Lee Atchison
Published by: O’Reilly Media
http://scalinginthe.cloud
“#1, Top DevOps Books to read in 2018”
– Apiumhub
“10 Tech Books for Summer Reading”
– Informaiton Week/Network Computing