Focusing on business applications in financial services, this presentations talks about what we mean by resilience, the challenges and solutions. It uses a case-study of the resiliency strategy in a tier-1 bank.

1. STRICTLY CONFIDENTIAL
Application Resilience: Challenges & Good Practice
City Business Club - ISITC Europe | July 2019
Dr Aled Sage
aled@cloudsoft.io

2. © Cloudsoft Corporation 2019
About Aled Sage
VP Engineering @ Cloudsoft
PhD and 20 years experience with
distributed systems
Expertise in Cloud, Devops and Automation
Across different verticals, including
financial services
@aledsage
aled@cloudsoft.io
linkedin.com/in/aledsage
github.com/aledsage© Cloudsoft Corporation 2019 2

3. © Cloudsoft Corporation 2019
1. What is resiliency?
2. Changing landscape
3. Case study
4. Problems & solutions
Agenda
© Cloudsoft Corporation 2019 3

4. © Cloudsoft Corporation 2019
• Business applications
• Service level objectives (SLOs)
Service level agreements (SLAs)
• Aim is not 100% availability
• “Availability” is not binary
• Design for failure
• Proactive and reactive
What is Resiliency
Google Site Reliability Engineers (SRE)
“Striving for Imperfection: Using an error budget
to move fast without compromising high
reliability”
CRM, payroll, analysis, trading systems,
settlement systems, …
Proactive: High availability (HA)
Avoid problems
Reactive: Know when something is wrong
Know how to respond
Repair
Disaster recovery (DR)© Cloudsoft Corporation 2019 4

5. © Cloudsoft Corporation 2019
Business Impact
© Cloudsoft Corporation 2019 5
Gartner
Why Business Leaders Don’t Care About the Cost of Downtime
9 April 2019

6. © Cloudsoft Corporation 2019
• Private and public cloud
• Kubernetes and Openshift (using containers)
• Bare-metal and mainframes
• Business wants agility
• Security and reliability still vital
• All while driving down costs
Changing Landscape
• Metro Bank, Monzo, Starling
• Facebook? Amazon?
© Cloudsoft Corporation 2019 6

7. © Cloudsoft Corporation 2019
Comparison with Disruptors
Incumbents Disruptors
Hybrid environment Everything is cloud-native
Chaos engineering?
“No way!”
Kill servers in production
Wall Street bank: decrease VM
provisioning to 9 hours
New container in seconds;
New Virtual Machines in minute(s)
3-6 month releases Small changes
several times a day
Improve the basics:
● Improve resiliency and
agility of existing apps
● Fit with old-world and the
future
© Cloudsoft Corporation 2019
100s of applications
Many lines of business
Small product set
Legacy high-value apps No legacy apps
7

8. © Cloudsoft Corporation 2019
APPLICATION-CENTRIC APPROACH
✓ Empower and involve application architects and developers
✓ Codify best practices and use of the bank’s strategic tools
✓ Consistent way to automate resiliency
Self-healing applications!
Your application can automatically detect and fix problems
Case-study: Tier-1 Investment Bank
ACHIEVEMENTS UNLOCKED
✓ Application-level Resiliency at scale
✓ Meets business & regulatory needs
✓ Across multiple evolving platforms
© Cloudsoft Corporation 2019 8

9. © Cloudsoft Corporation 2019
Challenges
© Cloudsoft Corporation 2019 9
Wide variation in
techniques and quality;
bespoke processes.
Involves many systems.
Fiefdoms & politics.
Resiliency is a
cross-cutting concern.
Manual procedures and
tribal knowledge.
Responses to
degradation or failure
not automated.
Manual Bespoke
Operations
fragmented
Rarely tested
Sufficient to meet
auditors’ requirements.
But little practice or
variation in scenarios;
manual testing is
expensive.

10. © Cloudsoft Corporation 2019
Problems and Solutions
Problems Solutions
Tribal knowledge, rarely tested Gamedays
Processes, environments and applications are
complicated and ever-changing
Post mortems & continual improvement
High availability and DR needs understanding of
application, but infrastructure-focused
Application-centric approach
Bespoke / inconsistent approaches Empower the application architects;
Codify and share best practices.
© Cloudsoft Corporation 2019 10

11. © Cloudsoft Corporation 2019
Problems and Solutions

12. © Cloudsoft Corporation 2019
Problems and Solutions
Problems Solutions
Manual runbooks Automated recovery
Involves many systems Encourage APIs and automation;
“glue code” to codify their use;
focus on common application architectures.
Runbooks and CMDB out-of-date “Living model” of the application;
Deployment and in-life management codified in this model.
© Cloudsoft Corporation 2019 12

13. © Cloudsoft Corporation 2019
• Restart the web-servers (e.g. on an out-of-memory)
• Repave servers or clusters (e.g. when a restart fails, or an update is
required)
• Auto-scale based on latency, load or time-of-day
• Manage DR within a region and/or across regions
Automated Recovery
© Cloudsoft Corporation 2019 13

14. © Cloudsoft Corporation 2019
Self Managing Systems: Autonomics
© Cloudsoft Corporation 2019 14
Self configuring
• Automated configuration of components and systems follows
high-level policies. Rest of system adjusts automatically and
seamlessly
Self healing
• System automatically detects, diagnoses, and repairs software and
hardware problems
Self optimizing
• Components and systems continually seek opportunities to
improve their own performance and efficiency
Self protecting
• System automatically defends against malicious attacks or
cascading failures. It uses early warning to anticipate and prevent
system wide failures

15. © Cloudsoft Corporation 2019
Cloudsoft AMP
© Cloudsoft Corporation 2019 15
AMP streamlines development, operations and governance for any
application on any cloud
Rapidly Scalable
Autonomic policy-based
management
Cloud Enabling
Liberate applications
from infrastructure
Powerful Tools
Consistent in-life management
tooling and features
Improve Visibility
Unified management
view of applications

16. © Cloudsoft Corporation 2019
Suggested Next Steps
© Cloudsoft Corporation 2019
This week
❏ Get the Gartner report: “Why Business Leaders Don’t Care About the Cost of Downtime” free of charge
❏ Talk to person(s) in charge of application resiliency
❏ Example post mortem from the last major incident; how was this knowledge shared and used?
❏ How are application authors involved in planning for and implementing resiliency?
❏ Is there consistent monitoring and recovery strategies (focusing 80:20 rule of existing apps)?
❏ How often is it tested? Are there gamedays?
Next 90 days
❏ Identify example app(s)
❏ Get a review from an expert of an example workload (“well-architected review”)
❏ Organise gamedays
❏ Introduce post mortems
Next 6 months
❏ Consider automation software at the application level for ‘low-hanging fruit’
❏ Involve and empower your application authors
16
get report for free
cloudsoft.io/report

17. 17
e: aled@cloudsoft.io
w: cloudsoft.io
Thank You!
Any Questions?