The document discusses the challenges of moving to cloud computing at Fidelity Investments. It describes Fidelity's adoption of a centralized cloud organization called Enterprise Cloud Computing to power digital services. It outlines some of the strategic, operational, and technical challenges of cloud adoption, including cloud strategy, security, compliance, operations, cost management, application design, and deployment. It emphasizes the importance of automation, DevOps practices, and designing applications to be cloud-native and take advantage of the scalability and elasticity of the cloud.

1. The Cloud Challenge
By Rick Moran
Head of Enterprise Cloud Computing Architecture, Fidelity Investments
https://www.linkedin.com/in/richardjmoran/

2. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Disclaimer
This presentation is a case study of the Fidelity Investments cloud experience
highlighting strategy, issues encountered and lessons learned. This is not an
endorsement or recommendation of any vendor product or service.

3. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
About Fidelity Investments and ECC
3
• We provide investment management and administration expertise to a diverse set of customers from
individual investors to businesses, financial advisors, and institutions with over 6.8 Billion AUM
• Fidelity has more than 10,000 technologists – approximately 1/4 of our workforce
• We deliver the best customer experience, we’ve always invested deeply in technology and have a
long track record of innovation, from offering the first mutual fund.
• We nurture an environment that encourages risk-taking and supports new idea generation.
• We focus constantly on what’s next and we take intelligent risks rather than follow the
crowd to make every product, service, and experience better for our customers.
• Enterprise Cloud Computing (ECC), is a centralized Cloud organization for Fidelity Investments: Our
mission is to power the next generation of Fidelity digital services for our 27 million customers.

4. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Build all the things…
Ship all the things…
Run all the things…
In the cloud!

5. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Epic Challenges
The Ice Bucket Challenge
The Cinnamon Challenge
5
The Cloud Challenge

6. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
The Cloud Landscape
6
Azure
GCPPublic
Hybrid
Private
Innovation
Machine Learning
Artificial Intelligence
Data Analytics
Risk
Audit
Compliance
Security
Lot’s of options and lot’s of questions

7. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
What is the cloud?
7
Scalable
Increase or decrease to meet demand
Dynamic
Self-service provisioning with RBAC
Pooled
Multi-tenant resources billed by usage
Elastic
Dynamically adjustable services

8. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Developer
Why Go to Cloud?
8
Business
Datacenter
Customer
Market Event
• Time to Market
• Elastic Scaling
• Utility Billing
• Innovation
Vendor

9. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Challenges
Cloud Strategy Cloud Maturity Cloud Security
Cloud Audit
and
Compliance
Cloud
Operations
Cloud Cost
Management
Cloud
Application
Design
Cloud
Application
Deployment
Cloud DevOps
Automation
Cloud
Portability
9

10. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
What’s your first move?
10

11. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Strategy
11
INDIVIDUALLY MANAGED BY EACH Line of
Business
FEDERATED APPROACH
• Allow LOB’s to focus on
customer value rather than
platform decisions
• Create ease for developers
• Ensure seamless end-to-end
security and compliance
• Certify the cloud provider
services that may be used
• Drive single data platform
strategy to support new
technologies like AI
• Automate to reduce potential for
human error
CENTRALIZED APPROACH
CENTRALLYMANAGED BY an Enterprise Cloud Service Broker
ContainerCloudCD
NativeCloudCD
VirtualMachineCD
Deliver differentiated value to our customers by simplifying our environment and the application
development experience to enable speed, security, compliance and reliability at scale.

12. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Strategy Principles
12
Leverage Open
Source
Buy versus build Minimal
customizations
Self-Service
Capabilities
Everything as a
Service with
API’s
Everything as
code
Full Automation 12-Factor design
architecture
Pace over
perfection
Multi-Cloud
Support
Scalable Cost
Model
Security and
Audit Support

13. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Are you ready?
13

14. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Readiness Model
14
Not Cloud Ready
• Minimal Cloud skill sets
• Very little automation
• Minimal DevOps practice
• Minimal Agile Practice
• No 12-Factor App design
• Minimal app observability
• Minimal Security Controls
Scan
• Growing Cloud skill sets
• Some CI/CD automation
• Emerging DevOps practice
• Emerging Agile Practice
• Some 12-Factor App design
• Emerging App Observability
• Some Security Controls
On the Path to Cloud Ready
Try
Cloud Ready
• Strong Cloud Skill sets
• Full CI/CD Automated
• Mature DevOps Practice
• Mature Agile Practice
• Mature 12-Factor App design
• Full App Observability
• Automated Security Controls
Scale

15. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Are you secure?
15

16. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Security – Principles
16
Encryption at rest and in motion
Enterprise managed encryption keys (BYOK)
Federated identity management integration
Role based access controls (IAM)
Network Isolation and policies at software product level
Ingress and egress access controls
Automated governance gates and controls
Monitoring, detection and auto-remediation

17. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Security – Shared Responsibility Model
17
Traditional Data
Centers
Monitoring Governance Policy
Incident Escalation
Public Cloud - CSP
Private Cloud
IAM CIRT
Threat &
Vulnerability
Pen Testing
Insider Threat
External Threat
Enterprise Security Policies and Standards
Enterprise Security
Responsible for all cyber
security across the firm
Centralized Cloud
Security
SaaS providers
Cloud Security
Responsible for cloud specific controls /
access / logging and monitoring

18. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Security – Automated Controls
18
Asset Documentation
Security Code Scan
Artifact Integrity Check
Approved Machine Images (AMI)
Infrastructure-as-code Scan
Penetration Test
Automation Gate
Applications
Preventative Controls Detective Controls Responsive Controls
Category Checks
Compute 60
Storage 20
Network 54
Identity & Access Mgmt 49
Encryption & Key Mgmt 12
Database 33
Governance 52
Total 755
Examples:
• Unauthorized OS Images
• Untagged Instances
• Restricted Geographic Regions
• Unencrypted S3 Buckets
• Global-Readable S3 Buckets
Automated Security & Compliance
Checks of “everything-as-code”
Automated Security &
Compliance Monitoring
Auto-Remediation of
Non-Compliant Configs
Upon detection, these
misconfigurations are
corrected in <15 secs
Pipeline Cloud

19. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Compliance and Audit
19
Corporate Standards
ISO/IEC 27001
Supporting Fidelity’s Enterprise Certification against the
different ISO standards: ISO/IEC 20000, ISO 14001, ISO
22301 as the BU use certification as business differentiator
SOC1, SOC2, SOC3
Cloud Security Alliance (CSA
Framework)
1.For use internally to reduce costs by
eliminating duplicate efforts and
increasing transparency.
2.Being adopted by E3C FinServ
Industry Council as an industry
standard
3. Fidelity is providing leadership in E3C
sharing ECC experience building
secure/compliant cloud solutions
Leveraged & updating existing Corporate Standards to support
regulatory demands while aligning to the compliance
requirements mapped within the CSA matrix
• Mapping
• Automation
• Management
NIST 800-53
Set of industry standard controls; prioritized by Enterprise
Cyber Security for implementation and tracking.
Audit as code with self-service reporting and dashboards for auditors
Category / Capability Description
SOC1
SOC2
SOC3
ISO27001
HIPPA
SOX
etc.
1.0 Access Management
1.1 RBAC
1.2 Logging
1.3 MFA
1.4 Use it or Lose it
…
2.0 Data Protection
2.1 Encryption
2.1.1 Crypto Algorithms
2.1.2 Levels
2.2 Data At Rest
2.2 Data In Transit
2.3 Key Management
…
Industry Compliance Standards
CSA Matrix

20. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Are you prepared to operate?
20

21. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Operations
21
TRACING
• Deep Dive Contextual Monitoring
Host system metrics and request traces on a single pane of glass for more context and easy
correlation
• Service Discovery & Mapping
Decompose application into component services and draw observed dependencies between
these services in real time
• Trace Search
Use tagging to filter application requests flowing through a service
ALERTING
• Anomaly Detection
Identify differentiation from past trends
while accounting for seasonal, day-of-
week and time-of-day patterns
• Composite monitor
Combine many individual monitors into
one so that you can define more
specific alert conditions
LOGGING
• Pipelines
Apply a list of sequential processors against a filtered subset of incoming
logs
• Log Monitoring
Define search queries and thresholds for event escalation
Infrastructure & SaaS Metrics Application Performance - APM
Centralized Logs
Ops team provide shared tools, process, visibility and tracking of
product success
Operational
Tools
Health
Visibility
Proactive
Operations
Operating
Process
CSP monitoring
Health dashboards
System Status
Performance analytics
Monitoring, logging platform
Notification capabilities
DevOps tools
Process automation
Change process
Major incident resolution
Chaos engineering
Health assessment
Capacity and Performance
framework and reporting

22. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Resource Management
Majority of cloud
spend is
infrastructure
• EC2
• EBS
• S3
• RDS
• Network
Utilization of cloud
infrastructure is low
• Generally under 20%
• Memory as a metric
• Over-sized machines
running all the time
Migration strategies
are driving existing
issues to cloud
• Lift and shift
• Legacy application
designed for VM and
physical
Cost Management
approaches
• Visibility of cost
• Reserved, Spot
instances and latest
machine types
• Auto-scaling and
automated instance
shut down
• Right sizing machines
22

23. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Do you have a good design?

24. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Design – Thinking Differently
Self Healing Systems
• Smart Applications
• Circuit Breakers
• Health check and Metrics
Frequent Rehydration
• The instance can go away at
any moment
• Frequent Rebuild with no
more patching
The DevOps pipeline is
key to success!
• Automated Code Reviews
• Quality checks at every step
Learn to love services!
• Attach to Services vs. Code it
yourself
• Services and events beat file
systems and infrastructure
App teams responsible
for Security
• BYOK or client-side encryption
• Data Validation
No interactive access to
production
• Love those logs and alerts!
• Visibility is through L&T
Know your networking
up front!
• Isolation at multiple levels
• Firewall rules, GTM routing
• Egress, Ingress, Peering
Design for the cloud
• Elasticity is Performance
• 12 Factor matters
• Stateless Applications
• Fault Tolerant
Deploying smaller apps
• Horizontal scaling (scale out)
• Memory, storage and CPU cost
• <= 1GB container
• Micro-Services
Efficient use of
infrastructure
• Auto-scaling to meet demand
• Destroying environments
when not needed

25. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Design – Resiliency
• Application Resiliency
• Application Fault Tolerance
• Application Health Checks
• Application Circuit Breakers
• Application Auto-Scaling (Horizontal)
• Platform Resiliency
• Platform Fault Tolerance
• Platform Auto-Scaling (Vertical)
• Platform Resource Management
• Platform Transaction Management
Smart Infrastructure Smart Applications

26. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Design – Cloud Application
Patterns
26

27. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Are you on target?
27

28. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Deployment – Runtime Platforms
28
Hardware
IaaS
CaaS
PaaS
FaaS
Strategic goal:
Place workloads where they will maximize value and best align to LOB strategies
Slower time
to market at
higher cost
to the
business
Faster time
to market
at lower
cost to the
business
FaaS
Paas
CaaS
IaaS
Hardware
Level of Effort to Support
(OS/Language/Security)

29. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Deployment – Platform as a
Service
Application
- # of instances
- App Name
Service Bindings
- Service 1
- Service 2
Routes
- Route 1
- Route 2
Cloud Foundry
Application Config - YAML
API
Garden
App 1
Instance 1
Stem Cell 2
Garden
Garden
Elastic Runtime
App Services
App 1
Instance 2
App 1
Instance 3
App 2
Instance 1
App 2
Instance 2App 1 App 2
Logging
Monitoring
Metrics
Caching
Stem Cell 1
Stem Cell 3
Infrastructure Management (BOSH)

30. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Deployment – Container as a
Service
Deployment
Pod1
- Container 1
- Container 2
- Replicas = 3
Pod 2
- Container 3
- Replicas = 2
Kubernetes Cluster Services
Application Config - YAML
API
Worker Node 1
Kublet
POD 1
Replica 1
POD 2
Replica 1
Worker Node 2
Kublet
POD 1
Replica 2
Worker Node 3
Kublet
POD 1
Replica 3
POD 2
Replica 2
Desired State Management
KCM
Node
Volume
Route
Service
Infrastructure Management (BOSH)Infrastructure Management (BOSH)

31. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Deployment– PaaS and CaaS
Models
Buildpack
Language Runtime
Stem Cell
Operating System
Ubuntu Windows
Warden
Container
GARDEN
Code
CI Build
Framework
Application
WAR
TAR
Assembly
Kubernetes
NHC
Azure
NHC
Azure
CF Push
Code
CI Build
Framework
Application
WAR
TAR
Assembly
Docker Build
B
U
I
L
D
P
A
C
K
D
O
C
K
E
R
Docker
Container
Base
Image

32. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Deployment– Platform Effort
Comparison
32
Outcome and capabilities before of technology

33. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Application Deployment – Runtime Platform
GuidanceApplication Runtime
Container + PaaS - Cloud Agnostic
Web Application Web Service
Micro-Service Batch Application
Asynchronous App
Container Runtime
Scheduling + Orchestration – Cloud Specific
Batch Application
Vendor Products
Non-Container
Native Cloud – Cloud Specific
Function as a Service
Infrastructure Services
Database ServicesMessaging ServicesMicro-Service
Web Application Web Service
EKS
Azure
Asynchronous App Data Analytics
Legacy Application
Legacy Application
AKS PKS

34. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Are you automated?

35. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
DevOps Automation – Putting it all Together
LOB Dev Team Responsibility - Central Responsibility - Security Gate -
Dev
QA
Prod
NHC
Security
Governance
Developer
Code Hub
Commit
Manifest Code
Artifact Repository
Build Artifact
OSS
Components ECC Base Images App Images
CI Pipeline
Build Artifact
Code Quality
Scan
Unit Test
App Images
Security Scan
& Fingerprint Artifacts
Continuous
Image Scans
Check
Fingerprint
Configuration
Service
Deploy
Tool
Functional
Tests
Logging &
Monitoring
Patterns
Standard OS and Language Runtimes
Fargate
EKS
Image Factory
Readiness
Check
Secrets
Management
CFT, ARM

36. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
DevOps Automation – Pipeline Outcomes
Standard OS and Language Runtimes
Continuous Image Scans
Security Scan & Fingerprint Artifacts
Fingerprinting Check
Configuration Service
Secrets Management
Readiness Check

37. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Do you have an exit strategy?
37

38. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Cloud Portability
38
Thoughtful choices about native service integration
Multi-Cloud deployment strategy
Full automation with everything as code
Containerization as first deployment choice
Azure GCP

39. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Get you head in the clouds….
41
Have a business strategy for cloud not just a technology strategy
Be careful about going all in on a single CSP
Watch out for the infrastructure bubble
Pay attention to the shared responsibility model
Make the leap from infrastructure reliability to application resiliency

40. Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Thank You

41. > Stay Connected.
Richard J. Moran: richard.j.moran@fmr.com
#springon
e
@s1
p