© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
How Hess Has Continued to Optimize
the AWS Cloud after Migrating
S u n n y S u n k a r a – A W S S o l u t i o n s A r c h i t e c t
E N T 2 1 8
H a i d e r A b d u l l a h – H e s s C o r p I T S y s t e m s S p e c i a l i s t – C l o u d A r c h i t e c t
N o v e m b e r 2 9 , 2 0 1 7

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to Expect from the Session
• Hess corporation introduction
• Hess migration journey
• Optimization strategies on AWS
• Hess optimization story
• Post migration challenges
• Solutions implemented
• Driving innovation
• Lessons learned

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About Hess—History

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About Hess—Core Business
EXPLORATION GLOBAL PRODUCTION
• Exploration strategy focuses on
proven and emerging oil-prone
plays that leverage our
capabilities
• Participated in significant oil
discovery offshore Guyana, with
estimated recoverable
resources of more than 1 billion
barrels of oil equivalent
• World class acreage position in
North Dakota Bakken
Formation, premier shale play
in U.S.
• Proven track record applying
lean manufacturing techniques
to drive down costs and
increase well productivity and
efficiency
• Top quartile performance in
offshore drilling and project
delivery
• Pioneer of sub-sea drilling
techniques in Denmark and
West Africa
• Seventh largest producer
offshore Gulf of Mexico

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About Hess—Key Differentiator
Embracing a Culture of Continuous Improvement to Deliver Top-Quartile Business Results
Our performance is driven by a distinctive lean culture that equips and empowers people at every level to
solve problems and spot opportunities for continuous improvement and innovation

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hess Migration Journey
First Divesture
• Divested Downstream business
• 170 instances, 6 months
Second Divesture
• Divested Retail business
• 90 instances, 3 months
• 90% lower migration cost
compared to first divesture
Migrating Core Business
• 550+ instances migrated
• Majority Windows workloads
• Commercial packaged
software suites
• Custom built applications
• SQL Server BYOL
• Oracle on RDS
• Greenfield installations and
migration tools
2013
2014
2016

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Encourage and Enable Teams to Optimize for…
Incentive,
skills, time
investment
Business
Needs
Service
Management
Application
Management
Security
Enterprise
Services
Cost
Management
Leverage AWS tools and
features to continually
improving operational costs
(Consolidated billing, RIs,
discounts)
Utilize latest AWS tools to
continuously improve service
management methods/
processes
Utilize AWS best practices and
tools (e.g. DevOps, CI/CD) to
continuously improve
application management
methods / tools
Continuously seek ways to
aggregate and improve shared
services

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost Optimization Is…
going from…
to…
pay for what you use
pay for what you need
USAGE OVER TIME
COSTPERUNIT

Where do you start?

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Five Pillars of Cost Optimization
Rightsizing Pick the right
pricing model
Increase elasticity
Measuring &
monitoring
Match usage to
storage class
Right-sizing

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pillar 1: Right-Sizing
Right-sizing
• Selecting the cheapest instance while meeting
performance needs
• Look at CPU, RAM, storage, and network
utilization to identify instances that could be
downsized

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pillar 2: Increase Elasticity
“Hug your workloads, not your servers”
AWS allows you to fit cost to today’s demand
Traditional IT tends to fit cost to peak demand expected in 3-5 years' time

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pillar 3: Leveraging the Right Pricing Model
Reserved Instance (RIs)
On-Demand
EC2 Spot

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RIs Are Best Used for Always-On Instances
Commitment level
1 year (approx. payback 7-9 months)
3 year (approx. payback 10-18 months)
AWS services offering RIs
Amazon EC2 & EC2 Hosts
Amazon RDS
Amazon Redshift
Amazon ElastiCache
Amazon DynamoDB*
Amazon CloudFront*
*Discount for commitment, but not a RI

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pillar 4: Leveraging the Right Storage Class
Elastic Block Store (EBS)
Simple Storage Service (S3)

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Use Ap p rop riate ly Size d EB S
y o u c a n m o d i f y i f n e e d e d w i t h z e r o d o w n t i m e *
* Decreasing volume size is possible, but requires a different process
Some older instances & volumes may not support zero downtime modification

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Object Storage Classes on Amazon S3
Active data Archive dataInfrequently accessed data
Standard
Hot
Standard - Infrequent Access
Warm
Amazon Glacier
Cold

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pillar 5: Measure and Monitor
% Instances turned off daily
% Of instances right-sized
% Always on resources covered by RIs
% RI utilization
What KPI makes sense for this
workload?
✔ ✔
✔ ✔
✘
✘
✘
✘
Set up metrics to define success and track progress

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Options

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Five Pillars of Cost Optimization
Rightsizing Pick the right
pricing model
Increase elasticity
Measuring &
monitoring
Match usage to
storage class
Right-sizing

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hess Customer Story

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Post-Migration Challenges Faced
Efficiency
• Proper capacity provisioning, optimized for cost &
performance
• Visibility/Transparency
• Ownership of resource usage
Management/Governance
• Operational tasks
• Security
Agility
• New instance provisioning (self-service)

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Provisioning Challenge

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Before We Go Further—Defining Serverless
AWS
Lambda Amazon EFS Amazon
S3
Amazon
DynamoDB
Amazon
ElastiCache
Amazon
RDS
Amazon API
Gateway*
Amazon
SES
Amazon
SQS
Amazon
SNS
AWS Step
Functions
Amazon
Kinesis

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Core Serverless Compute—AWS Lambda
No servers
to manage
Continuous
scaling
Sub-second
metering

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Challenge—Cost Optimization
Goals
• Provision resources at the proper levels
• Use resources only when necessary
• Provide transparency to costs AND waste
• Self-service using standardized toolsets
• Drive cultural change

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution—Cost Optimization
• Built easy to consume Spotfire Dashboards
• Implemented flexible ‘On/Off’ cost optimization schedules
• 50% of instances are on a schedule
31% savings as a % of total EC2 compute cost
75+% (compared to on-demand)
Savings for instances on schedule:

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Small Amounts Add Up Quickly…

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
{
"scheduleID": 3,
“stopEvents":[
{
"day": 1,
"times": ["1800"]
},
{
"day": 2,
"times": ["1800"]
}
...
],
"startEvents":[
{
"day": 1,
"times": ["0600"]
}
...
Cost Optimization Scheduler
Amazon
CloudWatch
Event
(every 5
minutes)
Amazon S3
Bucket
AWS
Lambda
Lambda
Function
Amazon
EC2
Instances
{
“Enabled": true,
“OptimizerType": “OnOff”,
“OptimizeScheduleID": 3,
}

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Walkthrough—Cost Optimization
Dashboards

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
PowerShell
Solution—Cost Optimization (Disk Utilization)
Amazon CloudWatch
Event
(once daily)
AWS Lambda
Lambda
Function
Amazon EC2
Target
Instances
Step Functions
DynamoDB
Lambda
Function
Table: DiskUtil
SQL Server
Simple Systems Manager

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Challenge—Management/Governance
Goals
• Be proactive: detect/avoid error scenarios
• Automate routine management tasks
• Reduce waste
• Raise awareness of Cloud capabilities

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution—Management/Governance
• Error Detection/Recovery
• Status Check Failures (create tickets for service management team) – every 15 minutes
• Restore Volumes for Instance – On-Demand
• Operational Governance
• Create Snapshots of Instances – every night
• Cleanup ‘expired’ snapshots – every night
• Service Limits to Increase – every night
• Enforcement of tags (propagate to Volumes/Snapshots/Generate Compliance report) – every night
• Cost Governance
• Orphaned Volumes Report & Cleanup – every night
• Cost Optimizer – every 5 minutes
• Extract cost information to SQL – every weekday
• Security
• Send Alert for Access Keys older than [x] days – every night
• Disable Access Keys not used in [x] days – every night
• Send alerts for S3 Buckets with Public access – every 15 minutes

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution—Management/Governance
Culture of Cloud first
• Immersion days
• Internal training
• Certifications
• External events
Alexa, how
many
orphaned
volumes?
Alexa, how
much oil did we
produce in
North Dakota
yesterday ?

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Walkthrough—Restoring Instances

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Challenge—Self-Service Provisioning
Goals
• Easy to use/standardized one page request form
• Approval workflow before build
• No login (to AWS Console) required
• Accessible from internal network
• Joined to Domain
• Local permissions granted to support owners
• Standard tools installed
• Quick turn-around time
• Agility—be able to add new build types
• Make it SERVERLESS!

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Walkthrough—Self-Service
Provisioning

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution—Provisioning (Submission Flow)

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution—Provisioning (Approval Flow)

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution—Provisioning (Build Flow)

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hess Monthly Cost for AWS Lambda

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
S u m m a r y o f M e c h a n i s m s : P r o c e s s , P o l i c y , T o o l s , & P e o p l e
3. Budgeting,
Orchestration & Approval
(balanced against agility)
4. Agree Tagging with
Stakeholders
Responsibility
1. Enable Architecting
for Cost
5. Address
Improper Tagging
6. Metrics, Targets,
Tools, and Review

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Path Forward
• Continue migration of remaining workloads
• Continuing to look for ways to optimize and automate
• AMI build process
• Containerization of apps
• Serverless adoption
• Continuous Improvement & Governance
• Onboarding to a managed service provider soon but
retaining optimization currently in place

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Questions

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
Please complete your evaluations

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Appendix