Temporaneous Test Environments
Fergal Hynes
Detailed VM control script walkthrough - #1 of 23
Source - AWS
Test Environments
Technology
TestX
Cost reduction
ContinuousX
Items
that
matters
most to
business
owners
Understand this…
Most organisations target COST as their first key metric
Gartner 2017
Become
relevant!
A Testers Tale..
• Shared environments
• Broken integrations
• Not scalable
• Changing config
Move to public cloud infrastructure can
range from 50%less expensive
to 20%more expensive…mmmh..
CIO Survey 2017:
New investment priorities in next
12 months:
#2 cloud
services/solutions
We’re heading to the cloud, be
on point
So..
A recent application development project, 20 machines
(excl. vcs, build mgmt. etc):
1,456 3,360
hours per week
Vs.
~60% resource saving
(& time = $!)
So..
74% of hours during the year are associated with nights, weekend and annual
leave
1 x 'managed' AWS c4.2xlarge $2.5k per
year saving over un-managed
20 x 'managed' AWS c4.2xlarge
$50k per year saving over un-
managed
200 x 'managed' AWS c4.2xlarge
$500k per year saving
over un-managed
AWS
c4.2xlarge
Event Driven Architecture for Test Environments #1
Event Driven Architecture for Test Environments #2
Be good at…
Analysis
Testing
Environment
Management
Environment Management
An opportunity to
contribute and stand out…
• Fresh technical challenge for testers
• Quick wins
• Valued within DevOps culture
Cloud may not be the best option
Cloud vs On-Prem
Be aware of TCO.
On-Prem involves
Hardware + Electricity +
Maintenance + Space + Risk
Complexity of choices requires a clear
strategy
Beware of information overload re.
services and cost structure…
(Extreme amount of resources are on the web)
You need a strategy
First steps..
Review all
supplier
offerings
Understand
pricing models
Turn off unused
resources
Get discounts /
use free
resources
Right-size all
resources
Use RIs for
always-on
Implement Cloud
Service Expense
Management
Encourage cost
awareness
Understand
pricing models
• AWS – Free-Tier, On-demand
(Premium), RIs and Spot
Instances
• Azure – PAYG, Compute (plans)
• GCE – Sustained Use,
Preemptible
Review frequently
Review all
supplier
offerings
• All use different models
• ‘Intuitive use’ review
• Feature review
• Security review
Turn off unused
resources
• Automate (can use tags..)
• Exercise care to ensure no
prod machines are
impacted
• Terminate != Stop!
Get discounts /
use free
resources
• Free tier prototyping
• 20k-120k credits
• Negotiate big stuff
Review frequently
Right-size all
resources
• Examine constraints
• CPU / RAM / IOPS
• Don’t copy on-prem, you might
have under/oversized –
Opportunity for waste analysis
(LEAN)
• Avoid overbuying
• ‘Burstable’ available
Use RIs for
always-on
• Plan well (years) and consider
B.E.
• Perfect for prod &
ContinuousX machines
• Large discounts (up to 75%)
• Many options available (get
educated)
Review frequently
Implement Cloud
Service Expense
Management
• Native
• Commercial 3rd Party
• Open source
Encourage cost
awareness
• Public display
• Gamification
• Reward
Review frequently
And yes…..
Measurement, always..
% rightsized
% always on using RIs
etc..
Getting Confident?
Tagging Plan Data I/O
Orphaned
Resources
Throttling Free Tools
Tagging Plan
• Naming Convention
• Track resources
• Useful for Turn-Off
strategy
Data I/O
• Internal comms within
cloud is free
• Ingress / Egress
• Unnecessary data egress
must be eliminated
Throttling
• Pause
resources
when not
required
Free Tools
• Many -
Supplier
tools incl.
alerts
Orphaned
Resources
• Terminate!
• Multiple cloud
account users
• Poor tagging
• Automate clean-
up
What’s possible .. quickly
Create a
cloud
account
Learn
with the
user
interface
Use CMD
line
Create +
Terminate
Stop
+ Start
Config
Execute
2 tests in
series
Get logs
off
Design
pipeline
Automate
pipeline
Identify
FOTU
envs
Triggers
Danger – Early VM Orchestration
Be the puppeteer..
Do the simple things
well yourself first, then
orchestrate
Classical first principle approach
serves you and your team well - Eat
the frog
Know how to change
your own wheel yourself
Continuous X..
Integration
Learning
Improvement
Deployment
Testing
Compliance
Think about
‘Abstract first’ approach
containerization
Pricing Dangers
Use pricing calculators but beware, it's a seller
view
Storage is complex, interesting and
potentially very expensive
Use 3rd party CESMs that span all suppliers
"By 2020, DevOps
Initiatives will lead 50%
of enterprises to
implement continuous
testing using
frameworks and open-
source quality tools" Critical Capabilities for Software
Test Automation - Gartner 2017
A Story
A SaaS enterprise product development
for 12 months for €? infrastructure cost
• ~20 VMs
• A lot of IOPS
And when our credits were used
up, we moved to another
infrastructure provider immediately
for more credit
The Mission
Become a Master of Cost
Reduction
Change your role in line with
Technology
Fight to change Security
Policies that discourage
cloud-tech
Employ DevOps principles - You will
attract talented staff!
Test in Prod
Bread-
crumbing
RCA of Prod
Incidents
Shift focus
from Unit
Tests to API
Hire DevOps
Staff
Become Security (Pen
Test) Experts
UB(T)P
Thank You!
https://www.linkedin.com/in/fergalhynes/

International software testing conference 2017 fergal hynes