This document discusses HERE Transit, a backend service for a Windows Phone mobile application. It has a large number of CloudFormation stacks across multiple regions and environments. Plain JSON would be difficult to work with due to the complexity, so the service uses Python code to generate CloudFormation templates. Examples are provided showing how configurations are pulled from files and templates are generated dynamically based on environment settings.

1. 1
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

2. 2
• What is HERE Transit?
• Why not plain JSON?
• Examples
Agenda
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

3. 33
Backend service for the Windows Phone mobile
application and (soon) other platforms.
3 regions.
7 CloudFormation stacks per production
deployment of the service.
6 development environments (“development”,
“data integration”, “functional testing”, “load
testing”, “qa”, “demo”): at least 2 CF stacks for
each.
And staging.
What is HERE Transit
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

4. 44
1. Big number of CF stacks
- some are functionally different (e.g.
ElasticSearch stack is functionally
different from Proxy stack)
- many of them have minor differences (we have
different scaling requirements for
production and development, every region
uses different AMI)
- some stacks are identical but the number of
stacks differs (we use substacks for data shards –
number of shards is different in production
and staging)
Why not plain JSON?
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

5. 55
2. Resources outside of CF stack
- every region has an EIP and a “special” SG
– need to keep track of them, but not to
add to CF (stacks come and go, but these
guys should be there always)
Why not plain JSON?
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

6. 66
3. CF in JSON is pain to read and edit
- python code is just more readable
- you are aware where code is and where
data is (not true with CF)
Why not plain JSON?
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

7. 77
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bashn",
"until $(cfn-init -v -c thor -s ",
{
"Ref": "AWS::StackName"
},
" -r CfnRootRole",
" --region ",
Why not plain JSON?
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

8. 88
Why not plain JSON?
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

9. 99
for idx in range(1, int(cfgs['kernel_groups']) + 1):
group = "G%d" % idx
group_st = "Kernels%sStack" % group
kernel_group_substack_template = Template()
# adding resources to a substack ...
Examples
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

10. 1010
user_data_tpl = jinja2.Template(open("user_data.sh", "r").read())
user_data_proxy = user_data_tpl.render(role = "proxy",
puppet_env = cfg['puppet_env'],
puppetmaster = cfg['puppetmaster_hostname'])
# cfg dictionary is generated out of toml config
Examples
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

11. 1111
if cfgf['env'] == 'pr' or cfgf['env'] == 'st':
healthcheck_url = "HTTP:5555/rest/info/v2/status?accessId=foobar"
else:
healthcheck_url = "TCP:5555"
Examples
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

12. 1212
loadbalancer = elb.LoadBalancer(
"MetaroutersELB",
AvailabilityZones=GetAZs(""),
Listeners=[
elb.Listener(
LoadBalancerPort="80",
InstancePort="5555",
Protocol="HTTP"
)
],
HealthCheck=elb.HealthCheck(
Target=healthcheck_urlTarget=healthcheck_url,,
HealthyThreshold="2",
UnhealthyThreshold="2",
Interval="5",
Timeout="3"
),
)
template.add_resource(loadbalancer)
Examples
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l

13. 1313
©
2
0
1
4
H
E
R
E
|
P
o
w
e
r
P
o
n
t
t
e
m
p
l