AETN Digital Media | Confidential and Proprietary
TEMPLATE ABSTRACTION AND
AUTOMATED DEPLOYMENTS TO FASTLY
A Practical Walkthrough on Real Life Experiences
AETN Digital Media | Confidential and Proprietary
Then ☹
Manual UI Based Implementation
AETN Digital Media | Confidential and Proprietary
30m-1hr x Envs
Manual Process Back Then
• Caching
• Shielding
• Geographical
• Device Detection
• Backends
• Redirects
• Proxies
• Authorization
www.aetv.com
• Caching
• Shielding
• Geographical
• Device Detection
• Backends
• Redirects
• Proxies
• Authorization
www.mylifetime.com
• Caching
• Shielding
• Geographical
• Device Detection
• Backends
• Redirects
• Proxies
• Authorization
www.fyi.tv
• Caching
• Shielding
• Geographical
• Device Detection
• Backends
• Redirects
• Proxies
• Authorization
www.history.com
AETN Digital Media | Confidential and Proprietary
And then we have more sites!
30m-1hr x Env
30m-1hr x Env
30m-1hr x Env
AETN Digital Media | Confidential and Proprietary
Even more URLS added
30m-1hr x Env
30m-1hr x Env
30m-1hr x Env
30m-1hr x Env
30m-1hr x Env
30m-1hr x Env
AETN Digital Media | Confidential and Proprietary
Digging into Legacy Code!
AETN Digital Media | Confidential and Proprietary
Manual Process Back Then!
AETN Digital Media | Confidential and Proprietary
Update (100)
URLS
3 environments
Weeks of
work=
Manual Process Back Then!
AETN Digital Media | Confidential and Proprietary
DevOps Principals
• Collaboration
• Test everything
• Treat Infrastructure as code
• Breakdown the barriers
• Automate everything
• Measure & monitor everything
• Support business and IT agility
• Work as one team end to end
AETN Digital Media | Confidential and Proprietary
And Now ☺
Continuous
Integration
Fastly VCL
Templates APIs
Automated
Testing
EQUALS Awesome!
AETN Digital Media | Confidential and Proprietary
Continuous Integration
Selenium
Automated Process
CI Tool
AETN Digital Media | Confidential and Proprietary
GLOBALFASTLY BOILER PLATE
Sub vcl_recv
Sub vcl_fetch
Sub vcl_hit
Sub vcl_miss
Sub vcl_deliver
Sub vcl_error
Sub vcl_pass
SERVICES
Application URLS
Main.vcl
Backends.vcl
custom.vcl
default.vcl
Fastly Boiler Plate
Sub vcl_recv
call aetn_recv_shield
call aetn_recv_geo
call aetn_recv_cache
aetn.vcl
Template Abstraction
call aetn_recv_device
sub aetn_recv_shield
sub aetn_recv_geo
sub aetn_recv_cache
sub aetn_recv_device
AETN Digital Media | Confidential and Proprietary
Environment Agnostic Implementation
• Separate Service for each Env: {env}-abc.com
• Abstract Env Specific Backends in separate VCLs: {env}-backends.vcl
Backend dev_abc_com {
.first_byte_timeout = 3s;
.connect_timeout = 1s;
.dynamic = true;
.max_connections = 200;
.between_bytes_timeout = 3s;
.share_key = ”COMMONBACKEND";
.port = "443";
.host = ”dev.abc.com";
.probe = {
.url = "/aetn-heartbeat.html";
.interval = 30s;
.timeout = 3s;
.window = 5;
.threshold = 2;
}
}
• Use GITFlow guidelines
• CI Tool reads the config file to deploy to correct env
"environments" : {
"dev" : {
"ref" : "refs/heads",
"servers" : [
{
"id" : ”ABC567defgzThQ"
}
]
},
AETN Digital Media | Confidential and Proprietary
CIDataFlowforManagingEnvAgnostic
CI ToolGitHUB
Config File
(Env to Service Mapping)
Process
qa-backend.vcl
Process
dev-backend.vcl
Process all common files
Process
prod-backend.vcl
Event Payload
Notification to
CI Tool
Process Payload
and identify Event Type
Identify
ENV and
Service ID
QA
FASTLY
Deploy Files and Activate
DEV PROD
Commit VCL
AETN Digital Media | Confidential and Proprietary
DeploymentWorkFlowUsingFastlyAPI
List of Files Modified
Service/
Common
Retrieve the list of Services
Parse the list of services
Retrieve the service and
common files
Fastly Dev Version
(/service/search?name)
Clone New Version
(/service/service_id/version/number/clone)
Deploy Files
(/service/service_id/version/version/vcl/{main})
Activate
(/service/service_id/version/number/activate)
Create Dev Version
(/service/service_id/version/number/clone)
GITHUB
Deployment
Object
Common
Service
No
Yes
AETN Digital Media | Confidential and Proprietary
Update (100)
URLS
3 environments
Weeks of
work=
Automated Process Now!
From
To
Update (100)
URLS
3 environments a few Minutes
=
To
AETN Digital Media | Confidential and Proprietary
Workflow in Action
Update TTL for all sites
AETN Digital Media | Confidential and Proprietary
• Create a feature branch
• GIT Clone Repo
$ git clone https://USERNAME:TOKEN_OR_PASSWORD@github.com/aenetworks/
REPO_NAME.git fastly
Cloning into 'fastly'...
remote: Counting objects: 11441, done.
remote: Compressing objects: 100% (127/127), done.
remote: Total 11441 (delta 71), reused 0 (delta 0), pack-reused 11313
Receiving objects: 100% (11441/11441), 1.67 MiB | 1.17 MiB/s, done.
Resolving deltas: 100% (7939/7939), done.
Checking connectivity... done.
=================================================================================
====================================
WorkflowinAction
$ git checkout -b sd-increase-ttl
Switched to a new branch 'sd-increase-ttl'
=================================================================================
=====================================
if ((beresp.status == 200) && req.url ~ ".(aif|aiff|au|avi|bin|bmp|cab|carb|cct|
cdf|class|css|doc|dcr|dtd|eot|exe|flv|gcf|gff|gif|grv|html|hqx|ico|ini|jpeg|jpg|
js|json|mov|mp3|nc|otf|pct|png|ppc|pws|svg|swa|swf|ttf|txt|vbs|w32|wav|wbmp|wml|
wmlc|wmls|wmlsc|xsd|zip|webp|jxr|hdp|wdp)($|?)" ) {
set beresp.ttl = 36h;
}
=================================================================================
========================================
AETN Digital Media | Confidential and Proprietary
WorkflowinAction
if ((beresp.status == 200) && req.url ~ ".(aif|aiff|au|avi|bin|bmp|cab|carb|cct|
cdf|class|css|doc|dcr|dtd|eot|exe|flv|gcf|gff|gif|grv|html|hqx|ico|ini|jpeg|jpg|
js|json|mov|mp3|nc|otf|pct|png|ppc|pws|svg|swa|swf|ttf|txt|vbs|w32|wav|wbmp|wml|
wmlc|wmls|wmlsc|xsd|zip|webp|jxr|hdp|wdp)($|?)" ) {
set beresp.ttl = 36h;
}
• Modify TTL in any editor
• Dev:
• GIT Commit and Push to Feature Branch
$ git commit -m "Increased the TTL" -a
[sd-increase-ttl f5d4bff] Increased the TTL
1 file changed, 1 insertion(+), 1 deletion(-)
$ git push origin sd-increase-ttl
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 716 bytes | 0 bytes/s, done.
Total 6 (delta 2), reused 0 (delta 0)
To https://USERNAME:TOKEN_OR_PASSWORD@github.com/aenetworks/REPO_NAME.git
* [new branch] sd-increase-ttl -> sd-increase-ttl
wrap
Today at 4:16 PM • 2KB Plain Text snippet • Private snippet shared with you
shift+enter to add a new line
• QA and Prod
• Follow GIT Flow to Create Pull request and release to appropriate Env
AETN Digital Media | Confidential and Proprietary
Thanks!

VCL template abstraction model and automated deployments to Fastly

  • 1.
    AETN Digital Media| Confidential and Proprietary TEMPLATE ABSTRACTION AND AUTOMATED DEPLOYMENTS TO FASTLY A Practical Walkthrough on Real Life Experiences
  • 2.
    AETN Digital Media| Confidential and Proprietary Then ☹ Manual UI Based Implementation
  • 3.
    AETN Digital Media| Confidential and Proprietary 30m-1hr x Envs Manual Process Back Then • Caching • Shielding • Geographical • Device Detection • Backends • Redirects • Proxies • Authorization www.aetv.com • Caching • Shielding • Geographical • Device Detection • Backends • Redirects • Proxies • Authorization www.mylifetime.com • Caching • Shielding • Geographical • Device Detection • Backends • Redirects • Proxies • Authorization www.fyi.tv • Caching • Shielding • Geographical • Device Detection • Backends • Redirects • Proxies • Authorization www.history.com
  • 4.
    AETN Digital Media| Confidential and Proprietary And then we have more sites! 30m-1hr x Env 30m-1hr x Env 30m-1hr x Env
  • 5.
    AETN Digital Media| Confidential and Proprietary Even more URLS added 30m-1hr x Env 30m-1hr x Env 30m-1hr x Env 30m-1hr x Env 30m-1hr x Env 30m-1hr x Env
  • 6.
    AETN Digital Media| Confidential and Proprietary Digging into Legacy Code!
  • 7.
    AETN Digital Media| Confidential and Proprietary Manual Process Back Then!
  • 8.
    AETN Digital Media| Confidential and Proprietary Update (100) URLS 3 environments Weeks of work= Manual Process Back Then!
  • 9.
    AETN Digital Media| Confidential and Proprietary DevOps Principals • Collaboration • Test everything • Treat Infrastructure as code • Breakdown the barriers • Automate everything • Measure & monitor everything • Support business and IT agility • Work as one team end to end
  • 10.
    AETN Digital Media| Confidential and Proprietary And Now ☺ Continuous Integration Fastly VCL Templates APIs Automated Testing EQUALS Awesome!
  • 11.
    AETN Digital Media| Confidential and Proprietary Continuous Integration Selenium Automated Process CI Tool
  • 12.
    AETN Digital Media| Confidential and Proprietary GLOBALFASTLY BOILER PLATE Sub vcl_recv Sub vcl_fetch Sub vcl_hit Sub vcl_miss Sub vcl_deliver Sub vcl_error Sub vcl_pass SERVICES Application URLS Main.vcl Backends.vcl custom.vcl default.vcl Fastly Boiler Plate Sub vcl_recv call aetn_recv_shield call aetn_recv_geo call aetn_recv_cache aetn.vcl Template Abstraction call aetn_recv_device sub aetn_recv_shield sub aetn_recv_geo sub aetn_recv_cache sub aetn_recv_device
  • 13.
    AETN Digital Media| Confidential and Proprietary Environment Agnostic Implementation • Separate Service for each Env: {env}-abc.com • Abstract Env Specific Backends in separate VCLs: {env}-backends.vcl Backend dev_abc_com { .first_byte_timeout = 3s; .connect_timeout = 1s; .dynamic = true; .max_connections = 200; .between_bytes_timeout = 3s; .share_key = ”COMMONBACKEND"; .port = "443"; .host = ”dev.abc.com"; .probe = { .url = "/aetn-heartbeat.html"; .interval = 30s; .timeout = 3s; .window = 5; .threshold = 2; } } • Use GITFlow guidelines • CI Tool reads the config file to deploy to correct env "environments" : { "dev" : { "ref" : "refs/heads", "servers" : [ { "id" : ”ABC567defgzThQ" } ] },
  • 14.
    AETN Digital Media| Confidential and Proprietary CIDataFlowforManagingEnvAgnostic CI ToolGitHUB Config File (Env to Service Mapping) Process qa-backend.vcl Process dev-backend.vcl Process all common files Process prod-backend.vcl Event Payload Notification to CI Tool Process Payload and identify Event Type Identify ENV and Service ID QA FASTLY Deploy Files and Activate DEV PROD Commit VCL
  • 15.
    AETN Digital Media| Confidential and Proprietary DeploymentWorkFlowUsingFastlyAPI List of Files Modified Service/ Common Retrieve the list of Services Parse the list of services Retrieve the service and common files Fastly Dev Version (/service/search?name) Clone New Version (/service/service_id/version/number/clone) Deploy Files (/service/service_id/version/version/vcl/{main}) Activate (/service/service_id/version/number/activate) Create Dev Version (/service/service_id/version/number/clone) GITHUB Deployment Object Common Service No Yes
  • 16.
    AETN Digital Media| Confidential and Proprietary Update (100) URLS 3 environments Weeks of work= Automated Process Now! From To Update (100) URLS 3 environments a few Minutes = To
  • 17.
    AETN Digital Media| Confidential and Proprietary Workflow in Action Update TTL for all sites
  • 18.
    AETN Digital Media| Confidential and Proprietary • Create a feature branch • GIT Clone Repo $ git clone https://USERNAME:TOKEN_OR_PASSWORD@github.com/aenetworks/ REPO_NAME.git fastly Cloning into 'fastly'... remote: Counting objects: 11441, done. remote: Compressing objects: 100% (127/127), done. remote: Total 11441 (delta 71), reused 0 (delta 0), pack-reused 11313 Receiving objects: 100% (11441/11441), 1.67 MiB | 1.17 MiB/s, done. Resolving deltas: 100% (7939/7939), done. Checking connectivity... done. ================================================================================= ==================================== WorkflowinAction $ git checkout -b sd-increase-ttl Switched to a new branch 'sd-increase-ttl' ================================================================================= ===================================== if ((beresp.status == 200) && req.url ~ ".(aif|aiff|au|avi|bin|bmp|cab|carb|cct| cdf|class|css|doc|dcr|dtd|eot|exe|flv|gcf|gff|gif|grv|html|hqx|ico|ini|jpeg|jpg| js|json|mov|mp3|nc|otf|pct|png|ppc|pws|svg|swa|swf|ttf|txt|vbs|w32|wav|wbmp|wml| wmlc|wmls|wmlsc|xsd|zip|webp|jxr|hdp|wdp)($|?)" ) { set beresp.ttl = 36h; } ================================================================================= ========================================
  • 19.
    AETN Digital Media| Confidential and Proprietary WorkflowinAction if ((beresp.status == 200) && req.url ~ ".(aif|aiff|au|avi|bin|bmp|cab|carb|cct| cdf|class|css|doc|dcr|dtd|eot|exe|flv|gcf|gff|gif|grv|html|hqx|ico|ini|jpeg|jpg| js|json|mov|mp3|nc|otf|pct|png|ppc|pws|svg|swa|swf|ttf|txt|vbs|w32|wav|wbmp|wml| wmlc|wmls|wmlsc|xsd|zip|webp|jxr|hdp|wdp)($|?)" ) { set beresp.ttl = 36h; } • Modify TTL in any editor • Dev: • GIT Commit and Push to Feature Branch $ git commit -m "Increased the TTL" -a [sd-increase-ttl f5d4bff] Increased the TTL 1 file changed, 1 insertion(+), 1 deletion(-) $ git push origin sd-increase-ttl Counting objects: 6, done. Delta compression using up to 4 threads. Compressing objects: 100% (6/6), done. Writing objects: 100% (6/6), 716 bytes | 0 bytes/s, done. Total 6 (delta 2), reused 0 (delta 0) To https://USERNAME:TOKEN_OR_PASSWORD@github.com/aenetworks/REPO_NAME.git * [new branch] sd-increase-ttl -> sd-increase-ttl wrap Today at 4:16 PM • 2KB Plain Text snippet • Private snippet shared with you shift+enter to add a new line • QA and Prod • Follow GIT Flow to Create Pull request and release to appropriate Env
  • 20.
    AETN Digital Media| Confidential and Proprietary Thanks!