SlideShare a Scribd company logo
1 of 89
Modern
Architecture
The
Worker
Pattern
Modern
Architecture
!"#$%&"'()'
!"#$%&"'()'%*+"+$,$-+%.-/$'%+&$%0'12"+$%!$3.'1+1$*%41+15"+1)-%6$#)',%73+%)#%899:;
%
<&1*%='$*$-+"+1)-%,">%3)-+"1-%#)'?"'/@A))B1-5%*+"+$,$-+*%+&"+%1-2)A2$%'1*B*C%.-3$'+"1-+1$*C%"-/%"**.,=+1)-*D%E#%"->%*.3&%
.-3$'+"1-+1$*%,"+$'1"A1F$%)'%1#%"->%)#%+&$%"**.,=+1)-*%=')2$*%1-3)''$3+C%+&$%'$*.A+*%)#%*"A$*#)'3$D3),C%1-3D%3).A/%/1##$'%
,"+$'1"AA>%#'),%+&$%'$*.A+*%$G='$**$/%)'%1,=A1$/%(>%+&$%#)'?"'/@A))B1-5%*+"+$,$-+*%?$%,"B$D%7AA%*+"+$,$-+*%)+&$'%+&"-%
*+"+$,$-+*%)#%&1*+)'13"A%#"3+%3).A/%($%/$$,$/%#)'?"'/@A))B1-5C%1-3A./1-5%"->%=')H$3+1)-*%)#%=')/.3+%)'%*$'213$%"2"1A"(1A1+>C%
*.(*3'1($'%5')?+&C%$"'-1-5*C%'$2$-.$*C%)'%)+&$'%#1-"-31"A%1+$,*%"-/%"->%*+"+$,$-+*%'$5"'/1-5%*+'"+$51$*%)'%=A"-*%)#%
,"-"5$,$-+%#)'%#.+.'$%)=$'"+1)-*C%*+"+$,$-+*%)#%($A1$#C%"->%*+"+$,$-+*%3)-3$'-1-5%-$?C%=A"--$/C%)'%.=5'"/$/%*$'213$*%
)'%+$3&-)A)5>%/$2$A)=,$-+*%"-/%3.*+),$'%3)-+'"3+*%)'%.*$%)#%).'%*$'213$*D
%
<&$%'1*B*%"-/%.-3$'+"1-+1$*%'$#$''$/%+)%"()2$%1-3A./$%I%(.+%"'$%-)+%A1,1+$/%+)%I%'1*B*%"**)31"+$/%?1+&%/$2$A)=1-5%"-/%
/$A12$'1-5%-$?%#.-3+1)-"A1+>%#)'%).'%*$'213$C%-$?%=')/.3+*%"-/%*$'213$*C%).'%-$?%(.*1-$**%,)/$AC%).'%="*+%)=$'"+1-5%
A)**$*C%=)**1(A$%#A.3+."+1)-*%1-%).'%)=$'"+1-5%'$*.A+*%"-/%'"+$%)#%5')?+&C%1-+$''.=+1)-*%)'%/$A">*%1-%).'%J$(%&)*+1-5C%
('$"3&%)#%).'%*$3.'1+>%,$"*.'$*C%+&$%).+3),$%)#%1-+$AA$3+."A%=')=$'+>%"-/%)+&$'%A1+15"+1)-C%'1*B*%"**)31"+$/%?1+&%=)**1(A$%
,$'5$'*%"-/%"3K.1*1+1)-*C%+&$%1,,"+.'$%,"'B$+%1-%?&13&%?$%)=$'"+$C%).'%'$A"+12$A>%A1,1+$/%)=$'"+1-5%&1*+)'>C%).'%"(1A1+>%
+)%$G="-/C%'$+"1-C%"-/%,)+12"+$%).'%$,=A)>$$*%"-/%,"-"5$%).'%5')?+&C%-$?%'$A$"*$*%)#%).'%*$'213$%"-/%*.33$**#.A%
3.*+),$'%/$=A)>,$-+C%).'%A1,1+$/%&1*+)'>%'$*$AA1-5%-)-@*"A$*#)'3$D3),%=')/.3+*C%"-/%.+1A1F"+1)-%"-/%*$AA1-5%+)%A"'5$'%
$-+$'='1*$%3.*+),$'*D%L.'+&$'%1-#)',"+1)-%)-%=)+$-+1"A%#"3+)'*%+&"+%3).A/%"##$3+%+&$%#1-"-31"A%'$*.A+*%)#%*"A$*#)'3$D3),C%1-3D%
1*%1-3A./$/%1-%).'%"--."A%'$=)'+%)-%L)',%8M@N%#)'%+&$%,)*+%'$3$-+%#1*3"A%K."'+$'%$-/$/%7='1A%OMC%PM88D%<&1*%/)3.,$-+*%
"-/%)+&$'*%3)-+"1-1-5%1,=)'+"-+%/1*3A)*.'$*%"'$%"2"1A"(A$%)-%+&$%!QR%L1A1-5*%*$3+1)-%)#%+&$%E-2$*+)'%E-#)',"+1)-%*$3+1)-%
)#%).'%J$(%*1+$D
%
7->%.-'$A$"*$/%*$'213$*%)'%#$"+.'$*%'$#$'$-3$/%1-%+&1*%)'%)+&$'%='$*$-+"+1)-*C%='$**%'$A$"*$*%)'%=.(A13%*+"+$,$-+*%"'$%
-)+%3.''$-+A>%"2"1A"(A$%"-/%,">%-)+%($%/$A12$'$/%)-%+1,$%)'%"+%"AAD%R.*+),$'*%?&)%=.'3&"*$%).'%*$'213$*%*&).A/%,"B$%
+&$%=.'3&"*$%/$31*1)-*%("*$/%.=)-%#$"+.'$*%+&"+%"'$%3.''$-+A>%"2"1A"(A$D%!"A$*#)'3$D3),C%1-3D%"**.,$*%-)%)(A15"+1)-%"-/%
/)$*%-)+%1-+$-/%+)%.=/"+$%+&$*$%#)'?"'/@A))B1-5%*+"+$,$-+*D
Ryan
Smith
Ryan
Smith
@ryandotsmith
ryandotsmith.heroku.com
The
Vault
1
Designed
for
the
Cloud
Goals
Goals


{
engineering
concepts
}
Goals


 {
engineering
concepts
}
concept
=>
implementation
UNIX
“Write
programs
that
do
one
thing
and
do
it
well.”
                                 ‐‐
Doug
McIlroy
Background
Apps
small
>
large
Informal
agreement
on
pre/post
conditions.
Dependencies
Common

  HTTP
  JSON
Queueing
Scheduling
Formal
Declaration
Easy
install
for
new
developers.
Consistency
in
Production.
Java     Maven

 Ruby     Bundler

Python      Pip

Node.js    NPM

Clojure    Lein
Environment
Production
  Staging
Development
Creating
apps
should
be
trivial.
Caveat
Caveat

Staging
Credit
Card
gateway
Caveat

Staging
Credit
Card
gateway
     $VERBOSE=‘true’
Application
Code

if ENV[‘VERBOSE’] == ‘true’
  log(‘message’)
end
Application
Code

if ENV[‘VERBOSE’] == ‘true’
  log(‘message’)
end
    Production
Environment

$ export VERBOSE=‘false’
Application
Code

if ENV[‘VERBOSE’] == ‘true’
  log(‘message’)
end
    Production
Environment

$ export VERBOSE=‘false’

  Development
Environment

$ export VERBOSE=‘true’
cat .env | xargs env
Services
Database
Database
 Cache
Database
  Cache
Message
Bus
Database
  Cache
Message
Bus
 Log
Drain
Database
     Cache
  Message
Bus
    Log
Drain
Heartbeat
Monitor
Everything
is
a
service.
New
Features
:)
    DBA
:(
CREATE
INDEX
PRO
TIP

CREATE INDEX CONCURRENTLY
Provision
a
new
DB
in
<
60s
Configured
in
environment.
database = DB.connect(
  ENV[“DATABASE_URL”]
);
App        App




      DB
App        App




      DB
Deploy
Developer
Deploys
Code
I
want




























for
my
deploy.
Incremental
Rollout
SELECT ... WHERE MOD(id, 4) = 0;
Rollback
Continuous
Deployment
Visibility
Science
Hypothesis
Hypothesis
Conclude
Implications
Hypothesis
Conclude
Implications
  Observe
Reality*
Logging
Think
of
your
logs
as
streams
of
data.
App      App        App




      Log Service




  Log Drain API
papertrail    Sinatra      Heroku API




heroku restart -a app_name -ps worker.5
App      App        App




      Log Service




  Log Drain API
log
=
event
stream
Administration
If
you
do
it
twice,
automate
it.
Conclusion
Small
Apps
Small
Apps
Dependencies
Small
Apps
 Dependencies
Environment
Vars
Small
Apps
 Dependencies
Environment
Vars
    Services
Small
Apps
 Dependencies
Environment
Vars
    Services
  Deployment
Small
Apps
 Dependencies
Environment
Vars
    Services
  Deployment
    Visibility
Focus
on
your
application
not
your
platform
finnish

More Related Content

What's hot

You're Doing It Wrong
You're Doing It WrongYou're Doing It Wrong
You're Doing It Wrongbostonrb
 
Using Actions and Filters in WordPress to Make a Plugin Your Own
Using Actions and Filters in WordPress to Make a Plugin Your OwnUsing Actions and Filters in WordPress to Make a Plugin Your Own
Using Actions and Filters in WordPress to Make a Plugin Your OwnBrian Hogg
 
Software Project Management
Software Project ManagementSoftware Project Management
Software Project ManagementWidoyo PH
 
Android L02 - Activities and Adapters
Android L02 - Activities and AdaptersAndroid L02 - Activities and Adapters
Android L02 - Activities and AdaptersMohammad Shaker
 
Gigigo Rails Workshop
Gigigo Rails WorkshopGigigo Rails Workshop
Gigigo Rails WorkshopAlex Rupérez
 
Sql FUNCTIONS
Sql FUNCTIONSSql FUNCTIONS
Sql FUNCTIONSAbrar ali
 
From Big to Massive – Scalability in AngularJS Applications
From Big to Massive – Scalability in AngularJS ApplicationsFrom Big to Massive – Scalability in AngularJS Applications
From Big to Massive – Scalability in AngularJS ApplicationsSebastian Fröstl
 
Entry-level PHP for WordPress
Entry-level PHP for WordPressEntry-level PHP for WordPress
Entry-level PHP for WordPresssprclldr
 
Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...
Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...
Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...Pilot
 
Activity streams lightning talk, DjangoCon 2011 Day 3
Activity streams lightning talk, DjangoCon 2011 Day 3Activity streams lightning talk, DjangoCon 2011 Day 3
Activity streams lightning talk, DjangoCon 2011 Day 3Steve Ivy
 
My first WordPress Plugin
My first WordPress PluginMy first WordPress Plugin
My first WordPress PluginAbbas Siddiqi
 

What's hot (13)

You're Doing It Wrong
You're Doing It WrongYou're Doing It Wrong
You're Doing It Wrong
 
Well
WellWell
Well
 
Using Actions and Filters in WordPress to Make a Plugin Your Own
Using Actions and Filters in WordPress to Make a Plugin Your OwnUsing Actions and Filters in WordPress to Make a Plugin Your Own
Using Actions and Filters in WordPress to Make a Plugin Your Own
 
Software Project Management
Software Project ManagementSoftware Project Management
Software Project Management
 
Android L02 - Activities and Adapters
Android L02 - Activities and AdaptersAndroid L02 - Activities and Adapters
Android L02 - Activities and Adapters
 
Routes
RoutesRoutes
Routes
 
Gigigo Rails Workshop
Gigigo Rails WorkshopGigigo Rails Workshop
Gigigo Rails Workshop
 
Sql FUNCTIONS
Sql FUNCTIONSSql FUNCTIONS
Sql FUNCTIONS
 
From Big to Massive – Scalability in AngularJS Applications
From Big to Massive – Scalability in AngularJS ApplicationsFrom Big to Massive – Scalability in AngularJS Applications
From Big to Massive – Scalability in AngularJS Applications
 
Entry-level PHP for WordPress
Entry-level PHP for WordPressEntry-level PHP for WordPress
Entry-level PHP for WordPress
 
Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...
Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...
Pilot Tech Talk #9 — Ember.js: Productivity without the fatigue by Jacek Gala...
 
Activity streams lightning talk, DjangoCon 2011 Day 3
Activity streams lightning talk, DjangoCon 2011 Day 3Activity streams lightning talk, DjangoCon 2011 Day 3
Activity streams lightning talk, DjangoCon 2011 Day 3
 
My first WordPress Plugin
My first WordPress PluginMy first WordPress Plugin
My first WordPress Plugin
 

Viewers also liked

Modern Architecture
Modern ArchitectureModern Architecture
Modern ArchitectureTom Clowers
 
HISTORY: Understanding Deconstructivism/ Critical Regionalism/ Phenomenology
HISTORY: Understanding Deconstructivism/ Critical Regionalism/ PhenomenologyHISTORY: Understanding Deconstructivism/ Critical Regionalism/ Phenomenology
HISTORY: Understanding Deconstructivism/ Critical Regionalism/ PhenomenologyArchiEducPH
 
ARKWIZ: Qualifiers Competition Briefing
ARKWIZ: Qualifiers Competition BriefingARKWIZ: Qualifiers Competition Briefing
ARKWIZ: Qualifiers Competition BriefingArchiEducPH
 
HISTORY: Romanesque Architecture
HISTORY: Romanesque ArchitectureHISTORY: Romanesque Architecture
HISTORY: Romanesque ArchitectureArchiEducPH
 
HISTORY: Early Christian Architecture
HISTORY: Early Christian ArchitectureHISTORY: Early Christian Architecture
HISTORY: Early Christian ArchitectureArchiEducPH
 
Emergence of modern architecture ppt
Emergence of modern architecture  pptEmergence of modern architecture  ppt
Emergence of modern architecture pptaziz khan
 

Viewers also liked (7)

Modern Architecture
Modern ArchitectureModern Architecture
Modern Architecture
 
Modern Architecture
Modern ArchitectureModern Architecture
Modern Architecture
 
HISTORY: Understanding Deconstructivism/ Critical Regionalism/ Phenomenology
HISTORY: Understanding Deconstructivism/ Critical Regionalism/ PhenomenologyHISTORY: Understanding Deconstructivism/ Critical Regionalism/ Phenomenology
HISTORY: Understanding Deconstructivism/ Critical Regionalism/ Phenomenology
 
ARKWIZ: Qualifiers Competition Briefing
ARKWIZ: Qualifiers Competition BriefingARKWIZ: Qualifiers Competition Briefing
ARKWIZ: Qualifiers Competition Briefing
 
HISTORY: Romanesque Architecture
HISTORY: Romanesque ArchitectureHISTORY: Romanesque Architecture
HISTORY: Romanesque Architecture
 
HISTORY: Early Christian Architecture
HISTORY: Early Christian ArchitectureHISTORY: Early Christian Architecture
HISTORY: Early Christian Architecture
 
Emergence of modern architecture ppt
Emergence of modern architecture  pptEmergence of modern architecture  ppt
Emergence of modern architecture ppt
 

Similar to Modern Architecture

Separating enterprise social apps from platforms
Separating enterprise social apps from platformsSeparating enterprise social apps from platforms
Separating enterprise social apps from platformsLee Bryant
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable CodeBaidu, Inc.
 
Redux. From twitter hype to production
Redux. From twitter hype to productionRedux. From twitter hype to production
Redux. From twitter hype to productionFDConf
 
Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013
Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013
Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013Amazon Web Services
 
Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesLindsay Holmwood
 
Hidden Docs in Angular
Hidden Docs in AngularHidden Docs in Angular
Hidden Docs in AngularYadong Xie
 
Танки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_ЯндексеТанки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_ЯндексеYandex
 
SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)Oswald Campesato
 
Redux. From twitter hype to production
Redux. From twitter hype to productionRedux. From twitter hype to production
Redux. From twitter hype to productionJenya Terpil
 
Implementation of GUI Framework part3
Implementation of GUI Framework part3Implementation of GUI Framework part3
Implementation of GUI Framework part3masahiroookubo
 
Eventsourcing with PHP and MongoDB
Eventsourcing with PHP and MongoDBEventsourcing with PHP and MongoDB
Eventsourcing with PHP and MongoDBJacopo Nardiello
 
Serverless in production, an experience report (London DevOps)
Serverless in production, an experience report (London DevOps)Serverless in production, an experience report (London DevOps)
Serverless in production, an experience report (London DevOps)Yan Cui
 
GHC Participant Training
GHC Participant TrainingGHC Participant Training
GHC Participant TrainingAidIQ
 
Working with Javascript in Rails
Working with Javascript in RailsWorking with Javascript in Rails
Working with Javascript in RailsSeungkyun Nam
 
QConSP 2015 - Dicas de Performance para Aplicações Web
QConSP 2015 - Dicas de Performance para Aplicações WebQConSP 2015 - Dicas de Performance para Aplicações Web
QConSP 2015 - Dicas de Performance para Aplicações WebFabio Akita
 

Similar to Modern Architecture (20)

Separating enterprise social apps from platforms
Separating enterprise social apps from platformsSeparating enterprise social apps from platforms
Separating enterprise social apps from platforms
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
 
Redux. From twitter hype to production
Redux. From twitter hype to productionRedux. From twitter hype to production
Redux. From twitter hype to production
 
Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013
Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013
Zero to Sixty: AWS Elastic Beanstalk (DMG204) | AWS re:Invent 2013
 
Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websites
 
Cpp tutorial
Cpp tutorialCpp tutorial
Cpp tutorial
 
Hidden Docs in Angular
Hidden Docs in AngularHidden Docs in Angular
Hidden Docs in Angular
 
Танки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_ЯндексеТанки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
 
Svcc 2013-d3
Svcc 2013-d3Svcc 2013-d3
Svcc 2013-d3
 
SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)
 
Redux. From twitter hype to production
Redux. From twitter hype to productionRedux. From twitter hype to production
Redux. From twitter hype to production
 
Implementation of GUI Framework part3
Implementation of GUI Framework part3Implementation of GUI Framework part3
Implementation of GUI Framework part3
 
Eventsourcing with PHP and MongoDB
Eventsourcing with PHP and MongoDBEventsourcing with PHP and MongoDB
Eventsourcing with PHP and MongoDB
 
Serverless in production, an experience report (London DevOps)
Serverless in production, an experience report (London DevOps)Serverless in production, an experience report (London DevOps)
Serverless in production, an experience report (London DevOps)
 
Android L01 - Warm Up
Android L01 - Warm UpAndroid L01 - Warm Up
Android L01 - Warm Up
 
实战Ecos
实战Ecos实战Ecos
实战Ecos
 
GHC Participant Training
GHC Participant TrainingGHC Participant Training
GHC Participant Training
 
CppTutorial.ppt
CppTutorial.pptCppTutorial.ppt
CppTutorial.ppt
 
Working with Javascript in Rails
Working with Javascript in RailsWorking with Javascript in Rails
Working with Javascript in Rails
 
QConSP 2015 - Dicas de Performance para Aplicações Web
QConSP 2015 - Dicas de Performance para Aplicações WebQConSP 2015 - Dicas de Performance para Aplicações Web
QConSP 2015 - Dicas de Performance para Aplicações Web
 

Recently uploaded

[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdfSandro Moreira
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelDeepika Singh
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 

Recently uploaded (20)

[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 

Modern Architecture

Editor's Notes

  1. Today I am talking about building apps for the cloud. \n
  2. But wait, weren&amp;#x2019;t you supposed to talk about the worker pattern? Yes. But I am not going to do that now.\n
  3. To that end, I am talking about building apps for the cloud. Some of the points I will make should be obvious in hindsight. However, some points I will make have not yet proven to be true so look out that!\n
  4. \n
  5. My name is Ryan Smith, you can find me on twitter via @ryandotsmith.\n
  6. \n
  7. I am an engineer at Heroku.\n
  8. I work in The Vault. The Vault is a team within Heroku that is responsible for innovating the usage &amp; billing components of our business. We track all of the events in our cloud an figure out how to make an invoice out of the events. We do all of this in realtime and with large amounts of data.\n
  9. One interesting note. All of The Vault&amp;#x2019;s components are hosted on Heorku. We have been doing all sorts of strange things related to self hosting these days. Personally, I love the idea. Isn&amp;#x2019;t self hosting a mind bend? I mean the wrote the C compiler in C.\n
  10. You would think that such an operation requires a large engineering team. However, we get by with a team of 1.\n
  11. Our team&amp;#x2019;s size would not be feasible if we maintained our own platform. Knowing this, The Vault&amp;#x2019;s apps were optimized for a cloud based deployment and therein lies the core ideas of this case study. I want to talk about the engineering practices that allowed us a cloud deployment.\n
  12. My goal for this talk is to present a set of engineering concepts that we have maintained in The Vault that have directly led to successful development. I will introduce the concept then follow it up by providing example of how we implement the concept. Lets get started with the first concept.\n
  13. My goal for this talk is to present a set of engineering concepts that we have maintained in The Vault that have directly led to successful development. I will introduce the concept then follow it up by providing example of how we implement the concept. Lets get started with the first concept.\n
  14. There is a lot to learn from the UNIX philosophy. \n
  15. One of the things we picked early on was the need to write many small apps instead of one large app. We have apps for all kinds of things.\n
  16. Web apps for credit card forms.\n
  17. Web apps for invoice archiving and current usage\n
  18. Background apps with no Web UI. Apps that process gigabytes of data.\n
  19. There was a time when all of our code was in 1 large monolithic app and it was quite painful.\nWhen I go to write a feature for an app I not only have to load the code into my editor, I also have to load it into my brain. At that point, I become the bottleneck.\n\n
  20. To that end, I started breaking down the apps into simpler components.\n\n
  21. you can also horizontally scale your app independently of other components\n
  22. I would be remiss if I didn&amp;#x2019;t inform you of the pitfalls in deploying many small apps in your system. \nThere will be problems in maintaining consistency across dependent APIs. However, The Vault has had great success in overcoming this challenge by setting up inter-team contracts. \n\nIf 1 or more components within our business are dependent upon our API then we draft an informal document describing what we promise to uphold. We typically keep this document in the doc directory of the applications code base. \n
  23. So the first concept was small apps and the next concept is dependency management.\n
  24. Most of our apps share a set of dependencies. Although some may vary by version, they are generally solutions to common problems. We use libraries that help us make HTTP requests, pase JSON, publish and consume on message queues\n
  25. We keep a machine readable list of dependencies in each one of our apps.\n
  26. This allows new developers a wonderful way to bootstrap the app.\n
  27. This also ensures that our production platform knows the exact version of dependency to install. \n
  28. Popular languages support this. It is good. I can&amp;#x2019;t imagine not using one of these tools with my apps.\n
  29. Even if your project has 0 dependencies, still include a Gemfie. At the end of the day, you are communicating to other developers what a project needs. If you project needs nothing, communicate that.\n
  30. \n
  31. For each app, we have both a staging and production deployment. The two deployments are identical. They are on the same platform, the same OS, they have the same codebase, the same dependencies... IDENTICAL.\n\nWe try and get our development environment identical as well. We develop BSD and Linux and execute the same commands that our server executes.\n
  32. It would be too much work if I had to create servers for each environment. Heroku allows me to quickly create apps.\n\nI don&amp;#x2019;t have time to provision hardware and setup my runtime.\n
  33. Of course we need to tweak a few settings between environments.\n\nWe use the sandbox API provided by our credit card gateway.\n\nWe use staging for experiments. On heroku it is very easy to load our production dataset into a staging database. So we take the opportunity to test the performance of new algorithms, tune SQL queries, etc....\n\nAnd in development we turn on all of the logging features in our libraries.\n\nThe point is that all of tweaks are defined in the environment via shell variables.\n
  34. Of course we need to tweak a few settings between environments.\n\nWe use the sandbox API provided by our credit card gateway.\n\nWe use staging for experiments. On heroku it is very easy to load our production dataset into a staging database. So we take the opportunity to test the performance of new algorithms, tune SQL queries, etc....\n\nAnd in development we turn on all of the logging features in our libraries.\n\nThe point is that all of tweaks are defined in the environment via shell variables.\n
  35. Our code base remains constant while our environment determines how the system should react.\n
  36. Our code base remains constant while our environment determines how the system should react.\n
  37. I keep a file in my project directory and name it .env\nThis is how I communicate what environment setting my app requires\n
  38. I keep a file in my project directory and name it .env\nThis is how I communicate what environment setting my app requires.\nIf you use foreman, this file will be loaded into your shell environment upon it&amp;#x2019;s startup.\n
  39. Everything is a service.\n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. Everything is a service. I don&amp;#x2019;t have time to configure and maintain all of the systems. I am a team of 1.\n
  47. The Vault, relies on Postgres. This service is provided by Heroku&amp;#x2019;s data team. This is definitely a feature we could not do without. I can&amp;#x2019;t imagine developing new features AND maintaining a database.\n
  48. The only thing remotely close to DBA work that I do is: CREATE INDEX.\n
  49. typically when you create an index it will lock your table. This is bad. Postgres will allow you to create an index without locking the table. Super useful for debugging production performance problems.\n
  50. Just like apps, cloud services can be brought online in seconds, not days.\n
  51. When we provision a database, our provider gives us a URI which we set as a shell variable in our environment. There are no configuration files to track, just set the proper variable in the environment.\n
  52. In my code, I reference a URL that is set in my environment.\n
  53. Because my database is just a URL, I can easily share this database among other apps in the cloud.\n\nThis makes me happy.\n
  54. \n
  55. As a team of 1, I develop the feature, test the system and roll it out to my customers. I have great insight into all of the moving parts of my system.\n
  56. This allows me to quickly ship features into production. When I fix a bug or ship a new UI improvement. I want this in production right away. I don&amp;#x2019;t want to be blocked by anything! I want prime for my deployment.\n
  57. Even though I have never shipped a bug into production, I like to know that I have a safety net. Incremental Rollout --one of the most valuable practices I have incorporated into The Vault&amp;#x2019;s repertoire. \n\n\n
  58. I can choose to enable a new feature for a portion of our customers. I deploy the change, watch my logs and if there is a problem, I quickly rollback to the last known commit.\n
  59. When I encounter an unusual amount of exceptions in my app, I rely on the ability to rollback.\n
  60. Developers deploying and incremental rollout describe a more general concept. And that is: Continuous deployment. As we fix bugs, develop features, we deploy the code. Since it takes &lt; 30 seconds to release code to production, it makes a lot of sense for us to push changes at will.\n\nThe days are long over when we deploy hundreds of commits.\n
  61. Visibility is quite important to a team of 1. I don&amp;#x2019;t have time to manually monitor all of my systems. So I rely on cloud technologies to alert me when interesting things happen.\n
  62. On a side note. I have been on a science kick lately. When troubleshooting complex routines or reasoning about the performance of my system, I start by employing the scientific method. \n\n\n
  63. The 3rd step is what I need help with. I need an easy way to make observations.\n
  64. The 3rd step is what I need help with. I need an easy way to make observations.\n
  65. The 3rd step is what I need help with. I need an easy way to make observations.\n
  66. Logging is what helps us observe reality\n
  67. I am not talking about log files. Forget about log files. Start treating your logs as first-class data. Data that streams from all of our apps.\n
  68. We point these streams at cloud logging services. Products like papertrailapp give you a web ui and APIs to do all sorts of analysis on your log data.\n
  69. 1 cool feature that we could not live without is log alerts. This service is pioneered by papertrailapp.com We setup search queries for our log data. Whenever a log message comes in that matches that criteria, papertrail will send a message into our group chat service.\n\nFor instance, I log errors when my worker apps start to slow down. When this happens, I get notified immediately in my team&amp;#x2019;s chat room.\n
  70. Not only can i receive campfire messages about this worker problem. I can also write a service that repairs my system for me.\n
  71. If we trust our logs to warn us, why not trust them with other tasks.\n
  72. What if we used the connivence of logging as a transport for data back to our applications.\n
  73. We started treating our logs as first-class data. Data that streams from all of our apps.\n
  74. Event streams are the future. This area is ripe for innovation and standardization.\n
  75. So after we realize that an app is misbehaving, we need some way of easily restarting the process. Jokingly, we call this activity: kicking the vault.\n
  76. \n
  77. \n
  78. There are a million different ways to skin this cat. The Vault has found an approach that works reasonably well. The approach I have outlined here is optimized for rapid change and feedback --the agile process. It is also optimized for scale. The size of my dataset grows geometrically; the size of my team grows linearly at best. These practices maintain sanity in our chaotic environment.\n \nThe concepts we have discussed will help you do the same.\n
  79. There are a million different ways to skin this cat. The Vault has found an approach that works reasonably well. The approach I have outlined here is optimized for rapid change and feedback --the agile process. It is also optimized for scale. The size of my dataset grows geometrically; the size of my team grows linearly at best. These practices maintain sanity in our chaotic environment.\n \nThe concepts we have discussed will help you do the same.\n
  80. There are a million different ways to skin this cat. The Vault has found an approach that works reasonably well. The approach I have outlined here is optimized for rapid change and feedback --the agile process. It is also optimized for scale. The size of my dataset grows geometrically; the size of my team grows linearly at best. These practices maintain sanity in our chaotic environment.\n \nThe concepts we have discussed will help you do the same.\n
  81. There are a million different ways to skin this cat. The Vault has found an approach that works reasonably well. The approach I have outlined here is optimized for rapid change and feedback --the agile process. It is also optimized for scale. The size of my dataset grows geometrically; the size of my team grows linearly at best. These practices maintain sanity in our chaotic environment.\n \nThe concepts we have discussed will help you do the same.\n
  82. There are a million different ways to skin this cat. The Vault has found an approach that works reasonably well. The approach I have outlined here is optimized for rapid change and feedback --the agile process. It is also optimized for scale. The size of my dataset grows geometrically; the size of my team grows linearly at best. These practices maintain sanity in our chaotic environment.\n \nThe concepts we have discussed will help you do the same.\n
  83. There are a million different ways to skin this cat. The Vault has found an approach that works reasonably well. The approach I have outlined here is optimized for rapid change and feedback --the agile process. It is also optimized for scale. The size of my dataset grows geometrically; the size of my team grows linearly at best. These practices maintain sanity in our chaotic environment.\n \nThe concepts we have discussed will help you do the same.\n
  84. \n
  85. \n