No 11
cavendish
square
11 july 2013
london
Thursday, July 11, 13
developer
VOCAB
Alex brandt
sr developer
Thursday, July 11, 13
MODULES
3
Thursday, July 11, 13
LIBRARIES
4
Thursday, July 11, 13
FRAMEWORKS
•The building blocks of an application - a set of common
software functions that take the work out writing all ...
6
Thursday, July 11, 13
7
Thursday, July 11, 13
8
Thursday, July 11, 13
LANGUAGES
9
Thursday, July 11, 13
10
Thursday, July 11, 13
APIs
11
Thursday, July 11, 13
APIs
12
Thursday, July 11, 13
What is a data
structure?
•images
• url
• stars
• *title
• caption
• variants
•small
•medium
•big
•tags
•*_id
•rolestars
•...
API tier/Application
STORAGE
APPLICATION
PRESENTATION
14
Thursday, July 11, 13
API tier/Application
STORAGE
APPLICATION
PRESENTATION
APPLICATION/API
14
Thursday, July 11, 13
Bindings
15
Thursday, July 11, 13
Bindings
16
Thursday, July 11, 13
Bindings
17
Thursday, July 11, 13
EVERYTHING
IS CODE
wayne walls
cloud evangelist
alex brandt
sr developer
Thursday, July 11, 13
Infrastructure
as code
19
Thursday, July 11, 13
20
Thursday, July 11, 13
21
Thursday, July 11, 13
Chef
Server
Chef
Workstation
Chef Client
knife
node
Chef Client
node
environment
22
Thursday, July 11, 13
•Nodes: Servers
•Environments: Groups of nodes (Dev/QA/Prod)
•Recipes: Describes the desired state of a node
•Cookbooks: G...
24
Thursday, July 11, 13
Puppet
CA
Puppet
Master
Puppet
Agent
https
Puppet
Agent
Puppet
Agent
25
Thursday, July 11, 13
•Nodes: Servers
•Environments: Groups of
nodes (Dev/QA/Prod)
•Manifests: Describes the
desired state of a node
•Catalog: G...
27
Thursday, July 11, 13
Salt
Master
Salt
Minion
Salt
Minion
Salt
Minion
28
Thursday, July 11, 13
•Nodes: Minions
•Environments: Groups of nodes (Dev/QA/Prod)
•States: Describes the desired state of a node
•Grains: infor...
Thinking like a developer
30
Thursday, July 11, 13
scrum Development
Product
Backlog
Sprint
Backlog
Sprint
Working increment
of the software
31
1. Design
5. QA 2. Build
4. R...
scrum Development
32
1. Design
5. QA 2. Build
4. Release 3. Configure
Thursday, July 11, 13
TYPES OF
DEVELOPMENT
33
Thursday, July 11, 13
TYPES OF
DEVELOPMENT
33
Test Driven Development
Thursday, July 11, 13
TYPES OF
DEVELOPMENT
33
Test Driven Development
Behavior Driven Development
Thursday, July 11, 13
TYPES OF
DEVELOPMENT
33
Test Driven Development
Behavior Driven Development Domain Driven Development
Thursday, July 11, 13
TYPES OF
DEVELOPMENT
33
Test Driven Development
Behavior Driven Development
Application Bliss
Domain Driven Development
Th...
Model View Controller (MVC)
UPDATES MANIPULATES
SEES USES
USER
MODEL
VIEW CONTROLLER
34
Thursday, July 11, 13
DEv tools
Thursday, July 11, 13
36
Thursday, July 11, 13
37
Thursday, July 11, 13
Gerrit
38
Thursday, July 11, 13
39
Thursday, July 11, 13
CODE
REVIEW
PROCESS
Thursday, July 11, 13
Our hero, the developer, submits code for review.
The code is reviewed in Gerrit by other developers, who
approve the chan...
POLYGLOT
PERSISTENCE
wayne walls
cloud evangelist
technical track
alex brandt
sr developer
Thursday, July 11, 13
A database, in the most general
sense, is an organized collection of
data.
Database Defined
http://glenn-packer.net
43
Thu...
ACID
•Atomicity: refers to a database’s ability to either fully process or fully roll back a
transaction
•Consistency: dat...
•A relational database is essentially a group of tables
or, to use the technical name, entities.
•Each table is made up of...
Some companies needed something more to accommodate their growing data
rdbms may not be
the best fit
46
Thursday, July 11,...
Enter
Non-Relational database
management systems
47
Thursday, July 11, 13
INTRODUCTION TO
NOSQL
• NoSQL Database (1998)
• NoREL
• “Not only SQL”
• NoSQL (2009)
48
Thursday, July 11, 13
NoSQL is difficult to define but most NoSQL databases will
share common characteristics
1. Non-relational
2. Distributed
3...
An “aggregate” is a
collection of related
objects that we wish
to treat as a unit
orders
customers
order lines
credit card...
There are several different
types of data stores that are
considered to be NoSQL
•Key/Value Databases
•Document Databases
...
Key/Value Database
•Primarily constructed of aggregates that have a Key or ID that’s
used to access the data
•The database...
Key/Value Database
•Good Use Cases:
• Storing Session
Information
• User Profiles &
Preferences
• Shopping Cart Data
• Bad...
•Like a Key-Value database, document databases primarily store
and access data using aggregates
•In contrast to the Key-Va...
document Databases
•Good Use Cases:
• Event Logging
• Blogging Platforms
• Real-time Analytics
• Bad Use Cases:
• Complex ...
• Can be looked at as a two-level aggregate
structure
• The first key is often described as a row ID
picking up the aggreg...
COLUMN-FAMILY
datastores
•Good Use Cases:
• Event Logging
• Blogging Platforms
• Counters (page visits)
• Expiring Usage
•...
•Unique compared to most NOSQL types because it was not
developed with the intention of scaling horizontally into a cluste...
Amanda
KNOWS
KNOWS
KNOWS
Robert
Gerry
Lamar
KNOWS
App 1
BUILT
BUILT
KNOWS
App 2
USES
BUILT
Christie
KNOWS
Destiny
Jessica
...
graph databases
•Good Use Cases:
• Connected Data
• Location-based services
•Recommendation
engines
• Bad Use Cases:
• Abi...
Companies using
nosql
61
DATABASE COMPANIES
Thursday, July 11, 13
THE FUTURE IS POLYGLOT
62
Neo4J
database
Cassandra
database
Couch
database
MongoDB
database
RELATIONAL
DATABASE
Graph-stru...
63
POLYGLOT PATTERN: cache
ORDER
TAKING
RESTAURANT
MANAGEMENT
REDIS
CACHE
MYSQL
DATABASE
SECONDFIRST
CONSUMER
RESTAURANT
O...
64
POLYGLOT PATTERN:
write to sql and NOSQL
NEW
TWEET
INSERT
LPUSHX
LTRIM
MySQL
TWEETS
FOLLOWS
FOLLOWERS
Redis: Timeline
F...
65
POLYGLOT PATTERN:
REPLICATE FROM
MYsql to NOSQL
QUERY
SERVICE
UPDATE
SERVICE
REDIS
MYSQL
DATABASE
SYSTEM
OF
RECORD
MATE...
66
Atomic
Consistent
Isolated
Durable
ACID to base
Basically Available
Soft state
Eventually consistent
Thursday, July 11,...
5 PILLARS OF
CLOUDINESS
wayne walls
cloud evangelist
alex brandt
sr developer
technical track
Thursday, July 11, 13
FIVE PILLARS
OF CLOUDINESS
1 2 3 4 5
Parallel Modular Horizontal Agile Secure
68
Thursday, July 11, 13
Parallel
Thursday, July 11, 13
Parallel - what
1M
Records
Answer
Node
70
Thursday, July 11, 13
Parallel - how
1M
Records
Answer
Node
1-250K
Node
251K-500K
Node
501K-750K
Node
751K-1M
71
Thursday, July 11, 13
Parallel - how
72
1M
Records
Answer
Node
1-250K
Node
251K-500K
Node
501K-750K
Node
751K-1M
Thursday, July 11, 13
Parallel - how
72
1M
Records
Answer
Node
1-250K
Node
251K-500K
Node
501K-750K
Node
751K-1M
Thursday, July 11, 13
Parallel - how
73
Thursday, July 11, 13
Parallel - how
DATA
CALCULATION
MAP
CALCULATION
REDUCE
DATA
CALC
D
C
R
D
D
C
R
D
D
C
R
D
SHUFFLE
DATA
73
Thursday, July 11...
Avoid Deadlock
74
Thursday, July 11, 13
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
X
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
X
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
X X
File B
Thread 2
Thursday, July 11, 13
Solution? Scheduling
Y
XA
B
A
B
Y
X
1
2
3
4
76
Thursday, July 11, 13
Parallel - why
1M
records
Answer
Node
1M
records
Answer
Node
1-250K
Node
251K-500K
Node
501K-750K
Node
750K-1M
77
Thursday...
Parallel - why
1M
records
Answer
Node
1M
records
Answer
Node
1-250K
Node
251K-500K
Node
501K-750K
Node
750K-1M
77
Thursday...
Parallel - why
78
Thursday, July 11, 13
Parallel - why
79
Thursday, July 11, 13
modular
Thursday, July 11, 13
ONE PERSON BUILDS A CAR
Modular - what
Example: Assembly Line
81
Thursday, July 11, 13
CHASSIS ENGINE BODY INTERIOR
Decoupled
Self-contained units
Data Communication
Modular - what
Example: Assembly Line
82
Th...
Modular - what
Modules within Modules
CHASSIS ENGINE BODY INTERIOR
83
Nested Modularity
Thursday, July 11, 13
CHASSIS ENGINE BODY INTERIOR
Modular - what
84
Modules within Modules
Nested Modularity
Thursday, July 11, 13
CHASSIS ENGINE BODY INTERIOR
Modular - what
SEATS FLOOR
DASHBOARD
GAS
AND
BRAKE
PEDALS
84
Modules within Modules
Nested Mo...
Small units are more maintainable
Allows us to tolerate some partitions (but not all)
Allows us to be parallel!
CHASSIS EN...
Direct communication (1:1)
Topic communication (1:N)
Fanout communication (1:ALL)
modular - Why
86
Thursday, July 11, 13
•Decouple requests
•Message Queue
•Make small units of work
•Define tiers by function
modular - how
87
Thursday, July 11, ...
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defect
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defect defect
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defect defect FAIL
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defect defect FAIL
Mandelbugs
Potential for
Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, Ju...
horizontal
Thursday, July 11, 13
2 GB
2 CORES
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
VERTICAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
VERTICAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 11, 13
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB
4 CORES
Thursday, July 1...
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB
4 CORES
Thu...
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 G...
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
Horizontal ...
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
SHARED NOTH...
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH S...
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
2 GB
2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH S...
Horizontal - why
91
Thursday, July 11, 13
Horizontal - why
92
Thursday, July 11, 13
ONE
SERVER
NEAR
CAPACITY
Horizontal - why
92
Thursday, July 11, 13
ONE
SERVER
NEAR
CAPACITY
ADD
ANOTHER
SERVER
Horizontal - why
92
Thursday, July 11, 13
ONE
SERVER
NEAR
CAPACITY
ADD
ANOTHER
SERVER
Horizontal - why
93
Thursday, July 11, 13
1234 MESSAGE QUEUE
94
Horizontal - HOW
Thursday, July 11, 13
1234MESSAGE QUEUE
95
Horizontal - HOW
Thursday, July 11, 13
1
2
3
4
96
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
1
2
3
4
96
56789101112 MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
97
5678 9101112MESSAGE QUEUE
1
2
3
4
Horizontal - HOW
Thursday, July 11, 13
97
5678 9101112
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
MESSAGE QUEUE
1
2
3
4
Horizontal - HOW
Thursday, July 11, 13
MESSAGE QUEUE
98
5
6
7
8
9
10
11
12
13141516
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
1
2
3
4
Horizontal - HOW
Thursday, Ju...
99
5678 9101112
13141516
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
100
5678 9101112
13141516
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
101
5678 9101112
13141516
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
101
5678 9101112
13141516
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Horizontal - HOW
DECREASE SERVERS
BAS...
MESSAGE QUEUE
102
5678 9101112
13
14
15
16
ADD MORE SERVERS
BASED ON
MESSAGE QUEUE
1234
Horizontal - HOW
DECREASE SERVERS
...
agile
Thursday, July 11, 13
Agile - WHAT
the power of moving
quickly and easily;
nimbleness
104
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
105
Agile - why Monday
Thursday, July 1...
Agile - why
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
106
Tuesday
Thursday, July ...
Agile - why
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
106
Tue...
Features In Progress Blocked Tabled Testing Complete
Feature AFeature BFeature C
Feature E
Feature G
Agile - why
107
Wedne...
Features In Progress Blocked Tabled Testing Complete
Feature AFeature BFeature C
Feature E
Feature D
Feature F
Feature G
A...
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
Feature F
Feature G...
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
Feature F
Feature G...
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature EFeature D
Feature F
Feature G
...
Agile - why
57%
14%
29%
Failed Successful Challenged
49%
42%
9%
Waterfall Agile
Source: The CHAOS Manifesto, The Standish ...
• CI/CD when
appropriate
• Quality is in the
eye of the
beholder
• Release fast,
release slow
Agile - How
Features In Prog...
IMAGE: blogs.bu.edu
Agile - How
• Increase quality
by reusing code
from partners
and libraries
112
Thursday, July 11, 13
Agile - How
113
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavio...
ENSURES QUALITY
Agile - How
113
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit...
ENSURES CONFIDENCE
Agile - How
114
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
U...
secure
Thursday, July 11, 13
Security - what
116
•All about Trust
•Provider
•Customer
•User
Thursday, July 11, 13
117
SECURITY - HOW
•Security at every layer
PRESENTATION
APPLICATION
STORAGE
Thursday, July 11, 13
117
SECURITY - HOW
•Security at every layer
PRESENTATION
APPLICATION
STORAGE
INTERFACE
SECURITY
ACROSS ALL LAYERS
Thursday...
118
SECURITY - HOW
•Use industry standards
•SSL
•Encryption
•At rest
•In-transit
Thursday, July 11, 13
119
SECURITY - HOW
•Authentication
•User authentication
•Manage yourself
•Offload to 3rd party
•Host-based Authentication
...
120
SECURITY - HOW
•Hybrid Cloud
•Store different
types of data in the
right place
•Sensitive data in on-
premise cloud
•N...
121
SECURITY - WHY
•Trustworthiness as an organization
•Cloud security is too important
to leave up to the end-user alone
...
BREAK
3:00 - 3:15
Thursday, July 11, 13
APPLICATION
dissection
alex brandt
sr developer
technical track
wayne walls
cloud evangelist
Thursday, July 11, 13
Data Schema
•images
• url
• stars
• *title
• caption
• variants
•small
•medium
•big
•tags
•*_id
•rolestars
•*image_id
•*ro...
Requirements
•List stars
•List roles’ images
•CRUD
•Tag searching
•Who liked what
•Social media
•Image resizing
•Added by ...
application decisions
API TIER
UT UR RS
WORKER
TIER
LB
AUTH
MQ DB
OBJECT
STORAGE
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balan...
application decisions
API TIER
UT UR RS
WORKER
TIER
AUTH
MQ DB
OBJECT
STORAGE
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balancer...
application decisions
API TIER
UT UR RS
WORKER
TIER
AUTH
MQ DB
OBJECT
STORAGE
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balancer...
application decisions
API TIER
UT UR RS
WORKER
TIER
MQ DB
OBJECT
STORAGE
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balancer
•Inf...
application decisions
API TIER
UT UR RS
WORKER
TIER
DB
OBJECT
STORAGE
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balancer
•Infras...
application decisions
API TIER
UT UR RS
WORKER
TIER
OBJECT
STORAGE
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balancer
•Infrastru...
application decisions
API TIER
UT UR RS
WORKER
TIER
CDN
SOCIAL
MEDIA
126
INTERNET
•Load Balancer
•Infrastructure
•Authenti...
application decisions
API TIER
UT UR RS
WORKER
TIER
CDN
126
INTERNET
•Load Balancer
•Infrastructure
•Authentication
•Messa...
application decisions
API TIER
UT UR RS
WORKER
TIER
126
INTERNET
•Load Balancer
•Infrastructure
•Authentication
•Message Q...
PARALLEL
API TIER
UT UR RS
INTERNET
WORKER
TIER
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
WORKER
TIER
API TIER
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
WORKER
TIER
API TIER
•Works in tandem
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
WORKER
TIER
API TIER
•Works in tandem
•Doesn’t wait for workers
127
Thursday...
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
•Update Tags, Update Relations,
Resize Workers operate in
tandem
WORKER
TIER...
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
•Update Tags, Update Relations,
Resize Workers operate in
tandem
•Non-blocki...
MODULAR
API TIER
UT UR RS
WORKER
TIER
INTERNET
128
Thursday, July 11, 13
•Separate tiers by functionality
MODULAR
API TIER
UT UR RS
WORKER
TIER
API TIER
UT UR RS
INTERNET
128
Thursday, July 11, 13
•Separate tiers by functionality
MODULAR
API TIER
UT UR RS
•Use MQ to decouple API from
worker
WORKER
TIER
INTERNET
128
Th...
•Separate tiers by functionality
MODULAR
API TIER
UT UR RS
•Use MQ to decouple API from
worker
•Separate workers to simpli...
Horizontal
API TIER
UT UR RS
WORKER
TIER
INTERNET
129
Thursday, July 11, 13
•Restful API for shared nothing
scale
Horizontal
API TIER
UT UR RS
WORKER
TIER
API TIER
INTERNET
129
Thursday, July 11, 13
•Restful API for shared nothing
scale
Horizontal
API TIER
UT UR RS
•Workers share no state, simply
work on input
WORKER
TI...
•Restful API for shared nothing
scale
Horizontal
API TIER
UT UR RS
•Workers share no state, simply
work on input
•Scale da...
AGILE
API TIER
UT UR RS
WORKER
TIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
WORKER
TIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
WORKER
TIER
AP...
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
•CI/CD is achi...
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
•CI/CD is achi...
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
•CI/CD is achi...
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
•CI/CD is achi...
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
•CI/CD is achi...
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are
individually testable
•CI/CD is achi...
SECURE
API TIER
UT UR RS
WORKER
TIER
INTERNET
131
Thursday, July 11, 13
•Use HTTPs for secure data
transmission
SECURE
API TIER
UT UR RS
WORKER
TIER
API TIER
INTERNET
131
Thursday, July 11, 13
•Use HTTPs for secure data
transmission
SECURE
API TIER
UT UR RS
•No host logins; managed by
Chef
WORKER
TIER
API TIER
UT ...
•Use HTTPs for secure data
transmission
SECURE
API TIER
UT UR RS
•No host logins; managed by
Chef
•StormPath for user
auth...
thank you!
@rackspaceUK
@Racker_beighton
@waynewalls
@alunduil
@jennbolestweets
Thursday, July 11, 13
Upcoming SlideShare
Loading in...5
×

Unlocked London - Technical Track

1,673

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,673
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Unlocked London - Technical Track

  1. 1. No 11 cavendish square 11 july 2013 london Thursday, July 11, 13
  2. 2. developer VOCAB Alex brandt sr developer Thursday, July 11, 13
  3. 3. MODULES 3 Thursday, July 11, 13
  4. 4. LIBRARIES 4 Thursday, July 11, 13
  5. 5. FRAMEWORKS •The building blocks of an application - a set of common software functions that take the work out writing all code from scratch. 5 Thursday, July 11, 13
  6. 6. 6 Thursday, July 11, 13
  7. 7. 7 Thursday, July 11, 13
  8. 8. 8 Thursday, July 11, 13
  9. 9. LANGUAGES 9 Thursday, July 11, 13
  10. 10. 10 Thursday, July 11, 13
  11. 11. APIs 11 Thursday, July 11, 13
  12. 12. APIs 12 Thursday, July 11, 13
  13. 13. What is a data structure? •images • url • stars • *title • caption • variants •small •medium •big •tags •*_id •rolestars •*image_id •*role_id •roles • *name • privileges • login • description • location • child_roles • _id 13 Thursday, July 11, 13
  14. 14. API tier/Application STORAGE APPLICATION PRESENTATION 14 Thursday, July 11, 13
  15. 15. API tier/Application STORAGE APPLICATION PRESENTATION APPLICATION/API 14 Thursday, July 11, 13
  16. 16. Bindings 15 Thursday, July 11, 13
  17. 17. Bindings 16 Thursday, July 11, 13
  18. 18. Bindings 17 Thursday, July 11, 13
  19. 19. EVERYTHING IS CODE wayne walls cloud evangelist alex brandt sr developer Thursday, July 11, 13
  20. 20. Infrastructure as code 19 Thursday, July 11, 13
  21. 21. 20 Thursday, July 11, 13
  22. 22. 21 Thursday, July 11, 13
  23. 23. Chef Server Chef Workstation Chef Client knife node Chef Client node environment 22 Thursday, July 11, 13
  24. 24. •Nodes: Servers •Environments: Groups of nodes (Dev/QA/Prod) •Recipes: Describes the desired state of a node •Cookbooks: Group of Recipes •Roles: Group of Cookbooks (Web) •Attribute: information about the current state of a node 23 Thursday, July 11, 13
  25. 25. 24 Thursday, July 11, 13
  26. 26. Puppet CA Puppet Master Puppet Agent https Puppet Agent Puppet Agent 25 Thursday, July 11, 13
  27. 27. •Nodes: Servers •Environments: Groups of nodes (Dev/QA/Prod) •Manifests: Describes the desired state of a node •Catalog: Group of Manifests •Facts: information about the current state of a node 26 Thursday, July 11, 13
  28. 28. 27 Thursday, July 11, 13
  29. 29. Salt Master Salt Minion Salt Minion Salt Minion 28 Thursday, July 11, 13
  30. 30. •Nodes: Minions •Environments: Groups of nodes (Dev/QA/Prod) •States: Describes the desired state of a node •Grains: information about the current state of a node 29 Thursday, July 11, 13
  31. 31. Thinking like a developer 30 Thursday, July 11, 13
  32. 32. scrum Development Product Backlog Sprint Backlog Sprint Working increment of the software 31 1. Design 5. QA 2. Build 4. Release 3. Configure Thursday, July 11, 13
  33. 33. scrum Development 32 1. Design 5. QA 2. Build 4. Release 3. Configure Thursday, July 11, 13
  34. 34. TYPES OF DEVELOPMENT 33 Thursday, July 11, 13
  35. 35. TYPES OF DEVELOPMENT 33 Test Driven Development Thursday, July 11, 13
  36. 36. TYPES OF DEVELOPMENT 33 Test Driven Development Behavior Driven Development Thursday, July 11, 13
  37. 37. TYPES OF DEVELOPMENT 33 Test Driven Development Behavior Driven Development Domain Driven Development Thursday, July 11, 13
  38. 38. TYPES OF DEVELOPMENT 33 Test Driven Development Behavior Driven Development Application Bliss Domain Driven Development Thursday, July 11, 13
  39. 39. Model View Controller (MVC) UPDATES MANIPULATES SEES USES USER MODEL VIEW CONTROLLER 34 Thursday, July 11, 13
  40. 40. DEv tools Thursday, July 11, 13
  41. 41. 36 Thursday, July 11, 13
  42. 42. 37 Thursday, July 11, 13
  43. 43. Gerrit 38 Thursday, July 11, 13
  44. 44. 39 Thursday, July 11, 13
  45. 45. CODE REVIEW PROCESS Thursday, July 11, 13
  46. 46. Our hero, the developer, submits code for review. The code is reviewed in Gerrit by other developers, who approve the change. Jenkins runs automated tests against the code change. Jenkins pushes the code to GitHub so others can build on the change. 41 Thursday, July 11, 13
  47. 47. POLYGLOT PERSISTENCE wayne walls cloud evangelist technical track alex brandt sr developer Thursday, July 11, 13
  48. 48. A database, in the most general sense, is an organized collection of data. Database Defined http://glenn-packer.net 43 Thursday, July 11, 13
  49. 49. ACID •Atomicity: refers to a database’s ability to either fully process or fully roll back a transaction •Consistency: database should ensure that all data written therein follows all rules and constraints specified in the database •Isolation: transactions must be processed securely and independently, without interfering with each other •Durability: the database must ensure that all committed transactions are saved permanently and cannot be accidentally erased, even in a database crash 44 Thursday, July 11, 13
  50. 50. •A relational database is essentially a group of tables or, to use the technical name, entities. •Each table is made up of rows (tuples) and columns (attributes). relational databases 45 Thursday, July 11, 13
  51. 51. Some companies needed something more to accommodate their growing data rdbms may not be the best fit 46 Thursday, July 11, 13
  52. 52. Enter Non-Relational database management systems 47 Thursday, July 11, 13
  53. 53. INTRODUCTION TO NOSQL • NoSQL Database (1998) • NoREL • “Not only SQL” • NoSQL (2009) 48 Thursday, July 11, 13
  54. 54. NoSQL is difficult to define but most NoSQL databases will share common characteristics 1. Non-relational 2. Distributed 3. Open-source 4. Horizontally Scalable 5. Schema-less NoSQL 49 Thursday, July 11, 13
  55. 55. An “aggregate” is a collection of related objects that we wish to treat as a unit orders customers order lines credit cards ID: 1001 line items: customer: Ann 0321293533 0321601912 0131495054 2 1 1 $48 $48 $39 $51 $39 $51 payment details: Card: Amex CC #: 12345 expires: 04/2001 aggregate 50 Thursday, July 11, 13
  56. 56. There are several different types of data stores that are considered to be NoSQL •Key/Value Databases •Document Databases •Column-Family Stores •Graph Databases 51 Thursday, July 11, 13
  57. 57. Key/Value Database •Primarily constructed of aggregates that have a Key or ID that’s used to access the data •The database cannot see structure within the aggregate 52 Thursday, July 11, 13
  58. 58. Key/Value Database •Good Use Cases: • Storing Session Information • User Profiles & Preferences • Shopping Cart Data • Bad Use Cases: • Data Relationships •Multi-operation Transactions •Query by Data 53 Thursday, July 11, 13
  59. 59. •Like a Key-Value database, document databases primarily store and access data using aggregates •In contrast to the Key-Value database, document data bases can see structure within the aggregate. •You are able to retrieve the whole aggregate or specific parts of it document Database 54 Thursday, July 11, 13
  60. 60. document Databases •Good Use Cases: • Event Logging • Blogging Platforms • Real-time Analytics • Bad Use Cases: • Complex Transactions •Queries against Varying Aggregate Structures 55 Thursday, July 11, 13
  61. 61. • Can be looked at as a two-level aggregate structure • The first key is often described as a row ID picking up the aggregate of interest • The second level values of the aggregates are referred to as columns • You are able to access a row as a whole or operations will allow you to pick out a particular column to get specific data COLUMN FAMILY 56 Thursday, July 11, 13
  62. 62. COLUMN-FAMILY datastores •Good Use Cases: • Event Logging • Blogging Platforms • Counters (page visits) • Expiring Usage • Bad Use Cases: • ACID transactions required • Early prototype applications 57 Thursday, July 11, 13
  63. 63. •Unique compared to most NOSQL types because it was not developed with the intention of scaling horizontally into a cluster •Data is stored as “nodes” that are connected by edges •Edges are directional in nature and must be specified •Queries need a starting point and them will traverse the edges to gather the data requested graph database 58 Thursday, July 11, 13
  64. 64. Amanda KNOWS KNOWS KNOWS Robert Gerry Lamar KNOWS App 1 BUILT BUILT KNOWS App 2 USES BUILT Christie KNOWS Destiny Jessica KNOWS App 3 BUILT USES USES USES KNOWS graph database 59 Thursday, July 11, 13
  65. 65. graph databases •Good Use Cases: • Connected Data • Location-based services •Recommendation engines • Bad Use Cases: • Ability to update specific aggregates 60 Thursday, July 11, 13
  66. 66. Companies using nosql 61 DATABASE COMPANIES Thursday, July 11, 13
  67. 67. THE FUTURE IS POLYGLOT 62 Neo4J database Cassandra database Couch database MongoDB database RELATIONAL DATABASE Graph-structured domain rules Columnar data access with decentralization Document structures Document structures with offline processing (Actors) (Actors) (Actors)(Actors) Asynchronous message passing MODULE 1 MODULE 2 MODULE 3 MODULE 4 IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg Thursday, July 11, 13
  68. 68. 63 POLYGLOT PATTERN: cache ORDER TAKING RESTAURANT MANAGEMENT REDIS CACHE MYSQL DATABASE SECONDFIRST CONSUMER RESTAURANT OWNER Thursday, July 11, 13
  69. 69. 64 POLYGLOT PATTERN: write to sql and NOSQL NEW TWEET INSERT LPUSHX LTRIM MySQL TWEETS FOLLOWS FOLLOWERS Redis: Timeline FOLLOWER 1 FOLLOWER 2 FOLLOWER 3 Avoid expensive MySQL joins Thursday, July 11, 13
  70. 70. 65 POLYGLOT PATTERN: REPLICATE FROM MYsql to NOSQL QUERY SERVICE UPDATE SERVICE REDIS MYSQL DATABASE SYSTEM OF RECORD MATERIALIZED VIEW READER WRITER replicate and denormalize query() update() Thursday, July 11, 13
  71. 71. 66 Atomic Consistent Isolated Durable ACID to base Basically Available Soft state Eventually consistent Thursday, July 11, 13
  72. 72. 5 PILLARS OF CLOUDINESS wayne walls cloud evangelist alex brandt sr developer technical track Thursday, July 11, 13
  73. 73. FIVE PILLARS OF CLOUDINESS 1 2 3 4 5 Parallel Modular Horizontal Agile Secure 68 Thursday, July 11, 13
  74. 74. Parallel Thursday, July 11, 13
  75. 75. Parallel - what 1M Records Answer Node 70 Thursday, July 11, 13
  76. 76. Parallel - how 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M 71 Thursday, July 11, 13
  77. 77. Parallel - how 72 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M Thursday, July 11, 13
  78. 78. Parallel - how 72 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M Thursday, July 11, 13
  79. 79. Parallel - how 73 Thursday, July 11, 13
  80. 80. Parallel - how DATA CALCULATION MAP CALCULATION REDUCE DATA CALC D C R D D C R D D C R D SHUFFLE DATA 73 Thursday, July 11, 13
  81. 81. Avoid Deadlock 74 Thursday, July 11, 13
  82. 82. Thread 1 Deadlock 75 Thread 2 Thursday, July 11, 13
  83. 83. Thread 1 Deadlock 75 Thread 2 Thursday, July 11, 13
  84. 84. File A Thread 1 Deadlock 75 Thread 2 Thursday, July 11, 13
  85. 85. File A Thread 1 Deadlock 75 Thread 2 Thursday, July 11, 13
  86. 86. File A File B Thread 1 Deadlock 75 Thread 2 Thursday, July 11, 13
  87. 87. File A File B Thread 1 Deadlock 75 Thread 2 Thursday, July 11, 13
  88. 88. File A File B Thread 1 Deadlock 75 File B Thread 2 Thursday, July 11, 13
  89. 89. File A File B Thread 1 Deadlock 75 File B Thread 2 Thursday, July 11, 13
  90. 90. File A File B File A Thread 1 Deadlock 75 File B Thread 2 Thursday, July 11, 13
  91. 91. File A File B File A Thread 1 Deadlock 75 File B Thread 2 Thursday, July 11, 13
  92. 92. File A File B File A Thread 1 Deadlock 75 X File B Thread 2 Thursday, July 11, 13
  93. 93. File A File B File A Thread 1 Deadlock 75 X File B Thread 2 Thursday, July 11, 13
  94. 94. File A File B File A Thread 1 Deadlock 75 X X File B Thread 2 Thursday, July 11, 13
  95. 95. Solution? Scheduling Y XA B A B Y X 1 2 3 4 76 Thursday, July 11, 13
  96. 96. Parallel - why 1M records Answer Node 1M records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 750K-1M 77 Thursday, July 11, 13
  97. 97. Parallel - why 1M records Answer Node 1M records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 750K-1M 77 Thursday, July 11, 13
  98. 98. Parallel - why 78 Thursday, July 11, 13
  99. 99. Parallel - why 79 Thursday, July 11, 13
  100. 100. modular Thursday, July 11, 13
  101. 101. ONE PERSON BUILDS A CAR Modular - what Example: Assembly Line 81 Thursday, July 11, 13
  102. 102. CHASSIS ENGINE BODY INTERIOR Decoupled Self-contained units Data Communication Modular - what Example: Assembly Line 82 Thursday, July 11, 13
  103. 103. Modular - what Modules within Modules CHASSIS ENGINE BODY INTERIOR 83 Nested Modularity Thursday, July 11, 13
  104. 104. CHASSIS ENGINE BODY INTERIOR Modular - what 84 Modules within Modules Nested Modularity Thursday, July 11, 13
  105. 105. CHASSIS ENGINE BODY INTERIOR Modular - what SEATS FLOOR DASHBOARD GAS AND BRAKE PEDALS 84 Modules within Modules Nested Modularity Thursday, July 11, 13
  106. 106. Small units are more maintainable Allows us to tolerate some partitions (but not all) Allows us to be parallel! CHASSIS ENGINE BODY INTERIOR modular - Why 85 Thursday, July 11, 13
  107. 107. Direct communication (1:1) Topic communication (1:N) Fanout communication (1:ALL) modular - Why 86 Thursday, July 11, 13
  108. 108. •Decouple requests •Message Queue •Make small units of work •Define tiers by function modular - how 87 Thursday, July 11, 13
  109. 109. Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  110. 110. defect Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  111. 111. defect defect Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  112. 112. defect defect defect Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  113. 113. defect defect defect defect Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  114. 114. defect defect defect defect FAIL Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  115. 115. defect defect defect defect FAIL Mandelbugs Potential for Upstream Complexity CHASSIS ENGINE BODY INTERIOR 88 Thursday, July 11, 13
  116. 116. horizontal Thursday, July 11, 13
  117. 117. 2 GB 2 CORES Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  118. 118. 2 GB 2 CORES VERTICAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  119. 119. 2 GB 2 CORES VERTICAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  120. 120. 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  121. 121. 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  122. 122. 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  123. 123. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  124. 124. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  125. 125. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  126. 126. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  127. 127. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  128. 128. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  129. 129. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS Horizontal - What 90 8 GB 4 CORES Thursday, July 11, 13
  130. 130. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS Horizontal - What 2 GB 2 CORES 90 8 GB 4 CORES Thursday, July 11, 13
  131. 131. Horizontal - why 91 Thursday, July 11, 13
  132. 132. Horizontal - why 92 Thursday, July 11, 13
  133. 133. ONE SERVER NEAR CAPACITY Horizontal - why 92 Thursday, July 11, 13
  134. 134. ONE SERVER NEAR CAPACITY ADD ANOTHER SERVER Horizontal - why 92 Thursday, July 11, 13
  135. 135. ONE SERVER NEAR CAPACITY ADD ANOTHER SERVER Horizontal - why 93 Thursday, July 11, 13
  136. 136. 1234 MESSAGE QUEUE 94 Horizontal - HOW Thursday, July 11, 13
  137. 137. 1234MESSAGE QUEUE 95 Horizontal - HOW Thursday, July 11, 13
  138. 138. 1 2 3 4 96 MESSAGE QUEUE Horizontal - HOW Thursday, July 11, 13
  139. 139. 1 2 3 4 96 56789101112 MESSAGE QUEUE Horizontal - HOW Thursday, July 11, 13
  140. 140. 97 5678 9101112MESSAGE QUEUE 1 2 3 4 Horizontal - HOW Thursday, July 11, 13
  141. 141. 97 5678 9101112 ADD MORE SERVERS BASED ON MESSAGE QUEUE MESSAGE QUEUE 1 2 3 4 Horizontal - HOW Thursday, July 11, 13
  142. 142. MESSAGE QUEUE 98 5 6 7 8 9 10 11 12 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1 2 3 4 Horizontal - HOW Thursday, July 11, 13
  143. 143. 99 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Horizontal - HOW Thursday, July 11, 13
  144. 144. 100 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Horizontal - HOW Thursday, July 11, 13
  145. 145. 101 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Horizontal - HOW Thursday, July 11, 13
  146. 146. 101 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Horizontal - HOW DECREASE SERVERS BASED ON MESSAGE QUEUE Thursday, July 11, 13
  147. 147. MESSAGE QUEUE 102 5678 9101112 13 14 15 16 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 Horizontal - HOW DECREASE SERVERS BASED ON MESSAGE QUEUE Thursday, July 11, 13
  148. 148. agile Thursday, July 11, 13
  149. 149. Agile - WHAT the power of moving quickly and easily; nimbleness 104 Thursday, July 11, 13
  150. 150. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C 105 Agile - why Monday Thursday, July 11, 13
  151. 151. Agile - why Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C 106 Tuesday Thursday, July 11, 13
  152. 152. Agile - why Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D 106 Tuesday Thursday, July 11, 13
  153. 153. Features In Progress Blocked Tabled Testing Complete Feature AFeature BFeature C Feature E Feature G Agile - why 107 Wednesday Thursday, July 11, 13
  154. 154. Features In Progress Blocked Tabled Testing Complete Feature AFeature BFeature C Feature E Feature D Feature F Feature G Agile - why 107 Wednesday Thursday, July 11, 13
  155. 155. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D Feature F Feature G Agile - why 108 Friday Thursday, July 11, 13
  156. 156. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D Feature F Feature G Feature H Feature I Agile - why 108 Friday Thursday, July 11, 13
  157. 157. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature EFeature D Feature F Feature G Feature H Feature I Agile - why 109 Friday - WK 2 Thursday, July 11, 13
  158. 158. Agile - why 57% 14% 29% Failed Successful Challenged 49% 42% 9% Waterfall Agile Source: The CHAOS Manifesto, The Standish Group, 2012 110 Thursday, July 11, 13
  159. 159. • CI/CD when appropriate • Quality is in the eye of the beholder • Release fast, release slow Agile - How Features In Progress Blocked Tabled Testing Complete 111 Thursday, July 11, 13
  160. 160. IMAGE: blogs.bu.edu Agile - How • Increase quality by reusing code from partners and libraries 112 Thursday, July 11, 13
  161. 161. Agile - How 113 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Thursday, July 11, 13
  162. 162. ENSURES QUALITY Agile - How 113 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Thursday, July 11, 13
  163. 163. ENSURES CONFIDENCE Agile - How 114 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Thursday, July 11, 13
  164. 164. secure Thursday, July 11, 13
  165. 165. Security - what 116 •All about Trust •Provider •Customer •User Thursday, July 11, 13
  166. 166. 117 SECURITY - HOW •Security at every layer PRESENTATION APPLICATION STORAGE Thursday, July 11, 13
  167. 167. 117 SECURITY - HOW •Security at every layer PRESENTATION APPLICATION STORAGE INTERFACE SECURITY ACROSS ALL LAYERS Thursday, July 11, 13
  168. 168. 118 SECURITY - HOW •Use industry standards •SSL •Encryption •At rest •In-transit Thursday, July 11, 13
  169. 169. 119 SECURITY - HOW •Authentication •User authentication •Manage yourself •Offload to 3rd party •Host-based Authentication •Security Groups •Firewalls •Configuration Management Systems Thursday, July 11, 13
  170. 170. 120 SECURITY - HOW •Hybrid Cloud •Store different types of data in the right place •Sensitive data in on- premise cloud •Non-sensitive on public cloud Thursday, July 11, 13
  171. 171. 121 SECURITY - WHY •Trustworthiness as an organization •Cloud security is too important to leave up to the end-user alone •Ownership gaps that leads to process and governance issues •Compliance •Logging and auditing •Regulatory requirements Thursday, July 11, 13
  172. 172. BREAK 3:00 - 3:15 Thursday, July 11, 13
  173. 173. APPLICATION dissection alex brandt sr developer technical track wayne walls cloud evangelist Thursday, July 11, 13
  174. 174. Data Schema •images • url • stars • *title • caption • variants •small •medium •big •tags •*_id •rolestars •*image_id •*role_id •roles • *name • privileges • login • description • location • child_roles • _id 124 Thursday, July 11, 13
  175. 175. Requirements •List stars •List roles’ images •CRUD •Tag searching •Who liked what •Social media •Image resizing •Added by tag •Hot tagged images 125 Thursday, July 11, 13
  176. 176. application decisions API TIER UT UR RS WORKER TIER LB AUTH MQ DB OBJECT STORAGE CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  177. 177. application decisions API TIER UT UR RS WORKER TIER AUTH MQ DB OBJECT STORAGE CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  178. 178. application decisions API TIER UT UR RS WORKER TIER AUTH MQ DB OBJECT STORAGE CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  179. 179. application decisions API TIER UT UR RS WORKER TIER MQ DB OBJECT STORAGE CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  180. 180. application decisions API TIER UT UR RS WORKER TIER DB OBJECT STORAGE CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  181. 181. application decisions API TIER UT UR RS WORKER TIER OBJECT STORAGE CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  182. 182. application decisions API TIER UT UR RS WORKER TIER CDN SOCIAL MEDIA 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  183. 183. application decisions API TIER UT UR RS WORKER TIER CDN 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  184. 184. application decisions API TIER UT UR RS WORKER TIER 126 INTERNET •Load Balancer •Infrastructure •Authentication •Message Queue •Database •Content Delivery •Social Media Thursday, July 11, 13
  185. 185. PARALLEL API TIER UT UR RS INTERNET WORKER TIER 127 Thursday, July 11, 13
  186. 186. •API Tier PARALLEL API TIER UT UR RS INTERNET WORKER TIER API TIER 127 Thursday, July 11, 13
  187. 187. •API Tier PARALLEL API TIER UT UR RS INTERNET WORKER TIER API TIER •Works in tandem 127 Thursday, July 11, 13
  188. 188. •API Tier PARALLEL API TIER UT UR RS INTERNET WORKER TIER API TIER •Works in tandem •Doesn’t wait for workers 127 Thursday, July 11, 13
  189. 189. •API Tier PARALLEL API TIER UT UR RS INTERNET •Update Tags, Update Relations, Resize Workers operate in tandem WORKER TIER UT UR RS •Works in tandem •Doesn’t wait for workers 127 Thursday, July 11, 13
  190. 190. •API Tier PARALLEL API TIER UT UR RS INTERNET •Update Tags, Update Relations, Resize Workers operate in tandem •Non-blocking actions WORKER TIER UT UR RS API TIER •Works in tandem •Doesn’t wait for workers 127 Thursday, July 11, 13
  191. 191. MODULAR API TIER UT UR RS WORKER TIER INTERNET 128 Thursday, July 11, 13
  192. 192. •Separate tiers by functionality MODULAR API TIER UT UR RS WORKER TIER API TIER UT UR RS INTERNET 128 Thursday, July 11, 13
  193. 193. •Separate tiers by functionality MODULAR API TIER UT UR RS •Use MQ to decouple API from worker WORKER TIER INTERNET 128 Thursday, July 11, 13
  194. 194. •Separate tiers by functionality MODULAR API TIER UT UR RS •Use MQ to decouple API from worker •Separate workers to simplify units of work WORKER TIER UT UR RS INTERNET 128 Thursday, July 11, 13
  195. 195. Horizontal API TIER UT UR RS WORKER TIER INTERNET 129 Thursday, July 11, 13
  196. 196. •Restful API for shared nothing scale Horizontal API TIER UT UR RS WORKER TIER API TIER INTERNET 129 Thursday, July 11, 13
  197. 197. •Restful API for shared nothing scale Horizontal API TIER UT UR RS •Workers share no state, simply work on input WORKER TIER INTERNET 129 Thursday, July 11, 13
  198. 198. •Restful API for shared nothing scale Horizontal API TIER UT UR RS •Workers share no state, simply work on input •Scale data across nodes (even DCs) WORKER TIER UT UR RS API TIER INTERNET 129 Thursday, July 11, 13
  199. 199. AGILE API TIER UT UR RS WORKER TIER INTERNET 130 Thursday, July 11, 13
  200. 200. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS WORKER TIER INTERNET 130 Thursday, July 11, 13
  201. 201. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable WORKER TIER API TIER UT UR RS INTERNET 130 Thursday, July 11, 13
  202. 202. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable •CI/CD is achievable by component WORKER TIER INTERNET 130 Thursday, July 11, 13
  203. 203. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable •CI/CD is achievable by component WORKER TIER INTERNET 130 Thursday, July 11, 13
  204. 204. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable •CI/CD is achievable by component WORKER TIER INTERNET 130 Thursday, July 11, 13
  205. 205. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable •CI/CD is achievable by component WORKER TIER INTERNET 130 Thursday, July 11, 13
  206. 206. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable •CI/CD is achievable by component WORKER TIER INTERNET 130 Thursday, July 11, 13
  207. 207. •Don’t re-invent ability or hosting AGILE API TIER UT UR RS •Split processes that are individually testable •CI/CD is achievable by component WORKER TIER API TIER INTERNET 130 Thursday, July 11, 13
  208. 208. SECURE API TIER UT UR RS WORKER TIER INTERNET 131 Thursday, July 11, 13
  209. 209. •Use HTTPs for secure data transmission SECURE API TIER UT UR RS WORKER TIER API TIER INTERNET 131 Thursday, July 11, 13
  210. 210. •Use HTTPs for secure data transmission SECURE API TIER UT UR RS •No host logins; managed by Chef WORKER TIER API TIER UT UR RS INTERNET 131 Thursday, July 11, 13
  211. 211. •Use HTTPs for secure data transmission SECURE API TIER UT UR RS •No host logins; managed by Chef •StormPath for user authentication WORKER TIER INTERNET 131 Thursday, July 11, 13
  212. 212. thank you! @rackspaceUK @Racker_beighton @waynewalls @alunduil @jennbolestweets Thursday, July 11, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×