0
margarine
dissection
alex brandt
sr developer
wayne walls
cloud evangelist
Wednesday, August 21, 13
2
WHAT IS MARGARINE?
Wednesday, August 21, 13
3
3 PARTS
Wednesday, August 21, 13
4
BLEND
TINGE
SPREAD
Wednesday, August 21, 13
5
BLEND
TINGE
SPREAD
Wednesday, August 21, 13
6
BLEND
TINGE
SPREAD
Wednesday, August 21, 13
7
WHAT DO WE CARE ABOUT?
Wednesday, August 21, 13
8
SIMPLICITY!
Wednesday, August 21, 13
9
DESIGN DISCLAIMER
Wednesday, August 21, 13
10
LB
FRONT END API
MQ
CONSUMER
LB
KEY/VALUE
STORE
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fanout
•articl...
11
BUT WHAT DOES IT ALL MEAN?!
Wednesday, August 21, 13
12
DATA TIER
Wednesday, August 21, 13
13
WHAT KIND OF DATA DO WE
NEED TO STORE?
Wednesday, August 21, 13
14
Content of Articles
User Information / Profiles
Subscriptions by Users
Tags by Users
Notations for Margin Notes
Wednesd...
15
Content of Articles
User Information / Profiles
Subscriptions by Users
Tags by Users
Notations for Margin Notes
Wednesd...
16
Content of Articles
User Information / Profiles
Subscriptions by Users
Tags by Users
Notations for Margin Notes
Wednesd...
17
Content of Articles
User Information / Profiles
Subscriptions by Users
Tags by Users
Notations for Margin Notes
Wednesd...
18
Content of Articles
User Information / Profiles
Subscriptions by Users
Tags by Users
Notations for Margin Notes
Wednesd...
19
DATA TIER DECISIONS
Wednesday, August 21, 13
20
Where to Store the Data
How to Organize the Data
How to Manage Users
API Strategy
Wednesday, August 21, 13
21
Where to Store the Data
How to Organize the Data
How to Manage Users
API Strategy
Wednesday, August 21, 13
Data Schema
•articles
• url
• text
• tags-index
• notations
• location
• note
• votes
• created_at-index
• etag
• parsed_a...
23
Where to Store the Data
How to Organize the Data
How to Manage Users
API Strategy
Wednesday, August 21, 13
24
Where to Store the Data
How to Organize the Data
How to Manage Users
API Strategy
Wednesday, August 21, 13
25
APPLICATION TIER
Wednesday, August 21, 13
26
APPLICATION TIER ACTION!
Wednesday, August 21, 13
27
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
ADVANCED OPERATIONS FOR
USERS & ARTICLES
Wednesday, August 21, 13
28
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
ADVANCED OPERATIONS FOR
USERS & ARTICLES
Wednesday, August 21, 13
29
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
ADVANCED OPERATIONS FOR
USERS & ARTICLES
Wednesday, August 21, 13
30
APPLICATION TIER DECISIONS
Wednesday, August 21, 13
31
HOW IS DATA PROCESSED?
WHERE IS DATA PROCESSED?
Wednesday, August 21, 13
32
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
ADVANCED OPERATIONS FOR
USERS & ARTICLES
Wednesday, August 21, 13
33
A LOT OF WORK HERE...
Wednesday, August 21, 13
34
SPLIT THE APPLICATION TIER!
Wednesday, August 21, 13
34
SPLIT THE APPLICATION TIER!
BLEND SPREAD
Wednesday, August 21, 13
35
ADVANCED OPERATIONS FOR USERS
ADVANCED OPERATIONS FOR ARTICLES
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
B...
36
HOW DOES ONE SPLIT THE
APPLICATION TIER?
Wednesday, August 21, 13
37
MESSAGE QUEUE, BUT WHICH?
Wednesday, August 21, 13
38
NOW THAT WE HAVE THAT
FIGURED OUT
Wednesday, August 21, 13
39
BLEND
SPREAD
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
ADVANCED OPERATIONS FOR USERS
ADVANCED OPERATIONS F...
40
BLEND
SEVEN CONSUMER/WORKER SERVICES
Create	
  Ar(cle
Update	
  References	
  
Sani(ze	
  HTML
Create	
  User
Update	
 ...
41
BLEND
SPREAD
CRUD OPERATIONS FOR USERS
CRUD OPERATIONS FOR ARTICLES
ADVANCED OPERATIONS FOR USERS
ADVANCED OPERATIONS F...
42
WEB TIER
Wednesday, August 21, 13
43
WHAT ARE WE SERVING OUR
CUSTOMERS?
Wednesday, August 21, 13
44
WEB PORTAL FOR...
Wednesday, August 21, 13
44
WEB PORTAL FOR...
Logins
Wednesday, August 21, 13
44
WEB PORTAL FOR...
Logins
Password Resets
Wednesday, August 21, 13
44
WEB PORTAL FOR...
Logins
Password Resets
Viewing and Tagging of Articles 
Wednesday, August 21, 13
44
WEB PORTAL FOR...
Logins
Password Resets
Viewing and Tagging of Articles 
Submission of Articles
Wednesday, August 21, ...
45
WEB TIER DECISIONS
Wednesday, August 21, 13
46
OPERATING SYSTEM / STACK
DATASTORE SOLUTION
LOAD BALANCING
DNS
CONTENT DELIVERY
Wednesday, August 21, 13
47
Ubuntu 12.04
python; python-flask
nginx in front of uWSGI
Wednesday, August 21, 13
48
OPERATING SYSTEM / STACK
DATASTORE SOLUTION
LOAD BALANCING
DNS
CONTENT DELIVERY
Wednesday, August 21, 13
49
CloudFiles as Key/Value Store
ObjectRocket as Document Store
Exceptional.io Redis2Go as Key/Value Store
Wednesday, Augu...
50
CloudFiles as Key/Value Store
ObjectRocket as Document Store
Exceptional.io Redis2Go as Key/Value Store
Wednesday, Augu...
51
CloudFiles as Key/Value Store
ObjectRocket as DocumentStore
Exceptional.io Redis2Go as Key/Value Store
Wednesday, Augus...
52
OPERATING SYSTEM / STACK
DATASTORE SOLUTION
LOAD BALANCING
DNS
CONTENT DELIVERY
Wednesday, August 21, 13
53
RACKSPACE
Wednesday, August 21, 13
54
OPERATING SYSTEM / STACK
DATASTORE SOLUTION
LOAD BALANCING
DNS
CONTENT DELIVERY
Wednesday, August 21, 13
55
Wednesday, August 21, 13
56
OPERATING SYSTEM / STACK
DATASTORE SOLUTION
LOAD BALANCING
DNS
CONTENT DELIVERY
Wednesday, August 21, 13
57
Tinge to CloudFiles
SEO Management
Wednesday, August 21, 13
58
Tinge to CloudFiles
SEO Management
Wednesday, August 21, 13
59
CANONICAL ARCHITECTURE
WITH PILLARS
Wednesday, August 21, 13
LB
FRONT END API
MQ
CONSUMER
LB
KEY/VALUE
STORE
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fanout
•article.s...
LB
FRONT END API
MQ
CONSUMER
LB
KEY/VALUE
STORE
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fanout
•article.s...
LB
FRONT END API
MQ
CONSUMER
LB
KEY/VALUE
STORE
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fanout
•article.s...
LB
FRONT END API
MQ
CONSUMER
LB
KEY/VALUE
STORE
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fanout
•article.s...
LB
FRONT END API
MQ
CONSUMER
LB
KEY/VALUE
STORE
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fanout
•article.s...
KEY/VALUE
STORE
OBJECT
STORE
LB
FRONT END API
MQ
CONSUMER
LB
DOCUMENT
STORE
INTERNET
Routing
•Topic
•users
•articles
•Fano...
66
https://github.com/raxsavvy/margarine
Contribute
Wednesday, August 21, 13
Upcoming SlideShare
Loading in...5
×

Butter Web Browsing with Margarine

1,246

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,246
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Butter Web Browsing with Margarine"

  1. 1. margarine dissection alex brandt sr developer wayne walls cloud evangelist Wednesday, August 21, 13
  2. 2. 2 WHAT IS MARGARINE? Wednesday, August 21, 13
  3. 3. 3 3 PARTS Wednesday, August 21, 13
  4. 4. 4 BLEND TINGE SPREAD Wednesday, August 21, 13
  5. 5. 5 BLEND TINGE SPREAD Wednesday, August 21, 13
  6. 6. 6 BLEND TINGE SPREAD Wednesday, August 21, 13
  7. 7. 7 WHAT DO WE CARE ABOUT? Wednesday, August 21, 13
  8. 8. 8 SIMPLICITY! Wednesday, August 21, 13
  9. 9. 9 DESIGN DISCLAIMER Wednesday, August 21, 13
  10. 10. 10 LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Wednesday, August 21, 13
  11. 11. 11 BUT WHAT DOES IT ALL MEAN?! Wednesday, August 21, 13
  12. 12. 12 DATA TIER Wednesday, August 21, 13
  13. 13. 13 WHAT KIND OF DATA DO WE NEED TO STORE? Wednesday, August 21, 13
  14. 14. 14 Content of Articles User Information / Profiles Subscriptions by Users Tags by Users Notations for Margin Notes Wednesday, August 21, 13
  15. 15. 15 Content of Articles User Information / Profiles Subscriptions by Users Tags by Users Notations for Margin Notes Wednesday, August 21, 13
  16. 16. 16 Content of Articles User Information / Profiles Subscriptions by Users Tags by Users Notations for Margin Notes Wednesday, August 21, 13
  17. 17. 17 Content of Articles User Information / Profiles Subscriptions by Users Tags by Users Notations for Margin Notes Wednesday, August 21, 13
  18. 18. 18 Content of Articles User Information / Profiles Subscriptions by Users Tags by Users Notations for Margin Notes Wednesday, August 21, 13
  19. 19. 19 DATA TIER DECISIONS Wednesday, August 21, 13
  20. 20. 20 Where to Store the Data How to Organize the Data How to Manage Users API Strategy Wednesday, August 21, 13
  21. 21. 21 Where to Store the Data How to Organize the Data How to Manage Users API Strategy Wednesday, August 21, 13
  22. 22. Data Schema •articles • url • text • tags-index • notations • location • note • votes • created_at-index • etag • parsed_at •users • username-unique index • email • name • hash (md5) 22 •subscribers • uuid4-user • uuid4-bookmark • subscribed_at Wednesday, August 21, 13
  23. 23. 23 Where to Store the Data How to Organize the Data How to Manage Users API Strategy Wednesday, August 21, 13
  24. 24. 24 Where to Store the Data How to Organize the Data How to Manage Users API Strategy Wednesday, August 21, 13
  25. 25. 25 APPLICATION TIER Wednesday, August 21, 13
  26. 26. 26 APPLICATION TIER ACTION! Wednesday, August 21, 13
  27. 27. 27 CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES ADVANCED OPERATIONS FOR USERS & ARTICLES Wednesday, August 21, 13
  28. 28. 28 CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES ADVANCED OPERATIONS FOR USERS & ARTICLES Wednesday, August 21, 13
  29. 29. 29 CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES ADVANCED OPERATIONS FOR USERS & ARTICLES Wednesday, August 21, 13
  30. 30. 30 APPLICATION TIER DECISIONS Wednesday, August 21, 13
  31. 31. 31 HOW IS DATA PROCESSED? WHERE IS DATA PROCESSED? Wednesday, August 21, 13
  32. 32. 32 CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES ADVANCED OPERATIONS FOR USERS & ARTICLES Wednesday, August 21, 13
  33. 33. 33 A LOT OF WORK HERE... Wednesday, August 21, 13
  34. 34. 34 SPLIT THE APPLICATION TIER! Wednesday, August 21, 13
  35. 35. 34 SPLIT THE APPLICATION TIER! BLEND SPREAD Wednesday, August 21, 13
  36. 36. 35 ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES BLEND SPREAD Wednesday, August 21, 13
  37. 37. 36 HOW DOES ONE SPLIT THE APPLICATION TIER? Wednesday, August 21, 13
  38. 38. 37 MESSAGE QUEUE, BUT WHICH? Wednesday, August 21, 13
  39. 39. 38 NOW THAT WE HAVE THAT FIGURED OUT Wednesday, August 21, 13
  40. 40. 39 BLEND SPREAD CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES RabbitMQ Wednesday, August 21, 13
  41. 41. 40 BLEND SEVEN CONSUMER/WORKER SERVICES Create  Ar(cle Update  References   Sani(ze  HTML Create  User Update  User Password  E-­‐mail Password  Change Wednesday, August 21, 13
  42. 42. 41 BLEND SPREAD CRUD OPERATIONS FOR USERS CRUD OPERATIONS FOR ARTICLES ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES RabbitMQ Wednesday, August 21, 13
  43. 43. 42 WEB TIER Wednesday, August 21, 13
  44. 44. 43 WHAT ARE WE SERVING OUR CUSTOMERS? Wednesday, August 21, 13
  45. 45. 44 WEB PORTAL FOR... Wednesday, August 21, 13
  46. 46. 44 WEB PORTAL FOR... Logins Wednesday, August 21, 13
  47. 47. 44 WEB PORTAL FOR... Logins Password Resets Wednesday, August 21, 13
  48. 48. 44 WEB PORTAL FOR... Logins Password Resets Viewing and Tagging of Articles  Wednesday, August 21, 13
  49. 49. 44 WEB PORTAL FOR... Logins Password Resets Viewing and Tagging of Articles  Submission of Articles Wednesday, August 21, 13
  50. 50. 45 WEB TIER DECISIONS Wednesday, August 21, 13
  51. 51. 46 OPERATING SYSTEM / STACK DATASTORE SOLUTION LOAD BALANCING DNS CONTENT DELIVERY Wednesday, August 21, 13
  52. 52. 47 Ubuntu 12.04 python; python-flask nginx in front of uWSGI Wednesday, August 21, 13
  53. 53. 48 OPERATING SYSTEM / STACK DATASTORE SOLUTION LOAD BALANCING DNS CONTENT DELIVERY Wednesday, August 21, 13
  54. 54. 49 CloudFiles as Key/Value Store ObjectRocket as Document Store Exceptional.io Redis2Go as Key/Value Store Wednesday, August 21, 13
  55. 55. 50 CloudFiles as Key/Value Store ObjectRocket as Document Store Exceptional.io Redis2Go as Key/Value Store Wednesday, August 21, 13
  56. 56. 51 CloudFiles as Key/Value Store ObjectRocket as DocumentStore Exceptional.io Redis2Go as Key/Value Store Wednesday, August 21, 13
  57. 57. 52 OPERATING SYSTEM / STACK DATASTORE SOLUTION LOAD BALANCING DNS CONTENT DELIVERY Wednesday, August 21, 13
  58. 58. 53 RACKSPACE Wednesday, August 21, 13
  59. 59. 54 OPERATING SYSTEM / STACK DATASTORE SOLUTION LOAD BALANCING DNS CONTENT DELIVERY Wednesday, August 21, 13
  60. 60. 55 Wednesday, August 21, 13
  61. 61. 56 OPERATING SYSTEM / STACK DATASTORE SOLUTION LOAD BALANCING DNS CONTENT DELIVERY Wednesday, August 21, 13
  62. 62. 57 Tinge to CloudFiles SEO Management Wednesday, August 21, 13
  63. 63. 58 Tinge to CloudFiles SEO Management Wednesday, August 21, 13
  64. 64. 59 CANONICAL ARCHITECTURE WITH PILLARS Wednesday, August 21, 13
  65. 65. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Wednesday, August 21, 13
  66. 66. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Wednesday, August 21, 13
  67. 67. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Wednesday, August 21, 13
  68. 68. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Wednesday, August 21, 13
  69. 69. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Wednesday, August 21, 13
  70. 70. KEY/VALUE STORE OBJECT STORE LB FRONT END API MQ CONSUMER LB DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize Parallel Modular Horizontally scalable Agile Secure Wednesday, August 21, 13
  71. 71. 66 https://github.com/raxsavvy/margarine Contribute Wednesday, August 21, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×