SlideShare a Scribd company logo
Architecture by accident

       Gleicon	
  Moraes	
  
Required Listening
Agenda

•    Architecture for data - even if you don’t want it
•    Databases
•    Message Queues
•    Cache
Architecture
“Everyone	
  has	
  a	
  plan	
  un4l	
  they	
  get	
  punched	
  in	
  
the	
  mouth”	
  –	
  Mike	
  Tyson	
  
	
  
Even if you dont want it ...

•  There is an innate architecture on everything
•  You may end up with more data than you had
   planned to
•  You may get away from your quick and dirty CRUD
•  You probably are querying more than one Database
•  At some point you laugh when your boss asks you
   about 'Integrating Systems'
•  Code turns into legacy - and so architectures
•  'Scattered' is not the same that 'Distributed'
It	
  usually	
  starts	
  like	
  this	
  


         App Server       Database
then

App Servers     Database
it

App Servers        Master DB




                   Slave DB
goes

App Servers       Master DB




                  Slave DB




   Cache
like
App Servers       Master DB




                                  Slave DB




   Cache



               Indexing Service
this
App Servers           Master DB




                                     Slave DB




 Cache




                  Indexing Service


 API Servers
and
                              App Servers       Master DB




Load Balancer/Reverse Proxy




                                                                Slave DB




                               Cache




                                             Indexing Service


                               API Servers
beyond
                              App Servers       Master DB




Load Balancer/Reverse Proxy




                                                                Slave DB




                               Cache




                                             Indexing Service


                               API Servers
       Auth Service
Problem is...
An architect s first work is apt to be spare and clean. He knows he
doesn t know what he s doing, so he does it carefully and with great
restraint.	
  
As he designs the first work, frill after frill and embellishment after
embellishment occur to him. These get stored away to be used next
time. Sooner or later the first system is finished, and the architect,
with firm confidence and a demonstrated mastery of that class of
systems, is ready to build a second system.	
  
This second is the most dangerous system a man ever designs. When
he does his third and later ones, his prior experiences will confirm each
other as to the general characteristics of such systems, and their
differences will identify those parts of his experience that are particular
and not generalizable.
	
  
The general tendency is to over-design the second system, using all
the ideas and frills that were cautiously sidetracked on the first one.
The result, as Ovid says, is a big pile. 	
  
                                                        — Frederick P. Brooks, Jr.
                                                        The Mythical Man-Month
Databases
Databases	
  
•  Not	
  an	
  off-­‐the-­‐shelf	
  architectural	
  duct	
  tape	
  
•  Not	
  only	
  rela4onal,	
  other	
  paradigms	
  
•  Usually	
  the	
  last	
  place	
  sought	
  for	
  op4miza4on	
  
•  Usually	
  the	
  first	
  place	
  to	
  accomodate	
  last	
  minute	
  
   changes	
  
•  Good	
  ideas	
  to	
  try	
  out:	
  Sharding	
  and	
  
   Denormaliza4on	
  
•  Some	
  of	
  your	
  problems	
  may	
  require	
  something	
  
   other	
  than	
  a	
  Rela4onal	
  Database	
  
Relevant RDBMS Anti-Patterns

–    Dynamic table creation
–    Table as cache
–    Table as queue
–    Table as log file
–    Distributed Global Locking
–    Stoned Procedures
–    Row Alignment
–    Extreme JOINs
–    Your ORM issue full queries for Dataset iterations
–    Throttle Control
Dynamic table creation
Problem: To avoid huge tables, "dynamic schema” is
created. For example, lets think about a document
management company, which is adding new facilities over
the country. For each storage facility, a new table is created:

item_id - row - column - stuff
1 - 10 - 20 - cat food
2 - 12 - 32 - trout

Side Effect: "dynamic queries", which will probably query a
"central storage" table and issue a huge join to check if you
have enough cat food over the country. It’s different from
Sharding.

Alternative:
- Document storage, modeling a facility as a document
-  Key/Value, modeling each facility as a SET
-  Sharding properly
Table as cache
Problem: Complex queries demand that a result be
stored in a separated table, so it can be queried
quickly. Worst than views


Alternative:
- Really ?
- Memcached
- Redis + AOF + EXPIRE
- Denormalization
Table as queue
Problem: A table which holds messages to be
completed. Worse, they must be sorted by date.

Alternative:
- RestMQ, Resque
- Any other message broker
- Redis (LISTS - LPUSH + RPOP)
- Use the right tool
Table as log file
Problem: A table in which data gets written as a log
file. From time to time it needs to be purged.
Truncating this table once a day usually is the first task
assigned to trainee DBAs.

Alternative:
- MongoDB capped collection
- Redis, and a RRD pattern
-  RIAK
Distributed Global Locking
Problem: Someone learns java and synchronize. A bit
later genius thinks that a distributed synchronize would
be awesome. The proper place to do that would be the
database of course. Start with a reference counter in a
table and end up with this:

> select COALESCE(GET_LOCK('my_lock',0 ),0 )

Plain and simple, you might find it embedded in a
magic class called DistributedSynchronize or
ClusterSemaphore. Locks, transactions and reference
counters (which may act as soft locks) doesn't belongs
to the database.
Stoned procedures
Problem: Stored procedures hold most of your
applications logic. Also, some triggers are used to - well
- trigger important data events.

SP and triggers has the magic property of vanishing of
our mind instantly, being impossible to keep versioned.

Alternative:
- Careful so you don’t use map/reduce as stoned
procedures.
- Use your preferred language for business stuff, and
let event handling to pub/sub or message queues.
Row Alignment
Problem: Extra rows are created but not used, just in
case. Usually they are named as a1, a2, a3, a4 and
called padding.

There's good will behind that, specially when version 1
of the software needed an extra column in a 150M lines
database and it took 2 days to run an ALTER TABLE.

Alternative:
- Document based databases as MongoDB and
CouchDB, where new atributes are local to the
document. Also, having no schema helps

- Column based databases may be not the best choice
if column creation need restart/migrations
Extreme JOINs
Problem: Business rules modeled as tables. Table
inheritance (Product -> SubProduct_A). To find the
complete data for a user plan, one must issue gigantic
queries with lots of JOINs.

Alternative:
- Document storage, as MongoDB
- Denormalization
- Serialized objects
Your ORM ...
Problem: Your ORM issue full queries for dataset
iterations, your ORM maps and creates tables which
mimics your classes, even the inheritance, and the
performance is bad because the queries are huge, etc,
etc

Alternative:
Apart from denormalization and good old common
sense, ORMs are trying to bridge two things with
distinct impedance.

There is nothing to relational models which maps
cleanly to classes and objects. Not even the basic unit
which is the domain(set) of each column. Black Magic ?
Throttle Control
Problem: A request tracker to create a throttle control by IP
address, login, operation or any other event using a relational
database	
  

Ranging from an update … select to a lock/transaction block,
no relational database would be the best place to do that.

Alternative: use memcached, redis or any other DHT which has
expiration by creating a key as
THROTLE:<IP>:YYYYMMDDHH and increment it. At first
glance sounds the same but the expiration will take care of
cleaning up old entries. Also search time is the same as looking
for a key.
No silver bullet
- Consider alternatives	
  

- Think outside the norm	
  

- Denormalize	
  

- Simplify	
  
Cycle of changes - Product A
1. There was the database model	
  
2. Then, the cache was needed. Performance was no good.	
  
3. Cache key: query, value: resultset	
  
4. High or inexistent expiration time [w00t]	
  

(Now there's a turning point. Data didn't need to change often.
Denormalization was a given with cache)	
  

5. The cache needs to be warmed or the app wont work.	
  
6. Key/Value storage was a natural choice. No data on MySQL
anymore.
Cycle of changes - Product B
1. Postgres DB storing crawler results.	
  
2. There was a counter in each row, and updating this counter
   caused contention errors.	
  
3. Memcache for reads. Performance is better.	
  
4. First MongoDB test, no more deadlocks from counter
   update.	
  
5. Data model was simplified, the entire crawled doc was
   stored.	
  
Stuff to think about
Think if the data you use aren't denormalized (cached)	
  

Most of the anti-patterns contain signs that a non-relational
route (or at least a partial route) may help.	
  

Are you dependent on cache ? Does your application fails when
there is no cache ? Does it just slows down ?	
  

Are you ready to think more about your data ? 	
  

Think about the way to put and to get back your data from the
database (be it SQL or NoSQL).	
  
Extra - MongoDB and Redis
The next two slides are here to show what is like to use
MongoDB and Redis for the same task.	
  

There is more to managing your data than stuffing it inside a
database. You gotta plan ahead for searches and migrations.	
  

This example is about storing books and searching between
them. MongoDB makes it simpler, just liek using its query
language. Redis requires that you keep track of tags and ids to
use SET operations to recover which books you want.	
  

Check http://rediscookbook.org and http://
cookbook.mongodb.org/ for recipes on data handling.
MongoDB/Redis recap - Books
MongoDB	
                                                                    Redis	
  
{ 	
  
 'id': 1,  	
                                                                                                                                     	
  
 'title' : 'Diving into Python',       	
                                    SET book:1 {'title' : 'Diving into Python',
                                                                                                                	
  
                         	
  
 'author': 'Mark Pilgrim',                                                   'author': 'Mark Pilgrim'}
                                                                                                                                                         	
  
 'tags': ['python','programming', 'computing']                 	
            SET book:2 { 'title' : 'Programing Erlang',
                                                                                                                       	
  
}	
                                                                          'author': 'Joe Armstrong'}
                                                                             SET book:3 { 'title' : 'Programing in Haskell',                                    	
  
{	
  
                                                                             'author': 'Graham Hutton'}                    	
  
 'id':2, 	
  
 'title' : 'Programing Erlang',           	
                                                            	
  
 'author': 'Joe Armstrong',     	
                                           SADD tag:python 1
                                                                             SADD tag:erlang 2        	
  
 'tags': ['erlang','programming', 'computing',
                                                 	
                          SADD tag:haskell 3          	
  
'distributedcomputing', 'FP']
 	
                                                                          SADD tag:programming 1 2 3                             	
  
}                                                                            SADD tag computing 1 2 3                        	
  
{	
  
                                                                             SADD tag:distributedcomputing 2                               	
  
 'id':3, 	
                                                                  SADD tag:FP 2 3   	
  
 'title' : 'Programing in Haskell',                     	
  
 'author': 'Graham Hutton',        	
  
 'tags': ['haskell','programming', 'computing', 'FP']                 	
  
} 	
  
MongoDB/Redis recap - Books
MongoDB	
                                                          Redis	
  
Search tags for erlang or haskell:            	
  
                           	
                                      SINTER 'tag:erlang' 'tag:haskell'	
  
db.books.find({"tags":
     { $in: ['erlang', 'haskell']    	
                                    	
  
                                                                   0 results
   }    	
                                                                                                                  	
  
 	
  
})                                                                 SINTER 'tag:programming' 'tag:computing'
                                                                                   	
  
                                                                   3 results: 1, 2, 3
Search tags for erlang AND haskell (no results)             	
     SUNION 'tag:erlang' 'tag:haskell'   	
  
db.books.find({"tags":     	
                                      2 results: 2 and 3     	
  
     { $all: ['erlang', 'haskell']     	
                                                                     	
  
   }    	
                                                         SDIFF 'tag:programming' 'tag:haskell'
                                                                                                                     	
  
 	
  
})                                                                 2 results: 1 and 2 (haskell is excluded)

This search yields 3 results          	
  
db.books.find({"tags":     	
  
     { $all: ['programming', 'computing']            	
  
   }    	
  
 	
  
})
Message Queues
Decoupling db writes with Message Queues
Coupled comment
Uncoupled comment - producer
Uncoupled comment - consumer
Async HTML Scrapper
Fetch Page
1st parse

Fetch Page
             Message Queue
1st parse

                             Consumer

Fetch Page
1st parse

Fetch Page
1st parse
M/R	
  
Fetch Data
Map(Fun)

Fetch Data
              Message Queue
Map(Fun)

                              Reduce

Fetch Data
Map(Fun)

Fetch Data
Map(Fun)
M/R	
  –	
  Wordcount(Map)	
  
M/R	
  –	
  Wordcount(Reduce)	
  
Cache
Cache
HTML processing - no cache




http://github.com/gleicon/vuvuzelr/proxy_no_cache.rb
HTML processing - Cached




http://github.com/gleicon/vuvuzelr/proxy.rb
Conclusion
Thanks	
  
•    @gleicon	
  
•    hQp://www.7co.cc	
  
•    hQp://github.com/gleicon	
  
•    gleicon@gmail.com	
  

More Related Content

What's hot

Building large scalable mission critical business applications on the web
Building large scalable mission critical business applications on the webBuilding large scalable mission critical business applications on the web
Building large scalable mission critical business applications on the web
Maurice De Beijer [MVP]
 
Supporting large scale React applications
Supporting large scale React applicationsSupporting large scale React applications
Supporting large scale React applications
Maurice De Beijer [MVP]
 
"Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets
 "Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets "Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets
"Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets
Fwdays
 
Tulsa tech fest 2010 - web speed and scalability
Tulsa tech fest 2010  - web speed and scalabilityTulsa tech fest 2010  - web speed and scalability
Tulsa tech fest 2010 - web speed and scalability
Jason Ragsdale
 
Illuminate - Performance Analystics driven by Machine Learning
Illuminate - Performance Analystics driven by Machine LearningIlluminate - Performance Analystics driven by Machine Learning
Illuminate - Performance Analystics driven by Machine Learning
jClarity
 
Prometheus lightning talk (Devops Dublin March 2015)
Prometheus lightning talk (Devops Dublin March 2015)Prometheus lightning talk (Devops Dublin March 2015)
Prometheus lightning talk (Devops Dublin March 2015)
Brian Brazil
 
Message Queues & Offline Processing with PHP
Message Queues & Offline Processing with PHPMessage Queues & Offline Processing with PHP
Message Queues & Offline Processing with PHP
marcelesser
 
Store
StoreStore
Store
ESUG
 
HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612
HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612
HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612
Avenga Germany GmbH
 
Why Puppet Sucks - Rob Terhaar
Why Puppet Sucks - Rob TerhaarWhy Puppet Sucks - Rob Terhaar
Why Puppet Sucks - Rob Terhaar
Datadog
 
4th Lecture: JSP and such
4th Lecture:  JSP and such4th Lecture:  JSP and such
4th Lecture: JSP and such
Manolis Vavalis
 
Life in the fast lane. Full speed XPages
Life in the fast lane. Full speed XPagesLife in the fast lane. Full speed XPages
Life in the fast lane. Full speed XPages
Ulrich Krause
 
Life In The FastLane: Full Speed XPages
Life In The FastLane: Full Speed XPagesLife In The FastLane: Full Speed XPages
Life In The FastLane: Full Speed XPages
Ulrich Krause
 
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...
Brian Brazil
 
Why Wordnik went non-relational
Why Wordnik went non-relationalWhy Wordnik went non-relational
Why Wordnik went non-relational
Tony Tam
 
Low latency microservices in java QCon New York 2016
Low latency microservices in java   QCon New York 2016Low latency microservices in java   QCon New York 2016
Low latency microservices in java QCon New York 2016
Peter Lawrey
 
HTTP/2 Prioritization
HTTP/2 PrioritizationHTTP/2 Prioritization
HTTP/2 Prioritization
Patrick Meenan
 
OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...
OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...
OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...
NETWAYS
 

What's hot (18)

Building large scalable mission critical business applications on the web
Building large scalable mission critical business applications on the webBuilding large scalable mission critical business applications on the web
Building large scalable mission critical business applications on the web
 
Supporting large scale React applications
Supporting large scale React applicationsSupporting large scale React applications
Supporting large scale React applications
 
"Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets
 "Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets "Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets
"Fast / Resilient / .NET – What to Choose?" Serhiy Kalinets
 
Tulsa tech fest 2010 - web speed and scalability
Tulsa tech fest 2010  - web speed and scalabilityTulsa tech fest 2010  - web speed and scalability
Tulsa tech fest 2010 - web speed and scalability
 
Illuminate - Performance Analystics driven by Machine Learning
Illuminate - Performance Analystics driven by Machine LearningIlluminate - Performance Analystics driven by Machine Learning
Illuminate - Performance Analystics driven by Machine Learning
 
Prometheus lightning talk (Devops Dublin March 2015)
Prometheus lightning talk (Devops Dublin March 2015)Prometheus lightning talk (Devops Dublin March 2015)
Prometheus lightning talk (Devops Dublin March 2015)
 
Message Queues & Offline Processing with PHP
Message Queues & Offline Processing with PHPMessage Queues & Offline Processing with PHP
Message Queues & Offline Processing with PHP
 
Store
StoreStore
Store
 
HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612
HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612
HTTP/2 turns 3 years old // Web Performance Meetup wao.io 20180612
 
Why Puppet Sucks - Rob Terhaar
Why Puppet Sucks - Rob TerhaarWhy Puppet Sucks - Rob Terhaar
Why Puppet Sucks - Rob Terhaar
 
4th Lecture: JSP and such
4th Lecture:  JSP and such4th Lecture:  JSP and such
4th Lecture: JSP and such
 
Life in the fast lane. Full speed XPages
Life in the fast lane. Full speed XPagesLife in the fast lane. Full speed XPages
Life in the fast lane. Full speed XPages
 
Life In The FastLane: Full Speed XPages
Life In The FastLane: Full Speed XPagesLife In The FastLane: Full Speed XPages
Life In The FastLane: Full Speed XPages
 
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...
No C-QL (Or how I learned to stop worrying, and love eventual consistency) (N...
 
Why Wordnik went non-relational
Why Wordnik went non-relationalWhy Wordnik went non-relational
Why Wordnik went non-relational
 
Low latency microservices in java QCon New York 2016
Low latency microservices in java   QCon New York 2016Low latency microservices in java   QCon New York 2016
Low latency microservices in java QCon New York 2016
 
HTTP/2 Prioritization
HTTP/2 PrioritizationHTTP/2 Prioritization
HTTP/2 Prioritization
 
OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...
OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...
OSDC 2015: Tudor Golubenco | Application Performance Management with Packetbe...
 

Viewers also liked

RestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message QueueRestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message Queue
Gleicon Moraes
 
Por trás da infraestrutura do Cloud - Campus Party 2014
Por trás da infraestrutura do Cloud - Campus Party 2014Por trás da infraestrutura do Cloud - Campus Party 2014
Por trás da infraestrutura do Cloud - Campus Party 2014
Gleicon Moraes
 
QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...
QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...
QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...
Gleicon Moraes
 
7Masters CSS | SMACSS, por Rafael Lyra
7Masters CSS | SMACSS, por Rafael Lyra7Masters CSS | SMACSS, por Rafael Lyra
7Masters CSS | SMACSS, por Rafael Lyra
iMasters
 
Arquitetura emergente - sobre cultura devops
Arquitetura emergente - sobre cultura devopsArquitetura emergente - sobre cultura devops
Arquitetura emergente - sobre cultura devops
Gleicon Moraes
 
The Redis API Akbar Ahmed, DynomiteDB
The Redis API Akbar Ahmed, DynomiteDBThe Redis API Akbar Ahmed, DynomiteDB
The Redis API Akbar Ahmed, DynomiteDB
Redis Labs
 
Locaweb cloud and sdn
Locaweb cloud and sdnLocaweb cloud and sdn
Locaweb cloud and sdn
Gleicon Moraes
 
Redis as a message queue
Redis as a message queueRedis as a message queue
Redis as a message queue
Brandon Lamb
 
Celery: The Distributed Task Queue
Celery: The Distributed Task QueueCelery: The Distributed Task Queue
Celery: The Distributed Task Queue
Richard Leland
 
DevOps & PostgreSQL - Provisionamento Ágil
DevOps & PostgreSQL - Provisionamento ÁgilDevOps & PostgreSQL - Provisionamento Ágil
DevOps & PostgreSQL - Provisionamento Ágil
instructbr
 
DNAD 2015 - Como a arquitetura emergente de sua aplicação pode jogar contra ...
DNAD 2015  - Como a arquitetura emergente de sua aplicação pode jogar contra ...DNAD 2015  - Como a arquitetura emergente de sua aplicação pode jogar contra ...
DNAD 2015 - Como a arquitetura emergente de sua aplicação pode jogar contra ...
Gleicon Moraes
 
Apache Spark - Intro to Large-scale recommendations with Apache Spark and Python
Apache Spark - Intro to Large-scale recommendations with Apache Spark and PythonApache Spark - Intro to Large-scale recommendations with Apache Spark and Python
Apache Spark - Intro to Large-scale recommendations with Apache Spark and Python
Christian Perone
 
Word Embeddings - Introduction
Word Embeddings - IntroductionWord Embeddings - Introduction
Word Embeddings - Introduction
Christian Perone
 
SQLアンチパターン読書会 「スパゲッティクエリ」
SQLアンチパターン読書会 「スパゲッティクエリ」SQLアンチパターン読書会 「スパゲッティクエリ」
SQLアンチパターン読書会 「スパゲッティクエリ」
makopi 23
 
Introduction to Thrift
Introduction to ThriftIntroduction to Thrift
Introduction to Thrift
Dvir Volk
 
API Gateway report
API Gateway reportAPI Gateway report
API Gateway report
Gleicon Moraes
 
Kicking ass with redis
Kicking ass with redisKicking ass with redis
Kicking ass with redis
Dvir Volk
 
Redis in Practice
Redis in PracticeRedis in Practice
Redis in Practice
Noah Davis
 
10分でわかる無料になったXamarin
10分でわかる無料になったXamarin10分でわかる無料になったXamarin
10分でわかる無料になったXamarin
Yoshito Tabuchi
 
Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2
Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2
Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2
Yoshito Tabuchi
 

Viewers also liked (20)

RestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message QueueRestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message Queue
 
Por trás da infraestrutura do Cloud - Campus Party 2014
Por trás da infraestrutura do Cloud - Campus Party 2014Por trás da infraestrutura do Cloud - Campus Party 2014
Por trás da infraestrutura do Cloud - Campus Party 2014
 
QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...
QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...
QCon SP 2015 - Advogados do diabo: como a arquitetura emergente de sua aplica...
 
7Masters CSS | SMACSS, por Rafael Lyra
7Masters CSS | SMACSS, por Rafael Lyra7Masters CSS | SMACSS, por Rafael Lyra
7Masters CSS | SMACSS, por Rafael Lyra
 
Arquitetura emergente - sobre cultura devops
Arquitetura emergente - sobre cultura devopsArquitetura emergente - sobre cultura devops
Arquitetura emergente - sobre cultura devops
 
The Redis API Akbar Ahmed, DynomiteDB
The Redis API Akbar Ahmed, DynomiteDBThe Redis API Akbar Ahmed, DynomiteDB
The Redis API Akbar Ahmed, DynomiteDB
 
Locaweb cloud and sdn
Locaweb cloud and sdnLocaweb cloud and sdn
Locaweb cloud and sdn
 
Redis as a message queue
Redis as a message queueRedis as a message queue
Redis as a message queue
 
Celery: The Distributed Task Queue
Celery: The Distributed Task QueueCelery: The Distributed Task Queue
Celery: The Distributed Task Queue
 
DevOps & PostgreSQL - Provisionamento Ágil
DevOps & PostgreSQL - Provisionamento ÁgilDevOps & PostgreSQL - Provisionamento Ágil
DevOps & PostgreSQL - Provisionamento Ágil
 
DNAD 2015 - Como a arquitetura emergente de sua aplicação pode jogar contra ...
DNAD 2015  - Como a arquitetura emergente de sua aplicação pode jogar contra ...DNAD 2015  - Como a arquitetura emergente de sua aplicação pode jogar contra ...
DNAD 2015 - Como a arquitetura emergente de sua aplicação pode jogar contra ...
 
Apache Spark - Intro to Large-scale recommendations with Apache Spark and Python
Apache Spark - Intro to Large-scale recommendations with Apache Spark and PythonApache Spark - Intro to Large-scale recommendations with Apache Spark and Python
Apache Spark - Intro to Large-scale recommendations with Apache Spark and Python
 
Word Embeddings - Introduction
Word Embeddings - IntroductionWord Embeddings - Introduction
Word Embeddings - Introduction
 
SQLアンチパターン読書会 「スパゲッティクエリ」
SQLアンチパターン読書会 「スパゲッティクエリ」SQLアンチパターン読書会 「スパゲッティクエリ」
SQLアンチパターン読書会 「スパゲッティクエリ」
 
Introduction to Thrift
Introduction to ThriftIntroduction to Thrift
Introduction to Thrift
 
API Gateway report
API Gateway reportAPI Gateway report
API Gateway report
 
Kicking ass with redis
Kicking ass with redisKicking ass with redis
Kicking ass with redis
 
Redis in Practice
Redis in PracticeRedis in Practice
Redis in Practice
 
10分でわかる無料になったXamarin
10分でわかる無料になったXamarin10分でわかる無料になったXamarin
10分でわかる無料になったXamarin
 
Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2
Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2
Xamarin 概要 @ 「Xamarin」って何? Wエバンジェリストによる特濃「Xamarin」勉強会 Rev2
 

Similar to Architecture by Accident

Front Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesFront Range PHP NoSQL Databases
Front Range PHP NoSQL Databases
Jon Meredith
 
Architectural anti-patterns for data handling
Architectural anti-patterns for data handlingArchitectural anti-patterns for data handling
Architectural anti-patterns for data handling
Gleicon Moraes
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skies
shnkr_rmchndrn
 
Architectural anti patterns_for_data_handling
Architectural anti patterns_for_data_handlingArchitectural anti patterns_for_data_handling
Architectural anti patterns_for_data_handling
Gleicon Moraes
 
MongoDB
MongoDBMongoDB
MongoDB
fsbrooke
 
Framing the Argument: How to Scale Faster with NoSQL
Framing the Argument: How to Scale Faster with NoSQLFraming the Argument: How to Scale Faster with NoSQL
Framing the Argument: How to Scale Faster with NoSQL
Inside Analysis
 
Using Cassandra with your Web Application
Using Cassandra with your Web ApplicationUsing Cassandra with your Web Application
Using Cassandra with your Web Application
supertom
 
Microsoft Openness Mongo DB
Microsoft Openness Mongo DBMicrosoft Openness Mongo DB
Microsoft Openness Mongo DB
Heriyadi Janwar
 
Web20expo Scalable Web Arch
Web20expo Scalable Web ArchWeb20expo Scalable Web Arch
Web20expo Scalable Web Arch
guest18a0f1
 
Web20expo Scalable Web Arch
Web20expo Scalable Web ArchWeb20expo Scalable Web Arch
Web20expo Scalable Web Arch
mclee
 
Web20expo Scalable Web Arch
Web20expo Scalable Web ArchWeb20expo Scalable Web Arch
Web20expo Scalable Web Arch
royans
 
Agile data lake? An oxymoron?
Agile data lake? An oxymoron?Agile data lake? An oxymoron?
Agile data lake? An oxymoron?
samthemonad
 
NoSql Databases
NoSql DatabasesNoSql Databases
NoSql Databases
Nimat Khattak
 
Hardware Provisioning
Hardware ProvisioningHardware Provisioning
Hardware Provisioning
MongoDB
 
MPTStore: A Fast, Scalable, and Stable Resource Index
MPTStore: A Fast, Scalable, and Stable Resource IndexMPTStore: A Fast, Scalable, and Stable Resource Index
MPTStore: A Fast, Scalable, and Stable Resource Index
Chris Wilper
 
How and when to use NoSQL
How and when to use NoSQLHow and when to use NoSQL
How and when to use NoSQL
Amazon Web Services
 
SQL or NoSQL, that is the question!
SQL or NoSQL, that is the question!SQL or NoSQL, that is the question!
SQL or NoSQL, that is the question!
Andraz Tori
 
Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...
Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...
Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...
Lucidworks
 
A Practical Multi-Tenant Cluster
A Practical Multi-Tenant ClusterA Practical Multi-Tenant Cluster
A Practical Multi-Tenant Cluster
Command Prompt., Inc
 
NoSQL and MongoDB
NoSQL and MongoDBNoSQL and MongoDB
NoSQL and MongoDB
Rajesh Menon
 

Similar to Architecture by Accident (20)

Front Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesFront Range PHP NoSQL Databases
Front Range PHP NoSQL Databases
 
Architectural anti-patterns for data handling
Architectural anti-patterns for data handlingArchitectural anti-patterns for data handling
Architectural anti-patterns for data handling
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skies
 
Architectural anti patterns_for_data_handling
Architectural anti patterns_for_data_handlingArchitectural anti patterns_for_data_handling
Architectural anti patterns_for_data_handling
 
MongoDB
MongoDBMongoDB
MongoDB
 
Framing the Argument: How to Scale Faster with NoSQL
Framing the Argument: How to Scale Faster with NoSQLFraming the Argument: How to Scale Faster with NoSQL
Framing the Argument: How to Scale Faster with NoSQL
 
Using Cassandra with your Web Application
Using Cassandra with your Web ApplicationUsing Cassandra with your Web Application
Using Cassandra with your Web Application
 
Microsoft Openness Mongo DB
Microsoft Openness Mongo DBMicrosoft Openness Mongo DB
Microsoft Openness Mongo DB
 
Web20expo Scalable Web Arch
Web20expo Scalable Web ArchWeb20expo Scalable Web Arch
Web20expo Scalable Web Arch
 
Web20expo Scalable Web Arch
Web20expo Scalable Web ArchWeb20expo Scalable Web Arch
Web20expo Scalable Web Arch
 
Web20expo Scalable Web Arch
Web20expo Scalable Web ArchWeb20expo Scalable Web Arch
Web20expo Scalable Web Arch
 
Agile data lake? An oxymoron?
Agile data lake? An oxymoron?Agile data lake? An oxymoron?
Agile data lake? An oxymoron?
 
NoSql Databases
NoSql DatabasesNoSql Databases
NoSql Databases
 
Hardware Provisioning
Hardware ProvisioningHardware Provisioning
Hardware Provisioning
 
MPTStore: A Fast, Scalable, and Stable Resource Index
MPTStore: A Fast, Scalable, and Stable Resource IndexMPTStore: A Fast, Scalable, and Stable Resource Index
MPTStore: A Fast, Scalable, and Stable Resource Index
 
How and when to use NoSQL
How and when to use NoSQLHow and when to use NoSQL
How and when to use NoSQL
 
SQL or NoSQL, that is the question!
SQL or NoSQL, that is the question!SQL or NoSQL, that is the question!
SQL or NoSQL, that is the question!
 
Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...
Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...
Near Real Time Indexing Kafka Messages into Apache Blur: Presented by Dibyend...
 
A Practical Multi-Tenant Cluster
A Practical Multi-Tenant ClusterA Practical Multi-Tenant Cluster
A Practical Multi-Tenant Cluster
 
NoSQL and MongoDB
NoSQL and MongoDBNoSQL and MongoDB
NoSQL and MongoDB
 

More from Gleicon Moraes

Como arquiteturas de dados quebram
Como arquiteturas de dados quebramComo arquiteturas de dados quebram
Como arquiteturas de dados quebram
Gleicon Moraes
 
A closer look to locaweb IaaS
A closer look to locaweb IaaSA closer look to locaweb IaaS
A closer look to locaweb IaaS
Gleicon Moraes
 
Semi Automatic Sentiment Analysis
Semi Automatic Sentiment AnalysisSemi Automatic Sentiment Analysis
Semi Automatic Sentiment Analysis
Gleicon Moraes
 
L'esprit de l'escalier
L'esprit de l'escalierL'esprit de l'escalier
L'esprit de l'escalier
Gleicon Moraes
 
Architectural Anti Patterns - Notes on Data Distribution and Handling Failures
Architectural Anti Patterns - Notes on Data Distribution and Handling FailuresArchitectural Anti Patterns - Notes on Data Distribution and Handling Failures
Architectural Anti Patterns - Notes on Data Distribution and Handling Failures
Gleicon Moraes
 
Patterns of fail
Patterns of failPatterns of fail
Patterns of fail
Gleicon Moraes
 
NoSQL and SQL Anti Patterns
NoSQL and SQL Anti PatternsNoSQL and SQL Anti Patterns
NoSQL and SQL Anti Patterns
Gleicon Moraes
 
Redis
RedisRedis
NoSql Introduction
NoSql IntroductionNoSql Introduction
NoSql Introduction
Gleicon Moraes
 

More from Gleicon Moraes (9)

Como arquiteturas de dados quebram
Como arquiteturas de dados quebramComo arquiteturas de dados quebram
Como arquiteturas de dados quebram
 
A closer look to locaweb IaaS
A closer look to locaweb IaaSA closer look to locaweb IaaS
A closer look to locaweb IaaS
 
Semi Automatic Sentiment Analysis
Semi Automatic Sentiment AnalysisSemi Automatic Sentiment Analysis
Semi Automatic Sentiment Analysis
 
L'esprit de l'escalier
L'esprit de l'escalierL'esprit de l'escalier
L'esprit de l'escalier
 
Architectural Anti Patterns - Notes on Data Distribution and Handling Failures
Architectural Anti Patterns - Notes on Data Distribution and Handling FailuresArchitectural Anti Patterns - Notes on Data Distribution and Handling Failures
Architectural Anti Patterns - Notes on Data Distribution and Handling Failures
 
Patterns of fail
Patterns of failPatterns of fail
Patterns of fail
 
NoSQL and SQL Anti Patterns
NoSQL and SQL Anti PatternsNoSQL and SQL Anti Patterns
NoSQL and SQL Anti Patterns
 
Redis
RedisRedis
Redis
 
NoSql Introduction
NoSql IntroductionNoSql Introduction
NoSql Introduction
 

Recently uploaded

Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 

Recently uploaded (20)

Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 

Architecture by Accident

  • 1. Architecture by accident Gleicon  Moraes  
  • 3. Agenda •  Architecture for data - even if you don’t want it •  Databases •  Message Queues •  Cache
  • 4. Architecture “Everyone  has  a  plan  un4l  they  get  punched  in   the  mouth”  –  Mike  Tyson    
  • 5. Even if you dont want it ... •  There is an innate architecture on everything •  You may end up with more data than you had planned to •  You may get away from your quick and dirty CRUD •  You probably are querying more than one Database •  At some point you laugh when your boss asks you about 'Integrating Systems' •  Code turns into legacy - and so architectures •  'Scattered' is not the same that 'Distributed'
  • 6. It  usually  starts  like  this   App Server Database
  • 7. then App Servers Database
  • 8. it App Servers Master DB Slave DB
  • 9. goes App Servers Master DB Slave DB Cache
  • 10. like App Servers Master DB Slave DB Cache Indexing Service
  • 11. this App Servers Master DB Slave DB Cache Indexing Service API Servers
  • 12. and App Servers Master DB Load Balancer/Reverse Proxy Slave DB Cache Indexing Service API Servers
  • 13. beyond App Servers Master DB Load Balancer/Reverse Proxy Slave DB Cache Indexing Service API Servers Auth Service
  • 14.
  • 15. Problem is... An architect s first work is apt to be spare and clean. He knows he doesn t know what he s doing, so he does it carefully and with great restraint.   As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used next time. Sooner or later the first system is finished, and the architect, with firm confidence and a demonstrated mastery of that class of systems, is ready to build a second system.   This second is the most dangerous system a man ever designs. When he does his third and later ones, his prior experiences will confirm each other as to the general characteristics of such systems, and their differences will identify those parts of his experience that are particular and not generalizable.   The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one. The result, as Ovid says, is a big pile.   — Frederick P. Brooks, Jr. The Mythical Man-Month
  • 17. Databases   •  Not  an  off-­‐the-­‐shelf  architectural  duct  tape   •  Not  only  rela4onal,  other  paradigms   •  Usually  the  last  place  sought  for  op4miza4on   •  Usually  the  first  place  to  accomodate  last  minute   changes   •  Good  ideas  to  try  out:  Sharding  and   Denormaliza4on   •  Some  of  your  problems  may  require  something   other  than  a  Rela4onal  Database  
  • 18.
  • 19. Relevant RDBMS Anti-Patterns –  Dynamic table creation –  Table as cache –  Table as queue –  Table as log file –  Distributed Global Locking –  Stoned Procedures –  Row Alignment –  Extreme JOINs –  Your ORM issue full queries for Dataset iterations –  Throttle Control
  • 20. Dynamic table creation Problem: To avoid huge tables, "dynamic schema” is created. For example, lets think about a document management company, which is adding new facilities over the country. For each storage facility, a new table is created: item_id - row - column - stuff 1 - 10 - 20 - cat food 2 - 12 - 32 - trout Side Effect: "dynamic queries", which will probably query a "central storage" table and issue a huge join to check if you have enough cat food over the country. It’s different from Sharding. Alternative: - Document storage, modeling a facility as a document -  Key/Value, modeling each facility as a SET -  Sharding properly
  • 21. Table as cache Problem: Complex queries demand that a result be stored in a separated table, so it can be queried quickly. Worst than views Alternative: - Really ? - Memcached - Redis + AOF + EXPIRE - Denormalization
  • 22. Table as queue Problem: A table which holds messages to be completed. Worse, they must be sorted by date. Alternative: - RestMQ, Resque - Any other message broker - Redis (LISTS - LPUSH + RPOP) - Use the right tool
  • 23. Table as log file Problem: A table in which data gets written as a log file. From time to time it needs to be purged. Truncating this table once a day usually is the first task assigned to trainee DBAs. Alternative: - MongoDB capped collection - Redis, and a RRD pattern -  RIAK
  • 24. Distributed Global Locking Problem: Someone learns java and synchronize. A bit later genius thinks that a distributed synchronize would be awesome. The proper place to do that would be the database of course. Start with a reference counter in a table and end up with this: > select COALESCE(GET_LOCK('my_lock',0 ),0 ) Plain and simple, you might find it embedded in a magic class called DistributedSynchronize or ClusterSemaphore. Locks, transactions and reference counters (which may act as soft locks) doesn't belongs to the database.
  • 25. Stoned procedures Problem: Stored procedures hold most of your applications logic. Also, some triggers are used to - well - trigger important data events. SP and triggers has the magic property of vanishing of our mind instantly, being impossible to keep versioned. Alternative: - Careful so you don’t use map/reduce as stoned procedures. - Use your preferred language for business stuff, and let event handling to pub/sub or message queues.
  • 26. Row Alignment Problem: Extra rows are created but not used, just in case. Usually they are named as a1, a2, a3, a4 and called padding. There's good will behind that, specially when version 1 of the software needed an extra column in a 150M lines database and it took 2 days to run an ALTER TABLE. Alternative: - Document based databases as MongoDB and CouchDB, where new atributes are local to the document. Also, having no schema helps - Column based databases may be not the best choice if column creation need restart/migrations
  • 27. Extreme JOINs Problem: Business rules modeled as tables. Table inheritance (Product -> SubProduct_A). To find the complete data for a user plan, one must issue gigantic queries with lots of JOINs. Alternative: - Document storage, as MongoDB - Denormalization - Serialized objects
  • 28. Your ORM ... Problem: Your ORM issue full queries for dataset iterations, your ORM maps and creates tables which mimics your classes, even the inheritance, and the performance is bad because the queries are huge, etc, etc Alternative: Apart from denormalization and good old common sense, ORMs are trying to bridge two things with distinct impedance. There is nothing to relational models which maps cleanly to classes and objects. Not even the basic unit which is the domain(set) of each column. Black Magic ?
  • 29. Throttle Control Problem: A request tracker to create a throttle control by IP address, login, operation or any other event using a relational database   Ranging from an update … select to a lock/transaction block, no relational database would be the best place to do that. Alternative: use memcached, redis or any other DHT which has expiration by creating a key as THROTLE:<IP>:YYYYMMDDHH and increment it. At first glance sounds the same but the expiration will take care of cleaning up old entries. Also search time is the same as looking for a key.
  • 30. No silver bullet - Consider alternatives   - Think outside the norm   - Denormalize   - Simplify  
  • 31. Cycle of changes - Product A 1. There was the database model   2. Then, the cache was needed. Performance was no good.   3. Cache key: query, value: resultset   4. High or inexistent expiration time [w00t]   (Now there's a turning point. Data didn't need to change often. Denormalization was a given with cache)   5. The cache needs to be warmed or the app wont work.   6. Key/Value storage was a natural choice. No data on MySQL anymore.
  • 32. Cycle of changes - Product B 1. Postgres DB storing crawler results.   2. There was a counter in each row, and updating this counter caused contention errors.   3. Memcache for reads. Performance is better.   4. First MongoDB test, no more deadlocks from counter update.   5. Data model was simplified, the entire crawled doc was stored.  
  • 33. Stuff to think about Think if the data you use aren't denormalized (cached)   Most of the anti-patterns contain signs that a non-relational route (or at least a partial route) may help.   Are you dependent on cache ? Does your application fails when there is no cache ? Does it just slows down ?   Are you ready to think more about your data ?   Think about the way to put and to get back your data from the database (be it SQL or NoSQL).  
  • 34. Extra - MongoDB and Redis The next two slides are here to show what is like to use MongoDB and Redis for the same task.   There is more to managing your data than stuffing it inside a database. You gotta plan ahead for searches and migrations.   This example is about storing books and searching between them. MongoDB makes it simpler, just liek using its query language. Redis requires that you keep track of tags and ids to use SET operations to recover which books you want.   Check http://rediscookbook.org and http:// cookbook.mongodb.org/ for recipes on data handling.
  • 35. MongoDB/Redis recap - Books MongoDB   Redis   {   'id': 1,     'title' : 'Diving into Python',   SET book:1 {'title' : 'Diving into Python',     'author': 'Mark Pilgrim', 'author': 'Mark Pilgrim'}   'tags': ['python','programming', 'computing']   SET book:2 { 'title' : 'Programing Erlang',   }   'author': 'Joe Armstrong'} SET book:3 { 'title' : 'Programing in Haskell',   {   'author': 'Graham Hutton'}   'id':2,   'title' : 'Programing Erlang',     'author': 'Joe Armstrong',   SADD tag:python 1 SADD tag:erlang 2   'tags': ['erlang','programming', 'computing',   SADD tag:haskell 3   'distributedcomputing', 'FP']   SADD tag:programming 1 2 3   } SADD tag computing 1 2 3   {   SADD tag:distributedcomputing 2   'id':3,   SADD tag:FP 2 3   'title' : 'Programing in Haskell',   'author': 'Graham Hutton',   'tags': ['haskell','programming', 'computing', 'FP']   }  
  • 36. MongoDB/Redis recap - Books MongoDB   Redis   Search tags for erlang or haskell:     SINTER 'tag:erlang' 'tag:haskell'   db.books.find({"tags": { $in: ['erlang', 'haskell']     0 results }       }) SINTER 'tag:programming' 'tag:computing'   3 results: 1, 2, 3 Search tags for erlang AND haskell (no results)   SUNION 'tag:erlang' 'tag:haskell'   db.books.find({"tags":   2 results: 2 and 3   { $all: ['erlang', 'haskell']     }   SDIFF 'tag:programming' 'tag:haskell'     }) 2 results: 1 and 2 (haskell is excluded) This search yields 3 results   db.books.find({"tags":   { $all: ['programming', 'computing']   }     })
  • 38. Decoupling db writes with Message Queues
  • 42. Async HTML Scrapper Fetch Page 1st parse Fetch Page Message Queue 1st parse Consumer Fetch Page 1st parse Fetch Page 1st parse
  • 43. M/R   Fetch Data Map(Fun) Fetch Data Message Queue Map(Fun) Reduce Fetch Data Map(Fun) Fetch Data Map(Fun)
  • 46.
  • 47. Cache
  • 48. Cache
  • 49. HTML processing - no cache http://github.com/gleicon/vuvuzelr/proxy_no_cache.rb
  • 50. HTML processing - Cached http://github.com/gleicon/vuvuzelr/proxy.rb
  • 51.
  • 53. Thanks   •  @gleicon   •  hQp://www.7co.cc   •  hQp://github.com/gleicon   •  gleicon@gmail.com