Your SlideShare is downloading. ×
  • Like
noSQL @ QCon SP
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

noSQL @ QCon SP

  • 3,719 views
Published

 

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,719
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
105
Comments
0
Likes
10

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. noSQL quarta-feira, 8 de setembro de 2010
  • 2. . hype quarta-feira, 8 de setembro de 2010
  • 3. história... quarta-feira, 8 de setembro de 2010
  • 4. modelos • Hierarchical (IMS): late 1960’s and 1970’s • Directed graph (CODASYL): 1970’s • Relational: 1970’s and early 1980’s • Entity-Relationship: 1970’s • Extended Relational: 1980’s • Semantic: late 1970’s and 1980’s • Object-oriented: late 1980’s and early 1990’s • Object-relational: late 1980’s and early 1990’s • Semi-structured (XML): late 1990’s to late 2000’s • The next big thing: ??? ref: What Goes Around Comes Around por Michael Stonebraker e Joey Hellerstein quarta-feira, 8 de setembro de 2010
  • 5. next big thing? quarta-feira, 8 de setembro de 2010
  • 6. definição... quarta-feira, 8 de setembro de 2010
  • 7. abaixo ao banco de dados relacional! quarta-feira, 8 de setembro de 2010
  • 8. abaixo ao banco de dados relacional! como bala de prata! quarta-feira, 8 de setembro de 2010
  • 9. momento histórico... quarta-feira, 8 de setembro de 2010
  • 10. quarta-feira, 8 de setembro de 2010
  • 11. resolver problemas específicos quarta-feira, 8 de setembro de 2010
  • 12. quais problemas? quarta-feira, 8 de setembro de 2010
  • 13. Architectural Anti Patterns Notes on Data Distribution and Handling Failures quarta-feira, 8 de setembro de 2010
  • 14. Required Listening: Frank Zappa - One size fits all quarta-feira, 8 de setembro de 2010
  • 15. Anti Patterns • Evolution from SQL Anti Patterns (NoSQL:br May 2010) • More than just RDBMS • Large volumes of data • Distribution • Architecture • Research on other tools • Message Queues, DHT, Job Schedulers, NoSQL • Indexing, Map/Reduce quarta-feira, 8 de setembro de 2010
  • 16. RDBMS Anti Patterns Not all things fit on a relational database, single ou distributed • The eternal table-as-a-tree • Dynamic table creation • Table as cache • Table as queue • Table as log file • Stoned Procedures • Row Alignment • Extreme JOINs • Your scheme must be printed in an A3 sheet. • Your ORM issue full queries for Dataset iterations quarta-feira, 8 de setembro de 2010
  • 17. Doing it wrong, Junior ! quarta-feira, 8 de setembro de 2010
  • 18. The eternal tree Problem: Most threaded discussion example uses something like a table which contains all threads and answers, relating to each other by an id. Usually the developer will come up with his own binary-tree version to manage this mess. id - parent_id -author - text 1 - 0 - gleicon - hello world 2 - 1 - elvis - shout ! Alternative: Document storage: { thread_id:1, title: 'the meeting', author: 'gleicon', replies:[ { 'author': elvis, text:'shout', replies:[{...}] } ] } quarta-feira, 8 de setembro de 2010
  • 19. Dynamic table creation Problem: To avoid huge tables, one must come with a "dynamic schema". For example, lets think about a document management company, which is adding new facilities over the country. For each storage facility, a new table is created: item_id - row - column - stuff 1 - 10 - 20 - cat food 2 - 12 - 32 - trout Now you have to come up with "dynamic queries", which will probably query a "central storage" table and issue a huge join to check if you have enough cat food over the country. Alternatives: - Document storage, modeling a facility as a document - Key/Value, modeling each facility as a SET quarta-feira, 8 de setembro de 2010
  • 20. Table as cache Problem: Complex queries demand that a result be stored in a separated table, so it can be queried quickly. Worst than views Alternatives: - Really ? - Memcached - Redis + AOF + EXPIRE - De-normalization quarta-feira, 8 de setembro de 2010
  • 21. Table as queue Problem: A table which holds messages to be completed. Worse, they must be ordered by time of creation. Corolary: Job Scheduler table Alternatives: - RestMQ, Resque - Any other message broker - Redis (LISTS - LPUSH + RPOP) - Use the right tool quarta-feira, 8 de setembro de 2010
  • 22. Table as log file Problem: A table in which data gets written as a log file. From time to time it needs to be purged. Truncating this table once a day usually is the first task assigned to new DBAs. Alternative: - MongoDB capped collection - Redis, and RRD pattern - RIAK quarta-feira, 8 de setembro de 2010
  • 23. Stoned procedures Problem: Stored procedures hold most of your applications logic. Also, some triggers are used to - well - trigger important data events. SP and triggers has the magic property of vanishing of our memories and being impossible to keep versioned. Alternative: - Now be careful so you dont use map/reduce as modern stoned procedures. Unfit for real time search/processing - Use your preferred language for business stuff, and let event handling to pub/sub or message queues. quarta-feira, 8 de setembro de 2010
  • 24. Row Alignment Problem: Extra rows are created but not used, just in case. Usually they are named as a1, a2, a3, a4 and called padding. There's good will behind that, specially when version 1 of the software needed an extra column in a 150M lines database and it took 2 days to run an ALTER TABLE. But that's no excuse. Alternative: - Quit being cheap. Quit feeling 'hacker' about padding - Document based databases as MongoDB and CouchDB, has no schema. New atributes are local to the document and can be added easily. quarta-feira, 8 de setembro de 2010
  • 25. Extreme JOINs Problem: Business stuff modeled as tables. Table inheritance (Product -> SubProduct_A). To find the complete data for a user plan, one must issue gigantic queries with lots of JOINs. Alternative: - Document storage, as MongoDB might help having important information together. - De-normalization - Serialized objects quarta-feira, 8 de setembro de 2010
  • 26. Your scheme fits in an A3 sheet Problem: Huge data schemes are difficult to manage. Extreme specialization creates tables which converges to key/value model. The normal form get priority over common sense. Product_A Product_B id - desc id - desc Alternatives: - De-normalization - Another scheme ? - Document store for flattening model - Key/Value - See 'Extreme JOINs' quarta-feira, 8 de setembro de 2010
  • 27. Your ORM ... Problem: Your ORM issue full queries for dataset iterations, your ORM maps and creates tables which mimics your classes, even the inheritance, and the performance is bad because the queries are huge, etc, etc Alternative: - Apart from denormalization and good old common sense, ORMs are trying to bridge two things with distinct impedance. - There is nothing to relational models which maps cleanly to classes and objects. Not even the basic unit which is the domain(set) of each column. Black Magic ? quarta-feira, 8 de setembro de 2010
  • 28. No silver bullet - Think about data handling and your system architecture - Think outside the norm - De-normalize - Simplify - Know stuff (Message queues, NoSQL, DHT) quarta-feira, 8 de setembro de 2010
  • 29. Cycle of changes - Product A 1.There was the database model 2.Then, the cache was needed. Performance was no good. 3.Cache key: query, value: resultset 4.High or inexistent expiration time [w00t] (Now there's a turning point. Data didn't need to change often. Denormalization was a given with cache) 5. The cache needs to be warmed or the app wont work. 6. Key/Value storage was a natural choice. No data on MySQL anymore. quarta-feira, 8 de setembro de 2010
  • 30. Cycle of changes - Product B 1.Postgres DB storing crawler results. 2.There was a counter in each row, and updating this counter caused contention errors. 3.Memcache for reads. Performance is better. 4.First MongoDB test, no more deadlocks from counter update. 5.Data model was simplified, the entire crawled doc was stored. quarta-feira, 8 de setembro de 2010
  • 31. Stuff to think about Think if the data you use aren't de-normalized somewhere (cached) Most of the anti-patterns signals that there are architectural issues instead of only database issues. The NoSQL route (or at least a partial NoSQL route) may simplify it. Are you dependent on cache ? Does your application fails when there is no cache ? Does it just slows down ? Think about the way to put and to get back your data from the database (be it SQL or NoSQL). quarta-feira, 8 de setembro de 2010
  • 32. arquitetura quarta-feira, 8 de setembro de 2010
  • 33. armazenamento de dados NÃO tem sido [a muito tempo] considerado parte de arquitetura quarta-feira, 8 de setembro de 2010
  • 34. cada escolha uma renúncia quarta-feira, 8 de setembro de 2010
  • 35. padrões quarta-feira, 8 de setembro de 2010
  • 36. how-to quarta-feira, 8 de setembro de 2010
  • 37. quarta-feira, 8 de setembro de 2010
  • 38. acid quarta-feira, 8 de setembro de 2010
  • 39. quarta-feira, 8 de setembro de 2010 (
  • 40. existe nosql acid quarta-feira, 8 de setembro de 2010
  • 41. quarta-feira, 8 de setembro de 2010 )
  • 42. CAP ref: The CAP Theorem por Seth Gilbert & Nancy Lynch quarta-feira, 8 de setembro de 2010
  • 43. C onsistency A vailability P artition Tolerance quarta-feira, 8 de setembro de 2010
  • 44. quarta-feira, 8 de setembro de 2010
  • 45. BASE ref: BASE: an Acid Alternative por Dan Pritchett quarta-feira, 8 de setembro de 2010
  • 46. B asically A vailable S oft State E eventually Consistent quarta-feira, 8 de setembro de 2010
  • 47. Eventually Consistency ref: Eventually Consistent por Werner Vogels quarta-feira, 8 de setembro de 2010
  • 48. eventual em inglês: irá ocorrer em algum momento eventual em português: pode ou não ocorrer quarta-feira, 8 de setembro de 2010
  • 49. Consitência em Momento Indeterminado @mdediana quarta-feira, 8 de setembro de 2010
  • 50. consistência W+R > N quarta-feira, 8 de setembro de 2010
  • 51. durabilidade ref: The End of an Architectural Era por Michael Stonebraker & al. quarta-feira, 8 de setembro de 2010
  • 52. ainda tem... ★ latência ★ performance ★ particionamento ★ distribuição ★ replicação quarta-feira, 8 de setembro de 2010
  • 53. lembre-se vc não está criando uma solução de escala intergaláctica com tolerância a falhas aleatórias entre datacenters espalhados em diversas localizações geográficas e outras dimensões quarta-feira, 8 de setembro de 2010
  • 54. sacou a importância da arquitetura? quarta-feira, 8 de setembro de 2010
  • 55. com tantas definições... com tantos conceitos... com tantos tradeoffs... com tantos.... quarta-feira, 8 de setembro de 2010
  • 56. como o nosql se tornou tão sexy e popular? quarta-feira, 8 de setembro de 2010
  • 57. apesar de tudo.... quarta-feira, 8 de setembro de 2010 é fácil usar!
  • 58. persitência poliglota quarta-feira, 8 de setembro de 2010
  • 59. Perguntas? quarta-feira, 8 de setembro de 2010
  • 60. Obrigado github.com/porcelli github.com/gleicon linkedin.com/in/alexandreporcelli linkedin.com/in/gleicon @porcelli @gleicon porcelli.com.br zenmachine.wordpress.com quarta-feira, 8 de setembro de 2010