SlideShare a Scribd company logo
1 of 11
Download to read offline
The future is:

                               NoSQL Databases

                              Polyglot Persistence
                                                                         Martin Fowler
   a note on the future of data storage in
   the enterprise, written primarily for those
   involved in the management of
   application development.                                              Pramod Sadalage




© Martin Fowler and Pramod Sadalage   Rendered: February 8, 2012 11:26
SQL has Ruled for two decades

        Store persistent data                                 Application Integration
          Storing large amounts of data                         Many applications in an enterprise
          on disk, while allowing                               need to share information. By
          applications to grab the bits                         getting all applications to use the
          they need through queries                             database, we ensure all these
                                                                applications have consistent, up-to-
                                                                date data

                                          Mostly Standard
                                           The relational model is widely
                                           used and understood. Interaction
                                           with the database is done with
                                           SQL, which is a (mostly) standard
    Concurrency Control                    language. This degree of
     Many users access the same            standardization is enough to keep
     information at the same time.         things familiar so people don’t                Reporting
     Handling this concurrency is          need to learn new things
                                                                                            SQL’s simple data model and
     difficult to program, so databases                                                     standardization has made it a
     provide transactions to help                                                           foundation for many reporting
     ensure consistent interaction.                                                         tools




      All this supported by Big Database Vendors and the separation of the DBA profession.
2
but SQL’s dominance is cracking

    Relational databases are designed                             But it’s cheaper and more effective
    to run on a single machine, so to                             to scale horizontally by buying lots of
    scale, you need buy a bigger                                  machines.
    machine




                                                                                           SQL
        SQL




                                        The machines in these large clusters are                   Google and Amazon were both early
                                        individually unreliable, but the overall cluster           adopters of large clusters, and both
                                        keeps working even as machines die - so the                eschewed relational databases.
                                        overall cluster is reliable.
                                        The “cloud” is exactly this kind of cluster, which          Google                   Bigtable
                                        means relational databases don’t play well with
                                        the cloud.
                                                                                                   Amazon                   Dynamo
                                        The rise of web services provides an effective
                                        alternative to shared databases for application
                                        integration, making it easier for different                Their efforts have been a large inspiration
                                        applications to choose their own data storage.             to the NoSQL community
3
so now we have NoSQL databases
                                                      examples include
    There is no standard definition of what NoSQL
    means. The term began with a workshop
    organized in 2009, but there is much
    argument about what databases can truly be
    called NoSQL.
    But while there is no formal definition, there
    are some common characteristics of NoSQL
    databases
       they don’t use the relational data model,
       and thus don’t use the SQL language
       they tend to be designed to run on a
       cluster
       they tend to be Open Source
       they don’t have a fixed schema, allowing      We should also remember Google’s
       you to store any data in any record           Bigtable and Amazon’s SimpleDB. While
                                                     these are tied to their host’s cloud
                                                     service, they certainly fit the general
                                                     operating characteristics




4
so this means we can
Reduce Development Drag                                              Embrace Large Scale
A lot of effort in application development is tied up in             The large scale clusters that we can support with
working with relational databases. Although Object/                  NoSQL databases allow us to store larger datasets
Relational Mapping frameworks have eased the load, the               (people are talking about petabytes these days) to
database is still a significant source of developer hours.           process large amounts of analytic data.
Often we can reduce this effort by choosing an alternative
database that’s more suited to the problem domain.                   Alternative data models also allow us to carry out many
                                                                     tasks more efficiently, allowing us to tackle problems
We often come across projects who are using relational               that we would have balked at when using only relational
databases because they are the default, not because they             databases
are the best choice for the job. Often they are paying a
cost, in developer time and execution performance, for
features they do not use.
                                                                                                  Danish Health Care
                                                                                                    Centralized record of drug
                                                                                                    prescriptions. Currently held in
                                                                                                    MySQL databases, but concerned
                                                                                                    about scale for both response time
                                                                                                    and availability. Migrated data to
                                                 DNC                                                Riak. [more...]
                                                  Searching 300 Million voters
                                                  information for 1 person with
                                                  addresses, emails, phones is
                                                  tough with a relational data store.
                                                                                               McLaren
    Guardian                                      MongoDB was used to store the
                                                  documents about the person.                    Streaming of telemetric data into
     New functionality uses Mongo rather          [more...]                                      MongoDB for later analysis. Orders
     than relational DB. They found                                                              of magnitude faster than relational
     Mongo’s document data model                                                                 (SQL Server). [more...]
     significantly easier to interact with for
5    their kind of application. [more...]
but this does not mean relational is dead


              the relational model is still             ACID transactions
              relevant                                    In order to run effectively on a
               The tabular model is suitable for          cluster, most NoSQL databases
               many kinds of data, particularly           have limited transactional
               when you need to pick apart data           capability. Often this is enough…
               and re-assemble it in different            but not always.
               ways for different purposes.




      Tools                                        Familiarity
       The long dominance of SQL                    NoSQL systems are still new, so
       means that many tools have been              people aren’t familiar with using
       written to work with SQL                     them. So we shouldn’t be using
       databases. Tooling for alternative           them on utility projects where
       datastores is much more limited.             their benefits would have less
                                                    impact.




6
this leads us to a world of

       Polyglot Persistence
                                        using multiple data storage technologies, chosen
                                        based upon the way data is being used by
                                        individual applications. Why store binary images
                                        in relational database, when there are better
                                        storage systems?




      Polyglot persistence will occur over the enterprise
      as different applications use different data storage
      technologies. It will also occur within a single
      application as different parts of an application’s
      data store have different access characteristics.      http://martinfowler.com/bliki/PolyglotPersistence.html

7
what might Polyglot Persistence look like?

                                                                        Needs high availability across               Rapidly traverse links
    Rapid access for reads              Needs transactional             multiple locations. Can merge                between friends, product
    and writes. No need to              updates. Tabular                inconsistent writes                          purchases, and ratings
    be durable                          structure fits data



                                               Speculative Retailers Web Application



                      User sessions               Financial Data         Shopping Cart          Recomendations


                         Redis                       RDBMS                   Riak                    Neo4J                 This is a very
                                                                                                                           hypothetical example,
                                                                                                                           we would not make
                                                                                                                           technology
                     Product Catalog                Reporting              Analytics            User activity logs         recommendations
                                                                                                                           without more
                       MongoDB                       RDBMS                 Cassandra               Cassandra
                                                                                                                           contextual information



                 Lots of reads, infrequent
                 writes. Products make                             Large-scale analytics on
                 natural aggregate                                 large cluster                  High volume of writes on
                                                                                                  multiple nodes
                                      SQL interfaces well with
                                      reporting tools
8
polyglot persistence provides lots of new
    opportunities for enterprises

           i.e. problems
                   Decisions                            Organizational Change
                    We have to decide what data          How will our data groups react to
                    storage technology to use, rather    this new technology?
                    than just go with relational

                   Immaturity                           Dealing with Eventual Consistency
                    NoSQL tools are still young, and    Paradigm
                    full of the rough edges that new
                    tools have.                          How will different stakeholders in the
                                                         enterprise data deal with data that
                    Furthermore since we don’t have      could be stale and how do you
                    much experience with them, we        enforce rules to sync data across
                    don’t know how to use them well,     systems
                    what the good patterns are, and
                    what gotchas are lying in wait.



9
what kinds of projects are candidates for
  polyglot persistence?
                                                                                       If you need to get to market quickly,
                                                                                       then you need to maximize
                                                                                       productivity of your development
                                                                                       team. If appropriate, polyglot
                                                                                       persistence can remove significant
                                              rapid time to market                     drag.



                Strategic               and      and/or

                                              data intensive

Most software projects are utility                                   Data intensiveness can come in
projects, i.e. they aren’t central to                                various forms
the competitive advantage of the                                        lots of data
company. Utility projects should not
                                                                        high availability
take on the risk and staffing
demands that polyglot persistence                                       lots of traffic: reads or writes
brings as the potential benefits are                                    complex data relationships
not there.
                                                                     Any of these may suggest non-
                                                                     relational storage, but its the exact
                                                                     nature of the data interaction that
                                                                     will suggest the best of the many
10                                                                   alternatives.
for more information…
              On the Web
                We are both active writers on our websites.
                Martin writes at http://martinfowler.com and
                Pramod at http://www.sadalage.com/.




              Forthcoming Book
                We are currently working on a introductory book
                to NoSQL databases, to be titled: NoSQL
                Distilled (see http://martinfowler.com/bliki/
                NosqlDistilled.html)



              Consulting and Delivery
                ThoughtWorks has carried out several projects
                delivering production systems using NoSQL
                technologies. To see if NoSQL is a good fit for
                your needs, and how we can help your delivery,
                contact your local ThoughtWorks office, which
                you can find at http://thoughtworks.com



11

More Related Content

What's hot

NoSQL Data Architecture Patterns
NoSQL Data ArchitecturePatternsNoSQL Data ArchitecturePatterns
NoSQL Data Architecture PatternsMaynooth University
 
Comparison between rdbms and nosql
Comparison between rdbms and nosqlComparison between rdbms and nosql
Comparison between rdbms and nosqlbharati k
 
2012 10 24_briefing room
2012 10 24_briefing room2012 10 24_briefing room
2012 10 24_briefing roomNuoDB
 
Sql vs NO-SQL database differences explained
Sql vs NO-SQL database differences explainedSql vs NO-SQL database differences explained
Sql vs NO-SQL database differences explainedSatya Pal
 
The Future of Distributed Databases
The Future of Distributed DatabasesThe Future of Distributed Databases
The Future of Distributed DatabasesNuoDB
 
Indic threads pune12-nosql now and path ahead
Indic threads pune12-nosql now and path aheadIndic threads pune12-nosql now and path ahead
Indic threads pune12-nosql now and path aheadIndicThreads
 
To SQL or NoSQL, that is the question
To SQL or NoSQL, that is the questionTo SQL or NoSQL, that is the question
To SQL or NoSQL, that is the questionKrishnakumar S
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?Venu Anuganti
 
Schema migrations in no sql
Schema migrations in no sqlSchema migrations in no sql
Schema migrations in no sqlDr-Dipali Meher
 
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBeyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBen Stopford
 
Cache and consistency in nosql
Cache and consistency in nosqlCache and consistency in nosql
Cache and consistency in nosqlJoão Gabriel Lima
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQLDon Demcsak
 

What's hot (20)

NoSQL Data Architecture Patterns
NoSQL Data ArchitecturePatternsNoSQL Data ArchitecturePatterns
NoSQL Data Architecture Patterns
 
Selecting best NoSQL
Selecting best NoSQL Selecting best NoSQL
Selecting best NoSQL
 
Polyglot Persistence
Polyglot Persistence Polyglot Persistence
Polyglot Persistence
 
Nosql
NosqlNosql
Nosql
 
Comparison between rdbms and nosql
Comparison between rdbms and nosqlComparison between rdbms and nosql
Comparison between rdbms and nosql
 
2012 10 24_briefing room
2012 10 24_briefing room2012 10 24_briefing room
2012 10 24_briefing room
 
Sql vs NO-SQL database differences explained
Sql vs NO-SQL database differences explainedSql vs NO-SQL database differences explained
Sql vs NO-SQL database differences explained
 
Datastores
DatastoresDatastores
Datastores
 
The Future of Distributed Databases
The Future of Distributed DatabasesThe Future of Distributed Databases
The Future of Distributed Databases
 
RDBMS vs NoSQL
RDBMS vs NoSQLRDBMS vs NoSQL
RDBMS vs NoSQL
 
Indic threads pune12-nosql now and path ahead
Indic threads pune12-nosql now and path aheadIndic threads pune12-nosql now and path ahead
Indic threads pune12-nosql now and path ahead
 
NoSQL databases
NoSQL databasesNoSQL databases
NoSQL databases
 
To SQL or NoSQL, that is the question
To SQL or NoSQL, that is the questionTo SQL or NoSQL, that is the question
To SQL or NoSQL, that is the question
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?
 
Schema migrations in no sql
Schema migrations in no sqlSchema migrations in no sql
Schema migrations in no sql
 
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBeyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
 
NoSQL Basics - A Quick Tour
NoSQL Basics - A Quick TourNoSQL Basics - A Quick Tour
NoSQL Basics - A Quick Tour
 
Cache and consistency in nosql
Cache and consistency in nosqlCache and consistency in nosql
Cache and consistency in nosql
 
Queues, Pools and Caches paper
Queues, Pools and Caches paperQueues, Pools and Caches paper
Queues, Pools and Caches paper
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQL
 

Similar to Nosql intro

Nosql-Module 1 PPT.pptx
Nosql-Module 1 PPT.pptxNosql-Module 1 PPT.pptx
Nosql-Module 1 PPT.pptxRadhika R
 
Module-1.pptx63.pptx
Module-1.pptx63.pptxModule-1.pptx63.pptx
Module-1.pptx63.pptxShrinivasa6
 
A Comparative Study of NoSQL and Relational Database.pdf
A Comparative Study of NoSQL and Relational Database.pdfA Comparative Study of NoSQL and Relational Database.pdf
A Comparative Study of NoSQL and Relational Database.pdfJennifer Roman
 
Ijaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseerIjaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseerijaprr_editor
 
Ijaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseerIjaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseerijaprr
 
Data management in cloud study of existing systems and future opportunities
Data management in cloud study of existing systems and future opportunitiesData management in cloud study of existing systems and future opportunities
Data management in cloud study of existing systems and future opportunitiesEditor Jacotech
 
SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...
SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...
SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...IJCERT JOURNAL
 
SQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdf
SQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdfSQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdf
SQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdfssusere444941
 
NoSql And The Semantic Web
NoSql And The Semantic WebNoSql And The Semantic Web
NoSql And The Semantic WebIrina Hutanu
 
Big Data using NoSQL Technologies
Big Data using NoSQL TechnologiesBig Data using NoSQL Technologies
Big Data using NoSQL TechnologiesAmit Singh
 
No sql databases explained
No sql databases explainedNo sql databases explained
No sql databases explainedSalil Mehendale
 
Comparative study of no sql document, column store databases and evaluation o...
Comparative study of no sql document, column store databases and evaluation o...Comparative study of no sql document, column store databases and evaluation o...
Comparative study of no sql document, column store databases and evaluation o...ijdms
 
1. introduction to no sql
1. introduction to no sql1. introduction to no sql
1. introduction to no sqlAnuja Gunale
 
Presentation on Databases in the Cloud
Presentation on Databases in the CloudPresentation on Databases in the Cloud
Presentation on Databases in the Cloudmoshfiq
 
Massive Data Analytics and the Cloud
Massive Data Analytics and the CloudMassive Data Analytics and the Cloud
Massive Data Analytics and the CloudBooz Allen Hamilton
 
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLA STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLijscai
 

Similar to Nosql intro (20)

No sql database
No sql databaseNo sql database
No sql database
 
Report 1.0.docx
Report 1.0.docxReport 1.0.docx
Report 1.0.docx
 
Nosql-Module 1 PPT.pptx
Nosql-Module 1 PPT.pptxNosql-Module 1 PPT.pptx
Nosql-Module 1 PPT.pptx
 
Module-1.pptx63.pptx
Module-1.pptx63.pptxModule-1.pptx63.pptx
Module-1.pptx63.pptx
 
A Comparative Study of NoSQL and Relational Database.pdf
A Comparative Study of NoSQL and Relational Database.pdfA Comparative Study of NoSQL and Relational Database.pdf
A Comparative Study of NoSQL and Relational Database.pdf
 
Ijaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseerIjaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseer
 
Ijaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseerIjaprr vol1-2-6-9naseer
Ijaprr vol1-2-6-9naseer
 
Data management in cloud study of existing systems and future opportunities
Data management in cloud study of existing systems and future opportunitiesData management in cloud study of existing systems and future opportunities
Data management in cloud study of existing systems and future opportunities
 
Report 2.0.docx
Report 2.0.docxReport 2.0.docx
Report 2.0.docx
 
SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...
SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...
SURVEY ON IMPLEMANTATION OF COLUMN ORIENTED NOSQL DATA STORES ( BIGTABLE & CA...
 
SQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdf
SQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdfSQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdf
SQL OR NoSQL DATABASES? CRITICAL DIFFERENCES.pdf
 
NoSql And The Semantic Web
NoSql And The Semantic WebNoSql And The Semantic Web
NoSql And The Semantic Web
 
Big Data using NoSQL Technologies
Big Data using NoSQL TechnologiesBig Data using NoSQL Technologies
Big Data using NoSQL Technologies
 
No sql databases explained
No sql databases explainedNo sql databases explained
No sql databases explained
 
Comparative study of no sql document, column store databases and evaluation o...
Comparative study of no sql document, column store databases and evaluation o...Comparative study of no sql document, column store databases and evaluation o...
Comparative study of no sql document, column store databases and evaluation o...
 
39 43
39 4339 43
39 43
 
1. introduction to no sql
1. introduction to no sql1. introduction to no sql
1. introduction to no sql
 
Presentation on Databases in the Cloud
Presentation on Databases in the CloudPresentation on Databases in the Cloud
Presentation on Databases in the Cloud
 
Massive Data Analytics and the Cloud
Massive Data Analytics and the CloudMassive Data Analytics and the Cloud
Massive Data Analytics and the Cloud
 
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQLA STUDY ON GRAPH STORAGE DATABASE OF NOSQL
A STUDY ON GRAPH STORAGE DATABASE OF NOSQL
 

More from Hoang Nguyen

GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and ApplicationsHoang Nguyen
 
Scrum - An introduction
Scrum - An introductionScrum - An introduction
Scrum - An introductionHoang Nguyen
 
Introduction to Cross-platform App Development
Introduction to Cross-platform App DevelopmentIntroduction to Cross-platform App Development
Introduction to Cross-platform App DevelopmentHoang Nguyen
 
Conistency of random forests
Conistency of random forestsConistency of random forests
Conistency of random forestsHoang Nguyen
 
Trust - Digital Signature
Trust - Digital SignatureTrust - Digital Signature
Trust - Digital SignatureHoang Nguyen
 
SOME SECURITY CHALLENGES IN CLOUD COMPUTING
SOME SECURITY CHALLENGES  IN CLOUD COMPUTINGSOME SECURITY CHALLENGES  IN CLOUD COMPUTING
SOME SECURITY CHALLENGES IN CLOUD COMPUTINGHoang Nguyen
 
Information, Data and Decision Making
Information, Data and Decision MakingInformation, Data and Decision Making
Information, Data and Decision MakingHoang Nguyen
 
Multiple processor systems
Multiple processor systemsMultiple processor systems
Multiple processor systemsHoang Nguyen
 
Multiprocessor Systems
Multiprocessor SystemsMultiprocessor Systems
Multiprocessor SystemsHoang Nguyen
 
Introduction to AOS course
Introduction to AOS courseIntroduction to AOS course
Introduction to AOS courseHoang Nguyen
 
Background Knowledge
Background KnowledgeBackground Knowledge
Background KnowledgeHoang Nguyen
 
Introduction to Information Security Course
Introduction to Information Security CourseIntroduction to Information Security Course
Introduction to Information Security CourseHoang Nguyen
 
Introduction to CNS Course
Introduction to CNS CourseIntroduction to CNS Course
Introduction to CNS CourseHoang Nguyen
 

More from Hoang Nguyen (20)

GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and Applications
 
Scrum - An introduction
Scrum - An introductionScrum - An introduction
Scrum - An introduction
 
ORM in Django
ORM in DjangoORM in Django
ORM in Django
 
Introduction to Cross-platform App Development
Introduction to Cross-platform App DevelopmentIntroduction to Cross-platform App Development
Introduction to Cross-platform App Development
 
Conistency of random forests
Conistency of random forestsConistency of random forests
Conistency of random forests
 
Trust - Digital Signature
Trust - Digital SignatureTrust - Digital Signature
Trust - Digital Signature
 
Key Exchange
Key ExchangeKey Exchange
Key Exchange
 
SOME SECURITY CHALLENGES IN CLOUD COMPUTING
SOME SECURITY CHALLENGES  IN CLOUD COMPUTINGSOME SECURITY CHALLENGES  IN CLOUD COMPUTING
SOME SECURITY CHALLENGES IN CLOUD COMPUTING
 
Stream ciphers
Stream ciphersStream ciphers
Stream ciphers
 
Classical ciphers
Classical ciphersClassical ciphers
Classical ciphers
 
Confidentiality
ConfidentialityConfidentiality
Confidentiality
 
Information, Data and Decision Making
Information, Data and Decision MakingInformation, Data and Decision Making
Information, Data and Decision Making
 
Multiple processor systems
Multiple processor systemsMultiple processor systems
Multiple processor systems
 
Multiprocessor Systems
Multiprocessor SystemsMultiprocessor Systems
Multiprocessor Systems
 
Introduction to AOS course
Introduction to AOS courseIntroduction to AOS course
Introduction to AOS course
 
Background Knowledge
Background KnowledgeBackground Knowledge
Background Knowledge
 
Introduction to Information Security Course
Introduction to Information Security CourseIntroduction to Information Security Course
Introduction to Information Security Course
 
Introduction to CNS Course
Introduction to CNS CourseIntroduction to CNS Course
Introduction to CNS Course
 
Dynamic Testing
Dynamic TestingDynamic Testing
Dynamic Testing
 
Static Testing
Static TestingStatic Testing
Static Testing
 

Recently uploaded

Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptxHansamali Gamage
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Alkin Tezuysal
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2DianaGray10
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxKaustubhBhavsar6
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4DianaGray10
 
AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024Brian Pichman
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsDianaGray10
 

Recently uploaded (20)

Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptx
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4
 
AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projects
 

Nosql intro

  • 1. The future is: NoSQL Databases Polyglot Persistence Martin Fowler a note on the future of data storage in the enterprise, written primarily for those involved in the management of application development. Pramod Sadalage © Martin Fowler and Pramod Sadalage Rendered: February 8, 2012 11:26
  • 2. SQL has Ruled for two decades Store persistent data Application Integration Storing large amounts of data Many applications in an enterprise on disk, while allowing need to share information. By applications to grab the bits getting all applications to use the they need through queries database, we ensure all these applications have consistent, up-to- date data Mostly Standard The relational model is widely used and understood. Interaction with the database is done with SQL, which is a (mostly) standard Concurrency Control language. This degree of Many users access the same standardization is enough to keep information at the same time. things familiar so people don’t Reporting Handling this concurrency is need to learn new things SQL’s simple data model and difficult to program, so databases standardization has made it a provide transactions to help foundation for many reporting ensure consistent interaction. tools All this supported by Big Database Vendors and the separation of the DBA profession. 2
  • 3. but SQL’s dominance is cracking Relational databases are designed But it’s cheaper and more effective to run on a single machine, so to to scale horizontally by buying lots of scale, you need buy a bigger machines. machine SQL SQL The machines in these large clusters are Google and Amazon were both early individually unreliable, but the overall cluster adopters of large clusters, and both keeps working even as machines die - so the eschewed relational databases. overall cluster is reliable. The “cloud” is exactly this kind of cluster, which Google Bigtable means relational databases don’t play well with the cloud. Amazon Dynamo The rise of web services provides an effective alternative to shared databases for application integration, making it easier for different Their efforts have been a large inspiration applications to choose their own data storage. to the NoSQL community 3
  • 4. so now we have NoSQL databases examples include There is no standard definition of what NoSQL means. The term began with a workshop organized in 2009, but there is much argument about what databases can truly be called NoSQL. But while there is no formal definition, there are some common characteristics of NoSQL databases they don’t use the relational data model, and thus don’t use the SQL language they tend to be designed to run on a cluster they tend to be Open Source they don’t have a fixed schema, allowing We should also remember Google’s you to store any data in any record Bigtable and Amazon’s SimpleDB. While these are tied to their host’s cloud service, they certainly fit the general operating characteristics 4
  • 5. so this means we can Reduce Development Drag Embrace Large Scale A lot of effort in application development is tied up in The large scale clusters that we can support with working with relational databases. Although Object/ NoSQL databases allow us to store larger datasets Relational Mapping frameworks have eased the load, the (people are talking about petabytes these days) to database is still a significant source of developer hours. process large amounts of analytic data. Often we can reduce this effort by choosing an alternative database that’s more suited to the problem domain. Alternative data models also allow us to carry out many tasks more efficiently, allowing us to tackle problems We often come across projects who are using relational that we would have balked at when using only relational databases because they are the default, not because they databases are the best choice for the job. Often they are paying a cost, in developer time and execution performance, for features they do not use. Danish Health Care Centralized record of drug prescriptions. Currently held in MySQL databases, but concerned about scale for both response time and availability. Migrated data to DNC Riak. [more...] Searching 300 Million voters information for 1 person with addresses, emails, phones is tough with a relational data store. McLaren Guardian MongoDB was used to store the documents about the person. Streaming of telemetric data into New functionality uses Mongo rather [more...] MongoDB for later analysis. Orders than relational DB. They found of magnitude faster than relational Mongo’s document data model (SQL Server). [more...] significantly easier to interact with for 5 their kind of application. [more...]
  • 6. but this does not mean relational is dead the relational model is still ACID transactions relevant In order to run effectively on a The tabular model is suitable for cluster, most NoSQL databases many kinds of data, particularly have limited transactional when you need to pick apart data capability. Often this is enough… and re-assemble it in different but not always. ways for different purposes. Tools Familiarity The long dominance of SQL NoSQL systems are still new, so means that many tools have been people aren’t familiar with using written to work with SQL them. So we shouldn’t be using databases. Tooling for alternative them on utility projects where datastores is much more limited. their benefits would have less impact. 6
  • 7. this leads us to a world of Polyglot Persistence using multiple data storage technologies, chosen based upon the way data is being used by individual applications. Why store binary images in relational database, when there are better storage systems? Polyglot persistence will occur over the enterprise as different applications use different data storage technologies. It will also occur within a single application as different parts of an application’s data store have different access characteristics. http://martinfowler.com/bliki/PolyglotPersistence.html 7
  • 8. what might Polyglot Persistence look like? Needs high availability across Rapidly traverse links Rapid access for reads Needs transactional multiple locations. Can merge between friends, product and writes. No need to updates. Tabular inconsistent writes purchases, and ratings be durable structure fits data Speculative Retailers Web Application User sessions Financial Data Shopping Cart Recomendations Redis RDBMS Riak Neo4J This is a very hypothetical example, we would not make technology Product Catalog Reporting Analytics User activity logs recommendations without more MongoDB RDBMS Cassandra Cassandra contextual information Lots of reads, infrequent writes. Products make Large-scale analytics on natural aggregate large cluster High volume of writes on multiple nodes SQL interfaces well with reporting tools 8
  • 9. polyglot persistence provides lots of new opportunities for enterprises i.e. problems Decisions Organizational Change We have to decide what data How will our data groups react to storage technology to use, rather this new technology? than just go with relational Immaturity Dealing with Eventual Consistency NoSQL tools are still young, and Paradigm full of the rough edges that new tools have. How will different stakeholders in the enterprise data deal with data that Furthermore since we don’t have could be stale and how do you much experience with them, we enforce rules to sync data across don’t know how to use them well, systems what the good patterns are, and what gotchas are lying in wait. 9
  • 10. what kinds of projects are candidates for polyglot persistence? If you need to get to market quickly, then you need to maximize productivity of your development team. If appropriate, polyglot persistence can remove significant rapid time to market drag. Strategic and and/or data intensive Most software projects are utility Data intensiveness can come in projects, i.e. they aren’t central to various forms the competitive advantage of the lots of data company. Utility projects should not high availability take on the risk and staffing demands that polyglot persistence lots of traffic: reads or writes brings as the potential benefits are complex data relationships not there. Any of these may suggest non- relational storage, but its the exact nature of the data interaction that will suggest the best of the many 10 alternatives.
  • 11. for more information… On the Web We are both active writers on our websites. Martin writes at http://martinfowler.com and Pramod at http://www.sadalage.com/. Forthcoming Book We are currently working on a introductory book to NoSQL databases, to be titled: NoSQL Distilled (see http://martinfowler.com/bliki/ NosqlDistilled.html) Consulting and Delivery ThoughtWorks has carried out several projects delivering production systems using NoSQL technologies. To see if NoSQL is a good fit for your needs, and how we can help your delivery, contact your local ThoughtWorks office, which you can find at http://thoughtworks.com 11