This is an introduction to relational and non-relational databases and how their performance affects scaling a web application.
This is a recording of a guest Lecture I gave at the University of Texas school of Information.
In this talk I address the technologies and tools Gowalla (gowalla.com) uses including memcache, redis and cassandra.
Find more on my blog:
http://schneems.com
A presentation I gave on Refactoring for the RIA Unleashed conference in 2011 up in Boston.
Video References:
Winston Wolfe - I Solve Problems form Pulp Fiction
http://www.youtube.com/watch?v=DO0d7dpA-K8
And the ready scene from The Last Samurai:
http://www.youtube.com/watch?v=QE3yMEfpk6E
Ruby Meditation 12 - 19.11.2016
His theme is Functional Ruby. How to forgot pain of side effects and start to write your Erlang in Ruby. Или просто очередная серебряная пуля для задач, которые вам никогда не попадутся.
#rubymeditation
Becoming a more productive Rails DeveloperJohn McCaffrey
A presentation by John McCaffrey of RailsPerformance.com on how to manage technical information, ask technical questions, expand Ruby and Rails knowledge, and work on interesting side projects for open source, non-profits or as a freelancer
Tutorial on SPARQL 1.1 given at SWAT4LS 2012 in paris to a full room. This material covers enough to get started and includes working with Topbraid Composer.
This is an introduction to relational and non-relational databases and how their performance affects scaling a web application.
This is a recording of a guest Lecture I gave at the University of Texas school of Information.
In this talk I address the technologies and tools Gowalla (gowalla.com) uses including memcache, redis and cassandra.
Find more on my blog:
http://schneems.com
A presentation I gave on Refactoring for the RIA Unleashed conference in 2011 up in Boston.
Video References:
Winston Wolfe - I Solve Problems form Pulp Fiction
http://www.youtube.com/watch?v=DO0d7dpA-K8
And the ready scene from The Last Samurai:
http://www.youtube.com/watch?v=QE3yMEfpk6E
Ruby Meditation 12 - 19.11.2016
His theme is Functional Ruby. How to forgot pain of side effects and start to write your Erlang in Ruby. Или просто очередная серебряная пуля для задач, которые вам никогда не попадутся.
#rubymeditation
Becoming a more productive Rails DeveloperJohn McCaffrey
A presentation by John McCaffrey of RailsPerformance.com on how to manage technical information, ask technical questions, expand Ruby and Rails knowledge, and work on interesting side projects for open source, non-profits or as a freelancer
Tutorial on SPARQL 1.1 given at SWAT4LS 2012 in paris to a full room. This material covers enough to get started and includes working with Topbraid Composer.
A walkthrough of various application performance tuning tools and a good workflow for where to start, from a presentation at WindyCityRails 2011 in Chicago, IL.
See the video, and more Web and Ruby/Rails Performance info at www.RailsPerformance.com
-John McCaffrey
Writing concurrent programs that can run in multiple threads and on multiple cores is crucial but daunting. Futures provides a convenient abstraction for many problem domains. The online course "Intermediate Scala" includes an up-to-date discussion of futures and the parts of java.util.concurrent that underlie the Scala futures implementation. Unlike Java's futures, Scala futures supports composition, transformations and sophisticated callbacks.
The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
This presentation is for enterprises that are considering adopting Scala. The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
Many companies and individuals these days release parts of their work as open source software. This benefits the entire software development community and brings are new set of challenges. Maintaining open source well takes time and effort. Abandoning a project can be very problematic for your users. How does one find a balance?
In this talk we’ll discuss how we did the impossible: make the users of more than 30 ClojureWerkz projects happy and still have a life.
Rainbows, Unicorns, and other Fairy Tales in the Land of Serverless DreamsJosh Carlisle
When done correctly Serverless offers fantastic potential but can also lead to spectacular failure when critical concepts are overlooked. With over a dozen Serverless implementations on Azure Functions over the last couple years, I’ve learned some lessons the hard way. In this talk, I will be sharing a few of the most impactful hard-earned lessons and how I was able to overcome them. I’ll be touching on topics ranging from considerations using traditional relational databases, managing service and data connections to managing complexity and increasing observability. The talk is done in the context of Azure Functions but whose concepts apply equally to all Serverless Platforms.
Creating a GraphQL API is more and more common for PHP developers. The task can seem complex but there are a lot of tools to help.
In this talk given at AFUP Paris PHP Meetup, I'm presenting GraphQL and why it is important. Then, I'm having a look at the existing libraries in PHP. Finally, I'm diving in the details of GraphQLite; a library that creates a GraphQL schema by analyzing your PHP code.
IronRuby has been around for a while. This presentation is about the practical uses of IronRuby. It contains several different use cases that you can immediately go and use to enhance your everyday work.
A Quick Preview of What You'll See at Qt World Summit 2016Qt
This year at #QtWS16, The Qt Company and our ecosystem will deliver several inspirational keynotes, insightful breakouts, take a look into the future of IoT and how Qt will be the enabler, putting you and your project ahead of the curve.
Here is a sneak peek at some of the things you will see in San Francisco at the Qt World Summit.
A walkthrough of various application performance tuning tools and a good workflow for where to start, from a presentation at WindyCityRails 2011 in Chicago, IL.
See the video, and more Web and Ruby/Rails Performance info at www.RailsPerformance.com
-John McCaffrey
Writing concurrent programs that can run in multiple threads and on multiple cores is crucial but daunting. Futures provides a convenient abstraction for many problem domains. The online course "Intermediate Scala" includes an up-to-date discussion of futures and the parts of java.util.concurrent that underlie the Scala futures implementation. Unlike Java's futures, Scala futures supports composition, transformations and sophisticated callbacks.
The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
This presentation is for enterprises that are considering adopting Scala. The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
Many companies and individuals these days release parts of their work as open source software. This benefits the entire software development community and brings are new set of challenges. Maintaining open source well takes time and effort. Abandoning a project can be very problematic for your users. How does one find a balance?
In this talk we’ll discuss how we did the impossible: make the users of more than 30 ClojureWerkz projects happy and still have a life.
Rainbows, Unicorns, and other Fairy Tales in the Land of Serverless DreamsJosh Carlisle
When done correctly Serverless offers fantastic potential but can also lead to spectacular failure when critical concepts are overlooked. With over a dozen Serverless implementations on Azure Functions over the last couple years, I’ve learned some lessons the hard way. In this talk, I will be sharing a few of the most impactful hard-earned lessons and how I was able to overcome them. I’ll be touching on topics ranging from considerations using traditional relational databases, managing service and data connections to managing complexity and increasing observability. The talk is done in the context of Azure Functions but whose concepts apply equally to all Serverless Platforms.
Creating a GraphQL API is more and more common for PHP developers. The task can seem complex but there are a lot of tools to help.
In this talk given at AFUP Paris PHP Meetup, I'm presenting GraphQL and why it is important. Then, I'm having a look at the existing libraries in PHP. Finally, I'm diving in the details of GraphQLite; a library that creates a GraphQL schema by analyzing your PHP code.
IronRuby has been around for a while. This presentation is about the practical uses of IronRuby. It contains several different use cases that you can immediately go and use to enhance your everyday work.
A Quick Preview of What You'll See at Qt World Summit 2016Qt
This year at #QtWS16, The Qt Company and our ecosystem will deliver several inspirational keynotes, insightful breakouts, take a look into the future of IoT and how Qt will be the enabler, putting you and your project ahead of the curve.
Here is a sneak peek at some of the things you will see in San Francisco at the Qt World Summit.
What do you need to keep in mind when using ORM, how it will affect your needs and what are the disadvantages of using and advantages of not using ORM.
Из презентации вы узнаете:
— как работает database/sql;
— интерфейс и реализации database/sql/driver;
— обзор популярных ORM и что с ними не так;
— как мы делали свой лучший ORM;
— и почему столько раз его переделывали.
Hibernate ORM: Tips, Tricks, and Performance TechniquesBrett Meyer
DevNexus 2014
Out-of-the-box, Hibernate ORM offers limited overhead and decent throughput. Early-stage applications enjoy the convenience of ORM/JPA with great performance. However, scaling your application into an enterprise-level system introduces more demanding needs.
This talk will describe numerous tips and techniques to both increase Hibernate ORM performance, as well as decrease overhead. These include some basic tricks, such as mapping and fetching strategies. Entity enhancement instrumentation, third-party second level caching, Hibernate Search, and more complex considerations will also be discussed. The talk will include live demonstrations techniques and their before-and-after results.
MongoDB and Ecommerce : A perfect combinationSteven Francia
Presentation given at the MongoDB NYC Meetup by Steve Francia, VP of Engineering at OpenSky. OpenSky uses MongoDB to develop the next ecommerce platform. OpenSky also uses Symfony 2, Doctrine 2, PHP 5.3, PHPUnit 3.5, jQuery, node.js, Git (with gitflow) and a touch of Java and Python. The OpenSky team contributes back to many of these technologies and employs core members of the Symfony 2 and Doctrine 2 teams.
Solr cloud the 'search first' nosql database extended deep divelucenerevolution
Presented by Mark Miller, Software Engineer, Cloudera
As the NoSQL ecosystem looks to integrate great search, great search is naturally beginning to expose many NoSQL features. Will these Goliath's collide? Or will they remain specialized while intermingling – two sides of the same coin.
Come learn about where SolrCloud fits into the NoSQL landscape. What can it do? What will it do? And how will the big data, NoSQL, Search ecosystem evolve. If you are interested in Big Data, NoSQL, distributed systems, CAP theorem and other hype filled terms, than this talk may be for you.
An introduction to core concepts in Apache Cassandra. We cover the evolution of database architecture as you try to scale a relational database to solve big data problems, and explain how Cassandra handles these problems efficiently.
The rise of NoSQL is characterized with confusion and ambiguity; very much like any fast-emerging organic movement in the absence of well-defined standards and adequate software solutions. Whether you are a developer or an architect, many questions come to mind when faced with the decision of where your data should be stored and how it should be managed. The following are some of these questions: What does the rise of all these NoSQL technologies mean to my enterprise? What is NoSQL to begin with? Does it mean "No SQL"? Could this be just another fad? Is it a good idea to bet the future of my enterprise on these new exotic technologies and simply abandon proven mature Relational DataBase Management Systems (RDBMS)? How scalable is scalable? Assuming that I am sold, how do I choose the one that fit my needs best? Is there a middle ground somewhere? What is this Polyglot Persistence I hear about? The answers to these questions and many more is the subject of this talk along with a survey of the most popular of NoSQL technologies. Be there or be square.
The relational database model was designed to solve the problems of yesterday’s data storage requirements. The massively connected world of today presents different problems and new challenges. We’ll explore the NoSQL philosophy, before comparing and contrasting the strengths and weaknesses of the relational model versus the NoSQL model. While stepping through real-world scenarios, we’ll discuss the reasons for choosing one solution over the other.
To complete this session, let’s demonstrate our findings with an application written with a NoSQL storage layer and explain the advantages that accrue from that decision. By taking a look at the new challenges we face with our data storage needs, we’ll examine why the principles behind NoSQL make it a better candidate as a solution, than yesterday’s relational model.
This is my deck from Cloud Conference Torino 2013 (http://www.cloudconf.it). I was the post-lunch speaker, so this one is more silly and there was a lot of off-deck riffing, so this is here only for posterity.
I initially planned to speak on cloud-specific stuff, this turned into an intro to MongoDB though.
6 reasons Jubilee could be a Rubyist's new best friendForrest Chang
(Video here: http://confreaks.com/videos/5014-RubyConf2014-6-reasons-jubilee-could-be-a-rubyist-s-new-best-friend or https://www.youtube.com/watch?feature=player_embedded&v=FFR0G89WXI8)
Rubyconf 2014 talk on Jubilee, a Vert.x module that runs rack apps.
Alternate titles
Beyond Rails while using Rails
Rails can't do everything I want and <fill> makes me want to cry
Rubyconf abstract
Do you do web development in Ruby? Have you been forced to go to node or other technologies just for concurrency/websockets etc. Do miss your gems, and tire of functionality you have to implement from scratch? Do you hate javascript?
Well no need to switch languages/platforms, Jubilee could be your new best friend.
Jubilee, a rack server on top of Vert.x gives you
* Concurrency
* Speed
* Easy Websockets support
* Shared Memory
* Access to the JVM ecosystem
* Ability to reuse your existing Ruby knowledge and gems
"Say Hello to your new friend" - Al Pacino
Native mobile application development with Flutter (Dart)Randal Schwartz
Create beautiful performant mobile apps for both iOS and Android using the revolutionary Flutter framework, originally from Google, now supported by a large community.
This slide deck was presented at Scale16x in Pasadena. The screencast is at https://www.youtube.com/watch?v=O7TXamVRSbY and the video recording is https://youtu.be/C3WipFZ0blQ?list=PL57quI9usf_vq2TH9NV5sMsbabkMGOnRA&t=13374 (timecode 3:42:54).
Introduction to Git, from ground zero. Includes basic concepts, most common commands and workflows. Describes rebase vs merge, what push and pull do, and what the SHA1 does for you.
So, you don't have time to read Damian Conway's "Perl Best Practices" book, to understand his "256 guidelines on the art of coding to help you write better Perl code"? Hear Randal Schwartz provide the executive summary, including pointing out where Randal disagrees with Damian, and why. This high-speed overview will help you understand "code layout, naming conventions, choice of data and control structures, program decomposition, interface design and implementation, modularity, object orientation, error handling, testing, and debugging." But using shorter words.
A brief introduction to the Dart programming language. I wrote this talk for FISL 16 (2015) and presented it there. The video is available at http://hemingway.softwarelivre.org/fisl16/high/40t/sala_40t-high-201507101800.ogv
Perl is 25, and I'm a bit more than twice that. I've spent half my life with Perl, and Perl has spent all of its life with me. The Perl culture influences and has been influenced by Stonehenge (my company) and therefore also by me. I retell the best parts of the behind-the-scenes events where Perl, Stonehenge, and Randal overlap.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
1. Forget the ORM!
Persistent data with
Non-traditional Databases
By Randal L. Schwartz,
Stonehenge Consulting Services, Inc.
<merlyn@stonehenge.com>
http://www.stonehenge.com/merlyn/
version 1.2 at 13 April 2010
9. Brief review of ORMs
• Objects in memory
• Tables in RDBMS
• Load objects from tables
10. Brief review of ORMs
• Objects in memory
• Tables in RDBMS
• Load objects from tables
• Store objects (and changes) back to tables
11. Brief review of ORMs
• Objects in memory
• Tables in RDBMS
• Load objects from tables
• Store objects (and changes) back to tables
• Typically mapped one attribute per column
12. Brief review of ORMs
• Objects in memory
• Tables in RDBMS
• Load objects from tables
• Store objects (and changes) back to tables
• Typically mapped one attribute per column
• Occasionally more complex mappings
16. But things change
• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
17. But things change
• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
• Updates often require custom SQL
18. But things change
• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
• Updates often require custom SQL
• Or live with updating more than needed
19. But things change
• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
• Updates often require custom SQL
• Or live with updating more than needed
• Detecting what has changed can be hard
23. One-to-many messes
• Messy when an attribute is a reference
• Or a non-DB data type, like a set
• Mapping involves joins
24. One-to-many messes
• Messy when an attribute is a reference
• Or a non-DB data type, like a set
• Mapping involves joins
• Left joins to get child rows
25. One-to-many messes
• Messy when an attribute is a reference
• Or a non-DB data type, like a set
• Mapping involves joins
• Left joins to get child rows
• Or multiple trips to get joined data
26. One-to-many messes
• Messy when an attribute is a reference
• Or a non-DB data type, like a set
• Mapping involves joins
• Left joins to get child rows
• Or multiple trips to get joined data
• Either way, expensive
29. Why this is bad
• If your car were an ORM...
• Reduce it to parts each night in garage
30. Why this is bad
• If your car were an ORM...
• Reduce it to parts each night in garage
• Rebuild it each morning
31. Why this is bad
• If your car were an ORM...
• Reduce it to parts each night in garage
• Rebuild it each morning
• Is this sane in the 21st century?
32. Why this is bad
• If your car were an ORM...
• Reduce it to parts each night in garage
• Rebuild it each morning
• Is this sane in the 21st century?
• Most ORMs generate the SQL on the fly
33. Why this is bad
• If your car were an ORM...
• Reduce it to parts each night in garage
• Rebuild it each morning
• Is this sane in the 21st century?
• Most ORMs generate the SQL on the fly
• Recompiling text on each hit?
34. Why this is bad
• If your car were an ORM...
• Reduce it to parts each night in garage
• Rebuild it each morning
• Is this sane in the 21st century?
• Most ORMs generate the SQL on the fly
• Recompiling text on each hit?
• Often called the “object relational
impedence mismatch”
36. How to solve it
• Rectangles don’t fit today’s data
37. How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
38. How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
39. How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• Two main options:
40. How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• Two main options:
• Document storage (typically JSON)
41. How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• Two main options:
• Document storage (typically JSON)
• Object storage
42. How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• Two main options:
• Document storage (typically JSON)
• Object storage
• (Generally) not possible if other apps still
need the data as RDB
46. Dropping ACID
• Automicity, Consistency, Isolation,
Durability
• Often expressed through transactions
• Most NoSQL offer no multi-update ACID
47. Dropping ACID
• Automicity, Consistency, Isolation,
Durability
• Often expressed through transactions
• Most NoSQL offer no multi-update ACID
• But atomic within a single doc update
48. Dropping ACID
• Automicity, Consistency, Isolation,
Durability
• Often expressed through transactions
• Most NoSQL offer no multi-update ACID
• But atomic within a single doc update
• Thus, think about your “schema” carefully
49. Dropping ACID
• Automicity, Consistency, Isolation,
Durability
• Often expressed through transactions
• Most NoSQL offer no multi-update ACID
• But atomic within a single doc update
• Thus, think about your “schema” carefully
• Ensure single doc update suffices
53. Eric Brewer’s CAP
• Consistency
• All working, or not working
• Yes, this is the “A” in ACID
54. Eric Brewer’s CAP
• Consistency
• All working, or not working
• Yes, this is the “A” in ACID
• Availability
55. Eric Brewer’s CAP
• Consistency
• All working, or not working
• Yes, this is the “A” in ACID
• Availability
• Is the service up and running?
56. Eric Brewer’s CAP
• Consistency
• All working, or not working
• Yes, this is the “A” in ACID
• Availability
• Is the service up and running?
• Partition Tolerance
57. Eric Brewer’s CAP
• Consistency
• All working, or not working
• Yes, this is the “A” in ACID
• Availability
• Is the service up and running?
• Partition Tolerance
• Can parts of it go offline safely?
58. Eric Brewer’s CAP
• Consistency
• All working, or not working
• Yes, this is the “A” in ACID
• Availability
• Is the service up and running?
• Partition Tolerance
• Can parts of it go offline safely?
• .... Pick any two
86. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
87. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive JavaScript shell
88. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive JavaScript shell
• In-place updates
89. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive JavaScript shell
• In-place updates
• Replication
90. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive JavaScript shell
• In-place updates
• Replication
• Auto-sharding
91. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive JavaScript shell
• In-place updates
• Replication
• Auto-sharding
• Bindings for many languages
92. MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive JavaScript shell
• In-place updates
• Replication
• Auto-sharding
• Bindings for many languages
• FLOSS Weekly #105
96. Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
• Implements Java Content Repository API
97. Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
• Implements Java Content Repository API
• Full text and XPath search
98. Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
• Implements Java Content Repository API
• Full text and XPath search
• Versioning, transactions, observation
99. Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
• Implements Java Content Repository API
• Full text and XPath search
• Versioning, transactions, observation
• Authentication
100. Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
• Implements Java Content Repository API
• Full text and XPath search
• Versioning, transactions, observation
• Authentication
• Object persistence using Object Content
Manager
105. MarkLogic Server
• Not open-source
• XML database
• Implements XQuery
• Full-text and structured search
106. MarkLogic Server
• Not open-source
• XML database
• Implements XQuery
• Full-text and structured search
• Native geospatial searches
107. MarkLogic Server
• Not open-source
• XML database
• Implements XQuery
• Full-text and structured search
• Native geospatial searches
• In heavy use by their customers
108. MarkLogic Server
• Not open-source
• XML database
• Implements XQuery
• Full-text and structured search
• Native geospatial searches
• In heavy use by their customers
• Scalable to “hundreds of terabytes”
109. MarkLogic Server
• Not open-source
• XML database
• Implements XQuery
• Full-text and structured search
• Native geospatial searches
• In heavy use by their customers
• Scalable to “hundreds of terabytes”
• Both native and RESTful APIs
123. eXist
• Open source (Java, LGPL)
• Native XML queries
• XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Can serve entire webapps!
124. eXist
• Open source (Java, LGPL)
• Native XML queries
• XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Can serve entire webapps!
• Used by large installations
125. eXist
• Open source (Java, LGPL)
• Native XML queries
• XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Can serve entire webapps!
• Used by large installations
• US State Department (history.state.gov)
126. eXist
• Open source (Java, LGPL)
• Native XML queries
• XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Can serve entire webapps!
• Used by large installations
• US State Department (history.state.gov)
• FLOSS Weekly #97
136. InfoGrid
• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
• MeshBase - self contained
• NetMeshBase - distributed knowledge
• API in front of other SQL and NoSQL DBs
137. InfoGrid
• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
• MeshBase - self contained
• NetMeshBase - distributed knowledge
• API in front of other SQL and NoSQL DBs
• Includes components for authentication
141. Neo4j
• Open source (Java, AGPL3)
• Commercial support/version available
• Java objects on disk
142. Neo4j
• Open source (Java, AGPL3)
• Commercial support/version available
• Java objects on disk
• Transactional
143. Neo4j
• Open source (Java, AGPL3)
• Commercial support/version available
• Java objects on disk
• Transactional
• Scalable (several billion nodes on single
machine)
144. Neo4j
• Open source (Java, AGPL3)
• Commercial support/version available
• Java objects on disk
• Transactional
• Scalable (several billion nodes on single
machine)
• Small footprint (JAR under 500KB)
145. Neo4j
• Open source (Java, AGPL3)
• Commercial support/version available
• Java objects on disk
• Transactional
• Scalable (several billion nodes on single
machine)
• Small footprint (JAR under 500KB)
• RDF mappings
149. AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
150. AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
• Interfaces in many common languages
151. AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
• Interfaces in many common languages
• Active development
152. AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
• Interfaces in many common languages
• Active development
• In use by FLOSS/commercial/government
153. AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
• Interfaces in many common languages
• Active development
• In use by FLOSS/commercial/government
• Queries with SPARQL
154. AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
• Interfaces in many common languages
• Active development
• In use by FLOSS/commercial/government
• Queries with SPARQL
• ... Protocol and RDF Query Language
161. Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
162. Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
• used by many large sites
163. Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
• used by many large sites
• Simple key/value storage
164. Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
• used by many large sites
• Simple key/value storage
• Keys up to 120 bytes, values 1 MB
165. Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
• used by many large sites
• Simple key/value storage
• Keys up to 120 bytes, values 1 MB
• Clientside horizontal scaling
166. Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
• used by many large sites
• Simple key/value storage
• Keys up to 120 bytes, values 1 MB
• Clientside horizontal scaling
• Can also store on disk (memcachedb)
170. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
171. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists, sets, hashes
172. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists, sets, hashes
• Master/slave replication supported
173. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists, sets, hashes
• Master/slave replication supported
• Many client languages available
174. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists, sets, hashes
• Master/slave replication supported
• Many client languages available
• Used by many big sites
175. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists, sets, hashes
• Master/slave replication supported
• Many client languages available
• Used by many big sites
• Github, Craigslist, Engine Yard, Guardian
176. Redis
• Open source (C, BSD)
• Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists, sets, hashes
• Master/slave replication supported
• Many client languages available
• Used by many big sites
• Github, Craigslist, Engine Yard, Guardian
• Can also store on disk
187. Amazon SimpleDB
• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
188. Amazon SimpleDB
• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
• Items have query-able attributes
189. Amazon SimpleDB
• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
• Items have query-able attributes
• No text search
190. Amazon SimpleDB
• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
• Items have query-able attributes
• No text search
• Build your own indexes
193. Berkeley DB
• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
194. Berkeley DB
• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
• Key/value pairs in various storage formats
195. Berkeley DB
• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
• Key/value pairs in various storage formats
• Transactional locking, HA features
196. Berkeley DB
• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
• Key/value pairs in various storage formats
• Transactional locking, HA features
• Widely used
200. Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
• Very efficient in both space and speed
201. Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
• Very efficient in both space and speed
• Simple key/value pairs
202. Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
• Very efficient in both space and speed
• Simple key/value pairs
• Multiple storage strategies
203. Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
• Very efficient in both space and speed
• Simple key/value pairs
• Multiple storage strategies
• Multiple client language interfaces
204. Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
• Very efficient in both space and speed
• Simple key/value pairs
• Multiple storage strategies
• Multiple client language interfaces
• Local storage only: no network interface
205. Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
• Very efficient in both space and speed
• Simple key/value pairs
• Multiple storage strategies
• Multiple client language interfaces
• Local storage only: no network interface
• But see Tokyo Tyrant
209. Cassandra
• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
210. Cassandra
• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availability/elastic through replication
211. Cassandra
• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availability/elastic through replication
• Used by Facebook, Digg, Twitter, Rackspace
212. Cassandra
• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availability/elastic through replication
• Used by Facebook, Digg, Twitter, Rackspace
• Eventually consistent
213. Cassandra
• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availability/elastic through replication
• Used by Facebook, Digg, Twitter, Rackspace
• Eventually consistent
• Queries can ask “majority” or “all”
217. GT.M
• Open Source (Mumps, GPL)
• Commercial support from Fidelity
• Distributed persistence of key/value pairs
218. GT.M
• Open Source (Mumps, GPL)
• Commercial support from Fidelity
• Distributed persistence of key/value pairs
• Transactions via optimistic concurrency
219. GT.M
• Open Source (Mumps, GPL)
• Commercial support from Fidelity
• Distributed persistence of key/value pairs
• Transactions via optimistic concurrency
• Everything you read must be unchanged
220. GT.M
• Open Source (Mumps, GPL)
• Commercial support from Fidelity
• Distributed persistence of key/value pairs
• Transactions via optimistic concurrency
• Everything you read must be unchanged
• Mumps-to-C-to-Mumps APIs
228. Mnesia
• Open source (Erlang, Erlang license)
• Key/value pairs
• Value is any Erlang datatype
229. Mnesia
• Open source (Erlang, Erlang license)
• Key/value pairs
• Value is any Erlang datatype
• Live reconfiguration
230. Mnesia
• Open source (Erlang, Erlang license)
• Key/value pairs
• Value is any Erlang datatype
• Live reconfiguration
• Supports transactions and distribution
234. Hbase
• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
• Push map/reduce down to shards
235. Hbase
• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
• Push map/reduce down to shards
• Runs on Hadoop Distributed File System
236. Hbase
• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
• Push map/reduce down to shards
• Runs on Hadoop Distributed File System
• Java, REST, Thrift APIs
237. Hbase
• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
• Push map/reduce down to shards
• Runs on Hadoop Distributed File System
• Java, REST, Thrift APIs
• Might be the DB behind Bing
250. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
251. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
• Supports object persistence
252. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
• Supports object persistence
• both Java and dot-net
253. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
• Supports object persistence
• both Java and dot-net
• Can replicate to traditional RDBMS
254. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
• Supports object persistence
• both Java and dot-net
• Can replicate to traditional RDBMS
• Supports class migration
255. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
• Supports object persistence
• both Java and dot-net
• Can replicate to traditional RDBMS
• Supports class migration
• ... if you provide the code
256. Db4o
• Open source (Java/C#, GPL)
• Commercial license/support available
• Supports object persistence
• both Java and dot-net
• Can replicate to traditional RDBMS
• Supports class migration
• ... if you provide the code
• Large community
262. InterSystems Cache
• Commercial
• “World’s fastest object database”
• Cross-platform
• Persistence of multidimensional arrays
• Similar to MUMPS/Pick data
263. InterSystems Cache
• Commercial
• “World’s fastest object database”
• Cross-platform
• Persistence of multidimensional arrays
• Similar to MUMPS/Pick data
• Can be embedded in web pages
275. GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
• Includes Java interface
• Automatic and/or guided class upgrades
276. GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
• Includes Java interface
• Automatic and/or guided class upgrades
• Horizontally scaleable
277. GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
• Includes Java interface
• Automatic and/or guided class upgrades
• Horizontally scaleable
• OOCL manages 40% of overseas traffic
278. GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
• Includes Java interface
• Automatic and/or guided class upgrades
• Horizontally scaleable
• OOCL manages 40% of overseas traffic
• JPM’s Kapital created the financial crisis :)
279. GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
• Includes Java interface
• Automatic and/or guided class upgrades
• Horizontally scaleable
• OOCL manages 40% of overseas traffic
• JPM’s Kapital created the financial crisis :)
• Integrates with Seaside
280. GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
• Includes Java interface
• Automatic and/or guided class upgrades
• Horizontally scaleable
• OOCL manages 40% of overseas traffic
• JPM’s Kapital created the financial crisis :)
• Integrates with Seaside
• Web apps with transparent persistence
283. Magma
• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
284. Magma
• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
285. Magma
• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-server model
286. Magma
• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-server model
• HA mode (multiple slaves ready for master)
287. Magma
• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-server model
• HA mode (multiple slaves ready for master)
• Transaction-based (commit/rollback)
288. Magma
• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-server model
• HA mode (multiple slaves ready for master)
• Transaction-based (commit/rollback)
• Works nicely with Seaside