This document discusses MongoDB replication and sharding. It begins with an introduction to MongoDB and its data model. It then covers replication with master-slave and replica sets. It also discusses sharding components including shard servers, config servers, and mongos routers. It provides examples of setting up replica sets, adding shards, and combining replica sets with sharding. The document encourages trying these techniques yourself and sharing experiences.
MongoDB supports replication for failover and redundancy. In this session we will introduce the basic concepts around replica sets, which provide automated failover and recovery of nodes. We'll cover how to set up, configure, and initiate a replica set; methods for using replication to scale reads; and proper architecture for durability.
MongoDB supports replication for failover and redundancy. In this session we will introduce the basic concepts around replica sets, which provide automated failover and recovery of nodes. We'll cover how to set up, configure, and initiate a replica set; methods for using replication to scale reads; and proper architecture for durability.
Webinaire 2 de la série « Retour aux fondamentaux » : Votre première applicat...MongoDB
Il s'agit du deuxième webinaire de la série « Retour aux fondamentaux » qui a pour but de vous présenter la base de données MongoDB. Dans ce webinaire, nous vous expliquerons comment créer une application de création de blogs dans MongoDB.
For the first time this year, 10gen will be offering a track completely dedicated to Operations at MongoSV, 10gen's annual MongoDB user conference on December 4. Learn more at MongoSV.com
MongoDB supports replication for failover and redundancy. In this session we will introduce the basic concepts around replica sets, which provide automated failover and recovery of nodes. We'll show you how to set up, configure, and initiate a replica set, and methods for using replication to scale reads. We'll also discuss proper architecture for durability.
MongoDB supports replication for failover and redundancy. In this session we will introduce the basic concepts around replica sets, which provide automated failover and recovery of nodes. We'll show you how to set up, configure, and initiate a replica set, and methods for using replication to scale reads. We'll also discuss proper architecture for durability.
As presented at Confoo 2013.
More than some arcane NoSQL tool, Redis is a simple but powerful swiss army knife you can begin using today.
This talk introduces the audience to Redis and focuses on using it to cleanly solve common problems. Along the way, we'll see how Redis can be used as an alternative to several common PHP tools.
One of the strongest points for using a NoSQL database is their focus on distribution — both for replication and sharding. This talks takes a short look at what replication is, why you should use it, and what is so difficult about it. We then take a look at MongoDB’s implementation in general and finally focus on what can go wrong. In a practical demo you see how to find the right balance between performance versus data safety and how to use it in your Java application.
This presentation will demonstrate how you can use the aggregation pipeline with MongoDB similar to how you would use GROUP BY in SQL and the new stage operators coming 3.4. MongoDB’s Aggregation Framework has many operators that give you the ability to get more value out of your data, discover usage patterns within your data, or use the Aggregation Framework to power your application. Considerations regarding version, indexing, operators, and saving the output will be reviewed.
MongoDB Europe 2016 - Debugging MongoDB PerformanceMongoDB
Asya is back, and so is Sherlock Holmes and his techniques to gather and analyze data from your poorly performing MongoDB clusters. In this advanced talk we take a deep look at all the diagnostic data that lives inside MongoDB - how to interrogate and interpret it to help you solve those frustrating performance bottlenecks that we all face occasionally.
In these slide following projects are presented:
* redis_wrap: A Pythonic wrapper that makes it nicer to work with builtin Redis datatypes
* redis_graph: A sample graph database
* redis_simple_queue: A simple queue implemented on Redis list structure
* bitmapist: a powerful analytics library using Redis bitmaps, great for retention and cohort tracking
* fixedlist: a fixed list structure that can optimize timelines (and other things)
* how to use Lua scripting for more advanced data structures and better performance
These are Python projects, but some of them (like bitmapist) have been ported to other languages.
This talk was given to PyCon Belarus on 31 Jan. 2015
Webinar: General Technical Overview of MongoDB for Dev TeamsMongoDB
In this talk we will focus on several of the reasons why developers have come to love the richness, flexibility, and ease of use that MongoDB provides. First we will give a brief introduction of MongoDB, comparing and contrasting it to the traditional relational database. Next, we’ll give an overview of the APIs and tools that are part of the MongoDB ecosystem. Then we’ll look at how MongoDB CRUD (Create, Read, Update, Delete) operations work, and also explore query, update, and projection operators. Finally, we will discuss MongoDB indexes and look at some examples of how indexes are used.
MySQL flexible schema and JSON for Internet of ThingsAlexander Rubin
My presentation at Oracle Open World Conference 2017: Using MySQL Flexible Schema (Document Store/JSON) for IoT
Tuesday, Oct 03, 11:30 a.m. - 12:15 p.m. | Marriott Marquis (Yerba Buena Level) - Salon 14
Storing data from sensors (Internet of Things) may be challenging in many respects, specifically due to the changing nature of the data. For example, if you have a fixed table structure and a sensor will need to store new property, it will be hard to make this change. This session discusses different options for implementing flexible schemas with MySQL 5.7 and MySQL 8.0, using JSON and calculated fields as well as the MySQL Document Store feature. It includes a demo with IoT devices where data is stored in MySQL 8.0.
MongoDB San Francisco 2013: Basic Sharding in MongoDB presented by Brandon Bl...MongoDB
Sharding allows you to distribute load across multiple servers and keep your data balanced across those servers. This session will review MongoDB’s sharding support, including an architectural overview, design principles, and automation.
Webinaire 2 de la série « Retour aux fondamentaux » : Votre première applicat...MongoDB
Il s'agit du deuxième webinaire de la série « Retour aux fondamentaux » qui a pour but de vous présenter la base de données MongoDB. Dans ce webinaire, nous vous expliquerons comment créer une application de création de blogs dans MongoDB.
For the first time this year, 10gen will be offering a track completely dedicated to Operations at MongoSV, 10gen's annual MongoDB user conference on December 4. Learn more at MongoSV.com
MongoDB supports replication for failover and redundancy. In this session we will introduce the basic concepts around replica sets, which provide automated failover and recovery of nodes. We'll show you how to set up, configure, and initiate a replica set, and methods for using replication to scale reads. We'll also discuss proper architecture for durability.
MongoDB supports replication for failover and redundancy. In this session we will introduce the basic concepts around replica sets, which provide automated failover and recovery of nodes. We'll show you how to set up, configure, and initiate a replica set, and methods for using replication to scale reads. We'll also discuss proper architecture for durability.
As presented at Confoo 2013.
More than some arcane NoSQL tool, Redis is a simple but powerful swiss army knife you can begin using today.
This talk introduces the audience to Redis and focuses on using it to cleanly solve common problems. Along the way, we'll see how Redis can be used as an alternative to several common PHP tools.
One of the strongest points for using a NoSQL database is their focus on distribution — both for replication and sharding. This talks takes a short look at what replication is, why you should use it, and what is so difficult about it. We then take a look at MongoDB’s implementation in general and finally focus on what can go wrong. In a practical demo you see how to find the right balance between performance versus data safety and how to use it in your Java application.
This presentation will demonstrate how you can use the aggregation pipeline with MongoDB similar to how you would use GROUP BY in SQL and the new stage operators coming 3.4. MongoDB’s Aggregation Framework has many operators that give you the ability to get more value out of your data, discover usage patterns within your data, or use the Aggregation Framework to power your application. Considerations regarding version, indexing, operators, and saving the output will be reviewed.
MongoDB Europe 2016 - Debugging MongoDB PerformanceMongoDB
Asya is back, and so is Sherlock Holmes and his techniques to gather and analyze data from your poorly performing MongoDB clusters. In this advanced talk we take a deep look at all the diagnostic data that lives inside MongoDB - how to interrogate and interpret it to help you solve those frustrating performance bottlenecks that we all face occasionally.
In these slide following projects are presented:
* redis_wrap: A Pythonic wrapper that makes it nicer to work with builtin Redis datatypes
* redis_graph: A sample graph database
* redis_simple_queue: A simple queue implemented on Redis list structure
* bitmapist: a powerful analytics library using Redis bitmaps, great for retention and cohort tracking
* fixedlist: a fixed list structure that can optimize timelines (and other things)
* how to use Lua scripting for more advanced data structures and better performance
These are Python projects, but some of them (like bitmapist) have been ported to other languages.
This talk was given to PyCon Belarus on 31 Jan. 2015
Webinar: General Technical Overview of MongoDB for Dev TeamsMongoDB
In this talk we will focus on several of the reasons why developers have come to love the richness, flexibility, and ease of use that MongoDB provides. First we will give a brief introduction of MongoDB, comparing and contrasting it to the traditional relational database. Next, we’ll give an overview of the APIs and tools that are part of the MongoDB ecosystem. Then we’ll look at how MongoDB CRUD (Create, Read, Update, Delete) operations work, and also explore query, update, and projection operators. Finally, we will discuss MongoDB indexes and look at some examples of how indexes are used.
MySQL flexible schema and JSON for Internet of ThingsAlexander Rubin
My presentation at Oracle Open World Conference 2017: Using MySQL Flexible Schema (Document Store/JSON) for IoT
Tuesday, Oct 03, 11:30 a.m. - 12:15 p.m. | Marriott Marquis (Yerba Buena Level) - Salon 14
Storing data from sensors (Internet of Things) may be challenging in many respects, specifically due to the changing nature of the data. For example, if you have a fixed table structure and a sensor will need to store new property, it will be hard to make this change. This session discusses different options for implementing flexible schemas with MySQL 5.7 and MySQL 8.0, using JSON and calculated fields as well as the MySQL Document Store feature. It includes a demo with IoT devices where data is stored in MySQL 8.0.
MongoDB San Francisco 2013: Basic Sharding in MongoDB presented by Brandon Bl...MongoDB
Sharding allows you to distribute load across multiple servers and keep your data balanced across those servers. This session will review MongoDB’s sharding support, including an architectural overview, design principles, and automation.
Webinar: Building Your First App in Node.jsMongoDB
Node.js is one of the fastest growing and most popular application frameworks today. It offers a set of features that is both useful and challenging to modern web and back-end developer. Most engineers think of it as a cool, simple, and fast application framework that makes iterative development easier.
In this webinar we will talk about how Node.js and MongoDB interact, discuss the best ways to get started, and walk through how to use MongoDB and Node.js in a set of use cases.
Webinar: Building Your First App in Node.jsMongoDB
Node.js is one of the fastest growing and most popular application frameworks today. It offers a set of features that is both useful and challenging to modern web and back-end developer. Most engineers think of it as a cool, simple, and fast application framework that makes iterative development easier.
In this webinar we will talk about how Node.js and MongoDB interact, discuss the best ways to get started, and walk through how to use MongoDB and Node.js in a set of use cases.
NoSQL databases are often touted for their performance and whilst it's true that they usually offer great performance out of the box, it still really depends on how you deploy your infrastructure. Dedicated vs cloud? In memory vs on disk? Spindal vs SSD? Replication lag. Multi data centre deployment.
This talk considers all the infrastructure requirements of a successful high performance infrastructure with hints and tips that can be applied to any NoSQL technology. It includes things like OS tweaks, disk benchmarks, replication, monitoring and backups.
Presented at NoSQL Roadshow Berlin 2013 by David Mytton.
Presentation on MongoDB and Node.JS. We describe how to do basic CRUD operations (insert, remove, update, find) how to aggregate using node.js. We also discuss a bit of Meteor, MEAN Stack and other ODMs and projects on Javascript and MongoDB
NoSQL databases are often touted for their performance and whilst it's true that they usually offer great performance out of the box, it still really depends on how you deploy your infrastructure. Dedicated vs cloud? In memory vs on disk? Spindal vs SSD? Replication lag. Multi data centre deployment.
This talk considers all the infrastructure requirements of a successful high performance infrastructure with hints and tips that can be applied to any NoSQL technology. It includes things like OS tweaks, disk benchmarks, replication, monitoring and backups.
A bit of history, frustration-driven development, and why and how we started looking into Puppet at Opera Software. What we're doing, successes, pain points and what we're going to do with Puppet and Config Management next.
MongoDB: Optimising for Performance, Scale & AnalyticsServer Density
MongoDB is easy to download and run locally but requires some thought and further understanding when deploying to production. At scale, schema design, indexes and query patterns really matter. So does data structure on disk, sharding, replication and data centre awareness. This talk will examine these factors in the context of analytics, and more generally, to help you optimise MongoDB for any scale.
Presented at MongoDB Days London 2013 by David Mytton.
OSDC 2012 | Scaling with MongoDB by Ross LawleyNETWAYS
MongoDB's architecture features built-in support for horizontal scalability, and high availability through replica sets. Auto-sharding allows users to easily distribute data across many nodes. Replica sets enable automatic failover and recovery of database nodes within or across data centers. This session will provide an introduction to scaling with MongoDB by one of the developers working on the project.
Beyond PHP - It's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
2. What we will do today?
- Following Replicated Shard
3. What is MongoDB?
● Document-oriented database
● in MongoDB you store JSON-like documents
with dynamic schemas
● Bridge the gap between key-value stores
and relational databases
5. Why MongoDB?
● Document-oriented
● High performance
● High availability
● Easy scalability
● Rich query language + MapReduce
6. Use Cases
Well Suited
● Archiving and event logging
● Document and Content Management Systems
● ECommerce Often in combination with an RDBMS for the final
order processing and accounting
● Gaming Small read/writes are a good fit for MongoDB
● Mobile Specifically, the server-side infrastructure of mobile
systems. Geospatial.
● Operational data store of a web site MongoDB is very good at
real-time inserts, updates, and queries. Specific web use
case examples:
● content management
● comment storage, management, voting
● user registration, profile, session data
● Real-time stats/analytics
7. What is a Document?
{'id':1, 'category_name' : 'Computer'}
8. Mongo Data Model
Key:
'category_name'
Value:
'Computer'
Field: key-value pair
'category_name':'Computer'
Document: set of fields
{'id':1, 'category_name' : 'Computer'}
9. Mongo Data Model
Collection: set of documents
(say categories)
{'id':1, 'category_name' : 'Computer'},
{'id':2, 'category_name' : 'Mobile'},
...
Database: set of collections
categories
products
members
11. First Run
$ mkdir -p /data/db/ Create a data path
and give permissions
$ chown -R mongod:mongod /data/db/
Start mongod deamon
$ mongod [--dbpath /data/db/] &
$ mongo Connect to mongod
> show dbs
> use admin
> show collections
12. Intro
CRUD Operations
> use testdb
switched to db testdb
> j = { name : "deneme" };
{"name" : "deneme"}
> t = { x : 3 };
{ "x" : 3 }
> db.col.save(j);
> db.col.insert(j); /* see the error message */
> db.col.save(t);
> for (var x = 1; x <= 20; x++) db.col.save({x:x, j:x*x})
>
13. Intro
> db.col.find();
{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }
{ "_id" : ObjectId("4c2209fef3924d31102bd84b"), "x" : 3 }
{ "_id" : ObjectId("4f971ab51c3fde3bc55f286f"), "x" : 1, "j" : 1 }
{ "_id" : ObjectId("4f971ab51c3fde3bc55f2870"), "x" : 2, "j" : 4 }
{ "_id" : ObjectId("4f971ab51c3fde3bc55f2871"), "x" : 3, "j" : 9 }
…
has more
> db.col.remove({"x":"3"});
>
16. Master-Slave Rep.
$ mkdir -p /data/db/ms Different data paths
$ mkdir -p /data/db/sl for each instances
$ bin/mongod --master [--port <port>] [--dbpath /data/masterdb/]
$ bin/mongod --slave [--port <port>] --source <masterhostname>[:<port>]
[--dbpath /data/slavedb/]
…
> db.printReplicationInfo() - on master
> db.printSlaveReplicationInfo() - on slave
> use admin
> db.runCommand({resync: 1})
17. ReplicaSet - Voting
Consensus Vote
For a node to be elected primary, it must receive a
majority of votes by the following formula
(floor(5/2)+1)
Arbiters - {_id: 2, host: 'localhost:27019', arbiterOnly:
true}
Reachable Node – heartbeat
Each replica set is limited to 12 total nodes and 7 voting nodes.