This document provides an introduction and overview of MongoDB. It begins with basic introductions to MongoDB and its terminology. It then covers MongoDB features like indexing, replication, auto-sharding, and monitoring. It also provides tips on topics like vertical scaling, building indexes, backups, and rollbacks. The document uses several examples to illustrate MongoDB concepts and operations.
Groovy speech I held last year for introducing a new JVM language as substitute of Java. Easy and intuitive, it offers new features unknow to its parent yet.
Scalable XQuery Processing with Zorba on top of MongoDBWilliam Candillon
Since a couple of years, the NoSQL movement has developed a variety of open-source document stores. Most of them focus on high availability, horizontal scalability, and are designed to run on commodity hardware. These products have gained great traction in the industry to store large amounts of flexible data (mostly JSON). In the meantime, XQuery has evolved to a standardized, full-fledged programming language for XML with native support for complex queries, indexes, updates, full-text search, and scripting. Moreover, JSON has recently been added as a first-level datatype into the language. As of today, it is without doubt the most robust and productive technology to process flexible data.
The aim of this talk is to showcase the benefits that can be achieved by integrating the Zorba XQuery Processor with MongoDB. We will introduce the 28msec platform that seamlessly stores, indexes, and manages flexible data entirely in XQuery. The data itself is stored in MongoDB. The platform leverages MongoDB’s indexes, sharding, and consistency guarantees to scale-out horizontally. The talk will conclude by showing a benchmark of the platform and discuss perspectives of the outlined approach.
MongoDB + Java - Everything you need to know Norberto Leite
Learn everything you need to know to get started building a MongoDB-based app in Java. We'll explore the relationship between MongoDB and various languages on the Java Virtual Machine such as Java, Scala, and Clojure. From there, we'll examine the popular frameworks and integration points between MongoDB and the JVM including Spring Data and object-document mappers like Morphia.
Groovy speech I held last year for introducing a new JVM language as substitute of Java. Easy and intuitive, it offers new features unknow to its parent yet.
Scalable XQuery Processing with Zorba on top of MongoDBWilliam Candillon
Since a couple of years, the NoSQL movement has developed a variety of open-source document stores. Most of them focus on high availability, horizontal scalability, and are designed to run on commodity hardware. These products have gained great traction in the industry to store large amounts of flexible data (mostly JSON). In the meantime, XQuery has evolved to a standardized, full-fledged programming language for XML with native support for complex queries, indexes, updates, full-text search, and scripting. Moreover, JSON has recently been added as a first-level datatype into the language. As of today, it is without doubt the most robust and productive technology to process flexible data.
The aim of this talk is to showcase the benefits that can be achieved by integrating the Zorba XQuery Processor with MongoDB. We will introduce the 28msec platform that seamlessly stores, indexes, and manages flexible data entirely in XQuery. The data itself is stored in MongoDB. The platform leverages MongoDB’s indexes, sharding, and consistency guarantees to scale-out horizontally. The talk will conclude by showing a benchmark of the platform and discuss perspectives of the outlined approach.
MongoDB + Java - Everything you need to know Norberto Leite
Learn everything you need to know to get started building a MongoDB-based app in Java. We'll explore the relationship between MongoDB and various languages on the Java Virtual Machine such as Java, Scala, and Clojure. From there, we'll examine the popular frameworks and integration points between MongoDB and the JVM including Spring Data and object-document mappers like Morphia.
Tokyo Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Records are organized in hash table, B+ tree, or fixed-length array.
The latest version of my PostgreSQL introduction for IL-TechTalks, a free service to introduce the Israeli hi-tech community to new and interesting technologies. In this talk, I describe the history and licensing of PostgreSQL, its built-in capabilities, and some of the new things that were added in the 9.1 and 9.2 releases which make it an attractive option for many applications.
You may all know that JSON is a subset of JavaScript, but… Did you know that HTML5 implements NoSQL databases? Did you know that JavaScript was recommended for REST by HTTP co-creator Roy T. Fielding himself? Did you know that map & reduce are part of the native JavaScript API? Did you know that most NoSQL solutions integrate a JavaScript engine? CouchDB, MongoDB, WakandaDB, ArangoDB, OrientDB, Riak…. And when they don’t, they have a shell client which does. The story of NoSQL and JavaScript goes beyond your expectations and opens more opportunities than you might imagine… What better match could you find than a flexible and dynamic language for schemaless databases? Isn’t an event-driven language what you’ve been waiting for to manage consistency? When NoSQL doesn’t come to JavaScript, JavaScript comes to NoSQL. And does it very well.
SQL? NoSQL? NewSQL?!? What's a Java developer to do? - PhillyETE 2012Chris Richardson
The database world is undergoing a major upheaval. NoSQL databases such as MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offering significantly better scalability and performance. But these databases have a very different and unfamiliar data model and APIs as well as a limited transaction model. Moreover, the relational world is fighting back with so-called NewSQL databases such as VoltDB, which by using a radically different architecture offers high scalability and performance as well as the familiar relational model and ACID transactions. Sounds great but unlike the traditional relational database you can’t use JDBC and must partition your data.
In this presentation you will learn about popular NoSQL databases – MongoDB, and Cassandra – as well at VoltDB. We will compare and contrast each database’s data model and Java API using NoSQL and NewSQL versions of a use case from the book POJOs in Action. We will learn about the benefits and drawbacks of using NoSQL and NewSQL databases.
MongoDB: Optimising for Performance, Scale & AnalyticsServer Density
MongoDB is easy to download and run locally but requires some thought and further understanding when deploying to production. At scale, schema design, indexes and query patterns really matter. So does data structure on disk, sharding, replication and data centre awareness. This talk will examine these factors in the context of analytics, and more generally, to help you optimise MongoDB for any scale.
Presented at MongoDB Days London 2013 by David Mytton.
How does a full-text search engine works? How is the index built and searched? Can I use PostgreSQL as a full-text search engine or should I go for a more specialised solution? How does one configure and use PostgreSQL search?
This presentation covers all those aspects, based on the work we did to index teowaki.com. It was presented at PgConf EU 2014 in Madrid
Map/Confused? A practical approach to Map/Reduce with MongoDBUwe Printz
Talk given at MongoDb Munich on 16.10.2012 about the different approaches in MongoDB for using the Map/Reduce algorithm. The talk compares the performance of built-in MongoDB Map/Reduce, group(), aggregate(), find() and the MongoDB-Hadoop Adapter using a practical use case.
Whether running load tests or migrating historic data, loading data directly into Cassandra can be very useful to bypass the system’s write path.
In this webinar, we will look at how data is stored on disk in sstables, how to generate these structures directly, and how to load this data rapidly into your cluster using sstableloader. We'll also review different use cases for when you should and shouldn't use this method.
Tokyo Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Records are organized in hash table, B+ tree, or fixed-length array.
The latest version of my PostgreSQL introduction for IL-TechTalks, a free service to introduce the Israeli hi-tech community to new and interesting technologies. In this talk, I describe the history and licensing of PostgreSQL, its built-in capabilities, and some of the new things that were added in the 9.1 and 9.2 releases which make it an attractive option for many applications.
You may all know that JSON is a subset of JavaScript, but… Did you know that HTML5 implements NoSQL databases? Did you know that JavaScript was recommended for REST by HTTP co-creator Roy T. Fielding himself? Did you know that map & reduce are part of the native JavaScript API? Did you know that most NoSQL solutions integrate a JavaScript engine? CouchDB, MongoDB, WakandaDB, ArangoDB, OrientDB, Riak…. And when they don’t, they have a shell client which does. The story of NoSQL and JavaScript goes beyond your expectations and opens more opportunities than you might imagine… What better match could you find than a flexible and dynamic language for schemaless databases? Isn’t an event-driven language what you’ve been waiting for to manage consistency? When NoSQL doesn’t come to JavaScript, JavaScript comes to NoSQL. And does it very well.
SQL? NoSQL? NewSQL?!? What's a Java developer to do? - PhillyETE 2012Chris Richardson
The database world is undergoing a major upheaval. NoSQL databases such as MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offering significantly better scalability and performance. But these databases have a very different and unfamiliar data model and APIs as well as a limited transaction model. Moreover, the relational world is fighting back with so-called NewSQL databases such as VoltDB, which by using a radically different architecture offers high scalability and performance as well as the familiar relational model and ACID transactions. Sounds great but unlike the traditional relational database you can’t use JDBC and must partition your data.
In this presentation you will learn about popular NoSQL databases – MongoDB, and Cassandra – as well at VoltDB. We will compare and contrast each database’s data model and Java API using NoSQL and NewSQL versions of a use case from the book POJOs in Action. We will learn about the benefits and drawbacks of using NoSQL and NewSQL databases.
MongoDB: Optimising for Performance, Scale & AnalyticsServer Density
MongoDB is easy to download and run locally but requires some thought and further understanding when deploying to production. At scale, schema design, indexes and query patterns really matter. So does data structure on disk, sharding, replication and data centre awareness. This talk will examine these factors in the context of analytics, and more generally, to help you optimise MongoDB for any scale.
Presented at MongoDB Days London 2013 by David Mytton.
How does a full-text search engine works? How is the index built and searched? Can I use PostgreSQL as a full-text search engine or should I go for a more specialised solution? How does one configure and use PostgreSQL search?
This presentation covers all those aspects, based on the work we did to index teowaki.com. It was presented at PgConf EU 2014 in Madrid
Map/Confused? A practical approach to Map/Reduce with MongoDBUwe Printz
Talk given at MongoDb Munich on 16.10.2012 about the different approaches in MongoDB for using the Map/Reduce algorithm. The talk compares the performance of built-in MongoDB Map/Reduce, group(), aggregate(), find() and the MongoDB-Hadoop Adapter using a practical use case.
Whether running load tests or migrating historic data, loading data directly into Cassandra can be very useful to bypass the system’s write path.
In this webinar, we will look at how data is stored on disk in sstables, how to generate these structures directly, and how to load this data rapidly into your cluster using sstableloader. We'll also review different use cases for when you should and shouldn't use this method.
This presentation will demonstrate how you can use the aggregation pipeline with MongoDB similar to how you would use GROUP BY in SQL and the new stage operators coming 3.4. MongoDB’s Aggregation Framework has many operators that give you the ability to get more value out of your data, discover usage patterns within your data, or use the Aggregation Framework to power your application. Considerations regarding version, indexing, operators, and saving the output will be reviewed.
Building a Scalable Distributed Stats Infrastructure with Storm and KairosDBCody Ray
Building a Scalable Distributed Stats Infrastructure with Storm and KairosDB
Many startups collect and display stats and other time-series data for their users. A supposedly-simple NoSQL option such as MongoDB is often chosen to get started... which soon becomes 50 distributed replica sets as volume increases. This talk describes how we designed a scalable distributed stats infrastructure from the ground up. KairosDB, a rewrite of OpenTSDB built on top of Cassandra, provides a solid foundation for storing time-series data. Unfortunately, though, it has some limitations: millisecond time granularity and lack of atomic upsert operations which make counting (critical to any stats infrastructure) a challenge. Additionally, running KairosDB atop Cassandra inside AWS brings its own set of challenges, such as managing Cassandra seeds and AWS security groups as you grow or shrink your Cassandra ring. In this deep-dive talk, we explore how we've used a mix of open-source and in-house tools to tackle these challenges and build a robust, scalable, distributed stats infrastructure.
Stata cheat sheet: programming. Co-authored with Tim Essam (linkedin.com/in/timessam). See all cheat sheets at http://bit.ly/statacheatsheets. Updated 2016/06/04
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
MongoDB 在盛大大数据量下的应用
1. MongoDB 在盛大大数据
量下的应用
郭理靖 @sn da ; gu ol ijing@ gm a il .c om ;
11年10月19日星期三
2. Sections
Basic introduction to MongoDB
Monitor MongoDB
Backup and Rollback of MongoDB
Case Study
11年10月19日星期三
3. What’s MongoDB
MongoDB (from "humongous") is a
scalable, high-performance, open
source, document-oriented
database.
Current version:2.0.0
11年10月19日星期三
6. Features
Document-oriented storage (JSON)
Full Index Support (Indexes)
Replication & High availability
Rich Document-based Queries and Updates
Map/Reduce
Auto-Sharding and GridFS
11年10月19日星期三
7. Terminology
RDBMS MongoDB
Table Collection
View/Row(s) JSON Document
Column name Field name
Index Index
Join Embedding & Linking
Partition Shard
Partition Key Shard Key
11年10月19日星期三
8. SELECT
mySQL db.runCommand({
MongoDB
Dim1, Dim2, 1 mapreduce: "DenormAggCollection",
SUM(Measure1) AS MSum, query: {
2
COUNT(*) AS RecordCount, filter1: { '$in': [ 'A', 'B' ] },
AVG(Measure2) AS MAvg, 3 filter2: 'C',
MIN(Measure1) AS MMin filter3: { '$gt': 123 }
MAX(CASE },
WHEN Measure2 < 100 4 map: function() { emit(
THEN Measure2 { d1: this.Dim1, d2: this.Dim2 },
END) AS MMax { msum: this.measure1, recs: 1, mmin: this.measure1,
FROM DenormAggTable mmax: this.measure2 < 100 ? this.measure2 : 0 }
WHERE (Filter1 IN (’A’,’B’)) );},
AND (Filter2 = ‘C’) 5 reduce: function(key, vals) {
AND (Filter3 > 123) var ret = { msum: 0, recs: 0, mmin: 0, mmax: 0 };
GROUP BY Dim1, Dim2 1 for(var i = 0; i < vals.length; i++) {
HAVING (MMin > 0) ret.msum += vals[i].msum;
ORDER BY RecordCount DESC ret.recs += vals[i].recs;
LIMIT 4, 8 if(vals[i].mmin < ret.mmin) ret.mmin = vals[i].mmin;
if((vals[i].mmax < 100) && (vals[i].mmax > ret.mmax))
ret.mmax = vals[i].mmax;
}
1 Grouped dimension columns are pulled return ret;
out as keys in the map function, },
reducing the size of the working set. finalize: function(key, val) {
6
2 Measures must be manually aggregated. 7 val.mavg = val.msum / val.recs;
return val;
3 Aggregates depending on record counts
},
Revision 4, Created 2010-03-06
Rick Osborne, rickosborne.org
must wait until finalization.
4 Measures can use procedural logic.
out: 'result1',
verbose: true
5 Filters have an ORM/ActiveRecord- });
looking style.
db.result1.
6 Aggregate filtering must be applied to
the result set, not in the map/reduce.
find({ mmin: { '$gt': 0 } }).
7 Ascending: 1; Descending: -1
sort({ recs: -1 }).
skip(4).
limit(8);
11年10月19日星期三
9. Indexes
Unique Indexes or Duplicate Values Indexes
Index of Single Key(Embedded key, Document
key) .Default index, ‘_id’: MongoID(GUID)
Index of Compound
Keys(db.user.ensureIndex({credit : -1, name: 1}))
Sparse Index. (A sparse index can only have one
field) //after 1.75
11年10月19日星期三
10. Geo Indexes
Geospatial index supported (good news for LBS)
db.c.find( {a:[50,50]} ) using index {a:’2d’}
db.c.find( {a:{$near:[50,50]}} ) using index
{a:’2d’}
Results are sorted closest - farthest
db.c.find( {a:{$within:{$box:[[40,40],
[60,60]]}}} ) using index {a:’2d’}
11年10月19日星期三
11. Atomic Operations
$set $unset $inc
$push - append a value to an array
$pushAll - append several values to an array
$pull - remove a value(s) from an existing array
$pullAll - remove several value(s) from an
existing array
$bit - bitwise operations
11年10月19日星期三
12. Tips
Update requires a write lock.
Write lock is “greedy” right now
Update in big collection is slow and block other
writes during the update
Query it first then update it will reduce the lock
time
11年10月19日星期三
14. Replica set
Replica sets are basically master/slave
replication, adding automatic failover and
automatic recovery of member nodes.
A Replica Set consists of two or more nodes that
are copies of each other. (i.e.: replicas)
The Replica Set automatically elects a Primary
(master) if there is no primary currently
available.
11年10月19日星期三
15. Replica set
Automated fail-over
Distribute read load
Simplified maintenance
A Replica Set can have passive members that
cannot become primary (for reading and backup)
A Replica Set can have delayed nodes (in case of
user error)
11年10月19日星期三
16. step down primary
If you step down primary manually when
primary is heavy, the replica set may can not
elect new primary. (it is fixed in 1.8.0)
May lose data when step down primary
manually. (sometimes, not test in 1.8.0 yet)
Clients may not know what happened.
11年10月19日星期三
17. What’s happened when
primary is down?
Replica set will elect new primary, but how?
what’s arbiter?
operator time
votes
priority
may lose data when primary is down
11年10月19日星期三
23. mongostat
inserts/s - # of inserts per second
query/s - # of queries per second
update/s - # of updates per second
delete/s - # of deletes per second
getmore/s - # of get mores (cursor batch)
per second
11年10月19日星期三
24. mongostat
command/s - # of commands per second
flushes/s - # of fsync flushes per second
mapped - amount of data mmaped (total data
size) megabytes
visze - virtual size of process in megabytes
res - resident size of process in megabytes
11年10月19日星期三
25. mongostat
faults/s - # of pages faults/sec (linux only)
locked - percent of time in global write lock
idx miss - percent of btree page misses
(sampled)
q t|r|w - lock queue lengths (total|read|
write)
conn - number of open connections
11年10月19日星期三
27. MongoDB with Numa
WARNING: You are running on a NUMA
machine.
We suggest launching mongod like this
to avoid performance problems:
numactl --interleave=all mongod
[other options]
11年10月19日星期三
28. Security
Auth is supported in Replica set/Master-Slaves,
Auto-Sharding
bind_ip
start auth in RS may have bug.(use http to check
the status and try restart mongods)
11年10月19日星期三
29. Memory
Memory-mapping
Keep indexes in memory
db.test.stats()
db.test.totalIndexSize()
RAM > indexes + hot data = better performance
11年10月19日星期三
30. some tips
No transactions
Fast-N-Loose by default (no safe/w/GLE)
Indexing order matters; query optimizer helps
One write thread,many query threads
Memory Mapped Data Files
BSON everywhere
11年10月19日星期三
31. Oplog of MongoDB
Oplog in Replica set:
local.system.replset: store config of replica set,
use rs.conf() to detail
local.oplog.rs: capped collection, use --
oplogSize to set size
local.replset.minvalid: for sync status
NO INDEX
11年10月19日星期三
33. Oplog of MongoDB
{ts:{},h{}, op:{},ns:{},o:{},o2:{} }
Ts: 8 bytes of time stamp by 4 bytes Unix timestamp + 4 bytes since the count said.
This value is very important, in the elections (such as master is down unit), new primary would
choose the biggest change as the ts new primary.
Op: 1 bytes of operation types, such as I said insert, d said the delete.
In the namespace ns: the operation.
O: the operation is corresponding to the document, that is, the content of the current operation
(such as the update operation to update when the fields and value)
O2: perform update operation in the condition is limited to the update, where to have this property
Among them, can be the op several situations:
h:// hash mongodb use to make sure we are reading the right flow of ops and aren't on an out-of-
date "fork"
11年10月19日星期三
35. Feature of Oplog
Replay same oplog is harmless
Replay old oplogs is harmless if the ts in last one
is newer than current database
11年10月19日星期三
36. Back up
mongoexport mongoimport
mongodump mongorestore
data consistency? use --oplog
use incremental back up
read oplog and replay (wordnik tools)
11年10月19日星期三
37. Rollback MongoDB
Use snapshot + oplog
Use delayed secondary +
oplog
11年10月19日星期三
38. Vertical Scala
Old server 8G RAM +200G dis
New server 64G RAM + 2T disk
What can I do if my service should always keep
online ?
11年10月19日星期三
39. Vertical Scala
You’re luck if using Replica set
move everything(include ip config) from
secondary to new service
step down primary, do the same thing above
well, problem solved
11年10月19日星期三
40. Build New Index
Build a index is quite easy.....
Build a index in a huge collections, such as
500GB ,and database is very buy, is it easy?
11年10月19日星期三
41. Build New Index
Take a snapshot of current Mongodb service
Build new index in snapshot
replay oplog of working Mongod
Replace the working Mongod
11年10月19日星期三
42. Best practices
Replica set: one for primary, one for secondary,
one for incremental back up with a arbiter.
11年10月19日星期三
43. Q&A
www.mongoic.com is online !
We’re hiring...
11年10月19日星期三