The document discusses MongoDB's free cloud monitoring service MMS and how it can be used to monitor and tune MongoDB performance. It provides examples of using MMS metrics and logs to diagnose two cases of performance issues - high replication lag due to insufficient bandwidth and slow queries causing high disk latency. The presentation recommends setting up MMS to collect metrics and receive alerts, and explores some key metrics and tools for log analysis to help identify and address bottlenecks.
Understanding how memory is managed with MongoDB is instrumental in maximizing database performance and hardware utilisation. This talk covers the workings of low level operating system components like the page cache and memory mapped files. We will examine the differences between RAM, SSD and hard disk drives to help you choose the right hardware configuration. Finally, we will learn how to monitor and analyze memory and disk usage using the MongoDB Management Service, linux administration commands and MongoDB commands.
Understanding how memory is managed with MongoDB is instrumental in maximizing database performance and hardware utilisation. This talk covers the workings of low level operating system components like the page cache and memory mapped files. We will examine the differences between RAM, SSD and hard disk drives to help you choose the right hardware configuration. Finally, we will learn how to monitor and analyze memory and disk usage using the MongoDB Management Service, linux administration commands and MongoDB commands.
David Mytton is a MongoDB master and the founder of Server Density. In this presentation David delves deeper into what's discussed in our how to monitor MongoDB tutorial (https://blog.serverdensity.com/monitor-mongodb/), with the aim of taking you through:
Key MongoDB metrics to monitor.
Non-critical MongoDB metrics to monitor.
Alerts to set for MongoDB on production.
Tools for monitoring MongoDB.
Determining the root cause of performance issues is a critical task for Operations. In this webinar, we'll show you the tools and techniques for diagnosing and tuning the performance of your MongoDB deployment. Whether you're running into problems or just want to optimize your performance, these skills will be useful.
Optimizing MongoDB: Lessons Learned at Localyticsandrew311
Tips, tricks, and gotchas learned at Localytics for optimizing MongoDB installs. Includes information about document design, indexes, fragmentation, migration, AWS EC2/EBS, and more.
Understanding and tuning WiredTiger, the new high performance database engine...Ontico
MongoDB 3.0 introduced the concept of different storage engine. The new engine known as WiredTiger introduces document level MVCC locking, compression and a choice between Btree or LSM indexes. In this talk you will learn about the storage engine architecture and specifically WiredTiger, and how to tune and monitor it for best performance.
MongoDB 3.0 представил новый концепт движков хранения. Новый движок известен как WiredTiger и предоставляет новый уровень документов MVCC фикс, компрессию и выбор между Btree или индексами LSM. В этом докладе вы поймете, как тюнить и мониторить архитектуры движка базы данных, а точнее WiredTiger для получения максимальной производительности.
PostgreSQL connections at scale was the presentation by our external speaker at our 8th opensource database meetup. The presentation helps you comprehend on database connections with its cost, gauge the need for a connection pooler, Pgbouncer overview with its features, monitoring, and deployment best practices.
Back to Basics Webinar 6: Production DeploymentMongoDB
This is the final webinar of a Back to Basics series that will introduce you to the MongoDB database. This webinar will guide you through production deployment.
Breaking the Oracle Tie; High Performance OLTP and Analytics Using MongoDBMongoDB
This talk is the story of design and implementation of Marketing Communication Suite at Persado. Marketing Communication Suite is a platform serving tens of customers ranging from telecoms to finance and web properties with persuasion marketing language messaging. Our platform uses a range of technologies with the most important being MongoDB for the online transactional and analytical processing of messages. Topics this talk will be about: MongoDB Aggregation vs. Mapreduce Data Modeling Deployment Architecture Migration Scenarios Hybrid Solutions
Development to Production with Sharded MongoDB ClustersSeveralnines
Severalnines presentation at MongoDB Stockholm Conference.
Presentation covers:
- mongoDB sharding/clustering concepts
- recommended dev/test/prod setups
- how to verify your deployment
- how to avoid downtime
- what MongoDB metrics to watch
- when to scale
Back to Basics Spanish Webinar 3 - Introducción a los replica setsMongoDB
Cómo crear un clúster de producción
Cómo crear un replica set
Cómo MongoDB gestiona la persistencia de los datos y cómo un conjunto de réplicas se recupera automáticamente de todo tipo de fallos
Back to Basics 2017: Introduction to ShardingMongoDB
Sharding is a method for distributing data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations by providing the capability for horizontal scaling.
This presentation will discuss implementing external authentication when using Percona Server for MongoDB and MongoDB Enterprise. It will review authentication using OpenLDAP or ActiveDirectory and ActiveDirectory with Kerberos.
The presentation will also include examples of the configurations required by these external directory services. It will also review the LDAP Authorization features introduced in MongoDB Enterprise 3.4.
Back to Basics Spanish 4 Introduction to shardingMongoDB
Cómo MongoDB amplía el rendimiento de las operaciones de escritura y maneja grandes tamaño de datos
Cómo crear un sharded cluster básico
Cómo elegir una clave de sharding
MongoDB performance tuning and monitoring with MMSNicholas Tang
Using the MongoDB Monitoring Service to monitor your MongoDB instance(s) and track down performance issues - including two real-world examples of how we tracked down problems using MMS to understand the environment, figure out what changed, and help us rapidly drill into a successful diagnosis.
Silicon Valley Code Camp 2014 - Advanced MongoDBDaniel Coupal
MongoDB presentation from Silicon Valley Code Camp 2014.
Walkthrough developing, deploying and operating a MongoDB application, avoiding the most common pitfalls.
David Mytton is a MongoDB master and the founder of Server Density. In this presentation David delves deeper into what's discussed in our how to monitor MongoDB tutorial (https://blog.serverdensity.com/monitor-mongodb/), with the aim of taking you through:
Key MongoDB metrics to monitor.
Non-critical MongoDB metrics to monitor.
Alerts to set for MongoDB on production.
Tools for monitoring MongoDB.
Determining the root cause of performance issues is a critical task for Operations. In this webinar, we'll show you the tools and techniques for diagnosing and tuning the performance of your MongoDB deployment. Whether you're running into problems or just want to optimize your performance, these skills will be useful.
Optimizing MongoDB: Lessons Learned at Localyticsandrew311
Tips, tricks, and gotchas learned at Localytics for optimizing MongoDB installs. Includes information about document design, indexes, fragmentation, migration, AWS EC2/EBS, and more.
Understanding and tuning WiredTiger, the new high performance database engine...Ontico
MongoDB 3.0 introduced the concept of different storage engine. The new engine known as WiredTiger introduces document level MVCC locking, compression and a choice between Btree or LSM indexes. In this talk you will learn about the storage engine architecture and specifically WiredTiger, and how to tune and monitor it for best performance.
MongoDB 3.0 представил новый концепт движков хранения. Новый движок известен как WiredTiger и предоставляет новый уровень документов MVCC фикс, компрессию и выбор между Btree или индексами LSM. В этом докладе вы поймете, как тюнить и мониторить архитектуры движка базы данных, а точнее WiredTiger для получения максимальной производительности.
PostgreSQL connections at scale was the presentation by our external speaker at our 8th opensource database meetup. The presentation helps you comprehend on database connections with its cost, gauge the need for a connection pooler, Pgbouncer overview with its features, monitoring, and deployment best practices.
Back to Basics Webinar 6: Production DeploymentMongoDB
This is the final webinar of a Back to Basics series that will introduce you to the MongoDB database. This webinar will guide you through production deployment.
Breaking the Oracle Tie; High Performance OLTP and Analytics Using MongoDBMongoDB
This talk is the story of design and implementation of Marketing Communication Suite at Persado. Marketing Communication Suite is a platform serving tens of customers ranging from telecoms to finance and web properties with persuasion marketing language messaging. Our platform uses a range of technologies with the most important being MongoDB for the online transactional and analytical processing of messages. Topics this talk will be about: MongoDB Aggregation vs. Mapreduce Data Modeling Deployment Architecture Migration Scenarios Hybrid Solutions
Development to Production with Sharded MongoDB ClustersSeveralnines
Severalnines presentation at MongoDB Stockholm Conference.
Presentation covers:
- mongoDB sharding/clustering concepts
- recommended dev/test/prod setups
- how to verify your deployment
- how to avoid downtime
- what MongoDB metrics to watch
- when to scale
Back to Basics Spanish Webinar 3 - Introducción a los replica setsMongoDB
Cómo crear un clúster de producción
Cómo crear un replica set
Cómo MongoDB gestiona la persistencia de los datos y cómo un conjunto de réplicas se recupera automáticamente de todo tipo de fallos
Back to Basics 2017: Introduction to ShardingMongoDB
Sharding is a method for distributing data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations by providing the capability for horizontal scaling.
This presentation will discuss implementing external authentication when using Percona Server for MongoDB and MongoDB Enterprise. It will review authentication using OpenLDAP or ActiveDirectory and ActiveDirectory with Kerberos.
The presentation will also include examples of the configurations required by these external directory services. It will also review the LDAP Authorization features introduced in MongoDB Enterprise 3.4.
Back to Basics Spanish 4 Introduction to shardingMongoDB
Cómo MongoDB amplía el rendimiento de las operaciones de escritura y maneja grandes tamaño de datos
Cómo crear un sharded cluster básico
Cómo elegir una clave de sharding
MongoDB performance tuning and monitoring with MMSNicholas Tang
Using the MongoDB Monitoring Service to monitor your MongoDB instance(s) and track down performance issues - including two real-world examples of how we tracked down problems using MMS to understand the environment, figure out what changed, and help us rapidly drill into a successful diagnosis.
Silicon Valley Code Camp 2014 - Advanced MongoDBDaniel Coupal
MongoDB presentation from Silicon Valley Code Camp 2014.
Walkthrough developing, deploying and operating a MongoDB application, avoiding the most common pitfalls.
Slides from workshop held on 12/14 in Asbury Park, NJ
http://www.meetup.com/Jersey-Shore-Tech/events/148118762/?gj=ro2_e&a=ro2_gnl&rv=ro2_e&_af_eid=148118762&_af=event
MongoDB 3.2 introduces a host of new features and benefits, including encryption at rest, document validation, MongoDB Compass, numerous improvements to queries and the aggregation framework, and more. To take advantage of these features, your team needs an upgrade plan.
In this session, we’ll walk you through how to build an upgrade plan. We’ll show you how to validate your existing deployment, build a test environment with a representative workload, and detail how to carry out the upgrade. By the end, you should be prepared to start developing an upgrade plan for your deployment.
Webinar: MongoDB and Drupal 8 - Life without SQLMongoDB
Drupal, the popular PHP content management system, has undergone vast changes since its last stable release more than two years ago. While Drupal 7 already allowed for some integration with MongoDB, you will be able to run Drupal without any SQL in Drupal 8. Learn how a few seemingly unrelated initiatives played perfectly into this and how you can use Drupal with MongoDB.
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelDaniel Coupal
MongoDB presentation from Silicon Valley Code Camp 2015.
Walkthrough developing, deploying and operating a MongoDB application, avoiding the most common pitfalls.
MongoDB San Francisco 2013: MongoDB for Collaborative Science presented by D...MongoDB
Scientific data sets are messy (loose data structures, evolving schemas) and large. MongoDB is becoming increasingly popular in the scientific computing space for precisely these reasons. We discuss the advantages of using MongoDB in scientific computing, and describe how we've built the Scientific Computing infrastructure for The Materials Project using MongoDB. We also discuss "warts" in the MongoDB implementation that affect our choices of how and when to use it.
Mastering MongoDB Atlas: Essentials of Diagnostics and Debugging in the Cloud...Mydbops
Diving deep into the essentials of MongoDB Atlas diagnostics and debugging, helps you ensure optimal performance for your cloud-based databases. Join us as we explore key strategies and best practices for effective database management in the cloud environment. Get ready to elevate your MongoDB Atlas experience and unlock the full potential of your cloud databases.
Similar to MongoDB Performance Tuning and Monitoring (20)
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
During this talk we'll navigate through a customer's journey as they migrate an existing MongoDB deployment to MongoDB Atlas. While the migration itself can be as simple as a few clicks, the prep/post effort requires due diligence to ensure a smooth transfer. We'll cover these steps in detail and provide best practices. In addition, we’ll provide an overview of what to consider when migrating other cloud data stores, traditional databases and MongoDB imitations to MongoDB Atlas.
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
MongoDB Kubernetes operator and MongoDB Open Service Broker are ready for production operations. Learn about how MongoDB can be used with the most popular container orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications. A demo will show you how easy it is to enable MongoDB clusters as an External Service using the Open Service Broker API for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB
Humana, like many companies, is tackling the challenge of creating real-time insights from data that is diverse and rapidly changing. This is our journey of how we used MongoDB to combined traditional batch approaches with streaming technologies to provide continues alerting capabilities from real-time data streams.
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB
Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe.
This talk covers:
Common components of an IoT solution
The challenges involved with managing time-series data in IoT applications
Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance.
How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts
At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB
Our clients have unique use cases and data patterns that mandate the choice of a particular strategy. To implement these strategies, it is mandatory that we unlearn a lot of relational concepts while designing and rapidly developing efficient applications on NoSQL. In this session, we will talk about some of our client use cases, the strategies we have adopted, and the features of MongoDB that assisted in implementing these strategies.
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB
Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch".
This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
When you need to model data, is your first instinct to start breaking it down into rows and columns? Mine used to be too. When you want to develop apps in a modern, agile way, NoSQL databases can be the best option. Come to this talk to learn how to take advantage of all that NoSQL databases have to offer and discover the benefits of changing your mindset from the legacy, tabular way of modeling data. We’ll compare and contrast the terms and concepts in SQL databases and MongoDB, explain the benefits of using MongoDB compared to SQL databases, and walk through data modeling basics so you feel confident as you begin using MongoDB.
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB
Query performance should be the unsung hero of an application, but without proper configuration, can become a constant headache. When used properly, MongoDB provides extremely powerful querying capabilities. In this session, we'll discuss concepts like equality, sort, range, managing query predicates versus sequential predicates, and best practices to building multikey indexes.
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB
Aggregation pipeline has been able to power your analysis of data since version 2.2. In 4.2 we added more power and now you can use it for more powerful queries, updates, and outputting your data to existing collections. Come hear how you can do everything with the pipeline, including single-view, ETL, data roll-ups and materialized views.
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
MongoDB Atlas Data Lake is a new service offered by MongoDB Atlas. Many organizations store long term, archival data in cost-effective storage like S3, GCP, and Azure Blobs. However, many of them do not have robust systems or tools to effectively utilize large amounts of data to inform decision making. MongoDB Atlas Data Lake is a service allowing organizations to analyze their long-term data to discover a wealth of information about their business.
This session will take a deep dive into the features that are currently available in MongoDB Atlas Data Lake and how they are implemented. In addition, we'll discuss future plans and opportunities and offer ample Q&A time with the engineers on the project.
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB
Virtual assistants are becoming the new norm when it comes to daily life, with Amazon’s Alexa being the leader in the space. As a developer, not only do you need to make web and mobile compliant applications, but you need to be able to support virtual assistants like Alexa. However, the process isn’t quite the same between the platforms.
How do you handle requests? Where do you store your data and work with it to create meaningful responses with little delay? How much of your code needs to change between platforms?
In this session we’ll see how to design and develop applications known as Skills for Amazon Alexa powered devices using the Go programming language and MongoDB.
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB
aux Core Data, appréciée par des centaines de milliers de développeurs. Apprenez ce qui rend Realm spécial et comment il peut être utilisé pour créer de meilleures applications plus rapidement.
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB
Il n’a jamais été aussi facile de commander en ligne et de se faire livrer en moins de 48h très souvent gratuitement. Cette simplicité d’usage cache un marché complexe de plus de 8000 milliards de $.
La data est bien connu du monde de la Supply Chain (itinéraires, informations sur les marchandises, douanes,…), mais la valeur de ces données opérationnelles reste peu exploitée. En alliant expertise métier et Data Science, Upply redéfinit les fondamentaux de la Supply Chain en proposant à chacun des acteurs de surmonter la volatilité et l’inefficacité du marché.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
MongoDB Performance Tuning and Monitoring
1. Technical Services Engineer, Palo Alto, CA @ MongoDB
Daniel Coupal
#MongoDB
Performance Tuning and
Monitoring Using MMS
2. 2 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Agenda
1. What and who is MMS for?
2. Performance and monitoring examples
3. Setting it up and getting around
4. Wrapping up
5. Q&A
4. 4 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
What is MMS?
The MongoDB Management Service
• a free Cloud service for monitoring and
managing your MongoDB clusters
• or available to run On-Prem for
customers with the Standard or
Enterprise Subscriptions
• tool that makes MongoDB easier to use
5. 5 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Who is MMS for?
• Developers
– Track bottlenecks
• Ops Team
– Monitor health of the clusters
– Backup databases
– Automate updates and add capacity
• MongoDB Technical Service Team
“For years, MongoDB has been planning to do monitoring right.“
“We want the Ops People to have the same enthusiasm as the Developers.”
6. 6 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
What is in MMS?
A. Monitoring
1. Cloud: Sept 2011
2. On-Prem: July 2013
B. Backups
1. Cloud: April 2013
2. On-Prem: April 2014
C. Automation
1. Cloud: April 2014 (Beta)
7. 7 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
What is in MMS monitoring?
A. Metric Collection and Reporting
B. Alerting
(Email, SMS, PagerDuty, HipChat, SNMP)
C. Event Tracking
D. Database Stats
E. Hardware Stats
F. Logs and Profile Data
8. 8 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
A. Metric Collection and Reporting
15. 15 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
How to do performance tuning?
1. Assess the problem and establish acceptable behavior
2. Measure the current performance
3. Find the bottleneck*
4. Remove the bottleneck
5. Re-test to confirm
6. Repeat
* - (This is often the hard part)
(Adapted from http://en.wikipedia.org/wiki/Performance_tuning )
16. 16 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Pro-Tip: know thyself
You have to recognize normal to know when it isn’t.
Source: http://www.flickr.com/photos/skippy/6853920/
17. 17 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Some handy metrics to watch
• Memory usage
• Opcounters
• Lock %
• Queues
• Background flush average
• Replication
– Replication oplog window
– Replication lag
18. 18 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Fun fact: oplog idempotency
• Operations in the oplog only affect the value once, so
they can be run multiple times safely.
• Examples
– If you increment n from 2 to 3, n = 3 is fine; n + 1 is not.
– Updating all documents that match a query is not, list of
documents to update is.
• Frequent, large updates means a big oplog to sync.
• Updates that change a set mean writing the entire
new version of the set to the oplog.
19. 19 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 1: replication lag
Scenario:
Customer reports 150,000s of replication lag.
Equals to almost 2 days of lag!
20. 20 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 1: replication lag
Some common causes of replication lag:
• Secondaries underspec’d vs primaries
• Access patterns between primary and
secondaries
• Insufficient bandwidth
• Foreground index builds on secondaries
“…when you have eliminated the impossible, whatever remains,
however improbable, must be the truth…” -- Sherlock Holmes
Sir Arthur Conan Doyle, The Sign of the Four
21. 21 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 1: replication lag
Example:
• ~1500 ops per minute (opcounters)
• 0.1 MB per object (average object size, local db)
• ~1500 ops/min / 60 seconds * 0.1 MB/op * 8b/B
=~ 20 mbps required bandwidth
Huge updates (oplog is idempotent) translated to 30
mbps, while they only had 10 mbps
22. 22 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Lesson: remember to use alerts!
Don’t wait until your secondaries fall off
your oplog!
23. 23 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Scenario:
• User-facing web application with social functionality.
• Customer was seeing significant performance
degradation after adding and removing an index from
their replicaset.
• Their replicaset had 2 visible data-bearing nodes,
each on real hardware, with dedicated 15K RPM disks
and a significant amount of RAM.
• Why were things slow?
24. 24 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Opcounters: queries rose a bit, however writes
were flat…
25. 25 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Background flush average: went up
considerably!
26. 26 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Queues: also went up considerably!
27. 27 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Journal stats: went up much higher than
the ops…
28. 28 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Connections: also went up…
29. 29 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Background flush average: consistent
until then
30. 30 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Opcounters: interesting… around July 9th
31. 31 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Page faults: something’s going on!
32. 32 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Local DB average object size: growing!
33. 33 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Now what?
Time to analyze the logs
What query or queries were going crazy?
And what sort of query would grow in size
without growing significantly in volume?
Remember:
Growing disk latency (caused by page faults?)
And journal/oplog entries growing even
though writes (inserts/updates) were flat.
34. 34 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Log analysis
The best tools for analyzing MongoDB
logs are included in mtools*:
• mlogfilter
filter logs for slow queries, collection scans, …
• mplotqueries
graph query response times and volumes
* https://github.com/rueckstiess/mtools
35. 35 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Log analysis (example syntax)
Show me queries that took more than
1000 ms from 6 am to 6 pm:
mlogfilter mongodb.log --from 06:00 --to
18:00 --slow 1000 > mongodb-filtered.log
Now, graph those queries:
mplotqueries --logscale mongodb-filtered.log
36. 36 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
37. 37 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Filter more!
--operation
Logarithmic!
--logscale
38. 38 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Sample query
Wed Jul 17 14:16:44 [conn60560] update x.y query: { e:
”[id1]" } update: { $addToSet: { fr: ”[id2]" } }
nscanned:1 nupdated:1 keyUpdates:1 locks(micros) w:889
6504ms
6.5 seconds to add a single value to a set!
39. 39 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
http://docs.mongodb.org/manual/reference/operator/addToSet/
The $addToSet operator adds a value to an array only if
the value is not in the array already. If the value is in the
array, $addToSet returns without modifying the array.
Consider the following example:
db.collection.update({field:value}, {$addToSet: {field:value1}});
Here, $addToSet appends value1 to the array stored in
field, only if value1 is not already a member of this array.
40. 40 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
https://jira.mongodb.org/browse/SERVER-8192
“IndexSpec::getKeys() finds the set of index keys for a given
document and index key spec. It's used when inserting / updating /
deleting a document to update the index entries, and also for
performing in memory sorts, deduping $or clauses and for other
purposes.
Right now extracting 10k elements from a nested object field within
an array takes on the order of seconds on a decently fast machine.
We could see how much we can optimize the implementation.”
41. 41 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
What else?!
• Wed Jul 17 14:11:59 [conn56541] update x.y query: {
e: ”[id1]" } update: { $addToSet: { fr: ”[id2]" } }
nscanned:1 nmoved:1 nupdated:1 keyUpdates:0
locks(micros) w:85145 11768ms
• Almost 12 seconds!
This time, there’s “nmoved:1”, too.
This means a document was moved on
disk, it outgrew the space allocated for it.
42. 42 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Wait, there’s more!
• Wed Jul 17 13:40:14 [conn28600] query x.y [snip]
ntoreturn:16 ntoskip:0 nscanned:16779 scanAndOrder:1
keyUpdates:0 numYields: 906 locks(micros) r:46877422
nreturned:16 reslen:6948 38172ms
• 38 seconds! Scanned 17k documents,
returned 16.
43. 43 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
What next?
Short term fix: disable the new feature for
the heaviest users! After that:
• rework the code to avoid $addToSet
• add indexes for queries scanning collections
• use powerOf2Sizes
(http://docs.mongodb.org/manual/reference/command/collMod/)
to reduce fragmentation and document moves
44. 44 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Example 2: slow performance
Did it work?
(Yes.)
(So far. ;) )
45. 45 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Monitoring: watch for warnings
MMS warns you if your systems
a) are running outdated versions
b) have startup warnings
c) if a mongod is publicly visible
Don’t ignore these warnings!
47. 47 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Setting up monitoring for the Cloud
http://mms.mongodb.com/help/monitoring/tutorial/
• Setup an account (Free)
• Install the agent
– Download the binary
– Set the API key in the configuration file
• Add your hosts
– Add a seed host, MMS will discover the cluster
• Optional: hardware stats through Munin-node
• Optional: enable logging and profiling
48. 48 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Setting up monitoring “On-Prem”
• Available (included) to Customers with
– Enterprise subscription
– OR Standard subscription
• One host to run the server for up to 400 hosts
• One RPM/Deb package to install
• Installing the backup requires more planning
and more resources (hosts, disks, …)
– The alternative is use MMS Backups in the Cloud,
we removed the complexity for you.
49. 49 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
MMS Monitoring Architecture
50. 50 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
List of monitored hosts
51. 51 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Notes
• Agent written in Go
• Failover: run multiple agents (1 primary)
• Hosts: use CNAMEs, especially on AWS!
• You can use a group per environment (each needs an
agent)
• Connections are over SSL
• On-Prem solution for Enterprise or Standard
customers that don’t want to use the hosted service
• Makes it easier for the technical services to help you!
53. 53 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
The Technical Services Team
• We are here to make you successful,
open tickets
– We support 1000s of customers
– Another customer may had the same issue
as you
• Throw any problems to the Technical
Services Team
54. 54 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
What’s next for MMS?
• Continuous updates
– Cloud: ~every 3 weeks
– On-Prem: ~every 3 months
• Automation service
• A lot of more information at:
MongoDB World 2014 @
NYC on June 23-25
55. 55 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Summary
• MMS is a great, free service
• Setup is easy
• Metrics and graphs are awesome
• Preventing failures even more awesome
• Using MMS makes it easier for the
Technical Services Team to help you
• Monitor your DBs like a Pro, use MMS
58. 58 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Other MMS References
• Tons of webinars and other presentations
http://www.mongodb.com/presentations
• Five MMS Monitoring Alerts to Keep Your MongoDB Deployment on Track
http://www.mongodb.com/blog/post/five-mms-monitoring-alerts-keep-your-
mongodb-deployment-track
• Digging into the meaning of some metrics
– Lock % : http://blog.mms.mongodb.com/post/78650784046/learn-about-lock-percentage-concurrency-
in-mongodb
– Replication Oplog Window : http://blog.mms.mongodb.com/post/77279440964/replica-set-health-is-
more-than-just-replication-lag
• How to fix some warnings reported by MMS
– http://docs.mongodb.org/manual/administration/production-notes/
(includes readahead settings)
59. 59 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Examining memory and disk
Memory: resident vs virtual vs non-mapped
60. 60 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Examining memory and disk
Page faults and Record Stats
61. 61 MongoDB Performance Tuning and Monitoring Using MMS, Daniel Coupal
Examining memory and disk
Background flush and Disk IO
(Check out http://www.wmarrow.com/strcalc/ )