MySQL flexible schema and JSON for Internet of ThingsAlexander Rubin
My presentation at Oracle Open World Conference 2017: Using MySQL Flexible Schema (Document Store/JSON) for IoT
Tuesday, Oct 03, 11:30 a.m. - 12:15 p.m. | Marriott Marquis (Yerba Buena Level) - Salon 14
Storing data from sensors (Internet of Things) may be challenging in many respects, specifically due to the changing nature of the data. For example, if you have a fixed table structure and a sensor will need to store new property, it will be hard to make this change. This session discusses different options for implementing flexible schemas with MySQL 5.7 and MySQL 8.0, using JSON and calculated fields as well as the MySQL Document Store feature. It includes a demo with IoT devices where data is stored in MySQL 8.0.
MySQL flexible schema and JSON for Internet of ThingsAlexander Rubin
My presentation at Oracle Open World Conference 2017: Using MySQL Flexible Schema (Document Store/JSON) for IoT
Tuesday, Oct 03, 11:30 a.m. - 12:15 p.m. | Marriott Marquis (Yerba Buena Level) - Salon 14
Storing data from sensors (Internet of Things) may be challenging in many respects, specifically due to the changing nature of the data. For example, if you have a fixed table structure and a sensor will need to store new property, it will be hard to make this change. This session discusses different options for implementing flexible schemas with MySQL 5.7 and MySQL 8.0, using JSON and calculated fields as well as the MySQL Document Store feature. It includes a demo with IoT devices where data is stored in MySQL 8.0.
Cluj Big Data Meetup - Big Data in PracticeSteffen Wenz
At the Cluj Big Data Meetup, we shared some insights into TrustYou's big data tech stack. Also we introduced two tools which we've found useful in our production jobs: Apache Pig and Luigi.
Also check out the code samples on GitHub: https://github.com/trustyou/meetups/tree/master/big-data
This presentation will demonstrate how you can use the aggregation pipeline with MongoDB similar to how you would use GROUP BY in SQL and the new stage operators coming 3.4. MongoDB’s Aggregation Framework has many operators that give you the ability to get more value out of your data, discover usage patterns within your data, or use the Aggregation Framework to power your application. Considerations regarding version, indexing, operators, and saving the output will be reviewed.
Mythbusting: Understanding How We Measure the Performance of MongoDBMongoDB
Benchmarking, benchmarking, benchmarking. We all do it, mostly it tells us what we want to hear but often hides a mountain of misinformation. In this talk we will walk through the pitfalls that you might find yourself in by looking at some examples where things go wrong. We will then walk through how MongoDB performance is measured, the processes and methodology and ways to present and look at the information.
Wprowadzenie do technologii Big Data / Intro to Big Data EcosystemSages
Introduction to Hadoop Map Reduce, Pig, Hive and Ambari technologies.
Workshop deck prepared and presented on September 5th 2015 by Radosław Stankiewicz.
During that the day participants had also the possibility to go through prepared tutorials and test their analysis on real cluster.
Building apps why you should bet on the webthebeebs
Building apps today is a complicated business, trying to get your creation in front of as many users possible, whilst delivering a great user experience, is a tall order. It's no wonder that many developers reduce the platforms they support and go native.
Using Web Technologies and, where required, JavaScript abstractions into Native code, you can support more devices and reach more users.
It's not necessarily easy, but with a pragmatic approach to testing, performance and design you can develop cross platform apps that beg the question: why should you ever bet on anything other than web technologies?
Weather of the Century: Design and PerformanceMongoDB
This talk walks you through how you can use MongoDB to store and analyze worldwide weather data from the entire 20th century in a graphical application.
"You shall not pass : anti-debug methodics"ITCP Community
How to save you app from hacking? We will discuss a protection mechanisms and ways to hack them.
Основные тезисы:
- Основы LLDB;
- Способы защиты от нежелательного дебага вашего приложения;
- Оценка эффективности этих способов;
- Способы обхода приведенных методик;
- Обфускация кода и другие полезные практики защиты;
- Небольшое демо по взлому:)
Mythbusting: Understanding How We Measure Performance at MongoDBMongoDB
Benchmarking, benchmarking, benchmarking. We all do it, mostly it tells us what we want to hear but often hides a mountain of misinformation. In this talk we will walk through the pitfalls that you might find yourself in by looking at some examples where things go wrong. We will then walk through how MongoDB performance is measured, the processes and methodology and ways to present and look at the information.
The X Dev API is a new protocol for non-blocking, asynchronous calls to MySQL. In this talk, Lior explores the benefits of working with this protocol and connectors, and the challenges we encountered during the process of adopting X Dev API in Wix Engineering.
Lior shares how we are incorporating the protocol in our massive multi dc architecture, and how it helps us, at Wix Engineering, rollout to production faster.
Cluj Big Data Meetup - Big Data in PracticeSteffen Wenz
At the Cluj Big Data Meetup, we shared some insights into TrustYou's big data tech stack. Also we introduced two tools which we've found useful in our production jobs: Apache Pig and Luigi.
Also check out the code samples on GitHub: https://github.com/trustyou/meetups/tree/master/big-data
This presentation will demonstrate how you can use the aggregation pipeline with MongoDB similar to how you would use GROUP BY in SQL and the new stage operators coming 3.4. MongoDB’s Aggregation Framework has many operators that give you the ability to get more value out of your data, discover usage patterns within your data, or use the Aggregation Framework to power your application. Considerations regarding version, indexing, operators, and saving the output will be reviewed.
Mythbusting: Understanding How We Measure the Performance of MongoDBMongoDB
Benchmarking, benchmarking, benchmarking. We all do it, mostly it tells us what we want to hear but often hides a mountain of misinformation. In this talk we will walk through the pitfalls that you might find yourself in by looking at some examples where things go wrong. We will then walk through how MongoDB performance is measured, the processes and methodology and ways to present and look at the information.
Wprowadzenie do technologii Big Data / Intro to Big Data EcosystemSages
Introduction to Hadoop Map Reduce, Pig, Hive and Ambari technologies.
Workshop deck prepared and presented on September 5th 2015 by Radosław Stankiewicz.
During that the day participants had also the possibility to go through prepared tutorials and test their analysis on real cluster.
Building apps why you should bet on the webthebeebs
Building apps today is a complicated business, trying to get your creation in front of as many users possible, whilst delivering a great user experience, is a tall order. It's no wonder that many developers reduce the platforms they support and go native.
Using Web Technologies and, where required, JavaScript abstractions into Native code, you can support more devices and reach more users.
It's not necessarily easy, but with a pragmatic approach to testing, performance and design you can develop cross platform apps that beg the question: why should you ever bet on anything other than web technologies?
Weather of the Century: Design and PerformanceMongoDB
This talk walks you through how you can use MongoDB to store and analyze worldwide weather data from the entire 20th century in a graphical application.
"You shall not pass : anti-debug methodics"ITCP Community
How to save you app from hacking? We will discuss a protection mechanisms and ways to hack them.
Основные тезисы:
- Основы LLDB;
- Способы защиты от нежелательного дебага вашего приложения;
- Оценка эффективности этих способов;
- Способы обхода приведенных методик;
- Обфускация кода и другие полезные практики защиты;
- Небольшое демо по взлому:)
Mythbusting: Understanding How We Measure Performance at MongoDBMongoDB
Benchmarking, benchmarking, benchmarking. We all do it, mostly it tells us what we want to hear but often hides a mountain of misinformation. In this talk we will walk through the pitfalls that you might find yourself in by looking at some examples where things go wrong. We will then walk through how MongoDB performance is measured, the processes and methodology and ways to present and look at the information.
The X Dev API is a new protocol for non-blocking, asynchronous calls to MySQL. In this talk, Lior explores the benefits of working with this protocol and connectors, and the challenges we encountered during the process of adopting X Dev API in Wix Engineering.
Lior shares how we are incorporating the protocol in our massive multi dc architecture, and how it helps us, at Wix Engineering, rollout to production faster.
MongoDB ne fonctionne pas comme les autres bases de données. Son modèle de données orienté documents, son partitionnement en gammes et sa cohérence forte sont bien adaptés à certains problèmes et moins adaptés à d'autres. Dans ce séminaire Web, nous étudierons des exemples réels d'utilisation de MongoDB mettant à profit ces fonctionnalités uniques. Nous évoquerons le cas de clients spécifiques qui utilisent MongoDB et nous verrons la façon dont ils ont implémenté leur solution. Nous vous montrerons également comment construire une solution du même type pour votre entreprise.
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...MongoDB
This session will be a case study of eBay’s experience running MongoDB for project Zoom, in which eBay stores all media metadata for the site. This includes references to pictures of every item for sale on eBay. This cluster is eBay's first MongoDB installation on the platform and is a mission critical application. Yuri Finkelstein, an Enterprise Architect on the team, will provide a technical overview of the project and its underlying architecture.
Video available here: http://vivu.tv/portal/archive.jsp?flow=783-586-4282&id=1270584002677
We all know that MongoDB is one of the most flexible and feature-rich databases available. In this webinar we'll discuss how you can leverage this feature set and maintain high performance with your project's massive data sets and high loads. We'll cover how indexes can be designed to optimize the performance of MongoDB. We'll also discuss tips for diagnosing and fixing performance issues should they arise.
MongoDB is the trusted document store we turn to when we have tough data store problems to solve. For this talk we are going to go a little bit off the path and explore what other roles we can fit MongoDB into. Others have discussed how to turn MongoDB’s capped collections into a publish/subscribe server. We stretch that a little further and turn MongoDB into a full fledged broker with both publish/subscribe and queue semantics, and a the ability to mix them. We will provide code and a running demo of the queue producers and consumers. Next we will turn to coordination services: We will explore the fundamental features and show how to implement them using MongoDB as the storage engine. Again we will show the code and demo the coordination of multiple applications.
MongoDB: Optimising for Performance, Scale & AnalyticsServer Density
MongoDB is easy to download and run locally but requires some thought and further understanding when deploying to production. At scale, schema design, indexes and query patterns really matter. So does data structure on disk, sharding, replication and data centre awareness. This talk will examine these factors in the context of analytics, and more generally, to help you optimise MongoDB for any scale.
Presented at MongoDB Days London 2013 by David Mytton.
Relational databases are central to web applications, but they have also been the primary source of pain when it comes to scale and performance. Recently, non-relational databases (also referred to as NoSQL) have arrived on the scene. This session explains not only what MongoDB is and how it works, but when and how to gain the most benefit.
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
During this talk we'll navigate through a customer's journey as they migrate an existing MongoDB deployment to MongoDB Atlas. While the migration itself can be as simple as a few clicks, the prep/post effort requires due diligence to ensure a smooth transfer. We'll cover these steps in detail and provide best practices. In addition, we’ll provide an overview of what to consider when migrating other cloud data stores, traditional databases and MongoDB imitations to MongoDB Atlas.
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
MongoDB Kubernetes operator and MongoDB Open Service Broker are ready for production operations. Learn about how MongoDB can be used with the most popular container orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications. A demo will show you how easy it is to enable MongoDB clusters as an External Service using the Open Service Broker API for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB
Humana, like many companies, is tackling the challenge of creating real-time insights from data that is diverse and rapidly changing. This is our journey of how we used MongoDB to combined traditional batch approaches with streaming technologies to provide continues alerting capabilities from real-time data streams.
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB
Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe.
This talk covers:
Common components of an IoT solution
The challenges involved with managing time-series data in IoT applications
Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance.
How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts
At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB
Our clients have unique use cases and data patterns that mandate the choice of a particular strategy. To implement these strategies, it is mandatory that we unlearn a lot of relational concepts while designing and rapidly developing efficient applications on NoSQL. In this session, we will talk about some of our client use cases, the strategies we have adopted, and the features of MongoDB that assisted in implementing these strategies.
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB
Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch".
This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
When you need to model data, is your first instinct to start breaking it down into rows and columns? Mine used to be too. When you want to develop apps in a modern, agile way, NoSQL databases can be the best option. Come to this talk to learn how to take advantage of all that NoSQL databases have to offer and discover the benefits of changing your mindset from the legacy, tabular way of modeling data. We’ll compare and contrast the terms and concepts in SQL databases and MongoDB, explain the benefits of using MongoDB compared to SQL databases, and walk through data modeling basics so you feel confident as you begin using MongoDB.
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB
Query performance should be the unsung hero of an application, but without proper configuration, can become a constant headache. When used properly, MongoDB provides extremely powerful querying capabilities. In this session, we'll discuss concepts like equality, sort, range, managing query predicates versus sequential predicates, and best practices to building multikey indexes.
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB
Aggregation pipeline has been able to power your analysis of data since version 2.2. In 4.2 we added more power and now you can use it for more powerful queries, updates, and outputting your data to existing collections. Come hear how you can do everything with the pipeline, including single-view, ETL, data roll-ups and materialized views.
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
MongoDB Atlas Data Lake is a new service offered by MongoDB Atlas. Many organizations store long term, archival data in cost-effective storage like S3, GCP, and Azure Blobs. However, many of them do not have robust systems or tools to effectively utilize large amounts of data to inform decision making. MongoDB Atlas Data Lake is a service allowing organizations to analyze their long-term data to discover a wealth of information about their business.
This session will take a deep dive into the features that are currently available in MongoDB Atlas Data Lake and how they are implemented. In addition, we'll discuss future plans and opportunities and offer ample Q&A time with the engineers on the project.
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB
Virtual assistants are becoming the new norm when it comes to daily life, with Amazon’s Alexa being the leader in the space. As a developer, not only do you need to make web and mobile compliant applications, but you need to be able to support virtual assistants like Alexa. However, the process isn’t quite the same between the platforms.
How do you handle requests? Where do you store your data and work with it to create meaningful responses with little delay? How much of your code needs to change between platforms?
In this session we’ll see how to design and develop applications known as Skills for Amazon Alexa powered devices using the Go programming language and MongoDB.
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB
aux Core Data, appréciée par des centaines de milliers de développeurs. Apprenez ce qui rend Realm spécial et comment il peut être utilisé pour créer de meilleures applications plus rapidement.
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB
Il n’a jamais été aussi facile de commander en ligne et de se faire livrer en moins de 48h très souvent gratuitement. Cette simplicité d’usage cache un marché complexe de plus de 8000 milliards de $.
La data est bien connu du monde de la Supply Chain (itinéraires, informations sur les marchandises, douanes,…), mais la valeur de ces données opérationnelles reste peu exploitée. En alliant expertise métier et Data Science, Upply redéfinit les fondamentaux de la Supply Chain en proposant à chacun des acteurs de surmonter la volatilité et l’inefficacité du marché.
2. 1. nearly ordered keys have nice properties db.coll.ensureIndex( { x : 1 } ) db.coll.insert( { x : rand() } )
3. 1. nearly ordered keys db.coll.ensureIndex( { x : 1 } ) db.coll.insert( { x : rand() } )
4. 1. nearly ordered keys db.coll.ensureIndex( { x : 1 } ) db.coll.insert( { x : new Date() } )
5. 2. use the embedded time in Object IDs > obj = db.mycoll.findOne(); > obj._id.getTimestamp(); > db.mycoll.find().sort({_id:1}); // time order
6. 3. Store UUIDs as BinData rather than as strings > help misc b = new BinData(subtype,base64str) create a BSON BinData value b.subtype() the BinData subtype (0..255) b.length() length of the BinData data in bytes b.hex() the data as a hex encoded string b.base64() the data as a base 64 encoded string b.toString() byte type = BINDATA; char my_fieldname[]; unsigned len = 16; // 16 bytes byte subtype = 3; // UUID char data[16]; // http://bsonspec.org/#/specification
8. 4. Use covered compound indexes to create materialized views* * coming soon! -- your current build may not be fast yet > // { a : ..., b : ..., c : ..., d : ..., e : ..., f : reallybigdata, g : ... } > t.ensureIndex( { b : 1, d : 1, e : 1 } ) > f.find( {}, { b : 1, d : 1, e : 1 } )
10. 6. making a replay log --diaglog or db._adminCommand( { diagLogging : 1 } ) Output is written to diaglog.bin_ 0 off. Also flushes any pending data to the file. 1 log writes 2 log reads 3 log both nc ''database_server_ip'' 27017 < ''somelog.bin'' | hexdump -c good for qa? good for bug reports?
13. thanks questions? [email_address] @dmerr Please follow @mongodb on twitter. Sign up with 10gen for a 1 day “db design sanity check and design tuning” session. Email [email_address] 10gen is hiring in Bay Area and New York. Up Next 11:00-11:30 MongoDB and the Democratic Party -- A Case Study How Sunlight Labs Uses MongoDB