Mass spectrometry is the gold standard for determining chemical compositions, with spectrometers often measuring the mass of a compound down to a single electron. This level of granularity produces an enormous amount of hierarchical data that doesn't fit well into rows and columns. In this talk, learn how Thermo Fisher is using MongoDB Atlas on AWS to allow their users to get near real-time insights from mass spectrometry experiments—a process that used to take days. We also share how the underlying database service used by Thermo Fisher was built on AWS.
MongoDB Days Silicon Valley: Introducing MongoDB 3.2MongoDB
Presented by:
Eliot Horowitz, CTO and Co-Founder, MongoDB
Richard Kreuter, VP of Professional Services, MongoDB
Andrew Erlichson, VP of Engineering, Developer Experience, MongoDB
The MongoDB Spark Connector integrates MongoDB and Apache Spark, providing users with the ability to process data in MongoDB with the massive parallelism of Spark. The connector gives users access to Spark's streaming capabilities, machine learning libraries, and interactive processing through the Spark shell, Dataframes and Datasets. We'll take a tour of the connector with a focus on practical use of the connector, and run a demo using both Spark and MongoDB for data processing.
The integration between Spring Framework and MongoDB tends to be somewhat unknown. This presentation shows the different projects that compose Spring ecosystem, Springdata, Springboot, SpringIO etc and how to merge between the pure JAVA projects to massive enterprise systems that require the interaction of these systems together.
Modern architectures are moving away from a "one size fits all" approach. We are well aware that we need to use the best tools for the job. Given the large selection of options available today, chances are that you will end up managing data in MongoDB for your operational workload and with Spark for your high speed data processing needs.
Description: When we model documents or data structures there are some key aspects that need to be examined not only for functional and architectural purposes but also to take into consideration the distribution of data nodes, streaming capabilities, aggregation and queryability options and how we can integrate the different data processing software, like Spark, that can benefit from subtle but substantial model changes. A clear example is when embedding or referencing documents and their implications on high speed processing.
Over the course of this talk we will detail the benefits of a good document model for the operational workload. As well as what type of transformations we should incorporate in our document model to adjust for the high speed processing capabilities of Spark.
We will look into the different options that we have to connect these two different systems, how to model according to different workloads, what kind of operators we need to be aware of for top performance and what kind of design and architectures we should put in place to make sure that all of these systems work well together.
Over the course of the talk we will showcase different libraries that enable the integration between spark and MongoDB, such as MongoDB Hadoop Connector, Stratio Connector and MongoDB Spark Native Connector.
By the end of the talk I expect the attendees to have an understanding of:
How they connect their MongoDB clusters with Spark
Which use cases show a net benefit for connecting these two systems
What kind of architecture design should be considered for making the most of Spark + MongoDB
How documents can be modeled for better performance and operational process, while processing these data sets stored in MongoDB.
The talk is suitable for:
Developers that want to understand how to leverage Spark
Architects that want to integrate their existing MongoDB cluster and have real time high speed processing needs
Data scientists that know about Spark, are playing with Spark and want to integrate with MongoDB for their persistency layer
MongoDB Schema Design: Practical Applications and ImplicationsMongoDB
Presented by Austin Zellner, Solutions Architect, MongoDB
Schema design is as much art as it is science, but it is central to understanding how to get the most out of MongoDB. Attendees will walk away with an understanding of how to approach schema design, what influences it, and the science behind the art. After this session, attendees will be ready to design new schemas, as well as re-evaluate existing schemas with a new mental model.
Database Trends for Modern Applications: Why the Database You Choose Matters MongoDB
Matt Kalan, Senior Solutions Architect, MongoDB
Matt will explain how modern technology requirements have changed the requirements of the database. In order to handle agile development, big data, cloud, APIs, continuous availability, and unlimited scale while lowering costs, new capabilities are required. Do you need to tolerate the impedance mismatch between an object model and the relational model, or is there another way? We will walk through the application development process, to the code level, to compare using an RDBMS with MongoDB.
MongoDB has been conceived for the cloud age. Making sure that MongoDB is compatible and performant around cloud providers is mandatory to achieve complete integration with platforms and systems. Azure is one of biggest IaaS platforms available and very popular amongst developers that work on Microsoft Stack.
MongoDB Days Silicon Valley: Introducing MongoDB 3.2MongoDB
Presented by:
Eliot Horowitz, CTO and Co-Founder, MongoDB
Richard Kreuter, VP of Professional Services, MongoDB
Andrew Erlichson, VP of Engineering, Developer Experience, MongoDB
The MongoDB Spark Connector integrates MongoDB and Apache Spark, providing users with the ability to process data in MongoDB with the massive parallelism of Spark. The connector gives users access to Spark's streaming capabilities, machine learning libraries, and interactive processing through the Spark shell, Dataframes and Datasets. We'll take a tour of the connector with a focus on practical use of the connector, and run a demo using both Spark and MongoDB for data processing.
The integration between Spring Framework and MongoDB tends to be somewhat unknown. This presentation shows the different projects that compose Spring ecosystem, Springdata, Springboot, SpringIO etc and how to merge between the pure JAVA projects to massive enterprise systems that require the interaction of these systems together.
Modern architectures are moving away from a "one size fits all" approach. We are well aware that we need to use the best tools for the job. Given the large selection of options available today, chances are that you will end up managing data in MongoDB for your operational workload and with Spark for your high speed data processing needs.
Description: When we model documents or data structures there are some key aspects that need to be examined not only for functional and architectural purposes but also to take into consideration the distribution of data nodes, streaming capabilities, aggregation and queryability options and how we can integrate the different data processing software, like Spark, that can benefit from subtle but substantial model changes. A clear example is when embedding or referencing documents and their implications on high speed processing.
Over the course of this talk we will detail the benefits of a good document model for the operational workload. As well as what type of transformations we should incorporate in our document model to adjust for the high speed processing capabilities of Spark.
We will look into the different options that we have to connect these two different systems, how to model according to different workloads, what kind of operators we need to be aware of for top performance and what kind of design and architectures we should put in place to make sure that all of these systems work well together.
Over the course of the talk we will showcase different libraries that enable the integration between spark and MongoDB, such as MongoDB Hadoop Connector, Stratio Connector and MongoDB Spark Native Connector.
By the end of the talk I expect the attendees to have an understanding of:
How they connect their MongoDB clusters with Spark
Which use cases show a net benefit for connecting these two systems
What kind of architecture design should be considered for making the most of Spark + MongoDB
How documents can be modeled for better performance and operational process, while processing these data sets stored in MongoDB.
The talk is suitable for:
Developers that want to understand how to leverage Spark
Architects that want to integrate their existing MongoDB cluster and have real time high speed processing needs
Data scientists that know about Spark, are playing with Spark and want to integrate with MongoDB for their persistency layer
MongoDB Schema Design: Practical Applications and ImplicationsMongoDB
Presented by Austin Zellner, Solutions Architect, MongoDB
Schema design is as much art as it is science, but it is central to understanding how to get the most out of MongoDB. Attendees will walk away with an understanding of how to approach schema design, what influences it, and the science behind the art. After this session, attendees will be ready to design new schemas, as well as re-evaluate existing schemas with a new mental model.
Database Trends for Modern Applications: Why the Database You Choose Matters MongoDB
Matt Kalan, Senior Solutions Architect, MongoDB
Matt will explain how modern technology requirements have changed the requirements of the database. In order to handle agile development, big data, cloud, APIs, continuous availability, and unlimited scale while lowering costs, new capabilities are required. Do you need to tolerate the impedance mismatch between an object model and the relational model, or is there another way? We will walk through the application development process, to the code level, to compare using an RDBMS with MongoDB.
MongoDB has been conceived for the cloud age. Making sure that MongoDB is compatible and performant around cloud providers is mandatory to achieve complete integration with platforms and systems. Azure is one of biggest IaaS platforms available and very popular amongst developers that work on Microsoft Stack.
Getting Started with MongoDB Using the Microsoft Stack MongoDB
Speaker: John Randolph, Sr. Software Developer, Gexa Energy
Level: 100 (Beginner)
Track: Developer
Gexa has implemented several applications using MongoDB as a document repository storing multiple types of files (PDF, XLS, CSV, etc.). This entry level session is intended to share what we’ve learned in developing and deploying our first applications in an on premise, Microsoft environment. We’ll provide architectural and development information about what we’ve done. The focus is to help get your projects up-to-speed more quickly. This will be useful to teams moving from pilot to production and for developers getting started with the .Net MongoDB drivers. Plenty of code samples will be shown. We’ll discuss our successful engagement with MongoDB Consulting to help us design and deploy a high-quality production environment.
What You Will Learn:
- Ideas how to store and retrieve documents of different sizes, types, and volumes. We’ll describe the storage, partitioning and indexing techniques used that provide sub-second retrieval from collections with over 100 million records.
- The issues addressed moving to production, including: backup, disaster recovery, SSL, using replica sets, implementing authorization and authentication, changing default setting, and creating a full path-to-production set of environments.
- A successful pattern for building applications with .Net, providing teams some ideas to jump-start their development along with tips and tricks for using the .Net drivers.
MongoDB and Hadoop: Driving Business InsightsMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
Webinar: Live Data Visualisation with Tableau and MongoDBMongoDB
MongoDB 3.2 introduces a new way for familiar Business Intelligence (BI) tools to access your real-time operational data – opening it up to data analysts and data scientist, enabling new insights to be discovered faster than ever before. Tableau accesses the JSON document data stored in MongoDB via this new BI connector. We will cover how the BI connector works by creating a relational view definition of a JSON data set that is then used to present a tabular SQL/ODBC interface to Tableau. Then we will set-up a live connection from Tableau Desktop to the MongoDB Connector for BI. Once we have Tableau Desktop and MongoDB connected, we will demonstrate the visual power of Tableau to explore the agile data storage of MongoDB. This webinar will cover:
What is the MongoDB BI Connector?
Setting up a connection from Tableau to the MongoDB BI Connector.
How to perform data discovery Tableau connected to MongoDB live data.
Publishing a Tableau Dashboard for sharing insights.
New generations of database technologies are allowing organizations to build applications never before possible, at a speed and scale that were previously unimaginable. MongoDB is the fastest growing database on the planet, and the new 3.2 release will bring the benefits of modern database architectures to an ever broader range of applications and users.
Webinar: Compliance and Data Protection in the Big Data Age: MongoDB Security...MongoDB
Data security and privacy are critical concerns in today’s connected world. Data analyzed from new sources such as social media, logs, mobile devices and sensor networks has become as sensitive as traditional transaction data generated by back-office systems. For this reason, big data technologies must evolve to meet the regulatory compliance standards demanded by industry and government. This session provides an overview of MongoDB’s security architecture, including authentication, authorization, auditing and encryption, collectively designed to to defend, detect and control access to valuable online big data.
Are you in the process of evaluating or migrating to MongoDB? We will cover key aspects of migrating to MongoDB from a RDBMS, including Schema design, Indexing strategies, Data migration approaches as your implementation reaches various SDLC stages, Achieving operational agility through MongoDB Management Services (MMS).
Webinar: Best Practices for Getting Started with MongoDBMongoDB
MongoDB adoption continues to grow at a record pace due to the significant enhancements in developer productivity and scalability that the database provides. Occasionally, however, organizations new to the technology make mistakes that limit their ability to leverage the significant advantages MongoDB provides. This webinar will discuss some of the common mistakes made by users when they first start working with MongoDB, how to identify when you've made those mistakes, and how to resolve them.
Eagle6 is a product that use system artifacts to create a replica model that represents a near real-time view of system architecture. Eagle6 was built to collect system data (log files, application source code, etc.) and to link system behaviors in such a way that the user is able to quickly identify risks associated with unknown or unwanted behavioral events that may result in unknown impacts to seemingly unrelated down-stream systems. This session is designed to present the capabilities of the Eagle6 modeling product and how we are using MongoDB to support near-real-time analysis of large disparate datasets.
Webinar: Schema Patterns and Your Storage EngineMongoDB
How do MongoDB’s different storage options change the way you model your data?
Each storage engine, WiredTiger, the In-Memory Storage engine, MMAP V1 and other community supported drivers, persists data differently, writes data to disk in different formats and handles memory resources in different ways.
This webinar will go through how to design applications around different storage engines based on your use case and data access patterns. We will be looking into concrete examples of schema design practices that were previously applied on MMAPv1 and whether those practices still apply, to other storage engines like WiredTiger.
Topics for review: Schema design patterns and strategies, real-world examples, sizing and resource allocation of infrastructure.
Webinar: Introducing the MongoDB Connector for BI 2.0 with TableauMongoDB
Pairing your real-time operational data stored in a modern database like MongoDB with first-class business intelligence platforms like Tableau enables new insights to be discovered faster than ever before.
Many leading organizations already use MongoDB in conjunction with Tableau including a top American investment bank and the world’s largest airline. With the Connector for BI 2.0, it’s never been easier to streamline the connection process between these two systems.
In this webinar, we will create a live connection from Tableau Desktop to a MongoDB cluster using the Connector for BI. Once we have Tableau Desktop and MongoDB connected, we will demonstrate the visual power of Tableau to explore the agile data storage of MongoDB.
You’ll walk away knowing:
- How to configure MongoDB with Tableau using the updated connector
- Best practices for working with documents in a BI environment
- How leading companies are using big data visualization strategies to transform their businesses
Move Fast with MongoDB Cloud Database - Atlas.
The workshop covered:
Deploying a MongoDB cluster in minutes
Query and manage data in MongoDB
Executing continuous backups and point-in-time restores, ensuring that you can meet any restore point objectives
View historical metrics in optimized dashboards, see what’s happening in your database live, configure alerts, and receive automated index suggestions to improve the performance of your cluster
Using MongoDB Charts and create visual representations of your data
How Auto Trader enables the UK's largest digital automotive marketplaceMongoDB
Often cited as one of the most successful digital transformations in the UK, Russell Warman talked through how their new ways of working, values and technology are helping Auto Trader to enable the UK's largest digital automotive marketplace and to become the UK’s most admired digital business.
Development time is wasted as the bulk of the work shifts from adding business features to struggling with the RDBMS. MongoDB, the leading NoSQL database, offers a flexible and scalable solution.
Getting Started with MongoDB Using the Microsoft Stack MongoDB
Speaker: John Randolph, Sr. Software Developer, Gexa Energy
Level: 100 (Beginner)
Track: Developer
Gexa has implemented several applications using MongoDB as a document repository storing multiple types of files (PDF, XLS, CSV, etc.). This entry level session is intended to share what we’ve learned in developing and deploying our first applications in an on premise, Microsoft environment. We’ll provide architectural and development information about what we’ve done. The focus is to help get your projects up-to-speed more quickly. This will be useful to teams moving from pilot to production and for developers getting started with the .Net MongoDB drivers. Plenty of code samples will be shown. We’ll discuss our successful engagement with MongoDB Consulting to help us design and deploy a high-quality production environment.
What You Will Learn:
- Ideas how to store and retrieve documents of different sizes, types, and volumes. We’ll describe the storage, partitioning and indexing techniques used that provide sub-second retrieval from collections with over 100 million records.
- The issues addressed moving to production, including: backup, disaster recovery, SSL, using replica sets, implementing authorization and authentication, changing default setting, and creating a full path-to-production set of environments.
- A successful pattern for building applications with .Net, providing teams some ideas to jump-start their development along with tips and tricks for using the .Net drivers.
MongoDB and Hadoop: Driving Business InsightsMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
Webinar: Live Data Visualisation with Tableau and MongoDBMongoDB
MongoDB 3.2 introduces a new way for familiar Business Intelligence (BI) tools to access your real-time operational data – opening it up to data analysts and data scientist, enabling new insights to be discovered faster than ever before. Tableau accesses the JSON document data stored in MongoDB via this new BI connector. We will cover how the BI connector works by creating a relational view definition of a JSON data set that is then used to present a tabular SQL/ODBC interface to Tableau. Then we will set-up a live connection from Tableau Desktop to the MongoDB Connector for BI. Once we have Tableau Desktop and MongoDB connected, we will demonstrate the visual power of Tableau to explore the agile data storage of MongoDB. This webinar will cover:
What is the MongoDB BI Connector?
Setting up a connection from Tableau to the MongoDB BI Connector.
How to perform data discovery Tableau connected to MongoDB live data.
Publishing a Tableau Dashboard for sharing insights.
New generations of database technologies are allowing organizations to build applications never before possible, at a speed and scale that were previously unimaginable. MongoDB is the fastest growing database on the planet, and the new 3.2 release will bring the benefits of modern database architectures to an ever broader range of applications and users.
Webinar: Compliance and Data Protection in the Big Data Age: MongoDB Security...MongoDB
Data security and privacy are critical concerns in today’s connected world. Data analyzed from new sources such as social media, logs, mobile devices and sensor networks has become as sensitive as traditional transaction data generated by back-office systems. For this reason, big data technologies must evolve to meet the regulatory compliance standards demanded by industry and government. This session provides an overview of MongoDB’s security architecture, including authentication, authorization, auditing and encryption, collectively designed to to defend, detect and control access to valuable online big data.
Are you in the process of evaluating or migrating to MongoDB? We will cover key aspects of migrating to MongoDB from a RDBMS, including Schema design, Indexing strategies, Data migration approaches as your implementation reaches various SDLC stages, Achieving operational agility through MongoDB Management Services (MMS).
Webinar: Best Practices for Getting Started with MongoDBMongoDB
MongoDB adoption continues to grow at a record pace due to the significant enhancements in developer productivity and scalability that the database provides. Occasionally, however, organizations new to the technology make mistakes that limit their ability to leverage the significant advantages MongoDB provides. This webinar will discuss some of the common mistakes made by users when they first start working with MongoDB, how to identify when you've made those mistakes, and how to resolve them.
Eagle6 is a product that use system artifacts to create a replica model that represents a near real-time view of system architecture. Eagle6 was built to collect system data (log files, application source code, etc.) and to link system behaviors in such a way that the user is able to quickly identify risks associated with unknown or unwanted behavioral events that may result in unknown impacts to seemingly unrelated down-stream systems. This session is designed to present the capabilities of the Eagle6 modeling product and how we are using MongoDB to support near-real-time analysis of large disparate datasets.
Webinar: Schema Patterns and Your Storage EngineMongoDB
How do MongoDB’s different storage options change the way you model your data?
Each storage engine, WiredTiger, the In-Memory Storage engine, MMAP V1 and other community supported drivers, persists data differently, writes data to disk in different formats and handles memory resources in different ways.
This webinar will go through how to design applications around different storage engines based on your use case and data access patterns. We will be looking into concrete examples of schema design practices that were previously applied on MMAPv1 and whether those practices still apply, to other storage engines like WiredTiger.
Topics for review: Schema design patterns and strategies, real-world examples, sizing and resource allocation of infrastructure.
Webinar: Introducing the MongoDB Connector for BI 2.0 with TableauMongoDB
Pairing your real-time operational data stored in a modern database like MongoDB with first-class business intelligence platforms like Tableau enables new insights to be discovered faster than ever before.
Many leading organizations already use MongoDB in conjunction with Tableau including a top American investment bank and the world’s largest airline. With the Connector for BI 2.0, it’s never been easier to streamline the connection process between these two systems.
In this webinar, we will create a live connection from Tableau Desktop to a MongoDB cluster using the Connector for BI. Once we have Tableau Desktop and MongoDB connected, we will demonstrate the visual power of Tableau to explore the agile data storage of MongoDB.
You’ll walk away knowing:
- How to configure MongoDB with Tableau using the updated connector
- Best practices for working with documents in a BI environment
- How leading companies are using big data visualization strategies to transform their businesses
Move Fast with MongoDB Cloud Database - Atlas.
The workshop covered:
Deploying a MongoDB cluster in minutes
Query and manage data in MongoDB
Executing continuous backups and point-in-time restores, ensuring that you can meet any restore point objectives
View historical metrics in optimized dashboards, see what’s happening in your database live, configure alerts, and receive automated index suggestions to improve the performance of your cluster
Using MongoDB Charts and create visual representations of your data
How Auto Trader enables the UK's largest digital automotive marketplaceMongoDB
Often cited as one of the most successful digital transformations in the UK, Russell Warman talked through how their new ways of working, values and technology are helping Auto Trader to enable the UK's largest digital automotive marketplace and to become the UK’s most admired digital business.
Development time is wasted as the bulk of the work shifts from adding business features to struggling with the RDBMS. MongoDB, the leading NoSQL database, offers a flexible and scalable solution.
Creating a Modern Data Architecture for Digital TransformationMongoDB
By managing Data in Motion, Data at Rest, and Data in Use differently, modern Information Management Solutions are enabling a whole range of architecture and design patterns that allow enterprises to fully harness the value in data flowing through their systems. In this session we explored some of the patterns (e.g. operational data lakes, CQRS, microservices and containerisation) that enable CIOs, CDOs and senior architects to tame the data challenge, and start to use data as a cross-enterprise asset.
Seminario Web MongoDB-Paradigma: Cree aplicaciones más escalables utilizando ...MongoDB
Las arquitecturas de microservicios han sido adoptados muy rápidamente, debido a su capacidad para proveer modularidad, escalabilidad y alta disponibilidad
En este seminario web grabado, nuestros expertos, Rubén Terceño de MongoDB y Miguel Garrido de Paradigma Digital le explican cómo se puede usar microservicios para:
Alinear las estructuras de tu organización
Realizar aplicaciones más rápidamente
Hacer un uso eficiente de tus recursos
Webinar: 10-Step Guide to Creating a Single View of your BusinessMongoDB
Organizations have long seen the value in aggregating data from multiple systems into a single, holistic, real-time representation of a business entity. That entity is often a customer. But the benefits of a single view in enhancing business visibility and operational intelligence can apply equally to other business contexts. Think products, supply chains, industrial machinery, cities, financial asset classes, and many more.
However, for many organizations, delivering a single view to the business has been elusive, impeded by a combination of technology and governance limitations.
MongoDB has been used in many single view projects across enterprises of all sizes and industries. In this session, we will share the best practices we have observed and institutionalized over the years. By attending the webinar, you will learn:
- A repeatable, 10-step methodology to successfully delivering a single view
- The required technology capabilities and tools to accelerate project delivery
- Case studies from customers who have built transformational single view applications on MongoDB.
AWS is an incredibly popular environment for running MongoDB deployments. Today you have many choices about instance type, storage, network config, security, how you configure MongoDB processes, and more. In addition, you now have options when it comes to tooling to help you manage and operate your deployment. In this session, we’ll take a look at several recommendations that can help you get the best performance out of AWS.
The importance of efficient data management for Digital TransformationMongoDB
Digital Transformation has developed from hype into a “standard” tool for businesses that need to modernise and compete. Experiencing pressure from new market entrants, incumbents are challenged on a daily basis to redefine their ways of doing business. This doesn’t only include people and processes, but of course also the underlying technology. With data being the force behind the most successful transformation stories in the past years, we are explored some of the challenges of legacy Information Management Systems, and look at new ways of managing Data in Motion, Data at Rest, and Data in Use to drive a successful Digital Transformation programme to gain a competitive advantage.
Back to Basics: My First MongoDB ApplicationMongoDB
This Back to Basics webinar series will introduce you to NoSQL and the MongoDB database. You will find out what MongoDB is, why you would use it, and what you would use it for.
View all the MongoDB World 2016 Poster Sessions slides in one place!
Table of Contents:
1: BigData DB Infrastructure for Modeling the Fly Brain
2: Taming the WiredTiger Cache
3: Sharding with MongoDB 3.2 Kick the tires and pop the hood!
4: Scaling Proactive Anomaly Detection
5: MongoTx: Transactions with Sharding and Queries
6: MongoDB: It’s Not Too Late To Shard
7: DLIFLC usage of MongoDB
Back to Basics 2017: Introduction to ShardingMongoDB
Sharding is a method for distributing data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations by providing the capability for horizontal scaling.
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. In this session, the concepts behind microservices, containers and orchestration was explained and how to use them with MongoDB.
Webinar: Working with Graph Data in MongoDBMongoDB
With the release of MongoDB 3.4, the number of applications that can take advantage of MongoDB has expanded. In this session we will look at using MongoDB for representing graphs and how graph relationships can be modeled in MongoDB.
We will also look at a new aggregation operation that we recently implemented for graph traversal and computing transitive closure. We will include an overview of the new operator and provide examples of how you can exploit this new feature in your MongoDB applications.
Webinar: Enterprise Data Management in the Era of MongoDB and Data LakesMongoDB
With so much talk of how Big Data is revolutionizing the world and how a data lake with Hadoop and/or Spark will solve all your data problems, it is hard to tell what is hype, reality, or somewhere in-between.
In working with dozens of enterprises in varying stages of their enterprise data management (EDM) strategy, MongoDB enterprise architect, Matt Kalan, sees the same challenges and misunderstandings arise again and again.
In this session, he will explain common challenges in data management, what capabilities are necessary, and what the future state of architecture looks like. MongoDB is uniquely capable of filling common gaps in the data lake strategy.
This session also includes a live Q&A portion during which you are encouraged to ask questions of our team.
This presentation covers best practices for running MongoDB on AWS. We also discuss how to utilize the automation features of MMS to spin up new clusters in minutes on AWS.
In this webinar, we will be covering general best practices for running MongoDB on AWS.
Topics will range from instance selection to storage selection and service distribution to ensure service availability. We will also look at any specific best practices related to using WiredTiger. We will then shift gears and explore recommended strategies for managing your MongoDB instance on AWS.
This session also includes a live Q&A portion during which you are encouraged to ask questions of our team.
eHarmony uses MongoDB to make it easier for couples to find each other. At the forefront of big data and machine learning, eHarmony’s matching system uses a flow algorithm to process a billion potential matches per day. The Compatibility Matching System® uses bi-directional, user-defined criteria to match members based on a comprehensive set of traits and preferences. The system was originally built on a relational database, but with over 51MM+ users, it took more than 2 weeks for the matching algorithm to run. By switching to MongoDB, eHarmony reduced the time to match by 95% to under 12 hours. Learn about eHarmony's matching system, its technology evaluation process, and how it has used MongoDB to make the happiest couples in the world.
AWS re:Invent 2016: How Thermo Fisher Is Reducing Mass Spectrometry Experimen...Amazon Web Services
Mass spectrometry is the gold standard for determining chemical compositions, with spectrometers often measuring the mass of a compound down to a single electron. This level of granularity produces an enormous amount of hierarchical data that doesn't fit well into rows and columns. In this talk, learn how Thermo Fisher is using MongoDB Atlas on AWS to allow their users to get near real-time insights from mass spectrometry experiments—a process that used to take days. We also share how the underlying database service used by Thermo Fisher was built on AWS.
How Thermo Fisher is Reducing Data Analysis Times from Days to Minutes with M...MongoDB
Speaker: Joseph Fluckiger, Senior Software Architect, ThermoFisher Scientific
Level: 200 (Intermediate)
Track: Atlas
Mass spectrometry is the gold standard for determining chemical compositions, with spectrometers often measuring the mass of a compound down to a single electron. This level of granularity produces an enormous amount of hierarchical data that doesn't fit well into rows and columns. In this talk, learn how Thermo Fisher is using MongoDB Atlas on AWS to allow their users to get near real-time insights from mass spectrometry experiments – a process that used to take days. We also share how the underlying database service used by Thermo Fisher was built on AWS.
What You Will Learn:
- How we modeled mass spectrometry data to enable us to write and read an enormous about of experimental data efficiently.
- Learn about the best MongoDB tools and patterns for .NET applications.
- Live demo of scaling a MongoDB Atlas cluster with zero down time and visualizing live data from a million dollar Mass Spectrometer stored in MongoDB.
AWS re:Invent 2016: Amazon Aurora Best Practices: Getting the Best Out of You...Amazon Web Services
Amazon Aurora is a fully managed relational database engine that provides higher performance, availability and durability than previously possible using conventional monolithic database architectures. After launching a year ago, we continued adding many new features and capabilities to Aurora. In this session AWS Aurora experts will discuss the best practices that will help you put these capabilities to the best use. You will also hear from Amazon Aurora customer Intercom on the best practices they adopted for moving live databases with over two billion rows to a new datastore in Amazon Aurora with almost no downtime or lost records.
Intercom was founded to provide a fundamentally new way for Internet businesses to communicate with customers at scale. For growing startups like Intercom, it’s natural for the load on datastores to grow on a weekly basis. The usual solution to this problem is to get a bigger box from AWS. But very soon you reach a point where bigger boat is not an option anymore. You will learn about the benefits of moving to such a datastore, the problems it introduced, and all about the new ability for scaling that was not there before.
Building Big Data Applications with Serverless Architectures - June 2017 AWS...Amazon Web Services
Learning Objectives:
- Use cases and best practices for serverless big data applications
- Leverage AWS technologies such as AWS Lambda and Amazon Kinesis
- Learn to perform ETL, event processing, ad-hoc analysis, real-time processing, and MapReduce with serverless
Building data processing applications is challenging and time-consuming, and often requires specialized expertise to deploy and operate. With serverless computing, you can perform real-time stream processing of multiple data types without needing to spin up servers or install software, allowing you to deploy big data applications quickly and more easily. Come learn how you can use AWS Lambda with Amazon Kinesis to analyze streaming data in real-time and then store the results in a managed NoSQL database such as Amazon DynamoDB. You’ll learn tips and tricks for doing in-line processing, data manipulation, and even distributed MapReduce on large data sets.
MongoDB 3.6 helps you *move at the speed of your data* - turning developers, operations teams, and analysts into a growth engine for the business. It enables new apps to be delivered to market faster, running reliably and securely at scale, and unlocking insights and intelligence in real time. Learn more: https://www.mongodb.com/mongodb-3.6
AWS re:Invent 2016: Big Data Mini Con State of the Union (BDM205)Amazon Web Services
Join us for this general session where AWS big data experts present an in-depth look at the current state of big data. Learn about the latest big data trends and industry use cases. Hear how other organizations are using the AWS big data platform to innovate and remain competitive. Take a look at some of the most recent AWS big data announcements, as we kick off the Big Data re:Source Mini Con.
This presentation will show you overview of Google Cloud Service and show step-by-step example with Wordpress to introduce each service on GCP
Google Cloud Study Jam Bangkok 2019 #1 and #2 at ITKMITL and CPE KU on October 19-20, 2019
Choose Right Stream Storage: Amazon Kinesis Data Streams vs MSKSungmin Kim
This presentation compares Amazon Kinesis Data Streams to Managed Streaming for Kafka (MSK) in both architectural perspective and operational perspective. In addition, it shows common architectural patterns: (1) Data Hub: Event-Bus, (2) Log Aggregation, (3) IoT, (4) Event sourcing and CQRS.
MongoDB Transactions are a major new piece of functionality for users. Early releases of the MongoDB database focused on applications without rigorous transactional semantics, as is common in NoSQL databases. However, some MongoDB users desire advanced transactional features, including multi-document transactions, point-in-time reads, and the choice of snapshot or read-committed isolation.
Data analytics master class: predict hotel revenueKris Peeters
We predict future revenues in hotels by solving the data science puzzle end-to-end: from infrastructure in the cloud and security, to data ingestion, data cleaning, feature building and model training and model scoring.
The video of this talk is here: https://www.facebook.com/datamindedbe/posts/1385820021562117
(HLS402) Getting into Your Genes: The Definitive Guide to Using Amazon EMR, A...Amazon Web Services
The key to fighting cancer through better therapeutics is a deep understanding of the basic biology of this disease at a cellular and molecular level. Comprehensive analysis of cancer mutations in specific tumors or cancer cell lines by using Life Technologies sequencing and real-time PCR systems generates gigabytes to terabytes of data every day. Our customers bring together this data in studies that seek to discover the genetic fingerprint of cancer. The data typically translates to millions of records in databases that require complex algorithmic processing, cross-application analysis, and interactive visualizations with real-time response (2-3 seconds) to enable users to consume large volumes of complex scientific information.
We have chosen the AWS platform to bring this new era of data analysis power to our customers by using technologies such as Amazon S3, ElastiCache, and DynamoDB for storage and fast access and Amazon EMR for parallelizing complex computations. Our talk tells the story with rich details about challenges and roadblocks in building data-intense, highly interactive applications in the cloud. We also highlight enhanced customer workflows and highly optimized applications with orders of magnitude improvement in performance and scalability.
6. DISZ - Webalkalmazások skálázhatósága a Google Cloud PlatformonMárton Kodok
Az előadás témája hogyan építhető fel egy rugalmas, jól skálázható szolgáltatás a felhőszolgáltatók platformjain. Hogyan lehet megoldani, hogy a szolgáltatás, amelynek induláskor legfeljebb néhány tíz vagy száz felhasználót kell kiszolgálnia, akár több ezer vagy nagyságrendekkel több felhasználót is képes legyen kiszolgálni rugalmasan? Hátradőlni és csodálni az autoscaling funkciót a Black Friday napján. Beszélni fogunk virtualizációról, platformszintű virtualizációről, szuperkönnyű alkalmazáskonténerekről, a munkaterhek közel valósidejű “pakolgatásával”. Bemutatásra kerül a Google Cloud Platform számos komponense. Bankok, biztosítók, webshopok és így tovább mind a cloudban látják a kitörési pontot.
Similar to How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to Minutes w/ MongoDB Atlas on AWS (20)
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
During this talk we'll navigate through a customer's journey as they migrate an existing MongoDB deployment to MongoDB Atlas. While the migration itself can be as simple as a few clicks, the prep/post effort requires due diligence to ensure a smooth transfer. We'll cover these steps in detail and provide best practices. In addition, we’ll provide an overview of what to consider when migrating other cloud data stores, traditional databases and MongoDB imitations to MongoDB Atlas.
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
MongoDB Kubernetes operator and MongoDB Open Service Broker are ready for production operations. Learn about how MongoDB can be used with the most popular container orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications. A demo will show you how easy it is to enable MongoDB clusters as an External Service using the Open Service Broker API for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB
Humana, like many companies, is tackling the challenge of creating real-time insights from data that is diverse and rapidly changing. This is our journey of how we used MongoDB to combined traditional batch approaches with streaming technologies to provide continues alerting capabilities from real-time data streams.
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB
Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe.
This talk covers:
Common components of an IoT solution
The challenges involved with managing time-series data in IoT applications
Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance.
How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts
At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB
Our clients have unique use cases and data patterns that mandate the choice of a particular strategy. To implement these strategies, it is mandatory that we unlearn a lot of relational concepts while designing and rapidly developing efficient applications on NoSQL. In this session, we will talk about some of our client use cases, the strategies we have adopted, and the features of MongoDB that assisted in implementing these strategies.
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB
Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch".
This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
When you need to model data, is your first instinct to start breaking it down into rows and columns? Mine used to be too. When you want to develop apps in a modern, agile way, NoSQL databases can be the best option. Come to this talk to learn how to take advantage of all that NoSQL databases have to offer and discover the benefits of changing your mindset from the legacy, tabular way of modeling data. We’ll compare and contrast the terms and concepts in SQL databases and MongoDB, explain the benefits of using MongoDB compared to SQL databases, and walk through data modeling basics so you feel confident as you begin using MongoDB.
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB
Query performance should be the unsung hero of an application, but without proper configuration, can become a constant headache. When used properly, MongoDB provides extremely powerful querying capabilities. In this session, we'll discuss concepts like equality, sort, range, managing query predicates versus sequential predicates, and best practices to building multikey indexes.
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB
Aggregation pipeline has been able to power your analysis of data since version 2.2. In 4.2 we added more power and now you can use it for more powerful queries, updates, and outputting your data to existing collections. Come hear how you can do everything with the pipeline, including single-view, ETL, data roll-ups and materialized views.
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
MongoDB Atlas Data Lake is a new service offered by MongoDB Atlas. Many organizations store long term, archival data in cost-effective storage like S3, GCP, and Azure Blobs. However, many of them do not have robust systems or tools to effectively utilize large amounts of data to inform decision making. MongoDB Atlas Data Lake is a service allowing organizations to analyze their long-term data to discover a wealth of information about their business.
This session will take a deep dive into the features that are currently available in MongoDB Atlas Data Lake and how they are implemented. In addition, we'll discuss future plans and opportunities and offer ample Q&A time with the engineers on the project.
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB
Virtual assistants are becoming the new norm when it comes to daily life, with Amazon’s Alexa being the leader in the space. As a developer, not only do you need to make web and mobile compliant applications, but you need to be able to support virtual assistants like Alexa. However, the process isn’t quite the same between the platforms.
How do you handle requests? Where do you store your data and work with it to create meaningful responses with little delay? How much of your code needs to change between platforms?
In this session we’ll see how to design and develop applications known as Skills for Amazon Alexa powered devices using the Go programming language and MongoDB.
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB
aux Core Data, appréciée par des centaines de milliers de développeurs. Apprenez ce qui rend Realm spécial et comment il peut être utilisé pour créer de meilleures applications plus rapidement.
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB
Il n’a jamais été aussi facile de commander en ligne et de se faire livrer en moins de 48h très souvent gratuitement. Cette simplicité d’usage cache un marché complexe de plus de 8000 milliards de $.
La data est bien connu du monde de la Supply Chain (itinéraires, informations sur les marchandises, douanes,…), mais la valeur de ces données opérationnelles reste peu exploitée. En alliant expertise métier et Data Science, Upply redéfinit les fondamentaux de la Supply Chain en proposant à chacun des acteurs de surmonter la volatilité et l’inefficacité du marché.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
19. MongoDB is a Swiss army knife
• Hierarchical data
• Relational data
• Queues
• File storage
• Device state
Amazon SQS
Amazon S3
Amazon IoT
20. Join example
• Version 3.2 introduced the $lookup operator
• SQL query
• MongoDB C# driver query
21. MongoDB has caught
up to relational DBs
Notably, we show that the MUPG (match,
unwind, project, group) fragment is
already at least as expressive as full
relational algebra over (the relational view
of) a single collection, and in particular
able to express arbitrary joins.
– Bolzano University in Italy
22. Hash-Based Sharding
Roles
Kerberos
On-Prem Monitoring
2.4
GA 2013
2.6
GA 2014
3.0
GA 2015
3.2
GA 2015
Headline Features by Release
$out
Index Intersection
Text Search
Field-Level Redaction
LDAP & x509
Auditing
Document Validation
$lookup
Fast Failover
Simpler Scalability
Aggregation ++
Encryption At Rest
In-Memory Storage
Engine
BI Connector
MongoDB Compass
APM Integration
Profiler Visualization
Auto Index Builds
Backups to File
System
Doc-Level
Concurrency
Compression
Storage Engine API
≤50 replicas
Auditing ++
Ops Manager
Linearizable reads
Intra-cluster compression
Views
Log Redaction
Graph Processing
Decimal
Collations
Faceted Navigation
Spark Connector ++
Zones ++
Aggregation ++
Auto-balancing ++
ARM, Power, zSeries
BI Connector ++
Compass ++
Hardware Monitoring
Server Pool
LDAP Authorization
Encrypted Backups
Cloud Foundry Integration
3.4
GA 2016Atlas
The evolution of MongoDB
1.0
2009
25. Inserting data: MongoDB vs. MySQL
• Inserting 1,615 chemical compound records into two parent-child tables.
• To optimize the MySQL query, we turned off foreign keys during insert and
used a string builder to create a bulk insert SQL statement. This improved
insert performance by a factor of 360.
• Compare to MongoDB.
Database Milliseconds Lines of code
MySQL not optimized 147,600 (2.5 minutes) 21
MySQL optimized 410 40
MongoDB 68 1
27. Selecting data: MongoDB vs. MySQL
• Query 600,000 rows of SampleCompound result data
• To optimize the MySQL select query, we created a dictionary to lookup child
records for each parent, this improved performance by a factor of 300,
optimization effort: 2 engineers and 2 weeks.
Database Seconds Lines of code
MySQL not optimized 2,400 (4.1 minutes) 20
MySQL optimized 8.2 29
MongoDB 17.5 7
29. Migrating to MongoDB reduced code by 3.5x
SQLite MongoDB
Data Layer Lines of Code 4271 1260
30. MongoDB compared to DynamoDB
MongoDB DynamoDB
Anywhere AWS
Rich Ad-hoc Query Language + IDE No Ad-hoc query language
Many operators (Joins, Aggregation, etc.) Fewer operators
Excellent Performance Excellent Performance
Easy to deploy (with Atlas) Easy to Deploy each table
Adding tables requires no configuration
changes
Adding tables requires additional configuration
and cost
Easy to use from AWS services but not
natively integrated
Native integration with AWS Services: IAM,
VPC, Lambda, Kinesis
Released in 2009 Released in 2012
31. MongoDB vs. S3 performance
Download 220 KB object from MongoDB was 7x faster cold, and 3x faster when warm
MongoDB Amazon S3
Retrieve document first time
68 ms 468 ms
Retrieve document second time 13 ms 38 ms
32. MongoDB vs. S3 performance
MongoDB 11x faster than S3 in the use case of partial document loading
MongoDB S3
Data size 400 Bytes 2.1 MB
Performance 19 ms 214 ms
40. Fully managed MongoDB clusters
Customer only needs to choose the
shape and size of the cluster
● Instance size (CPU and RAM)
● Replication factor
● Number of shards
● Disk space
● Disk speed
Screenshot of create dialog
Cluster features
41. VPC peering
IP address whitelist
SCRAM-SHA-1 authentication
readWriteAnyDatabase
enableSharding
clusterMonitor
SSL
Using well-known CA
Trust system CAs by default
Security features
43. AWS Account X—Region Y
VPC (Customer N)
Availability Zone A Availability Zone B Availability Zone C
Subnet A Subnet B Subnet C
mongod—27017 mongod—27017 mongod—27017
Customer container with replica set
44. AWS Account X—Region Y
VPC (Customer N)
Availability Zone A Availability Zone B Availability Zone C
Subnet A Subnet B Subnet C
Customer container with sharded cluster
shard0
S
shard1
S
shard2 config
shard0
S
shard1
S
shard2 config
shard0
S
shard1
S
shard2 config
45. mongod—27017 mongod—27017 mongod—27017
One security group per VPC applied to
all Amazon EC2 instances
Three classes of security rules:
● MongoDB traffic between cluster
members
● MongoDB traffic between application
and clusters
● SSH traffic between production
support jump box and EC2 instance
App Server Jump Box
IP firewall using security groups
ThermoFisher is the biggest company you’ve never heard about, we strive to be the world leader in serving science. We have 50,000 employees around the world. Our goal is to make the world healthier, cleaner and safer.
One of the products we make is a Mass Spectrometer.
At the core of the instrument is ping-pong ball size metal cylinder called an Orbitrab. Which spins ionized molecules around for distances of several kilometers in a fraction of a second and measures their masses very accurately.
It turns out there are quite a few applications for this capability.
ThermoFisher Mass Spectrometry instruments are used to detect Pollutants, if it is bad for you, our instruments will detect it.
One of our customers is the Karolinska institute in Sweden, (this is the same university responsible for giving out Nobel prizes) and they processes 100k samples per year serving all of Sweden. Each of their high resolution instruments produces 100TB data per year.
For me, making the world a cleaner, safer place is personally meaningful. My son Landon was born with a Cleft lip and Pallet which is caused at least in part by exposure of the baby at a very early age (pea size) to some environmental condition: mercury, lead, a volatile organic. So preventing other children from being born with birth with defects and having safe and healthy lives is one thing that motivates me to come to work every day.
The next mission to mars in 2020 will carry a mass spec known as the Mars Organic Molecule Analyzer, or MOMA, which contains a design based on a ThermoFisher Linear Ion Trap Mass Spectrometer.
Mars rover is not running MongoDB, but maybe as the NASA trend continues for using commercial products and Thermo increasingly adapts MongoDB, MongoDB will ship on a Mars Rover some day. You definitely couldn’t run DynamoDB on the mars rover, but you could run Mongo.
----
http://science.gsfc.nasa.gov/sed/bio/veronica.t.pinnick
https://ep70.eventpilot.us/web/planner.php?id=ASMS16
Mars Organic Molecule Analyzer (MOMA) Mass Spectrometer: Performance Testing in GC-MS and LD-MS Modes of Operation
Our mass spectrometers are used in major sporting events to ensure an even playing field by detecting banned performance enhancing drugs.
[optional]
If an athlete is using synthetic Testosterone, the instruments are sufficiently sensitive and the analytical techniques sufficiently advanced to detect the difference between synthetic and natural testosterone.
[extra]
We have a marketing contract with CBS for any CSI TV shoes they use ThermoFisher equipment.
[reference]
http://www.nbcnews.com/storyline/2016-rio-summer-olympics/rio-olympics-top-anti-doping-scientist-cheats-will-probably-be-n573531
So this is what beer looks like in a mass spec. This is 100 samples of various types of beer. Each one of the variations in these peaks represents the unique flavonoids that make a product unique and give it a distinct smell and flavor.
Our mass spectrometers are used for product authenticity studies.
Any MythBuster fans out there? Adam Savage actually spoke at the keynote of MongoDB world 2016 in New York, so that is why I am a Mongo fan, never mind the technical merits.
In 2009 The Mythbusters Adam and Jamie use ThermoFisher Mass Spectrometer to determine if soda cans have rat pee on them. Really great episode, just search for “Rat Pee Soda”.
In the experiment, they take 1000 soda cans and let rats run and pee all over them. And then take soda cans from local convenience stores and compare the two sets of cans using a black light. Using the black light, both sets look similar. Organic material glowing under the black light. However, when they take the rat pee cans and the convenience store cans to the Stanford analytical lab, the mass spectrometer is able to conclusively determine that no rat pee is found on the convenience store cans.
[reference]
Episode 135
http://www.dailymotion.com/video/x2n9enp (Starting at minute 7:30 Jamie and Adam visit Stanford lab and use Thermo Mass Specs)
Jamie Says quote “These Mass Spectrometers are extremely accurate, they can detect down to a femptomole, and if it says they aren’t in there, its not in there.”
Adam was very relieved by this result and drank a soda.
To keep things interesting, I am going to do a live demo. This is always a risky proposition when trying to remote monitor a complex instrument that is more expensive than my house using a network that is potentially unpredictable.
Let me focus on one of our application that just rolled out to production called “Instrument Connect” built using Mongo Atlas.
“Instrument Connect” allows our customers to connect their mass spectrometers to the ThermoFisher cloud built on AWS.
Customers can monitor instrument status from anywhere in the world and receive notification of any errors that occur. Instrument data is streamed up to the cloud where it can take advantage of the incredible processing power of the AWS cloud and users from around the world can collaborate on the experiments and results.
The database which stores instrument status is MongoDB Atlas.
We also built a prototype integration with Amazon Alexa allowing us to control the instrument with voice commands.
[Demo outline]
Open MS Instrument connect dashboard.
Open Atlas Dashboard
This is the mass spec we will be remote monitoring.
I didn’t have the budget to bring the instrument with me on stage so I’ll use remote desktop.
Humor: Apparently this is the only shirt I own.
ThermoFisher is increasingly using MongoDB in its applications.
Mass Spectrometers have become so sensitive that they can measure the mass of a molecule down to the electron. This results in a huge amount of data.
Rich query language includes partial document updates,
MongoDB can store many types of data. Using MongoDB allows us to simplify our infrastructure. It also allows us to use a single set of tools for managing our data and our applications.
Now that MongoDB supports join operations, we can store both relational and document data in the same database. This greatly expands the type of application that can be built on MongoDB and simplifies our deployment since we only have one database rather than two.
MongoDB has climbed to the number 4 slot on db-engines ranking of most popular databases. This is based on metrics including job postings, stack overflow questions and google searches. Mongo is only behind Oracle, MySql, and SqlServer. Oracle which was first released in 1979, Sql Server in 1989, MySql in 1999 and MongoDB in 2009. Remarkable that MongoDB has made up so much ground on relational database technology which is 40 years old and doesn’t show any sign of slowing down.
Let me talk for a moment about some performance, scalability and cost comparisons that we did with MySql vs. MongoDB
We apply the same scientific rigor as our customers when making a decision on which database to use.
(remove fro AWS)
MySQL not optimized: 21 lines
MySQL optimized: 40 lines
MongoDB: 1 line
TODO: run test with larger data set.
MySQL not optimized: 21 lines
MySQL optimized: 40 lines
MongoDB: 1 line
If I were to reduce my presentation to one slide, this would be that slide. This is a staggeringly awesome improvement in developer productivity.
MySQL not optimized: 21 lines
MySQL optimized: 40 lines
MongoDB: 1 line
Similar number of lines of code and performance.
SQL Injection: Nice advantage of MongoDB is that the queries are strongly typed and no chance of SQL injection. After all these years SQL injection is still the number one security threat.
TODO: measure performance
The application used in the major sporting event in summer Rio sporting event - TraceFinder switched from XML and SQLite to MongoDB.
We could probably reduce this even further, but there is a dramatic decrease in cyclomatic complexity.
TODO: measure cyclomatic complexity.
Here I am contrasting DynamoDB and MongoDB, I think that as usual the answer to which database is best for your application is “It depends”, but here is some information to help you make that choice for your application. Both of these databases are very good and I think both will continue to grow in popularity at a much faster rate than relational databases.
Like me product manager says, I don’t need better decisions, I need better choices. And as a customer it is great to have multiple good database choices.
The most obvious difference is that Dynamo is an AWS-only service and Mongo runs anywhere.
Rich query language:
With MongoDB, I can answer questions like which instruments had the highest utilization last month or what is the pump pressure were I see my pumps begin to fail. And I can find this out without writing any code using ad-hoc queries.
With MongoDB, I can take advantage of rich features like joins, document validation, strongly typed queries, decimal data type, views, graph queries, and grouping, the aggregation pipeline, map-reduce, native spark connector, etc.
According to db-engines, MongoDB and DynamoDB are the two top rated database in the category of Document DBs. MongoDB has a score of 325 and DyamoDB a score of 29.
Native Integration with AWS: For example, if you want to take advantage of the native triggers available to execute a Lambda statement
Please don’t interpret this slide as you should always use MongoDB over S3. That would not be wise. S3 would far out perform MongoDB in other scenarios. In this particular case, MongoDB is a much better choice.
This measurement was taken by running C# code from EC2 instance in AWS US-East region.
The title of this slide might strike you as odd, comparing S3 with MongoDB.
S3 is an powerful AWS service which can be used to store multi gigabyte files and tiny JSON objects. It is a key-value store but by carefully selecting keys you can use S3 like a simple database with tables and rows a set of S3 objects with the same key prefix can function like a database table, the advantage is that you have a very inexpensive, serverless, highly available database. But as your application gets more complex you miss out on the rich query capabilities of a full relational or document database.
For our Real-time chromatogram we realized a couple orders of magnitude in savings in network and CPU consumption on our application servers by not having to download the entire S3 object and filter it down, we were able to do this instead on the database.
[Reference]
Performance measurement code: "C:\_git\CloudAgent\srcapi\Ironclad.Bootstrap\Repo\RealtimeChroDalBootstrap.cs"
[Note]
Serialzed JSON to S3 using Newtonsoft to S3 which is 20% larger objects compared with Mongo Bson. (storage on disk is even more of a contrast)
Please don’t interpret this slide as you should always use MongoDB over S3. That would not be wise. S3 would far out perform MongoDB in other scenarios. In this particular case, MongoDB is a much better choice.
This measurement was taken by running C# code from EC2 instance in AWS US-East region.
The title of this slide might strike you as odd, comparing S3 with MongoDB.
S3 is an powerful AWS service which can be used to store multi gigabyte files and tiny JSON objects. It is a key-value store but by carefully selecting keys you can use S3 like a simple database with tables and rows a set of S3 objects with the same key prefix can function like a database table, the advantage is that you have a very inexpensive, serverless, highly available database. But as your application gets more complex you miss out on the rich query capabilities of a full relational or document database.
For our Real-time chromatogram we realized a couple orders of magnitude in savings in network and CPU consumption on our application servers by not having to download the entire S3 object and filter it down, we were able to do this instead on the database.
[Reference]
Performance measurement code: "C:\_git\CloudAgent\srcapi\Ironclad.Bootstrap\Repo\RealtimeChroDalBootstrap.cs"
[Note]
Serialzed JSON to S3 using Newtonsoft to S3 which is 20% larger objects compared with Mongo Bson. (storage on disk is even more of a contrast)
With all the time we are saving writing and optimizing data layer code, we are able to invest in improving our algorithms, improving the user experience, and improving the processing infrastructure.
We have used MongoDB in a single server configuration, but we did not have expertise in cluster management, and don’t necessarily want to. When Atlas was announced in July of this year, we immediately jumped on board, ease of deploying MongoDB was the one thing holding us back.
With a weekend of work I switched from Dynamo to MongoDB in one day. Switching databases two months before going to production is not something I necessarily recommend. On Monday I asked my boss, did you notice anything different about the application, any downtime, he said now. I told him that I switched the database to Mongo, and he was not enthusiastic
Switching has turned out to be a great decision, and we have other applications looking to make the same switch.
Robustness: We were storing some of our data in DynamoDB and some in S3 because Dynamo is expensive. But you can do partial updates of S3 documents we since moved this data to MongoDB have improved the robustness and performance significantly.
We had a significant number of outages on Dynamo because we didn’t have sufficient throughput on our Dynamo tables, due to short spikes in traffic
Rate of development: Adding a new collection is much easier than adding a table in Dynamo. I don’t have to write a cloud formation script each time I want to provision a new table. I don’t have to provision read/write capacity for each table of the database independently.
Data analytics: Writing ad-hoc queries to answer questions like “give me a count of instruments per user”, or “what are my most active instruments” was not possible with DynamoDB because it doesn’t (didn’t) have a standalone query language or IDE.
Ability to run outside cloud as well as inside the cloud