The document discusses microservices and containers. It defines microservices as small, independent services with well-defined interfaces that allow for decentralized control and independent deployments. Containers are presented as a way to package and run microservices using technologies like Docker. Orchestration with systems like Kubernetes and Mesos is described as a way to automate deployment, linking, and maintenance of multiple containers across infrastructure. MongoDB is discussed as a good fit for microservices due to its flexibility, redundancy, scalability, and simplicity.
MongoDB Evenings Toronto - Monolithic to Microservices with MongoDBMongoDB
Monolithic to Microservices with MongoDB: Building Highly Available Services
Shawn McCarthy, Senior Solutions Architect, MongoDB
MongoDB Evenings Toronto
Infusion Offices
September 27, 2016
Powering Microservices with MongoDB, Docker, Kubernetes & Kafka – MongoDB Eur...Andrew Morgan
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver.
Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Need an identical copy of your application stack in multiple environments? Build your own container image and then your entire development, test, operations, and support teams can launch an identical clone environment.
Containers are revolutionizing the entire software lifecycle: from the earliest technical experiments and proofs of concept through development, test, deployment, and support. Orchestration tools manage how multiple containers are created, upgraded and made highly available. Orchestration also controls how containers are connected to build sophisticated applications from multiple, microservice containers.
This presentation introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.
From Monolithic to Microservices in 45 MinutesMongoDB
This document discusses moving from monolithic applications to microservices architectures. It begins by defining monolithic applications and how they can become difficult to scale. It then introduces the concepts of decoupling applications and microservices as an architecture where independent processes communicate via APIs. Some benefits discussed are improved scalability, release cycles, and fault tolerance. The document provides examples of microservices for tasks like text search and recommendations. It concludes by recommending starting with a monolithic approach and refactoring to microservices as needs require for scalability.
Orchestration Patterns for Microservices with Messaging by RabbitMQVMware Tanzu
Companies looking to speed up their software development are adopting microservices architectures (MSA). Building applications as groups of smaller components with fewer dependencies helps companies such as Comcast, Capital One, Uber, and Netflix deliver more frequent releases and thus innovate faster.
An important consideration in adopting an MSA is deciding how individual services should communicate between each other. Adding a message queue such as RabbitMQ to handle interservice messages can improve communication by:
- Simplifying our services so they only need to know how to talk to the messenger service.
- Abstracting communication by having the messenger service handle sophisticated orchestration patterns.
- Scaling message throughput by increasing the cluster size of the messenger service.
In this webinar we'll discuss:
- Requirements for communicating between microservices
- Typical messaging patterns in microservice architectures
- Use cases where RabbitMQ shines
- How to use the RabbitMQ service for Pivotal Cloud Foundry to deploy and run your applications
We’ll also demonstrate how to deploy RabbitMQ in Pivotal Cloud Foundry, and how to incorporate it in microservices-based applications.
Presenters: Greg Chase, Pivotal and Dan Baskette, Pivotal
Nodeconf Barcelona 2015 presentation exploring several ways of building microservices in an asynchronous way. Presented the concept of a broker as an alternative to a multiple point-to-point architecture.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/20SN0dP.
Tammer Saleh talks about the mistakes people make when building a microservices architecture. He also talks about: when microservices are appropriate, and where to draw the lines between services, dealing with performance issues, testing and debugging techniques, managing a polyglot landscape and the explosion of platforms, managing failure and graceful degradation. Filmed at qconlondon.com.
Tammer Saleh is a long time developer, leader, and author of the acclaimed book *Rails AntiPatterns*. Saleh is currently building the Cloud Foundry platform at Pivotal.
Spring is the most popular and productive enterprise Java development framework in the world, and has always provided developers with portability and choice. The cloud should be no different. Spring applications work flawlessly on all the major platform-as-a-service clouds including Heroku, Google App Engine, and Cloud Foundry. This session will focus on how to design, and create, modern enterprise applications using Spring 3 that are portable across cloud environments.
MongoDB .local Bengaluru 2019: Using MongoDB Services in Kubernetes: Any Plat...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MongoDB Evenings Toronto - Monolithic to Microservices with MongoDBMongoDB
Monolithic to Microservices with MongoDB: Building Highly Available Services
Shawn McCarthy, Senior Solutions Architect, MongoDB
MongoDB Evenings Toronto
Infusion Offices
September 27, 2016
Powering Microservices with MongoDB, Docker, Kubernetes & Kafka – MongoDB Eur...Andrew Morgan
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver.
Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Need an identical copy of your application stack in multiple environments? Build your own container image and then your entire development, test, operations, and support teams can launch an identical clone environment.
Containers are revolutionizing the entire software lifecycle: from the earliest technical experiments and proofs of concept through development, test, deployment, and support. Orchestration tools manage how multiple containers are created, upgraded and made highly available. Orchestration also controls how containers are connected to build sophisticated applications from multiple, microservice containers.
This presentation introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.
From Monolithic to Microservices in 45 MinutesMongoDB
This document discusses moving from monolithic applications to microservices architectures. It begins by defining monolithic applications and how they can become difficult to scale. It then introduces the concepts of decoupling applications and microservices as an architecture where independent processes communicate via APIs. Some benefits discussed are improved scalability, release cycles, and fault tolerance. The document provides examples of microservices for tasks like text search and recommendations. It concludes by recommending starting with a monolithic approach and refactoring to microservices as needs require for scalability.
Orchestration Patterns for Microservices with Messaging by RabbitMQVMware Tanzu
Companies looking to speed up their software development are adopting microservices architectures (MSA). Building applications as groups of smaller components with fewer dependencies helps companies such as Comcast, Capital One, Uber, and Netflix deliver more frequent releases and thus innovate faster.
An important consideration in adopting an MSA is deciding how individual services should communicate between each other. Adding a message queue such as RabbitMQ to handle interservice messages can improve communication by:
- Simplifying our services so they only need to know how to talk to the messenger service.
- Abstracting communication by having the messenger service handle sophisticated orchestration patterns.
- Scaling message throughput by increasing the cluster size of the messenger service.
In this webinar we'll discuss:
- Requirements for communicating between microservices
- Typical messaging patterns in microservice architectures
- Use cases where RabbitMQ shines
- How to use the RabbitMQ service for Pivotal Cloud Foundry to deploy and run your applications
We’ll also demonstrate how to deploy RabbitMQ in Pivotal Cloud Foundry, and how to incorporate it in microservices-based applications.
Presenters: Greg Chase, Pivotal and Dan Baskette, Pivotal
Nodeconf Barcelona 2015 presentation exploring several ways of building microservices in an asynchronous way. Presented the concept of a broker as an alternative to a multiple point-to-point architecture.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/20SN0dP.
Tammer Saleh talks about the mistakes people make when building a microservices architecture. He also talks about: when microservices are appropriate, and where to draw the lines between services, dealing with performance issues, testing and debugging techniques, managing a polyglot landscape and the explosion of platforms, managing failure and graceful degradation. Filmed at qconlondon.com.
Tammer Saleh is a long time developer, leader, and author of the acclaimed book *Rails AntiPatterns*. Saleh is currently building the Cloud Foundry platform at Pivotal.
Spring is the most popular and productive enterprise Java development framework in the world, and has always provided developers with portability and choice. The cloud should be no different. Spring applications work flawlessly on all the major platform-as-a-service clouds including Heroku, Google App Engine, and Cloud Foundry. This session will focus on how to design, and create, modern enterprise applications using Spring 3 that are portable across cloud environments.
MongoDB .local Bengaluru 2019: Using MongoDB Services in Kubernetes: Any Plat...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
Accelerating the Path to Digital with a Cloud Data StrategyMongoDB
This document discusses accelerating digital transformation through a cloud data strategy using MongoDB.
It begins by outlining MongoDB's capabilities as a cloud data platform, including its use by over 3000 enterprises. The document then discusses how time to market has replaced cost as the primary driver for cloud adoption. It also outlines considerations for choosing a cloud data platform like deployment flexibility, reducing complexity, agility, resiliency, scalability, cost, and security.
The document then provides an overview of MongoDB's cloud offerings, including MongoDB Atlas on public clouds, MongoDB Ops Manager for private clouds, and MongoDB Stitch for backend services. It also discusses best practices for replatforming applications from relational databases to MongoDB in the cloud.
Putting The 'M' In MBaaS—Red Hat Mobile Client Development Platform (Jay Balu...Red Hat Developers
When you hear the term "MBaaS," or "Red Hat Mobile," there is usually a lot of discussion about powerful scaling, back-end integrations, hosting options, containerization, etc. However, we can't forget what that "M" stands for, and why the platforms exist in the first place, which is to develop and deliver top-notch mobile applications to your users. In this session, we'll review what makes all of this possible—client SDKs, hybrid solutions like Cordova, and Xamarin, and our own Build Farm and Unified Push server. Not stopping there, our AppForms support makes it a snap to tie in back-end systems all the way to your app. And this is all backed by various templates, guides, and new open source resources that will help you get started and join the fun.
Building Reactive Applications With Node.Js And Red Hat JBoss Data Grid (Gald...Red Hat Developers
Node.js is a very popular framework for developing asynchronous, event-driven, reactive applications. Red Hat JBoss Data Grid, an in-memory distributed database designed for fast access to large volumes of data and scalability, has recently gained compatibility with Node.js letting reactive applications use it as a persistence layer. Thanks to near caching, JBoss Data Grid offers excellent response times for data queried regularly, and its continuous remote event support means data can get pushed from the data grid to the Node.js application instead of having to wait for the data grid to serve it. In this session, we'll show how to build Node.js applications that use JBoss Data Grid as a persistence layer.
MongoDB .local London 2019: Migrating a Monolith to MongoDB Atlas – Auto Trad...MongoDB
Over the last 12 months at Auto Trader, we have been focusing our energy on moving our on premise workloads to Google Cloud Platform, and that includes our database architecture.
Join me as we explore how we have migrated from on premise MongoDB clusters to a microservice aligned database architecture on MongoDB Atlas using Infrastructure as Code, and how we are integrating MongoDB into the Auto Trader Delivery Platform.
Transforming a Large Mission-Critical E-Commerce Platform from a Relational A...MongoDB
Speaker: Gaurav Goyal, Sr IT Architect, Cisco Systems Inc
Speaker: Dharmesh Panchmatia, Director, Cisco Systems Inc
Level: 200 (Intermediate)
Track: RDBMS to MongoDB
Cisco’s e-commerce platform is a suite of 35 different applications and 300+ services that powers product configuration, pricing, quoting, and order booking across all Cisco product lines including hardware, software, services and subscriptions. It’s a B2B platform used by Cisco Sales Team, Partners and Direct Customers, serving 140,000 unique users across the globe, handling 4 million transactions per day. The Benefits of migrating to MongoDB were as follows: 1) 5x performance improvement, 2) Fault tolerant architecture, 3) Continuous deployments and upgrades with zero downtime, 4) Faster application development.
What You Will Learn:
- How to transform your e-commerce platform to enable cloud native architecture.
- Bulk data migration in real time between relational databases & MongoDB.
- Best practices for brownfield migration for mission critical systems.
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
An introduction to KrakenD, the ultra-high performance API Gateway with middlewares. An opensource tool built using go that is currently serving traffic in major european sites.
MongoDB .local London 2019: Modern Data Backup and Recovery from On-premises ...MongoDB
The document discusses MongoDB's approach to backup and recovery across on-premises and cloud environments. It describes the evolution from complex on-premises backup architectures utilizing multiple agents and daemons to a simpler approach using WiredTiger checkpoints stored directly in object storage. It also outlines Atlas' multi-region disaster recovery capabilities and ability to restore from queryable cloud provider snapshots at a granular level. Future improvements may include agents writing directly to backups, incremental checkpoints, and optimizations for selective restores.
How Yelp Leapt to Microservices with More than a Message Queueconfluent
Without seeing what’s wrong with today’s messaging queues, it can be initially confusing to view Apache Kafka as more. By adding additional functionality, true storage, and guarantees it opens opportunities to take full advantage of a publish/subscribe model.
Joined by Yelp’s Justin Cunningham we’ll see how their infrastructure has quickly evolved. Powered by Kafka, Yelp has made the leap to microservices and is seeing the benefits of efficiency and performance.
Speakers:
Justin Cunningham
Technical Lead, Software Engineer, Yelp
Gehrig Kunz
Technical Product Marketing Manager, Confluent
Should you use traditional REST APIs to bind services together? Or is it better to use a richer, more loosely-coupled protocol? This talk will dig into how we piece services together in event driven systems, how we use a distributed log (event hub) to create a central, persistent history of events and what benefits we achieve from doing so. Apache Kafka is a perfect match for building such an asynchronous, loosely-coupled event-driven backbone. Events trigger processing logic, which can be implemented in a more traditional as well as in a stream processing fashion. The talk will show the difference between a request-driven and event-driven communication and show when to use which. It highlights how the modern stream processing systems can be used to
hold state both internally as well as in a database and how this state can be used to further increase independence of other services, the primary goal of a Microservices architecture.
MongoDB .local London 2019: New Encryption Capabilities in MongoDB 4.2: A Dee...MongoDB
Many applications with high-sensitivity workloads require enhanced technical options to control and limit access to confidential and regulated data. In some cases, system requirements or compliance obligations dictate a separation of duties for staff operating the database and those who maintain the application layer. In cloud-hosted environments, certain data are sometimes deemed too sensitive to store on third-party infrastructure. This is a common pain for system architects in the healthcare, finance, and consumer tech sectors — the benefits of managed, easily expanded compute and storage have been considered unavailable because of data confidentiality and privacy concerns.
This session will take a deep dive into new security capabilities in MongoDB 4.2 that address these scenarios, by enabling native client-side field-level encryption, using customer-managed keys. We will review how confidential data can be securely stored and easily accessed by applications running on MongoDB. Common query design patterns will be presented, with example code demonstrating strong end-to-end encryption in Atlas or on-premise. Implications for developers and others designing systems in regulated environments will be discussed, followed by a Q&A with senior MongoDB security engineers.
Microservices architecture is discussed along with Platform as a Service (PaaS), multi-tenancy, and DevOps. Key aspects of successful services like subscription-based models are highlighted. Techniques used by companies like Amazon, Google, Netflix, Facebook, and Twitter to enable continuous delivery and deployment are examined. Issues around managing scalability with microservices are also covered.
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)Red Hat Developers
Red Hat JBoss Fuse integration services delivers cloud-based integration based on OpenShift by Red Hat to deliver continuous delivery of tested, production-ready integration solutions. Utilizing a drag and drop, code-free UI and combining that with the integration power of Apache Camel, Fuse integration services is the next generation iPaaS. In this session, we'll walk you through why iPaaS is important, the current Fuse integration services roadmap, and the innovation happening in open source community projects to make this a reality.
Building distributed systems is challenging. Luckily, Apache Kafka provides a powerful toolkit for putting together big services as a set of scalable, decoupled components. In this talk, I'll describe some of the design tradeoffs when building microservices, and how Kafka's powerful abstractions can help. I'll also talk a little bit about what the community has been up to with Kafka Streams, Kafka Connect, and exactly-once semantics.
Presentation by Colin McCabe, Confluent, Big Data Day LA
Debugging Microservices - key challenges and techniques - Microservices Odesa...Lohika_Odessa_TechTalks
Microservice architecture is widespread our days. It comes with a lot of benefits and challenges to solve. Main goal of this talk is to go through troubleshooting and debugging in the distributed micro-service world. Topic would cover:
main aspects of the logging,
monitoring,
distributed tracing,
debugging services on the cluster.
About speaker:
Andrеy Kolodnitskiy is Staff engineer in the Lohika and his primary focus is around distributed systems, microservices and JVM based languages.
Majority of time engineers spend debugging and fixing the issues. This talk will be dedicated to best practicies and tools Andrеys team uses on its project which do help to find issues more efficiently.
RedisConf18 - Redis in Dev, Test, and Prod with the OpenShift Service CatalogRedis Labs
This document discusses using Redis in development, test, and production environments with the OpenShift Service Catalog.
It demonstrates using Redis for iterative development with ephemeral instances in development. In testing, it shows production-like configurations with immutable infrastructure, recovery testing, and zero-downtime deployments. For production, it notes the Service Catalog can provide targeted Redis instances and make external services discoverable. It promotes the Open Service Broker API and OpenShift Service Catalog for expanding service options.
Distributed Enterprise Monitoring and Management of Apache Kafka (William McL...HostedbyConfluent
Managing a distributed system like Apache Kafka can be extremely challenging, especially when you try to approach monitoring and managing from a single centralized GUI approach. In this talk come here and see a demo of a more decoupled approach to Kafka management and Kafka Monitoring where data is centralized but access is is distributed to scale to enterprise deployments, CICD pipelines and much much more.
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Lucas Jellema
This presentation gives an brief overview of the history of relational databases, ACID and SQL and presents some of the key strentgths and potential weaknesses. It introduces the rise of NoSQL - why it arose, what is entails, when to use it. The presentation focuses on MongoDB as prime example of NoSQL document store and it shows how to interact with MongoDB from JavaScript (NodeJS) and Java.
This document discusses using .NET Core and Docker for microservices. It begins with an overview of why Docker and microservices are useful. It then discusses why .NET Core and Microsoft technologies are good choices for building microservices. The document demonstrates creating a simple .NET Core app as a Docker container. It also discusses microservices patterns like having a database per service and isolating service instances. The document concludes with information about prerequisites for the demos and asking if there are any questions.
Scaling and Orchestrating Microservices with OSGi - N Bartlettmfrancis
This document discusses how OSGi services can be used to implement microservices and enable their orchestration and scaling. It describes how OSGi services have supported capabilities like runtime assembly, software components, and continuous delivery since before the term "microservices" was coined. The document argues that OSGi services align with many characteristics of microservices, like independent deployability, but with OSGi additionally enforcing encapsulation where discipline is needed with other approaches. It also discusses how OSGi remote services and discovery allow services to be scaled horizontally across processes and machines while enabling dynamic availability and pluggability. The document demonstrates these concepts with an OSGi-based microservices orchestration platform.
Webinar: Enabling Microservices with Containers, Orchestration, and MongoDBMongoDB
Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Need an identical copy of your application stack in multiple environments? Build your own container image and then your entire development, test, operations, and support teams can launch an identical clone environment.
Containers are revolutionizing the entire software lifecycle: from the earliest technical experiments and proofs of concept through development, test, deployment, and support. Orchestration tools manage how multiple containers are created, upgraded and made highly available. Orchestration also controls how containers are connected to build sophisticated applications from multiple, microservice containers.
This webinar introduces the concepts behind containers and orchestration, then explains the available technologies and how to use them with MongoDB. Finally, you will see a demonstration of exactly how to create a MongoDB replica set on Docker and Kubernetes within the Google Cloud.
Accelerating the Path to Digital with a Cloud Data StrategyMongoDB
This document discusses accelerating digital transformation through a cloud data strategy using MongoDB.
It begins by outlining MongoDB's capabilities as a cloud data platform, including its use by over 3000 enterprises. The document then discusses how time to market has replaced cost as the primary driver for cloud adoption. It also outlines considerations for choosing a cloud data platform like deployment flexibility, reducing complexity, agility, resiliency, scalability, cost, and security.
The document then provides an overview of MongoDB's cloud offerings, including MongoDB Atlas on public clouds, MongoDB Ops Manager for private clouds, and MongoDB Stitch for backend services. It also discusses best practices for replatforming applications from relational databases to MongoDB in the cloud.
Putting The 'M' In MBaaS—Red Hat Mobile Client Development Platform (Jay Balu...Red Hat Developers
When you hear the term "MBaaS," or "Red Hat Mobile," there is usually a lot of discussion about powerful scaling, back-end integrations, hosting options, containerization, etc. However, we can't forget what that "M" stands for, and why the platforms exist in the first place, which is to develop and deliver top-notch mobile applications to your users. In this session, we'll review what makes all of this possible—client SDKs, hybrid solutions like Cordova, and Xamarin, and our own Build Farm and Unified Push server. Not stopping there, our AppForms support makes it a snap to tie in back-end systems all the way to your app. And this is all backed by various templates, guides, and new open source resources that will help you get started and join the fun.
Building Reactive Applications With Node.Js And Red Hat JBoss Data Grid (Gald...Red Hat Developers
Node.js is a very popular framework for developing asynchronous, event-driven, reactive applications. Red Hat JBoss Data Grid, an in-memory distributed database designed for fast access to large volumes of data and scalability, has recently gained compatibility with Node.js letting reactive applications use it as a persistence layer. Thanks to near caching, JBoss Data Grid offers excellent response times for data queried regularly, and its continuous remote event support means data can get pushed from the data grid to the Node.js application instead of having to wait for the data grid to serve it. In this session, we'll show how to build Node.js applications that use JBoss Data Grid as a persistence layer.
MongoDB .local London 2019: Migrating a Monolith to MongoDB Atlas – Auto Trad...MongoDB
Over the last 12 months at Auto Trader, we have been focusing our energy on moving our on premise workloads to Google Cloud Platform, and that includes our database architecture.
Join me as we explore how we have migrated from on premise MongoDB clusters to a microservice aligned database architecture on MongoDB Atlas using Infrastructure as Code, and how we are integrating MongoDB into the Auto Trader Delivery Platform.
Transforming a Large Mission-Critical E-Commerce Platform from a Relational A...MongoDB
Speaker: Gaurav Goyal, Sr IT Architect, Cisco Systems Inc
Speaker: Dharmesh Panchmatia, Director, Cisco Systems Inc
Level: 200 (Intermediate)
Track: RDBMS to MongoDB
Cisco’s e-commerce platform is a suite of 35 different applications and 300+ services that powers product configuration, pricing, quoting, and order booking across all Cisco product lines including hardware, software, services and subscriptions. It’s a B2B platform used by Cisco Sales Team, Partners and Direct Customers, serving 140,000 unique users across the globe, handling 4 million transactions per day. The Benefits of migrating to MongoDB were as follows: 1) 5x performance improvement, 2) Fault tolerant architecture, 3) Continuous deployments and upgrades with zero downtime, 4) Faster application development.
What You Will Learn:
- How to transform your e-commerce platform to enable cloud native architecture.
- Bulk data migration in real time between relational databases & MongoDB.
- Best practices for brownfield migration for mission critical systems.
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
An introduction to KrakenD, the ultra-high performance API Gateway with middlewares. An opensource tool built using go that is currently serving traffic in major european sites.
MongoDB .local London 2019: Modern Data Backup and Recovery from On-premises ...MongoDB
The document discusses MongoDB's approach to backup and recovery across on-premises and cloud environments. It describes the evolution from complex on-premises backup architectures utilizing multiple agents and daemons to a simpler approach using WiredTiger checkpoints stored directly in object storage. It also outlines Atlas' multi-region disaster recovery capabilities and ability to restore from queryable cloud provider snapshots at a granular level. Future improvements may include agents writing directly to backups, incremental checkpoints, and optimizations for selective restores.
How Yelp Leapt to Microservices with More than a Message Queueconfluent
Without seeing what’s wrong with today’s messaging queues, it can be initially confusing to view Apache Kafka as more. By adding additional functionality, true storage, and guarantees it opens opportunities to take full advantage of a publish/subscribe model.
Joined by Yelp’s Justin Cunningham we’ll see how their infrastructure has quickly evolved. Powered by Kafka, Yelp has made the leap to microservices and is seeing the benefits of efficiency and performance.
Speakers:
Justin Cunningham
Technical Lead, Software Engineer, Yelp
Gehrig Kunz
Technical Product Marketing Manager, Confluent
Should you use traditional REST APIs to bind services together? Or is it better to use a richer, more loosely-coupled protocol? This talk will dig into how we piece services together in event driven systems, how we use a distributed log (event hub) to create a central, persistent history of events and what benefits we achieve from doing so. Apache Kafka is a perfect match for building such an asynchronous, loosely-coupled event-driven backbone. Events trigger processing logic, which can be implemented in a more traditional as well as in a stream processing fashion. The talk will show the difference between a request-driven and event-driven communication and show when to use which. It highlights how the modern stream processing systems can be used to
hold state both internally as well as in a database and how this state can be used to further increase independence of other services, the primary goal of a Microservices architecture.
MongoDB .local London 2019: New Encryption Capabilities in MongoDB 4.2: A Dee...MongoDB
Many applications with high-sensitivity workloads require enhanced technical options to control and limit access to confidential and regulated data. In some cases, system requirements or compliance obligations dictate a separation of duties for staff operating the database and those who maintain the application layer. In cloud-hosted environments, certain data are sometimes deemed too sensitive to store on third-party infrastructure. This is a common pain for system architects in the healthcare, finance, and consumer tech sectors — the benefits of managed, easily expanded compute and storage have been considered unavailable because of data confidentiality and privacy concerns.
This session will take a deep dive into new security capabilities in MongoDB 4.2 that address these scenarios, by enabling native client-side field-level encryption, using customer-managed keys. We will review how confidential data can be securely stored and easily accessed by applications running on MongoDB. Common query design patterns will be presented, with example code demonstrating strong end-to-end encryption in Atlas or on-premise. Implications for developers and others designing systems in regulated environments will be discussed, followed by a Q&A with senior MongoDB security engineers.
Microservices architecture is discussed along with Platform as a Service (PaaS), multi-tenancy, and DevOps. Key aspects of successful services like subscription-based models are highlighted. Techniques used by companies like Amazon, Google, Netflix, Facebook, and Twitter to enable continuous delivery and deployment are examined. Issues around managing scalability with microservices are also covered.
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)Red Hat Developers
Red Hat JBoss Fuse integration services delivers cloud-based integration based on OpenShift by Red Hat to deliver continuous delivery of tested, production-ready integration solutions. Utilizing a drag and drop, code-free UI and combining that with the integration power of Apache Camel, Fuse integration services is the next generation iPaaS. In this session, we'll walk you through why iPaaS is important, the current Fuse integration services roadmap, and the innovation happening in open source community projects to make this a reality.
Building distributed systems is challenging. Luckily, Apache Kafka provides a powerful toolkit for putting together big services as a set of scalable, decoupled components. In this talk, I'll describe some of the design tradeoffs when building microservices, and how Kafka's powerful abstractions can help. I'll also talk a little bit about what the community has been up to with Kafka Streams, Kafka Connect, and exactly-once semantics.
Presentation by Colin McCabe, Confluent, Big Data Day LA
Debugging Microservices - key challenges and techniques - Microservices Odesa...Lohika_Odessa_TechTalks
Microservice architecture is widespread our days. It comes with a lot of benefits and challenges to solve. Main goal of this talk is to go through troubleshooting and debugging in the distributed micro-service world. Topic would cover:
main aspects of the logging,
monitoring,
distributed tracing,
debugging services on the cluster.
About speaker:
Andrеy Kolodnitskiy is Staff engineer in the Lohika and his primary focus is around distributed systems, microservices and JVM based languages.
Majority of time engineers spend debugging and fixing the issues. This talk will be dedicated to best practicies and tools Andrеys team uses on its project which do help to find issues more efficiently.
RedisConf18 - Redis in Dev, Test, and Prod with the OpenShift Service CatalogRedis Labs
This document discusses using Redis in development, test, and production environments with the OpenShift Service Catalog.
It demonstrates using Redis for iterative development with ephemeral instances in development. In testing, it shows production-like configurations with immutable infrastructure, recovery testing, and zero-downtime deployments. For production, it notes the Service Catalog can provide targeted Redis instances and make external services discoverable. It promotes the Open Service Broker API and OpenShift Service Catalog for expanding service options.
Distributed Enterprise Monitoring and Management of Apache Kafka (William McL...HostedbyConfluent
Managing a distributed system like Apache Kafka can be extremely challenging, especially when you try to approach monitoring and managing from a single centralized GUI approach. In this talk come here and see a demo of a more decoupled approach to Kafka management and Kafka Monitoring where data is centralized but access is is distributed to scale to enterprise deployments, CICD pipelines and much much more.
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Lucas Jellema
This presentation gives an brief overview of the history of relational databases, ACID and SQL and presents some of the key strentgths and potential weaknesses. It introduces the rise of NoSQL - why it arose, what is entails, when to use it. The presentation focuses on MongoDB as prime example of NoSQL document store and it shows how to interact with MongoDB from JavaScript (NodeJS) and Java.
This document discusses using .NET Core and Docker for microservices. It begins with an overview of why Docker and microservices are useful. It then discusses why .NET Core and Microsoft technologies are good choices for building microservices. The document demonstrates creating a simple .NET Core app as a Docker container. It also discusses microservices patterns like having a database per service and isolating service instances. The document concludes with information about prerequisites for the demos and asking if there are any questions.
Scaling and Orchestrating Microservices with OSGi - N Bartlettmfrancis
This document discusses how OSGi services can be used to implement microservices and enable their orchestration and scaling. It describes how OSGi services have supported capabilities like runtime assembly, software components, and continuous delivery since before the term "microservices" was coined. The document argues that OSGi services align with many characteristics of microservices, like independent deployability, but with OSGi additionally enforcing encapsulation where discipline is needed with other approaches. It also discusses how OSGi remote services and discovery allow services to be scaled horizontally across processes and machines while enabling dynamic availability and pluggability. The document demonstrates these concepts with an OSGi-based microservices orchestration platform.
Webinar: Enabling Microservices with Containers, Orchestration, and MongoDBMongoDB
Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Need an identical copy of your application stack in multiple environments? Build your own container image and then your entire development, test, operations, and support teams can launch an identical clone environment.
Containers are revolutionizing the entire software lifecycle: from the earliest technical experiments and proofs of concept through development, test, deployment, and support. Orchestration tools manage how multiple containers are created, upgraded and made highly available. Orchestration also controls how containers are connected to build sophisticated applications from multiple, microservice containers.
This webinar introduces the concepts behind containers and orchestration, then explains the available technologies and how to use them with MongoDB. Finally, you will see a demonstration of exactly how to create a MongoDB replica set on Docker and Kubernetes within the Google Cloud.
This document outlines a plan to analyze gaps in an organization's governance and metrics for enterprise architecture modernization. The plan involves 7 projects that will develop an enterprise architecture framework, reference models, taxonomy, and assessments of the current and target states of the business, IT, technology, data, and application architectures. The projects will be carried out over 4 quarters.
Unlocking Operational Intelligence from the Data LakeMongoDB
The document discusses operationalizing data lakes by integrating MongoDB with Hadoop to enable both real-time and batch processing capabilities. It describes how MongoDB can be used to power operational applications with low-latency access to analytics models generated from raw data stored in Hadoop, while Hadoop is still used for its batch processing and analytics capabilities on large datasets. By combining both technologies, companies can unlock insights from their data lakes and avoid being part of the 70% of Hadoop projects that fail to meet objectives due to skills and integration challenges.
The rise of microservices - containers and orchestrationAndrew Morgan
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. In this session, the concepts behind containers and orchestration will be explained and how to use them with MongoDB.
Real World Application Orchestration Made Easy on VMware vCloud Air, vSphere ...Nati Shalom
Looking for application orchestration in a hybrid or multi-cloud environment? You’ve got to hear about TOSCA orchestration. TOSCA (Topology and Orchestration Specification for Cloud Applications), brought to you by the same people who brought us XML, enables you to seamlessly migrate your workloads across environments or build a hybrid deployment that runs simultaneously across the VMware cloud offering.
Join our Cloud Online Meetup to learn how Cloudify’s TOSCA-compliant orchestration can be your common management interface across the VMware cloud offering, OpenStack and heterogeneous cloud environments.
Speakers:
Nati Shalom, Founder and CTO at GigaSpaces, is a thought leader in Cloud Computing and Big Data Technologies. Shalom was recently recognized as a Top Cloud Computing Blogger for CIOs by The CIO Magazine and his blog is listed as an excellent blog by YCombinator. Shalom is the founder and also one of leaders of OpenStack Israel group, and is a frequent presenter at industry conferences.
Paco Gomez, Senior Solution Architect at VMware vCloud Air. Paco evaluates and integrates strategic solutions that help vCloud Air clients benefit from VMware's hybrid cloud and application services. Paco is a seasoned technologist, having extensive experience in diverse fields including mainframes, distributed systems, enterprise development, cloud computing, mobile, assistive technology, electrical engineering and embedded systems. Across his career, Paco has held positions in consulting, sales engineering
MongoDB Ops Manager is the easiest way to manage/monitor/operationalize your MongoDB footprint across your enterprise. Ops Manager automates key operations such as deployments, scaling, upgrades, and backups, all with the click of a button and integration with your favorite tools. It also provide the ability to monitor and alert on dozens of platform specific metrics. In this webinar, we'll cover the components of Ops Manager, as well as how it integrates and accelerates your use of MongoDB.
Developing Java based microservices ready for the world of containersClaus Ibsen
The so-called experts are saying microservices and containers will
change the way we build, maintain, operate, and integrate
applications. This talk is intended for Java developers who wants to hear and see how you can develop Java microservices that are ready to run in containers.
In this talk we will build a set of Java based Microservices that uses a mix of technologies with Apache Camel, Spring Boot and WildFly Swarm.
You will see how we can build small discrete microservices with these Java technologies and build and deploy on the Kubernets container platform.
We will discuss practices how to build distributed and fault tolerant microservices using technologies such as Kubernetes Services, Camel EIPs, and Netflixx Hysterix.
And the self healing and fault tolerant aspects of the Kubernetes platform is also discussed and demoed when we let the chaos monkeys loose killing containers.
This talk is a 50/50 mix between slides and demo.
The talk was presented at JDKIO on September 13th 2016.
Cloud Migration, Application Modernization and Security for PartnersAmazon Web Services
As AWS continues to expand, enterprise customers are increasingly looking to our partner ecosystem to assist in migrating their workloads to the cloud. This session describes the challenges, lessons learned and best practices for large scale application migrations. We will use real examples from our consulting partners and AWS Professional Services to illustrate how to move workloads to the cloud while modernizing the associated applications to take advantage of AWS’ unique benefits. We will also dive into how to use an array of AWS services and features to improve a customer’s security posture as they are migrating and once they are up and running in the cloud.
Oscon 2017: Build your own container-based system with the Moby projectPatrick Chanezon
Build your own container-based system
with the Moby project
Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.
Container Shangri-La Attaining the Promise of Container ParadiseXebiaLabs
Find out from Rob Stroud, CPO of XebiaLabs and former DevOps Analyst at Forrester Research, where containers fall short and how to bridge the gap between the promise of containers and the realities of complex enterprise application delivery.
MongoDB Europe 2016 - Powering Microservices with Docker, Kubernetes, and KafkaMongoDB
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. This session introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.
Microservices, Containers and Docker
This document provides an overview of microservices, containers, and Docker. It begins by defining microservices as an architectural style where applications are composed of independent, interchangeable components. It discusses benefits of the microservices style such as independent deployability, efficient scaling, and design autonomy. The document then introduces containers as a way to package applications and their dependencies to run uniformly across various environments. It compares containers to virtual machines. Finally, it describes Docker as an open source tool that automates deployment of applications into containers, providing portability and management of containers. The document concludes by discussing the need for container orchestration at scale.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes can schedule containers across a cluster of nodes, provide basic health checking and recovery of containers, and expose containers to the internet. Some key aspects include using microservices, container orchestration, continuous integration/delivery (CI/CD), and deployment automation.
Rob Davies presentation during Red Hat's "Microservices Journey with Apache Camel" that took place in Atlanta on 10/04/16 and in Minneapolis on 10/06/16.
Mastering MongoDB on Kubernetes, the power of operators DoKC
Link: https://youtu.be/Pi5ueyl_1jU
https://go.dok.community/slack
https://dok.community/
ABSTRACT OF THE TALK
During my first talk for DoK community I want to walk you through the world of NoSQL database MongoDB and Kubernetes Operators - Community Edition, Enterprise Edition (MongoDB and Ops Manager on K8s), and Atlas operator, highlight the most important capabilities, talk about use cases and challenges, the theory will be mixed with a live demos!
BIO
I'm a SRE / NoSQL / DevOps professional. I hold CKA, CKAD, CKS, also I’m MongoDB Certified DBA and MongoDB Champion. I have experience with multiple cloud providers, Kubernetes, different types of K8s operators (Strimzi, RabbitMQ Cluster Operator), but especially MongoDB K8s Operator. I also work with KEDA. Since 2017, I have been a speaker at MongoDB conferences all around the world (USA, China, Europe).
KEY TAKE-AWAYS FROM THE TALK
I would like to share the best practices of running NoSQL database - MongoDB on Kubernetes also I want to show how to manage Atlas (MongoDB cloud) via K8s operator
https://www.mongodb.com/developer/community-champions/arkadiusz-borucki/
An RSVP app designed to be deployed by the dockers on the Kubernetes Minikube Cluster. Front end with flask framework and MongoDB as a backend database.
Youtube video:https://youtu.be/KnjnQj-FvfQ
From CoreOS to Kubernetes and Concourse CIDenis Izmaylov
This document summarizes Denis Izmaylov's presentation on moving from CoreOS to Kubernetes. It discusses how CoreOS is primarily an OS focused on containers and not well-suited for managing microservices. Kubernetes provides a more complete platform for deploying and managing containerized applications at scale through concepts like pods, services, labels, and controllers. It allows achieving goals like fault tolerance, fast growth, and continuous delivery that were difficult with just CoreOS. The presentation also covers how the speaker's company developed a one-click installer to simplify Kubernetes cluster setup and management.
Lana Kalashnyk presented on transitioning to Java microservices on Docker. Key points included:
- Microservices involve breaking applications into small, independent services that communicate via APIs. Docker containers help deploy and manage microservices.
- The presentation demonstrated a Java microservice that polls a Bitcoin node for block height updates. It was packaged into a Docker container using Wildfly Swarm and exposed via REST APIs.
- A React web page displayed the data from the microservice. This illustrated how microservices and containers could replace outdated .NET web services.
- Benefits of microservices include independent deployability, fault isolation, and infrastructure automation using containers. Challenges include managing transactions and data
The Containers Ecosystem, the OpenStack Magnum Project, the Open Container In...Daniel Krook
Presentation at the OpenStack Summit in Tokyo, Japan on October 27, 2015.
http://sched.co/49x0
The technology industry has been abuzz about cloud workload containerization since the open source Docker project became a phenomenon in early 2014.
Meanwhile, an OpenStack Containers Team was formed and the Magnum project launched to provide users with a convenient Containers-as-a-Service solution for OpenStack environments.
As the potential of both technologies emerged, many wanted to see shared governance over the baseline container specification and runtime technology to ensure an open cloud ecosystem.
This past June, a new group was formed with a goal of creating open, industry standards around container formats and runtimes, called the Open Container Initiative (http://www.opencontainers.org).
So how will OpenStack Magnum influence - and be influenced by - the new OCI group? Why is the OCI under the stewardship of the Linux Foundation? What is the scope of the OCI effort? What project goals and/or principles will guide their work?
Attend this session to learn the following:
* A brief history of the open container ecosystem and the major benefits that containerization provides
* An overview of the Magnum CaaS plugin architecture and design goals
* Insider details on the the progress of the Linux Foundation Open Container Initiative (and the related Cloud Native Computing Foundation)
* What it all means for deploying container orchestration engines on your cloud with OpenStack Magnum
Megan Kostick - Software Engineer, Cloud and Open Source Technologies, IBM
Daniel Krook - Senior Software Engineer, Cloud and Open Source Technologies, IBM
Jeffrey Borek - WW Program Director, Open Technologies and Partnerships, Cloud Computing
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka Mario Ishara Fernando
This document discusses microservices and containers. It provides an overview of microservices architecture compared to monolithic architecture, highlighting that microservices are composed of many small, independent services with separate deployments and databases. It then discusses containers and how Docker is used to package and run applications in isolated containers. Finally, it introduces Kubernetes as a container orchestration system to manage and scale multiple containerized applications across a cluster of machines.
(SACON) Anand Tapikar - Attack vectors of Kubernetes infra. Are we on right ...Priyanka Aash
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. K8s groups containers that make up an application into logical units for easy management and discovery. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. As organizations accelerate their adoption of containers and container orchestrators, they will need to take necessary steps to protect such a critical part of their compute infrastructure.
How this topic is relevant 1 out of 5 organization going for container installation Container security attack vectors are rising Recently major vulnerability discovered in containers and got good media attention Duration (Mentioned on sacon.io, if not as per program committee call).
Containers are widely used today for consolidation and cloud native applications, with emerging uses in edge computing and serverless technologies. Containers provide efficiency, repeatability, isolation, and portability. Traditional uses include consolidation to reduce infrastructure costs and improve server utilization. Cloud native applications are also well suited to containers due to their microservices architecture enabling rapid deployment and scalability. For day 2 operations, container orchestration platforms like Kubernetes are used for lifecycle management, placement, and service discovery. Container storage interfaces provide pluggable storage solutions, while observability tools help debug complex microservices interactions through application performance monitoring and distributed tracing. Open APIs and standards are important to avoid vendor lock-in and allow flexibility in building, deploying and
Powering Microservices with Docker, Kubernetes, Kafka, & MongoDBMongoDB
This session introduced technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration – and most importantly, how to exploit them for stateful services such as MongoDB.
What You Will Learn:
* Why organizations are choosing to use microservice architectures, what benefits they deliver, and when they should – or shouldn't – be used.
* Technologies that are used to build microservices – including containers, orchestration, and messaging systems.
* Why MongoDB is a good fit for microservices and what special steps need to be taken to make them work well together.
Powering Microservices with Docker, Kubernetes, Kafka, & MongoDBMongoDB
This document discusses powering microservices architectures with containers, Docker, Kubernetes, Kafka, and MongoDB. It begins with an overview of microservices and why they are used, then covers containers and Docker. It describes how Kafka can be used to connect microservices and discusses orchestration with Kubernetes and Mesos. The document explains why MongoDB is a good fit for microservices due to its flexibility, scalability, and simplicity. It provides examples of running MongoDB with containers and orchestrating it using Kubernetes.
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg SchadNETWAYS
Apache Mesos has the ability to run on every private and cloud instance, anywhere. In this talk, Jörg Schad (Software Developer at Mesosphere) will explain the momentum behind the “single computer” abstraction that has put Mesos at the center of one of the most exciting architecture shifts in recent information technology history. He will explain how Mesos is enabling application developers and devops to redefine their responsibilities and shorten the amount of time it takes to write and ship production code. Jörg will outline how Mesos is empowering the new class of “datacenter developers” to program directly against datacenter resources, and draw correlations to how the Linux revolutionized the server industry.
This document provides an overview of Container as a Service (CaaS) with Docker. It discusses key concepts like Docker containers, images, and orchestration tools. It also covers DevOps practices like continuous delivery that are enabled by Docker. Specific topics covered include Docker networking, volumes, and orchestration with Docker Swarm and compose files. Examples are provided of building and deploying Java applications with Docker, including Spring Boot apps, Java EE apps, and using Docker for builds. Security features of Docker like content trust and scanning are summarized. The document concludes by discussing Docker use cases across different industries and how Docker enables critical transformations around cloud, DevOps, and application modernization.
Similar to The Rise of Microservices - Containers and Orchestration (20)
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
This presentation discusses migrating data from other data stores to MongoDB Atlas. It begins by explaining why MongoDB and Atlas are good choices for data management. Several preparation steps are covered, including sizing the target Atlas cluster, increasing the source oplog, and testing connectivity. Live migration, mongomirror, and dump/restore options are presented for migrating between replicasets or sharded clusters. Post-migration steps like monitoring and backups are also discussed. Finally, migrating from other data stores like AWS DocumentDB, Azure CosmosDB, DynamoDB, and relational databases are briefly covered.
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
MongoDB Kubernetes operator and MongoDB Open Service Broker are ready for production operations. Learn about how MongoDB can be used with the most popular container orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications. A demo will show you how easy it is to enable MongoDB clusters as an External Service using the Open Service Broker API for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB
Humana, like many companies, is tackling the challenge of creating real-time insights from data that is diverse and rapidly changing. This is our journey of how we used MongoDB to combined traditional batch approaches with streaming technologies to provide continues alerting capabilities from real-time data streams.
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB
Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe.
This talk covers:
Common components of an IoT solution
The challenges involved with managing time-series data in IoT applications
Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance.
How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts
At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB
Our clients have unique use cases and data patterns that mandate the choice of a particular strategy. To implement these strategies, it is mandatory that we unlearn a lot of relational concepts while designing and rapidly developing efficient applications on NoSQL. In this session, we will talk about some of our client use cases, the strategies we have adopted, and the features of MongoDB that assisted in implementing these strategies.
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB
Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch".
This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
When you need to model data, is your first instinct to start breaking it down into rows and columns? Mine used to be too. When you want to develop apps in a modern, agile way, NoSQL databases can be the best option. Come to this talk to learn how to take advantage of all that NoSQL databases have to offer and discover the benefits of changing your mindset from the legacy, tabular way of modeling data. We’ll compare and contrast the terms and concepts in SQL databases and MongoDB, explain the benefits of using MongoDB compared to SQL databases, and walk through data modeling basics so you feel confident as you begin using MongoDB.
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB
The document discusses guidelines for ordering fields in compound indexes to optimize query performance. It recommends the E-S-R approach: placing equality fields first, followed by sort fields, and range fields last. This allows indexes to leverage equality matches, provide non-blocking sorts, and minimize scanning. Examples show how indexes ordered by these guidelines can support queries more efficiently by narrowing the search bounds.
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB
Aggregation pipeline has been able to power your analysis of data since version 2.2. In 4.2 we added more power and now you can use it for more powerful queries, updates, and outputting your data to existing collections. Come hear how you can do everything with the pipeline, including single-view, ETL, data roll-ups and materialized views.
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB
The document describes a methodology for data modeling with MongoDB. It begins by recognizing the differences between document and tabular databases, then outlines a three step methodology: 1) describe the workload by listing queries, 2) identify and model relationships between entities, and 3) apply relevant patterns when modeling for MongoDB. The document uses examples around modeling a coffee shop franchise to illustrate modeling approaches and techniques.
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
MongoDB Atlas Data Lake is a new service offered by MongoDB Atlas. Many organizations store long term, archival data in cost-effective storage like S3, GCP, and Azure Blobs. However, many of them do not have robust systems or tools to effectively utilize large amounts of data to inform decision making. MongoDB Atlas Data Lake is a service allowing organizations to analyze their long-term data to discover a wealth of information about their business.
This session will take a deep dive into the features that are currently available in MongoDB Atlas Data Lake and how they are implemented. In addition, we'll discuss future plans and opportunities and offer ample Q&A time with the engineers on the project.
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB
Virtual assistants are becoming the new norm when it comes to daily life, with Amazon’s Alexa being the leader in the space. As a developer, not only do you need to make web and mobile compliant applications, but you need to be able to support virtual assistants like Alexa. However, the process isn’t quite the same between the platforms.
How do you handle requests? Where do you store your data and work with it to create meaningful responses with little delay? How much of your code needs to change between platforms?
In this session we’ll see how to design and develop applications known as Skills for Amazon Alexa powered devices using the Go programming language and MongoDB.
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB
aux Core Data, appréciée par des centaines de milliers de développeurs. Apprenez ce qui rend Realm spécial et comment il peut être utilisé pour créer de meilleures applications plus rapidement.
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB
Il n’a jamais été aussi facile de commander en ligne et de se faire livrer en moins de 48h très souvent gratuitement. Cette simplicité d’usage cache un marché complexe de plus de 8000 milliards de $.
La data est bien connu du monde de la Supply Chain (itinéraires, informations sur les marchandises, douanes,…), mais la valeur de ces données opérationnelles reste peu exploitée. En alliant expertise métier et Data Science, Upply redéfinit les fondamentaux de la Supply Chain en proposant à chacun des acteurs de surmonter la volatilité et l’inefficacité du marché.
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDBMongoDB
Chaque entreprise devient une entreprise de logiciels, fournissant des solutions client pour accéder à une variété de services et d'informations. Les entreprises commencent maintenant à valoriser leurs données et à obtenir de meilleures informations pour l'entreprise. Un défi crucial consiste à s'assurer que ces données sont toujours disponibles et sécurisées pour être conformes aux objectifs commerciaux de l'entreprise et aux contraintes réglementaires des pays. MongoDB fournit la couche de sécurité dont vous avez besoin, venez découvrir comment sécuriser vos données avec MongoDB.
Generative Classifiers: Classifying with Bayesian decision theory, Bayes’ rule, Naïve Bayes classifier.
Discriminative Classifiers: Logistic Regression, Decision Trees: Training and Visualizing a Decision Tree, Making Predictions, Estimating Class Probabilities, The CART Training Algorithm, Attribute selection measures- Gini impurity; Entropy, Regularization Hyperparameters, Regression Trees, Linear Support vector machines.
We are pleased to share with you the latest VCOSA statistical report on the cotton and yarn industry for the month of May 2024.
Starting from January 2024, the full weekly and monthly reports will only be available for free to VCOSA members. To access the complete weekly report with figures, charts, and detailed analysis of the cotton fiber market in the past week, interested parties are kindly requested to contact VCOSA to subscribe to the newsletter.
Build applications with generative AI on Google CloudMárton Kodok
We will explore Vertex AI - Model Garden powered experiences, we are going to learn more about the integration of these generative AI APIs. We are going to see in action what the Gemini family of generative models are for developers to build and deploy AI-driven applications. Vertex AI includes a suite of foundation models, these are referred to as the PaLM and Gemini family of generative ai models, and they come in different versions. We are going to cover how to use via API to: - execute prompts in text and chat - cover multimodal use cases with image prompts. - finetune and distill to improve knowledge domains - run function calls with foundation models to optimize them for specific tasks. At the end of the session, developers will understand how to innovate with generative AI and develop apps using the generative ai industry trends.
Codeless Generative AI Pipelines
(GenAI with Milvus)
https://ml.dssconf.pl/user.html#!/lecture/DSSML24-041a/rate
Discover the potential of real-time streaming in the context of GenAI as we delve into the intricacies of Apache NiFi and its capabilities. Learn how this tool can significantly simplify the data engineering workflow for GenAI applications, allowing you to focus on the creative aspects rather than the technical complexities. I will guide you through practical examples and use cases, showing the impact of automation on prompt building. From data ingestion to transformation and delivery, witness how Apache NiFi streamlines the entire pipeline, ensuring a smooth and hassle-free experience.
Timothy Spann
https://www.youtube.com/@FLaNK-Stack
https://medium.com/@tspann
https://www.datainmotion.dev/
milvus, unstructured data, vector database, zilliz, cloud, vectors, python, deep learning, generative ai, genai, nifi, kafka, flink, streaming, iot, edge
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Marlon Dumas
This webinar discusses the limitations of traditional approaches for business process simulation based on had-crafted model with restrictive assumptions. It shows how process mining techniques can be assembled together to discover high-fidelity digital twins of end-to-end processes from event data.
2. 2
Agenda
1. Why Microservices
2. What are Microservices
3. Best practices
4. Containers
5. Orchestration
6. MongoDB & Microservices
7. Business benefits
8. When to use Microservices
9. Use Cases
10.Resources
3. 3
Microservices - The Attraction
Web
Scale
Speed
Iterate
Agile
ReactIsolate
Maintain
Empower
13. 13
Conway’s Law
"Any organization that designs a system will
inevitably produce a design whose structure
is a copy of the organization's
communication structure.”
– Melvin Conway
15. 15
Containers – Powering Microservices
Real world shipping containers
• Same container efficiently transports goods by road,
rail, and sea
• Contents remain untouched through all those
environments; no repacking needed
• Ubiquitous and standardized
• Simple to use – open, fill, close
• Contents of each container safe from the others
• Know how much space the container consumes
16. 16
Containers – Powering Microservices
Software containers
• Build an image containing the full application stack only
once
• Spin up many containers from the same image in
multiple environments
– Laptop, data center, cloud
– Development, QA, production, support
• Simple to use & efficient
• Contents of each container isolated from the others
– Storage, memory, namespace
• Constrain resources available to each container
– Storage, memory, CPU, IO
17. VM VMVM
VMs Containers
Bare Metal
Host Operating System
Hypervisor
Guest OS
Libraries
Apps
Service
Guest OS
Libraries
Apps
Service
Guest OS
Libraries
Apps
Service
Container ContainerContainer
Bare Metal
Host Operating System
Docker Engine
Libraries
Libraries
Apps
Libraries
Apps
Service ServiceService
18. 18
Docker
The most popular container technology
• Simple to use and has a rich ecosystem
• 100,000+ images available from Docker Hub
– Including mongo hub.docker.com/_/mongo/
– Syncs with GitHub projects
• Define new images built upon base images
• Define interfaces between containers
• LINUX, (and now) Windows, and OS X
• Runs on bare metal, VMs, and cloud. Cloud
providers supply the Docker infrastructure (e.g.
Google Container Engine)
22. 22
Containers and Microservices
Microservices built by combining multiple
containers
• Build sophisticated services from many small,
focused processes (containers)
– Well defined APIs between components
– Each component can use different libraries,
middleware & programming languages
• Modular, decoupled architecture simplifies
maintenance and enables reuse
• Fault tolerant
• Scalable
Cmglee
24. 24
Orchestration
Automated deployment, connecting, and
maintenance of multiple containers
• Provision hosts
• Instantiate containers
• Reschedule failed containers
• Link containers through defined interfaces
• Expose services to the outside world
• Scale out and back in
25. 25
Orchestration – Kubernetes
Created by Google, feature-rich and widely
adopted
• Automated container deployment and ‘replication’
• On-line scale out/in
• Rolling upgrades
• HA – automatic rescheduling of failed containers
• Exposure of network ports to external apps
• Load balancing over groups of containers providing
a service
• Provided as a service by Google Compute Engine
26. 26
Orchestration – Apache Mesos
Designed to scale to 10,000s of physical
servers; used by Twitter, Airbnb & Apple
• Developer writes code to turn application into a
framework to run on Mesos
• Less feature rich than Kubernetes; considers many
functions such as load balancing, rescheduling, and
scaling to be a higher level function
– Project exists to run Kubernetes as a Mesos
framework
• Foundation for distributed systems
– Apache Aurora, Chronos, Marathon
27. 27
Choosing an Orchestration Framework
Factors to consider…
• Integration with existing DevOps frameworks?
• Number of hosts?
• Bare metal, VMs, or cloud deployment?
• Automated High Availability?
• Grouping and load balancing?
• Existing skills?
• Install your own orchestration framework or use as a
service?
28. 28
Security
Containers provide opportunities to
improve security
• Containers provide isolation; resources can only be
accessed from outside through explicitly provided
APIs
• Resources can be rationed
• A container’s role can be very narrow – remove
anything not required
• Images and containers should be kept current;
rolling upgrades with Kubernetes or Aurora
• Typically log into container as root so restrict access
29. 29
Orchestrating MongoDB
Orchestrating MongoDB containers
requires special treatment as it’s a
distributed, stateful application…
• State should survive rescheduling; use Kubernetes’
persistent volumes abstraction
• Replica Set members must communicate with each
other; expose external IP addresses/ports which
survive rescheduling
• Replica Set must be initialized from exactly one
member
• MongoDB must still be monitored and backed up –
MongoDB Cloud Manager
35. 35
Resources
• Case Study – FuboTV
https://www.mongodb.com/blog/post/leaf-in-the-wild-leading-
soccer-streaming-service-fubotv-scales-its-business-with-
mongodb-docker-containers-and-kubernetes
• Case Study – Square Enix
https://www.mongodb.com/blog/post/leaf-in-the-wild-square-
enix-scales-tomb-raider-hitman-absolution-deus-ex-and-more-
on-mongodb
• “Enabling Microservices – Containers &
Orchestration Explained” white paper
https://www.mongodb.com/collateral/microservices-containers-
and-orchestration-explained
• “Microservices: The Evolution of Building Modern
Applications” white paper
https://www.mongodb.com/collateral/microservices-the-
evolution-of-building-modern-applications
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
Monolith is like Spaghetti
Changing anything impacts everything else.
<= 1990s
Pre-SOA (monolithic)
Tight coupling
For a monolith to change, all must agree on each change. Each change has unanticipated effects requiring careful testing beforehand
Service Oriented Architecture is like a Pie
Easier to remove or change a piece but still need to make sure that it fits back together properly
2000s
Traditional SOA
Looser coupling
Elements in SOA are developed more autonomously but must be coordinated with others to fit into the overall design
SOA was billed as a way of delivering code re-use but people ended up designing for efficiency instead.
Microservices are like Cupcakes
Can add new ones with different flavors, remove ones that you no longer need, add more pink ones if there’s greater demand
Developers can create and activate new microservices without prior coordination with others. Their adherence to MSA principles makes continuous delivery of new or modified services possible
Greater modularity, looser coupling.
Started in the web and mobile app world, moving to Enterprise. Big in media and startups
Plan for flexibility rather than reuse
Each of the ovals represents a microservice.
Each source of social media feeds has its own microservice which is specialised in interfacing with the relevant API.
Each of those microservices passes messages to the ‘feed merge’ microservice which can then make them available for further microservices to work with.
Communication between the microservices is over the network – they can be local to the same machine or distributed.
Best practice is for each microservice to be stateless and to have its own database or schema
Individual microservices can be updated in isolation or even removed if their role is no longer needed
When a new role (or even a change to an existing one) appears, best practice is to implement a new microservice rather than extending an existing one.
When a new role (or even a change to an existing one) appears, best practice is to implement a new microservice rather than extending an existing one.
Microservices allow scale-out.
Each type of microservice can be scaled independently – add extra instances just for the functions that are being overworked.
Multiple instances of each service can provide High Availability
An alternate view is that each microservice is much larger – in this case User Account, Product Catalog, Inventory & Order. The key is to do what makes sense for your enterprise: which pieces need scaling independently, which would you like to upgrade independently, how does it fit into your organisation?
Functions are decoupled into four separate services (separate WAR files) and organized by business boundaries
Web pages, Classes, Config files to a service are grouped together
Services can evolve and iterate independently as each team can develop, test, and deploy code separately
Best practice is for each microservice to have its own database.
Melvin Edward Conway is a computer scientist, computer programmer, and hacker who coined what's now known as Conway's Law in 1967.
-Organizations design systems which copy the organization
-If the parts of the organization do not closely reflect the essential parts of the product, or if the relationship between organizations do not reflect relationship between product parts, than the project will be in trouble
-Make sure the organization is compatible with the product architecture
Best practice is for each microservice to be small enough that a single developer should be able to understand its entire codebase (think in hundreds rather than 10,000s of lines of code.
The code for a microservice should be owned by the organization responsible for that function; for example the marketing development team should own the microservice responsible for sending nurture track emails.
Netflix was one of the pioneers of Micrservices with their ”Cloud Native” approach – it was really all about being able to scale their development organisation.
To do useful work, microservices need a way of communicating – Apache Kafka
Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
Examples of **events** (or **messages**) include:
A periodic sensor reading such as the current temperature
A user adding an item to the shopping cart in an online store
A Tweet being sent with a specific hashtag
A log entry generated for each click in a web application
Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.
Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
A microservice can be upgraded and then catch up on everything it missed
Or even apply its updated business logic to the full history of events
A new microservice can be added and it can be brought up to speed with everything that’s gone before
If one service is generating more work than another can keep up with then Kafka operates as a buffer
[Apache Aurora](http://aurora.apache.org/) – a highly scalable service scheduler for long-running services and `cron` jobs; it's used by Twitter. Aurora extends Mesos by adding rolling updates, service registration, and resource quotas.
[Chronos](https://github.com/mesos/chronos) – a fault tolerant service scheduler, to be used as a replacement for `cron`, to orchestrate scheduled jobs within Mesos.
[Marathon](https://mesosphere.github.io/marathon/) – a simple to use service scheduler; it builds upon Mesos and Chronos by ensuring that two Chronos instances are running.
Key Message here:
Containers were originally assumed to be stateless but there’s now a demand for running DBs in a container.
Kubernetes has been adding some features to help (e.g. persistent volumes)
Still some gotchas for MongoDB but we have a white paper explaining exactly how to build and deploy a replica set with Docker and Kubernetes
Because we’re using external IP addresses, we can instead create the replica set with nodes in different regions
DevOps & Continuous Delivery
Low impact & risk; update one container at a time
Replicating environments
Instantiate clones for development, QA, production, support…
Scalability
Add and remove containers based on demand
Accurate Testing
Confident your stack exactly matches what’s in production
Isolation
Safely run multiple environments on the same hosts
Performance
Minimal impact from container overhead
High Availability
Redundancy from multiple containers fulfilling a role
Faster Time to Market
Organisational alignment
Cost reduction
More efficient use of infrastructure
Fast is more important than elegant.
Change in the application’s functionality and usage is frequent.
Change occurs at different rates within the application, so functional isolation and simple integration are more important than module cohesiveness.
Functionality is easily separated into simple, isolatable components.
When you have the developer/DevOps skillsets.
Where development org boundaries match service boundaries.
Don’t forget that you’re building a distributed system -> complexity but there are precedents to read up on.
One argument is that you shouldn’t bother with microservices unless you need either:
- Scale your team
- Design for change
Sagrada Familia – designed by Gaudi; construction started on March 19, 1882. Expected to be finished in 2026.
Flexible data model
Fits in with the need to be agile
Monitoring & automation
You have lots of moving parts in a microservice architecture – need to monitor and automate as much as possible
Redundancy
MongoDB replica sets -> can perform online upgrades and easily cope with rescheduling
Scalability
MongoDB sharding lets your database scale just as easily as your stateless microservices
Remember web-scale?
Best practice for each service to have its own schema or database; MongoDB’s simplicity helps
GAP moved their purchase order system from a monolith architecture to microservices. Due to MongoDB’s flexible schema, it took just 75 days to build the new system. When requirements changed and they had to add new types of purchase orders, it took days instead of months.
FuboTV is a North American soccer streaming service. Using Microservices with Kubernetes, Docker & MongoDB. Isolation means that they can use a single cluster of machines (in Google Cloud) for dev, QA & production. Very birsty application – scalability lets them handle 100x increases in traffic.
Otto – the key was to have an architecture that fits with their organization. Microservices empower loosely couple development teams (business, project management, IT). This is all enabling Fast test & deployment + Iterative, Continuous Delivery
Backcountry.com is an online specialty retailer that sells outdoor clothing and gear. The driver to Microservices for them was a growing, distributed development team. As more and more developers joined and made contributions to the code, the schemas became convoluted and harder to maintain; contributing to 20% of the Scrum backlog. Taking advantage of MongoDB’s flexible data model, Backcountry was able to iterate faster, reduce development time, and mitigate technical debt.