Speaker: Andrew Morgan
Organizations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Replicate your complete application for your development, test, operations, and support teams. This session introduces you to technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration, and most importantly, how to exploit them for stateful services such as MongoDB.
A basic introduction to Kubernetes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
Virtualization, Containers, Docker and scalable container management servicesabhishek chawla
In this presentation we take you through the concept of virtualization which includes the different types of virtualizations, understanding the Docker as a software containerization platform like Docker's Architecture, Building and running custom images in Docker containers, Scalable container management services which include overview of Amazon ECS & kubernetes and how at LimeTray we harnessed the power of kubernetes for scalable automated deployment of our microservices.
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaEdureka!
****** DevOps Training : https://www.edureka.co/devops ******
This DevOps Jenkins Tutorial on what is Jenkins ( Jenkins Tutorial Blog Series: https://goo.gl/JebmnW ) will help you understand what is Continuous Integration and why it was introduced. This tutorial also explains how Jenkins achieves Continuous Integration in detail and includes a Hands-On session around Jenkins by the end of which you will learn how to compile a code that is present in GitHub, Review that code and Analyse the test cases present in the GitHub repository. The Hands-On session also explains how to create a build pipeline using Jenkins and how to add Jenkins Slaves.
The Hands-On session is performed on an Ubuntu-64bit machine in which Jenkins is installed.
To learn how Jenkins can be used to integrate multiple DevOps tools, watch the video titled 'DevOps Tools', by clicking this link: https://goo.gl/up9iwd
Check our complete DevOps playlist here: http://goo.gl/O2vo13
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...Edureka!
***** Kubernetes Certification Training: https://www.edureka.co/kubernetes-certification *****
This Edureka tutorial on "What is Kubernetes" will give you an introduction to one of the most popular Devops tool in the market - Kubernetes, and its importance in today's IT processes. This tutorial is ideal for beginners who want to get started with Kubernetes & DevOps. The following topics are covered in this training session:
1. Need for Kubernetes
2. What is Kubernetes and What it's not
3. How does Kubernetes work?
4. Use-Case: Kubernetes @ Pokemon Go
5. Hands-on: Deployment with Kubernetes
DevOps Tutorial Blog Series: https://goo.gl/P0zAfF
A basic introduction to Kubernetes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
Virtualization, Containers, Docker and scalable container management servicesabhishek chawla
In this presentation we take you through the concept of virtualization which includes the different types of virtualizations, understanding the Docker as a software containerization platform like Docker's Architecture, Building and running custom images in Docker containers, Scalable container management services which include overview of Amazon ECS & kubernetes and how at LimeTray we harnessed the power of kubernetes for scalable automated deployment of our microservices.
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaEdureka!
****** DevOps Training : https://www.edureka.co/devops ******
This DevOps Jenkins Tutorial on what is Jenkins ( Jenkins Tutorial Blog Series: https://goo.gl/JebmnW ) will help you understand what is Continuous Integration and why it was introduced. This tutorial also explains how Jenkins achieves Continuous Integration in detail and includes a Hands-On session around Jenkins by the end of which you will learn how to compile a code that is present in GitHub, Review that code and Analyse the test cases present in the GitHub repository. The Hands-On session also explains how to create a build pipeline using Jenkins and how to add Jenkins Slaves.
The Hands-On session is performed on an Ubuntu-64bit machine in which Jenkins is installed.
To learn how Jenkins can be used to integrate multiple DevOps tools, watch the video titled 'DevOps Tools', by clicking this link: https://goo.gl/up9iwd
Check our complete DevOps playlist here: http://goo.gl/O2vo13
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...Edureka!
***** Kubernetes Certification Training: https://www.edureka.co/kubernetes-certification *****
This Edureka tutorial on "What is Kubernetes" will give you an introduction to one of the most popular Devops tool in the market - Kubernetes, and its importance in today's IT processes. This tutorial is ideal for beginners who want to get started with Kubernetes & DevOps. The following topics are covered in this training session:
1. Need for Kubernetes
2. What is Kubernetes and What it's not
3. How does Kubernetes work?
4. Use-Case: Kubernetes @ Pokemon Go
5. Hands-on: Deployment with Kubernetes
DevOps Tutorial Blog Series: https://goo.gl/P0zAfF
Docker Explained | What Is A Docker Container? | Docker Simplified | Docker T...Edureka!
( ** DevOps Training: https://www.edureka.co/devops ** )
This Docker Explained PPT will explain to you the fundamentals of Docker with a hands-on. Below are the topics covered in the PPT:
Problems Before Docker
Virtualization vs Containerization
What is Docker?
How does Docker work?
Docker Components
Docker Architecture
Docker Compose & Docker Swarm
Hands-On
The most important element in Jenkins architecture is the Jenkins slave. It is a java executable running on a remote machine that hear the requests from the Jenkins master instance .
Read more...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...Edureka!
This DevOps Docker Tutorial on what is docker ( Docker Tutorial Blog Series: https://goo.gl/32kupf ) will help you understand how to use Docker Hub, Docker Images, Docker Container & Docker Compose. This tutorial explains Docker's working Architecture and Docker Engine in detail. This Docker tutorial also includes a Hands-On session around Docker by the end of which you will learn to pull a centos Docker Image and spin your own Docker Container. You will also see how to launch multiple docker containers using Docker Compose. Finally, it will also tell you the role Docker plays in the DevOps life-cycle.
The Hands-On session is performed on an Ubuntu-64bit machine in which Docker is installed.
An in depth overview of Kubernetes and it's various components.
NOTE: This is a fixed version of a previous presentation (a draft was uploaded with some errors)
An introduction to docker; the concepts; how to use it and why. The presentation is mainly based on the following presentation by docker, but with added info about Docker Compose and Docker Swarm.
https://www.slideshare.net/Docker/docker-101-nov-2016
#container #docker #Trifork #TriforkSelected #GotoConf
Introduction to Microsoft Azure Well Architected Framework in Italian - Session 1 of 6
Introduzione a Microsoft Azure Well Architected Framework in Italiano - Sessione 1 di 6
Modulo 1: introduzione, principi e concetti base
Github Actions enables you to create custom software development lifecycle workflows directly in your Github repository. These workflows are made out of different tasks so-called actions that can be run automatically on certain events.
Powering Microservices with Docker, Kubernetes, Kafka, and MongoDBMongoDB
Speaker: Andrew Morgan, Principal Product Marketing Manager, MongoDB
Level: 100 (Beginner)
Track: Microservices
Organizations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Replicate your complete application for your development, test, operations, and support teams. This session introduces you to technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration – and most importantly, how to exploit them for stateful services such as MongoDB.
What You Will Learn:
- Why organizations are choosing to use microservice architectures, what benefits they deliver, and when they should - or shouldn't - be used.
- Technologies that are used to build microservices – including containers, orchestration and messaging systems.
- Why MongoDB is a good fit for microservices and what special steps need to be taken to make them work well together.
Docker Explained | What Is A Docker Container? | Docker Simplified | Docker T...Edureka!
( ** DevOps Training: https://www.edureka.co/devops ** )
This Docker Explained PPT will explain to you the fundamentals of Docker with a hands-on. Below are the topics covered in the PPT:
Problems Before Docker
Virtualization vs Containerization
What is Docker?
How does Docker work?
Docker Components
Docker Architecture
Docker Compose & Docker Swarm
Hands-On
The most important element in Jenkins architecture is the Jenkins slave. It is a java executable running on a remote machine that hear the requests from the Jenkins master instance .
Read more...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...Edureka!
This DevOps Docker Tutorial on what is docker ( Docker Tutorial Blog Series: https://goo.gl/32kupf ) will help you understand how to use Docker Hub, Docker Images, Docker Container & Docker Compose. This tutorial explains Docker's working Architecture and Docker Engine in detail. This Docker tutorial also includes a Hands-On session around Docker by the end of which you will learn to pull a centos Docker Image and spin your own Docker Container. You will also see how to launch multiple docker containers using Docker Compose. Finally, it will also tell you the role Docker plays in the DevOps life-cycle.
The Hands-On session is performed on an Ubuntu-64bit machine in which Docker is installed.
An in depth overview of Kubernetes and it's various components.
NOTE: This is a fixed version of a previous presentation (a draft was uploaded with some errors)
An introduction to docker; the concepts; how to use it and why. The presentation is mainly based on the following presentation by docker, but with added info about Docker Compose and Docker Swarm.
https://www.slideshare.net/Docker/docker-101-nov-2016
#container #docker #Trifork #TriforkSelected #GotoConf
Introduction to Microsoft Azure Well Architected Framework in Italian - Session 1 of 6
Introduzione a Microsoft Azure Well Architected Framework in Italiano - Sessione 1 di 6
Modulo 1: introduzione, principi e concetti base
Github Actions enables you to create custom software development lifecycle workflows directly in your Github repository. These workflows are made out of different tasks so-called actions that can be run automatically on certain events.
Powering Microservices with Docker, Kubernetes, Kafka, and MongoDBMongoDB
Speaker: Andrew Morgan, Principal Product Marketing Manager, MongoDB
Level: 100 (Beginner)
Track: Microservices
Organizations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Replicate your complete application for your development, test, operations, and support teams. This session introduces you to technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration – and most importantly, how to exploit them for stateful services such as MongoDB.
What You Will Learn:
- Why organizations are choosing to use microservice architectures, what benefits they deliver, and when they should - or shouldn't - be used.
- Technologies that are used to build microservices – including containers, orchestration and messaging systems.
- Why MongoDB is a good fit for microservices and what special steps need to be taken to make them work well together.
Powering Microservices with Docker, Kubernetes, Kafka, & MongoDBMongoDB
This session introduced technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration – and most importantly, how to exploit them for stateful services such as MongoDB.
What You Will Learn:
* Why organizations are choosing to use microservice architectures, what benefits they deliver, and when they should – or shouldn't – be used.
* Technologies that are used to build microservices – including containers, orchestration, and messaging systems.
* Why MongoDB is a good fit for microservices and what special steps need to be taken to make them work well together.
Powering Microservices with MongoDB, Docker, Kubernetes & Kafka – MongoDB Eur...Andrew Morgan
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver.
Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Need an identical copy of your application stack in multiple environments? Build your own container image and then your entire development, test, operations, and support teams can launch an identical clone environment.
Containers are revolutionizing the entire software lifecycle: from the earliest technical experiments and proofs of concept through development, test, deployment, and support. Orchestration tools manage how multiple containers are created, upgraded and made highly available. Orchestration also controls how containers are connected to build sophisticated applications from multiple, microservice containers.
This presentation introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.
MongoDB Europe 2016 - Powering Microservices with Docker, Kubernetes, and KafkaMongoDB
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. This session introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
MongoDB Kubernetes operator and MongoDB Open Service Broker are ready for production operations. Learn about how MongoDB can be used with the most popular container orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications. A demo will show you how easy it is to enable MongoDB clusters as an External Service using the Open Service Broker API for MongoDB
Making your Life Easier with MongoDB and Kafka (Robert Walters, MongoDB) Kafk...HostedbyConfluent
Kafka Connect makes it possible to easily integrate data sources like MongoDB! In this session we will first explore how MongoDB enables developers to rapidly innovate through the use of the document model. We will then put the document model to life and showcase how to integrate MongoDB and Kafka through the use of the MongoDB Connector with Apache Kafka. Finally, we will explore the different ways of using the connector including the new Confluent Cloud integration.
Cloud-native .NET Microservices mit KubernetesQAware GmbH
BASTA! 2017, Mainz: Talk von Mario-Leander Reimer (@LeanderReimer, Cheftechnologe bei QAware).
Cloud-Größen wie Google, Twitter und Netflix haben die Kernbausteine ihrer Infrastruktur quelloffen verfügbar gemacht. Das Resultat aus vielen Jahren Cloud-Erfahrung ist nun frei zugänglich, und jeder kann seine eigenen Cloud-nativen Anwendungen entwickeln – Anwendungen, die in der Cloud zuverlässig laufen und fast beliebig skalieren. Die einzelnen Bausteine wachsen zu einem großen Ganzen zusammen, dem Cloud-Native-Stack. In dieser Session stellen wir die wichtigsten Konzepte und aktuellen Schlüsseltechnologien kurz vor. Anschließend implementieren wir einen einfachen Microservice mit .NET Core und Steeltoe OSS und bringen ihn zusammen mit ausgewählten Bausteinen für Service-Discovery und Konfiguration schrittweise auf einem Kubernetes-Cluster zum Laufen.
The rise of microservices - containers and orchestrationAndrew Morgan
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. In this session, the concepts behind containers and orchestration will be explained and how to use them with MongoDB.
Tech Talk: DevOps at LeanIX @ Startup Camp BerlinLeanIX GmbH
DevOps at LeanIX - Presentation during Startup Camp Berlin 2015. Covering tools like Docker, Jenkins and Ansible.
===
LeanIX offers an innovative software-as-a-service solution for Enterprise Architecture Management (EAM), based either in a public cloud or the client’s data center.
Companies like Adidas, Axel Springer, Helvetia, RWE, Trusted Shops and Zalando use LeanIX Enterprise Architecture Management tool.
Free Trial: http://bit.ly/LeanIXDemoS
Docker moves very fast, with an edge channel released every month and a stable release every 3 months. Patrick will talk about how Docker introduced Docker EE and a certification program for containers and plugins with Docker CE and EE 17.03 (from March), the announcements from DockerCon (April), and the many new features planned for Docker CE 17.05 in May.
This talk will be about what's new in Docker and what's next on the roadmap
Enabling Hybrid Workflows with Docker/Mesos @OrbitzSteve Hoffman
Video here: http://bit.ly/oww-mesoscon2015-video
The promise of Mesos to create a datacenter-centric resource pool is a great idea. However, a decade old company with many different types of existing systems and unique workloads can't simply mass migrate to Mesos. That doesn't mean Mesos can't pay a huge part in unifying disparate workloads that construct existing systems while opening the door to new Mesos-centric development.
In this presentation, Steve will cover 3 very different use cases that are being unified under Apache Mesos including: Automated builds of existing code artifacts, automated builds of AWS AMIs via packer in EC2, and rolling deploys & supervision of Docker microservices across environments. The addition of Docker on Mesos has enabled this, but that doesn't mean that all Mesos slaves should be identical. See how the addition of localized support services on slaves can compliment your Docker on Mesos use cases.
Running MongoDB Enterprise on KubernetesAriel Jatib
Video : https://www.youtube.com/watch?v=vmIOCYZRZu4&t=2908s
Slides from Jason Mimicks presentation at the June 2018 Chicago Kubernetes Meetup - video here : https://youtu.be/vmIOCYZRZu4?t=48m28s
Luca Mezzaira ha presentato il case study di DAZN spiegando cosa sono le micro-architetture e focalizzandosi sui micro-frontends.
Durante l'intervento, il relatore ci ha spiegato come DAZN si è approcciato allo sviluppo della propria piattaforma evolvendola da una single page application a micro-frontends.
E' seguito poi uno spazio per domande e curiosità riguardo la tematica affrontata nell’intervento.
MongoDB.local DC 2018: MongoDB Ops Manager + KubernetesMongoDB
MongoDB Ops Manager is an enterprise-grade end-to-end database management, monitoring, and backup solution. Kubernetes has clearly won the orchestration-platform "wars". In this session we'll take a deep dive on how you can leverage both these technologies to host your MongoDB deployments within your Kubernetes infrastructure whether that's OpenShift, PKS, Azure AKS, or just upstream. This talk will review the core technologies, such as containers, Kubernetes, and MongoDB Ops Manager. You'll also have a chance to see real-live demos of MongoDB running on Kubernetes and managed with MongoDB Ops Manager with the MongoDB Enterprise Kubernetes Operator.
DevOps is a set of practices that automates the processes between software development and IT teams, in order that they can build, test, and release software faster and more reliably.
Docker and Cloud - Enables for DevOps - by ACA-ITStijn Wijndaele
DevOps is gericht op het tot stand brengen van een cultuur binnen organisaties waardoor het ontwikkelen, valideren en releasen van software sneller, meer betrouwbaar en frequenter kan verlopen. Om dit te realiseren staan het automatiseren van het 'software delivery process' en de bijhorende infrastructurele veranderingen centraal. Door de opkomst van 'Microservice Architecture' neemt het belang hiervan nog verder toe.
Similar to Powering Microservices with Docker, Kubernetes, Kafka, and MongoDB (20)
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
During this talk we'll navigate through a customer's journey as they migrate an existing MongoDB deployment to MongoDB Atlas. While the migration itself can be as simple as a few clicks, the prep/post effort requires due diligence to ensure a smooth transfer. We'll cover these steps in detail and provide best practices. In addition, we’ll provide an overview of what to consider when migrating other cloud data stores, traditional databases and MongoDB imitations to MongoDB Atlas.
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB
Humana, like many companies, is tackling the challenge of creating real-time insights from data that is diverse and rapidly changing. This is our journey of how we used MongoDB to combined traditional batch approaches with streaming technologies to provide continues alerting capabilities from real-time data streams.
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB
Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe.
This talk covers:
Common components of an IoT solution
The challenges involved with managing time-series data in IoT applications
Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance.
How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts
At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB
Our clients have unique use cases and data patterns that mandate the choice of a particular strategy. To implement these strategies, it is mandatory that we unlearn a lot of relational concepts while designing and rapidly developing efficient applications on NoSQL. In this session, we will talk about some of our client use cases, the strategies we have adopted, and the features of MongoDB that assisted in implementing these strategies.
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB
Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch".
This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB
MongoDB Kubernetes operator is ready for prime-time. Learn about how MongoDB can be used with most popular orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications.
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB
These days, everyone is expected to be a data analyst. But with so much data available, how can you make sense of it and be sure you're making the best decisions? One great approach is to use data visualizations. In this session, we take a complex dataset and show how the breadth of capabilities in MongoDB Charts can help you turn bits and bytes into insights.
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
When you need to model data, is your first instinct to start breaking it down into rows and columns? Mine used to be too. When you want to develop apps in a modern, agile way, NoSQL databases can be the best option. Come to this talk to learn how to take advantage of all that NoSQL databases have to offer and discover the benefits of changing your mindset from the legacy, tabular way of modeling data. We’ll compare and contrast the terms and concepts in SQL databases and MongoDB, explain the benefits of using MongoDB compared to SQL databases, and walk through data modeling basics so you feel confident as you begin using MongoDB.
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB
Join this talk and test session with a MongoDB Developer Advocate where you'll go over the setup, configuration, and deployment of an Atlas environment. Create a service that you can take back in a production-ready state and prepare to unleash your inner genius.
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB
Query performance should be the unsung hero of an application, but without proper configuration, can become a constant headache. When used properly, MongoDB provides extremely powerful querying capabilities. In this session, we'll discuss concepts like equality, sort, range, managing query predicates versus sequential predicates, and best practices to building multikey indexes.
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB
Aggregation pipeline has been able to power your analysis of data since version 2.2. In 4.2 we added more power and now you can use it for more powerful queries, updates, and outputting your data to existing collections. Come hear how you can do everything with the pipeline, including single-view, ETL, data roll-ups and materialized views.
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB
Are you new to schema design for MongoDB, or are you looking for a more complete or agile process than what you are following currently? In this talk, we will guide you through the phases of a flexible methodology that you can apply to projects ranging from small to large with very demanding requirements.
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
MongoDB Atlas Data Lake is a new service offered by MongoDB Atlas. Many organizations store long term, archival data in cost-effective storage like S3, GCP, and Azure Blobs. However, many of them do not have robust systems or tools to effectively utilize large amounts of data to inform decision making. MongoDB Atlas Data Lake is a service allowing organizations to analyze their long-term data to discover a wealth of information about their business.
This session will take a deep dive into the features that are currently available in MongoDB Atlas Data Lake and how they are implemented. In addition, we'll discuss future plans and opportunities and offer ample Q&A time with the engineers on the project.
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB
Virtual assistants are becoming the new norm when it comes to daily life, with Amazon’s Alexa being the leader in the space. As a developer, not only do you need to make web and mobile compliant applications, but you need to be able to support virtual assistants like Alexa. However, the process isn’t quite the same between the platforms.
How do you handle requests? Where do you store your data and work with it to create meaningful responses with little delay? How much of your code needs to change between platforms?
In this session we’ll see how to design and develop applications known as Skills for Amazon Alexa powered devices using the Go programming language and MongoDB.
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB
aux Core Data, appréciée par des centaines de milliers de développeurs. Apprenez ce qui rend Realm spécial et comment il peut être utilisé pour créer de meilleures applications plus rapidement.
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB
Il n’a jamais été aussi facile de commander en ligne et de se faire livrer en moins de 48h très souvent gratuitement. Cette simplicité d’usage cache un marché complexe de plus de 8000 milliards de $.
La data est bien connu du monde de la Supply Chain (itinéraires, informations sur les marchandises, douanes,…), mais la valeur de ces données opérationnelles reste peu exploitée. En alliant expertise métier et Data Science, Upply redéfinit les fondamentaux de la Supply Chain en proposant à chacun des acteurs de surmonter la volatilité et l’inefficacité du marché.
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDBMongoDB
Chaque entreprise devient une entreprise de logiciels, fournissant des solutions client pour accéder à une variété de services et d'informations. Les entreprises commencent maintenant à valoriser leurs données et à obtenir de meilleures informations pour l'entreprise. Un défi crucial consiste à s'assurer que ces données sont toujours disponibles et sécurisées pour être conformes aux objectifs commerciaux de l'entreprise et aux contraintes réglementaires des pays. MongoDB fournit la couche de sécurité dont vous avez besoin, venez découvrir comment sécuriser vos données avec MongoDB.
MongoDB .local Paris 2020: Les bonnes pratiques pour sécuriser MongoDB
Powering Microservices with Docker, Kubernetes, Kafka, and MongoDB
1. # M D B l o c a l
Andrew Morgan
Powering Microservices with Docker,
Kubernetes, Kafka, & MongoDB
MongoDB
London
@andrewmorgan
2. # M D B l o c a l
Agenda
Microservic
es
What, Why,
When
MongoDB
Why
Containers
Docker
Messaging
Kafka
Orchestrati
on
Kubernetes
Deploying
MongoDB
Options, How
1 2 3 4 5 6
4. # M D B l o c a l
Why Use Microservices? (tl;dr WebScale)
Speed Change Maintain Scale Empower
Build MVP quickly Rapid iterations Simple components Product Team == Component
React to market Isolated impact Team Committees
5. # M D B l o c a l
Why Use Microservices? (tl;dr WebScale)
Speed Change Maintain Scale Empower
Build MVP quickly Rapid iterations Simple components Product Team == Component
React to market Isolated impact Team Committees
6. # M D B l o c a l
Why Use Microservices? (tl;dr WebScale)
Speed Change Maintain Scale Empower
Build MVP quickly Rapid iterations Simple components Product Team == Component
React to market Isolated impact Team Committees
7. # M D B l o c a l
Why Use Microservices? (tl;dr WebScale)
Speed Change Maintain Scale Empower
Build MVP quickly Rapid iterations Simple components Product Team == Component
React to market Isolated impact Team Committees
8. # M D B l o c a l
Why Use Microservices? (tl;dr WebScale)
Speed Change Maintain Scale Empower
Build MVP quickly Rapid iterations Simple components Product Team == Component
React to market Isolated impact Team Committees
9. # M D B l o c a l
Why Use Microservices? (tl;dr WebScale)
Speed Change Maintain Scale Empower
Build MVP quickly Rapid iterations Simple components Product Team == Component
React to market Isolated impact Team Committees
26. # M D B l o c a l
Software containers
• 1 image -> Many containers
• Laptop, DC, cloud
• Dev, QA, production, support
• Efficient
• Isolation
• Constraints
Containers – Powering Microservices
27. # M D B l o c a l
• Simple to use
• 100K+ images on Docker
Hub
• Build images from images
• Platforms
• Linux, OS X, Windows
• Laptop, VM, Cloud,…
• Cloud services
Docker
28. # M D B l o c a l
docker run -d mongo
Run MongoDB
29. # M D B l o c a l
FROM debian:jessie-slim
RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
...
ARG MONGO_PACKAGE=mongodb-org
ENV MONGO_VERSION 3.4.9
...
RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list”
RUN set -x
&& apt-get update
&& apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION
${MONGO_PACKAGE}-server=$MONGO_VERSION
${MONGO_PACKAGE}-shell=$MONGO_VERSION
${MONGO_PACKAGE}-mongos=$MONGO_VERSION
${MONGO_PACKAGE}-tools=$MONGO_VERSION
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/mongodb
&& mv /etc/mongod.conf /etc/mongod.conf.orig
RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db
/data/configdb
VOLUME /data/db /data/configdb
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 27017
CMD ["mongod"]
https://github.com/docker-library/mongo
30. # M D B l o c a l
FROM debian:jessie-slim
RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
...
ARG MONGO_PACKAGE=mongodb-org
ENV MONGO_VERSION 3.4.9
...
RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list”
RUN set -x
&& apt-get update
&& apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION
${MONGO_PACKAGE}-server=$MONGO_VERSION
${MONGO_PACKAGE}-shell=$MONGO_VERSION
${MONGO_PACKAGE}-mongos=$MONGO_VERSION
${MONGO_PACKAGE}-tools=$MONGO_VERSION
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/mongodb
&& mv /etc/mongod.conf /etc/mongod.conf.orig
RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db
/data/configdb
VOLUME /data/db /data/configdb
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 27017
CMD ["mongod"]
https://github.com/docker-library/mongo
31. # M D B l o c a l
FROM debian:jessie-slim
RUN groupadd -r mongodb && useradd -r -g mongodb
mongodb
...
ARG MONGO_PACKAGE=mongodb-org
ENV MONGO_VERSION 3.4.9
...
RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list”
RUN set -x
&& apt-get update
&& apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION
${MONGO_PACKAGE}-server=$MONGO_VERSION
${MONGO_PACKAGE}-shell=$MONGO_VERSION
${MONGO_PACKAGE}-mongos=$MONGO_VERSION
${MONGO_PACKAGE}-tools=$MONGO_VERSION
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/mongodb
&& mv /etc/mongod.conf /etc/mongod.conf.orig
RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db
/data/configdb
VOLUME /data/db /data/configdb
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 27017
CMD ["mongod"]
https://github.com/docker-library/mongo
32. # M D B l o c a l
FROM debian:jessie-slim
RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
...
ARG MONGO_PACKAGE=mongodb-org
ENV MONGO_VERSION 3.4.9
...
RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list”
RUN set -x
&& apt-get update
&& apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION
${MONGO_PACKAGE}-server=$MONGO_VERSION
${MONGO_PACKAGE}-shell=$MONGO_VERSION
${MONGO_PACKAGE}-mongos=$MONGO_VERSION
${MONGO_PACKAGE}-tools=$MONGO_VERSION
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/mongodb
&& mv /etc/mongod.conf /etc/mongod.conf.orig
RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db /data/configdb
VOLUME /data/db /data/configdb
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 27017
CMD ["mongod"]
https://github.com/docker-library/mongo
33. # M D B l o c a l
FROM debian:jessie-slim
RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
...
ARG MONGO_PACKAGE=mongodb-org
ENV MONGO_VERSION 3.4.9
...
RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list”
RUN set -x
&& apt-get update
&& apt-get install -y
${MONGO_PACKAGE}=$MONGO_VERSION
${MONGO_PACKAGE}-server=$MONGO_VERSION
${MONGO_PACKAGE}-shell=$MONGO_VERSION
${MONGO_PACKAGE}-mongos=$MONGO_VERSION
${MONGO_PACKAGE}-tools=$MONGO_VERSION
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/mongodb
&& mv /etc/mongod.conf /etc/mongod.conf.orig
RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db
/data/configdb
VOLUME /data/db /data/configdb
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 27017
CMD ["mongod"]
https://github.com/docker-library/mongo
34. # M D B l o c a l
Many small, focused containers ->
sophisticated services
• Well defined APIs
• Independent languages &
libraries
• Modular: easy maintenance +
reuse
• Fault tolerant
• Scalable
Microservice Architectures Built on
Containers
36. # M D B l o c a l
Connecting the Microservices – Kafka
Producer 987 123...
Topic A
Consumer
New Old
37. # M D B l o c a l
Connecting the Microservices – Kafka
Producer
987 123...
Topic A
Consumer
Producer Consumer
38. # M D B l o c a l
Connecting the Microservices – Kafka
Producer
987 123...
Partition 0
Topic A
Consumer
Producer Consumer
435 123...
Partition 1
39. # M D B l o c a l
Connecting the Microservices – Kafka
Producer
LEADER
Topic A / Partition 0
Broker 1
FOLLOWER
Topic A / Partition 1
FOLLOWER
Topic A / Partition 0
Broker 2
LEADER
Topic A / Partition 1
40. # M D B l o c a l
Connecting the Microservices – Kafka
Producer
Producer
Producer
9
8
7
123
...
Partition 0
4
3
5
123
...
Partition 1
7
3
2
123
...
Partition N
Topic A
Topic B
7
6
5
123
...
Partition 0
Consumer
Consumer
42. # M D B l o c a l
Created by Google, feature-rich and
widely adopted
• Deployment and ‘replication’
• On-line scale out/in
• Rolling upgrades
• High Availability
• Persistence
• Ports
• Load balancing
• Google Compute Engine
Kubernetes
44. # M D B l o c a l
How to use MongoDB with Containers
Twitter
Ingest
Snapchat
Ingest
Feed
merge
Facebook
Ingest
Whatsapp
Ingest
Snapchat
Ingest
Snapchat
Ingest
MongoDB Atlas
45. # M D B l o c a l
How to use MongoDB with Containers
Twitter
Ingest
Snapchat
Ingest
Feed
merge
Facebook
Ingest
Whatsapp
Ingest
Snapchat
Ingest
Snapchat
Ingest
Kubernetes
Ops Mgr
agent
Ops Mgr
agent
Ops Mgr
agent
46. # M D B l o c a l
How to use MongoDB with Containers
Twitter
Ingest
Snapchat
Ingest
Feed
merge
Facebook
Ingest
Whatsapp
Ingest
Snapchat
Ingest
Snapchat
Ingest
Kubernetes
mongod
mongod
mongod
47. # M D B l o c a l
Kubernetes Building Blocks
POD Host Host Service
50. # M D B l o c a l
Beta in Kubernetes 1.5+
• Stable, predictable, unique
network identifiers
• IP addresses may change
• Stable, persistent storage
• Ordered, graceful deployment
and scaling (0 N-1)
• Ordered, graceful deletion and
termination (N-1 0)
StatefulSets
51. # M D B l o c a l
MongoDB Replica
Set as StatefulSet
rs.initiate()
rs.add('mongo-1.mongo:27017')
rs.add('mongo-2.mongo:27017')
52. # M D B l o c a l
• Enabling Microservices: Containers & Orchestration Explained
https://www.mongodb.com/collateral/microservices-containers-and-orchestration-explained
• Microservices: The Evolution of Building Modern Applications
https://www.mongodb.com/collateral/microservices-the-evolution-of-building-modern-applications
• Data Streaming with Apache Kafka & MongoDB
https://www.mongodb.com/collateral/data-streaming-with-apache-kafka-and-mongodb
• Guidance and examples
http://k8smongodb.net/
References
53. # M D B l o c a l
MongoDB & Microservices in the Wild
2 mins
A microservice architecture is one where you break an application down into multiple processes; where each one:
Is self-contained
Has a single, well defined role
The microservices communicate with each other using network technologies (even if running on the same host)
3 min.
Web & Mobile Apps => Enterprise
Micro Web Services
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
Web & Mobile Apps => Enterprise
Micro Web Services
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
Web & Mobile Apps => Enterprise
Micro Web Services
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
Web & Mobile Apps => Enterprise
Micro Web Services
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
Web & Mobile Apps => Enterprise
Micro Web Services
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
Web & Mobile Apps => Enterprise
Micro Web Services
Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
Benefits of microservices:
- Development Speed
- Rapid Iteration
Evolve quickly, continuous deployment
Isolate impact of changes to existing functions or just add a new one
Reactive development
Maintainable
Independent, empowered work teams
5 mins.
Add a new flavor independently.
Chef making chocolate knows that well but need not know the others
Blueberry goes out of fashion, remove them
Need more green cakes, add them
Improved pink frosting, throw out the old ones and the new ones.
Microservices are like Cupcakes
Can add new ones with different flavors, remove ones that you no longer need, add more pink ones if there’s greater demand
Developers can create and activate new microservices without prior coordination with others. Their adherence to MSA principles makes continuous delivery of new or modified services possible
Greater modularity, looser coupling.
Started in the web and mobile app world, moving to Enterprise. Big in media and startups
Plan for flexibility rather than reuse
7 mins.
Each of the ovals represents a microservice.
Each source of social media feeds has its own microservice which is specialised in interfacing with the relevant API.
Each of those microservices passes messages to the ‘feed merge’ microservice which can then make them available for further microservices to work with.
Communication between the microservices is over the network – they can be local to the same machine or distributed.
Best practice is for each microservice to be stateless and to have its own database or schema
Individual microservices can be updated in isolation or even removed if their role is no longer needed
When a new role (or even a change to an existing one) appears, best practice is to implement a new microservice rather than extending an existing one.
When a new role (or even a change to an existing one) appears, best practice is to implement a new microservice rather than extending an existing one.
Microservices allow scale-out.
Each type of microservice can be scaled independently – add extra instances just for the functions that are being overworked.
Multiple instances of each service can provide High Availability
9 mins
Fast > Elegant
Sagrada Familia –1882-2026 (144 years).
Frequent, localised changes
Localised scaling
Upgrades
Only if:
Scaling team
Designing for change
Fast is more important than elegant.
Change in the application’s functionality and usage is frequent.
Change occurs at different rates within the application, so functional isolation and simple integration are more important than module cohesiveness.
Functionality is easily separated into simple, isolatable components.
When you have the developer/DevOps skillsets.
Where development org boundaries match service boundaries.
Don’t forget that you’re building a distributed system -> complexity but there are precedents to read up on.
One argument is that you shouldn’t bother with microservices unless you need either:
- Scale your team
- Design for change
Sagrada Familia – designed by Gaudi; construction started on March 19, 1882. Expected to be finished in 2026.
11 mins
13 mins
Software containers
Build an image containing the full application stack only once
Spin up many containers from the same image in multiple environments
Laptop, data center, cloud
Development, QA, production, support
Simple to use & efficient
Contents of each container isolated from the others
Storage, memory, namespace
Constrain resources available to each container
Storage, memory, CPU, IO
16 mins
The most popular container technology
Simple to use and has a rich ecosystem
100,000+ images available from Docker Hub
Including mongo hub.docker.com/_/mongo/
Syncs with GitHub projects
Define new images built upon base images
Define interfaces between containers
LINUX, (and now) Windows, and OS X
Runs on bare metal, VMs, and cloud. Cloud providers supply the Docker infrastructure (e.g. Google Container Engine)
Not best practice
The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg <varname>=<value> flag.
The ENV instruction sets the environment variable <key> to the value <value>.The environment variables set using ENV will persist when a container is run from the resulting image.
20 mins
Microservices built by combining multiple containers
Build sophisticated services from many small, focused processes (containers)
Well defined APIs between components
Each component can use different libraries, middleware & programming languages
Modular, decoupled architecture simplifies maintenance and enables reuse
Fault tolerant
Scalable
21 mins
21 mins
3,6 Change Streams
To do useful work, microservices need a way of communicating – Apache Kafka
Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
Examples of **events** (or **messages**) include:
A periodic sensor reading such as the current temperature
A user adding an item to the shopping cart in an online store
A Tweet being sent with a specific hashtag
A log entry generated for each click in a web application
Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.
Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
A microservice can be upgraded and then catch up on everything it missed
Or even apply its updated business logic to the full history of events
A new microservice can be added and it can be brought up to speed with everything that’s gone before
If one service is generating more work than another can keep up with then Kafka operates as a buffer
To do useful work, microservices need a way of communicating – Apache Kafka
Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
Examples of **events** (or **messages**) include:
A periodic sensor reading such as the current temperature
A user adding an item to the shopping cart in an online store
A Tweet being sent with a specific hashtag
A log entry generated for each click in a web application
Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.
Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
A microservice can be upgraded and then catch up on everything it missed
Or even apply its updated business logic to the full history of events
A new microservice can be added and it can be brought up to speed with everything that’s gone before
If one service is generating more work than another can keep up with then Kafka operates as a buffer
To do useful work, microservices need a way of communicating – Apache Kafka
Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
Examples of **events** (or **messages**) include:
A periodic sensor reading such as the current temperature
A user adding an item to the shopping cart in an online store
A Tweet being sent with a specific hashtag
A log entry generated for each click in a web application
Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.
Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
A microservice can be upgraded and then catch up on everything it missed
Or even apply its updated business logic to the full history of events
A new microservice can be added and it can be brought up to speed with everything that’s gone before
If one service is generating more work than another can keep up with then Kafka operates as a buffer
To do useful work, microservices need a way of communicating – Apache Kafka
Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
Examples of **events** (or **messages**) include:
A periodic sensor reading such as the current temperature
A user adding an item to the shopping cart in an online store
A Tweet being sent with a specific hashtag
A log entry generated for each click in a web application
Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.
Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
A microservice can be upgraded and then catch up on everything it missed
Or even apply its updated business logic to the full history of events
A new microservice can be added and it can be brought up to speed with everything that’s gone before
If one service is generating more work than another can keep up with then Kafka operates as a buffer
To do useful work, microservices need a way of communicating – Apache Kafka
Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
Examples of **events** (or **messages**) include:
A periodic sensor reading such as the current temperature
A user adding an item to the shopping cart in an online store
A Tweet being sent with a specific hashtag
A log entry generated for each click in a web application
Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.
Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
A microservice can be upgraded and then catch up on everything it missed
Or even apply its updated business logic to the full history of events
A new microservice can be added and it can be brought up to speed with everything that’s gone before
If one service is generating more work than another can keep up with then Kafka operates as a buffer
23 mins
23 mins
Cloud Native Computing Foundation
Created by Google, feature-rich and widely adopted
Automated container deployment and ‘replication’
On-line scale out/in
Rolling upgrades
HA – automatic rescheduling of failed containers
Exposure of network ports to external apps
Load balancing over groups of containers providing a service
Provided as a service by Google Compute Engine
25.5 mins
27.5 mins
29.5 mins
Kubernetes.
Single Pod/comtainer/mongod in a ReplicationController.
Use external IP addresses (other IP addresses & hostnames are local to Kubernetes cluster and they change)
Refer to white paper for details
31 mins
31.5 mins
33.5 mins
35.5 mins
36.5 mins
Gap (flexibility): Monolith -> microservice (75 days). New types of PO took just days
FuboTV (scalability). Single cluster for dev, QA + production. Cope with 100x bursts. Run MongoDB on Kubernetes
Otto (arch == org). Fast, iterative delivery
Backcountry (> distributed dev team): Schema changes were taking 20% of dev time. Flexible schema
Compare The Market (Use Docker, Kafka, MongoDB & Ops Manager).
GAP moved their purchase order system from a monolith architecture to microservices. Due to MongoDB’s flexible schema, it took just 75 days to build the new system. When requirements changed and they had to add new types of purchase orders, it took days instead of months.
FuboTV is a North American soccer streaming service. Using Microservices with Kubernetes, Docker & MongoDB. Isolation means that they can use a single cluster of machines (in Google Cloud) for dev, QA & production. Very birsty application – scalability lets them handle 100x increases in traffic.
Otto – the key was to have an architecture that fits with their organization. Microservices empower loosely couple development teams (business, project management, IT). This is all enabling Fast test & deployment + Iterative, Continuous Delivery
Backcountry.com is an online specialty retailer that sells outdoor clothing and gear. The driver to Microservices for them was a growing, distributed development team. As more and more developers joined and made contributions to the code, the schemas became convoluted and harder to maintain; contributing to 20% of the Scrum backlog. Taking advantage of MongoDB’s flexible data model, Backcountry was able to iterate faster, reduce development time, and mitigate technical debt.
Compare The Market: In the cloud, each microservice, or logical grouping of related microservices, is provisioned with its own MongoDB replica set running in Encrypted storage engine to further reduce our security-related surface area. Use Docker, Kafka & MongoDB.