3 DevOps teams at the Dutch National Police are building big data applications in a private cloud using microservices architecture. They develop using techniques like continuous delivery, short feedback loops, and embrace of change. Challenges include balancing performance and security with stateless services, minimizing dependencies between teams, and preventing monolithic frontends. Future plans include zero downtime deployments, cross-functional product teams, and a modular frontend.
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014Piyush Kumar
OPEN SOURCE DEVELOPERS CONFERENCE http://osdconf.in/
★ April 26-27th, Noida ★
Keynote Session By Piyush Kumar (Lead of Infrastructure and Website Operations at MakeMyTrip)
Hot to build continuously processing for 24/7 real-time data streaming platform?GetInData
You can read our blog post about it here: https://getindata.com/blog/how-to-build-continuously-processing-for-24-7-real-time-data-streaming-platform/
Hot to build continuously processing for 24/7 real-time data streaming platform?
Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen
Slides of the presentation about Kubernetes practices and learnings at NU.nl.
This presentation was the first of two at the Dutch Kubernetes meetup at the Sanoma Netherlands offices, that took place on Sept. 5th 2019
NetflixOSS Meetup S3 E1, covering latest components in Distributed Databases, Telemetry systems, Big Data tools and more. Speakers from Netflix, IBM Watson, Pivotal and Nike Digital
Diagnosing Problems in Production: Cassandra Summit 2014Jon Haddad
At the 2014 Cassandra summit we covered how to ensure that your production experience with Cassandra is top notch by identifying the proper tools that should be put in place beforehand, and what tools you need to identify problems in real time.
Presented by Jon Haddad & Blake Eggleston
Python is a great programming language that works great with Cassandra. If your goal is to get your project into production quickly and iterate fast, Python is a great solution.
These slides are an introduction to the hands on portion from GitHub. https://github.com/rustyrazorblade/python-presentation
DefCore: The Interoperability Standard for OpenStackMark Voelker
This presentation provides an introduction to the OpenStack DefCore Committee, which is working to create interoperability standards for OpenStack Powered clouds. You'll gain insight into the interoperability challenges of OpenStack clouds, and learn how DefCore creates it's Guidelines. Learn why the Technical Committee, Board of Directors, end users, and vendors have a seat at the table. You'll laugh, you'll cry, you'll immediately want to stop talking about cloud computing and go watch science fiction all night.
This talk was originally presented at the Triangle OpenStack Meetup Group's September 21, 2015 meeting in Durham, NC. A recording can be found here (this talk starts at the 46:10 mark): https://vmware.webex.com/vmware/lsr.php?RCID=a51f9e6882f54ccab8b715c8c0162484
A new revision with updates was given at a meeting of the China Open Source Cloud League on May 20, 2016 in Beijing. The slides here on Slideshare represent that presentation.
OpenStack: Toward a More Resilient CloudMark Voelker
Since it's inception over four years ago, OpenStack has become the most popular open source software for building many types of clouds in part due to the flexibility it provides. As more adoption increases, interest has increased in building OpenStack clouds on a highly available control plane infrastructure. In this talk we will provide an introduction to today's OpenStack community and software, then dive deeper into how to build more highly available, scalable OpenStack architectures. - See more at: http://www.percona.com/news-and-events/percona-university-smart-data-raleigh/openstack-toward-more-resilient-cloud#sthash.wicdUMdH.dpuf
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)Tim Bozarth
Slides from Tim Bozarth's (@timbozarth) QCon 2017 presentation (https://qconnewyork.com/ny2017/presentation/zero-production-ready-minutes)
Abstract:
The fabric of Netflix's approach to building new highly-available services is evolving. The Runtime Platform Team is focused on improving developer productivity while simultaneously making it simpler to build and maintain the high-availability services that Netflix expects. Starting with application generation, and leveraging a new approach to communication between services (RPC), we're simplifying what's needed to build a fast, reliable, and optimized service capable of delivering a fantastic customer experience.
We'll be sharing how Netflix is enabling engineers to go from "zero" to "production ready" in minutes - incorporating best-practices learned through years in the cloud. We will also share the story of transitioning from our home-grown RPC machinery to open-source standards, how we recognized when it was the right time to walk away from our own creations, and how our new approach is improving team velocity across Netflix engineering.
Dorance Martinez Cortes' presentation on customizing Nagios. The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference.
Delivering big content at NBC News with RavenDBJohn Bennett
RavenDB is a schema-less document database that offers fully ACID transactions, fast and flexible search, replication, sharding, and a simple RESTful API wrapped by clients in a growing number of languages. In this session, we will discuss the experience of developing and maintaining a RavenDB-backed CMS for one of the largest news sites in the US.
We'll cover:
- Supporting rapid evolution of the content/data model.
- Indexing for full-text, map-reduce, geospatial and other types of search.
- Replicating and sharding across servers and data centers for high-availability.
- Deploying with no downtime.
- Handling huge traffic spikes.
Scalable and Reliable Logging at PinterestKrishna Gade
At Pinterest, hundreds of services and third-party tools that are implemented in various programming languages generate billions of events every day. To achieve scalable and reliable low latency logging, there are several challenges: (1) uploading logs that are generated in various formats from tens of thousands of hosts to Kafka in a timely manner; (2) running Kafka reliably on Amazon Web Services where the virtual instances are less reliable than on-premises hardware; (3) moving tens of terabytes data per day from Kafka to cloud storage reliably and efficiently, and guaranteeing exact one time persistence per message.
In this talk, we will present Pinterest’s logging pipeline, and share our experience addressing these challenges. We will dive deep into the three components we developed: data uploading from service hosts to Kafka, data transportation from Kafka to S3, and data sanitization. We will also share our experience in operating Kafka at scale in the cloud.
Microservices in action at the Dutch National Police - Bert Jan Schrijver - C...Codemotion
At the Cloud, Big Data and Internet division of the Dutch National Police, 4 DevOps teams use the latest open source technology to build high tech, cloud native web applications using Spring Boot, Angular 5, Spark, Kafka and Jenkins 2. I'll share our experiences and real-world use cases for microservices. I’ll show how 4 teams work together on one product and I’ll talk about how we apply the principles of DevOps and Continuous Delivery. I’ll show how we handle security, build pipelines, test automation, performance tests, service discovery, automated deployments, monitoring and more!
GeekOut 2017 - Microservices in action at the Dutch National PoliceBert Jan Schrijver
Microservices? At the Police? Definitely!
At the Cloud, Big Data and Internet division of the Dutch National Police, 3 DevOps teams use the latest open source technology to build high tech, cloud native web applications in a private cloud. These applications are used to support various types of police work with data from open, online sources and are built using Spring Boot, Angular 4, Spark, Kafka and Jenkins 2.
In this session, I'll share our experiences and real-world use cases for microservices. I’ll explain our architecture, why we chose it, which challenges we face and what this all brings us. I’ll show how 3 teams work together on one product, loosely based on the models used by Spotify and Netflix, and I’ll talk about how we apply the principles of DevOps and Continuous Delivery. I’ll show how we handle security, build pipelines, test automation, performance tests, automated deployments and monitoring.
You’ll leave this session with an understanding of how this approach enables us to have the agility of a startup within the large Police organisation.
J-Spring 2017 - Microservices in action at the Dutch National PoliceBert Jan Schrijver
Microservices? At the Police? Definitely! At the Cloud, Big Data and Internet division of the Dutch National Police, 3 DevOps teams use the latest open source technology to build high tech web applications. These applications are used to police work with data from open, online sources and are built using Spring Boot, Angular 2.x, Spark, Kafka and Jenkins 2. In this session, I'll share our experiences and real-world use cases for microservices. I’ll explain our architecture, why we chose it, which challenges we face and what this all brings us. I’ll show how 3 teams work together on one product, loosely based on the models used by Spotify and Netflix, and I’ll talk about how we apply the principles of DevOps and Continuous Delivery. I’ll show how we handle security, build pipelines, test automation, performance tests, automated deployments and monitoring. You’ll leave this session with an understanding of how this approach enables us to have the agility of a startup within the large Police organisation.
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014Piyush Kumar
OPEN SOURCE DEVELOPERS CONFERENCE http://osdconf.in/
★ April 26-27th, Noida ★
Keynote Session By Piyush Kumar (Lead of Infrastructure and Website Operations at MakeMyTrip)
Hot to build continuously processing for 24/7 real-time data streaming platform?GetInData
You can read our blog post about it here: https://getindata.com/blog/how-to-build-continuously-processing-for-24-7-real-time-data-streaming-platform/
Hot to build continuously processing for 24/7 real-time data streaming platform?
Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen
Slides of the presentation about Kubernetes practices and learnings at NU.nl.
This presentation was the first of two at the Dutch Kubernetes meetup at the Sanoma Netherlands offices, that took place on Sept. 5th 2019
NetflixOSS Meetup S3 E1, covering latest components in Distributed Databases, Telemetry systems, Big Data tools and more. Speakers from Netflix, IBM Watson, Pivotal and Nike Digital
Diagnosing Problems in Production: Cassandra Summit 2014Jon Haddad
At the 2014 Cassandra summit we covered how to ensure that your production experience with Cassandra is top notch by identifying the proper tools that should be put in place beforehand, and what tools you need to identify problems in real time.
Presented by Jon Haddad & Blake Eggleston
Python is a great programming language that works great with Cassandra. If your goal is to get your project into production quickly and iterate fast, Python is a great solution.
These slides are an introduction to the hands on portion from GitHub. https://github.com/rustyrazorblade/python-presentation
DefCore: The Interoperability Standard for OpenStackMark Voelker
This presentation provides an introduction to the OpenStack DefCore Committee, which is working to create interoperability standards for OpenStack Powered clouds. You'll gain insight into the interoperability challenges of OpenStack clouds, and learn how DefCore creates it's Guidelines. Learn why the Technical Committee, Board of Directors, end users, and vendors have a seat at the table. You'll laugh, you'll cry, you'll immediately want to stop talking about cloud computing and go watch science fiction all night.
This talk was originally presented at the Triangle OpenStack Meetup Group's September 21, 2015 meeting in Durham, NC. A recording can be found here (this talk starts at the 46:10 mark): https://vmware.webex.com/vmware/lsr.php?RCID=a51f9e6882f54ccab8b715c8c0162484
A new revision with updates was given at a meeting of the China Open Source Cloud League on May 20, 2016 in Beijing. The slides here on Slideshare represent that presentation.
OpenStack: Toward a More Resilient CloudMark Voelker
Since it's inception over four years ago, OpenStack has become the most popular open source software for building many types of clouds in part due to the flexibility it provides. As more adoption increases, interest has increased in building OpenStack clouds on a highly available control plane infrastructure. In this talk we will provide an introduction to today's OpenStack community and software, then dive deeper into how to build more highly available, scalable OpenStack architectures. - See more at: http://www.percona.com/news-and-events/percona-university-smart-data-raleigh/openstack-toward-more-resilient-cloud#sthash.wicdUMdH.dpuf
Netflix: From Zero to Production-Ready in Minutes (QCon 2017)Tim Bozarth
Slides from Tim Bozarth's (@timbozarth) QCon 2017 presentation (https://qconnewyork.com/ny2017/presentation/zero-production-ready-minutes)
Abstract:
The fabric of Netflix's approach to building new highly-available services is evolving. The Runtime Platform Team is focused on improving developer productivity while simultaneously making it simpler to build and maintain the high-availability services that Netflix expects. Starting with application generation, and leveraging a new approach to communication between services (RPC), we're simplifying what's needed to build a fast, reliable, and optimized service capable of delivering a fantastic customer experience.
We'll be sharing how Netflix is enabling engineers to go from "zero" to "production ready" in minutes - incorporating best-practices learned through years in the cloud. We will also share the story of transitioning from our home-grown RPC machinery to open-source standards, how we recognized when it was the right time to walk away from our own creations, and how our new approach is improving team velocity across Netflix engineering.
Dorance Martinez Cortes' presentation on customizing Nagios. The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference.
Delivering big content at NBC News with RavenDBJohn Bennett
RavenDB is a schema-less document database that offers fully ACID transactions, fast and flexible search, replication, sharding, and a simple RESTful API wrapped by clients in a growing number of languages. In this session, we will discuss the experience of developing and maintaining a RavenDB-backed CMS for one of the largest news sites in the US.
We'll cover:
- Supporting rapid evolution of the content/data model.
- Indexing for full-text, map-reduce, geospatial and other types of search.
- Replicating and sharding across servers and data centers for high-availability.
- Deploying with no downtime.
- Handling huge traffic spikes.
Scalable and Reliable Logging at PinterestKrishna Gade
At Pinterest, hundreds of services and third-party tools that are implemented in various programming languages generate billions of events every day. To achieve scalable and reliable low latency logging, there are several challenges: (1) uploading logs that are generated in various formats from tens of thousands of hosts to Kafka in a timely manner; (2) running Kafka reliably on Amazon Web Services where the virtual instances are less reliable than on-premises hardware; (3) moving tens of terabytes data per day from Kafka to cloud storage reliably and efficiently, and guaranteeing exact one time persistence per message.
In this talk, we will present Pinterest’s logging pipeline, and share our experience addressing these challenges. We will dive deep into the three components we developed: data uploading from service hosts to Kafka, data transportation from Kafka to S3, and data sanitization. We will also share our experience in operating Kafka at scale in the cloud.
Microservices in action at the Dutch National Police - Bert Jan Schrijver - C...Codemotion
At the Cloud, Big Data and Internet division of the Dutch National Police, 4 DevOps teams use the latest open source technology to build high tech, cloud native web applications using Spring Boot, Angular 5, Spark, Kafka and Jenkins 2. I'll share our experiences and real-world use cases for microservices. I’ll show how 4 teams work together on one product and I’ll talk about how we apply the principles of DevOps and Continuous Delivery. I’ll show how we handle security, build pipelines, test automation, performance tests, service discovery, automated deployments, monitoring and more!
GeekOut 2017 - Microservices in action at the Dutch National PoliceBert Jan Schrijver
Microservices? At the Police? Definitely!
At the Cloud, Big Data and Internet division of the Dutch National Police, 3 DevOps teams use the latest open source technology to build high tech, cloud native web applications in a private cloud. These applications are used to support various types of police work with data from open, online sources and are built using Spring Boot, Angular 4, Spark, Kafka and Jenkins 2.
In this session, I'll share our experiences and real-world use cases for microservices. I’ll explain our architecture, why we chose it, which challenges we face and what this all brings us. I’ll show how 3 teams work together on one product, loosely based on the models used by Spotify and Netflix, and I’ll talk about how we apply the principles of DevOps and Continuous Delivery. I’ll show how we handle security, build pipelines, test automation, performance tests, automated deployments and monitoring.
You’ll leave this session with an understanding of how this approach enables us to have the agility of a startup within the large Police organisation.
J-Spring 2017 - Microservices in action at the Dutch National PoliceBert Jan Schrijver
Microservices? At the Police? Definitely! At the Cloud, Big Data and Internet division of the Dutch National Police, 3 DevOps teams use the latest open source technology to build high tech web applications. These applications are used to police work with data from open, online sources and are built using Spring Boot, Angular 2.x, Spark, Kafka and Jenkins 2. In this session, I'll share our experiences and real-world use cases for microservices. I’ll explain our architecture, why we chose it, which challenges we face and what this all brings us. I’ll show how 3 teams work together on one product, loosely based on the models used by Spotify and Netflix, and I’ll talk about how we apply the principles of DevOps and Continuous Delivery. I’ll show how we handle security, build pipelines, test automation, performance tests, automated deployments and monitoring. You’ll leave this session with an understanding of how this approach enables us to have the agility of a startup within the large Police organisation.
Whar are microservices and microservices architecture (MSA) How we reach them? Are they the same or SoA or not? When to use them? What are the key characteristics?
Slides of my talk given in #Gapand2017 in Andorra
Dutch Oracle Architects Platform - Reviewing Oracle OpenWorld 2017 and New Tr...Lucas Jellema
Not since the rise of Service Oriented Architecture (and the supporting Fusion Middleware technology) over a decade ago have we seen so much rapid change in terms of application and infrastructure architecture. Cloud, Microservices and DevOps are perhaps the most explicit examples – but many other developments in technology, architecture and even the industry at large have an impact on how enterprises consider and employ IT – such as machine learning, IoT, blockchain.
In this session for (infrastructure, solution, application, enterprise, security, data) architects – we will present the main stories, roadmaps and technologies from Oracle OpenWorld 2017 (and JavaOne) that influence, shape and enable architecture. We will brainstorm together on the consequences of the new directions outlined by Oracle – and coming our way from other quarters. We are seeing a a lot of change. New opportunities arise – that may become challenges or threats if we fail to recognize and embrace the change in time. This session will help us all to get a better handle on the winds in enterprise IT in general and in Oracle land in particular.
Among the topics we will present and discuss are:
- The Only Way is Up – the inevitable and imminent move from on premises to the cloud, and upwards in the stack – from IaaS to SaaS
- Security and Ops in a hybrid landscape (multiple clouds & on premises, multiple technologies & interaction channels)
- Autonomous Database – what, when, how
- Oracle’s cloud strategy, High PaaS and Low PaaS, Open [source] technology (star of the show: Apache Kafka) and the commodization of the traditional Oracle platform
- Container and Cloud Native at Oracle Cloud (Docker, Kubernetes Container Platform, Wercker, Istio Service Mesh, CNCF)
- Serverless
- Java Reborn – for microservices and cloud, modularized (highlights from the JavaOne conference)
- Disruptive: Blockchain, IoT, Machine Learning
.Net Microservices with Event Sourcing, CQRS, Docker and... Windows Server 20...Javier García Magna
Good technical practices you can follow with (micro)services but can be applied to almost anything: discovery (microphone/consul), security, resilience (polly), composition, ssecurity (jwt/oauth2)... And then an example with a CQRS application, and how docker can be used in Windows 2016. Lastly a brief summary of what Service Fabric is and its programming models.
Following simple patterns of good application design can allow you to scale your application for your customers easily. This presentation dives into the 12 factor application design and demo how this applies to containers and deployments on Amazon ECS and Fargate. We'll take a look at tooling that can be used to simplify your workflow and help you adopt the principles of the 12 factor application.
We are on the cusp of a new era of application development software: instead of bolting on operations as an after-thought to the software development process, Kubernetes promises to bring development and operations together by design.
So many times our customers need a simple routine that can be executed on a routine basis but the solution doesn’t need to be an elaborate solution without going the trouble of setting servers and other infrastructure. Serverless computer is the abstraction of servers, infrastructure, and operating systems and make getting solutions to your customer’s needs much quicker and cheaper. During this session we will look at how Azure Functions will enable you to run code on-demand without having to explicitly provision or manage infrastructure.
The evolution of micro services architecture. Mainframe, Midrange, Client Server, SOA. Best practices of microservices. Load balancing, BigData, design patterns. When and why to use microservices.
In this webinar, Michael Nash of BoldRadius explores the Typesafe Reactive Platform.
The Typesafe Reactive Platform is a suite of technologies and tools that support the creation of reactive applications, that is, applications that handle the kind of responsiveness requirements, data volume, and user load that was out of practical reach only a few years ago.
From analysis of the human genome to wearable technology to communications at a massive scale, BoldRadius has the premier team of experts with decades of collective experience in designing and building these types of applications, and in helping teams adopt these tools.
Business and IT agility through DevOps and microservice architecture powered ...Lucas Jellema
IT needs to run in production in order to generate business value. DevOps is among other things a way of thinking focusing on production software. A business application requires a tailor made platform to generate business value. The combination of application and its platform is a DevOps product. The DevOps team has full responsibility for that product through its entire lifecycle.
The microservices architecture promises flexibility, scalability, and optimal use of compute resources. Via independent components with well-defined scope and responsibility, interface, and ownership that are evolved and managed in an automated DevOps process, this architecture leverages current technologies and hard-learned insights from past decades.
This session defines the objectives of Business with IT, of microservices and DevOps and introduces Containers and the container platform Kubernetes as crucial ingredients for making DevOps happen.
Triangle Devops Meetup covering Netflix open source, cloud architecture, and what Andrew did in his first year working as a senior software engineer in the cloud platform group.
DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CDDevOps_Fest
CI/CD process has been something your DevOps engineer purpose-built for your team. But with Kubernetes & cloud-native, that’s becoming “legacy.” The rising level of platform abstraction allows all the good practices that the industry has developed over time to be integrated, hidden, and simplified behind just one practice called “GitOps.” That simplified world is what Jenkins X enables.
We will discuss GitOps, Jenkins X, and how that combination drastically simplifies cloud-native web app development. You’ll understand why traditional DevOps is not suitable in a Kubernetes and cloud-native world, explore GitOps principles and discover how they facilitate high-velocity app development.
And finally, Kohsuke will make a fool of himself by talking about the future — now that Jenkins X simplifies the CD process, where is the next frontier?
AWS re:Invent 2016: Moving Mission Critical Apps from One Region to Multi-Reg...Amazon Web Services
In gaming, low latencies and connectivity are bare minimum expectations users have while playing online on PlayStation Network. Alex and Dustin share key architectural patterns to provide low latency, multi-region services to global users. They discuss the testing methodologies and how to programmatically map out a large dependency multi-region deployment with data-driven techniques. The patterns shared show how to adapt to changing bottlenecks and sudden, several million request spikes. You’ll walk away with several key architectural patterns that can service users at global scale while being mindful of costs.
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
Talk presented at Kubernetes Community Day, New York, May 2024.
Technical summary of Multi-Cluster Kubernetes Networking architectures with focus on 4 key topics.
1) Key patterns for Multi-cluster architectures
2) Architectural comparison of several OSS/ CNCF projects to address these patterns
3) Evolution trends for the APIs of these projects
4) Some design recommendations & guidelines for adopting/ deploying these solutions.
ER(Entity Relationship) Diagram for online shopping - TAEHimani415946
https://bit.ly/3KACoyV
The ER diagram for the project is the foundation for the building of the database of the project. The properties, datatypes, and attributes are defined by the ER diagram.
This 7-second Brain Wave Ritual Attracts Money To You.!nirahealhty
Discover the power of a simple 7-second brain wave ritual that can attract wealth and abundance into your life. By tapping into specific brain frequencies, this technique helps you manifest financial success effortlessly. Ready to transform your financial future? Try this powerful ritual and start attracting money today!
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
4. The police protects the
democracy, maintains the law
and is the authority on the
streets. Around 65.000 people
work at the Dutch police, of
which over 1500 IT
professionals.
Dutch National Police
5. CLOUD
PLATFORM
ANALYSE
PATRONEN
BIG DATA
SECURITY3 DevOps teams are building
high tech big data web
applications in a private cloud
environment. These
applications support police
related themes.
Product line
Cloud | Big Data | Internet
7. • 3 teams, separate backlogs
• Overall planning at start of sprint
• Minimal planning ritual
• Usability tests as part of sprint
• (Almost) no meetings
• Phabricator as tool of choice
Methodology
8. • Continuous Delivery & DevOps
• Short feedback loops
• Embrace change
• Minimal dependencies outside team
• Invest in people, not in products
• Open, transparent, verifiable
Culture
Source: http://kids.nationalgeographic.com/explore/countries/netherlands/#netherlands-tulip-fields.jpg
9. • How to handle support, monitoring and
(pro-active) maintenance during sprint?
Make this a role that cycles through the
team.
“Operator of the day” and “Developer of
the day”
Culture
Source: http://kids.nationalgeographic.com/explore/countries/netherlands/#netherlands-tulip-fields.jpg
11. • End-to-end security and encryption
• Version control for everything
• Horizontally scalable, no single points of failure
• No runtime dependencies on external systems
• Standardised naming
• Right tool for the job (polyglot)
Architecture
23. Source: https://www.google.com/about/datacenters
• How to manage 100’s of physical servers,
1000’s of virtual servers and 1000’s of
desktops?
Infrastructure as code.
Terraform for cloud, Puppet for servers,
Ansible for desktops
Platform
26. • How to decouple building a feature from
release and deployment?
Use feature toggles.
Frontend
27. • How to prevent creating a monolithic
frontend?
Apply microservice principles to the
frontend too.
Create re-usable frontend components as
standalone component libraries.
Frontend
29. • Small in size, single responsibility
• Runs in its own process
• Independently develop, deploy, upgrade, scale
• Has its own data store
• Distributed by default
• Potentially heterogeneous/polyglot
• Light-weight communication
Anatomy of a microservice
30. • Did we build microservices right from the start?
Nope.
• Why did we move to a microservices architecture?
• Scalability: both for performance/load and
development teams
• Modularity: independent development and
deployment of each part of the system
• The ‘cool factor’ helped bit ;-)
The path to microservices
31. • How did we move to microservices?
• Split the existing system in modules (bounded
contexts) - defined by business functionality
• For each module, create a microservice
• For every microservice, check if it now only
serves 1 part of the domain
• If so: cool. If not: repeat - split it up more.
The path to microservices
32. • Spring Boot, Java 8, Maven
• Stateless
• 1 service in 1 jar on 1 JVM on 1 host (and 1 git project)
• Minimal amount of shared code:
• Security
• Logging and metrics
• Past: high available via load balancers
• Present: moving from LB’s to service discovery
Backend
33. Service discovery is an advanced pattern.
Do you need it?
Probably not.
Do we need it?
We think we do.
Service discovery
34. Service discovery
Why do we use service discovery?
• HA & load balancing without single points of failure
• Direct secure end-to-end client-service comm.
• Kerberos: picky on DNS verification
• Dynamic horizontal scaling
• Resilience: automated response to failure
• Location transparency
• Zero downtime deployments
37. • Feature branch based development
• Master branch must always be releasable
• Test environment on OpenStack runs the
master branch
• Feature branches only live locally and on
the CI server
Development
38. • How to locally run a system that consists
of ~50 services?
Run only the component(s) you work on
locally.
For other components, local env connects
to test env on OpenStack
Development
39. • Unit tests
• Mutation tests
• Service/integration tests: Spring boot
integration, embedded in-memory data
stores, REST assured
• End-to-end test: Protractor
• Load tests: Gatling
Testing
40. • How to test feature branches in a
microservices environment?
Spin up branch version of component in a
container on the CI environment.
For dependencies, connect to test env.
Avoid changing multiple components at once.
Testing
43. • How to manage >100 builds with a single
team?
Create modular, reusable build
definitions.
(See https://virtualjug.com/pipeline-as-code-building-
continuous-delivery-pipelines-with-jenkins-2/)
Build tools
44. • Every push to master is a release
• Config embedded in executable jar
• Deployments via Rundeck and Puppet
Deployments
45. • How to know when, what and which version(s)
to deploy?
• Minimise administration and think time.
• Test environment: deploy component on commit
• Everything from test -> acc during sprint
• Everything from acc -> prod after sprint
• Single component test -> acc -> prod when needed
Deployments
46. • Logging and dashboards via Graylog
• Metrics:
• Spring Boot actuator
• Distributed tracing with Zipkin
• Grafana for graphs
• Kafka stats via Burrow
• Monitoring via Sensu and Flapjack
Running in production
48. Challenges
01
Share as little as possible; prefer
duplication over coupling.
Sharing code between services
04Authentication and authorisation
happen at every request. Find the
balance between performance and
security.
Running stateless has a cost
When moving fast,
don’t forget to finish up before
starting something new.
Switching focus has a cost
06
Throwing something away and
starting over can work out better
than refactoring.
Don’t be afraid to rebuild03
Microservices are not just for the
backend. Modularity is just as
important on the frontend.
Monolithic frontend
02
Minimalize dependencies on
other teams, or it will slow you
down.
Cross functional team
composition is vital
05
and lessons learned
@bjschrijver
49. Looking ahead
Upgrades and fixes without users even
noticing.
0-downtime deployments
Our plans for the (near) future.
@bjschrijver
Cross-functional teams with vertical
(full stack) responsibilities.
Product teams
Split the frontend in products and re-
usable components.
Modular frontend
There is no silver bullet here, but useful
tools and practices do exist.
Automated security testing
Get the teams the information they
need, but only when they need it.
Better dashboards and alerting