Cloud native applications are popular these days – applications that run in the cloud reliably und scale almost arbitrarily. They follow three key principles: They are built and composed as microservices, they are packaged and distributed in containers and the containers are executed dynamically in the cloud. In this hands-on session we will show how to build, package and deploy cloud native Java EE applications on top of DC/OS - fully automated with Gradle using cloud native infrastructure like Consul, Fabio, Hystrix and Prometheus. And for the fun of it we will be using an off-the-shelf DJ pad, programmed with nothing else than the Java Sound API, to demonstrate the core concepts and to visualize and remote control DC/OS.
A lot of data is best represented as time series: Operational data, financial data, and even in data warehouses the dominant dimension is often time. We present Chronix, a time series database based on Apache Solr and Spark which is able to handle trillions of time series data points and perform interactive queries. Chronix Spark is open source software and battle-proven at a German car manufacturer and an international telco.
We demonstrate several use cases of Chronix from real-life. Afterwards we lift the curtain and deep-dive into the Chronix architecture esp. how we're using Solr to store time series data and how we've hooked up Solr with Spark. We provide some benchmarks showing how Chronix has outperformed other time series databases in both performance and storage-efficiency.
Chronix is open source under the Apache License (http://chronix.io).
A Fast and Efficient Time Series Storage Based on Apache SolrQAware GmbH
OSDC 2016, Berlin: Talk by Florian Lautenschlager (@flolaut, Senior Software Engineer at QAware)
Abstract: How to store billions of time series points and access them within a few milliseconds? Chronix! Chronix is a young but mature open source project that allows one for example to store about 15 GB (csv) of time series in 238 MB with average query times of 21 ms. Chronix is built on top of Apache Solr a bulletproof distributed NoSQL database with impressive search capabilities. In this code-intense session we show how Chronix achieves its efficiency in both respects by means of an ideal chunking, by selecting the best compression technique, by enhancing the stored data with (pre-computed) attributes, and by specialized query functions.
This session is recommended for anyone interested in building real-time streaming applications using AWS. In this session, you will get a deep understanding of how data can be ingested by Amazon Kinesis and made available for real-time analysis and processing. We’ll also show how you can leverage the Kinesis client to make your applications highly available and fault tolerant. We’ll explore various design considerations in implementing real-time solutions and explain key concepts against the backdrop of an actual use case. Finally, we’ll situate stream processing in the broader context of your big data applications.
A lot of data is best represented as time series: Operational data, financial data, and even in data warehouses the dominant dimension is often time. We present Chronix, a time series database based on Apache Solr and Spark which is able to handle trillions of time series data points and perform interactive queries. Chronix Spark is open source software and battle-proven at a German car manufacturer and an international telco.
We demonstrate several use cases of Chronix from real-life. Afterwards we lift the curtain and deep-dive into the Chronix architecture esp. how we're using Solr to store time series data and how we've hooked up Solr with Spark. We provide some benchmarks showing how Chronix has outperformed other time series databases in both performance and storage-efficiency.
Chronix is open source under the Apache License (http://chronix.io).
A Fast and Efficient Time Series Storage Based on Apache SolrQAware GmbH
OSDC 2016, Berlin: Talk by Florian Lautenschlager (@flolaut, Senior Software Engineer at QAware)
Abstract: How to store billions of time series points and access them within a few milliseconds? Chronix! Chronix is a young but mature open source project that allows one for example to store about 15 GB (csv) of time series in 238 MB with average query times of 21 ms. Chronix is built on top of Apache Solr a bulletproof distributed NoSQL database with impressive search capabilities. In this code-intense session we show how Chronix achieves its efficiency in both respects by means of an ideal chunking, by selecting the best compression technique, by enhancing the stored data with (pre-computed) attributes, and by specialized query functions.
This session is recommended for anyone interested in building real-time streaming applications using AWS. In this session, you will get a deep understanding of how data can be ingested by Amazon Kinesis and made available for real-time analysis and processing. We’ll also show how you can leverage the Kinesis client to make your applications highly available and fault tolerant. We’ll explore various design considerations in implementing real-time solutions and explain key concepts against the backdrop of an actual use case. Finally, we’ll situate stream processing in the broader context of your big data applications.
Lessons from Cassandra & Spark (Matthias Niehoff & Stephan Kepser, codecentri...DataStax
We built an application based on the principles of CQRS and Event Sourcing using Cassandra and Spark. During the project we encountered a number of challenges and problems with Cassandra and the Spark Connector.
In this talk we want to outline a few of those problems and our actions to solve them. While some problems are specific to CQRS and Event Sourcing applications most of them are use case independent.
About the Speakers
Matthias Niehoff IT-Consultant, codecentric AG
works as an IT-Consultant at codecentric AG in Germany. His focus is on big data & streaming applications with Apache Cassandra & Apache Spark. Yet he does not lose track of other tools in the area of big data. Matthias shares his experiences on conferences, meetups and usergroups.
Stephan Kepser Senior IT Consultant and Data Architect, codecentric AG
Dr. Stephan Kepser is an expert on cloud computing and big data. He wrote a couple of journal articles and blog posts on subjects of both fields. His interests reach from legal questions to questions of architecture and design of cloud computing and big data systems to technical details of NoSQL databases.
Beyond the Query – Bringing Complex Access Patterns to NoSQL with DataStax - ...StampedeCon
Learn how to model beyond traditional direct access in Apache Cassandra. Utilizing the DataStax platform to harness the power of Spark and Solr to perform search, analytics, and complex operations in place on your Cassandra data!
MongoDB Chunks - Distribution, Splitting, and MergingJason Terpko
MongoDB Chunks – Distribution, Splitting, and Merging This presentation will discuss how the distribution of chunks can create scenarios where you may need to manually move, split, or merge chunks in your sharded cluster. Scenarios requiring these actions can exist with both optimal and sub-optimal shard keys. This can be especially true when a new change has been introduced to the workload. It may cause hot-spotting, unbalanced storage, or empty chunks.
Escape from Hadoop: Ultra Fast Data Analysis with Spark & CassandraPiotr Kolaczkowski
We present the basic functionality of the official DataStax spark-cassandra connector - how to load cassandra tables as Spark RDDs and how to save Spark RDDs to Cassandra.
Massively Scalable Real-time Geospatial Data Processing with Apache Kafka and...Paul Brebner
This presentation will explore how we added location data to a scalable real-time anomaly detection application, built around Apache Kafka, and Cassandra.
Kafka and Cassandra are designed for time-series data, however, it’s not so obvious how they can process geospatial data. In order to find location-specific anomalies, we need a way to represent locations, index locations, and query locations.
We explore alternative geospatial representations including: Latitude/Longitude points, Bounding Boxes, Geohashes, and go vertical with 3D representations, including 3D Geohashes.
For each representation we also explore possible Cassandra implementations including: Clustering columns, Secondary indexes, Denormalized tables, and the Cassandra Lucene Index Plugin.
To conclude we measure and compare the query throughput of some of the solutions, and summarise the results in terms of accuracy vs. performance to answer the question “Which geospatial data representation and Cassandra implementation is best?”
Updated version of presentation for 30 April 2020 Melbourne Distributed Meetup (online)
This tutorial will guide you through the many considerations when deploying a sharded cluster. We will cover the services that make up a sharded cluster, configuration recommendations for these services, shard key selection, use cases, and how data is managed within a sharded cluster. Maintaining a sharded cluster also has its challenges. We will review these challenges and how you can prevent them with proper design or ways to resolve them if they exist today. There will be lab sessions at the end of some chapters so please have your laptops with you.
Building a Fast, Resilient Time Series Store with Cassandra (Alex Petrov, Dat...DataStax
Cassandra is awesome for many things. One of the things it's awesome for is Time Series. Combining the power of Cassandra with APIs of existing Time Series tools, such as Graphite can yield interesting results.
Cyanite is a Time Series aggregator and store built on top of Cassandra. It's fully compatible with Graphite, can serve as a plug-in replacement for Graphite and Graphite web.
Cyanite is using SASI indexes to make glob metric path queries, can query and aggregate, store, display and analyse metrics from hundreds and thousands of servers.
Which data modelling practices work best for Time Series, which new awesome Cassandra features you can use to make your Time Series analysis better.
About the Speaker
Alex Petrov Software Engineer, DataStax
Polyglot programmer. Interested in algorithms, distributed systems, algebra and high performance solutions.
A database trigger is a stored procedure that is executed when specific actions occur within a database. Triggers fit perfectly on a relational schema (foreign keys) and are implemented as a built-in functionality on popular relational database like MySQL.
MongoDB does not have any support for triggers, mainly due to the lack of support for foreign keys. Even if it usually considered an antipattern, there are use cases in MongoDB that benefit from a partially-relational schema. The lack of triggers is an obstacle for a partially-relational schema but there can be workarounds for simulating trigger behavior.
This presentation will guide you through different ways to implement triggers in MongoDB. We will cover the topics streams, tailable cursors, and hooks. We will demonstrate coding examples for each topic and we will explain pros and cons of each implementation.
Kafka is a distributed event streaming platform that has become very popular in the past couple of years.
In a MongoDB-Kafka architecture, MongoDB may be configured as both a Sink and a Source. With sink we mean to ingest events from your Kafka topics directly into MongoDB collections, exposing the data to your services for efficient querying, enrichment, and analytics. With source we mean publish data changes from MongoDB into Kafka topics for streaming to consuming apps.
We are going to cover both scenarios (using MongoDB as Sink and Source), by demonstrating ways to efficiently connect each datastore to the other. At the same time, we will cover use-cases and best practices when using Kafka and MongoDB together.
New Indexing and Aggregation Pipeline Capabilities in MongoDB 4.2Antonios Giannopoulos
MongoDB 4.2 comes GA soon delivering some amazing new features on multiple areas. In this talk, we will focus on the new capabilities of the aggregation framework. We are going to cover the new operators and expressions. At the same time, we will explore how updates commands can now use the aggregation framework operators. We are also going to present aggregation framework improvements focusing on the on-demand materialized views. Finally, we are going to explore the wildcard indexes introduced in MongoDB 4.2 and how they change the way we design documents and build queries/aggregations. We will also make a reference to the new index build system.
Massively Scalable Real-time Geospatial Data Processing with Apache Kafka and...Paul Brebner
Geospatial data makes it possible to leverage location, location, location! Geospatial data is taking off, as companies realize that just about everyone needs the benefits of geospatially aware applications. As a result there are no shortages of unique but demanding use cases of how enterprises are leveraging large-scale and fast geospatial big data processing. The data must be processed in large quantities - and quickly - to reveal hidden spatiotemporal insights vital to businesses and their end users. In the rush to tap into geospatial data, many enterprises will find that representing, indexing and querying geospatially-enriched data is more complex than they anticipated - and might bring about tradeoffs between accuracy, latency, and throughput.This presentation will explore how we added location data to a scalable real-time anomaly detection application, built around Apache Kafka, and Cassandra. Kafka and Cassandra are designed for time-series data, however, it’s not so obvious how they can process geospatial data. In order to find location-specific anomalies, we need a way to represent locations, index locations, and query locations. We explore alternative geospatial representations including: Latitude/Longitude points, Bounding Boxes, Geohashes, and go vertical with 3D representations, including 3D Geohashes. To conclude we measure and compare the query throughput of some of the solutions, and summarise the results in terms of accuracy vs. performance to answer the question “Which geospatial data representation and Cassandra implementation is best?”
Spark and Cassandra with the Datastax Spark Cassandra Connector
How it works and how to use it!
Missed Spark Summit but Still want to see some slides?
This slide deck is for you!
The overall evolution towards microservices has caused a lot of IT leaders to radically rethink architectures and platforms. One can hardly keep up with the rapid onslaught on new distributed technologies. The same people who just asked yesterday "how can we deploy Docker containers?", are now asking "how can we operate Kubernetes-as-a-Service on-premise?", and are about to start asking "how can we operate the open source frameworks of our choice, such as Spark, TensorFlow, HDFS, and more, as a service across hybrid clouds?”. This session will discuss: Challenges of orchestrating and operating.
While we move towards microservices architecture, and are managing too many services, the issue of service discovery arises. This becomes a bigger issue when we are autoscaling. So we need a system to be able to resolve a service to a specific machine in our deployment.
An introduction to Dropwizard, as well as examples of how to integrate it with Spring DI and Spring Security. Prepared for a presentation at the Houston Java User Group. Code samples at : https://github.com/jacek99/dropwizard-spring-di-security-onejar-example
Lessons from Cassandra & Spark (Matthias Niehoff & Stephan Kepser, codecentri...DataStax
We built an application based on the principles of CQRS and Event Sourcing using Cassandra and Spark. During the project we encountered a number of challenges and problems with Cassandra and the Spark Connector.
In this talk we want to outline a few of those problems and our actions to solve them. While some problems are specific to CQRS and Event Sourcing applications most of them are use case independent.
About the Speakers
Matthias Niehoff IT-Consultant, codecentric AG
works as an IT-Consultant at codecentric AG in Germany. His focus is on big data & streaming applications with Apache Cassandra & Apache Spark. Yet he does not lose track of other tools in the area of big data. Matthias shares his experiences on conferences, meetups and usergroups.
Stephan Kepser Senior IT Consultant and Data Architect, codecentric AG
Dr. Stephan Kepser is an expert on cloud computing and big data. He wrote a couple of journal articles and blog posts on subjects of both fields. His interests reach from legal questions to questions of architecture and design of cloud computing and big data systems to technical details of NoSQL databases.
Beyond the Query – Bringing Complex Access Patterns to NoSQL with DataStax - ...StampedeCon
Learn how to model beyond traditional direct access in Apache Cassandra. Utilizing the DataStax platform to harness the power of Spark and Solr to perform search, analytics, and complex operations in place on your Cassandra data!
MongoDB Chunks - Distribution, Splitting, and MergingJason Terpko
MongoDB Chunks – Distribution, Splitting, and Merging This presentation will discuss how the distribution of chunks can create scenarios where you may need to manually move, split, or merge chunks in your sharded cluster. Scenarios requiring these actions can exist with both optimal and sub-optimal shard keys. This can be especially true when a new change has been introduced to the workload. It may cause hot-spotting, unbalanced storage, or empty chunks.
Escape from Hadoop: Ultra Fast Data Analysis with Spark & CassandraPiotr Kolaczkowski
We present the basic functionality of the official DataStax spark-cassandra connector - how to load cassandra tables as Spark RDDs and how to save Spark RDDs to Cassandra.
Massively Scalable Real-time Geospatial Data Processing with Apache Kafka and...Paul Brebner
This presentation will explore how we added location data to a scalable real-time anomaly detection application, built around Apache Kafka, and Cassandra.
Kafka and Cassandra are designed for time-series data, however, it’s not so obvious how they can process geospatial data. In order to find location-specific anomalies, we need a way to represent locations, index locations, and query locations.
We explore alternative geospatial representations including: Latitude/Longitude points, Bounding Boxes, Geohashes, and go vertical with 3D representations, including 3D Geohashes.
For each representation we also explore possible Cassandra implementations including: Clustering columns, Secondary indexes, Denormalized tables, and the Cassandra Lucene Index Plugin.
To conclude we measure and compare the query throughput of some of the solutions, and summarise the results in terms of accuracy vs. performance to answer the question “Which geospatial data representation and Cassandra implementation is best?”
Updated version of presentation for 30 April 2020 Melbourne Distributed Meetup (online)
This tutorial will guide you through the many considerations when deploying a sharded cluster. We will cover the services that make up a sharded cluster, configuration recommendations for these services, shard key selection, use cases, and how data is managed within a sharded cluster. Maintaining a sharded cluster also has its challenges. We will review these challenges and how you can prevent them with proper design or ways to resolve them if they exist today. There will be lab sessions at the end of some chapters so please have your laptops with you.
Building a Fast, Resilient Time Series Store with Cassandra (Alex Petrov, Dat...DataStax
Cassandra is awesome for many things. One of the things it's awesome for is Time Series. Combining the power of Cassandra with APIs of existing Time Series tools, such as Graphite can yield interesting results.
Cyanite is a Time Series aggregator and store built on top of Cassandra. It's fully compatible with Graphite, can serve as a plug-in replacement for Graphite and Graphite web.
Cyanite is using SASI indexes to make glob metric path queries, can query and aggregate, store, display and analyse metrics from hundreds and thousands of servers.
Which data modelling practices work best for Time Series, which new awesome Cassandra features you can use to make your Time Series analysis better.
About the Speaker
Alex Petrov Software Engineer, DataStax
Polyglot programmer. Interested in algorithms, distributed systems, algebra and high performance solutions.
A database trigger is a stored procedure that is executed when specific actions occur within a database. Triggers fit perfectly on a relational schema (foreign keys) and are implemented as a built-in functionality on popular relational database like MySQL.
MongoDB does not have any support for triggers, mainly due to the lack of support for foreign keys. Even if it usually considered an antipattern, there are use cases in MongoDB that benefit from a partially-relational schema. The lack of triggers is an obstacle for a partially-relational schema but there can be workarounds for simulating trigger behavior.
This presentation will guide you through different ways to implement triggers in MongoDB. We will cover the topics streams, tailable cursors, and hooks. We will demonstrate coding examples for each topic and we will explain pros and cons of each implementation.
Kafka is a distributed event streaming platform that has become very popular in the past couple of years.
In a MongoDB-Kafka architecture, MongoDB may be configured as both a Sink and a Source. With sink we mean to ingest events from your Kafka topics directly into MongoDB collections, exposing the data to your services for efficient querying, enrichment, and analytics. With source we mean publish data changes from MongoDB into Kafka topics for streaming to consuming apps.
We are going to cover both scenarios (using MongoDB as Sink and Source), by demonstrating ways to efficiently connect each datastore to the other. At the same time, we will cover use-cases and best practices when using Kafka and MongoDB together.
New Indexing and Aggregation Pipeline Capabilities in MongoDB 4.2Antonios Giannopoulos
MongoDB 4.2 comes GA soon delivering some amazing new features on multiple areas. In this talk, we will focus on the new capabilities of the aggregation framework. We are going to cover the new operators and expressions. At the same time, we will explore how updates commands can now use the aggregation framework operators. We are also going to present aggregation framework improvements focusing on the on-demand materialized views. Finally, we are going to explore the wildcard indexes introduced in MongoDB 4.2 and how they change the way we design documents and build queries/aggregations. We will also make a reference to the new index build system.
Massively Scalable Real-time Geospatial Data Processing with Apache Kafka and...Paul Brebner
Geospatial data makes it possible to leverage location, location, location! Geospatial data is taking off, as companies realize that just about everyone needs the benefits of geospatially aware applications. As a result there are no shortages of unique but demanding use cases of how enterprises are leveraging large-scale and fast geospatial big data processing. The data must be processed in large quantities - and quickly - to reveal hidden spatiotemporal insights vital to businesses and their end users. In the rush to tap into geospatial data, many enterprises will find that representing, indexing and querying geospatially-enriched data is more complex than they anticipated - and might bring about tradeoffs between accuracy, latency, and throughput.This presentation will explore how we added location data to a scalable real-time anomaly detection application, built around Apache Kafka, and Cassandra. Kafka and Cassandra are designed for time-series data, however, it’s not so obvious how they can process geospatial data. In order to find location-specific anomalies, we need a way to represent locations, index locations, and query locations. We explore alternative geospatial representations including: Latitude/Longitude points, Bounding Boxes, Geohashes, and go vertical with 3D representations, including 3D Geohashes. To conclude we measure and compare the query throughput of some of the solutions, and summarise the results in terms of accuracy vs. performance to answer the question “Which geospatial data representation and Cassandra implementation is best?”
Spark and Cassandra with the Datastax Spark Cassandra Connector
How it works and how to use it!
Missed Spark Summit but Still want to see some slides?
This slide deck is for you!
The overall evolution towards microservices has caused a lot of IT leaders to radically rethink architectures and platforms. One can hardly keep up with the rapid onslaught on new distributed technologies. The same people who just asked yesterday "how can we deploy Docker containers?", are now asking "how can we operate Kubernetes-as-a-Service on-premise?", and are about to start asking "how can we operate the open source frameworks of our choice, such as Spark, TensorFlow, HDFS, and more, as a service across hybrid clouds?”. This session will discuss: Challenges of orchestrating and operating.
While we move towards microservices architecture, and are managing too many services, the issue of service discovery arises. This becomes a bigger issue when we are autoscaling. So we need a system to be able to resolve a service to a specific machine in our deployment.
An introduction to Dropwizard, as well as examples of how to integrate it with Spring DI and Spring Security. Prepared for a presentation at the Houston Java User Group. Code samples at : https://github.com/jacek99/dropwizard-spring-di-security-onejar-example
WildFly Swarm: Criando Microservices com Java EE 7George Gastaldi
Apresentado no TDC 2016 - Florianópolis
Microservices é a arquitetura do momento. Todos estão falando sobre Spring Boot, NodeJS, DropWizard e outros frameworks, mas nenhum deles é tão completa quanto a oferecida pela especificação Java EE. Nesta sessão live-coding você vai aprender sobre o WildFly Swarm, e como transformar um projeto Java EE 7 qualquer em um microservice.
Simple REST-APIs with Dropwizard and SwaggerLeanIX GmbH
During the VOXXED Days in Berlin on 29 January 2016 Bernd Schönbach from LeanIX demonstrated an easy way to create well documented and implemented REST-APIs using the Dropwizard Library for the implementation and Swagger for easy Documentation.
===
LeanIX offers an innovative software-as-a-service solution for Enterprise Architecture Management (EAM), based either in a public cloud or the client’s data center.
Companies like Adidas, Axel Springer, Helvetia, RWE, Trusted Shops and Zalando use LeanIX Enterprise Architecture Management tool.
Free Trial: http://bit.ly/LeanIXFreeTrial
Slides accompanying a presentation on Dropwizard I gave at the DevIgnition conference ( www.devignition.com ) on April 29, 2016. The sample code is on GitHub at https://github.com/sleberknight/dropwizard-devignition-2016
You have heard how containers are great for running microservices, but what is needed to get microservices to run in production at scale? In this session, we explore the reasoning and concepts behind microservices and how containers simplify building microservices based applications. We will show how you can easily launch microservices on Amazon EC2 Container Service and how you can use ELB and Route 53 to easily do service discovery between microservices.
Presented by: Danny Fezer, Solutions Architect, Amazon Web Services
Customer Guest: Liz Duke, Technical Delivery Manager, Irdeto
J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"Daniel Bryant
Microservice platforms are finally becoming a reality: Mesos, Kubernetes, and a whole bunch of PaaS-style offerings are available, but the reality is that these platforms still don’t provide everything you need in order to build a fully functional microservice ecosystem. Come to this session to learn about the essential deployment, orchestration, and glue components that often have to be self-assembled. The presentation begins by looking at deployment techniques and tools and examines where to test (QA, staging, or production), how to test (integration and contracts), and how to separate deployment and release. It then discusses orchestration, configuration, and service discovery. Finally it looks at essential glue such as logging, monitoring, and alerting.
Microservices is a software architectural method where you decompose complex applications into smaller, independent services. Containers are great for running small decoupled services, but how do you coordinate running microservices in production at scale and what AWS services do you use?
In this session, we will explore the reasoning and concepts behind microservices and how containers simplify building microservices based applications. We will also demonstrate how you can easily launch microservices on Amazon EC2 Container Service and how you can use ELB and Route 53 to easily do service discovery between microservices.
Cloud Native Microservices with Spring CloudConor Svensson
In this talk we are going to discuss some of the key components of Spring Cloud. This includes the Netflix OSS integrations for Spring Boot apps which include Service Discovery (Eureka), Circuit Breaker (Hystrix), Intelligent Routing (Zuul) and Client Side Load Balancing (Ribbon). We will also touch on the Spring Cloud centralised configuration server and deploy these apps to Cloud Foundry.
Spring IO 2016 - Spring Cloud Microservices, a journey inside a financial entityToni Jara
The presentation explains the journey from a monolithic architecture to Spring Cloud Microservices for application development inside a financial entity, along with the transition to DevOps strategies… a journey that has just begun…
Building REST APIs with Spring Boot and Spring CloudKenny Bastani
In this talk I will introduce you to Spring Cloud, a set of tools for building cloud-native JVM applications. We will take a look at some of the common patterns for microservice architectures and how to use Cloud Foundry to deploy multiple microservices to the cloud.
We will also dive into a microservices example project of a cloud-native application built using Spring Boot and Spring Cloud. Using this example project, I'll show you how to use Lattice to spin up a microservice cluster on AWS. We will then explore what a cloud-native application looks like when using self-describing REST APIs that link multiple microservices together.
Microservices - java ee vs spring boot and spring cloudBen Wilcock
A short presentation on the clear differences in approach between the top two Java frameworks for microservices, service orientation and web-service development - Java EE (JEE) and Spring (Spring Boot and Spring Cloud).
Service discovery in a microservice architecture using consulJos Dirksen
Presentation I gave at Nextbuild 2016. Gives an overview of how Consul can be used in microservice architecture. Accompanying examples and demo can be found here: https://github.com/josdirksen/next-build-consul
Through this presentation you will gain a good understanding of how the clean architecture pattern is implemented at Taxibeat. What issues the Android Taxibeat team has faced so far and what solutions we came up with. Of course, the benefits of clean architecture will also be discussed along with the way we managed to build two fast paced iterative apps that share functionality.
The mainstreaming of containerization and microservices is raising a critical question by both developers and operators: how do we debug all this?
Debugging microservices applications is a difficult task. The state of the application is spread across multiple microservices, and it is hard to get a holistic view of the state of the application. Currently debugging of microservices is assisted by openTracing, which helps in tracing of a transaction or workflow for post-mortem analysis, and linkerd and itsio which monitor the network to identify latency problems. These tools however, do not allow to monitor and interfere with the application during run time.
In this talk, we will describe and demonstrate common debugging techniques and we will introduce Squash, a new tool and methodology.
This talk (delivered at QConLondon 2016) covers the evolution of Coursera's nearline architecture, delves into our latest generation system, and then covers the flagship application of the architecture (evaluating programming assignments).
8 - OpenShift - A look at a container platform: what's in the boxKangaroot
Many already have some familiarity with containers, and maybe even with Kubernetes. But what's the difference between those and a container platform? In this session the goal is to look at OpenShift, Red Hat's container platform based on Kubernetes. We see what it's made out of, what makes it tick, and what the future of OpenShift & Kubernetes holds.
Bringing order to the chaos! - Paulo Lopes - Codemotion Amsterdam 2018Codemotion
Chaos Engineering is an emerging discipline, but even before the first computer was built Failure was already there! Fast forward to today, how do you handle Failure? Do you deny it or accept it? Is your network really reliable? Is "the cloud", "the solution"? In this hands-on talk, I will explore the basic concepts of Chaos Engineering and demonstrate as a microservice application can be prepared for the chaos. I'll illustrate how to prepare a plan, break things so you will be ready when failure comes by! This talk is about going big or go home! Rolling back is not a plan!
GeeCON 2017 - TestContainers. Integration testing without the hassleAnton Arhipov
TestContainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
Easing offline web application development with GWTArnaud Tournier
At this current time, HTML5 APIs are mature enough so that the web browser can now be a very good platform for applications that were before only implemented as native applications : offline applications with locally stored data, embedded SQL engines, etc. Although there are many good Javascript frameworks out there, the Java language allows to build, maintain, debug and work with ease on really big applications (> 100,000 LOC).
You'll discover in this presentation all the tools we assembled to make an application available with its data 100% of the time, even without internet!
"JavaME + Android in action" CCT-CEJUG Dezembro 2008Vando Batista
Mini-cursos de JavaME e Android no evento do CEJUG Café com Tapioca, em Dezembro de 2008.
1. Introdução: overview do desenvolvimento em Java para dispositivos portáteis/móveis
2. Java ME in action: tutorial hands-on de desenvolvimento (mini-curso)
3. Android in action: tutorial hands-on de desenvolvimento (mini-curso)
Autor: Vando Batista
One aspect is crucial, when preserving IT value through cloud-friendly migration of legacy applications: Transparency. You need full visibility on the source application landscape, on the migration process, and on the target cloud environment. Without transparency, your effort, time targets and also quality go overboard –- you’ll experience an agonizing blind flight with a high risk of accidents. In the keynote, I will show the most important measures and tools with which you can create the visibility that has proven decisive in our many hundreds of migration projects.
There will be a future where container workloads and serverless platforms are BFF. An essential building block on this way is Source2Image. It provides the magic of source code being transformed automatically into an executable container image containing all required runtime components. Think of it as a black box continuous integration server for containerized applications. The talk will introduce, showcase, and compare leading Source2Image open source projects.
The need for speed – transforming insurance into a cloud-native industryJosef Adersberger
Die Versicherungsindustrie steht vor großen Herausforderungen: Neben etablierten Prinzipien der Skalenökonomie müssen mehr und mehr Prinzipien der Geschwindigkeits- und Plattformökonomie berücksichtigt werden. Ein wesentlicher Schritt auf dem Weg in eine Geschwindigkeitsökonomie ist die agile Transformation des Unternehmens, um schnelle Reaktionen auf ver ändertes Kundenverhalten ermöglichen. Der Vortrag zeigt, welche Schlüsselrolle Cloud- und Plattformtechnologien in diesem Transformationsprozess einnehmen und wie sich schnell sichtbare Erfolge erzielen lassen.
The good, the bad, and the ugly of migrating hundreds of legacy applications ...Josef Adersberger
Wir haben bei der Allianz innerhalb von 17 Monaten eine Container Plattform in der Public Cloud aufgebaut und in einem ersten Schritt 144 Java Legacy Anwendungen cloud-ready gemacht und dorthin migriert. Im Vortrag zeigen wir, was dabei unsere Erfolgsrezepte und größten Hindernisse waren. Es geht dabei unter anderen darum, wie man eine große Anwendungslandschaft auf ihre Cloud-Readiness hin analysiert und wie man eine industrialisierte Migration von Anwendungen auf eine Cloud Plattform etabliert.
Patterns and Pains of Migrating Legacy Applications to KubernetesJosef Adersberger
Running applications on Kubernetes can provide a lot of benefits: more dev speed, lower ops costs, and a higher elasticity & resiliency in production. Kubernetes is the place to be for cloud native apps. But what to do if you’ve no shiny new cloud native apps but a whole bunch of JEE legacy systems? No chance to leverage the advantages of Kubernetes? Yes you can!
We’re facing the challenge of migrating hundreds of JEE legacy applications of a German blue chip company onto a Kubernetes cluster within one year.
The talk will be about the lessons we've learned - the best practices and pitfalls we've discovered along our way.
Istio is the cool new kid on the service mesh block: it can be deployed without the need for any change on the microservice-side and enhances their communication paths with encryption, resiliency, identity and access management, observability with metrics and traces and policy enforcement. In this lightning talk I'll talk you through an Istio sample application incorporating all major features. The sample will be released on github and will also run on minikube.
Kubernetes und Docker sind trotz des hohen Verbreitungsgrads noch relativ junge Technologien. Viele Menschen machen gerade gute und teilweise auch schmerzliche Erfahrungen mit beiden. Der Vortrag bietet einen Katalog an Patterns und Antipatterns bei der Entwicklung von Anwendungen auf Basis Kubernetes und Docker. Der Katalog repräsentiert dabei die Erfahrung aus mehreren Industrieprojekten, die es bis in Produktion geschafft haben.
Es geht darum, was man bei Docker-Files und Kubernetes-Deskriptoren richtig und falsch machen kann; welche Architekturbausteine man einsetzen sollte; wie die Continuous Delivery Pipeline gestaltet werden sollte und wie Anwendungen auf Cloud-Native-Plattformen gut betreibbar und diagnostizierbar gemacht werden können.
The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications ...Josef Adersberger
Running applications on Kubernetes can provide a lot of benefits: more dev speed, lower ops costs, and a higher elasticity & resiliency in production. Kubernetes is the place to be for cloud native apps. But what to do if you’ve no shiny new cloud native apps but a whole bunch of JEE legacy systems? No chance to leverage the advantages of Kubernetes? Yes you can!
We’re facing the challenge of migrating hundreds of JEE legacy applications of a major German insurance company onto a Kubernetes cluster within one year. We're now close to the finish line and it worked pretty well so far.
The talk will be about the lessons we've learned - the best practices and pitfalls we've discovered along our way. We'll provide our answers to life, the universe and a cloud native journey like:
- What technical constraints of Kubernetes can be obstacles for applications and how to tackle these?
- How to architect a landscape of hundreds of containerized applications with their surrounding infrastructure like DBs MQs and IAM and heavy requirements on security?
- How to industrialize and govern the migration process?
- How to leverage the possibilities of a cloud native platform like Kubernetes without challenging the tight timeline?
Dataservices - Processing Big Data The Microservice WayJosef Adersberger
We see a big data processing pattern emerging using the Microservice approach to build an integrated, flexible, and distributed system of data processing tasks. We call this the Dataservice pattern. In this presentation we'll introduce into Dataservices: their basic concepts, the technology typically in use (like Kubernetes, Kafka, Cassandra and Spring) and some architectures from real-life.
Anwendungen nativ für den Betrieb in der Cloud auszulegen, ist der Architekturstil der Stunde: Microservices, 12-Factor Apps und Serverless-Architecturen sind en vogue. Daneben gibt es Java EE, das sich über Jahre bewährt hat beim Bau von Java-Anwendungen fürs Unternehmen. Java-EE-Anwendungen im modernen Cloud-Native-Stil zu entwickeln- ist kein Widerspruch, sondern ein Zugewinn: Man kann damit Enterprise-Anwendungen bauen, die reif für die Cloud-Ära sind.
Der Vortrag zeigt am laufenden Beispiel, wie man eine Cloud-Native-Java-EE-Anwendung entwickelt und wie sich Java-EE-APIs wie JAX-RS, CDI und JPA integrieren mit Cloud-Native-Infrastruktur wie DC/OS, Kubernetes, Hystrix, Traefik, Consul und Docker. Dabei wird nicht nur blanke Technologie gezeigt, sondern auch das Thema Cloud Native Java EE auf Architekturebene betrachtet.
Users leave thousands of traces per second on a successful ecommerce site. It’s very pragmatic to analyse and react on this trace event stream in realtime. This is called clickstream analysis. In the talk I’ll present a software architecture based on Apache Spark which is able to process thousands of clickstream events per second. A product based on this architecture is in production since mid 2015 and is still performing well. The building blocks of the architecture beside Spark are Kafka to handle the inbound event stream, Spark Streaming for initial stream processing and Parquet as serialization format. I argue why we’ve chosen these technologies and what experiences we had in developing, launching and operating the product.
Software-Sanierung: Wie man kranke Systeme wieder gesund macht.Josef Adersberger
Manche Softwaresysteme sind Intensivpatienten: Sie können nur mit hohem Aufwand weiter am Leben gehalten werden. Sämtliches Budget wird verbraucht, um Qualitätsschulden zurück zu zahlen: Bugs fixen; Performance tunen; Sicherheitslöcher stopfen; Architekturbrüche auflösen.
Wir stellen im Vortrag die 4 Schritte der Software-Sanierung vor:
Erstens, die Probleme erfassen. Hierbei hilft uns ein Diagnoseverfahren, das der Medizin entlehnt ist. Zweitens, die Ursachen identifizieren. Hier zählt es, die richtigen (Open-Source-) Werkzeuge einzusetzen zur statischen Analyse von Code- und Architekturanomalien und zur dynamischen Analyse von Laufzeitanomalien. Drittens, Maßnahmen ableiten und umsetzen sowie einen wirksamen und vernünftig priorisierten Maßnahmenkatalog erstellen und daraus Refactoringpläne ableiten und ausführen. Viertens, die Erfolge messen also bestimmen, wie wirksam die Maßnahmen wirklich waren, ob das Ziel schon erreicht ist, oder ob weitere Maßnahmen angegangen werden müssen.
Neben dem eigentlichen Ablauf beschreiben wir eine Reihe an Methoden, Best Practices und Werkzeuge, die dabei erfolgskritisch sind.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
10. JEE IS NEITHER UGLY NOR FOCUSED ON MONOLITHIC APPS
▸ Java EE by itself is modular und lightweight in the recent versions.
(to be honest: Spring feels more heavyweight than JEE to me from
time to time)
▸ Java EE micro containers allow to modularize applications into
self-contained runnable units.
▸ How to develop enterprise applications based on Java EE is well
understood and knowledge is widespread.
▸ API is mature and standardized. Implementations are battle
proven.
14. JEE IS NOT AN OVERALL MONOLITH. BUT PEOPLE TEND TO RUN
JEE APPS AS MONOLITHS WITHIN HEAVY WEIGHT APP SERVERS.
DESIGN CODE RUN
startup-cluster.sh
JEE Impl.
JEE App Server
15. JEE MICRO CONTAINER TO THE RESCUE: ENABLING MICROSERVICES ON JEE
DESIGN CODE RUN
main()
JEE impl. parts
16. A CLOUD OF JEE MICRO CONTAINERS. MISSION ACCOMPLISHED?
image: https://www.dreamstime.com
17. a cloud is not nothing!
what’s inside our cloud?
19. CLUSTER VIRTUALIZATION (computing, network, storage, memory)
CLUSTER RESOURCE MANAGER
CLUSTER ORCHESTRATOR
APPLICATIONS
CONTAINER
CLUSTER RESOURCES
MICROSERVICE PLATFORM
API Gateway
Micro Container
Configuration & Coordination
Diagnosability &
Monitoring
Infrastructure-as-a-Service Bare Metal Local Host
Service
Client
Service Discovery
DevOps Interface
‣ deploy
‣ rollback
‣ scale
‣ configure
Diagnose Interface
‣ analyze logs
‣ analyze metrics
‣ analyze traces
Edge Interface
‣ request service
‣ authenticate
20. CLUSTER VIRTUALIZATION (computing, network, storage, memory)
CLUSTER RESOURCE MANAGER
CLUSTER ORCHESTRATOR
APPLICATIONS
CONTAINER
CLUSTER RESOURCES
MICROSERVICE PLATFORM
API Gateway
Micro Container
Configuration & Coordination
Diagnosability &
Monitoring
Infrastructure-as-a-Service Bare Metal Local Host
Service
Client
Service Discovery
DevOps Interface
‣ deploy
‣ rollback
‣ scale
‣ configure
Diagnose Interface
‣ analyze logs
‣ analyze metrics
‣ analyze traces
Edge Interface
‣ request service
‣ authenticate
how to provide the right
resources for container
execution?
how to decouple from
physical hardware?
how to run (containerized)
applications on a cluster?
how to detect and solve
operational anomalies?
how to call other microservices
resilient and responsive?
how to execute a microservice
and embed it within the platform? how to provide cluster-wide
consensus on config values etc.?
how to expose microservice
endpoints to the internet?
how to manage
microservice endpoints
within the whole
platform?
22. CLUSTER VIRTUALIZATION (computing, network, storage, memory)
CLUSTER RESOURCE MANAGER
CLUSTER ORCHESTRATOR
APPLICATIONS
CONTAINER
CLUSTER RESOURCES
MICROSERVICE PLATFORM
API Gateway
Micro Container
Configuration & Coordination
Diagnosability &
Monitoring
Infrastructure-as-a-Service Bare Metal Local Host
Service
Client
Service Discovery
DevOps Interface
Diagnose InterfaceEdge Interface
23. THE ZWITSCHER JEE SHOWCASE
ZWITSCHER-APP-HISTORY ZWITSCHER-APP-WIKIPEDIAZWITSCHER-APP-CHUCK
ZWITSCHER-APP-BOARD
keyword results + keyword history
random joke
(because every chuck
norris joke matches on
every keyword)
JDBC Open API
ZWITSCHER-APP-CHUCK
fallback
https://github.com/adersberger/cloud-native-zwitscher-jee
27. JEE MICRO CONTAINER COMPARISON CHART
Payara Micro Wildfly Swarm TomEE+ kumuluzEE
Servlets et al. x x x x
WebSockets x x x
JSF x x x
JAX-RS x x x x
JAX-WS x x
EJB *2 x x x
CDI x x x x
JTA x x x
JCA x x
JMS x x
JPA x x x x
Bean Validation x x x x
JBatch x x
Concurrency x x
JCache x x
JEE version JEE 7 JEE 7 JEE 6, JEE 7 part.*1 JEE 7 part.
Packaging WAR + JAR JAR JAR classes + JARs
Startup time 4s 4s 2s 1s
Size 57MB 83 MB 44 MB 15 MB
*1) http://tomee.apache.org/javaee7-status.html
*2) http://stackoverflow.com/questions/13487987/where-to-use-ejb-3-1-and-cdi
28. @Path(“/joke") @RequestScoped
public class ChuckJokeResource {
@Inject
private IcndbIntegration icndb;
@GET
@Produces("application/json")
public Response getJoke() {
Map<String, String> json = new HashMap<>();
json.put("message", icndb.getRandomJoke());
return Response.ok(json).build();
}
}
@ApplicationPath("/chuck")
public class ChuckJokeApplication extends ResourceConfig {
public ChuckJokeApplication() {
super();
register(ChuckJokeResource.class);
}
}
JAX-RS WITH CDI
bean-discovery-mode="all"
29. public class Main {
/**
* Starts the microservice container.
*
* Requires environment variable "PORT" according
* on what port to listen.
*
* @param args no arguments evaluated
*/
public static void main(String[] args) {
com.kumuluz.ee.EeApplication.main(args);
}
}
JEE MICRO CONTAINER STARTUP
30. TESTING
@RunWith(CdiTestRunner.class)
public class TestChuckJokeResource {
@Inject
private ChuckJokeResource chuckJokeResource;
@Test
public void testChuckJokeResource() {
Response response = chuckJokeResource.getJoke();
assertThat(
response.getStatusInfo().getStatusCode(),
equalTo(200));
}
}
41. SERVICE DISCOVERYSERVICE REGISTRATION
SERVICE LOOKUP
SERVICE HEALTH CHECKING
& API GATEWAYAPI EXPOSITION & REQUEST ROUTING
AUTHENTICATION & AUTORISATION
LOAD BALANCING & SHEDDING
RATE LIMITING
REQUEST MONITORING & AUDITING
42. THE BIG PICTURE
MICRO
SERVICE
SERVICE DISCOVERY
register
web-facing
services
lookup
internal
endpoints
web requests
lookup routes
API GATEWAY
health
checks (metrics servlet)
java.security.Security.setProperty("networkaddress.cache.ttl", "0" );
43. SERVICE REGISTRATION WITHIN A JEE MICROSERVICE
@ApplicationPath("/chuck")
public class ChuckJokeApplication extends ResourceConfig {
@Inject
public ChuckJokeApplication(
@ConsulFabio IServiceDiscovery serviceDiscovery) {
super();
//Register service
serviceDiscovery.registerService(
"zwitscher-chuck", "/chuck/joke");
}
} Service Name URL Path to Service
44. SERVICE REGISTRATION: THE HEAVY LIFTING
/**
* Registeres a service
*
* see https://github.com/eBay/fabio/wiki/Service-Configuration
*/
public synchronized void registerService(String serviceName, String servicePath) {
String applicationHost = getOutboundHost();
int applicationPort = getOutboundPort();
HostAndPort consulEndpoint = getConsulHostAndPort();
logger.info("Will register service on host {} and port {} at consul endpoint {}",
applicationHost, applicationPort, consulEndpoint.toString());
//generate unique serviceId
String serviceId = serviceName + "-" + applicationHost + ":" + applicationPort;
String fabioServiceTag = "urlprefix-" + servicePath;
//point healthcheck URL to dropwizard metrics healthcheck servlet
URL serviceUrl = UrlBuilder.empty()
.withScheme("http")
.withHost(applicationHost)
.withPort(applicationPort)
.withPath("/metrics/ping").toUrl();
// Service bei Consul registrieren inklusive einem Health-Check auf die URL des REST-Endpunkts.
logger.info("Registering service with ID {} and NAME {} with healthcheck URL {} and inbound ROUTE {}",
serviceId, serviceName, serviceUrl, fabioServiceTag);
//use consul API to register service
ConsulClient client = new ConsulClient(consulEndpoint.toString());
NewService service = new NewService();
service.setId(serviceId);
service.setName(serviceName);
service.setPort(applicationPort);
service.setAddress(applicationHost);
List<String> tags = new ArrayList<>();
tags.add(fabioServiceTag);
service.setTags(tags);
//register health check
NewService.Check check = new NewService.Check();
check.setHttp(serviceUrl.toString());
check.setInterval(ConsulFabioServiceDiscovery.HEALTHCHECK_INTERVAL + "s");
service.setCheck(check);
client.agentServiceRegister(service);
}
public static String getOutboundHost() {
String hostName = System.getenv(HOSTNAME_ENVVAR);
String host = System.getenv(HOST_ENVVAR);
if (hostName == null && host == null) return DEFAULT_HOST;
else if (host != null) return host;
else {
File etcHosts = new File("/etc/hosts");
List<String> lines;
try {
lines = Files.readLines(etcHosts, Charset.defaultCharset());
} catch (IOException e) {
return DEFAULT_HOST;
}
for (String line: lines){
if (!line.trim().startsWith("#") && !line.trim().isEmpty()) {
String[] etcEntry = line.split("s+");
if (etcEntry[1].equals(hostName)) return etcEntry[0];
}
}
return DEFAULT_HOST;
}
}
public static int getOutboundPort() {
String portEnv = System.getenv(PORT_ENVVAR);
if (portEnv == null) return DEFAULT_PORT;
return Integer.valueOf(portEnv);
}
public static HostAndPort getConsulHostAndPort() {
String consulEnv = System.getenv(CONSUL_ENVVAR);
if (consulEnv == null) return HostAndPort.fromString(CONSUL_DEFAULT_HOSTANDPORT);
else return HostAndPort.fromString(consulEnv);
}
@ConsulFabio
@ApplicationScoped
public class ConsulFabioServiceDiscovery implements IServiceDiscovery {
figure out Consul-visible host name and port
compose health check
add meta data for Fabio
register service at Consul
(as well as API doc and diagnosability endpoints)
49. FOCUS ON SHORT ROUND TRIPS WITH MULTIPLE RUN MODES
In-Process
Local Cluster
Remote Cluster
‣ longer round trip time
‣ but: more realistic
Group with dependencies:
Single applications:
53. CLOUD NATIVE JEE 101
SUMMARY
▸ Implementing JEE microservices on DC/OS is simple & fun.
▸ The JEE APIs are out of the box not cloud native but can easily be
enhanced with the required functionality. You can use our
JCloudEE util classes to glue JEE together with cloud native tech.
▸ Short round trip times are essential to be productive. You need
full portability and automation from local host up to the cloud.
55. https://github.com/qaware/kubepad
‣ First things first: Will be renamed to
CloudPad soon!
‣ Controlling a DC/OS cluster
with a DJ pad.
‣ Written in fancy Kotlin.
‣ Turned out to be really helpful for cloud
native newbies to better grasp cluster
orchestration.
‣ Kicky colored lights and well-hidden
snake game easter egg. Set colors with
app label.
61. READING CONFIGURATION VALUES
@Inject
private ConfigurationProvider config;
//…
String messageTag = config.getProperty("messageTag", String.class);
try to read config value in
Consul
fallback to file if Consul is
not available or no config
value is set in Consul
62. THE CONSUL AGENT AND CONFIG ENVIRONMENT CAN BE SET
WITH A ENV VAR.
"env": {
"CONFIG_ENV" : "zwitscher"
}
The config environment (e.g. “zwitscher”, “zwitscher-test”, “zwitscher-prod”). Refers to
a path in Consul or to a local path where the config file is.