Zero-downtime deployment on Kubernetes with Hazelcast

COOL WAYS TO GET STARTED Join us for a live InfluxDB training to learn how to easily ingest at scale in a matter of seconds to help you build powerful time series based applications. Join our 45-minute demos with experts who will showcase key InfluxDB features and answer questions live from the audience. After attending this training, attendees will be able to: Use sample data sets to try out various visualization options Utilize the available data ingestion methods to construct a data pipeline to InfluxDB Leverage Notebooks to collaborate with team members Gain best practices for InfluxDB, Telegraf and Flux

How EnerKey Using InfluxDB Saves Customers Millions by Detecting Energy Usage...

INTRODUCING: CREATE PIPELINE

This document introduces MemSQL Pipelines, which allow for exactly-once data ingestion semantics when streaming data from Kafka into MemSQL. MemSQL Pipelines provide a native way to extract, transform, and load external data into MemSQL tables. They offer a scalable and highly performant ETL process across a distributed cluster. The document explains streaming semantics like at least once and exactly once delivery, and how MemSQL Pipelines coordinate with Kafka to enable exactly-once ingestion through offset tracking. It presents the architecture of MemSQL Pipelines and demonstrates their use through a live demo.

The Future of ETL Isn't What It Used to Be

Speaker: Gwen Shapira, Principal Data Architect, Confluent Join Gwen Shapira, Apache Kafka® committer and co-author of ""Kafka: The Definitive Guide,"" as she presents core patterns of modern data engineering and explains how you can use microservices, event streams and a streaming platform like Apache Kafka to build scalable and reliable data pipelines designed to evolve over time. This is part 1 of 3 in Streaming ETL - The New Data Integration series. Watch the recording: https://videos.confluent.io/watch/q7roRtNZBnjiT9C3ii88fo?.

Data & Analytics Forum: Moving Telcos to Real Time

MemSQL is a real-time database that allows users to simultaneously ingest, serve, and analyze streaming data and transactions. It is an in-memory distributed relational database that supports SQL, key-value, documents, and geospatial queries. MemSQL provides real-time analytics capabilities through Streamliner, which allows one-click deployment of Apache Spark for real-time data pipelines and analytics without batch processing. It is available in free community and paid enterprise editions with support and additional features.

O'Reilly Media Webcast: Building Real-Time Data Pipelines

As our customers tap into new sources of data or modify to existing data pipelines, we are often asked questions like: What technologies should we consider? Where can we reduce data latency? How can we simplify our data architecture? To eliminate the guesswork, we teamed up with Ben Lorica, Chief Data Scientist at O’Reilly Media to host a webcast centered around building real-time data pipelines.

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

InfluxDB Live Product Training

How EnerKey Using InfluxDB Saves Customers Millions by Detecting Energy Usage...

INTRODUCING: CREATE PIPELINE

The Future of ETL Isn't What It Used to Be

Data & Analytics Forum: Moving Telcos to Real Time

O'Reilly Media Webcast: Building Real-Time Data Pipelines

How to Enable Industrial Decarbonization with Node-RED and InfluxDB

Graphite Energy’s thermal energy storage (TES) platform encourages clients to offset their traditional energy consumption with low-cost renewable energy sources. Their customers include manufacturers, mines, steelmakers and aluminum plants. IIoT data is collected about energy usage, fuel consumption, temperatures, solar panels, wind farms, process steam and air dryers. Discover how Graphite Energy uses InfluxDB to monitor their zero-emission energy solution. In this webinar, Byron Ross will dive into: Graphite Energy’s approach to reducing their clients’ carbon footprint Their methodology to collecting sensor data used to make their operations more green Why they chose a time series database over a data historian

Live Coding a KSQL Application

Internet of Things and Multi-model Data Infrastructure

Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience Lond...

Achieving scale and performance using cloud native environment

Rakuten Group, Inc.

ID Platform Product can be used by every Rakuten Group Companies and can easily serve millions of users. Multi-Region product challenges are many, example: - Ensure 4 9’s availability - Management across each region - Alerting and Monitoring across each region - Auto scaling (Scale up and Scale down) across each region - Performance (vertical scale up) - Cost - DB Consistency Across Multiple Regions - Resiliency At Ecosystem Platform Layer for Rakuten, we handle each of these and this presentation is about how we handle these challenging scenarios.

Modeling the Smart and Connected City of the Future with Kafka and Spark

- Modeling the Smart and Connected City of the Future with Kafka and Spark discusses using Kafka, Spark, and MemSQL to build a real-time data pipeline for a hypothetical "MemCity" that captures data from 1.4 million households. - The document outlines the components of the "Real-Time Trinity" - Kafka for a high-throughput message queue, Spark for data transformation, and MemSQL for real-time data serving and analytics. - It also introduces MemSQL Streamliner, which is designed to simplify the creation of real-time data pipelines through a graphical interface and one-click deployment of integrated Apache Spark clusters.

How Microsoft Built and Scaled Cosmos

Cosmos is a large-scale data processing system used by thousands at Microsoft to process exabytes of data across clusters of over 50,000 servers. It provides a SQL-like language and allows teams to easily share and join data. This drives huge scalability requirements. The Apollo scheduler was developed to maximize cluster utilization while minimizing latency for heterogeneous workloads at cloud scale. Later, JetScope was created to support lower latency interactive queries through intermediate result streaming and gang scheduling while maintaining fault tolerance.

From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...

This document discusses SunPower's transition from a legacy SQL server to a Confluent & Kafka monitoring system. Some key points: - SunPower has over 6 million devices in the field sending data every 5 minutes that they need to monitor to ensure 20-year performance guarantees. - They want to understand performance challenges like weather or faults using data to deliver customer value. - They implemented Kafka as a central system to blend sensor and other data sources to generate value and allow engineering teams access to enriched data. - Future plans include panel-level monitoring in an app to quickly identify issues and determine resolutions.

Stateful, Stateless and Serverless - Running Apache Kafka® on Kubernetes

This document provides an overview of a webinar on Kubernetes operators presented by Joe Beda. It discusses: - Kubernetes allows for developer and operator productivity through improved workflows and better resource efficiency. - Operators are domain-specific controllers that manage both a software application and related Kubernetes objects, and help automate common operational tasks through code. - The Kafka operator example shows how an operator can handle tasks like identity, discovery, storage and logging/metrics for Kafka deployments on Kubernetes through a custom resource definition.

PowerStream Demo

Live Coding a KSQL Application

Join us as we build a complete streaming application with KSQL. There will be plenty of hands-on action, plus a description of our thought process and design choices along the way. Look out for advice on best practices and handy tips and tricks as we go. This is part 2 out of 3 in the Empowering Streams through KSQL series. The code from the talk is available here: https://gist.github.com/rmoff/7efa882dfd808dbab4eb7b8e6f9eda16.

Aengus Rooney [Grafana] | What's New with Grafana and InfluxDB | InfluxDays E...

The document summarizes new features in Grafana and InfluxDB, including: 1. A unified data model provides a consistent way to define data sources, conventions, user defaults, and override rules across visualizations. 2. Transformations allow users to manipulate and combine time series data from different sources in sensible ways, such as applying calculations or defining labels in a database to appear as fields in panels. 3. The document proposes a demonstration of building a coin exchange trading terminal with Grafana and Flux in 15 minutes by recreating the Kraken UI as a Grafana terminal using calculations from Flux.

InfluxData Architecture for IoT | Noah Crowley | InfluxData

InfluxEnterprise Architectural Patterns by Dean Sheehan, Senior Director, Pre...

Container Monitoring Best Practices Using AWS and InfluxData by Gunnar Aasen

Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber

Speaker: Yupeng Fu, Staff Engineer, Uber High availability and reliability are important requirements to Uber services, and the services shall tolerate datacenter failures in a region and fail over to another region. In this talk, we will present the active-active Apache Kafka® at Uber and how it facilitates disaster discovery across regions for Uber services. In particular, we will highlight the key components including topic replication, topic aggregation, offsets sync and then walk through several use cases of their disaster recovery strategy using active-active Kafka. Lastly, we will present several interesting challenges and the future work planned. Yupeng Fu is a staff engineer in Uber Data Org leading the streaming data platform. Previously, he worked at Alluxio and Palantir, building distributed data analysis and storage platforms. Yupeng holds a B.S. and an M.S. from Tsinghua University and did his Ph.D. research on databases at UCSD.

Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal

ASRC Federal created the Mission Operator Assist (MOA) tool to extend human capabilities through AI/ML for NOAA. MOA ingests system log data from on-orbit satellite constellations and applies machine learning to greatly improve real-time situational awareness. MOA uses a collection of tools, including Kafka for multi-subscriber communications, all hosted through AWS Cloud Services and Kubernetes Containers for microservices. Like many traditional on-premises systems, satellite ground station operations are undergoing a renaissance as they increasingly become enabled by cloud. During this session, the audience will learn about the satellite communications chain, and best practices and lessons learned in creating a data pipeline with Kafka for high throughput and scalability while displaying high quality situational awareness to mission operators. We will discuss our goals centered around establishing event-driven streaming for satellite logs so our machine learning becomes real-time and supporting a multi-subscriber approach for various Kafka topics. Listeners will also learn how a multi-subscriber approach using Kafka, helped us auto scale logstash based on how many messages are in the queue and other microservices.

Using InfluxDB for Full Observability of a SaaS Platform by Aleksandr Tavgen,...

Aleksandr Tavgen from Playtech, the world’s largest online gambling software supplier, will share how they are using InfluxDB 2.0, Flux, and the OpenTracingAPI to gain full observability of their platform. In addition, he will share how InfluxDB has served as the glue to cope with multiple sets of time series data, especially in the case of understanding online user activity — a use case that is normally difficult without the math functions now available with Flux.

Schemas, streams, and grocery stores

Jay Kreps | Kafka Summit NYC 2019 Keynote (Events Everywhere) | CEO, Confluent

Jay Kreps is the CEO of Confluent, Inc., a company backing the popular Apache Kafka® messaging system. Prior to founding Confluent, he was formerly the lead architect for data infrastructure at LinkedIn. He is among the original authors of several open source projects including Project Voldemort (a key-value store). Apache Kafka (a distributed messaging system) and Apache Samza (a stream processing system).

Zero-downtime Deployment on Kubernetes

All Things Open

Presented by: Nicolas Fränkel Presented at the All Things Open 2021 Raleigh, NC, USA Raleigh Convention Center Abstract: Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day because we need to handle state. Though it would be much easier to have stateless apps, and despite our best efforts in this direction, state is found in (at least) two places: sessions and databases. You need to think keeping the state while stopping and starting application nodes. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema change with the help of Hazelcast, while keeping the service up during the entire update process.

JCon - Zero-Downtime-Deployment with Kubernetes, Spring Boot and Flyway

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

What's hot

How to Enable Industrial Decarbonization with Node-RED and InfluxDB

Live Coding a KSQL Application

Internet of Things and Multi-model Data Infrastructure

Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience Lond...

Achieving scale and performance using cloud native environment

Rakuten Group, Inc.

Modeling the Smart and Connected City of the Future with Kafka and Spark

How Microsoft Built and Scaled Cosmos

From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...

Stateful, Stateless and Serverless - Running Apache Kafka® on Kubernetes

PowerStream Demo

Live Coding a KSQL Application

Aengus Rooney [Grafana] | What's New with Grafana and InfluxDB | InfluxDays E...

InfluxData Architecture for IoT | Noah Crowley | InfluxData

InfluxEnterprise Architectural Patterns by Dean Sheehan, Senior Director, Pre...

Container Monitoring Best Practices Using AWS and InfluxData by Gunnar Aasen

Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber

Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal

Using InfluxDB for Full Observability of a SaaS Platform by Aleksandr Tavgen,...

Schemas, streams, and grocery stores

Jay Kreps | Kafka Summit NYC 2019 Keynote (Events Everywhere) | CEO, Confluent

What's hot (20)

How to Enable Industrial Decarbonization with Node-RED and InfluxDB

Live Coding a KSQL Application

Internet of Things and Multi-model Data Infrastructure

Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience Lond...

Achieving scale and performance using cloud native environment

Modeling the Smart and Connected City of the Future with Kafka and Spark

How Microsoft Built and Scaled Cosmos

From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...

Stateful, Stateless and Serverless - Running Apache Kafka® on Kubernetes

PowerStream Demo

Live Coding a KSQL Application

Aengus Rooney [Grafana] | What's New with Grafana and InfluxDB | InfluxDays E...

InfluxData Architecture for IoT | Noah Crowley | InfluxData

InfluxEnterprise Architectural Patterns by Dean Sheehan, Senior Director, Pre...

Container Monitoring Best Practices Using AWS and InfluxData by Gunnar Aasen

Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber

Kafka Migration for Satellite Event Streaming Data | Eric Velte, ASRC Federal

Using InfluxDB for Full Observability of a SaaS Platform by Aleksandr Tavgen,...

Schemas, streams, and grocery stores

Jay Kreps | Kafka Summit NYC 2019 Keynote (Events Everywhere) | CEO, Confluent

Similar to Zero-downtime deployment on Kubernetes with Hazelcast

Zero-downtime Deployment on Kubernetes

All Things Open

JCon - Zero-Downtime-Deployment with Kubernetes, Spring Boot and Flyway

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

VoxxedDays Cluj - Zero-downtime deployment with Kubernetes, Spring Boot and F...

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

DevOpsDays Madrid - Zero-downtime deployment with Kubernetes, Spring Boot and...

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

XP Days Ukraine - Zero-downtime deployment with Kubernetes, Spring Boot and F...

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

OSCONF Koshi - Zero downtime deployment with Kubernetes, Flyway and Spring Boot

Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.

Tech talks - 3 performance improvements

JavaDay Istanbul - 3 improvements in your microservices architecture

BigData conference - Introduction to stream processing

This document discusses stream processing and summarizes a presentation about the topic. It introduces Hazelcast Jet as a stream processing engine and covers open data standards like GTFS. It also describes a demo that uses GTFS data to enrich public transit vehicle position updates in real-time using Hazelcast Jet. The presentation discusses streaming approaches, benefits over batch processing, and provides an overview of stream processing concepts.

YaJUG/Kaiserslautern JUG - 3 easy improvements in your microservices architec...

While microservices offer better scalability, they actually decrease performance and resiliency. I’ll show 3 areas in which it’s possible to cope with that. While a microservices architecture is more scalable than a monolith, it has a direct hit on performance. To cope with that, one performance improvement is to set up a cache. It can be configured for database access, for REST calls or just to store session state across a cluster of server nodes. In this demo-based talk, I’ll show how Hazelcast In-Memory Data Grid can help you in each one of those areas and how to configure it. Hint: it’s much easier than one would expect.

Istanbul JUG - 3 performance improvements with Hazelcast IMDG in your microse...

YAJUG - 3 Idées d’amélioration pour vos Architectures Microservices

Si une architecture en microservices est plus évolutive qu'une approche monolithique, elle a toutefois un impact direct sur les performances. Pour adresser cela, une amélioration de performance peut être obtenue en mettant en oeuvre un cache. II peut servir aux accés bases de données, aux invocations REST ou simplement pour conserver un état de session dans un cluster de noeuds serveur. Dans cette présentation basée sur des démonstrations, je vais vous montrer comment la solution Hazelcast "In Memory Data Grid" peut vous aider dans chacun de ces domaines et comment le configurer. Un petit indice: c'est beaucoup plus simple que vous pourriez le penser.

Voxxed Days Cluj - 3 performance improvements with Hazelcast IMDG in your mic...

go>tech world - 3 performance improvements with Hazelcast IMDG in your micros...

While a microservices architecture is more scalable than a monolith, it has a direct hit on performance. To cope with that, one performance improvement is to set up a cache. It can be configured for database access, for REST calls or just to store session state across a cluster. In this demo-based talk, I’ll show how Hazelcast In-Memory Data Grid can help you in each one of those areas and how to configure it. Hint: it’s much easier than one would expect. Takeaways: • Microservices are not for everyone • Microservices actually decrease performance but allow for horizontal scaling • Hazelcast IMDG can with the performance hit caused by microservices (or clustered application in general)

Devclub.lv - Introduction to stream processing

GeekOut - Configuration Management with Kubernetes, a Spring-Boot use-case

Come discover what options Kubernetes allows in terms of configuration management! Docker images are supposed to (and should be) immutable. Given that the same image should be promoted from the development environment to the production environment, it needs to be parameterized in each environment. Kubernetes provides different ways to do that. On the other, the Spring Boot platform allows a wide range of configuration options. In this talk, I’ll go through (and demo!) what common options are available, and what pro(s)/con(s) each of them have.

SouJava- 3 easy performance improvements in your microservices architecture

ConFoo - 3 performance improvements

Build User-Facing Analytics Application That Scales Using StarRocks (DLH).pdf

Albert Wong

This article isn't just about building an application, it's about empowering your users to unleash the full potential of their data. We'll delve into the world of StarRocks, a cutting-edge engine for real-time analytics, and guide you through the process of constructing user-facing applications that deliver fast, insightful, and visually captivating experiences. Dive into the Why: First, we'll explore the compelling reasons to choose StarRocks. We'll demystify its ability to handle high-velocity data streams and deliver sub-second query responses, making it ideal for applications where real-time insights are king. Laying the Foundation: Next, we'll embark on a step-by-step journey through the application development process. We'll discuss: Data Ingestion: Learn how to seamlessly integrate real-time data sources like Kafka and Pulsar into your application, ensuring a constant flow of fresh insights. Building the Core: We'll explore various options for structuring your application's backend, showcasing the pros and cons of different frameworks and libraries. Querying Made Easy: Delve into StarRocks' intuitive SQL interface and discover how to craft powerful queries that unlock the treasure trove of data hiding within. Visualization Wonderland: Explore the vast landscape of data visualization tools compatible with StarRocks. We'll showcase ways to transform complex data into interactive dashboards, compelling charts, and captivating reports that resonate with your users. Beyond the Basics: We'll push the boundaries by exploring advanced features like: Materialized Views: Discover how to pre-compute data summaries for even faster querying. Security and Access Control: Learn how to implement robust security measures to protect sensitive data while granting appropriate access levels to different user groups. Customization and Scalability: Explore options for tailoring the application to your specific needs and ensuring it can gracefully handle growing data volumes and user traffic. Conclusion: By the end of this article, you'll be equipped with the knowledge and tools to build user-facing analytics applications that empower your users to harness the power of real-time data and make confident, data-driven decisions. This journey isn't just about building an application, it's about transforming how your users interact with data, unlocking a world of actionable insights and limitless possibilities. So, are you ready to conquer complexity and unleash the power of StarRocks? Dive into this article and start building your next game-changing user-facing analytics application!

TidalScale Overview

Pete Jarvis

TidalScale has created a software defined computer. At TidalScale, we have created a simple cost-effective way for a data scientist, an analyst, an engineer, a scientist, a database administrator, or a software developer to access a group of servers through a single operating system instance as if it were a single supercomputer. This dramatically simplifies development, while reducing software scaling complexity not to mention a dramatic cost saving in hardware and software. We configure hosted hardware into one or more TidalPods. Each TidalPod is a virtual supercomputer comprising a set of commodity servers configured with the TidalScale HyperKernel. What the user sees is standard Linux, FreeBSD or Windows running with the sum of all memory, processors, networks, and I/O. The secret sauce is the HyperKernel that fools the guest OS into thinking it’s running directly on a huge, expensive machine when in fact it’s running on a set of smaller, less expensive servers. We offer an incredibly simple user experience. • Define the computer size you want (Number of CPU, Amount of Memory), boot the virtual machine, then login to the computer… Thus, we enable a simple cost-effective way for a data scientist, an analyst, an engineer, a scientist, a database administrator, or a software developer to access a group of servers in a Datacenter through a single operating system instance as if it were a single supercomputer. This dramatically simplifies development, while reducing software scaling complexity not to mention a dramatic cost saving in hardware and software.

Similar to Zero-downtime deployment on Kubernetes with Hazelcast (20)

Zero-downtime Deployment on Kubernetes

JCon - Zero-Downtime-Deployment with Kubernetes, Spring Boot and Flyway

VoxxedDays Cluj - Zero-downtime deployment with Kubernetes, Spring Boot and F...

DevOpsDays Madrid - Zero-downtime deployment with Kubernetes, Spring Boot and...

XP Days Ukraine - Zero-downtime deployment with Kubernetes, Spring Boot and F...

OSCONF Koshi - Zero downtime deployment with Kubernetes, Flyway and Spring Boot

Tech talks - 3 performance improvements

JavaDay Istanbul - 3 improvements in your microservices architecture

BigData conference - Introduction to stream processing

YaJUG/Kaiserslautern JUG - 3 easy improvements in your microservices architec...

Istanbul JUG - 3 performance improvements with Hazelcast IMDG in your microse...

YAJUG - 3 Idées d’amélioration pour vos Architectures Microservices

Voxxed Days Cluj - 3 performance improvements with Hazelcast IMDG in your mic...

go>tech world - 3 performance improvements with Hazelcast IMDG in your micros...

Devclub.lv - Introduction to stream processing

GeekOut - Configuration Management with Kubernetes, a Spring-Boot use-case

SouJava- 3 easy performance improvements in your microservices architecture

ConFoo - 3 performance improvements

Build User-Facing Analytics Application That Scales Using StarRocks (DLH).pdf

TidalScale Overview

More from Nicolas Fränkel

SnowCamp - Adding search to a legacy application

Most applications evolve to a point where they need to provide search capabilities. But updating an application is always a risk. Plus, sometimes, you don’t have access to the source code. The easiest way to access the data is by getting them directly from the database. The initial load is the easiest step. However, how do you keep the search index in sync with the database? How do you keep the latency between the search store and the source of truth, so your users don’t have to wait for the next run of the batch to access the newest changes? In this live coding session, we will show you how you can solve this issue by connecting Elasticsearch to the database with a touch of Hazelcast.

Un CV de dévelopeur toujours a jour

On dit que GitHub est le CV d'un développeur. Un rapide coup d'œil à votre historique de commit et les recruteurs savent tout de vous. Cette approche comporte quelques problèmes. La plupart des entreprises ne publient même pas leur code sous une licence Open Source. Si vous travaillez pour l'une d'entre elles, et si vous n'êtes pas un développeur Open Source les soirs et les week-ends, alors vous n'avez aucune chance. Récemment, GitHub a permis un certain degré de personnalisation de son profil. Ainsi, même si votre historique de commit a plus de blanc que de vert, vous pouvez fournir un bon point d'entrée pour les employeurs potentiels. Mais ça ne vaut que l'effort que vous y mettez et les données perdent leur valeur rapidement. Pourtant, avec un peu de travail et l'aide d'outils d'automatisation (tels que GitHub Actions), vous pouvez présenter un profil toujours à jour.

jLove - A Change-Data-Capture use-case: designing an evergreen cache

When one’s app is challenged with poor performances, it’s easy to set up a cache in front of one’s SQL database. It doesn’t fix the root cause (e.g. bad schema design, bad SQL query, etc.) but it gets the job done. If the app is the only component that writes to the underlying database, it’s a no-brainer to update the cache accordingly, so the cache is always up-to-date with the data in the database. Things start to go sour when the app is not the only component writing to the DB. Among other sources of writes, there are batches, other apps (shared databases exist unfortunately), etc. One might think about a couple of ways to keep data in sync i.e. polling the DB every now and then, DB triggers, etc. Unfortunately, they all have issues that make them unreliable and/or fragile. You might have read about Change-Data-Capture before. It’s been described by Martin Kleppmann as turning the database inside out: it means the DB can send change events (SELECT, DELETE and UPDATE) that one can register to. Just opposite to Event Sourcing that aggregates events to produce state, CDC is about getting events out of states. Once CDC is implemented, one can subscribe to its events and update the cache accordingly. However, CDC is quite in its early stage, and implementations are quite specific. In this talk, I’ll describe an easy-to-setup architecture that leverages CDC to have an evergreen cache.

ADDO - Your own Kubernetes controller, not only in Go

In Kubernetes, operators allow the API to be extended to your heart content. If one task requires too much YAML, it’s easy to create an operator to take care of the repetitive cruft, and only require a minimum amount of YAML. On the other hand, since its beginnings, the Go language has been advertised as closer to the hardware, and is now ubiquitous in low-level programming. Kubernetes has been rewritten from Java to Go, and its whole ecosystem revolves around Go. For that reason, It’s only natural that Kubernetes provides a Go-based framework to create your own operator. While it makes sense, it requires organizations willing to go down this road to have Go developers, and/or train their teams in Go. While perfectly acceptable, this is not the only option. In fact, since Kubernetes is based on REST, why settle for Go and not use your own favorite language? In this talk, I’ll describe what is an operator, how they work, how to design one, and finally demo a Java-based operator that is as good as a Go one.

TestCon Europe - Mutation Testing to the Rescue of Your Tests

Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant? Come discover mutation testing and make sure your never forget another assert again. In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn’t mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant. In this talk, Nicolas will explain how Code Coverage is computed and what its inherent flaw is. Afterwards, he will describe how Mutation Testing work and how it helps pointing out code that is tested but leave out corner cases. He will also demo PIT, a Java production-grade framework that enables Mutation Testing.

OSCONF Jaipur - A Hitchhiker's Tour to Containerizing a Java application

As “the Cloud” becomes more and more widespread, now is a good time to assess how you can containerize your Java application. I assume you’re able to write a a Dockerfile around the generated JAR. However, each time the application’s code will change, the whole image will need to be rebuilt. If you’re deploying to a local Kubernetes cluster environment, this increases that much the length of the feedback loop. In this demo-based talk, I’ll present different ways to get your Java app in a container: Dockerfile, Jib, and Cloud Native Buildpacks. We will also have a look at what kind of Docker image they generate, how they layer the images, whether those images are compatible with skaffold, etc.

GeekcampSG 2020 - A Change-Data-Capture use-case: designing an evergreen cache

OSCONF Hyderabad - Shorten all URLs!

At a point in the past, it was forecast that Java would die, but the JVM platform would be its legacy. And in fact, for a long time, the JVM has been tremendously successful. Wikipedia itself lists a bunch of languages that run on it, some of them close to Java e.g. Kotlin, some of them very remote e.g. Clojure. But nowadays, the Cloud is becoming ubiquitous. Containerization is the way to go to alleviate some of the vendor lock-in issues. Kubernetes is a de facto platform. If a container needs to be killed for whatever reason (resource consumption, unhealthy, etc.), a new one needs to replace it as fast as possible. In that context, the JVM seems to be a dead-end: its startup time is huge in comparison to a native process. Likewise, it consumes a lot of memory that just increase the monthly bill. What does that mean for us developers? Has all the time spent in learning the JVM ecosystem been invested with no hope of return over investment? Shall we need to invest even more time in new languages, frameworks, libraries, etc.? That is one possibility for sure. But we can also leverage our existing knowledge, and embrace the Cloud and containers ways with the help of some tools. In this talk, I’ll create a simple URL shortener with a “standard” stack: Kotlin, JAX-RS and Hazelcast. Then, with the help of Quarkus and GraalVM, I’ll turn this application into a native executable with all Cloud/Container related work has been moved to the build process.

JOnConf - A CDC use-case: designing an Evergreen Cache

This document discusses using change data capture (CDC) and Hazelcast Jet to build an evergreen cache that remains in sync with a database. It covers alternatives to cache invalidation like polling and triggers, introduces CDC and the Debezium implementation, and proposes a Jet job that watches database change events, analyzes them, and updates the cache accordingly to solve the cache freshness problem.

London In-Memory Computing Meetup - A Change-Data-Capture use-case: designing...

JUG Tirana - Introduction to data streaming

This talk will be about the reasons behind the new stream processing model, how it compare to the old batch model, what are their pros and cons, and a list of existing technologies implementing stream processing with their most prominent characteristics. It will contain details of one possible use-case of data streaming that is not possible with batches: display in (near) real-time all trains in Switzerland and their position on a map, beginning with an overview of all the requirements and the design. Finally, using an OpenData endpoint and the Hazelcast platform,showing a working demo implementation of it.

Java.IL - Your own Kubernetes controller, not only in Go!

vJUG - Introduction to data streaming

The document discusses stream processing and provides an overview of Hazelcast Jet. It begins with explaining why streaming is useful and describes different streaming approaches like event-driven programming. It then provides details on Hazelcast Jet, including its concepts of pipelines and jobs. The document also discusses open data standards like GTFS and demonstrates a sample streaming pipeline that enriches public transportation data from open APIs.

London Java Community - An Experiment in Continuous Deployment of JVM applica...

A couple of years ago, continuous integration in the JVM ecosystem meant Jenkins. Since that time, a lot of other tools have been made available. But new tools don’t mean new features, just new ways. Besides that, what about continuous deployment? There’s no tool that allows deploying new versions of a JVM-based application without downtime. The only way to achieve zero downtime is to have multiple nodes deployed on a platform, and let that platform achieve that e.g. Kubernetes. And yet, achieving true continuous deployment of bytecode on one single JVM instance is possible if one changes one’s way of looking at things. What if the compilation could be seen as changes? What if those changes could be stored in a data store, and a listener on this data store could stream those changes to the running production JVM via the Attach API? In this talk, we'll demo exactly that using Hazelcast and Hazelcast Jet - but it’s possible to re-use the principles that will be shown using other streaming technologies.

OSCONF - Your own Kubernetes controller: not only in Go

This document discusses creating Kubernetes operators and controllers using different programming languages besides Go. It suggests that a Java-based controller is possible using the GraalVM, which allows creating native executables from Java bytecode. Key points covered include what controllers and operators are, that no specific technology stack is required, and that the JVM could be a good option for controller development with GraalVM's support for polyglot programming and creating native applications.

vKUG - Migrating Spring Boot apps from annotation-based config to Functional

Migrating Spring Boot apps from annotation-based config to Functional with Kotlin - Nicolas Fränkel In the latest years, there has been some push-back against frameworks, and more specifically annotations: some call them magic. Obviously, they make understanding the flow of the application harder. Spring and Spring Boot latest versions go along this trend, by offering an additional way to configure beans with explicit code instead of annotations. It’s declarative in the sense it looks like configuration, though it’s based on Domain-Specific Language(s). This talk aims to demo a step-by-step process to achieve that.

AllTheTalks.online - A Streaming Use-Case: And Experiment in Continuous Deplo...

Nicolas Fränkel proposes an experimental use case of continuous deployment of bytecode by dynamically loading Java agents into a running JVM using the Attach API. The agent would use the Instrumentation API to modify bytecode of running applications by redefining classes in order to deploy changes made by a continuous integration pipeline without restarting the JVM. Current limitations include a lack of tagging and an inability to add/remove fields or methods via redefinition. Improvements could include streaming changes directly from the CI pipeline and adding metadata to classes from the Git tag.

ING Meetup - Migrating Spring Boot Config Annotations to Functional with Kotlin

In the latest years, there has been some push-back against frameworks, and more specifically annotations: some call them magic. Obviously, they make understanding the flow of the application harder. Spring and Spring Boot latest versions go along this trend, by offering an additional way to configure beans with explicit code instead of annotations. It's declarative in the sense it looks like configuration, though it's based on Domain-Specific Language(s). This talk aims to demo a step-by-step process to achieve that.

JUG SF - Introduction to data streaming

While “software is eating the world”, those who are able to best manage the huge mass of data will emerge out on the top. The batch processing model has been faithfully serving us for decades. However, it might have reached the end of its usefulness for all but some very specific use-cases. As the pace of businesses increases, most of the time, decision makers prefer slightly wrong data sooner, than 100% accurate data later. Stream processing - or data streaming - exactly matches this usage: instead of managing the entire bulk of data, manage pieces of them as soon as they become available. In this talk, I’ll define the context in which the old batch processing model was born, the reasons that are behind the new stream processing one, how they compare, what are their pros and cons, and a list of existing technologies implementing the latter with their most prominent characteristics. I’ll conclude by describing in detail one possible use-case of data streaming that is not possible with batches: display in (near) real-time all trains in Switzerland and their position on a map. I’ll go through the all the requirements and the design. Finally, using an OpenData endpoint and the Hazelcast platform, I’ll try to impress attendees with a working demo implementation of it.

SCALE - Stream processing and Open Data, a match made in Heaven