This document discusses using reactive event-driven architecture with Grails applications. It introduces the Platform-Core plugin, which provides an events API for publishing and listening to events within Grails applications and across plugins. It also discusses several other plugins that integrate Platform-Core events with technologies like Spring Integration, Atmosphere for client-side events, and Vert.x for clustering and scalability. The document provides code examples of publishing and handling events and also discusses configuring events using a DSL.
Apache Kafka and Apache Pulsar are both popular messaging frameworks. Apache Kafka has a big user base and people will want to know how Kafka and Pulsar are either the same or different in many respects. This talk will cover the key differences and how Pulsar adds new features that missing in Kafka.
We will cover:
The architectural differences and similarities in Pulsar and Kafka. Show use of BookKeeper and what that allows.
The Producer API and functionality differences. Show HelloWorld for both.
The Consumer API and functionality differences. Show HelloWorld for both.
The core use case and functionality differences. Show Pulsar as handling all of Kafka’s use cases and new ones that aren’t possible with Kafka.
This talk will allow people who are choosing between Kafka and Pulsar to have a more accurate and in-depth understanding of the differences between them. For companies considering a switch from Kafka to Pulsar, this talk will give them the cheatsheet to go back and make a more informed decision.
Cloud Native Spring - The role of Spring Cloud after Kubernetes became a main...Orkhan Gasimov
Presentation of my talk about Spring Cloud features that can integrate with AWS, GCP and Azure turning Spring Cloud into a distributed platform that is capable to work with different environments like Kubernetes, Cloud or Local with adoption of Spring abstractions.
Pulsar Watermarking - Pulsar Virtual Summit Europe 2021StreamNative
This session is a technical deep-dive into the Apache Pulsar community's plan to support event-time watermarking in a Pulsar topic. The motivation is to simplify and improve the correctness of stream processing applications.
Deep Dive into the Pulsar Binary Protocol - Pulsar Virtual Summit Europe 2021StreamNative
To achieve maximum performance, some important choices have been made when designing the Pulsar binary protocol.
This session will explain how Pulsar implements all the features of a high quality streaming protocol such as frame multiplexing, session establishment, keep-alive, flow control, authentication and authorisation, encoding, zero-copy capabilities and more.
Building Fast and Scalable Persistence Layers with Spring Data JPAVMware Tanzu
SpringOne 2021:
Session Title: Building Fast and Scalable Persistence Layers with Spring Data JPA
Speaker: Thorben Janssen, Freelancer at Self-employed
Apache Kafka and Apache Pulsar are both popular messaging frameworks. Apache Kafka has a big user base and people will want to know how Kafka and Pulsar are either the same or different in many respects. This talk will cover the key differences and how Pulsar adds new features that missing in Kafka.
We will cover:
The architectural differences and similarities in Pulsar and Kafka. Show use of BookKeeper and what that allows.
The Producer API and functionality differences. Show HelloWorld for both.
The Consumer API and functionality differences. Show HelloWorld for both.
The core use case and functionality differences. Show Pulsar as handling all of Kafka’s use cases and new ones that aren’t possible with Kafka.
This talk will allow people who are choosing between Kafka and Pulsar to have a more accurate and in-depth understanding of the differences between them. For companies considering a switch from Kafka to Pulsar, this talk will give them the cheatsheet to go back and make a more informed decision.
Cloud Native Spring - The role of Spring Cloud after Kubernetes became a main...Orkhan Gasimov
Presentation of my talk about Spring Cloud features that can integrate with AWS, GCP and Azure turning Spring Cloud into a distributed platform that is capable to work with different environments like Kubernetes, Cloud or Local with adoption of Spring abstractions.
Pulsar Watermarking - Pulsar Virtual Summit Europe 2021StreamNative
This session is a technical deep-dive into the Apache Pulsar community's plan to support event-time watermarking in a Pulsar topic. The motivation is to simplify and improve the correctness of stream processing applications.
Deep Dive into the Pulsar Binary Protocol - Pulsar Virtual Summit Europe 2021StreamNative
To achieve maximum performance, some important choices have been made when designing the Pulsar binary protocol.
This session will explain how Pulsar implements all the features of a high quality streaming protocol such as frame multiplexing, session establishment, keep-alive, flow control, authentication and authorisation, encoding, zero-copy capabilities and more.
Building Fast and Scalable Persistence Layers with Spring Data JPAVMware Tanzu
SpringOne 2021:
Session Title: Building Fast and Scalable Persistence Layers with Spring Data JPA
Speaker: Thorben Janssen, Freelancer at Self-employed
Leveraging the power of SolrCloud and Spark with OpenShiftQAware GmbH
Kubernetes/Cloud-Native-Meetup September 2018, Munich : Talk by Franz Wimmer (@zalintyre, Software Engineer at QAware)
Abstract: One of the most commonly used big data processing frameworks is Apache Spark. Spark manages to process large datasets with parallelization. Solr is a search platform based on Lucene. Solr can be distributed across a cluster using ZooKeeper for configuration management. Both applications can be combined to create performant Big Data applications.
But what if you want to scale up horizonally and add a node? In a manual setup, you'd have to install the new node manually. Cluster orchestrators like OpenShift claim to solve this problem.
This talk shows how to put Spark, Solr and ZooKeeper into containers, which can then be scaled individually inside a cluster using OpenShift. We will cover OpenShift details like DeploymentConfigs, StatefulSets, Services, Routes and Persistent Volumes and install a complete, failsafe and horizontally scaleable SolrCloud / Spark / Zookeeper cluster in seconds.
You will also learn about the drawbacks and pitfalls of running Big Data applications inside an OpenShift cluster.
OpenShift/Kubernetes to Splunk log integrationMichiel Kalkman
This presentation covers OpenShift/Kubernetes platform-level log integration with Splunk using fluent-bit and was presented at the Melbourne Splunk meetup 2018-04.
In our deep-dive technical series, we look at the strategic importance of having Baas as part of your API Management solution. Apigee Edge API BaaS enables web and mobile app developers to link their apps to a cloud datastore and provide features including user management, push notifications, geolocation services, and more.
Episode 4: Operating Kubernetes at Scale with DC/OSMesosphere Inc.
You’ve installed your Kubernetes cluster on DC/OS — now what? Operating Kubernetes efficiently can be challenging. In the final episode of our Kubernetes series, we will share best practices for operating your DC/OS Kubernetes cluster and maintaining performance. During this presentation, Joerg Schad and Chris Gaun show you how to successfully operate Kubernetes at scale in your environment.
During this session, we discuss:
1. How to upgrade DC/OS and Kubernetes with no downtime
2. How DC/OS guards against failure and enables fault domains that are resistant to outages within racks, availability zones, or cloud environments
3. How the monitoring and metrics capabilities on DC/OS improve operational analytics and help you get the most from your cluster
4. How cloud bursting extends your on-prem environment with resources from the cloud to handle spikes in your workload
The Serverless Paradigm, OpenWhisk and FIWAREAlex Glikson
Outline:
1. Overview of Serverless
2. OpenWhisk – open source ‘Serverless’ platform
3. Challenges of Serverless
4. Serverless and FIWARE
Alex Glikson
Cloud Platforms, IBM Research
Architect, FIWARE Cloud Hosting
CCF 4 XAP has been designed to exploit XAP capabilities on the cloud and leverage XAP scalability, low latency and high-throughput features when deployed in such dynamic environment
Devfest 2023 - Service Weaver Introduction - Taipei.pdfKAI CHU CHUNG
In modern software development, decentralized applications are increasingly common. Decentralized applications can split applications into multiple independent services, each service can be developed, deployed and managed independently.
Service Weaver is a decentralized application development framework provided by Google Cloud. It helps you develop, deploy and manage decentralized applications easily.
In this session, Google Cloud developer expert Kai-Chu Chung will introduce the basic concepts and usage of Service Weaver.
Event-based APIs are becoming more popular, enabling developers to craft new integrations and solutions that go beyond the original design of an API. Yet, there remains a challenge: how can teams design thoughtful event-based APIs that are long-lasting, evolvable, and discoverable? This talk will dive into the design practices of event-based APIs, including tips for determining which protocol(s) you should select, design patterns we should apply, and anti-patterns should we avoid. We will also look at how AI and tools such as ChatGPT are starting to shape the next generation of APIs.
Delivered on May 10, 2023 for the EDA Summit
4156 Twist and cloud-how ibm customers make cics dancenick_garrod
InterConnect 2015 Session 4156 Twist and Cloud - How customers make CICS dance. Putting CICS into the cloud, is someone actually doing this? Since 2012 CICS TS Version 5 has been introducing and continuously enhancing its capabilities to support a Platform As A Service approach. Join speakers from CICS Technical Sales to hear about their experiences and how their customers gained value from cloud capabilities in CICS.
In this WebHack talk I shared my experience about microservices, Docker, Kubernetes and Kong, an API gateway by Mashape. Since they are based on a real working system, this slides is majorly for how to build the whole thing up, not about detailed internal implementation. Although I included some details and reference in order to make it more comprehensive.
Leveraging the power of SolrCloud and Spark with OpenShiftQAware GmbH
Kubernetes/Cloud-Native-Meetup September 2018, Munich : Talk by Franz Wimmer (@zalintyre, Software Engineer at QAware)
Abstract: One of the most commonly used big data processing frameworks is Apache Spark. Spark manages to process large datasets with parallelization. Solr is a search platform based on Lucene. Solr can be distributed across a cluster using ZooKeeper for configuration management. Both applications can be combined to create performant Big Data applications.
But what if you want to scale up horizonally and add a node? In a manual setup, you'd have to install the new node manually. Cluster orchestrators like OpenShift claim to solve this problem.
This talk shows how to put Spark, Solr and ZooKeeper into containers, which can then be scaled individually inside a cluster using OpenShift. We will cover OpenShift details like DeploymentConfigs, StatefulSets, Services, Routes and Persistent Volumes and install a complete, failsafe and horizontally scaleable SolrCloud / Spark / Zookeeper cluster in seconds.
You will also learn about the drawbacks and pitfalls of running Big Data applications inside an OpenShift cluster.
OpenShift/Kubernetes to Splunk log integrationMichiel Kalkman
This presentation covers OpenShift/Kubernetes platform-level log integration with Splunk using fluent-bit and was presented at the Melbourne Splunk meetup 2018-04.
In our deep-dive technical series, we look at the strategic importance of having Baas as part of your API Management solution. Apigee Edge API BaaS enables web and mobile app developers to link their apps to a cloud datastore and provide features including user management, push notifications, geolocation services, and more.
Episode 4: Operating Kubernetes at Scale with DC/OSMesosphere Inc.
You’ve installed your Kubernetes cluster on DC/OS — now what? Operating Kubernetes efficiently can be challenging. In the final episode of our Kubernetes series, we will share best practices for operating your DC/OS Kubernetes cluster and maintaining performance. During this presentation, Joerg Schad and Chris Gaun show you how to successfully operate Kubernetes at scale in your environment.
During this session, we discuss:
1. How to upgrade DC/OS and Kubernetes with no downtime
2. How DC/OS guards against failure and enables fault domains that are resistant to outages within racks, availability zones, or cloud environments
3. How the monitoring and metrics capabilities on DC/OS improve operational analytics and help you get the most from your cluster
4. How cloud bursting extends your on-prem environment with resources from the cloud to handle spikes in your workload
The Serverless Paradigm, OpenWhisk and FIWAREAlex Glikson
Outline:
1. Overview of Serverless
2. OpenWhisk – open source ‘Serverless’ platform
3. Challenges of Serverless
4. Serverless and FIWARE
Alex Glikson
Cloud Platforms, IBM Research
Architect, FIWARE Cloud Hosting
CCF 4 XAP has been designed to exploit XAP capabilities on the cloud and leverage XAP scalability, low latency and high-throughput features when deployed in such dynamic environment
Devfest 2023 - Service Weaver Introduction - Taipei.pdfKAI CHU CHUNG
In modern software development, decentralized applications are increasingly common. Decentralized applications can split applications into multiple independent services, each service can be developed, deployed and managed independently.
Service Weaver is a decentralized application development framework provided by Google Cloud. It helps you develop, deploy and manage decentralized applications easily.
In this session, Google Cloud developer expert Kai-Chu Chung will introduce the basic concepts and usage of Service Weaver.
Event-based APIs are becoming more popular, enabling developers to craft new integrations and solutions that go beyond the original design of an API. Yet, there remains a challenge: how can teams design thoughtful event-based APIs that are long-lasting, evolvable, and discoverable? This talk will dive into the design practices of event-based APIs, including tips for determining which protocol(s) you should select, design patterns we should apply, and anti-patterns should we avoid. We will also look at how AI and tools such as ChatGPT are starting to shape the next generation of APIs.
Delivered on May 10, 2023 for the EDA Summit
4156 Twist and cloud-how ibm customers make cics dancenick_garrod
InterConnect 2015 Session 4156 Twist and Cloud - How customers make CICS dance. Putting CICS into the cloud, is someone actually doing this? Since 2012 CICS TS Version 5 has been introducing and continuously enhancing its capabilities to support a Platform As A Service approach. Join speakers from CICS Technical Sales to hear about their experiences and how their customers gained value from cloud capabilities in CICS.
In this WebHack talk I shared my experience about microservices, Docker, Kubernetes and Kong, an API gateway by Mashape. Since they are based on a real working system, this slides is majorly for how to build the whole thing up, not about detailed internal implementation. Although I included some details and reference in order to make it more comprehensive.
What's next for Reactive Systems ? Next is making apps communications reactive. This keynote presentation was given with Steve Gury from Facebook Engineering.
This is your one stop shop introduction to get oriented to the world of reactive programming. There are lots of such intros out there even manifestos. We hope this is the one where you don't get lost and it makes sense. Get a definition of what "reactive" means and why it matters. Learn about Reactive Streams and Reactive Extensions and the emerging ecosystem around them. Get a sense for what going reactive means for the programming model. See lots of hands-on demos introducing the basic concepts in composition libraries using RxJava and Reactor.
1. Reactive Grails
Event-driven architecture made easy
Stephane Maldini – Consultant @ SpringSource/VMware*
* Changes may apply in the next couple of months
3. In Memory use cases
• Loosely coupled
– Plugin oriented
– Testable handlers without mocking
• Declarative logic
– Workflow by contract
• Threading scalability
– Scale to multicores without digging too deep in your code
3
4. In Cloud use cases
• Sharing workload
– And on demand !
– One machine can handle the task: using a shared work queue
• Synchronizing
– All machines need to work: Cache invalidation, indexing...
• Communicating with Service API
– Layering at machine level instead of software level
4
5. The big picture
Application
Service Listener
Service Listener Point to point
Events
Bus Service Listener
Service Listener
Service Listener
5
6. The big picture
Application
Service Listener
Service Listener Publish/Subscribe
Events
Bus Service Listener
Service Listener
Service Listener
6
7. The big picture
Cloud
App
App Publish/Subscribe
Events
Bus App
App
App
7
9. Grails on bus : current plugins
• Spring Events
– Use ApplicationContext to propagate ApplicationEvents
– Code boilerplate
• Routing (Apache Camel)
– Super flexible
– Simple but powerful
– Routing DSL but not so Grailsy
• Falcone Utils
– Fully integrated with Grails
– Not maintained
9
10. Platform-Core
• This plugin purpose is to provide key features for todays
application development and plugin oriented architecture
• Provided by Marc Palmer and your guest
• Initiated mid 2012, currently version 1.0.RC2:
– Nearly complete API stabilization
– Integrated with grails development
• Fully extensible, API driven design
10
11. Platform-Core: API
Configuration
Navigation
Security
Injection
Events
11
13. Platform-Core : Events
• Grails Apps and Plugins can use Platform-Core/Events to:
– Listen for plugins/app events
– Avoid topic name conflicts using namespacing support
– Do in-memory eventing
– Do Asynchronous calls (default)
– Increase in flexibility if required
13
14. Platform-Core : Events
• Add this to your BuildConfig RIGHT NOW :
– compile :platform-core:1.0.RC2
14
15. Events : Sending Events
class
UserController{
def
registration(){
def
user
=
new
User(params).save()
if(user){
//non-‐blocking
call,
will
trigger
application
listeners
for
this
topic
event('mailRegistration',
user)
//blocking
call
:
//event('mailRegistration',
user).waitFor()
//can
also
be
written
like
that
//event
topic:'mailRegistration',
data:user
//and
if
you
need
to
reuse
the
current
thread
//event
topic:'mailRegistration',
data:user,
fork:false
render(view:'sendingRegistrationMail')
}else{
render(view:'errorRegistration')
}
}
}
15
16. Events : Listening Events
class
UserService{
//use
method
name
'mailRegistration'
as
topic
name
//can
also
use
custom
topic
name
using
topic
arg:
@Listener(topic='test')
@grails.events.Listener
def
mailRegistration(User
user){
sendMail{
to
user.mail
subject
"Confirmation"
html
g.render(template:"userMailConfirmation")
}
}
//Can
also
receive
an
EventMessage
to
get
more
information
on
this
particular
event)
@grails.events.Listener(topic="mailRegistration")
def
mailRegistration2(org.grails.plugin.platform.events.EventMessage
msg){
sendMail{
to
msg.data.mail
subject
"Confirmation"
html
g.render(template:"userMailConfirmation")
}
}
}
16
17. Events : DSL
events
=
{
//prevents
any
events
in
gorm
namespace
'*'
namespace:'gorm',
disabled:true
//filters
any
events
on
'testTopic'
where
data
<=
2
testTopic
filter:{it
>
2}
//filters
any
events
on
'testTopic2'
where
data
is
not
a
TestTopic
class
type
testTopic2
filter:TestTopic
//filters
any
events
on
'testTopicX'
using
boolean
method
from
service
testTopicX
filter:ctx.myService.&someMethod
//only
if
using
events-‐push
plugin,
allows
client-‐side
listener
on
this
topic
testTopic3
browser:true
//Default
Error
Handling,
Global
Reply
Handling,
timeout
and
fork
testTopicD
onError:{},
onReply:{},
timeout:1000l
testTopicD2
fork:false
}
17
18. Platform-Core : Events, more!
• Listen to GORM events
– using the same API
• Reply handlers
– Return values from your listeners to pass them back to caller
• Errors handler
– Listen for any exceptions
18
20. Events : Plugins
• Leveraging and enriching the Platform-core API
• May update events registering/dispatching engine
• Combine them and achieve kool kombos
• Currently 3 plugins in development
– Events-SI : Spring Integration backed events
– Events-Push : Events to the browser using Atmosphere
– Vertx : Vertx integration and platform-core backed events
20
21. grails-events-si
• Change in-memory default mechanism:
– @Listener now generates a channel + endpoint
– event methods now send through Spring Integration gateway
– This channel can be overriden using the naming convention
– Super flexible, connect to the world
21
23. grails-events-si + Groovy DSL
• Based on the excellent module from David Turanski:
– https://github.com/SpringSource/spring-integration-dsl-groovy
• Integration with Events plugin registry and publishing
– Easy to add queuing behavior
– Easy to transform Pub/Sub to Point 2 Point
– Additional declarative logic (filtering, system integration etc)
• Still possible to fallback to standard BeanBuilder
23
25. How to send through RabbitMQ in minutes
With this in BuildConfig :
runtime ":rabbitmq:1.0.0 "
runtime ":events-si:1.0.M4-SNAPSHOT"
beans = { !
xmlns siAmqp:
'http://www.springframework.org/schema/integration/amqp' !
!
siAmqp.'publish-subscribe-channel'(id: 'gorm://afterInsert') !
!
siAmqp.'publish-subscribe-channel'(id: 'gorm://afterDelete') !
!
siAmqp.'publish-subscribe-channel'(id: 'gorm://afterUpdate') !
!
}!
25
26. grails-events-push
• And if you want to listen for events... in Javascript
– Like Socket.IO or Vertx
– Like a free bird
– But not so free, prevent from listening everything
– But which damn protocol using ??
26
31. grails-vertx
• No hands clustering
– Using Vert.x Hazelcast support
• Super scalable events processor
– Mom’ said Event loop is darn good at handling high concurrent
load
• Deploying features as verticles
– WIN for hot reloading plugins/services
31
34. Roadmaps
• Platform-Core
– Work Queue / Point to Point eventing
– Response Streaming
• Events-SI
– Overridable listeners with Queue channels
• Events-Push
– More on filtering
– ... per request security (with PlatformCore security)
34
35. Roadmaps
• Some more Vertx love:
– Better integration
– Specific DSL to manage verticles
– SockJS support
– request dispatching
• Vertx Back / Grails Front
• Vertx Front / Grails Back ?
– Rename it Testx
35