Serverless (also known as function-as-a-service) is fast emerging as an effective architecture for event-driven applications. Apache OpenWhisk is one of the more popular open-source cloud serverless platforms, and has first-class support for Kafka as a source of events. Come to this session for an introduction to building microservices without servers using OpenWhisk. Ill describe the challenges to building applications using serverless stacks, and the serverless design patterns to help you get started. Ill give a demonstration of how you can use Kafka Connect to invoke serverless actions, and how serverless can be an effective way to host event-processing logic.

1. © 2019 IBM Corporation
Building serverless apps with Kafka
Dale Lane
An introduction to serverless with Apache OpenWhisk
IBM Event StreamsApache Kafka
@dalelane

2. © 2019 IBM Corporation
Agenda
§ What is “serverless”?
§ Implementations
§ Apache OpenWhisk
§ Demos

3. © 2019 IBM Corporation
What is “serverless”?

4. © 2019 IBM Corporation
Serverless
§An architectural pattern where your code is run:
– in response to events
– only on-demand on a per-request basis

5. © 2019 IBM Corporation
Evolution of deployment patterns
Monolith

6. © 2019 IBM Corporation
Evolution of deployment patterns
Microservice
Microservice
Microservice
Monolith

7. © 2019 IBM Corporation
Evolution of deployment patterns
Function
Function
Function
Function
Function
Function
Function
Function
Microservice
Microservice
Microservice
Monolith

8. © 2019 IBM Corporation
Evolution of deployment patterns
Bare metal Virtual
machines
Containers Orchestration Serverless

9. © 2019 IBM Corporation
Function
FaaS
Platform
PaaS
Infrastructure
IaaS
Container
CaaS
Bare metal
Evolution of deployment patterns

10. © 2019 IBM Corporation
Definition
github.com/cncf/wg-serverless

11. © 2019 IBM Corporation
Definition
github.com/cncf/wg-serverless

12. © 2019 IBM Corporation
Definition
applications that do not require server management
applications bundled as one or more functions
[developers] no longer need to spend time and resources on server
provisioning, maintenance, updates, scaling and capacity planning

13. © 2019 IBM Corporation
How is serverless different?
Monolith Microservice
Microservice
Microservice
Function
Function
Function
Function
Function
Function
Function
Function

14. © 2019 IBM Corporation
How is serverless different?

15. © 2019 IBM Corporation
How is serverless different?

16. © 2019 IBM Corporation
How is serverless different?
£ $ €
kr Fr
¥

17. © 2019 IBM Corporation
Example use cases
§ Event processing
§ Data processing
§ Scheduled tasks
§ Scalable REST APIs

18. © 2019 IBM Corporation
The Twelve Factor App

19. © 2019 IBM Corporation
Serverless is good for:
§ Infrequent or sporadic demand
§ Focused, short duration processing
§ Highly dynamic workloads
§ Stateless workloads
§ Inconsistent demand
§ Scheduled infrequent tasks
§ Flexibility

20. © 2019 IBM Corporation
Implementations

21. © 2019 IBM Corporation
Implementations – platform

22. © 2019 IBM Corporation
Implementations – platform

23. © 2019 IBM Corporation
Implementations – platform

24. © 2019 IBM Corporation
Implementations – platform

25. © 2019 IBM Corporation
Implementations – platform

26. © 2019 IBM Corporation
Implementations – hosted

27. © 2019 IBM Corporation
Implementations – private cloud

28. © 2019 IBM Corporation
Implementations – installable

29. © 2019 IBM Corporation
Implementations
s.cncf.io

30. © 2019 IBM Corporation
Serverless framework
serverless.com/framework

31. © 2019 IBM Corporation
Apache OpenWhisk

32. © 2019 IBM Corporation
OpenWhisk
openwhisk.apache.org

33. © 2019 IBM Corporation
OpenWhisk – who

34. © 2019 IBM Corporation
OpenWhisk – where

35. © 2019 IBM Corporation
OpenWhisk concepts

36. © 2019 IBM Corporation
OpenWhisk concepts
event

37. © 2019 IBM Corporation
OpenWhisk concepts
event action

38. © 2019 IBM Corporation
OpenWhisk concepts
event action
function (eventobj)
{
// do something
return result;
}

39. © 2019 IBM Corporation
OpenWhisk languages

40. © 2019 IBM Corporation
OpenWhisk languages

41. © 2019 IBM Corporation
OpenWhisk concepts
event action

42. © 2019 IBM Corporation
OpenWhisk concepts
event actiontrigger

43. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent

44. © 2019 IBM Corporation
OpenWhisk concepts
actionrule
event
actionrule
actionrule
actionrule
trigger

45. © 2019 IBM Corporation
OpenWhisk concepts
rule
event
rule
rule
rule
trigger
trigger
trigger
trigger
action
event
event
event

46. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
} hello.js

47. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
} hello.js

48. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
}
$ wsk action create helloworld hello.js
ok: created action helloworld
hello.js

49. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
}
$ wsk action create helloworld hello.js
ok: created action helloworld
$ wsk action invoke helloworld
ok: invoked /guest/helloworld with id 7df9c9dfbb
hello.js

50. © 2019 IBM Corporation
OpenWhisk actions
blocking
non-blocking

51. © 2019 IBM Corporation
OpenWhisk actions
blocking
non-blocking
periodic

52. © 2019 IBM Corporation
OpenWhisk actions
action action action

53. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent

54. © 2019 IBM Corporation
OpenWhisk implementation
proxy

55. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller

56. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store

57. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
service registry
invoker

58. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
service registry
invoker
invoker
invoker
invoker
invoker
invoker

59. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

60. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

61. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

62. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

63. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

64. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

65. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

66. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

67. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0

68. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0

69. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0

70. © 2019 IBM Corporation
Resource limits
github.com/apache/incubator-openwhisk

71. © 2019 IBM Corporation
Demos – installable

72. © 2019 IBM Corporation
Deployment options

73. © 2019 IBM Corporation
git clone https://github.com/apache/incubator-openwhisk-devtools.git
cd incubator-openwhisk-devtools/docker-compose
make quick-start

74. © 2019 IBM Corporation
OpenWhisk event providers
trigger ruleevent

75. © 2019 IBM Corporation
cd incubator-openwhisk-devtools/docker-compose
make create-provider-cloudant
make create-provider-kafka

76. © 2019 IBM Corporation
git clone https://github.com/apache/incubator-openwhisk-package-kafka.git
cd incubator-openwhisk-package-kafka
export OPENWHISK_AUTH=$OPENWHISK_HOME/ansible/files/auth.whisk.system
./installKafka.sh $OPENWHISK_AUTH $SYSTEM_IP_ADDR
http://$SYSTEM_IP_ADDR:5984
local_
$SYSTEM_IP_ADDR

77. © 2019 IBM Corporation
Demo

78. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS

79. © 2019 IBM Corporation
kafka-topics.sh --create --topic MY.EVENTS
--replication-factor 1 --partitions 1
--zookeeper $SYSTEM_IP_ADDR:3181

80. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger

81. © 2019 IBM Corporation
wsk trigger create myKafkaTrigger
-f /whisk.system/messaging/kafkaFeed
-p brokers '["9.174.23.208:3092"]'
-p topic MY.EVENTS

82. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger

83. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action

84. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action

85. © 2019 IBM Corporation
wsk action create sendToSlack slack-action.js

86. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action

87. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
slackOnKafkaMessages
sendToSlack
action
action

88. © 2019 IBM Corporation
wsk rule create slackOnKafkaMessages
myKafkaTrigger
sendToSlack

89. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
slackOnKafkaMessages
sendToSlack
action
action

90. © 2019 IBM Corporation

91. © 2019 IBM Corporation
Reminder

92. © 2019 IBM Corporation
Demos – using a hosted service

93. © 2019 IBM Corporation
Demo
Orders Voyages Ships Containers Alerts
VoyagesOrder
Order
Management
Microservice
Fleet/Ships
Simulator
Microservice
Voyages
Management
Microservice
Kafka Streams
Kafka

94. © 2019 IBM Corporation

95. © 2019 IBM Corporation
Demo

96. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
“alerts”
kafkaAlert
trigger
processAlert
action
action

97. © 2019 IBM Corporation

98. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
“alerts”
kafkaAlert
trigger
processAlert
action
action

99. © 2019 IBM Corporation
Orders Voyages Ships Containers Alerts
VoyagesOrder
Order
Management
Microservice
Fleet/Ships
Simulator
Microservice
Voyages
Management
Microservice
Kafka
Kafka Streams

100. © 2019 IBM Corporation
Summary

101. © 2019 IBM Corporation
Summary / roundup
§An architectural pattern where your code is run:
– in response to events
– only on-demand on a per-request basis

102. © 2019 IBM Corporation
Getting started
twitter.com/openwhisk
medium.com/openwhisk
slack.openwhisk.org
youtube.com/channel/UCbzgShnQk8F43NKsvEYA1SA
#openwhisk
developer.ibm.com/code/patterns/category/serverless
serverlessconf.io

103. © 2019 IBM Corporation
Thank you
Dale Lane
Software Engineer, IBM Event Streams
IBM Event Streams - ibm.com/cloud/event-streams
https://slack-invite-ibm-cloud-tech.mybluemix.net/