WAMP as a platform for composite SOA applications and its implementation on Lua

WAMP as a platform
for composite SOA
applications and its
implementation on Lua
Konstantin Burkalev
@KSDaemon

Few words about me…
2
• Software architect
• 15+ years in IT
• Author and host of podcasts: SDCast & RadioJS
• Active member of WAMP protocol development group
• Author and maintainer of a few WAMP implementations,
some of them are written in Lua

Talk plan
Example of system using WAMP
What is WAMP and how it works
WAMP as basis for SOA applications
Application fields
WAMP client in Lua
WAMP router in Nginx/Lua
3

Living room
Task: Smart Home System
4

Bath room
Living room
4

Bath room
Living room
4
User

The temperature sensor detects
the temperature raise
Bath room
Living room
4
User

The client is notified
Logged in database
Bath room
Living room
4
User

The system turns on air conditioning
Bath room
Living room
4
User

The temperature in the room decreases
Bath room
Living room
4
User

The system turns off the air conditioning
Bath room
Living room
4
User

The humidity in the bathroom rises
Bath room
Living room
4
User

The system turns on a fan
Bath room
Living room
4
User

The humidity in the bathroom drops
Bath room
Living room
4
User

The system turns off the fan
Bath room
Living room
4
User

Bath room
Living room
4
User
Admin

Admin configures threshold values
Bath room
Living room
4
User
Admin

Admin searches new devices
Bath room
Living room
4
User
Admin

The system informs about
found devices
Bath room
Living room
4
User
Admin

What is WAMP
• It stands for "Web Application Messaging Protocol"
5

What is WAMP
• Is intended for communication between software components in
near real-time
5

What is WAMP
near real-time
• Provides two mechanisms:
• Publish/Subscribe (PubSub)
• Remote Procedure Calls (RPC)
5

What is WAMP
near real-time
• Bi-directional, ordered
5

What is WAMP
near real-time
• Bi-directional, ordered
• For communication between clients is required WAMP-router
5

Why WAMP?
• Open specification
6

Why WAMP?
• Thoughtful Publish / Subscribe
6

Why WAMP?
• Thoughtful Remote Procedure Calls
6

Why WAMP?
• Message routing
6

Why WAMP?
• Message routing
• Cross-platform
6

Why WAMP?
• Message routing
• Cross-platform
• There are implementations in more than 15 languages
6

WAMP alternatives
7
Technology PubSub RPC
Routed
RPC
Web native
Cross
Language
Open
Standard
WAMP ✔ ✔ ✔ ✔ ✔ ✔
AJAX - ✔ - ✔ ✔ -
AMQP ✔ (✔) - - ✔ ✔
Java JMS ✔ - - - - ✔
JSON-RPC - ✔ - ✔ ✔ ✔
MQTT ✔ - - - ✔ ✔
REST - ✔ - ✔ ✔ -
SOAP - ✔ - ✔ ✔ ✔
socket.io ✔ - - ✔ - -
XMPP ✔ - - ✔ ✔ ✔
ZMQ ✔ - - - ✔ -

WAMP architecture
10
Client.
Can provide
or consume
resources such as
subscriptions and RPC.

WAMP architecture
11
Router.
Deals with
authorization and
routing
messages
between clients.

WAMP architecture
12
Peer.
Can be a client
or a router,
must have
at least one role.

WAMP architecture
13
Role.
Defines
features of the node.
There are 4 client roles
and 2 roles of the router.

WAMP architecture
14
Session.
It is necessary for
identification and
authorization of clients,
announce the roles
of nodes.

WAMP architecture
15
Realm.
domain, namespace,
single space
within which
resources
and message routing
takes place.

WAMP architecture
16
Transport.
You can use:
Websocket, Raw TCP,
Unix sockets and even
HTTP Long poll.

WAMP architecture
17
Serializer.
All messages are
serialized using
JSON/MessagePack/
CBOR.
message

WAMP architecture
18
Resource
identifiers, URI.
Required for
topics, procedures
and errors.
For example:
Topic: «sensor.events»
RPC: «discover.devices»
message

Basic WAMP concepts
• Client. Can provide or consume resources, such as subscriptions and RPC.
• Router. Deals with authorization and message routing between clients.
• Peer. Can be a client or a router, must have at least one role.
• Role. Determines the capabilities of the node. There are 4 client roles and 2 router roles.
• Session. It is necessary to identify and authorize clients, announce the role of nodes.
• Realm. Domain, namespace, within which messages are routed.
• Transport. You can use: WebSockets, Raw TCP, Unix sockets and even HTTP Long poll.
• Serializer. Messages are serialized using JSON / MessagePack / CBOR / etc.
• Resource identifiers, URIs. Required for topics, procedures and errors.
19

WAMP: Features
• Basic profile: basic features of RPC and PubSub
20

WAMP: Features
• Advanced profile:
20

WAMP: Features
• Advanced profile:
• Challenge-Response Authentication
• Progressive call results, call timeout, call canceling
• Publication / call trust levels
• Caller / publisher identification
• Pattern based registration / subscription
• Shared registration, sharded registration / subscription
• Event history
• Meta API & Procedure / topic reflection
20

WAMP: session establishment
21
Client Router
Realm

21
HELLO
Client Router
Realm
[1, "homeapp", { "roles": { "publisher": {}, "subscriber": {} } }]

21
HELLO
WELCOME
Client Router
Realm
[2, 9129137332, { "roles": { "broker": {} } }]

21
HELLO
WELCOME
ABORT
Client Router
Realm
[3, {"message": "The realm does not exist."}, "wamp.error.no_such_realm"]

21
HELLO
WELCOME
GOODBYE (init by client)
ABORT
Client Router
Realm
[6, {"message": "The host is shutting down now."}, "wamp.error.system_shutdown"]

21
HELLO
WELCOME
ABORT
GOODBYE
Client Router
Realm
[6, {}, "wamp.error.goodbye_and_out"]

21
HELLO
WELCOME
ABORT
GOODBYE
GOODBYE (init by router)
Client Router
Realm
[6, {}, "wamp.error.close_realm"]

21
HELLO
WELCOME
ABORT
GOODBYE
GOODBYE
Client Router
Realm
[6, {}, "wamp.error.goodbye_and_out"]

21
HELLO
WELCOME
ABORT
GOODBYE
GOODBYE
Client Router
Realm

WAMP: Publish / Subscribe
22
Publisher Broker Subscriber
Realm

22
SUBSCRIBE
Realm
[32, 713845233, {}, "sensor.events"]

22
SUBSCRIBE
SUBSCRIBED
Realm
[33, 713845233, 5512315355]

22
SUBSCRIBE
SUBSCRIBED
ERROR
Realm
[8, 32, 713845233, {}, "wamp.error.not_authorized"]

22
SUBSCRIBE
SUBSCRIBED
ERROR
PUBLISH
Realm
[16, 239714735, {}, "sensor.events", [], {"sensorId ": 25, "temp": 27}]

22
SUBSCRIBE
SUBSCRIBED
ERROR
PUBLISH
PUBLISHED
Realm
[17, 239714735, 4429313566]

22
SUBSCRIBE
SUBSCRIBED
ERROR
PUBLISH
PUBLISHED
ERROR
Realm
[8, 16, 239714735, {}, "wamp.error.not_authorized"]

22
SUBSCRIBE
SUBSCRIBED
ERROR
PUBLISH
PUBLISHED
ERROR
EVENT
Realm
[36, 5512315355, 4429313566, {}, [], {"sensorId ": 25, "temp": 27}]

22
SUBSCRIBE
SUBSCRIBED
UNSUBSCRIBE
ERROR
PUBLISH
PUBLISHED
ERROR
EVENT
Realm
[34, 85346237, 5512315355]

22
SUBSCRIBE
SUBSCRIBED
UNSUBSCRIBE
UNSUBSCRIBED
ERROR
PUBLISH
PUBLISHED
ERROR
EVENT
Realm
[35, 85346237]

22
SUBSCRIBE
SUBSCRIBED
UNSUBSCRIBE
UNSUBSCRIBED
ERROR
ERROR
PUBLISH
PUBLISHED
ERROR
EVENT
Realm
[8, 34, 85346237, {}, "wamp.error.no_such_subscription"]

22
SUBSCRIBE
SUBSCRIBED
UNSUBSCRIBE
UNSUBSCRIBED
ERROR
ERROR
PUBLISH
PUBLISHED
ERROR
EVENT
Realm

WAMP: Remote Procedure Calls
23
Caller Dealer Callee
Realm

23
REGISTERRealm
[64, 25349185, {}, "dcs.fan.control"]

23
REGISTER
REGISTERED
Realm
[65, 25349185, 2103333224]

23
REGISTER
REGISTERED
ERROR
Realm
[8, 64, 25349185, {}, "wamp.error.procedure_already_exists"]

23
REGISTER
REGISTERED
ERROR
CALL
Realm
[48, 7814135, {}, "dcs.fan.control", [true]]

23
REGISTER
REGISTERED
ERROR
CALL
INVOCATION
Realm
[68, 6131533, 9823528, {}, [true]]

23
REGISTER
REGISTERED
ERROR
CALL
INVOCATION
YIELD
Realm
[70, 6131533, {}]

23
REGISTER
REGISTERED
ERROR
CALL
RESULT
INVOCATION
YIELD
Realm
[50, 7814135, {}]

23
REGISTER
REGISTERED
ERROR
CALL
RESULT
INVOCATION
YIELD
ERROR
Realm
[8, 68, 6131533, {}, "homeapp.error.hardware_not_found"]

23
REGISTER
REGISTERED
ERROR
CALL
RESULT
INVOCATION
YIELD
ERROR
ERROR
Realm
[8, 48, 7814135, {}, "homeapp.error.hardware_not_found"]

23
REGISTER
REGISTERED
UNREGISTER
ERROR
CALL
RESULT
INVOCATION
YIELD
ERROR
ERROR
Realm
[66, 788923562, 2103333224]

23
REGISTER
REGISTERED
UNREGISTER
UNREGISTERED
ERROR
CALL
RESULT
INVOCATION
YIELD
ERROR
ERROR
Realm
[67, 788923562]

23
REGISTER
REGISTERED
UNREGISTER
UNREGISTERED
ERROR
ERROR
CALL
RESULT
INVOCATION
YIELD
ERROR
ERROR
Realm
[8, 66, 788923562, {}, "wamp.error.no_such_registration"]

23
REGISTER
REGISTERED
UNREGISTER
UNREGISTERED
ERROR
ERROR
CALL
RESULT
INVOCATION
YIELD
ERROR
ERROR
Realm

WAMP as a basis for SOA applications
24

Distributed architecture
24

Loose coupling of components
24

A simple and intuitive interface for interaction
24

Unified Routing
24

Unified Routing
Independence from programming language
24

Unified Routing
Re-usability of components
24

Unified Routing
Re-usability of components
WAMP satisfies the requirements of SOA
24

WAMP as ESB
25
WAMP
Router
Service
Providers
Service
Providers
Service
Providers
Service
Providers
Service
Providers
Consumers
Consumers
Consumers
Consumers
Consumers
call RPC
Subscribe to topics
Register RPC
Publish events
Subscribe to topics
Publish events
Сервер
Сервер
Server

WAMP in heterogeneous systems
26
WAMP
Router
File
uploader
(python)
Web
Client
(js)
Backoffice
Client
(c#)
Video
processing
(c++)
Image
processing
(go)
Logger
(ruby)
External
API
Gateway
Client
REST API
(nodejs)
Mobile
Client
(swift)
Backoffice
REST API
(Java)
DB
Mobile
Client
(java)
Realm
Realm
Realm
Сервер
Сервер
Server

WAMP Application fields
27
Distributed applications with a large number of services

27
Smart homes, Smart cities, Internet of things in General

27
Systems with long-running processes

27
Easy modern alternative to XMPP, XMLRPC

27
Easy modern alternative to XMPP, XMLRPC
Network multiplayer games

WAMP: real projects
• Kadecot API Server. Sony Computer Science Laboratories
28

WAMP: real projects
• ParaView. Parallel Data Analysis and Visualization Application, Kitware
28

WAMP: real projects
• Record.Evolution. Data-warehouse and Business Intelligence
28

WAMP: real projects
• SmartME smart city project. MDS Lab at the University of Messina
28

WAMP: real projects
• SmartME smart city project. MDS Lab at the University of Messina
• ButterflyDNS. Web UI frontend for BIND DLZ
• Gollard VISION Server Management Subsystem
28

Health monitor
Nginx
Server Management Subsystem
29
WAMP
Router
Admin
App
(c#)
Shaman
RPC: set.network
RPC: get.network Publish: status.change
RPC: get.status
RPC: get.time
RPC: set.time

Loowy: WAMP client in Lua
30
• Works in Lua/Luajit environment on top of lua-ev

30
• Supports JSON and Msgpack serializers

30
• Supports a number of features from Advanced Profile

30
• Automatic reconnection and re-subscription

30
• Automatic reconnection and re-subscription
• TODO: Raw socket transport
• TODO: Meta API
• TODO: Challenge-Response Authorization (WAMP-CRA)
• TODO: work on the basis of coroutines

Loowy: api calls examples
31
client = loowy.new("ws://localhost/ws/", { realm = "gv" })
client:publish('sensor.events', { sensorId = 12, temperature = 27})
client:subscribe('sensor.events', function(event)
-- process received event
end)
client:call('dcs.fan.control', true, {
onSuccess = function()
-- fan turned on successfully
end,
onError = function(err)
-- error occurred
end
})

Loowy: Technology stack
32
• Lua 5.1+, Luajit 2.0
• lua-websockets (which uses luasocket under the hood)
• lua-ev
• lua-rapidjson
• lua-MessagePack (optional)

Wiola: WAMP router in Lua
• Works on top of Nginx/Lua module
33

• Uses Redis for runtime storage
33

33

• TODO: Raw socket transport
• TODO: Meta API
• TODO: lua-resty-postgres
33

Wiola: Technology stack
34
• Nginx or OpenResty, simply put lua-nginx-module
• lua-resty-websocket
• lua-resty-redis
• Redis Server
• lua-rapidjson
• lua-resty-hmac (optional, is required for WAMP-CRA)
• lua-MessagePack (optional)

Wiola: Nginx config
35
location /ws/ {
set $wiola_socket_timeout 10;
set $wiola_max_payload_len 65535;
lua_check_client_abort on;
header_filter_by_lua_file /lib/wiola/headers.lua;
content_by_lua_file /lib/wiola/handler.lua;
}

Nginx
Wiola: Common scheme
36
Web
Client
init_by_lua
headers_filter_by_lua
content_by_lua
Redis
Server

Wiola: JSON caveats and limitations
37

• cJSON
37

• cJSON
• you cannot specify how to serialize empty tables
37

• cJSON
• lua-resty-libcjson
37

• cJSON
• not able to work with large numbers (int > 32bit)
37

• cJSON
lua-rapidjson
37

• cJSON
lua-rapidjson
• So far, no new problems have been found :)
37

Wiola: Runtime storage options
38

38
• Global variables in memory

38
• are created every time on request handler

38
• problems with concurrent requests

38
• Module variables loaded in init_by_lua

38
• You can read in all workers

38
• But the write, thanks to CoW, will create a copy of the module for the VM

38
• ngx.shared.DICT

38
• ngx.shared.DICT
• The size of each dictionary is specified in the config file and can not be changed

38
• ngx.shared.DICT
• The size of each dictionary is specified in the config file and can not be changed
External storage: Redis, Memcache, PostgreSQL, MySQL and others

Wiola: Особенности. lua_code_cache
39

39
• lua_code_cache off;

39
• it is convenient to debug the code

39
• every require() is executed again in each request

39
• Even the modules loaded into init_by_lua are re-executed

39
• lua_code_cache on;

39
• for production environment

39
• each require() is executed once and is cached

39
• each require() is executed once and is cached
• whenever you change the code, you need to reload nginx

Wiola: Publication in luarocks, opm
40

40
• luarocks: there is one part of
the dependencies, there is no
other:

40
other:
- lua-resty-websocket

40
other:
- lua-resty-redis

40
other:
- lua-resty-redis
✓ rapidjson

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack
• opm (OpenResty Package
Manager): accurate to
"reverse":

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack
"reverse":
✓ lua-resty-websocket

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack
"reverse":
✓ lua-resty-redis

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack
"reverse":
✓ lua-resty-redis
- rapidjson

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack
"reverse":
✓ lua-resty-redis
- rapidjson
✓ lua-resty-hmac

40
other:
- lua-resty-redis
✓ rapidjson
✓ lua-resty-hmac
✓ lua-messagepack
"reverse":
✓ lua-resty-redis
- rapidjson
✓ lua-resty-hmac
- lua-messagepack

Useful links about WAMP
• WAMP main site: http://wamp-proto.org/
• WAMP spec on github: github.com/wamp-proto/wamp-proto
• Mailing list: https://groups.google.com/forum/#!forum/wampws
• Wiola: https://github.com/KSDaemon/wiola
• Loowy: https://github.com/KSDaemon/Loowy
• Wampy.js: https://github.com/KSDaemon/wampy.js
• Wiola docker image: https://hub.docker.com/r/421p/wiola/
• Crossbar Router: https://crossbar.io/ и https://demo.crossbar.io/
41

WAMP: Conclusion
• A simple and open Protocol
42

WAMP: Conclusion
• RPC and PubSub in one transport/protocol
42

WAMP: Conclusion
• Any transport and serialization
42

WAMP: Conclusion
• Allows you to build distributed applications with loosely coupled
components
42

WAMP: Conclusion
components
• There are implementations in many languages
42

WAMP: Conclusion
components
• Open source
42

WAMP: Conclusion
components
• Open source
• A small but friendly community
42

WAMP: Conclusion
• Useful architectural solution
43

WAMP: Conclusion
• A great option for your IoT project! (IMHO: better than MQTT)
43

WAMP: Conclusion
• Try WAMP:
43

WAMP: Conclusion
• Try WAMP:
• Already have nginx/openresty? — try Wiola (bit.ly/wiola-router)
43

WAMP: Conclusion
• Try WAMP:
• Are you using a docker?
43

WAMP: Conclusion
• Try WAMP:
• docker pull 421p/wiola
43

WAMP: Conclusion
• Try WAMP:
• docker pull crossbario/crossbar
43

WAMP: Conclusion
• Try WAMP:
• docker pull crossbario/crossbar
• WAMP client for every taste and color (wamp-proto.org/implementations/)
43

Thank you for attention!
And your questions :)
44
Konstantin Burkalev
• Mail: kostik@ksdaemon.ru
• Twitter: @KSDaemon
• GitHub: KSDaemon
• Blog: blog.ksdaemon.ru
• SDCast: sdcast.ksdaemon.ru

WAMP as a platform for composite SOA applications and its implementation on Lua

More Related Content

What's hot

Similar to WAMP as a platform for composite SOA applications and its implementation on Lua

Recently uploaded

WAMP as a platform for composite SOA applications and its implementation on Lua