This was the very first presentation about VerneMQ. A high-performance, distributed MQTT message broker developed in Erlang. Presented at the Paris Erlang User Group. The presentation gives a very short introduction to the MQTT protocol, describes the VerneMQ system architecture, and goes into some details about the implemented clustering and extensibility features.

1. erlang.paris Meetup / June 29th 2015, Paris / André Graf (@der_graf)
VerneMQ - A distributed MQTT message broker
developed in Erlang

2. Intro
- André Graf, Co-Founder at Erl.io (2012)
- Programming in Erlang since 2008
- Basel Switzerland
@der_graf

3. Simple Publish/Subscribe protocol
over TCP/IP that utilizes
a centralized message broker
MQTT

4. Different use cases…
MQTT

5. https://www.flickr.com/mmmavocado/

6. https://www.flickr.com/photos/christianhaugen

7. https://www.flickr.com/photos/janitors

8. Different use cases…
...but similar requirements.
MQTT

9. constrained devices in
constrained environments
MQTT

10. | different QoS Levels
| small header sizes (min 2B)
| durable sessions
| last will testament
MQTT

11. Some MQTT Facts
- Current Spec Version 3.1.1
- open OASIS Standard since 2014
- Invented 1998 at IBM and Arcom

12. VerneMQ Architecture from a
bird’s eye perspective

13. Erlang VM
ninenines/ranch
ninenines/cowboy
vmq_server
vmq_plugin
helium/eleveldb
helium/plumtree
uwiger/jobsvmq_commons
feuerlabs/
exometer_core
vmq_passwd
vmq_acl
vmq_bridge
vmq_systree
vmq_snmp
vmq_graphite
larshesel/
vmq_elixir

14. erlio/vernemq release project
basho/nodepackage
basho/cuttlefish
basho/clique
rebar/rebar3

15. vmq_reg_trie
<<gen_server>>
3 processes / MQTT connection
vmq_server
vmq_session
<<gen_fsm>>
vmq_ranch
<<ranch_protocol>>
vmq_queue
<<gen_fsm>>start_link start_link
vmq_reg
<<gen_server>>
vmq_reg_sup
<<supervisor>>
vmq_reg_trie
<<gen_server>>
monitor
ETSETS

16. vmq_reg_trie
<<gen_server>>
vmq_server
vmq_session
<<gen_fsm>>
vmq_ranch
<<ranch_protocol>> sync
vmq_reg
<<gen_server>>
vmq_reg_sup
<<supervisor>>
vmq_reg_trie
<<gen_server>>
ETS
vmq_session
<<gen_fsm>>
vmq_ranch
<<ranch_protocol>>
vmq_queue
<<gen_fsm>>async batch/async
match
fold
MQTT publish example
TCP Conn Client A
TCP Conn Client B
ETS
PUB
/a/b/c
SUB
/a/b/+

17. Clustering VerneMQ

18. Distribution
What is ACTIVELY distributed?
| Subscriber Data
| Online Messages
| Cluster Status Information
| ...

19. Distribution
What is NOT distributed?
| offline messages (future)

20. Distributing Subscriber Data
- Initially using Mnesia
- Move to Plumtree/LevelDB
Paradigm Shift:
Consistent -> eventual Consistent

21. Distributing online Messages
- Initially over Erlang Distribution layer
- Move to own communication layer
- buffering in case of ‘nodedown’s
- batching requests for higher throughput

22. Extending VerneMQ

23. erlio/vmq_plugin
%% iterates over all Plugins serving the Hook 'HookName'
vmq_plugin:all(HookName :: atom(), HookArgs :: [any()]).
%% iterates over all Plugins serving the Hook 'HookName'
%% until one returns 'ok' | {'ok', any()}
vmq_plugin:all_till_ok(HookName :: atom(), HookArgs :: [any()]).
%% calls the top priority Plugin that serves the Hook 'HookName'
vmq_plugin:only(HookName :: atom(), HookArgs :: [any()]).

24. erlio/vmq_plugin
%% enabling an OTP Application Plugin
vmq_plugin_mgr:enable_plugin(Application :: atom()).
%% in Application.app.src
{env, [
{vmq_plugin_hooks, [
{HookName, Module, Function, Arity, []}
...
]}
]}

25. erlio/vmq_plugin
Available Hooks:
- auth_on_register/5
- auth_on_publish/6
- auth_on_subscribe/3
- on_register/3
- on_publish/6
- on_subscribe/3
- on_deliver/6 (planned)

26. erlio/vmq_plugin
Available Hooks:
- msg_store_write_sync/2
- msg_store_write_async/2
- msg_store_delete_sync/1
- msg_store_delete_async/1
- msg_store_fold/2
- msg_store_read/1

27. erlio/vmq_plugin
Checkout vmq_plugin and the Demo Plugin App
https://github.com/erlio/vmq_plugin
https://github.com/dergraf/vernemq_demo_plugin

28. To Sum Up
- MQTT is a great Protocol for various use
cases (monitoring/metering/mobile/..)
- VerneMQ brings scalability and powerful
extensibility to the MQTT world.
- Erl.io provides support and development
services for VerneMQ

29. http://verne.mq | http://erl.io
@der_graf | @vernemq | @erlio_basel
Merci Beaucoup!
Je suis content de répondre a vos questions.

30. References
https://verne.mq
http://mqtt.org
https://github.com/erlio/vernemq
https://github.com/erlio/vmq_plugin
https://github.com/dergraf/vernemq_demo_plugin
https://github.com/helium/plumtree
http://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf
https://groups.google.com/forum/#!topic/mqtt/c1dlCdA5eOc