Academy #2
4th November 2015
Mickaël Rémond, @mickael
Questions
ejabberd questions
• What are backends ? What backends are available in ejabberd and how do they work ?
XMPP questions
• How can I learn and experiment with XMPP ?
• What are message carbons and how do they work ?
• Why do avatars / carbons not work in MUC rooms ?
ejabberd: Routing messages in a statefull world
ejabberd leverage both stateless and statefull features:
• Stateless:
• ejabberd is an XMPP server: Its goal is to route XMPP packets between JID.
• Statefull:
• In most case ejabberd depends on data (user base, contact list, …) or produce data (message archive, ...)
To deploy an ejabberd that is as stateless as possible, ejabberd leverages backends.
What are ejabberd backends ?
• Backends are pluggable modules you can configure to define where you would like to store part or all of your
data.
• Backends provide the data to the feature modules of ejabberd.
• They can be read-write or read-only if you do not need some of the ejabberd associated features:
• For example, if you handle user management elsewhere in your infrastructure, you can use a user back-
end that can only authenticate users but not create them.
Available backends
• Modules that need data backends generally support:
• Internal database: Mnesia
• RDBMS: MySQL / MariaDB, Postgres, SQLServer / Azure SQL, SQLite
• NOSQL: Riak
• Some of them support additional backends (i.e authentication, shared roster):
• LDAP / Active Directory / Azure Directory
• ejabberd SaaS provides ReST API description to implement HTTPS backend for
• User authentication,
• Contact list (rosters),
• Message archiving
• Custom backends in ejabberd-contribs: i.e. HTTP authentication.
Example configuration
You generally need a global parameter for RDBMS:
• Example of MySQL connection:
odbc_type: mysql
odbc_server: "server.company.com"
odbc_port: 3306 # the default
odbc_database: "mydb"
odbc_username: "user1"
odbc_password: "**********"
odbc_pool_size: 5
You can then reference that back-end for authentication:
auth_method: odbc
Or in some modules configuration on a granular basis:
modules:
mod_roster: [{db_type: odbc}]
ejabberd
architecture
model
Mobile - Desktop - Web Browser
XMPP - 5222
Websocket / Bosh:
HTTP - 80
HTTPS - 443
ejabberd Instant Messaging
ejabberd cluster
...
Load balancers
XMPP - 5222
User base
(i.e. MySQL)
User calls
ejabberd backend API
select one or several backends
ejabberd service
ejabberd architecture
model
ejabberd internal database
Offline messages
Last seen
Privacy lists
Pubsub nodes
Push tokens (APNS / GCM)
…
Contact service
(i.e. webservice)
Roster calls
Message archive
(i.e. Riak)
Message
Archiving calls
…
Data backends
to integrate
with infrastructure
and scale
Data can be moved to backends
to make ejabberd more stateless
ejabberd internals: Feature modules rely on data modules -
backends
Properties of ejabberd backends
• Backend can be combined:
• Different features can use different backends to leverage different backend properties:
• Massive amount of data
• Speed
• …
• Backend are API based:
• They are developed using ejabberd API.
• It means you can write your own backend to meet special needs.
• => Selecting the right backend or deciding to write your own for the each module is part of ejabberd solution
design.
What are sessions backends ?
• We have a special other type of backend call session backend.
• Sessions are transient data that serve to locate a user connection in the the cluster when he is online.
• As a default in ejabberd Community Edition, sessions are stored in Mnesia database.
• This is not permanent data.
• However, for scalability or convenience, you can select other backends:
• Redis
• RDBMS: MySQL, MariaDB, Postgres
• ejabberd Business Edition clustering
How can I learn and experiment with XMPP ?
• Developers often get confused when dealing with XMPP because it is quite a large protocol with a lot of
extensions.
• A common approach is to learn from the client libraries. This is often a bad idea to start from here are libraries
often try to abstract XMPP protocol.
• => Starting from the library hide what is XMPP about and can limit understanding of the protocol.
• I recommend to learn XMPP at the XMPP packet / text protocol level.
• To experiment, I am often using PSI client XMPP console to check traffic, analyse packets received and send,
and craft and send my own packets.
What are Message Carbons and how do they work ?
• Message Carbons are defined in XEP-0280.
• This is a feature that is intended so that all connected clients can receive all parts of the message
conversation to avoid online clients having only a part of the messages.
• It does so by having the server
• Send copies of your own messages to your other connected clients that want to receive the copies.
• Send copies of incoming messages to your other connected resources.
• Enabled by sending XMPP IQ to explicitely enable carbons for your session.
PSI XMPP debug
session demo
Why do avatars / carbons not work in MUC rooms ?
• One-to-one chat in XMPP rely on different features set than Multi-User chat.
• Multi-User Chat is a kind of proxy that allows you to discuss indirectly with other users.
• It means that some features that works on one-to-one chat may or may not be usable, depending on the
context:
• Avatars: Avatars are available for contact that are:
• in your roster
• or that you can at least know the JID.
MUC being semi anonymous, if you are not admin, you may not be able to retrieve VCard avatars as you
do not know the user JID.
Even if you have the JID, if you use PEP based avatars, user must have accepted you so that you can
see the avatar.
• Message carbons: XEP-0280 explicitely exclude MUC messages.
See you at next

XMPP Academy #2

  • 1.
    Academy #2 4th November2015 Mickaël Rémond, @mickael
  • 2.
    Questions ejabberd questions • Whatare backends ? What backends are available in ejabberd and how do they work ? XMPP questions • How can I learn and experiment with XMPP ? • What are message carbons and how do they work ? • Why do avatars / carbons not work in MUC rooms ?
  • 3.
    ejabberd: Routing messagesin a statefull world ejabberd leverage both stateless and statefull features: • Stateless: • ejabberd is an XMPP server: Its goal is to route XMPP packets between JID. • Statefull: • In most case ejabberd depends on data (user base, contact list, …) or produce data (message archive, ...) To deploy an ejabberd that is as stateless as possible, ejabberd leverages backends.
  • 4.
    What are ejabberdbackends ? • Backends are pluggable modules you can configure to define where you would like to store part or all of your data. • Backends provide the data to the feature modules of ejabberd. • They can be read-write or read-only if you do not need some of the ejabberd associated features: • For example, if you handle user management elsewhere in your infrastructure, you can use a user back- end that can only authenticate users but not create them.
  • 5.
    Available backends • Modulesthat need data backends generally support: • Internal database: Mnesia • RDBMS: MySQL / MariaDB, Postgres, SQLServer / Azure SQL, SQLite • NOSQL: Riak • Some of them support additional backends (i.e authentication, shared roster): • LDAP / Active Directory / Azure Directory • ejabberd SaaS provides ReST API description to implement HTTPS backend for • User authentication, • Contact list (rosters), • Message archiving • Custom backends in ejabberd-contribs: i.e. HTTP authentication.
  • 6.
    Example configuration You generallyneed a global parameter for RDBMS: • Example of MySQL connection: odbc_type: mysql odbc_server: "server.company.com" odbc_port: 3306 # the default odbc_database: "mydb" odbc_username: "user1" odbc_password: "**********" odbc_pool_size: 5 You can then reference that back-end for authentication: auth_method: odbc Or in some modules configuration on a granular basis: modules: mod_roster: [{db_type: odbc}]
  • 7.
    ejabberd architecture model Mobile - Desktop- Web Browser XMPP - 5222 Websocket / Bosh: HTTP - 80 HTTPS - 443 ejabberd Instant Messaging ejabberd cluster ... Load balancers XMPP - 5222 User base (i.e. MySQL) User calls ejabberd backend API select one or several backends ejabberd service ejabberd architecture model ejabberd internal database Offline messages Last seen Privacy lists Pubsub nodes Push tokens (APNS / GCM) … Contact service (i.e. webservice) Roster calls Message archive (i.e. Riak) Message Archiving calls … Data backends to integrate with infrastructure and scale Data can be moved to backends to make ejabberd more stateless
  • 8.
    ejabberd internals: Featuremodules rely on data modules - backends
  • 9.
    Properties of ejabberdbackends • Backend can be combined: • Different features can use different backends to leverage different backend properties: • Massive amount of data • Speed • … • Backend are API based: • They are developed using ejabberd API. • It means you can write your own backend to meet special needs. • => Selecting the right backend or deciding to write your own for the each module is part of ejabberd solution design.
  • 10.
    What are sessionsbackends ? • We have a special other type of backend call session backend. • Sessions are transient data that serve to locate a user connection in the the cluster when he is online. • As a default in ejabberd Community Edition, sessions are stored in Mnesia database. • This is not permanent data. • However, for scalability or convenience, you can select other backends: • Redis • RDBMS: MySQL, MariaDB, Postgres • ejabberd Business Edition clustering
  • 11.
    How can Ilearn and experiment with XMPP ? • Developers often get confused when dealing with XMPP because it is quite a large protocol with a lot of extensions. • A common approach is to learn from the client libraries. This is often a bad idea to start from here are libraries often try to abstract XMPP protocol. • => Starting from the library hide what is XMPP about and can limit understanding of the protocol. • I recommend to learn XMPP at the XMPP packet / text protocol level. • To experiment, I am often using PSI client XMPP console to check traffic, analyse packets received and send, and craft and send my own packets.
  • 12.
    What are MessageCarbons and how do they work ? • Message Carbons are defined in XEP-0280. • This is a feature that is intended so that all connected clients can receive all parts of the message conversation to avoid online clients having only a part of the messages. • It does so by having the server • Send copies of your own messages to your other connected clients that want to receive the copies. • Send copies of incoming messages to your other connected resources. • Enabled by sending XMPP IQ to explicitely enable carbons for your session.
  • 13.
  • 14.
    Why do avatars/ carbons not work in MUC rooms ? • One-to-one chat in XMPP rely on different features set than Multi-User chat. • Multi-User Chat is a kind of proxy that allows you to discuss indirectly with other users. • It means that some features that works on one-to-one chat may or may not be usable, depending on the context: • Avatars: Avatars are available for contact that are: • in your roster • or that you can at least know the JID. MUC being semi anonymous, if you are not admin, you may not be able to retrieve VCard avatars as you do not know the user JID. Even if you have the JID, if you use PEP based avatars, user must have accepted you so that you can see the avatar. • Message carbons: XEP-0280 explicitely exclude MUC messages.
  • 15.

Editor's Notes

  • #2 TODO: Publish video recording URL. Video recording: https://youtu.be/-dqQfCpw98E