Messaging, interoperability and log aggregation - a new framework
Xmpp intro 2014
1. Y O U S E E - M A R C H 2 0 1 4
S T E F F E N L A R S E N
!
!
E M A I L / X M P P :
S L A R S E N @ B R A I N T R U S T. D K
!
T W I T T E R : @ Z O O L D K
X M P P
I N T R O
2. T H I S P R E S E N TAT I O N
• Intro and presentation of the protocol
• What is it?, Usage
• Examples
• Architecture and the basics
• Advanced Topics and loose talk
• Extensions
• Plugins and components
3. X M P P I N T R O
• What is it?
• XMPP - eXtensible Message and Presence Protocol
• Real time message system with presence
• Routes small snipplets of XML - called stanzas
• An Open Standard (RFC 6120 / 6121)
• Earlier known as Jabber - 1998 (now Cisco trademark)
4. X M P P - W H AT & W H O ?
• Who uses it? Most of you use it every day..
• Google (gtalk, hangouts (not federated), android push messages (GCM))
• Cisco (webex)
• Apple iMessage and other
• Facebook chat
• whatsapp (FB purchased an open standard protocol for $19 billion!) heh!
• Viber
• Chesspark (chessgame)
• Eve (Sci-fi game, in-game chat)
• Even Micro$oft - their msg system
5. M Y X M P P I N V O LV E M E N T
• Developed XMPP since 2007 (Ms.Thesis, pidgin plugin doing video/
VOIP, later gtalk)
• Member of XMPP Standard Foundation (XSF)
• Now also editor for new extensions (XEPs)
• Done XMPP design and impl. for:
• Open Source: Tigase, Strophe-plugins, XMPPConsole etc.
• John Doe company. etc. for Nordija (Set-top-boxes and over-the-top TV and IPTV)
• Thrane&Thrane / Cobham (Maritime Satellite Communication project - Maritime rescue service)
• Danske Spil / BetWare (Game software for Bingo etc)
• Livestation in-video group chat
6. X M P P A N D Y O U S E E
• Use Presence for watching the number of concurrent clients
and limit it
• Send out notifications for specific users, devices and versions
of software
• Broadcast important notifications
• Second Screen stuff
• remote PVR
• Exchange bandwidth? and set limits
7. X M P P B A S I C S
• What can you do with it?
• Everything you imagine (almost)!
• More than just a chat, its ALL about real-time,
presence and routing messages!
• Real-time web, push it!
8. X M P P B A S I C S
• Basic Services and applications
• Presence (know who is online and what they do!): Taxi apps (like cabulous etc)
• Contact / Buddy lists
• Messaging (1-1 or group chat), customer service: Yammer, Google, JIVE
• Push Notifications (individual or publish/subscribe): google GCM push, buddycloud, energy trading
software
• Service discovery and capability advertisement (know device type in realtime)
• Geolocation and notifications
• Video/Voice over IP (VOIP) - WebRTC IS coming!
• Whiteboarding and collaborative text editing etc.
9. X M P P B A S I C S - W H Y X M P P ?
• XMPP is an Open Standard (like HTTP) - XSF and the council does the work openly.
(thus all clients, servers and libs work together)
• Routing and presence out-of-the box!
• Secure - channel encryption and authentication
• Proven tech. Almost 15 years of dev.
• Extensibility!.. We have above 300 extensions to the XMPP core.
• Its a application protocol that is transport agnostic - use normal TCP, HTTP (old
school polling), BOSH (long polling), or websockets.
• Huge community and open standard community that works!
• Decentralised. no stand-alone silos (hello whatsapp and FB!..)
11. X M P P A R C H I T E C T U R E
• XMPP can consist of many components:
• Servers (many through federation) - your own or public ones! federate!
• Plugins
• Components
• Clients (typically many..)
• Proxies
• Database, LDAP, SIP and other systems (gateways)
12. X M P P A D D R E S S I N G & R O U T I N G
• Standard routing through JID
• full JID: <id>@<domain>/<resource> (many logins/clients pr. user)
• bare JID: <id>@<domain> (autom. resource by server)
• Anonymous login (typically used in multiuser chat and customer support
systems)
• Domain routing like normal mail (smtp).
13. X M P P C O N N E C T I O N L I F E C Y C L E -
S I M P L E
• Connect with login or anonymous
• Get resource (or supply it when logging in)
• Send presence and capabilities to sever that you are
online
• Get roster (friends list)
14. X M P P C O N N E C T I O N L I F E C Y C L E
• Client initiates a connection (typ. TCP) to the XMPP server over secure socket (SSL / TLS)
• Opens an XML <stream> for a session
• Negotiates stream features and SASL negotiation (Simple Authentication and Security Layer)
• Typ. auth mechanisms: PLAIN, DIGEST-MD5, SCAM, EXTERNAL etc.
• The client get bound to a resource, if none given by client
• The client and server can now communicate with Stanza primitives
• The clients sends out a presence Stanza
• The client get his roster and does his stuff
• The client ends session when done with his stuff
• </stream> tag is send and TCP connection is closed again.
15. C O M M U N I C AT I O N P R I M I T I V E S
• Stanzas (xml snipplets that flow async.)
• <Presence/>
• <Message/>
• <IQ/> (Information/Query)
16. S TA N Z A S : P R E S E N C E
• Sets and shows the availability of the entity that is connected.
• chat
• away
• dnd (do not disturb)
• Presence for the system and the subscribers of the entity (e.g.
buddylist)
• Specialized Presence extension (PEP) shows what music you
are playing.. (see adium and pidgin for example)
17. S TA N Z A S : P R E S E N C E
// initial simple presence
<presence/>
!
// sending away to subscribers
<presence>
<show>away</show>
</presence>
<presence>
!
<show>dnd</show>
<status>I am working.. </status>
</presence>
!
// before logging off..
<presence type='unavailable'/>
18. S TA N Z A S : P R E S E N C E
// direct presence
!
<presence from='juliet@example.com balcony’
to='romeo@example.net'/>
!
// Capabilities.. version, voip etc.
!
<presence from='romeo@example.net'>
<c xmlns='http://jabber.org/protocol/caps'
hash='sha-1'
node='http://psi-im.org'
ver='q07IKJEyjvHSyhy//CH0CxmKi8w='/>
</presence>
19. S TA N Z A S : M E S S A G E
• Using it to communicate with other clients
• TO attribute
• full JID: direct messaging to specific entity (e.g. device/computer).
• bare JID: will route to one or more of the connected resources - depends of status,
presence, message type etc.
• Type attribute (chat, normal, groupchat, headline etc) -
presentation and routing
• Body Element - the message
20. S TA N Z A S : M E S S A G E
// normal message (let the server decide where to route the message)
<message
from='juliet@example.com/balcony'
id='ktx72v49'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Hi romeo where art thou?</body>
</message>
// direct message (in cases where you want to hit specific client / devices (yousee eg.)
<message
from='juliet@example.com/balcony'
id='ktx72v50'
to='romeo@example.net/thecastle'
type='chat'
xml:lang='en'>
<body>will you come and visit?</body>
</message>
21. S TA N Z A S : I Q
• A more request / response model. For getting
information and query the server / entities.
• Request: GET / SET
• Response: RESULT / ERROR
• Using it for getting roster, ad-hoc command and
statistics for server (if admin).. etc.
22. S TA N Z A S : I Q
Example 1: User requests current roster from server
!
UC: <iq from='romeo@example.net/orchard'
id='hf61v3n7'
type='get'>
<query xmlns='jabber:iq:roster'/>
</iq>
!
Example 2: User receives roster from server
!
US: <iq id='hf61v3n7'
to='romeo@example.net/orchard'
type='result'>
<query xmlns='jabber:iq:roster'>
<item jid='juliet@example.com'
name='Juliet'
subscription='both'>
<group>Friends</group>
</item>
<item jid='benvolio@example.org'
name='Benvolio'
subscription='to'/>
<item jid='mercutio@example.org'
name='Mercutio'
subscription='from'/>
</query>
</iq>
23. I I ) A D VA N C E D T O P I C S
• Extensions
• Components, plugins and extensions
• Internal routing and modifications via. plugins
24. X M P P E X T E N S I O N S
• XSF defines a set of XMPP Extension Protocols (XEPs).
• > 300 at the moment
• Jingle (Gtalk VOIP)
• MUC (multiuser chat / group chat),
• file transfer
• publish/subscribe
• XMPP over Web sockets
• XMPP over HTTP - BOSH (long polling)
• XMPP over serverless messaging (Bonjour)
• Compression algos.
• Dataforms and ad-hoc commands
• IoT extensions for data collection etc.
• etc.
25. E X T E N S I O N S A N D N E W F O C U S A R E A S
• The following areas spins of a lot of extensions at the moment:
• WebRTC - federation and discovery
• UPnP cloud - UPnP is going use XMPP standard for interconnecting
devices!
• Internet of Things (IoT ) / M2M (IoT with security and federation?! - wow)
• Home devices
• Second Screen
• Big data / data collecting (for data gathering devices like
thermostats, motor services etc)
27. A D VA N C E D : P L U G I N S
• Located on the server (bound to server, not that portable)
• External Auth: either plain db, LDAP, REST or others.
• Nice if you already have your users and just want to enable
real-time push or other features from XMPP. Use your own
DB or REST call (like yspro)
• Make your own routing strategy if it does not exist yet
• Filter messages
• Collect statistics
28. A D VA N C E D : C O M P O N E N T S
• Server is easily extensible with components (typ. business logic)
• External - connects to server or vice versa
• Handshakes and auth. with the server to get trusted
• Its a protocol - hence not bound to server and thereby portable
• Can be written in any language - just implement the simple protocol
• Can interact on the whole domain or subdomain
• Can alter the stanzas: to and from fields
• Can send out new Stanzas etc.
29. A D VA N C E D : C O M P O N E N T S
!
• Examples:
• Gateways to skype, msn, ICQ, AOL, IRC.
• Amazon SQS for messaging queueing - send messages to your users through other
transports
• REST endpoints
• Server starts the communication. This is done over another transport, internal satellite
(thrane & thrane) and sends a go when the client can establish a connection.
• Make your client seem present without having a session yet (strange but usable). Used
for satellite comm because of latency.
• YouSee.. disconnect after a number of sessions.
30. X M P P V S D I A L ( C H R O M E C A S T )
• DIAL protocol (by Netflix) is a small subset of XMPP and can’t really be compared (i’ll do it
anyway) :-) - used for multiscreen / second screen only!
• DIAL is a simple REST service which is fine! - can do discovery of a DIAL enabled device (multicast
to google) and startup a browser and a URL - thats it!
• You have to sign registry to control the 1’st screen app in DIAL
• SILOED! - DO a multicast to google (dependent on 3’rd party vendor)
• DIAL lacks security
• DIAL offers no form of pairing and authentication
• Needs to be on the same network
• DIAL have no means of presence
• Chromecast implementation -
• The registry is entirely up to Google. Sign with the devil!
• Offers Netflix and YouTube on a stick - works.
31. < T H E E N D / > . .
• Contact me at
• Email/XMPP: slarsen@braintrust.dk
• Twitter: @zooldk
• LinkedIn: http://dk.linkedin.com/in/zooldk
!
• Illustrations by Adrian Teh (Too lazy to draw my self..)
32. L I N K S U M M A RY
• XMPP standard foundation: http://xmpp.org
• Server list: http://xmpp.org/xmpp-software/servers/
• Client list: http://xmpp.org/xmpp-software/clients/
• Libraries: http://xmpp.org/xmpp-software/libraries/