WebCamp Ukraine 2016: Instant messenger with Python. Back-end development

Instant messenger with Python
Back-end development
Viacheslav Kakovskyi
WebCamp 2016

Me!
@kakovskyi
Python Developer at SoftServe
Contributor of Atlassian HipChat — Python 2, Twisted
Maintainer of KPIdata — Python 3, asyncio
2

Agenda
● What is 'instant messenger'?
● Related projects from my experience
● Messaging protocols
● Life of messaging platform
● Lessons learned
● Summary
● Further reading
3

What is 'instant messenger'?
4

● online chat
● real-time delivery
● short messages
5

● history search
● file sharing
● mobile push notifications
● video calling
● bots and integrations
6

Related projects from my experience
● Hosted chat for teams and enterprises
● Founded in 2009 by 3 students
● 100 000+ connected users
● 100+ nodes
● REST API for integrations and bots
● Built with Python 2 and Twisted
7

Messaging protocols
Protocol is about:
● Message format
● Allowed types of messages
● Limitations
● Routine
○ How to encode data?
○ How to establish/close connection?
○ How to authenticate?
○ How to encrypt?
8

Messaging protocols
● OSCAR (1997)
● XMPP (1999)
● Skype (2003)
● WebSocket-based (2011)
● MQTT, MTProto, DHT-based, etc.
9

XMPP
● XMPP - signaling protocol
● BOSH - transport protocol
● Started from Jabber in 1999
● XML as a message format
● Stanza - basic unit in XMPP
● Types of stanzas:
○ Message
○ Presence
○ Info/Query
10

XMPP
● Extensions defined by XEPs (XMPP Extension
Protocols):
○ Bidirectional-streams Over Synchronous
HTTP (BOSH)
○ Serverless messaging
○ File transfer and etc.
11

XMPP: Establishing a connection
12
Client:
<?xml version='1.0'?>
<stream:stream to='example.com' xmlns='jabber:client'
xmlns:stream='http://etherx.jabber.org/streams'
version='1.0'>
Server:
<?xml version='1.0'?>
<stream:stream from='example.com' id='someid'
xmlns='jabber:client' xmlns:stream='http://etherx.jabber.
org/streams' version='1.0'>

XMPP: Sending a message
13
Client:
<message from='juliet@example.com' to='romeo@example.net'
xml:lang='en'>
<body>Art thou not Romeo, and a Montague?</body>
</message>
Server:
<message from='romeo@example.net' to='juliet@example.com' xml:
lang='en'>
<body>
Neither, fair saint, if either thee dislike.
</body>
</message>

XMPP: Closing a connection
14
Client:
</stream:stream>
Server:
</stream:stream>

XMPP: Pros
● Robust and standardized
● Extendable via XEPs
● Secured
● Native support of multi-sessions
● A lot of clients implementations
15

XMPP: Cons
● Overhead
○ Presence
○ Downloading the World on startup
● XML
○ Large documents
○ Expensive parsing
16

XMPP and Python
● Servers:
○ TwistedWords - good place to start
○ Tornado-based example
○ aioxmpp
○ XMPPFlask
○ Punjab - BOSH-server on Twisted
17

XMPP and Python
● Clients:
○ SleekXMPP - mature and solid
○ Slixmpp - asyncio-support
○ TwistedWords
○ Wokkel - Twisted-based
○ xmpp.py
● JS-client: Strophe.js
18

WebSocket-based solutions
● WebSocket - transport protocol
● Standardized in 2011 by W3C
● Full-duplex communication channel
● JSON as a message format
● Custom message types
19

WebSocket: Establishing a connection
20
Client:
GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
Server:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

WebSocket: Sending a message
21
Client:
{
"type": "message",
"ts": 1469563519,
"user": "kakovskyi",
"text": "Hello, @WebCamp!"
}
Server:
{
"type": "notification",
"ts": 1469563519,
"user": "WebCamp Bot",
"text": "Howdy @kakovskyi?"
}

WebSocket: Closing a connection
22
Client:
0x8
Server:
0x8

WebSocket: Pros
● Supported by majority of browsers
● Low latency
● Small bandwidth
● Easy to start development
23

WebSocket: Cons
● Needs development of signaling protocol
● Timeouts/reconnections should be additionally
handled
24

WebSocket and Python
● Servers:
○ Autobahn - Twisted and asyncio
implementations
○ aiohttp
○ Tornado
○ Flask-SocketIO
○ Flask-Sockets
25

WebSocket and Python
● Clients:
○ Autobahn
○ aiohttp
○ Tornado-based example
○ Vanilla websocket-client
● JS-client: SocketIO
26

Life of messaging platform
● Authentication
● Access control checks
● Delivery
○ Messages
○ User's presence
○ Push notifications
● History retrieval
● History search
27

● Parsing
○ Protocol
○ Message content
● Dealing with file uploads
○ Security checks
○ Thumbnails distribution
● Multi-session support
● Reconnection handling
● Rate-limiting
28

● Server keeps connections open for every client
● High amount of long-lived concurrent connections
● Multithreaded approach isn't efficient due to overhead
● Requires usage of a select implementation on backend:
○ poll
○ epoll
○ kqueue
● Usage of asynchronous Python frameworks is preferred
for high loaded solutions
29

● Authentication
○ OAuth2
○ Run encryption operations in a separate Python thread
○ Cache users identities with Redis/Memcached
● Access-control checks
○ Make the checks lightweight and cheap
○ Raise an exception when operation isn't permitted
30
EAFP: Easier to ask for forgiveness than permission

Delivery
● Make message delivery fault-tolerant
● Limit size of a message
● Filter content of messages:
○ Users like to send chars that break all the things
● Reduce presence traffic, it could be a bottleneck for large chats
● Use asynchronous broker for delivery when a user is offline
(email or push)
○ Celery
○ RQ
○ Amazon Simple Queue Service
○ Huey
31

● Push notifications
■ Vendors
● Amazon SNS
● APNS
● Google Cloud Messaging
● Firebase Cloud Messaging
■ Python tools
● PyAPNs
● Python-GCM
● Pusher
● Be careful with device registration
● Make delivery of pushes fault-tolerant
32

History retrieval
● Return last messages for every chat instantly
○ Use double writes
■ In-memory queue only for last messages
■ Persistent storage for all the things
● Majority of history retrievals is for the last days
○ Let's optimize the case
● Index messages by date
33

History search
● ElasticSearch is the default solution for full-
text search
● @a_soldatenko: What is the best full text
search engine for Python?
● Add timing for search requests
34

Parsing
● Protocol
○ Avoid to use Pure Python parsers
■ ujson
■ lxml
○ Run benchmarks against your typical cases
● Message content
○ Be careful with regular expressions
■ re2
■ pyre2
○ Alternative parsers in Python
35

Dealing with file uploads
● Security checks
○ File upload vulnerabilities
○ Image upload
■ Decompression bomb
■ Other vulnerabilities with Pillow
○ Amazon S3 as file storage
■ boto
■ aiobotocore
■ botornado
● Thumbnails distribution
○ Delegate that to S3
○ Requested by a client even if not needed
36

● Multi-session support
○ Set expiration time
○ Be ready to handle up to 4x sessions per user simultaneously
■ Desktop
■ Mobile
■ Tablet
■ Laptop
● Reconnection handling
○ Spin a proxy layer between messaging server and clients
● Rate-limiting
○ Limit amount of operations per user/group for heavy stuff
○ Leaky bucket
○ Throttling
37

Lessons learned
● Bursty traffic
○ Load testing is a must, but not always enough
■ Locust
■ Yandex Tank
● Reconnect storm could be a big deal
○ We should handle that on platform and client-side
● AWS issues make bad customers experience
○ Put nodes in Multi-AZ
38

Lessons learned
● Incidents prevention is cheaper than resolution
○ Grab stats and metrics about your services and
storages
■ Redis for per-chat stats
■ StatsD
■ Grafana
○ Be notified when something starts going wrong
■ Elastalert
■ Monit
■ DataDog
39

Lessons learned
● Don't stick with one language/stack
○ Python is great, but for some cases Go, Ruby or
PHP are more suitable from product side
○ Avoid business logic duplication in several repos,
spin a service and just call the endpoint
● Releasing new features only for certain groups makes
product management easier
○ LaunchDarkly
40

Lessons learned
● Don’t F**k the Customer
○ Provide unit/integration tests with every PR
○ Have development environment same as prod
○ Have staging environment same as prod
○ Make deployments fast
○ Rollback faster
○ Have a fallback plan
41

Summary
● Select a messaging protocol which aligns with your needs
● WebSocket + JSON could be the thing for new projects
● Usage of asynchronous frameworks is preferred
● Execute blocking operations in a separate thread
● Collect metrics for common services operations
● Caching saves a lot of time
● Use C or Cython-based solutions for CPU-bound tasks
● Have fast release/deploy/rollback cycle
● Python is great, but don't hesitate to pick other tools
43

Further reading
● How HipChat Stores and Indexes Billions of Messages Using ElasticSearch
● @kakovskyi: Maintaining a high load Python project for newcomers
● HipChat: Important improvements to staging, presence & database storage
● HipChat and the little connection that could
● Elasticsearch at HipChat: 10x faster queries
● Atlassian: How IT and SRE use ChatOps to run incident management
● A Study of Internet Instant Messaging and Chat Protocols
● What Is Async, How Does It Work, And When Should I Use It?
● Leaky Bucket & Tocken Bucket - Traffic shaping
● A guide to analyzing Python performance
● Why Leading Companies Dark Launch - LaunchDarkly Blog
● @bmwant: Asyncio-stack for web development
44

Questions?
45
Viacheslav Kakovskyi
viach.kakovskyi@gmail.com
@kakovskyi
Instant messenger with Python
Back-end development

WebCamp Ukraine 2016: Instant messenger with Python. Back-end development

WebCamp Ukraine 2016: Instant messenger with Python. Back-end development

Recommended

More Related Content

What's hot

What's hot(20)

Viewers also liked

Viewers also liked(16)

Similar to WebCamp Ukraine 2016: Instant messenger with Python. Back-end development

Similar to WebCamp Ukraine 2016: Instant messenger with Python. Back-end development(20)

Recently uploaded

Recently uploaded(20)

WebCamp Ukraine 2016: Instant messenger with Python. Back-end development