OWD - Push Notification Server Architecture [DEVCON1_2012]

7,301 views

Published on

More info at: http://www.lacofa.es/index.php/general/notification-server or http://www.lacofa.es/index.php/general/notification-server?lang=en

Published in: Technology
0 Comments
19 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,301
On SlideShare
0
From Embeds
0
Number of Embeds
286
Actions
Shares
0
Downloads
0
Comments
0
Likes
19
Embeds 0
No embeds

No notes for slide

OWD - Push Notification Server Architecture [DEVCON1_2012]

  1. 1. Push Notification Server Architecture talkTelefonica Digital28th November 2012
  2. 2. 01Introduction
  3. 3. 01 Push Notification Server Who are we? • Fernando Rodríguez Sela frsela@tid.es @mangiacaprini • Guillermo López Leal gll@tid.es @willyarandaOWD 3Telefonica Digital
  4. 4. 01 Push Notification Server The project • Objective: Develop a PUSH Notification Server  Mobile network friendly  No developer registration  Easy to use  Scalable  Based on web technologies http://www.lacofa.es/index.php/general/notification-server https://github.com/telefonicaid/notification_serverOWD 4Telefonica Digital
  5. 5. 02 Technologies usedOWD 5Telefonica Digital
  6. 6. 02 Node.JS Javascript on the server … awesome ! • Server side platform based on the V8 JS engine (Chrome) • Event-driven based • Non-blocking I/O model • Very fast and low fingerprint • High performance with heavy load • http://nodejs.orgOWD 6Telefonica Digital
  7. 7. 02 MongoDB NO-SQL database • Document oriented storage • JSON style documents (BSON) • High performance • Scalability (replication & autosharding) • http://mongodb.orgOWD 7Telefonica Digital
  8. 8. 02 RabbitMQ Asynchronous messaging queue system • Robust and fast messaging • Small software  12.000 lines of erlang code  Erlang is highly concurrent and used for real time HA software • Multiple delivery schemes. We use publish/subscribe with round-robin  ActiveMQ dont support round-robin out of the box • High performance  Outstands ActiveMQ performance in persistent mode • Scalability (Clustering mode)  Far better than ActiveMQ options (master-slave) • http://rabbitmq.comOWD 8Telefonica Digital
  9. 9. 03 Mobile network issuesOWD 9Telefonica Digital
  10. 10. 03 The problem Current problem – a bit of history • Designed without considering the mobile network • Push servers cant connect directly to the device  Long polling  Keep alives • Consequences  Signalling storms  Draining users batteries  Low QoS • Network stress • Real case: WhatsApp & Telefonica november 2012OWD 10Telefonica Digital
  11. 11. 03 The problem Current problem – Radio states & Why signalling storms RRC Idle • Battery consume 1 relative unit Cell_PCH • Battery consume <2 relative units URA_PCH • Battery consume ≤ Cell_PCH Cell_FACH • Battery consume 40 relative units Cell_DCH • Battery consume 100 relative units Each keep-alive message force move the handset from the IDLE state to a shared channelOWD 11Telefonica Digital
  12. 12. 04 State of the artOWD 12Telefonica Digital
  13. 13. 04 Current Situation Cellular PS network App Server 1 Internet 1 Polling App 2 App Server N OS Two different behaviours: Polling: 1. Polling applications check periodically if there is new information in their App Servers. 2. Most of the requests are not delivering data, creating network congestion: both RRC and TCP connections open/close in core. Keep-alives: 1. The application keeps an permanently open TCP connection with their server, by sending keep-alives. 2. The server can reach the application trough this TCP connection.OWD 13Telefonica Digital
  14. 14. 04 Current SituationOWD 14Telefonica Digital
  15. 15. 04 Current SituationOWD 15Telefonica Digital
  16. 16. 04 Solutions: WAP Push Cellular PS network Application server Internet Encoded WAP push (SMS) WAP Push to e.g. WAP Push Proxy +44802123456 Gateway @ppg.genie.co.uk WAP Push Initiator Server Cellular CS network • The push message starts at the WAP Push Initiator and it is sent to the Push Proxy Gateway specified in the destination address of the push message. All addresses are relative to the WAP push proxy gateway (PPG). The PPG sends the SMS messages through CS to the phone identified using its MSISDN. Then, the Mobile opens the PS connection. • Two variants: • Service Indication: On receiving a WAP Push the handset will automatically give the user the option to access the WAP content. • Service Load: Directly open de browser to display the WAP content without user interaction.OWD 16Telefonica Digital
  17. 17. 04 Solutions: Apple Push Notification Service (APNS) & Google Cloud Messaging (GCM) Application Server Cellular PS network 5 Internet 1 3 2 NS Server 4 6 1. The application registers for push notifications. The OS asks NS Server for a token. 2. The application receives the device token. 3. The app sends the token to the Application Server. 4. The mobile keeps 1 single permanently open TCP connection with the NS server, by sending keep-alives. 5. The Application Server sends the push notifications to the NS Server with their token. 6. The NS sends the push notification to the app through the open TCP connection.OWD 17Telefonica Digital
  18. 18. 05 Our solutionOWD 18Telefonica Digital
  19. 19. OWD 19Telefonica Digital
  20. 20. 05 Firefox OS Push Notification Service Application Server Cellular PS network Internet 1 4 3 2 Push Notification Server 5 http://push.telefonica.es 1. The mobile application registers for push notifications in the Push Notification Server. The Push Notification Server assigns a token to the device or application, and stores the private IP of the mobile associated to that token. 2. The mobile app receives the “URL to notify me” containing the Push Notification Server address and the token (e.g.: http://push.telefonica.es/token=123456789) 3. The mobile app sends the “URL to notify me” to its Application Server. 4. The Application Server sends the push notifications to that URL (the Push Notification Server with their token). 5. The Push Notification Server sends the push notification to the mobile app. As there is not an open TCP connection (but the device has a private IP), the network will page the mobile to “find it” and forward the packet to it.OWD 20Telefonica Digital
  21. 21. 05 Firefox OS Push Notification Service • No need to keep any open TCP connection. => No keep-alives • Open and standard solution •W3C, OMA •Other interested carriers • No registration needed • No quotas • No fees • SecureOWD 21Telefonica Digital
  22. 22. 05 The problem Proposed solution • Objective: Avoid Keep-Alive and open connections  Reduce required network resources  Reduce battery consume (increasing the radio IDLE status) • How to achieve it?  Notification server should have “direct vision” with the handset • Avoid maintaining open sockets – WakeUps sent via WAP-PUSH, UDP or TCP packets • The device listens for those notifications – Then, retrieve the notificationOWD 22Telefonica Digital
  23. 23. 05 Interested carriers We are not aloneOWD 23Telefonica Digital
  24. 24. 06 Proposed protocolOWD 24Telefonica Digital
  25. 25. 06 Proposed protocol Actors • WebApp (WA) – Web • Notification Server (NS) – Application which receives Server dedicated to manage asynchronous notifications all PUSH notifications and responsible to deliver to the correct UA/WA. • User Agent (UA) – Web • AppServer (AS) – Application rendering engine which offers Server – This is the server a simple API to the WA to deployed by the WA receive notifications. Also the developer UA shall maintain a communication channel with the notification server.OWD 25Telefonica Digital
  26. 26. 06 Proposed protocol BoxesOWD 26Telefonica Digital
  27. 27. 06 Proposed protocol APIs • (1) API defined by W3C. Used • (3) Private Protocol designed by the application to register it by the application developer. into the NS and receives Should be used to inform the notifications over the AS about the publicURL given publicURL given by the NS to the WA by the NS (2) API used by the UA to • (4) REST API used to sent register a communication notifications from the AS to channel with the NS in order the WA to receives notifications. This channel should be different based on the situation: WiFi or external networks (WebSocket) Same TCP Network (UDP) ...OWD 27Telefonica Digital
  28. 28. 06 Proposed protocol Tokens and how we identify each actor • NS uses tokens to identify UAs and WAs  UAToken → Cryptographic token (AES)  WAToken → Identify an application installation  Public Key (PBK) → Identify each application & used for signing  AppToken → SHA256(WAToken + PBK)OWD 28Telefonica Digital
  29. 29. 06 Proposed protocol Kind of deliveries • To one user and one device  The WAToken given is unique and secret • To one user and multiple devices  The WAToken given is shared between user devices and secret • To all the users of concrete application  The same WAToken is used in all the instances (or group of them)  Obviously, not a secretOWD 29Telefonica Digital
  30. 30. 06 Proposed protocol How it works 1) Get a valid UAToken through any secure mechanism 2) On UA starts 1) Open a WebSocket with the configured NS 2) Register itself sending his UAToken and some network parameters 3) Register applications by sending WAToken + PBK 1) Receive a notification URL for each registered application 4) Each app. Send his own notification URL to their ASOWD 30Telefonica Digital
  31. 31. 06 Proposed protocol How it works 1) AS POST a notification to the public notification URL 2) The Notification server: 1) Verify signature 2) Delivery: 1) If open WebSocket, just send the notification to the recipients 2) Else: 1) If UDP: send a wakeup packet 1) The UA connects to the NS and retrieves the notification.OWD 31Telefonica Digital
  32. 32. 07 Our architectureOWD 32Telefonica Digital
  33. 33. 07 Architecture Refreshing diagramOWD 33Telefonica Digital
  34. 34. 07 Internal Architecture Multiple Servers Message Queue User Agent Web Socket Application Server User Agent UDP NO-SQL MongoDBOWD 34Telefonica Digital
  35. 35. 07 Inside NS Registration Message Queue 1 User Agent Web Socket Application 2 Server User Agent UDP NO-SQL MongoDBOWD 35Telefonica Digital
  36. 36. 07 Inside NS Send a notification Message 4 Queue 6 User Agent 4 3 Web Socket Application 1 5 Server User Agent UDP 2 5 NO-SQL MongoDBOWD 36Telefonica Digital
  37. 37. 08 SecurityOWD 37Telefonica Digital
  38. 38. 08 Security & Privacy PvK / PbK – Signed • Check signature for each notification  Avoid malicious senders  Verify the origin  Private Key (PvK) on AS / Public Key (PbK) on WA  On the WA registration process, WA provides the WAToken and a PbKOWD 38Telefonica Digital
  39. 39. 08 Security & Privacy publicURL • A publicURL ↔ WAToken & PbK  PublicURL = https://server.domain.com/notify/APPTokenOWD 39Telefonica Digital
  40. 40. 08 Security & Privacy Possible attacks • An evil AS wants to send notifications  Need to know the Private Key • An evil WA wants to receive notifications from another WA  Need to know the WAToken which SHALL be a secret • An evil device wants to register as another one  Need to know the UAToken (managed by the OS) • Try DoS, flood messages, … → abuse controls on server sideOWD 40Telefonica Digital
  41. 41. 03 Security & Privacy Security always should be improved ! Were open to suggestionsOWD 41Telefonica Digital
  42. 42. 09 Advantages for developersOWD 42Telefonica Digital
  43. 43. 09 We need to give some sweets to the developers The way to spread the useOWD 43Telefonica Digital
  44. 44. 09 We need to give some sweets to the developers The way to spread the use • Easy to use API  Based on Web technologies && standarization in progress • Reduce developer deployment costs • More efficient use of networks && battery • No registration process needed and no subscriptions • Bigger payloads and more messages per appOWD 44Telefonica Digital
  45. 45. 10 Issues found during DeploymentOWD 45Telefonica Digital
  46. 46. 10 Issues found Theory vs practice • RabbitMQ  Use persistence only when you need to • We prefer using MongoDB  Chose carefully your configuration (publish-subscribe, round robin, priority) • MongoDB:  Use sharding and think for a good sharding key  Delete your SQL knowledge and start from zero • Self-contained documents • Node.JS:  collapses (100% CPU) when reaches ulimit • A lot of connections == a lot of file descriptors • Raise ulimit and file descriptors (inodes)  make tests (Travis-ci)  Explore variations for your algorithms • Delete vs de-referenceOWD 46Telefonica Digital
  47. 47. 11 Standarization worksOWD 47Telefonica Digital
  48. 48. 11 Standarization Convert it in the standard solution for pushing • W3C interested • OMA interested • A lot of carriers want to push this solutionOWD 48Telefonica Digital
  49. 49. 11 Standarization Convert it in the standard solution for pushing • W3C WebApps Working Group: Specifying Push device API. Telefónica co- editing the Push API draft specification together with AT&T. Recently progressed to FWPD (First Public working Draft), meaning it has been accepted as a working group draft specification • OMA AOI (Always online Infrastructure): Defining a Push framework, with same objectives as the Firefox OS one. Telefonica has successfully contributed to align OMA AOI requirements with Firefox OS push framework. Recently started the discussion on the best architecture to fulfill the requirements, to which Telefonica is also contributing.OWD 49Telefonica Digital
  50. 50. 11 Proposed protocol Standarization http://dvcs.w3.org/hg/push/raw-file/default/index.htmlOWD 50Telefonica Digital
  51. 51. 12 Next steps What for V2?OWD 51Telefonica Digital
  52. 52. 12 The future (IPv6) App Server N Internet Every UE will have a public IP, but the operator’s firewall will block connections from the applications servers. It will be necessary to have rules in the firewall or proxy to accept some incoming connections. That rules should keep in mind what applications the UE has. ¿Should present solution be compatible with IPv6?OWD 52Telefonica Digital
  53. 53. 12 Some improvement ideas • Support priority  Enqueue low priority messages • TTL • Backup PINGs when the device establish any other connection • On open connections send messages as keep-alive response • Support WAP PUSH for wake-up (require carrier integration) • Abuse control • Carriers with multiple private sub-networks • More control about sent notifications  Is the UA connected? (presence)  Is the notification delivered?OWD 53Telefonica Digital
  54. 54. 13 Questions?OWD 54Telefonica Digital
  55. 55. OWD 55Telefonica Digital

×