Your SlideShare is downloading. ×
0
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Funambol Server Architecture
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Funambol Server Architecture

6,997

Published on

Published in: Education, Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,997
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
429
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Funambol Server Architecture Stefano Nichele
  • 2. Overview <ul><li>Funambol server components </li></ul><ul><li>Synchronization overview </li></ul><ul><li>Data synchronization service architecture </li></ul><ul><li>Push overview </li></ul><ul><li>Push Connection service </li></ul><ul><li>InboxListener and PIMListener </li></ul>
  • 3. Funambol Server Architecture We will refer to those as services . The DS service is the primary service , without that the Funambol Server cannot run. And we are going to change some names.... Core component of the Funambol Server. It provides synchronization and push engines Allows push mail by checking email accounts Allows pim push by checking the database Allows Client TCP Push DataSynchronization Server Inbox Listener Pim Listener CTP Server DataSynchronization service Pim Listener service Push Connection service Inbox Listener service
  • 4. Funambol Server Architecture Synchronization Push DataSynchronization service Pim Listener service Push Connection service Inbox Listener service DB Mail Server
  • 5. Synchronization <ul><li>Process to make equal two sets of data (repository, database) </li></ul><ul><li>One-way : data is coming just from one source (i.e. backup, restore) </li></ul><ul><li>Two-way : data is coming from both sources </li></ul>PDA Backup Repo PDA My laptop
  • 6. Synchronization <ul><li>Two different modes: </li></ul><ul><li>Full-sync : the sources are synchronized by comparing the whole sets of data (OMA terminology: slow-sync) </li></ul><ul><li>Fast-sync : the sources are synchronized by comparing just the changes that occurred since the last sync. </li></ul>
  • 7. SyncML <ul><li>Open standard for data synchronization </li></ul><ul><li>XML based </li></ul><ul><li>Supported by major companies such as Nokia, Motorola....Funambol :-) </li></ul><ul><li>Since 2002 defined by Open Mobile Alliance (http://www.openmobilealliance.org) </li></ul>
  • 8. SyncML – an example <SyncML xmlns='SYNCML:SYNCML1.2'> <SyncHdr> <VerDTD>1.2</VerDTD> <VerProto>SyncML/1.2</VerProto> <SessionID>1</SessionID> <MsgID>3</MsgID> <Target> <LocURI>IMEI:004400061769830</LocURI> </Target> <Source> <LocURI>http://syncserver.com/servlets/SyncML</LocURI> </Source> ... </SyncHdr> <SyncBody> <Status> <CmdID>1</CmdID> <MsgRef>3</MsgRef> <CmdRef>0</CmdRef> <Cmd>SyncHdr</Cmd> <TargetRef>http://syncserver.com/servlets/SyncML</TargetRef> <SourceRef>IMEI:004400061769830</SourceRef> <Data>200</Data> </Status> </SyncBody> </SyncML>
  • 9. Data Synchronization service <ul><li>It's the core and the only required service of the Funambol server (primary service) </li></ul><ul><li>It governs synchronization and push </li></ul><ul><li>Extensible and highly configurable </li></ul><ul><li>SyncML 1.1 and 1.2 </li></ul>
  • 10. Data Synchronization service Admin Manager Pipeline Manager Nokia Synclet Razr Synclet Push Engine Push Sender WebServices API Plugins Manager Plugin A Plugin B Framework Device Inventory SMS Service SMS Provider Synchr. Engine Email Sync Source PIM Sync Source Google Sync Source Officer
  • 11. Data Synchronization service <ul><li>It allows additional processing and manipulation of the messages exchanged between the server and the client. </li></ul><ul><li>The kind of processing that is performed in the pipeline is a message level processing, such as the manipulation of the message elements. </li></ul><ul><li>Possible applications are: </li></ul><ul><ul><li>Encoding/decoding of item content </li></ul></ul><ul><ul><li>Item filtering </li></ul></ul><ul><ul><li>Item ordering </li></ul></ul><ul><ul><li>Message decoration adding/removing elements on a custom basis </li></ul></ul>Pipeline Manager Nokia Synclet Razr Synclet
  • 12. Data Synchronization service <ul><li>The idea behind the message processing pipeline is to be able to modify both incoming and outgoing messages: </li></ul><ul><li>in the former case, we want to be able to manipulate the message before it goes into the sync engine </li></ul><ul><li>instead, in the latter, we want to be able to change the message returned by the sync engine before sending it to the client. </li></ul><ul><li>Input and output pipeline components are called Synclet. </li></ul>Pipeline Manager Nokia Synclet Razr Synclet Synchr. Engine Email Sync Source PIM Sync Source Google Sync Source Officer S E R V L E T Input pipeline Output pipeline Pipeline Manager Raw data SyncML SyncML
  • 13. Data Synchronization service <ul><li>How are the synclets to perform selected ? </li></ul><ul><li>Synclets for any device </li></ul><ul><li>Synclets just for some devices: based on the user-agent of the requests </li></ul>Example of synclet configuration file <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?> <java version=&quot;1.5.0_08&quot; class=&quot;java.beans.XMLDecoder&quot;> <object class=&quot;com.funambol.foundation.synclet.BeanShellSynclet&quot;> <void property=&quot;header&quot;> <string> user-agent </string> </void> <void property=&quot;pattern&quot;> <string> NokiaE61 </string> </void> <void property=&quot;script&quot;> <string>com/funambol/server/engine/pipeline/phones-support/bsh/NokiaE65in.bsh</string> </void> </object> </java> Pipeline Manager Nokia Synclet Razr Synclet
  • 14. Data Synchronization service <ul><li>Main responsibilities: </li></ul><ul><ul><li>authenticate and authorize the user </li></ul></ul><ul><ul><li>identify the sources and the destinations of data to be synchronized </li></ul></ul><ul><ul><li>identify what data needs to be updated/added/deleted </li></ul></ul><ul><ul><li>determine how updates must be applied </li></ul></ul><ul><ul><li>detect conflicts </li></ul></ul><ul><ul><li>resolve conflicts </li></ul></ul>Sync. Engine Email SS PIM SS Google SS Officer
  • 15. Data Synchronization service <ul><li>Deep look at the synchronization engine </li></ul>SyncML Messages Sync Session Handler Sync4jEngine Sync4jStrategy Email Sync Source PIM Sync Source Officer It represents the handler for a SyncML session. It coordinates and handles messages as dictated by the protocol. It coordinates the sync process delegating almost all the sync logic to Sync4jStrategy Given two sources of data, it performs the real synchronization process It checks if the credentials provided by the client are valid and if the user is authorized. A SyncSource is the interface versus the data store Sync. Engine Email SS PIM SS Google SS Officer
  • 16. Data Synchronization service <ul><li>How can you interact with the engine process and how can you synchronize your data repository ? </li></ul><ul><li>Officer : it's the component that authenticates and authorizes a user. It's important since implementing your own officer you are able to authenticate a user on your user database. </li></ul><ul><li>SyncSource : it's the class used to access to the server data repository. You can develop your own PIM syncsource but also your source for MP3 or for whatever you want synchronization. </li></ul><ul><li>For more details, see the Developer Guide. </li></ul>Sync. Engine Email SS PIM SS Google SS Officer
  • 17. Data Synchronization service <ul><li>The Data Synchronization service provides the required infrastructure to allow you an easy integration with your SMS Gateway. </li></ul><ul><li>SMS Service </li></ul><ul><li>it's the entry point for the SMS infrastructure </li></ul><ul><li>no specific SMS gateway logic </li></ul><ul><li>SMS Provider </li></ul><ul><li>it's the real interface towards your preferred SMS gateway (i.e. SubitoSMS or Clickatell) </li></ul>SMS Infrastructure SMS Message Recipient Sender SMS Service SMS Provider SMS Gateway Usually this is an HTTP request SMS Service SMS Provider
  • 18. Data Synchronization service <ul><li>What can I do in the SMS Service ? </li></ul><ul><li>decorate the recipient/sender with additional information (like carrier/country) </li></ul><ul><li>work as proxy towards more than one SMSProvider selecting the one to use based on the recipient </li></ul><ul><li>provide smart functionalities (for example using the delivery detail/status information, a sms can be discharged if the previous one is still pending) </li></ul><ul><li>Out of the box we provide a simple SMS Service that forwards requests to an SMS Provider that just logs the messages. </li></ul>SMS Service SMS Provider
  • 19. Data Synchronization service <ul><li>Plug-ins : configurable objects that can be “deployed” in the DS service in order to extend its functionalities. For instance a simple plug-in could send an email to the system administrator to notify him for startup and shutdown events. </li></ul><ul><li>By now the plug-ins are notified just for startup and shutdown events. </li></ul><ul><li>As for the other configurable components, they can be deployed without stopping the DS service just copying the xml bean file under com/funambol/server/plugin . </li></ul>Plugins Manager Plugin A Plugin B
  • 20. Data Synchronization service <ul><li>It's the core of the administration API. It can be accessed locally or remotely using web services. </li></ul><ul><li>It exposes the administration API via web services. </li></ul><ul><li>It's the component that handles devices and their capabilities </li></ul>....the latest DS service components Admin Manager WebServices API Device Inventory
  • 21. Funambol Server Architecture Push DataSynchronization service Pim Listener service Push Connection service Inbox Listener service DB Mail Server
  • 22. Push <ul><li>Push sequence in summary: </li></ul><ul><li>1. the Funambol server detects that a device needs to sync </li></ul><ul><li>2. the Funambol server sends a push message to the device. This is pkg #0 in the OMA specs </li></ul><ul><li>3. the device starts a sync so that its data is updated with the one on the server </li></ul>Funambol Server Data Repository Data Repository 1 2 3
  • 23. Push <ul><li>How can the server send the push message to the device ? </li></ul><ul><li>SMS seems the easiest way but SMSs are not free of charge </li></ul><ul><li>The client can run a TCP server so the server can open a TCP connection and sends the push message over that </li></ul><ul><li>The client can open a connection versus the server and keep it always open, ready for push message from the server </li></ul><ul><li>Funambol Server supports out of the box connection-less and connection-oriented push but SMS Push is easily implementable. </li></ul>STP Push CTP Push SMS Push SMS Push Connection-less Push Connection-oriented Push
  • 24. Push <ul><li>Who does what ? </li></ul>DS service Mail Server InboxListener service PIMListener service PIM DB Mail Server WS API Push Engine Push Connection service The Inbox Listener at regular time checks on the mail server if there are users to push If there are devices to push, the IL and the PIM listener call the DS service admin API The Push Engine tries to send the push message using connection-less push The Push Connection service tries to push the device using an existing connection The Push Engine asks to the Push Connection service to push the device using connection-oriented push The PIM Listener at regular time checks on the DB if there are devices to push 1 1 2 3 4 5
  • 25. Data Synchronization service <ul><li>Main responsibilities: </li></ul><ul><li>Handling push :-) </li></ul>Push Engine Push request WS Admin API Notification Engine Notification Builder Notification Sender DB The NotificationEngine uses a NotificationBuilder to create the Push message The NotificationEngine checks in the DB if there are old push messages The sender is the component used to deliver a push message to a device If the sender is not able to deliver a push message, it is stored in the database 1 2 3 4 Push Engine Push Sender
  • 26. Data Synchronization service <ul><li>Builder and sender configurable per device </li></ul><ul><li>Two available builder implementations (for SyncML 1.1 and for SyncML 1.2) </li></ul><ul><li>Just one sender implementation that tries: </li></ul><ul><ul><li>connection-less push </li></ul></ul><ul><ul><li>connection-oriented push (if the previous one fails) </li></ul></ul>SMS Service SMS Provider Push Engine Push Sender
  • 27. Connection-less push <ul><li>Main concepts: </li></ul><ul><li>the client must be able to run a TCP server waiting for connections from the server </li></ul><ul><li>when a push message must be sent, the DS service opens a socket and writes the message </li></ul><ul><li>the DS service must know the client IP address </li></ul><ul><li>the IP address must be reachable from the DS service </li></ul>
  • 28. Connection-less push IP Address Pending Push IP Address <ul><li>Address notification: process used by the client to notify its IP address </li></ul><ul><li>The device must notify its address any time it changes ! So it must be able to detect when it changes. </li></ul>Address Listener Servlet DB If the server is able to send the pending push to the client, the IP address is stored in the DB. 1 2 3 The device sends its IP address to the server The server verifies if the client address is reachable trying to send the pending push messages
  • 29. Connection-less push <ul><li>Issues: </li></ul><ul><li>The client must be able to run a TCP server </li></ul><ul><li>The client must be able to detect when its IP address changes </li></ul><ul><li>The IP address must be reachable from the server </li></ul>
  • 30. Connection-oriented push <ul><li>Main concepts: </li></ul><ul><li>The client opens a connection on the server and keeps it open </li></ul><ul><li>The server uses that connection to deliver push messages </li></ul><ul><li>...really simple ;-) </li></ul>
  • 31. Connection-oriented push UDP channel Connecting Pushing Push Connection Service 1 The client opens the connection and sends its credentials DS service WS API Push Engine 2 The Push Connection service calls the DS service WS API to authenticate the client The Push Sender sends a multicast UDP message with the push message 2 1 The Push Connection service receives the UDP message and checks if the recipient is connected 3 If the device is connected, the push message is written on the already open (by the client)connection
  • 32. Push Connection service <ul><li>Main challenge: </li></ul><ul><li>It must be able to handle millions of connections </li></ul><ul><li>Features: </li></ul><ul><li>Java Standard Application </li></ul><ul><li>Based on javax.nio </li></ul><ul><li>Based on Apache Mina (network application framework) </li></ul><ul><li>Clusterable </li></ul>
  • 33. InboxListener and PIMListener <ul><li>Both do the same work, at regular time they must check the data repository </li></ul><ul><li>Both based on the PushListener Framework... </li></ul>Push Listener Framework InboxListener Task InboxListener Push Listener Framework PimListener Task PIMListener
  • 34. Push Framework Listener <ul><li>Framework ready to use to implement any listener. By now used for: </li></ul><ul><ul><li>InboxListener </li></ul></ul><ul><ul><li>PIMListener </li></ul></ul><ul><ul><li>ExchangeListener </li></ul></ul><ul><ul><li>OXListener </li></ul></ul><ul><li>Pretty similar to a scheduler with basic functionalities for push and for HA/scalability </li></ul><ul><li>Java Standard Application (no J2EE) </li></ul><ul><li>Based on Jdk 1.5 thread features </li></ul><ul><li>JMX interface </li></ul><ul><li>Pluggable architecture </li></ul>
  • 35. Push Framework Listener <ul><li>Hey, this is not push.....it's polling ! </li></ul>UDP notification <ul><li>Polling mode works with all mail servers </li></ul><ul><li>Notifiable mode can be used just in some cases </li></ul>InboxListener service Mail Server Polling mode InboxListener service Mail Server Notifiable mode
  • 36. Push Framework Listener Task executor Tasks queue Scheduled tasks One Shot tasks DB Registry Monitor Plugin Manager UDP Listener At startup it loads from the database the list of tasks to perform and publish it in the tasks queue Thread Pool. Any free thread pickup from the queue a task and starts its execution. It can contain scheduled tasks or “One Shot” tasks It allows notifiable mode ... ... User 1 User 2 User 3 User 4
  • 37. Push Framework Listener <ul><li>How does it scale ? </li></ul>Instance 1 of 3 Instance 2 of 3 Instance 0 of 3 Cluster It executes the tasks with (id % 3) = 0 It executes the tasks with (id % 3) = 1 It executes the tasks with (id % 3) = 2 InboxListener service A InboxListener service C InboxListener service B Any instance know its index and how many instances are in the cluster Tasks Repository The cluster is obtained using JGroups
  • 38. Push Framework Listener <ul><li>HA – High availability </li></ul>Cluster InboxListener service A InboxListener service C InboxListener service B Tasks Repository Instance 1 of 3 Instance 2 of 3 Instance 0 of 3 It executes the tasks with (id % 3) = 0 It executes the tasks with (id % 3) = 1 It executes the tasks with (id % 3) = 2 Instance 1 of 2 Instance 0 of 2 It executes the tasks with (id % 2) = 0 It executes the tasks with (id % 2) = 1
  • 39. Conclusions Monitoring Monitoring WS API Connection-less push Request for connection-oriented push Connection-oriented push Synchronization DS service Mail Server InboxListener service PIMListener service PIM DB Mail Server WS API Push Engine Push Connection service 1 1 2 3 4 5 Sync Engine 6
  • 40. References <ul><li>Funambol Forge - https://www.forge.funambol.org/ </li></ul><ul><li>Funambol Developer Guide </li></ul><ul><li>http://download.forge.objectweb.org/sync4j/funambol-v7-developers-guide.pdf </li></ul><ul><li>Funambol design documents </li></ul><ul><li>https://core.forge.funambol.org/wiki/FunambolDesignDocuments </li></ul><ul><li>Open Mobile Alliance - http://www.openmobilealliance.org </li></ul><ul><li>myFunambol - http://my.funambol.com </li></ul><ul><li>Funambol mailing lists - https://core.forge.funambol.org/ds/viewForums.do </li></ul>
  • 41. The End

×