Contributing to OSS in
                 a commercial non-OSS
                 environment




                            ...
Overview

□
    What is Mailtrust
□
    What OSS does Mailtrust use
□
    How has Mailtrust contributed to Funambol
□
    ...
□
    Formerly Webmail.us
□
    Now a division of Rackspace
       ◊   Recently went public
□
    Business-class email hos...
Mailtrust and Open Source

□
    Heavily Used         □
                             Main Contributions
       ◊   PHP    ...
Contributions to Funambol

□
    DS-Server
       ◊   Webmail Connector
□
    Outlook Client
       ◊   Support for 'custo...
Goals for Sync Service

□
    Synchronize shared data
       ◊   Read-only
□
    Give back to open source community
      ...
System Architecture Evolution, Part 1

Initial design:
●
  Very Simple
●
  Connector hit DBs directly

                   ...
System Architecture Evolution, Part 1

Lessons Learned:
●
  2 code bases to maintain!
●
  Data validation duplicated
●
  P...
System Architecture Evolution, Part 2

Introducing the Webmail-Sync API
●
  Implementation of SyncSource
   interface in A...
System Architecture Evolution, Part 2

Lessons Learned
●
  PHP has an execution time limit
    ●
        Don't do too much...
System Architecture Evolution, Part 3

Batching calls to API
●
  Can't batch calls w/ standard code
    ●
        addItem ...
System Architecture Evolution, Part 3

Lessons Learned:
●
  The Strategy is complicated
●
  Test, Test, Test
●
  Funambol ...
System Architecture Evolution, Part 4

New Problems
●
  Cannot upload to webmail and
    sync at the same time!
●
  Do not...
System Architecture Evolution, Part 4

Lessons Learned:
●
  Dedicate systems whenever                   Webmail
possible
●...
System Architecture Evolution, Part 5

New Problems, Again!
●
  Network connectivity issues                       Webmail
...
System Architecture Evolution, Part 5

Lessons Learned:
●
  Don't retry 'dangerous' commands                              ...
System Architecture – Misc details

□
    SyncSourceRedirectSynclet
       ◊   Replaces remote URI in requests
       ◊   ...
System Architecture – Misc details

□
    SessionConnectionLimiterSynclet
       ◊   Disconnects a session that has lasted...
System Architecture – Misc details

□
    Authentication
       ◊   Moved queries to mysql stored procedures
       ◊   Ke...
Summary

□
    Use abstractions to hide proprietary
    details
       ◊   Write open logic, and share it!
□
    Physicall...
Summary, cont.

□
    Rely on configurations for details
       ◊   Funambol does this already – model after
            t...
Summary, cont.

□
    Coordinate with your community
       ◊   Communication is key
       ◊   Work out a way to share co...
The End

Questions? Comments?

mtaczak@mailtrust.com   23
Upcoming SlideShare
Loading in …5
×

Contributing to OSS in a commercial non-OSS environment

1,341 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,341
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Contributing to OSS in a commercial non-OSS environment

  1. 1. Contributing to OSS in a commercial non-OSS environment Mike Taczak mtaczak@mailtrust.com 25/11/08 Mailtrust, a division of Rackspace www.mailtrust.com | www.rackspace.com
  2. 2. Overview □ What is Mailtrust □ What OSS does Mailtrust use □ How has Mailtrust contributed to Funambol □ Design challenges □ Review of integration strategies 2
  3. 3. □ Formerly Webmail.us □ Now a division of Rackspace ◊ Recently went public □ Business-class email hosting □ Noteworthy Webmail ◊ Full-featured AJAX-y webmail application ◊ Competitive collaboration suite 3
  4. 4. Mailtrust and Open Source □ Heavily Used □ Main Contributions ◊ PHP ◊ Funambol ◊ mySQL ◊ Dovecot ◊ Hadoop ◊ Postfix ◊ amavisd ◊ policyd ◊ Many others 4
  5. 5. Contributions to Funambol □ DS-Server ◊ Webmail Connector □ Outlook Client ◊ Support for 'custom fields' □ Blackberry PIM Client ◊ Initial development in-house ◊ Now part of Funambol's core clients □ iPhone client ◊ 2 weeks in Italy 5
  6. 6. Goals for Sync Service □ Synchronize shared data ◊ Read-only □ Give back to open source community ◊ But keep proprietary systems private □ Focus on a few highly refined clients ◊ Outlook ◊ Blackberry ◊ Windows Mobile 6
  7. 7. System Architecture Evolution, Part 1 Initial design: ● Very Simple ● Connector hit DBs directly Webmail Webmail Database Data Webmail DS-Server Authen Module tication User Database 7
  8. 8. System Architecture Evolution, Part 1 Lessons Learned: ● 2 code bases to maintain! ● Data validation duplicated ● Proprietary DB schemas in open source code! Webmail ● We could not launch with this architecture Webmail Database Data Webmail DS-Server Authen Module tication User Database 8
  9. 9. System Architecture Evolution, Part 2 Introducing the Webmail-Sync API ● Implementation of SyncSource interface in API form ● Data parsing now in php ● Implemented a large Vobj library Webmail ● Hope to open-source it! Database ● Uses existing structures and validation ● HTTP + jsON ● Connector generic enough for other uses ● Clear separation between Funambol and Webmail Webmail DS-Server API Webmail Module 9
  10. 10. System Architecture Evolution, Part 2 Lessons Learned ● PHP has an execution time limit ● Don't do too much at once ● Know your technologies intimately ● Distance matters Webmail ● Reliability, speed deteriorates Database ● Retry failed requests ● getSyncItemByKey is slow ● Page calls and cache data DS-Server Webmail 2000 API Webmail Module Miles! 10
  11. 11. System Architecture Evolution, Part 3 Batching calls to API ● Can't batch calls w/ standard code ● addItem requires returning a GUID ● Went one layer higher ● New BatchedSyncStrategy Webmail ● New BatchedSyncSource Database ● Completely compatible with original ● Reduced calls by 50-90% Strategy Webmail DS-Server API Webmail Module 11
  12. 12. System Architecture Evolution, Part 3 Lessons Learned: ● The Strategy is complicated ● Test, Test, Test ● Funambol is very flexible to these kind of changes Webmail Database Strategy Webmail DS-Server API Webmail Module 12
  13. 13. System Architecture Evolution, Part 4 New Problems ● Cannot upload to webmail and sync at the same time! ● Do not want load from API processing affecting webmail Webmail users Database Strategy Webmail DS-Server API Webmail Module 13
  14. 14. System Architecture Evolution, Part 4 Lessons Learned: ● Dedicate systems whenever Webmail possible ● Scale services independently Webmail Database Dedicated Strategy Webmail Webmail DS-Server API Module For Sync 14
  15. 15. System Architecture Evolution, Part 5 New Problems, Again! ● Network connectivity issues Webmail caused duplicates ● Requests to addItems failed, and were retried ● Sync was hitting webmail databases too hard ● 50 calendars can have a lot of data Webmail Database Dedicated Strategy Webmail Webmail DS-Server API Module For Sync 15
  16. 16. System Architecture Evolution, Part 5 Lessons Learned: ● Don't retry 'dangerous' commands Webmail ● Database connections are limited resources ● Cache everything reasonable! ● Database requests ● API requests! Webmail ● Open source has a solution Database memcached memcached Dedicated Strategy Webmail Webmail DS-Server API Module For Sync 16
  17. 17. System Architecture – Misc details □ SyncSourceRedirectSynclet ◊ Replaces remote URI in requests ◊ Uses content types sent by client, has defaults ◊ 'contacts' => 'contacts-vcard21' or 'contacts-sif' ◊ Easier for users to configure devices ◊ Guarantees clients get the preferred type, if available 17
  18. 18. System Architecture – Misc details □ SessionConnectionLimiterSynclet ◊ Disconnects a session that has lasted 'too long' ◊ Protects service from runaway clients 18
  19. 19. System Architecture – Misc details □ Authentication ◊ Moved queries to mysql stored procedures ◊ Keeps DB schema out of code and configuration ◊ Allows installations to redefine these procedures 19
  20. 20. Summary □ Use abstractions to hide proprietary details ◊ Write open logic, and share it! □ Physically and logically separate open and closed source programs ◊ API separated synchronization logic from our proprietary systems and data ◊ Stored procedures hide our database behind an interface 20
  21. 21. Summary, cont. □ Rely on configurations for details ◊ Funambol does this already – model after them ◊ Requires flexible code, just the kind open source needs □ Work at a great company ◊ The kind that values open source ◊ The kind that lets you give back! 21
  22. 22. Summary, cont. □ Coordinate with your community ◊ Communication is key ◊ Work out a way to share code and plans often ◊ Still working on this one ;) □ Go to Italy! 22
  23. 23. The End Questions? Comments? mtaczak@mailtrust.com 23

×