2. Current situation: Z-Push 1.5.10
• Z-Push 1.x code base 5 years old
• Development & improvement of
1.5.x for 1,5 years
• Considered very stable
• Generally good support of devices
• Supports ActiveSync 2.5
(comparable with MSEX 2003)
– Early days of mobility
– Only plaintext email possible
(with the exception of iOS)
3. Not optimal ‚things„ with Z-Push 1.5
• Re-creation of the ActiveSync profiles of the users necessary in
many cases
• Synchronization states are hardly manageable
– Broken/missing states cause duplicates on mobile
• No effective error detection
• Invalid mapi data “drives devices crazy”
– Battery drain
– „incoming loops‟ (same item created various times on the server)
– Never ending synchronisations
• Many concurrent push connections (especially from iOS)
• Code extensibility limited
4. Key features for the administrator
• More control
– About connected users and devices
– Remote-wipe even with changed User password or without store
• Improved logging
– Different log levels
– Ability to write full logs for certain users
– Fail2ban support
• Support friendly
– Generally no need to recreate profiles on the mobiles
5. Stability improvements I
• Semantic checks on data
Every data object is checked before it is sent to the mobile,
e.g. “is the end date of an appointment before the start date”
Object can be ignored if a critical criteria is not met.
Results in [WARN] level log lines and are also in the device details:
SyncObject->Check(): object from type SyncContact: parameter
'picture' is longer than 49152. Check failed (4)
• Incoming loop detection (failstate)
Assures that created items on the mobile are created ONCE on the server.
Does not generate a specific log entry
6. Stability improvements II
Loop detection
– Detects when mobiles do not understand received data. The amount of
data is reduced to 1 object trying to identify which object is causing the
issue.
– When problematic item is identified it is ignored. This is logged on
[WARN] log level. This message is visible in z-push-admin.
– Users could get error messages on the mobile
“Synchronization not possible at the moment”.
Retrying several time (refresh) generally solves this.
– In some cases the “Mail app” (iOS) needs to be stopped and restarted.
Device reboot is also a good option.
7. Stability improvements III
Detection of loop in different processes
• Interprocess loop detection
Example:
1. Sync on an invalid folder -> triggers a FolderSync
2. FolderSync does not solve the issue
3. Sync on the invalid folder is issued again -> loops
After 2 retries a full re-sync is executed automatically
• Segfault detection
Process did not terminate successfully.
Logs message in [ERROR] log level:
LoopDetection->ProcessLoopDetectionPreviousConnectionFailed():
Command 'Sync' at 07/06/2012 17:03:51 with pid '4986'
terminated unexpectedly or is still running.
8. Real push notifications
• In combination with ZCP 7.0.6+ real notifications are available by
using a “changes sink”!
Polling vs. Notification
3
ZCP 1 Z-Push 2
ZCP 4 Z-Push
Z-Push registers with the server
Z-Push checks (1) with the on which folders it wants to be
server every X seconds (2) notified on changes (3).
if changes are available
On change, server notifies Z-Push (4).
9. New tools – z-push-top (live statistics)
• ZCP 7.0.6+ implements changes sink (see “Sink ...”)
• Fallback method is polling (says “Polling ...”)
10. z-push-top
• Several actions available
– Type “h” or “help” in z-push-admin for full list
– Possible to grep and tail directly on the log from tool
– Filter listed output (option “f”) to only see certain commands, user, pid or
device
– Activate “wide” view for large screens (option “w”)
• Hint: You can create a symbolic link on your server to easily access
z-push-top and z-push-admin
ln –s /usr/share/z-push/z-push-top.php /usr/sbin/z-push-top
ln –s /usr/share/z-push/z-push-admin.php /usr/sbin/z-push-admin
Note: to execute these commands you need reading and writing
permissions to the logs and states
11. Device Management with z-push-admin
• User administration (details about connected devices, remote-wipe,
resynchronization)
• Administrative clean up (reset loop detection)
12. New tools – z-push-admin (II)
• Also available over SOAP interface
13. Device management
With z-push-admin
• See which users have connected an ActiveSync device:
-a list
• See details about one or all devices:
-a list –u USERNAME –d DEVICE
• Remote wipe a device
-a wipe –u USERNAME –d DEVICE
• Remove states of all or a single devices of a user
-a remove –u USERNAME [-d DEVICE]
Note: if devices are still connected they will re-provision and re-sync
• Trigger re-sync of a device
-a remove –u USERNAME [-d DEVICE]
Note: remove and re-sync are similar, but no re-provision is necessary and
dates like “first synchronization” stays intact.
14. Partial & Empty Synchronization
Multiple folders per request (device decision)
• Z-Push 1.5: one sync request is always for one folder
• Z-Push 2.0: one sync request can be for one or multiple folders
“Partial” and “empty” sync
• Partial: request sync for one or more folders, but also expects
changes for “all known synchronized folders”
• Empty: request sync without sending data, which means:
give changes for “all known synchronized folders”
15. Heartbeat Synchronization
• Is a combined PING and SYNC request
• Initializes like a normal sync but blocks on requested folders if there
are no changes atm
• On change:
the new data is sent immediately in the response
• Partial/Empty & Heartbeat can be combined
• Much lower latency till change is on the mobile
– one response instead of one response and one full sync request
• Less bandwidth consumption
• May look confusing in z-push-top, as command “Sync”
16. Real notifications & heartbeat functionality
“BlackBerry Experience”
with ActiveSync
Currently supported by
Android 4 and
Windows Phone 7+
17. Configuration options
• Timezone configuration is important for Z-Push 2
– If no timezone is set (default), Europe/Amsterdam is used
• State directory:
STATE_DIR /var/lib/z-push/
• Provisioning settings (as in 1.5.x)
PROVISIONING and LOOSE_PROVISIONING
• Amount of items to be synchronized (default all)
SYNC_FILTERTIME_MAX SYNC_FILTERTYPE_ALL
18. Configuration options – Logging options
• Log directory:
LOGFILEDIR /var/log/z-push/
• Log level (default)
LOGLEVEL LOGLEVEL_INFO
• Available:
LOGLEVEL_OFF, LOGLEVEL_FATAL, LOGLEVEL_ERROR
LOGLEVEL_WARN, LOGLEVEL_INFO, LOGLEVEL_DEBUG
Containing user data:
LOGLEVEL_WBXML wbxml data (for support!)
LOGLEVEL_DEVICEID as wbxml but with device id
LOGLEVEL_WBXMLSTACK wbxml processing stack
19. Configuration options – Advanced logging
• Separate log for certain users
LOGUSERLEVEL LOGLEVEL_DEVICEID
Users which should be logged are named:
$specialLogUsers = array(‘mo’, ‘j@d.com’);
• Fail2ban log:
LOGAUTHFAIL true
Writes [WARN] message into the default log
20. Configuration: synchronize Public folder
• Before synchronizing a folder, Z-Push will check if user has at least
“Secretary permissions” on this folder
• Globally configured for all users on system
$additionalFolders = array(
array(
'store' => "SYSTEM",
'folderid' => "7bd2229f7df92680b0700000000",
'name' => “Learn as SPAM",
'type' => SYNC_FOLDER_TYPE_USER_MAIL,
),
• Several folders of several stores can be synchronized
• Data can be discovered using a script available for ZCP in
z-push/backends/zarafa/listfolders.php
21. State management
• Z-Push 2 states are by default located in
/var/lib/z-push
• Directory needs to be writeable for apache
• Below this directory, several subdirectories are created
• States can be deleted manually!
– same operation which z-push-admin resync/remove options do!
• States are located by device using the last 2 characters of the
device id, e.g.
Device id States location
Appl789789Z7G /var/lib/z-push/g/7
• State files begin with the device id, e.g.
Appl789789Z7G-74cad955-d390-4f02-b4a1-feb576f02a46-<xx>
22. States II
• The states have a fixed structure
<device-id>-<sync-uuid>-[type]-<counter>
• Structure in detail
– Appl789789Z7G the device id
– devicedata main device information
– 74cad955-d390-feb576f02a46 Unique key for a folder (uuid)
– <no-type> normal synchronization state
– hc HierarchyCache
– fd FolderData (used e.g. for ping)
– fs FailSave data (used by incoming
loop protection)
– bs Backend storage (IMAP backend)
23. Ping Tracking
• concurrent PING connections
– iOS opens a new ping connection every time an ActiveSync item is
accessed
– These connections run up to 1 hour on the server and block a apache
process/thread and memory each
– Each ping connection registers an own sink or worse: does polling
• PingTracking solves this
– Active PING connections check regularly if newer ping processes are
available. If so, the older connections terminates.
– This is done every refresh cycle
(default 30 seconds - parameter PING_INTERVAL in config.php)
– Only one ping connection stays active
25. Upgrading 1.5 -> 2.0
• States are incompatible
– Old states should be removed or not be copied
– All the devices need to fully resynchronize
– May cause some higher server load
– May take some time (amount of users + items)
• All data is automatically removed from the mobile
• Data is then resynchronized
• Some users will see errors
– Device sometimes needs “time” and several sync cycles
– Users should press “refresh” repeatedly
– On iOS restart mail app or reboot device helps
26. Upgrading steps
• Install & configure z-push 2
– Set timezone
– Check/Create state and log folders
– State and log folders need to be writeable for apache
• Add dummy configuration to apache for testing & reload
• Test in browser
– Check for errors in /var/log/z-push/z-push.log
• Update apache config & reload
• Check progress with z-push-top
28. What usually goes wrong
• Missing writing permissions to state & log files
– Synchronization will not occur, errors will be in log file
• Missing packages
– Z-Push 2 uses shared memory for some operations
– Some OS like RHEL need package php-process
– Z-push-top requires php5-pcntl on some systems, e.g. SUSE
• Appointments are listed with wrong time on mobiles
– Z-Push server timezone is not set
– Timezone on mobile is not set or wrong
• High latency
– If ZCP server is “too far away” connections can time out
29. Next step ?
• Z-Push 2.0 released today !
Give it a try!
Better/More New
• Control • Features
• Flexibility • Tools
• Manageability • Supported devices
• Compatibility • User friendlier (profiles)
• Performance
30. Z-Push 1.5 Support
• Complementary support until 31.12.2012
• Upgrade support afterwards
• Be aware that with Z-Push 1.5 more BYOD
incompatibilities
are to be expected
• Currently Android 4 &
BlackBerry Playbook
are already not
supported with 1.5.x
31. Where does it go?
• MDM plugin for Webapp with new features
– More synchronization status information
– Messages/objects which could not be synchronized
– Open shared folders (contacts & calendars only on iOS)
• Planning 2.x:
– Encrypted & signed emails
– Auto-discovery
– Policy management tool
• Packaging
– Easier installation & upgrade
32. Keep in touch
• Announcements on Zarafa Release & Beta list
Subscribe at http://zarafa.com/newsletter
• Public Bug tracker: http://z-push.sf.net/tracker
• Latest downloads: http://z-push.sf.net/download
(beta & finals)
• Sources at BerliOS: https://svn.berlios.de/wsvn/z-push/
• Compatibility list: http://z-push.sf.net/compatibility
Always up-to-date list about known devices and potential compatibility
issues.
