Presentation of Sebastian Kummer about debugging Z-Push at Zarafa SummerCamp 2011

1. Z-Push: advanced debugging
Sebastian Kummer

2. Debugging Z-Push
Most of the time you don't see issues immediately!
Debugging is not that easy:
You need logs, they will be

3. What “usually” goes wrong – States
Problems with states
•
missing permissions
•
missing states after upgrading
States represent the synchronization state of mobile & folder on server
Missing states mean:
•
Server “does not know” what is already on the mobile, sends
EVERYTHING again → duplications on mobile
•
Some stuff (like sending, creating & editing items) “works” somehow
•
Could cause loops
In Z-Push 2 a mobile will be fully resynchronized when states are missing
or invalid → missing states for many users could cause HIGH LOAD!

4. What “usually” goes wrong – Old versions
Old versions – especially 1.2.x and 1.3.x:
•
Compatibility issues
•
Fields not being synchronized
•
Some invitations/recurrences broken (also with 1.4.x)
•
Unimplemented features
•
Broken encodings - “Vielen Dank fÃŒr Ihre – Reservierung”
•
Nokia issues (duplicated folders)

5. What “usually” goes wrong – Loops
Loops
•
Creating 1000000(0000000000) duplicates
•
High load on server
•
High use of mobile bandwidth (could $$$$)
Loops are nasty and dynamic! Requests are triggered BY THE MOBILE.
Most common causes:
•
Broken items (most of times calendar, e.g. recurrences)
•
Timeouts (server takes to long to process request)
•
Fatal errors or Apache/PHP Segfaults (rarely)
•
Incompatibilities (mobile tries over and over again)

6. Approaching loops - I This is what you
see in the log
Mobile Server
Idle Has a change
Synch. request 1. Reads state
2. calculates delta
3. exports change
Retry
4. MAPI → ActiveSync
5. streams change
Waits for answer to mobile
Failure!
Timeout 6. writes new state
SEGFAULT
Idle
Any step (1-6) can fail!
Any step could take to long!
Segfault could occur!
time
Connection could fail (entering tunnel?)

7. Approaching loops - II This is what you
see in the log – looks great!
Mobile Server
Idle Has a 'broken' change
Synch. request 1. Reads state
2. calculates delta
3. exports change
Retry
4. MAPI → ActiveSync
5. streams change
Waits for answer to mobile
Success!
6. writes new state
Processes answer
Faulty item is sent to the mobile!
Mobile does not know how to
ERROR interpret item → error!
time

8. Approaching loops - II This is what you
see in the log – looks great!
Mobile Server
Idle Has a 'broken' change
'Broken' ...? 1. Reads state
Synch. request
2. calculates delta
Syntactical errors exports change
3.
Retry - Missing elementary properties, like subject
4. MAPI → ActiveSync
- Missing start or end date of change item
5. streams calendar
Waits for answer to mobile
Success! errors
Semantic 6. writes new state
- start date after end date
Processes answer - value not as expected (e.g. string vs. hex)
- Recurrence exception: the mobile!
Faulty item is sent to
- Mobile does not know how to
missing/faulty basedate
- interpret item → error! (syntax)
not a recurrence at all
time ERROR

9. 'Kamikaze loop' - III This is what you
see in the log – looks great!
Mobile Server
Has NEW item Has a ('broken') change
1. Reads state
Synch. request 2. creates item on server
3. calculates delta
Retry 4. exports changes
5. MAPI → ActiveSync
Success/failure!
Waits for answer 6. streams new ID
or timeout... 7. streams changes
Doesn't really 8. writes new state
matter...
Anything after 2 fails..
Anything else goes wrong
(previous slides)
time ERROR

10. Kamikaze user

11. How an iPhone receives an email
“ I can see the first lines of the mail in the preview, but when I open it
just says 'no body available'.....! “
iPhone Server
Idle Has a change
Only 1K of plain email
Synch. request
Retrieving and processing
Success
Waits and processes
Fetch with HTML
Fetch entire mail 1. Retrieving
Failure 2. Processing as RFC822
Processing
SEGFAULT
Idle
time

12. How to fix?
Error checking in loops
•
some syntax checking
•
only few semantics checks possible
•
Force lower maxItems
•
Logging on user level
•
New log levels, incl. Deviceid (when synching
several devices with same account

13. Questions?
Thank you for your attention!
More information at:
http://z-push.sf.net/forums
http://z-push.sf.net/tracker
http://z-push.sf.net/download
http://z-push.sf.net/compatibility
Contact: sebastian@zarafa.com