The document discusses monitoring IBM Notes Traveler. It provides details on using the "tell traveler status" and "tell traveler stat show" commands to check the status and view statistics of the Traveler server. The statistics can help analyze issues like CPU usage, mail database open times, and connection latencies. The document also mentions that some statistics are available in Domino Administrator to monitor Traveler.

1. 1
IBM Notes Traveler Daily Business
engage 2015
René Winkelmeyer
midpoints GmbH

2. 2
René Winkelmeyer
Head of Development
midpoints GmbH
http://www.midpoints.de
IBM Advanced Business Partner
IBM Design Partner (Notes Domino, Mobile, Verse)
Apple Enterprise Developer und MDM Group Member
Samsung Enterprise Alliance Partner
Services
- Enterprise Mobility Service
- Mobile Device und Application Management
- IBM Notes Traveler und IBM Mobile Connect
About me

3. 3
• Skype
muenzpraeger
• Twitter
muenzpraeger
• LinkedIn
muenzpraeger
• Slideshare
muenzpraeger
• Web
http://blog.winkelmeyer.com
http://www.midpoints.de
• Mail
mail@winkelmeyer.com
rene.winkelmeyer@midpoints.de
OpenNTF
• File Navigator
• Generic NSF View Widget for IBM Connections
About me
René Winkelmeyer
Head of Development

4. 4
Agenda
§ Administration basics
§ Monitoring
§ Tuning
§ Troubleshooting
§ NEXT

5. 5
Agenda
§ Administration basics
§ Monitoring
§ Tuning
§ Troubleshooting
§ NEXT

6. 6
Administration Basics – High Availability
IBM Notes Traveler allows two operating modes:
§ Standalone Traveler Server
§ LotusTraveler.nsf
§ Local Java Derby Database
§ High Availability (HA) Traveler Server Pool
§ Traveler-“Cluster“
§ Remote RDBMS (IBM DB2 or M$ SQL)

7. 7
Administration Basics – Big Picture Traveler
Traveler Server
HTTP
TASK
Traveler
OSGi
SERVLET
Traveler
TASKSSL
Domino
Directory
Server
Config LotusTraveler
Default
Settings
Derby
Rel.-DB
/ntsdb
Notes
notes.ini
Domino
& HTTP
Security
SQL

8. 8
Administration Basics – High Availability
§ Benefits of a HA environment
§ Better scaling (Standalone should operate to max. 2.500
devices)
§ Failover capabilites, no downtime of Traveler services
§ Servers can be updated any time (there‘s never a good time for
downtime...)

9. 9
Administration Basics – High Availability
Domino Mail
Domino Mail
Domino Mail
Traveler
Server
HTTPS
Notes
Traveler
Server
DB2 / SQL Server
DB2 / SQL Server
DB2/SQL
HTTP(S)
IBM Notes
Traveler
Service Pool
Load Balancer
IP-Sprayer
Reverse Proxy
(i.e. IBM Mobile Connect)
DB2 / SQL
Backend
Server in HA

10. 10
IBM Notes Traveler – High Availability
§ Two or more Domino servers with installed Traveler addon work in a
so called „Traveler Service Pool“
§ All pool members use the same state database
§ The state database is centrally organized (IBM DB2 or M$ SQL)

11. 11
IBM Notes Traveler – High Availability
What‘s a pool?

12. 12
IBM Notes Traveler – Traveler Service Pool
§ Every user can be served by every Pool member server
§ All Pool member servers are equal (in terms of service)
§ Travelers own Availabiltiy Index (AI) is used for internal Load
Balancing using TCP ports 50125/50126
§ Each user is bound to a single server within the pool for his Master
Monitoring Sesssion (MM or User Session)
§ This Master Monitor Server is responsible for mail database
synchronisation

13. 13
IBM Notes Traveler – HA Load Balancing
tell traveler HADR show
Domino ID Host IP:SrvrPort,SrvltPort Alive Server Servlet Last HB AI Users Devices
L1/NETWORK 330 s1.network.com 10.3.1.1:50125,50126 true true true 2014-08-26 96 2315 1179
L2/NETWORK 337 s2.network.com 10.3.1.2:50125,50126 true true true 2014-08-26 100 556 1102
L3/NETWORK 585 s3.network.com 10.3.1.3:50125,50126 true true true 2014-08-26 99 1630 1140
L4/NETWORK 580 s4.network.com 10.3.1.4:50125,50126 true true false 2014-08-26 100 0 346
L5/NETWORK 505 s5.network.com 10.3.1.5:50125,50126 true true true 2014-08-26 100 311 1106
AI = Traveler internal Availability Index (AI)
Users = Master Monitor Session / one per user
Devices = HTTP Sessions (Devices last seen) / one session per device
Note: L4/NETWORK has been restarted

14. 14
IBM Notes Traveler – HA Load Balancing
• Traveler Availability Index (AI)
• Master Monitor Server (MM) per user
• User Load Balancing Bias
+ 10 Bias for local server
+ 20 Bias for current MM
• Load Balancing algorithm
• AI calculated per server
• Chooses the highest AI (incl.
Bias) and defines that server as
MM
• All devices (of this user) are
routed to the current MM
• The MM isn‘t allowed to re-
balance within 10 minutes
HTTP –Task
Servlet
HTTP –Task
Servlet
Traveler –Task
AI = 75
Traveler –Task
AI = 80
Server 1 Server 2
AI 75 + BIAS 10 + BIAS 20 > AI 80
AI 75 + BIAS 20 > AI 80 + BIAS 10
TCP 50125

15. 15
Administration Basics – Webfrontend
Starting with 8.5.3. UP1 Traveler administration is done by using a
XPages application

16. 16
Administration Basics
The webfrontend is unfortunately not sufficient – you‘ll often need the
Domino server console for troubleshooting

17. 17
Administration Basics
Current user status – tell traveler user <user name>
tell traveler user Detlev Poettgen
CN=Detlev Poettgen/O=midpoints does not have sufficient access rights to the
database mail/dpoettge.nsf.
…
tell traveler user rene.winkelmeyer@midpoints.de
IBM Notes Traveler has validated that it can access the database mail/rwinkelm.nsf.
Encrypting, decrypting and signing messages are not enabled because the Notes ID is
not in the mail file or the ID vault.
……

18. 18
Administration Basics
IBM Notes Traveler has validated that it can access the database mail/rwinkelm.nsf.
Encrypting, decrypting and signing messages are not enabled because the Notes ID is not in the
mail file or the ID vault.
Canonical Name: CN=Rene Winkelmeyer/O=midpoints
Internet Address: rene.winkelmeyer@midpoints.de
Master Server: Traveler01/srv/midpoints-trav, version 9
Master Server Locked: Jan 21, 2015 11:05 AM, type=Soft
Home Mail Server: CN=mail01/OU=srv/O=midpoints
Home Mail File: mail/rwinkelm.nsf
Current Mail Server: CN=mail01/OU=srv/O=midpoints Release 9.0
Current Mail File: mail/rwinkelm.nsf
Mail File Replicas: [CN=mail02/OU=srv/O=midpoints, mail/rwinkelm.nsf], [CN=mail01/OU=srv/
O=midpoints, mail/rwinkelm.nsf]
ACL for Rene Winkelmeyer/midpoints: Access=Editor
Capabilities=create,update,read,delete,copy Missing Capabilities=none
ACL for Traveler01/srv/midpoints-trav: Access=Manager
Capabilities=create,update,read,delete,copy Missing Capabilities=none
Notes ID: Mail File does not contain the Notes ID.
Auto Sync User State: Monitoring disabled
Last Prime Sync: Monday, Jan, 2015 2:31:11 PM CEST
Banned Documents: 0…

19. 19
Administration Basics
Devices:
Device ID: ApplC38JCFABDTWG
Device Description: ApplC38JCFABDTWG
Security Policy Status: No policy
Security State: Clear
Approval State: Not required
Last Sync: Never
Auto Sync Device State: Inactive
Device offline time: Monday, Jan 20, 2015 2:39:42 PM CEST
Auto Sync Connection State: Disconnected
Auto Sync Applications to Synchronize: folder, mail, calendar, contact, serviceability, security
Auto Sync Change Flags: folder:add, mail:add(4:add), serviceability:configGet/configSet

20. 20
Administration Basics
Pipe command output to file – tell traveler –f <filepath> user <user
name>
Watch out:
Command leaves zero byte validation file .sem in the directory which
doesn‘t get cleaned up automatically.
tell traveler –f /tmp/user.txt user rwinkelm
Output for command ‘-f /tmp/user.txt show rwinkelm’ can be found at /tmp/user.txt.
[root@incinerate tmp]# ls -lrt
insgesamt 3240
-rw-rw-r--. 1 domino domino 424 20. Jan 13:19 user.txt
-rw-rw-r--. 1 domino domino 0 20. Jan 13:19 user.txt.sem

21. 21
Administration Basics
Get SQL content via Domino console – tell traveler sql „<QUERY>“
tell traveler sql "SELECT HOSTNAME FROM TS_GLOBAL“
[04683:00036-3357230848] Command 'SELECT HOSTNAME FROM TS_GLOBAL' was
completed successfully.
[04683:00036-3357230848] HOSTNAME |
[04683:00036-3357230848] incinerate.midpoints.net |
[04683:00036-3357230848] hellfire.midpoints.net |
[04683:00036-3357230848] aryastark.midpoints.net |

22. 22
Agenda
§ Administration basics
§ Monitoring
§ Tuning
§ Troubleshooting
§ NEXT

23. 23
Monitoring – Server-Task
Notes Traveler is part of the „Server Tasks“ within Domino
Administrator (added in 2013, check if your domadmin.nsf is updated)

24. 24
Monitoring
You‘ll find the default statistic values for yellow/red statuses in an IBM
wiki
§ http://www-10.lotus.com/ldd/dominowiki.nsf/dx/
Status_command_considerations_and_examples_LNT853
Traveler server status – tell traveler status
tell traveler status
The IBM Notes Traveler task has been running since Tue May 14 12:31:09 BST 2014.
The IBM Notes Traveler availability index is currently 100 while servicing 431 users.
The last successful device sync was on Sat Jan 22 15:05:15 BST 2015.
The overall status of IBM Notes Traveler is Green.

25. 25
Monitoring – Statistics
§ Issuing tell traveler stat show on the Domino console brings you all
Traveler statistics
§ http://www-10.lotus.com/ldd/dominowiki.nsf/xpDocViewer.xsp?
lookupName=Administering+IBM+Notes+Traveler
+9.0.1#action=openDocument&res_title=System_stat_results_A
901&content=pdcontent

26. 26
Monitoring – Statistics
Statistics are also available using Domino Administrator. Some are very
useful – some have only an informational level.

27. 27
Monitoring – Statistics
Analyzing connection issues and latencies
tell traveler stat show
[0A8C:0068-0924] Availability.Index.060-070 = 1
[0A8C:0068-0924] Availability.Index.090-100 = 48
[0A8C:0068-0924] Availability.Index.Current = 100
….
[0A8C:0068-0924] DCA.DB_CLOSE = 54
[0A8C:0068-0924] DCA.DB_OPEN = 41
[0A8C:0068-0924] DCA.DB_OPEN.Time.Histogram..000-001 = 37
[0A8C:0068-0924] DCA.DB_OPEN.Time.Histogram.CN=Traveler01/OU=srv/
O=midpoints-trav.000-001 = 4
….

28. 28
Monitoring – Statistics
§ CPU.Pct.070-080
CPU.Pct.080-090
CPU.Pct.090-100
§ Will be set when the CPU usage is within the defined range (i. e.
70-80% of the first parameter)
§ Important parameter as Traveler performance can be affected
heavily if CPU usage is 78% or more.

29. 29
Monitoring – Statistics
§ DCA.C.DB_OPEN.Time.Histogram.<server>.<bucket>
§ A „bucket“ defines the time in seconds which was needed to
open a mail file.
§ This statistic helps to analyze latencies and connection issues to
remote mail servers.
tell traveler stat show
….
[0A8C:0068-0924] DCA.DB_OPEN.Time.Histogram..000-001 = 37
[0A8C:0068-0924] DCA.DB_OPEN.Time.Histogram.CN=Traveler01/OU=srv/
O=midpoints-trav.000-001 = 4
….

30. 30
Monitoring – End-To-End
§ All shown mechanisms for analyzing Traveler health and status are
missing some key elements:
§ Is Traveler available from external and can a device
synchronize?
§ Are all involved components available (i. e. Internet connectivity,
Firewall, Load Balancer, Reverse Proxy, Network to mail
servers, mail server itself etc.)
§ An administrator needs to know issues before the CxO calls during
BBQ.

31. 31
Monitoring – End-To-End
§ Check if Traveler infrastructure works from external
§ HTTP(S) request to /traveler?action=getStatus
§ HTTP response code 200 mean: Traveler is available
§ Validations
§ Traveler access (Load Balancer, Proxy, authentication)
§ HTTP task active
§ Traveler task active
§ Mail server reachable
§ Traveler database (SQL backend) available

32. 32
Monitoring – End-To-End

33. 33
Monitoring – End-To-End
§ Traveler Mail Delivery Confirmation Message
§ Send a mail to a specific mail account (which is set on a device)
§ Device fetches mail via Traveler
§ The sender will receive a confirmation mail if the mail has been
delivered to the device
§ If the sender doesn‘t get the confirmation mail – Houston, we
have a problem!

34. 34
Monitoring – End-To-End
§ Traveler Mail Delivery Confirmation Message
§ Available since Traveler 9.0.0.1 IF2
§ Must be explicitly activated via notes.ini
NTS_MAIL_DELIVERY_CONFIRMATION=true
§ Allowed senders can be explicitly set
NTS_MAIL_DELIVERY_CONFIRMATION_SENDERS=MonitorM
ail1/Company

35. 35
Monitoring – End-To-End
§ Traveler Mail Delivery Confirmation Message
§ Mail-Subject must begin with <$Confirm>
§ Additional keywords/combinations:
<$Confirm,RemoveOnDelivery>
<$Confirm,SuppressSaveInSentItems>
<$Confirm,RemoveOnDelivery,SuppressSaveInSentItems>
§ http://www-10.lotus.com/ldd/dominowiki.nsf/dx/
Using_Mail_delivery_confirmation_messages_with_IBM_Notes_
Traveler

36. 36
Monitoring – Port checks
§ 80 / 443 - HTTP/HTTPS
§ 50125 - Communication
Servlet => Traveler task
Traveler task => Traveler task
§ 50126 - Communication
Traveler task => Servlet
§ Port checks should always be combined with further/other checks.
§ Port 50125 may i. e. answer but the Traveler task may not reach
the SQL backend

37. 37
Monitoring – Know your devices
Knowing your devices (os, os version type etc) is critical.

38. 38
Agenda
§ Administration basics
§ Monitoring
§ Tuning
§ Troubleshooting
§ NEXT

39. 39
Tuning
§ Always watch the amount of devices in your infrastructure and
update your configuration accordingly.
§ Important parameters are i. e.
§ CPU and RAM
§ HTTP threads
§ Memory cache
§ Maximum memory size
§ Address cache
§ Request size

40. 40
Tuning
Assign CPU and RAM more then needed. 64bit is always
recommended.

41. 41
Tuning
§ HTTP threads => 1,2 * number of devices per server
(default: 100 32bit / 400 64bit)
§ Watch out: RAM for all threads will be allocated a HTTP task start

42. 42
Tuning
§ Maximum cached users:
Number of users per server
§ Cached user expiration level:
Recommendation: 28.800 sec (8h)

43. 43
Tuning
§ Maximum Memory Size: varies depending of the user/device
numbers
§ Should be minimum 1.024 MB (rule of thumb: always ¼ of available
memory)

44. 44
Tuning
Memory – tell traveler mem
CPU and Memory (MB) Usage History
Date CPU Pct Java Mem C Mem Avl Indx # Users # Errors # DB Co
2015-01-02 15:01:53 BST 0.01 56 1307 100 5 1 0
2015-01-02 15:16:53 BST 0.01 44 1307 100 5 1 0
2015-01-02 15:31:53 BST 0.01 73 1307 100 5 1 0
2015-01-02 15:46:53 BST 0.01 39 1306 100 5 1 0
2015-01-02 16:01:53 BST 0.01 53 1306 100 5 1 0
2015-01-02 16:16:53 BST 0.01 66 1307 100 5 1 0
2015-01-02 16:31:53 BST 0.01 87 1307 100 5 1 0
…
Current Memory Usage
Java Memory Usage
Max Total 1024 MB
Current Total 96 MB
Free 940 MB (92 percent of Max Total)
Allocated 84 MB (8 percent of Max Total)
C Memory Usage
Allocated 1293 MB (33 percent of Total Physical)
Current Usage
Java 84 MB
C 1293 MB

45. 45
Tuning
§ Traveler performance varies depending on multiple factors like mail
database size and sync filter settings.
§ You‘ll see that in the size of the Derby / SQL database
§ 850 User, 500 MB quota, no filter => 1 GB
§ 850 User, no quota (Ø 2 GB), no filter => 5 GB
§ 2.000 User, no quota (Ø 1,5 GB), filter ”1 year” => 4 GB
§ Largest seen Derby database: 17 GB (1.000 users, no filter)
U * (D * (700 * V + 4000)) = Recommended DB space available in
bytes

46. 46
Tuning
Usage – tell traveler dbusage
tell traveler dbusage
IBM Notes Traveler Database Statistics
Accounts: 11
Devices: 23
Total device documents: 35307
Device documents synced: 12124
Device documents filtered: 23183
Domino documents: 11056
Highest Total Usage Documents Percentage
------------------------------------------------------------------------------------------------
Rene Winkelmeyer/midpoints 7229 65.39
Petar Petrov/midpoints 1471 13.30
Detlev Poettgen/midpoints 1302 11.78
Michael Ingendoh/midpoints 819 7.41
Benjamin Gaisser/midpoints 235 2.13

47. 47
Tuning
Usage – tell traveler dbusage
Mail documents: 14321
Highest Mail usage Documents Percentage EMail filter
------------------------------------------------------------------------------------------------------------
Rene Winkelmeyer/midpoints 6773 72.66 unlimited
Petar Petrov/midpoints 1254 13.45 unlimited
Detlev Poettgen/midpoints 743 7.97 30 days
Michael Ingendoh/midpoints 399 4.28 14 days
Benjamin Gaisser/midpoints 152 1.63 30 days
Calendar documents: 10085
…

48. 48
Tuning

49. 49
Tuning
§ Standalone (Derby)
§ Frequent defragementation
§ HA (IBM DB2 / M$ SQL)
§ Frequent Runstats for table and index status
§ Index ReOrg => TALK TO YOUR RDBMS-ADMIN
§ Translog check (size and storage)

50. 50
Tuning – DON‘T DO THIS AT HOME
§ Get all Traveler configuration parameters – tell traveler config
ADMINP_POLL_INTERVAL = 0
DEBUG_OUTFILE = "”
LOGFILE_DIR = "”
NTS_64_BIT = false (default = true)
NTS_ACCESS_ALLOW = "*:TravelerUsers" (default = "")
NTS_ACCESS_DENY = "”
NTS_ACCESS_ENABLED = true
….
NTS_FONT_CONVERSION = 0
NTS_FORCE_GC_MEMORY_LEVEL = 5
NTS_FORCE_OUTBOX_MAIL_TO_DRAFTS = false
NTS_FORCE_START = false
NTS_HOST_IP_ADDR = "11.11.11.100" (default = "")
NTS_HTTP_HEADERS_RESPONSE_X_IBM_TRAVELER_HOST = ""
NTS_IGNORE_TIMEZONE_ERROR = false
NTS_INSTALLATION_TYPE = ON_PREMISE
NTS_INSTALL_INSTANCE = ""
NTS_INTERNAL_BATCH_UPDATES = false
…

51. 51
Tuning – DON‘T DO THIS AT HOME
§ Nearly 800 (mostly undocumented) parameters!!!

52. 52
Tuning – DON‘T DO THIS AT HOME
§ You should change them only if you they are documented or IBM
support tells you to use them
§ NTS_AUTOSTART_HTTP
§ NTS_PUSH_APNS_SERVER
§ NTS_ROUTE_LOCAL_BIAS
§ NTS_STATUS_DATA_DIR_FREE_GIGABYTES_RED

53. 53
Agenda
§ Administration basics
§ Monitoring
§ Tuning
§ Troubleshooting
§ NEXT

54. 54
Troubleshooting – Device setup
§ Can the device connect to the Traveler server?
§ Can the device open the Traveler website (/traveler)?
§ Can the user authenticate himself (wrong password)?
§ Is Internet Lockout active (you should use it if you don‘t have a
secure reverse proxy in front of Traveler)?
§ Is the user allowed to use the Traveler server?

55. 55
Troubleshooting – Device setup
§ Can the Traveler server connect to the user‘s mail server?
§ Is the Traveler server allowed to connect to the user‘s mail server?
§ Is a cross-certificate for the Traveler server missing (if Traveler is
hosted in another domain)?
§ Has Traveler Manager access rights (incl. Delete) for the mail
database?
§ Has the user Editor access rights (incl. Delete) for the mail
database?

56. 56
Troubleshooting – Device setup
§ Is „Replication of Unread Marks“ set in the mail database
properties?
§ Has the mail database quota been reached (Traveler creates/uses
two profile documents in the mail database)?

57. 57
Troubleshooting – Logs
§ Central Log Directory
IBM_TECHNICAL_SUPPORT/traveler/logs
§ Enable Logging per User
tell traveler log adduser finest <username>
tell traveler log removeuser <username>
§ Dump user information
tell traveler dump <username>

58. 58
Troubleshooting – Logs
§ Collect Information for a PMR and upload
tell traveler pmr <pmr_number>
§ If you cannot do that use the following
tell traveler systemdump
tell traveler log collect
Check IBM_TECHNICAL_SUPPORTtravelerlogs<timestamp>

59. 59
Agenda
§ Administration basics
§ Monitoring
§ Tuning
§ Troubleshooting
§ NEXT

60. 60
NEXT – IBM Verse iOS App

61. 61
NEXT – IBM Verse iOS App

62. 62
NEXT – FreeTime-Search on iOS

63. 63
NEXT – More stuff
§ Trash folder synch on iOS
§ GCM Push Notifications for Android
§ ...

64. 64
Q & A!