Would your users like their IBM Notes client to perform faster? Do some applications and clients seem to load slowly? Re-written for IBM Notes 9, learn where to look to find out what’s wrong - and resolve it. Find out how to debug your client, deal with outdated ODS, network latency and application performance issues and more importantly understand why you should care. Gather best practices on how to streamline location and connection documents and why the catalog.nsf is so important. Improve your IBM Notes client installations to provide a better experience for happier administration and happier end users! And last but not least, Christoph will show you how to increase the startup time of virtualized IBM Notes Clients (Citrix / vmware / etc) up to 70%.

1. A Performance Boost for your IBM Notes Client
DanNotes 2014

2. 2
Christoph Adler
Technical Account Manager – panagenda
IBM (Lotus) Notes / Domino since 2001
Consulting for projects
 Administration
 Migration / Consolidation
 Client Management
 Application Management
Since 2012 TAM at panagenda with core competence in
 Notes Client Management
 Domino Infrastructure Analysis and Optimizations
[...]
 Frequent traveler
 Projects with many different companies in different
countries

3. 3

4. 4
Agenda
A Performance Boost for your IBM Notes Client
Your IBM Notes Client: Version 9.0.1
What makes your Notes Client START slow
What makes your Notes Client PERFORM slow
Performance Tips and Tricks

5. 5
Before we start ...
I love Notes.

6. 6
Your IBM Notes Client: Version 9.0.1 FP2
> 20.000 41.000 files (Notes 6/7 | 8/9 Basic used to have ~550) = +7.400%
 17.000 21.800 files in Notes Programframework in 1.800 2.150 subdirectories
 7.500 8.800 (largely undocumented) .properties files
 1.500 1.600 jar files
 1.200 1.300 HTML files
 3.000 1.182 files in Notes Dataworkspace in 2.400subdirectories, out of which 1,850 are empty 700 subdirectories
 900 180 (largely undocumented) XML files
 100+ 80 (largely undocumented) .properties files
 Which is
 20.000 41.000 files in 4.600 4.860 subdirectories
 8.500+ 9.000+ largely undocumented files
 Many (many many) subdirectories are plain empty, especially in Dataworkspace
 ... and around 100 NSFs/NTFs (=0.5 0.2 %)
 ... and the good old notes.ini file

7. 7
Your IBM Notes Client: Version 9.0.1 FP2 - Continued
Add over 80 preference/dialog boxes
• Each dialog has ~15 settings at average
For a total of 1,200+ settings sprayed across
• address book (very few)
• notes.ini (few)
• and XML files (lots)
Plus any plugins, widgets,
add-ons, etc. “you“ or the user
may have added to the Notes
client install

8. 8
Your IBM Notes Client: Version 9.0.1 FP2 - Continued
Search
Sidebar
Shortcut
Buttons
Toolbars
Bookmarkbar

9. 9
Your IBM Notes Client: Version 9.0.1 FP2 - Continued
Workspace
(nsf)
Physical nsfs 1, 2, 3, … n

10. 10
Agenda
A Performance Boost for your IBM Notes Client
Your IBM Notes Client: Version 9.0.1
What makes your Notes Client START slow
What makes your Notes Client PERFORM slow
Performance Tips and Tricks

11. 11
What makes your Notes Client START slow
If you‘re not on at least 9.0.1 yet  9.0.1 is a significant improvement
 Pre-loading the client might also be recommended
EOM of IBM Lotus products
 http://www-01.ibm.com/software/lotus/support/lifecycle/

12. 12
What makes your Notes Client START slow - Continued
~50% of the entire data volume in the Data
directory changes on a DAILY basis in
most setups
 5 - 10% in Dataworkspace ( up to 300 files)
 DISABLE Antivirus scanning for at least *.ns* in
Data Directory. Period.

13. 13
What makes your Notes Client START slow - Continued
Client crashes/database fixups
 especially bad for network based data directories clients
Outdated Hardware
 You need >1-2 GB memory if you want to use Notes AND something else
 Old fixed disks with fragmentation
 Fun Exercise: Buy a small Solid State Disk and make Notes boot off of that ;-)
Tons of startup programs/scripts launching during the “boot storm”
 if the disk is unable to keep up, it‘s not Notes' fault
 Again, see http://bit.ly/ACzO6Z

14. 14
What makes your Notes Client START slow - Continued
Outdated Data directories
 Virtually 100% of all NTFs (Laptops: 90%) can be removed by
 Cleaning up old Notes 4+5+6 templates
 using ini:SharedDataDirectory= (also with Single User install!)
Operating System
 Windows 7 boots ~40% faster than XP
 Windows 8 ~55% faster than XP
 ~20% faster than Windows 7, supposedly

15. 15
What makes your Notes Client START slow - ODS
ODS = On Disk Structure
 ODS 16 = Notes 2
 ODS 17 = Notes 3
 ODS 20 = Notes 4 (oder Templates)
 ODS 41 = Notes 5
 ODS 43 = Notes 6 & 7
 ODS 48 = Notes 8
 ODS 51 = Notes 8.5 & 9
 ODS 52 = Notes 9.0.1
The difference between ODS 43 and 51 = up to 80% LESS FILE I/O
This helps with ANY client AND server hardware

16. 16
What makes your Notes Client START slow – ODS - Continued
CAUTION: The ODS is not automatically updated when upgrading clients
 Except for names.nsf, bookmark.nsf, and desktop*.dsk/ndk – IF – you upgrade properly
(think ini:Setup/TemplateSetup=)
 ODS 48 requires Notes >= 8 and CREATE_R8_DATABASES=1 in notes.ini for NEW db's
 ODS 52 requires Notes >= 9.0.1 and CREATE_R9_DATABASES=1 in notes.ini for NEW db's
Good News for Notes > 8.5
 Use NSF_UpdateODS=1 with CREATE_R85/9_DATABASES=1 for a one-time upgrade of most local databases
 NOTE: USE WITH EXTREME CARE IF YOUR DATA DIRECTORIES ARE ON A NETWORK DRIVE!
 NOTE: End users can't access databases during compact

17. 17
What makes your Notes Client START slow – ODS - Continued
Impact examples
 Startup time of a Notes 8.5.2 client with 3 ODS 20 apps in Notes data = 10 seconds
 After ODS upgrade: 2 seconds
 Reduced File I/O of your disks/SAN/NAS after ODS 41 to 51 upgrade by 60%
 Removing 70% of all old files in Data directories on SAN/NAS = 45% less managed storage (backup)
The bad news: There is no way to tell what is “out there”
 3rd party tools or random inspections are highly recommended

18. 18
IBM is helping!

19. 19
What makes your Notes Client START fast
Eclipse Framework Performance Improvement
Speeds up starting the IBM Notes client up to 65 %
ONLY usable in virtual environments (e. g. Citrix XenApp, etc.)
ONLY usable with a IBM Notes Roaming Solution

20. 20
What makes your Notes Client START fast - Continued
Eclipse Framework Performance Improvement
Example: Configuration of an IBM Notes 9.0.1 FP2 Client
1. With a non-admin user (OS + Notes) run an initial setup of the Notes client.
2. Start the Notes client 3x and wait then wait for each 2-3minutes
3. Shutdown the Notes client
4. Create the the following directory  c:ProgramDataIBMNotesDataCommon
5. Copy the workspace directory from %LocalAppData%IBMNotesData to
c:ProgramDataIBMNotesDataCommon
6. Delete the following files/directories out of c:ProgramDataIBMNotesDataCommonworkspace
a) UDM
b) Logs  all but .prov2install
c) cache  if available
d) .metadata.pluginscom.ibm.collaboration.*
e) .metadata.pluginscom.ibm.rcp.personality.framework
f) .metadata.pluginscom.ibm.rcp.security.auth

21. 21
Live demo

22. 22
Agenda
A Performance Boost for your IBM Notes Client
Your IBM Notes Client: Version 9.0.1
What makes your Notes Client START slow
What makes your Notes Client PERFORM slow
Performance Tips and Tricks

23. 23
What makes your Notes Client PERFORM slow
ODS of databases (again)
Hardware (again)
Notes.ini
 EXTMGR_ADDINS = (not just) local virus scanners
 Disable Notes “plugins“ like virus scanners etc.
 they increase Network traffic by up to 200 – 250%!
Data directory on the network vs.. fixed disk
 Disk performance is the #1 bottleneck (think SSD vs. normal HDD ; 7.200rpm vs. 5.400rpm ;
network drive vs. local fixed disk = latency/bandwidth limitation)

24. 24
What makes your Notes Client PERFORM slow – Port settings
Port settings are included in the notes.ini
 Type / Active / Encryption / Compression / Timeout
Bad idea
 LAN0 / COMx / DisabledPorts  should be removed
Good idea
 Use TCPIP according to the client type
 PORTS = TCPIP
 TCPIP = TCP,0,15,0,,12320,  Desktops
 TCPIP = TCP,0,15,0,,45088,  Laptops
 TCPIP = TCP,0,15,0,,12288,  Citrix / VDI
 Riverbed !?
Don‘t forget your Servers!

25. 25
What makes your Notes Client PERFORM slow - Continued
Install types: Basic vs. Standard
 Basic client starts up fast, Standard client, well … not so fast
Roaming Users
 Try replicating 80MB each time you logon to Notes before use!
Connection documents
 Pointing to outdated server names or IP addresses
 Low priority connection documents, which can cause connectivity failures
Location documents
 IP addresses for mailserver  breaks Policies
 Outdated mail and home server information
 Outdated mail file path
 Outdated Catalog server
 In general: catalog.nsf  see next slide

26. 26
What makes your Notes Client PERFORM slow – The catalog.nsf problem
User in Hamburg
Sends Doc|DB-Link to user in Frankfurt
Unfortunately, the user in Frankfurt does not
open the database from a local server
but from the one in Hamburg
... or even in Augsburg (Alphatbeth!)

27. 27
Agenda
A Performance Boost for your IBM Notes Client
Your IBM Notes Client: Version 9.0.1
What makes your Notes Client START slow
What makes your Notes Client PERFORM slow
Performance Tips and Tricks

28. 28
Make Notes faster = 1
Search for the file jvm.properties underneath [LotusNotesProgramDir]frameworkrcp (might
be in subdirectory ...deploy)
 Open it in a text editor
 First line to look for is: vmarg.Xmx=-Xmx256m
 Change it to e.g. vmarg.Xmx=-Xmx1024m (Maximum is ½ of physical memory; do NOT change it to more than 1024m
regardless of how much memory you have)
Change vmarg.Xms=-Xms48m to ¼ of what you changed Xmx to;
e.g. if you changed vmarg.Xmx to -Xmx1024m then use vmarg.Xms=-Xms256m
NOT recommended for production / normal end users; for admins & developers only!
 But in reality, who else counts ;)

29. 29
Performance Measuring - Client Clocking
NOTE: All of this should only be done on “ONE” client – slows down client, creates huge log
files!
Use the following to see and measure client performance
CLIENT_CLOCK=1
CONSOLE_LOG_ENABLED=1
Debug_Outfile=c:debug.txt
Debug_Console=1
DEBUG_TCP_ALL=1 (use only if needed)
Fileserver or local disk Performance Debugging is virtually impossible
If on >8.5 try adding DEBUGGINGWCTENABLED=16386 in Notes >= 8.5

30. 30
Performance Measuring - Client Clocking - Continued
Client_Clock=1 leads to a debug file of the following format:

31. 31
Performance Measuring - Client Clocking - Continued
OpenNTF – NRPC Parser
Link:
http://www.openntf.org/internal/home.nsf/project.xsp?action=openDocument&name=Notes%20RPC%20Parser
Developed by:
Jamie Magee, MartinScott Consulting LLC, US
Andrew Magerman, Magerman Collaborative Software GmbH, Switzerland

32. 32
Performance Measuring - Network Latency
Network Latency has a major impact on perceived client performance for users that (try to)
work on server-side databases
For example
• Number of NRPC calls x
(Latency Up + Latency Down) =
how long it takes for a client to
“talk something through”
with the server
• 30 Calls for opening the mailfile x
(60 ms download + 60 ms upload)
= 3.600 ms to talk through what's
necessary to open the mailfile

33. 33
Performance Tuning – Understanding cache.ndk
We've all been “advised to”/guilty of deleting the cache.ndk in an effort to cure Notes
performance problems/symptoms
 Some companies even delete cache.ndk on a scheduled basis
HOWEVER, let us explain the impact of that on Notes performance ...
 Increases traffic between client and server by 4,000% (forty times more traffic, yes)
(10,000+% if the data directory is on a network drive)
 The more databases end users use, the worse it gets
 There is always a root cause – deleting cache.ndk only fixes symptoms
(since 1992 I've had to delete my cache.ndk only once(!))
 Let's look at some detailed traffic analysis for just ONE application (the mail file) of ONE user ...

34. 34
Performance Tuning – Understanding cache.ndk - Continued
After deleting cache.ndk Keeping / NOT deleting cache.ndk
100 users =
150 MB vs. 3,7 MB
1000 users =
1.5 GB vs. 37 MB
Excluding Fileservers!

35. 35
Summary
Update/clean out your ODS, databases and templates and hardware
Standardize whenever and however possible
 This will save you hugely on support time/cost
Keep your users version/fix packs current
 Each release brings performance improvements but also resource demands
Work around your inability to see clients
 Write an inventory agent, login script, get a 3rd party tool
Any challenge can be overcome with Knowledge and Tools
 All client issues are solvable
 Be careful with thinking that removing the client/going to the cloud solves everything

36. 36
The end is near ...
Spending 30 minutes on
Performance Improvements can be compared to
a walk on the tip of the iceberg.

37. 37
Q & A

38. 38
Thank you !

39. 39
Christoph Adler
Technical Account Manager
panagenda GmbH – Solutions for Infrastructure Analysis and Optimization
● Lahnstraße 17 ● 64646 Heppenheim (Germany)
● Cell: +49 172 494 4141
● Phone: +49 6252 679 39-52 ● Fax: +49 6252 305 2842
● E-Mail: christoph.adler@panagenda.com
Contact