Speaker: Christoph Adler, Senior Consultant, panagenda Abstract: Virtualizing HCL Notes 11.0.1 FP1 (incl. Language Packs, Fix Packs, Connections Plugins for Notes and more) is an effective way to standardize your Notes client infrastructure, reduce costs for workstation hardware and give your users a consistent experience. In this session, you will learn how to implement, configure and tune HCL Notes 11.0.1 FP1 on platforms like Citrix XenApp/XenDesktop and get the most out of it by dramatically reducing start-up times (up to 70%), bringing a much better performance and increasing the stability into the Notes client. Beside a live demo on what we call “The Workspace (folder) Improvement” you'll also get some worst practices stories as Chris shares his experiences from real-world client virtualization projects, how these stories can help you and a detailed recipe on ”How to run the Installation and initial configuration of Notes in the best possible way for your very own infrastructure”.

1. Virtual, Faster, Better!
How to deploy HCL Notes 11.0.1 FP1 for CitrixVDI
Christoph Adler - panagenda

2. HCL Notes 11.0.1 FP1 Performance Boost Re-Reloaded

3. Christoph Adler - Senior Consultant at panagenda
19+ years of Lotus / IBM / HCL Digital Solutions experience
Focused on
• Notes Client Management (incl. HCL Nomad)
• Analysis and Optimization of Notes/Domino infrastructures
• Application Modernization
• Cloud Migrations
Good to know
• Obviously German (accent)
• Bass player & geek
• Loves music, football (soccer) and beer

4. https://www.slideshare.net/christophadler50

5. 1. Introduction
2. Roaming
3. HCL Notes 11.0.1 FP1 Installation
4. Live Demo - Workspace Improvement
5. Q & A
Agenda

6. Make Your Data Work For You
Introduction

7. What are virtual environments (in this case)
Examples
• VDI – Virtual Desktop Infrastructure
– VMWare Horizon View OR Citrix XenDesktop
• TS – Terminal Services/Server
– Citrix XenApp

8. What we typically see at customers
Aged Notes installations in virtual environments that are working “somehow”
– mostly Notes data directories on network drives which cause
• long startup times of HCL Notes (up to 60 sec)
• hangs and wait times due to interruptions and poor network latency
• high network traffic as HCL Notes is constantly reading / writing data
• frequently corrupted local applications
• permanently occupied disk space on file servers / SAN / NAS
– at least 120MB per user
– often because of old and outdated files, even templates

9. What we typically do at customers
• Sparking their enthusiasm for HCL Notes
– especially in virtual environments, HCL Notes can be optimized to start up fast, run
smooth and contribute to a positive end user IT experience by
• switching to local Multi-user installations
• frequently maintaining contents of HCL Notes data folders
• relieving network, file servers and/or SAN/NAS systems
• Encouraging administrators to spend a little more time on troubleshooting and
analyzing HCL Notes
– Solve root causes, not symptoms!

10. Make Your Data Work For You
Roaming

11. To roam or not to roam: that is the question
• Given the previous reasons why HCL Notes in virtual environments should
not have any files in the network, the following logical questions arise
– How can personal HCL Notes data folders of end users be maintained locally on the
virtual environment servers?
• Think of having N servers in the farm where personal end user data potentially
needs to be on every single server
• How do end users get their most recent HCL Notes configuration no matter
on which server in the farm they log on?
• How to ensure that disk space used by HCL Notes data folders is cleared
after log off?

12. To roam or not to roam: that is the question (cont.)
• How to avoid transfers of large HCL Notes data folders per user during log on and
log off which increase wait times and cause network traffic?
• The answer is: Roaming!
– To provide end users with their latest HCL Notes configuration, their data needs to be
roamed onto the server they log on to start their session
– Roaming can be achieved through own procedures (e. g. logon script),
Domino Roaming or third-party solutions

13. To roam or not to roam: that is the question (cont.)
Log on to
virtual session
Roam data
Start
HCL Notes
Log on to
HCL Notes
(happens here:
logon script,
3rd-party
solutions)
Roam data
(or here: MarvelClient
Roaming)
Work with
HCL Notes
Roam data
(or here:
Domino
Roaming)
Shutdown
HCL Notes
Log off from
virtual session
Backup
data
Backup
data
(happens here:
Domino Roaming
or
MarvelClient
Roaming)
(or here)
Clear data

14. Make Your Data Work For You
HCL Notes 11.0.1 FP1 Installation

15. Basic stuff
• Use HCL’s default paths
– Program directory
C:Program Files(x86)HCLNotes
– Shared data directory
C:ProgramDataHCLNotesDatashared
– Data directory
C:Users<USERNAME>AppDataLocalHCLNotesData
• MultiUser install only → please!
• Use Install parameter “CITRIX=1” (at least if TS involved)
• No need to make changes to MSI package (scripts can help here)

16. Command line(s)
• Uninstall & cleanup first
– HCL NICE (Notes Install Cleanup Executable):
https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0040768
"C:InstallNotesV11nice.exe" -rp –wipe –qn
– rd /S /Q "C:Program Files (x86)IBMNotes"
– rd /S /Q "C:Program Files (x86)IBMLotus"
– rd /S /Q "C:ProgramDataLotus"
– rd /S /Q "C:ProgramDataIBMNotes"

17. Command line(s)
• Install
– Notes Client 11.0.1
“C:InstallNotes_1101_Win_English.exe" -s -a /s /v"SETMULTIUSER=1 CITRIX=1 /qn /lime
"c:InstallNotes_1101_Win_English.log“
– Multilingual User Interface Group 1
“C:InstallNotes_1101_MUI_G1ML.exe" -s -a /s
/v"ADDLOCAL=BrazilianPortuguese,French,German,Italian,Japanese,Korean,SimplifiedChinese,Spanish,Traditi
onalChinese CITRIX=1 /qn /lime "c:InstallNotes_1101_MUI_G1ML.log“
– Fix Pack 1
“C:InstallHCL_Notes_11.0.1FP1_Win.exe" -s -a /s /v“METHOD=INSTALL CITRIX=1 /qn /lime
"c:InstallHCL_Notes_11.0.1FP1_Win.log“

18. ODS = On Disk Structure
• ODS 16 = Notes 2
• ODS 17 = Notes 3
• ODS 20 = Notes 4
• ODS 41 = Notes 5
• ODS 43 = Notes 6 & 7
• ODS 48 = Notes 8
• ODS 51 = Notes 8.5 & 9
• ODS 52 = Notes 9.0.1
• ODS 53 = Notes 10 & 11
The difference between ODS 43 and 53 = up to 80% LESS FILE I/O
ODS53

19. ODS53 (cont.)
To upgrade the ODS of all shared templates in a virtual HCL Notes Multi-User
installation you can follow these steps
(includes ODS 53 for new databases created by the user)
• Add the following line to the shared notes.ini in %ProgramData%HCLNotesData
– CREATE_R10_DATABASES=1
• Make sure that the Administrator account used to log on to the virtual server does
not have a personal HCL Notes Data directory yet (in %LocalAppData%)
• From a command prompt type the following commands
– cd “%ProgramData%HCLNotesDataShared”
– “%ProgramFiles(x86)%HCLNotesncompact.exe” -C -ODS -*

20. Notes.ini
• A properly configured shared notes.ini can be essential for smooth running virtual
HCL Notes environments
• The shared notes.ini is used by every HCL Notes client session for which there’s no
personal HCL Notes data folder yet
• It’s located in %ProgramData%HCLNotesData
• It is used as a template/stub notes.ini out of which all parameters are transferred
into a users personal notes.ini upon the very first HCL Notes start
• Find some useful shared notes.ini parameters on the next slide
(additionally to the standard entries)

21. Notes.ini (cont.)
• Ports=TCPIP
• TCPIP=TCP,0,15,0,,12288,
– Note that this sets port compression to OFF, which is appropriate in virtual environments; Port
compression adds 5% CPU overhead and with 60 concurrent sessions this would result in
needing 3 dedicated CPUs for just port compression
• DISABLE_CLIENTRECORD=1
– This disables logging of client information back into the public address book as all virtual
environment users share the same servers in the farm anyway

22. Notes.ini (cont.)
• CREATE_R10_DATABASES=1
– Enables HCL Notes V11 Beta-1 to use the latest ODS version for newly created local applications.
Make sure to not use CREATE_R85_DATABASES (IBM Lotus Notes 8.5) or
CREATE_R9_DATABASES (IBM Notes 9.0.1) to avoid the use of an older ODS in HCL Notes 11.0.1
• Log=,1,0,7,40000
– This disables logging, which is optional but gains a slight performance improvement
– This could cause issues when applications needs an existing log.nsf → please check before!
• ConfigFile=C:ProgramDataHCLNotesDataconfig.txt
– For ConfigFile details see next topic

23. Notes.ini (cont.)
All parameters for copy & paste
Ports=TCPIP
TCPIP=TCP,0,15,0
DISABLE_CLIENTRECORD=1
CREATE_R10_DATABASES=1
Log=,1,0,7,40000
ConfigFile=C:ProgramDataHCLNotesDataconfig.txt
[always keep a blank line at the bottom!]

24. Config File
• HCL Notes can be set up unattended by using a so called ConfigFile – often
referred to as config.txt or setup.txt
• The ConfigFile needs to be referenced in the shared notes.ini using the parameter
ConfigFile=Drive: or UNCfolderfilename.txt (see previous topic)
• For more details, the following HCL Knowledge Article provides some very good
examples on how to create a ConfigFile as generic as possible
− https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0036482

25. Config File (cont.)
Example ConfigFile for copy & paste
Username=%USERNAME%
Domino.Name=Hubserver/Acme
Domino.Address=servername.acme.com
Domino.Port=TCPIP
Domino.Server=1
AdditionalServices=-1

26. ShareClasses Improvement
• In HCL Notes the jvm.shareclasses file (*_xpdplat_.jvm_G41L00) is stored in each users
Dataworkspace.configorg.eclipse.osgi folder and has a fixed size of 64MB
• To optimize file I/O, speed up the HCL Notes Client start and save storage (960MB per
server == 15users), this file can be shared across all users of the virtual environment server
1. Create the following folder “%ProgramData%HCLNotesDataShareClasses“
and set folder rights for “everyone” to read / write access
2. Change the value of this parameter “jvm.shareclasses.loc=“ in
%ProgramFiles(x86)%HCLNotesframeworkrcpdeployjvm.properties
• from “${rcp.data}/.config/org.eclipse.osgi“
• to “C:/ProgramData/HCL/Notes/Data/ShareClasses“
→ it’s important to use “/” instead of “”

27. HCL Notes 11.0.1 FP1 - jvm.properties file
To speed up the HCL Notes JVM it’s useful to optimize memory settings
Open %ProgramFiles(x86)%HCLNotesframeworkrcpdeployjvm.properties
– Modify vmarg.Xmx from “-Xmx256m” to “-Xmx1024m”
– Modify vmarg.Xms from “-Xms64m” to “-Xms256m”
– Change vmarg.Xmca=-Xmca512k
The implicit rules
– Do not change vmarg.Xmx to more than -Xmx1024m regardless of how much memory you have
– vmarg.Xms should always be ¼ of what you changed Xmx to
Note
– The above defaults have been changed by IBM/HCL over the last couple of Notes releases.
You might find other values than Xmx256m and Xms64m in your setups.

28. The Out Of Memory Mystery In Notes 11
• Some of our customers who upgraded to Notes 11 have experienced out of memory
errors in the Notes client. That always ended up in freezes or crashes of HCL Notes
11.x
• The weird thing about this:
− The computer had plenty of available memory, and
− Task Manager showed that Notes was using a normal amount of memory, like 250MB or so
• Using Performance Monitor for more detailed troubleshooting
− “Private Bytes” - the amount of committed memory the application is using
− “Virtual Bytes” - the amount of virtual address space the application has reserved
More information:
https://www.panagenda.com/2020/08/the-out-of-memory-mystery-in-notes-11/

29. The Out Of Memory Mystery In Notes 11 (cont.)
More information:
https://www.panagenda.com/2020/08/the-out-of-memory-mystery-in-notes-11/

30. The Out Of Memory Mystery In Notes 11 (cont.)
• When we were testing, we found that out of memory errors were happening at the point that
the client hit the 2GB mark in the virtual address space monitor.
• This 2GB mark is important because the Notes client is still a 32-bit application. One of the
side-effects of this is it only has 2GB of virtual address space to play with (really 4GB, but half
of that is claimed by the kernel). 64-bit applications have a much, much larger virtual address
space (they sometimes reserve terabytes of space), but 32-bit applications need to be careful.
• SIDEBAR: A few clarifications about virtual address space:
− it’s completely different from virtual memory
− it doesn’t tell you how much memory the application is actually using
− it has nothing to do with how much memory is on the computer
− it’s a way of mapping memory that might be used by the application
− More information here about virtual address space
https://docs.microsoft.com/en-us/windows-hardware/drivers/gettingstarted/virtual-address-spaces
More information:
https://www.panagenda.com/2020/08/the-out-of-memory-mystery-in-notes-11/

31. The Out Of Memory Mystery In Notes 11 (cont.)
• Solution
− The immediate fix was to reduce the Java heap size used by the Notes client
vmarg.Xmx=-Xmx512m
− See HCL‘s Technote about this:
https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0038188
• Why wasn’t this a problem before?
− When you compile a 32-bit application, there is a flag you can set called /LARGEADDRESSAWARE.
This allows the application’s process to use a full 4GB of virtual address space on a 64-bit operating
system, instead of the 2GB we were seeing above.
− It turns out that the java.exe and notes2.exe files on the Notes 9 client were compiled with the
/LARGEADDRESSAWARE flags, but the java.exe and notes2.exe files on the Notes 11 client were not.
− Notes 9 used the IBM version of the Java JVM but Notes 11 uses the OpenJDK OpenJ9 version. The
OpenJ9 32-bit distribution wasn’t compiled with /LARGEADDRESSAWARE until just recently
(version 8u262, from June 2020), so the Notes 11 client didn’t compile with that flag either.
− In other words, Notes 9 had a full 4GB virtual address space, but Notes 11.x only has 2GB.
More information:
https://www.panagenda.com/2020/08/the-out-of-memory-mystery-in-notes-11/

32. HCL Notes 11.0.1 FP1 - jvm.properties file
To speed up the HCL Notes JVM it’s useful to optimize memory settings
Open %ProgramFiles(x86)%HCLNotesframeworkrcpdeployjvm.properties
– Change vmarg.Xmx=-Xmx512m → or maybe to -Xmx768m
– Change vmarg.Xms=-Xms256m
– Change vmarg.Xmca=-Xmca512k
– Remove the value “singleJVM,” from the parameter “vmarg.Dshare=”

33. Workspace Improvement
• Every HCL Notes Client start in a virtual environment is a first (setup) start and takes
~20 secs on average
• ~65% - 75% of time during first starts is used to build the workspace folder (at least 13 sec)
• A typical new workspace folder (HCL Notes 11.0.1 FP1)
consists of
– >220 files
– >160 folders
– >85 MB (including 64MB *_xpdplat_.jvm_G41L00 file)
• Building the workspace folder causes lots of file I/O

34. Workspace Improvement (cont.)
Accelerate the HCL Notes Client start to be >50% faster (with less file I/O)
1. With a non-admin user (Windows + HCL Notes) run an initial setup of HCL Notes
2. Start HCL Notes and wait for 1,5 minutes
3. Exit HCL Notes
4. Start HCL Notes again and wait for another 1,5 minutes
5. Exit HCL Notes
6. Create the following folder → %ProgramData%HCLNotesDataCommon
7. Copy the workspace folder from %LocalAppData%HCLNotesData to
%ProgramData%HCLNotesDataCommon

35. Workspace Improvement (cont.)
8. Delete the following files / folders in
%ProgramData%HCLNotesDataCommonworkspace
a) UDM
b) Logs → all but .prov2install
c) .metadata.pluginscom.ibm.collaboration.*
d) .metadata.pluginscom.ibm.rcp.personality.framework
e) .metadata.pluginscom.ibm.rcp.security.auth
f) .metadata.pluginscom.ibm.rcp.managedsettings
g) .metadata.pluginsorg.eclipse.e4.workbench

36. Workspace Improvement (cont.)
Important:
• You must recreate the workspace folder when your Installation changes (another Notes
version, Fix Pack, etc.)
• If you want to include the workspace folder to your installation package, you also must
include the following file:
– C:Program Files (x86)HCLNotesframeworkrcprcplauncher.properties

37. Make Your Data Work For You
Workspace Improvement
Live

38. Windows Registry
If you change the Data Directory – let’s say from “H:NotesData” to
“%LocalAppData%HCLNotesData”
• Please ensure that the following Registry-Entry will be deleted during Windows-Logon (always)
HKEY_CURRENT_USERSoftwareLotusNotesNotesIniPath

39. DEP
• Data Execution Prevention
• Exclude
– C:Program Files (x86)HCLNotesnotes.exe
– C:Program Files (x86)HCLNotesnlnotes.exe
– C:Program Files (x86)HCLNotesnsd.exe
– C:Program Files (x86)HCLNotesframeworkrcpeclipseplugins
com.ibm.rcp.base_10.0.0.20200318-1334win32x86notes2.exe

40. -Commercial break-
• Do you want to fully (100%) automate everything what I’ve talked about
in the last 40minutes?

41. Q & A
Christoph Adler
@cadler80
Senior Consultant
panagenda
christoph.adler@panagenda.com

42. Thank you!
Christoph Adler
@cadler80
Senior Consultant
panagenda
christoph.adler@panagenda.com

43. HCL Notes 11.0.1 FP1 Performance Boost Re-Reloaded