1. OPENNTF WEBINARS
April OpenNTF Webinar:
Domino Administration Best
Practices

2. AGENDA
• Welcome – Howard Greenberg and Graham Acres
• Heather Hottenstein, HCL Ambassador
• Roberto Boccadoro, HCL Ambassador
• Serdar Basegmez, HCL Ambassador
• John Paganetti, HCL (for Q and A)
• Q and A - All

3. TRIBUTE TO NATHAN FREEMAN
• Nathan passed away last week
• Co-founder of OpenNTF
• His vision built this community
• Technical genius and a great person
• Donations to the family appreciated
• https://www.gofundme.com/f/ntf-needs-your-
help

4. ASKING QUESTIONS
• First Question – Will this be recorded?
• Yes, view on YouTube!!!
• https://www.youtube.com/user/OpenNTF
• Use the Questions Pane in GoToWebinar
• We will get to your questions at the end of
the webinar
• The speakers will respond to your questions
verbally
• (not in the Questions pane)
• Please keep all questions related to the
topics that our speakers are discussing!!!
• Unrelated Question => post at:
• http://openntf.slack.com/

5. THANKS TO THE OPENNTF SPONSORS
• HCL made a significant contribution to help our
organization
• Funds these webinars!
• Contests like Hackathons
• Running the organization
• Prominic donates all IT related services
• Cloud Hosting for OpenNTF
• Infrastructure management for HCL Domino and Atlassian
Servers
• System Administration for day-to-day operation

6. THIS IS OUR COMMUNITY
• Join us and get involved!
• We are all volunteers
• No effort is too small
• If your idea is bigger than you can do on your own, we
can connect you to a team to work on it
• Test or help or modify an existing project
• Write guides or documentation
• Add reviews on projects / stars on Snippets

7. NEXT WEBINAR
• TBD

8. OPENNTF WEBINARS
Domino Administration Best Practices

9. SPEAKERS
• Heather Hottenstein, RPR Wyatt – HCL Ambassador
• Roberto Boccadoro, ELD Engineering, HCL Ambassador
• Serdar Basegmez, HCL Ambassador

10. SERVER PLATFORM – 11.0.1X
• Operating System
• Windows
• 2019, 2016, 2012 R2 – Standard and Datacenter Edition
• LINUX
• Red Hat Enterprise Server 8.x, 7.4+
• SUSE Linux Enterprise Server 15.0+, 12.0+
• CentOS Server 8.x, 7.4x
• Docker
• AS/400
• V7 r2
• V7 r3
• V7 r4 (On IBM Power 8, 9)
• AIX
• 7.2 TL1+
• On premises vs Cloud

11. FILE SYSTEM STRUCTURE
• Separate drives/directories for Domino program and data
directories
• Do NOT install in c:program files...
• Avoid brand named directories: opt/ibm/…. /opt/hcl/….
• Use the KISS rule: c:Domino, d:DominoData

12. NAMING STRUCTURE
• Unless really needed, use just an O and do not use OUs
• Makes administration simpler
• Good for 90% of the cases
• Document certifier password
• Set certifier expiration to 100 years

13. SERVER DOCUMENT - BASICS
• Internet site documents - Enabled
• Automatic server recovery configuration
• Run NSD to collect diagnostic information - Enabled
• Automatically restart server after fault/crash - Enabled
• Maximum fault limits
• Mail fault notification

14. SERVER DOCUMENT - SECURITY
• Groups vs Explicit entries
• Full Access Administrators vs Administrators
• Servers are NOT admins
• Programmability Restrictions
• Compare public keys – Enforce key checking for all Notes users
and Domino servers
• Log public key mismatches – Log key mismatches for all Notes
users and Domino servers
• Check passwords on Notes IDs - Enabled
• Internet authentication
• Server Access
• Create databases & templates, new replicas
• Termination groups
• Trusted Servers

15. SERVER DOCUMENT – PORTS
• Net Address – FQN vs TCPIP address
• Cluster traffic
• Internet Ports
• Enforce server access settings - Enabled
• TCP/IP Port – Redirect to SSL
• Authentication options
• Name & Password – Yes
• Anonymous - No
• HTTPPublicUrls=/redir.nsf/*
• Mail – SMTP Outbound – TCP/IP Port status
• Negotiated SSL

16. SERVER DOCUMENT – SERVER TASKS
• Administration Process
• Delayed request settings
• Start executing on – list all days
• Agent Manager
• Max concurrent agents
• Max LotusScript/Java execution time

17. SERVER DOCUMENT – INTERNET
PROTOCOLS
• HTTP
• Number active threads
• Traveler devices X 1.2
• Allow HTTP clients to browse databases - No
• Log files - Enabled
• Domlog.nsf
• HTTP persistent connections - Disabled
• Domino Web Engine
• Maximum Post data

18. SERVER DOCUMENT – TRANSACTION
LOGGING
• Log path – separate from Data drive
• Logging styles
• Set and do not change
• After you change the logging style, Domino assigns a new DBIID to each
database. You must restart the server and perform another full backup
• Database maintenance
• DBIID and Backups
• Compact
• Fixup -J

19. SERVER DOCUMENT - DAOS
• Mail servers
• Configuration considerations
• DAOS Estimator Tool
• http://www.dominonews.com/dnews.nsf/documents/DA
OS%20Estimator%20tool?opendocument
• Tier 2 storage
• Always use a separate disk for DAOS and another for
transaction logging

20. SERVER DOCUMENT – NOTES TRAVELER
• Maximum Memory Size/JVM
• Access server
• MDM
• HA Pool

21. CONFIGURATION DOC - SECURITY
• Check vault first, then directory
• When enabled, this feature allows HCL Verse, HCL iNotes
and other web users with Notes IDs to provide their web
name and Notes ID password, to authenticate to the
Domino server. This allows the users to remember one
password, the Notes ID password.
• Enforce Internet Password Lockout

22. CONFIGURATION DOC – ROUTER/SMTP
• Number of mailboxes – 2+
• Address lookup – Fullname only
• Relay host for messages leaving the local internet
domain
• Maximum message size
• Deny messages to be sent to the following external
internet domains - *
• Deny messages from the following internet hosts to be
sent to external internet domains - *
• Exclude these connecting hosts from anti-relay checks
• IP addresses listed in square brackets – [192.10.10.1]

23. CONFIGURATION DOC – ROUTER/SMTP
• Allow connections from the following SMTP internet
hostnames/IP addresses
• Deny mail rules forwarding to external internet domains
• Server Mail Rules – block .zip, .exe
• Message Tracking – Enabled
• SSL negotiated over TCP/IP port - Enabled
• Out-of-Office type – Service
• Automatically process dead mail - Enabled

24. CONFIGURATION DOC – NOTES.INI
WHY THE CONFIGURATION DOCUMENT?
• HTTPJVMMaxHeapSize
• JavaUse64BitJVM
• TNEFEnableConversion
• NIF_VIEW_USAGE_ENABLED
• Create_R*_Databases
• UPDATERS=#CPUs
• REPLICATORS=#CPUs
• LOG_REPLICATION=1
• LOG_SESSIONS=1
• LOG_VIEW_EVENTS=1
• HTTPDisableMethods=Trace
• Cluster_Admin_On
• D10_ENABLE_REPAIR
• FTBasePath
• NIFNSFEnable
• NIFBasepath
• Debug variables
• MailFileDisableCompactAbort=1

25. ALWAYS USE SSL
• Some things will not even work with HTTP, e.g. Traveler,
Sametime mobile client will stop supporting HTTP soon.
• Mandatory for Verse
• Redirect port 80 to TLS
• Enforce server access setting
• Add those lines to notes.ini
• DISABLE_SSLV3=1
• SSL_DISABLE_TLS_10=1
• Disable weak ciphers
• SSLCipherSpec notes.ini setting is ignored

26. SETTING UP SSL IN V11
• Do not use the native Domino CA app. Use OpenSSL
and kyrtool.
• 3 ways to do it
• Create a self signed certificate (good for testing)
https://support.hcltechsw.com/csm?id=kb_article&sys_id=
a3ff10361b926cd4534c4159cc4bcb01&spa=1
• Obtain a certificate from a CA
https://support.hcltechsw.com/csm?id=kb_article&sys_id=
fb7ba618dbf6e89ca45ad9fcd3961966&spa=1
• Use LE4D , a free Domino app from Midpoints that lets you
request and renew free SSL certificates issued by
LetsEncrypt

27. SETTING UP SSL IN V12
• Piece of cake ☺
• Use the new CertMgr db
• More in a webinar in June

28. DISABLE NOT USED PORTS

29. WHITELIST ACTIVE CONTENT FILTER
(ACF) FOR INOTES AND VERSE
• The ACF is used to remove potentially harmful active
content from HTML messages such as JavaScript™,
Java™, and ActiveX. A whitelist filter removes all entities
except those in the whitelist. A blacklist filter (used in
previous releases and still the default in this release)
retains all entities except those in the blacklist. Blacklist
filters need to be continuously maintained to guard
against threats from new markup patterns. Whitelist
filters are considered a best practice because they are
explicit about the patterns that are allowed. ACF is
available for iNotes and Verse, it does not apply to the
Notes client.

30. PROGRAM DOCUMENTS
• Scheduled server tasks
• Database maintenance
• DBMT
• runs copy-style compact operations
• purges deletion stubs
• expires soft deleted entries
• updates views
• reorganizes folders
• merges full-text indexes
• updates unread lists
• ensures that critical views are created for failover
• System databases are not compacted
• -compactThreads 8 -updallThreads 8 -range 2:00AM 7:00AM -
compactNdays 5 -force 1
• Remove ServerTasksAt2=Updall
• Compact -B

31. DOMINO CERTIFICATE AUTHORITY
• Domino Directory Administration Server
• Domino Certifier IDs – Not SSL
• Tools – Certification – Migrate Certifier
• ICL database
• User registrations, renames and recertifications

32. ID VAULT SETUP
• Centralized storage of live Notes IDs
• Required for Verse on Premises
• Automatically created in Domino 12
• Configuration – Tools – ID Vaults – Create
• IBM_ID_Vaultdbname.nsf
• Vault ID – backup!
• Volt administrator
• Password resets
• Configuration – ID Vaults – Manage
• Create ID Vault replicas
• Assigning users

33. POLICIES AND SETTINGS DOCUMENTS
• Explicit vs Organizational
• Registration Settings
• Setup
• Desktop
• Marvel Client
• Archiving
• Security
• Mail
• Notes Traveler

34. DOMINO EMAIL - OPTIONS
• Notes client
• Web Browser
• iNotes (will likely go away after V12)
• Verse on Premises
• Mobile Devices - Traveler

35. DOMINO SERVER MONITORING
• Domino Domain Management
• DDM.NSF
• DDM Probes
• EVENTS4.NSF
• View and Manage events
• Third party products
• OS resource monitoring
• RAM, CPU, Disk
• Agents
• Tell amgr schedule

36. DOMINO SERVER MAINTENANCE
• Database maintenance
• Program documents
• System database maintenance
• Log.nsf, domlog.nsf, mail.box
• Domino Fix packs + upgrades
• OS patches
• Anti virus/spam updates

37. USEFUL DOCUMENTATION FROM HCL
• Domino cookbook
• Domino upgrade guide
• https://support.hcltechsw.com/csm?id=kb_article&sysp
arm_article=KB0077811&sys_kb_id=bf5c8b72dbe2a41ca
45ad9fcd3961961

38. TIPS AND TRICKS FOR DEVELOPERS
Approved for Test and Development Servers only!

39. AUTHENTICATION MECHANISMS
• Mostly, we don’t really care…
• However…
• Customised Login/Logout
• Simulating the same authentication scheme
• Apps providing service for “weirdies”
• Many options:
• Basic Authentication
• Session Authentication (Single Server vs Multi-server)
• SAML, OAuth, IAM, etc. ==> Not today!

40. AUTHENTICATION: BASIC
HCL Domino Server
Browser Request: GET /path/database.nsf
Response: 401 Unauthorized
WWW-Authenticate: Basic realm=“/path”
Request: GET /path/database.nsf
Authorization: Basic SXQncyBiaWdnZXIgaW5zaWRlIQ==
Response: 200 OK

41. AUTHENTICATION: SESSION
HCL Domino Server
Browser Request: GET /path/database.nsf
Response: 200 OK
Login Form (text/html)
Request: POST names.nsf?Login
Form Data with UserName + Password + RedirectTo
Response: 200 OK
Target Content + Authentication Cookie
401?

42. SINGLE SERVER VS. MULTI-SERVER
• Single Server
• Server creates a cookie “DomAuthSessId”
• Server keeps a list of authenticated sessions
• Cookie is only valid for single server
• Multiple servers (SSO)
• Server creates a cookie “LtpaToken” (customizable)
• Token is hashed with the username and expiration time
• Multiple Servers share a secret key to hash/verify the token.
• Server doesn’t keep track of users (except for monitoring)

43. DIFFERENCES
Basic Single Server Multi-Server
Name of the Cookie: No cookie DomAuthSessId
LtpaToken
(Configurable)
Expiration is kept… On Browser On Server On Cookie
Timeout depends on… Browser Session Last request Cookie Creation
tell Http Show Users None Accurate Inaccurate
On HTTP Restart Continue Need Authentication Continue

44. SOME TIPS ON AUTHENTICATION
• When Session Authentication is enabled,
• Unauthenticated/unauthorized requests returns “200 OK”
• 401/404 expected in modern web architecture
• Tip: Override Session Authentication
• Multi-server session cannot be extended, token cannot be
canceled.
• Tip: Remove LtpaToken cookie for manual logout
• Tip: Keep the expiration time long enough
• XPages Session ≠ Authentication Session
• XPages session ~ SessionID cookie
• Specific to the browser session

45. TIPS FOR XPAGES DEVELOPERS
Always use Internet Sites!
Additional settings
Multiple domains
More practical for testing
Enabled from the server document
Need site document for all protocols (e.g. IMAP,
POP3, SMTP, etc.)

46. TIPS FOR XPAGES DEVELOPERS
• Allowed methods and Domino Access Services
• Relevant Internet Site Document Configuration
• Important for RESTful developers

47. TIPS FOR XPAGES DEVELOPERS
• Server-wide xsp.properties
• Go to “[domino-data]properties” on the server
• The sample file is the documentation for all properties.

48. TIPS FOR XPAGES DEVELOPERS
• XPages app connecting to a remote server
• XPages as a front-end application layer
• Data in another NSF, even in another server
• “Trusted Servers” will be useful!
• It’s not for production
• Low performance
• Great to access real data from the production

49. TIPS FOR XPAGES DEVELOPERS
• Debugging HTTP Thread
• tell http debug thread on | off ==> Default level
• tell http debug postdata on | off ==> for client POST data
• tell http debug responsedata on | off ==> for server response data
• Save some space!
• tell http debug lastonly on | off ==> Keep only the last request!
• For more options…
• https://support.hcltechsw.com/kb_view.do?sysparm_article=KB0032210

50. TIPS FOR XPAGES DEVELOPERS
• Use XPages Log File Reader from OpenNTF
• https://www.openntf.org/p/xpages log file reader
• Send your virtual kudos to Jakob Majkilde!

51. JVM CUSTOMIZATION
• notes.ini parameters for JVM Memory
• HTTPJVMMaxHeapSize ==> JVM heap for HTTP
• JavaMaxHeapsize ==> JVM heap for the rest
• Default values for Domino 8.5+ and 64-bit
• HTTPJVMMaxHeapSize=1024M
• JavaMaxHeapsize=256M

52. JVM CUSTOMIZATION
• Add JVM arguments via notes.ini
• Create a text file with JVM arguments
• JavaOptionsFile=c:pathtojvm.txt
• Very useful to customize JVM!
• Testing different locales
• Setting TLS protocols
• Additional debugging
• Tweak third party libraries

53. JVM CUSTOMIZATION
• Modify Java security policy (like a pro!)
• /[domino]/jvm/lib/security/java.policy ==> do not use!
• /[user-home]/.java.policy ==> will persist!
• What is [user-home]?
• Linux: /local/notes (notes is the user for domino service)
• Windows (Run as a service): C:WindowsSystem32configsystemprofile
• Windows (Run as an app): C:UsersJANE.DOE
• Technote:
• https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0085173
• Reminder and Correction:
• /[domino]/jvm/lib/security/java.pol ==> Obsolete as of R11+

54. SECURITY TIPS
• Careful with the HTTPEnableConnectorHeaders
• Normally, it should be “0”
• It allows an attacker to impersonate any user!
• Only for “behind the proxy” scenarios.
• In case, Domino HTTP should be secured with Firewall.
Image is from Wikipedia. Refer to Jesper Kiaer for more details. https://nevermind.dk/nevermind/blog.nsf/subject/security-hole-leaves-ibm-domino-server-wide-open---part-one

55. SECURITY TIPS
• Use a different Domino domain for Test/development servers
• Testing and UAT servers are wide open for breaches!
• Open relay attacks
• Insecure passwords for test users
• Remote debugging (XPages/Agents)
• Intel about production

56. Q&A TIME!

57. QUESTIONS?
Use the GoToWebinar Questions Pane
Please keep all questions related to the
topics that our speakers are discussing!!!
Unrelated Question => post at:
http://openntf.slack.com/