Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

April, 2021 OpenNTF Webinar - Domino Administration Best Practices

While installing a new HCL Domino server is a relatively straight forward task, configuring the server properly requires knowledge. Lacking this knowledge means that several key steps may be missed resulting in a server with potential security and performance issues. Additionally there are several key features that will save you time on administration of the server. Domino server settings also affect the performance and security of custom applications. Even if you are a developer you should be aware of the options available when configuring a server.

Join our incredibly experienced presenters as they share their many years of Domino expertise. They will cover the finer details to correctly setup a Domino server environment that is optimized for performance, security and sustainable administration. Plus use this information presented in this webinar to modify and improve your existing server environment.

Presenters:
Heather Hottenstein, HCL Ambassador
Roberto Boccadoro, HCL Ambassador
Serdar Basegmez, HCL Ambassador

Additonal Panelists (Q and A)
John Paganetti, HCL

  • Be the first to comment

  • Be the first to like this

April, 2021 OpenNTF Webinar - Domino Administration Best Practices

  1. 1. OPENNTF WEBINARS April OpenNTF Webinar: Domino Administration Best Practices
  2. 2. AGENDA • Welcome – Howard Greenberg and Graham Acres • Heather Hottenstein, HCL Ambassador • Roberto Boccadoro, HCL Ambassador • Serdar Basegmez, HCL Ambassador • John Paganetti, HCL (for Q and A) • Q and A - All
  3. 3. TRIBUTE TO NATHAN FREEMAN • Nathan passed away last week • Co-founder of OpenNTF • His vision built this community • Technical genius and a great person • Donations to the family appreciated • https://www.gofundme.com/f/ntf-needs-your- help
  4. 4. ASKING QUESTIONS • First Question – Will this be recorded? • Yes, view on YouTube!!! • https://www.youtube.com/user/OpenNTF • Use the Questions Pane in GoToWebinar • We will get to your questions at the end of the webinar • The speakers will respond to your questions verbally • (not in the Questions pane) • Please keep all questions related to the topics that our speakers are discussing!!! • Unrelated Question => post at: • http://openntf.slack.com/
  5. 5. THANKS TO THE OPENNTF SPONSORS • HCL made a significant contribution to help our organization • Funds these webinars! • Contests like Hackathons • Running the organization • Prominic donates all IT related services • Cloud Hosting for OpenNTF • Infrastructure management for HCL Domino and Atlassian Servers • System Administration for day-to-day operation
  6. 6. THIS IS OUR COMMUNITY • Join us and get involved! • We are all volunteers • No effort is too small • If your idea is bigger than you can do on your own, we can connect you to a team to work on it • Test or help or modify an existing project • Write guides or documentation • Add reviews on projects / stars on Snippets
  7. 7. NEXT WEBINAR • TBD
  8. 8. OPENNTF WEBINARS Domino Administration Best Practices
  9. 9. SPEAKERS • Heather Hottenstein, RPR Wyatt – HCL Ambassador • Roberto Boccadoro, ELD Engineering, HCL Ambassador • Serdar Basegmez, HCL Ambassador
  10. 10. SERVER PLATFORM – 11.0.1X • Operating System • Windows • 2019, 2016, 2012 R2 – Standard and Datacenter Edition • LINUX • Red Hat Enterprise Server 8.x, 7.4+ • SUSE Linux Enterprise Server 15.0+, 12.0+ • CentOS Server 8.x, 7.4x • Docker • AS/400 • V7 r2 • V7 r3 • V7 r4 (On IBM Power 8, 9) • AIX • 7.2 TL1+ • On premises vs Cloud
  11. 11. FILE SYSTEM STRUCTURE • Separate drives/directories for Domino program and data directories • Do NOT install in c:program files... • Avoid brand named directories: opt/ibm/…. /opt/hcl/…. • Use the KISS rule: c:Domino, d:DominoData
  12. 12. NAMING STRUCTURE • Unless really needed, use just an O and do not use OUs • Makes administration simpler • Good for 90% of the cases • Document certifier password • Set certifier expiration to 100 years
  13. 13. SERVER DOCUMENT - BASICS • Internet site documents - Enabled • Automatic server recovery configuration • Run NSD to collect diagnostic information - Enabled • Automatically restart server after fault/crash - Enabled • Maximum fault limits • Mail fault notification
  14. 14. SERVER DOCUMENT - SECURITY • Groups vs Explicit entries • Full Access Administrators vs Administrators • Servers are NOT admins • Programmability Restrictions • Compare public keys – Enforce key checking for all Notes users and Domino servers • Log public key mismatches – Log key mismatches for all Notes users and Domino servers • Check passwords on Notes IDs - Enabled • Internet authentication • Server Access • Create databases & templates, new replicas • Termination groups • Trusted Servers
  15. 15. SERVER DOCUMENT – PORTS • Net Address – FQN vs TCPIP address • Cluster traffic • Internet Ports • Enforce server access settings - Enabled • TCP/IP Port – Redirect to SSL • Authentication options • Name & Password – Yes • Anonymous - No • HTTPPublicUrls=/redir.nsf/* • Mail – SMTP Outbound – TCP/IP Port status • Negotiated SSL
  16. 16. SERVER DOCUMENT – SERVER TASKS • Administration Process • Delayed request settings • Start executing on – list all days • Agent Manager • Max concurrent agents • Max LotusScript/Java execution time
  17. 17. SERVER DOCUMENT – INTERNET PROTOCOLS • HTTP • Number active threads • Traveler devices X 1.2 • Allow HTTP clients to browse databases - No • Log files - Enabled • Domlog.nsf • HTTP persistent connections - Disabled • Domino Web Engine • Maximum Post data
  18. 18. SERVER DOCUMENT – TRANSACTION LOGGING • Log path – separate from Data drive • Logging styles • Set and do not change • After you change the logging style, Domino assigns a new DBIID to each database. You must restart the server and perform another full backup • Database maintenance • DBIID and Backups • Compact • Fixup -J
  19. 19. SERVER DOCUMENT - DAOS • Mail servers • Configuration considerations • DAOS Estimator Tool • http://www.dominonews.com/dnews.nsf/documents/DA OS%20Estimator%20tool?opendocument • Tier 2 storage • Always use a separate disk for DAOS and another for transaction logging
  20. 20. SERVER DOCUMENT – NOTES TRAVELER • Maximum Memory Size/JVM • Access server • MDM • HA Pool
  21. 21. CONFIGURATION DOC - SECURITY • Check vault first, then directory • When enabled, this feature allows HCL Verse, HCL iNotes and other web users with Notes IDs to provide their web name and Notes ID password, to authenticate to the Domino server. This allows the users to remember one password, the Notes ID password. • Enforce Internet Password Lockout
  22. 22. CONFIGURATION DOC – ROUTER/SMTP • Number of mailboxes – 2+ • Address lookup – Fullname only • Relay host for messages leaving the local internet domain • Maximum message size • Deny messages to be sent to the following external internet domains - * • Deny messages from the following internet hosts to be sent to external internet domains - * • Exclude these connecting hosts from anti-relay checks • IP addresses listed in square brackets – [192.10.10.1]
  23. 23. CONFIGURATION DOC – ROUTER/SMTP • Allow connections from the following SMTP internet hostnames/IP addresses • Deny mail rules forwarding to external internet domains • Server Mail Rules – block .zip, .exe • Message Tracking – Enabled • SSL negotiated over TCP/IP port - Enabled • Out-of-Office type – Service • Automatically process dead mail - Enabled
  24. 24. CONFIGURATION DOC – NOTES.INI WHY THE CONFIGURATION DOCUMENT? • HTTPJVMMaxHeapSize • JavaUse64BitJVM • TNEFEnableConversion • NIF_VIEW_USAGE_ENABLED • Create_R*_Databases • UPDATERS=#CPUs • REPLICATORS=#CPUs • LOG_REPLICATION=1 • LOG_SESSIONS=1 • LOG_VIEW_EVENTS=1 • HTTPDisableMethods=Trace • Cluster_Admin_On • D10_ENABLE_REPAIR • FTBasePath • NIFNSFEnable • NIFBasepath • Debug variables • MailFileDisableCompactAbort=1
  25. 25. ALWAYS USE SSL • Some things will not even work with HTTP, e.g. Traveler, Sametime mobile client will stop supporting HTTP soon. • Mandatory for Verse • Redirect port 80 to TLS • Enforce server access setting • Add those lines to notes.ini • DISABLE_SSLV3=1 • SSL_DISABLE_TLS_10=1 • Disable weak ciphers • SSLCipherSpec notes.ini setting is ignored
  26. 26. SETTING UP SSL IN V11 • Do not use the native Domino CA app. Use OpenSSL and kyrtool. • 3 ways to do it • Create a self signed certificate (good for testing) https://support.hcltechsw.com/csm?id=kb_article&sys_id= a3ff10361b926cd4534c4159cc4bcb01&spa=1 • Obtain a certificate from a CA https://support.hcltechsw.com/csm?id=kb_article&sys_id= fb7ba618dbf6e89ca45ad9fcd3961966&spa=1 • Use LE4D , a free Domino app from Midpoints that lets you request and renew free SSL certificates issued by LetsEncrypt
  27. 27. SETTING UP SSL IN V12 • Piece of cake ☺ • Use the new CertMgr db • More in a webinar in June
  28. 28. DISABLE NOT USED PORTS
  29. 29. WHITELIST ACTIVE CONTENT FILTER (ACF) FOR INOTES AND VERSE • The ACF is used to remove potentially harmful active content from HTML messages such as JavaScript™, Java™, and ActiveX. A whitelist filter removes all entities except those in the whitelist. A blacklist filter (used in previous releases and still the default in this release) retains all entities except those in the blacklist. Blacklist filters need to be continuously maintained to guard against threats from new markup patterns. Whitelist filters are considered a best practice because they are explicit about the patterns that are allowed. ACF is available for iNotes and Verse, it does not apply to the Notes client.
  30. 30. PROGRAM DOCUMENTS • Scheduled server tasks • Database maintenance • DBMT • runs copy-style compact operations • purges deletion stubs • expires soft deleted entries • updates views • reorganizes folders • merges full-text indexes • updates unread lists • ensures that critical views are created for failover • System databases are not compacted • -compactThreads 8 -updallThreads 8 -range 2:00AM 7:00AM - compactNdays 5 -force 1 • Remove ServerTasksAt2=Updall • Compact -B
  31. 31. DOMINO CERTIFICATE AUTHORITY • Domino Directory Administration Server • Domino Certifier IDs – Not SSL • Tools – Certification – Migrate Certifier • ICL database • User registrations, renames and recertifications
  32. 32. ID VAULT SETUP • Centralized storage of live Notes IDs • Required for Verse on Premises • Automatically created in Domino 12 • Configuration – Tools – ID Vaults – Create • IBM_ID_Vaultdbname.nsf • Vault ID – backup! • Volt administrator • Password resets • Configuration – ID Vaults – Manage • Create ID Vault replicas • Assigning users
  33. 33. POLICIES AND SETTINGS DOCUMENTS • Explicit vs Organizational • Registration Settings • Setup • Desktop • Marvel Client • Archiving • Security • Mail • Notes Traveler
  34. 34. DOMINO EMAIL - OPTIONS • Notes client • Web Browser • iNotes (will likely go away after V12) • Verse on Premises • Mobile Devices - Traveler
  35. 35. DOMINO SERVER MONITORING • Domino Domain Management • DDM.NSF • DDM Probes • EVENTS4.NSF • View and Manage events • Third party products • OS resource monitoring • RAM, CPU, Disk • Agents • Tell amgr schedule
  36. 36. DOMINO SERVER MAINTENANCE • Database maintenance • Program documents • System database maintenance • Log.nsf, domlog.nsf, mail.box • Domino Fix packs + upgrades • OS patches • Anti virus/spam updates
  37. 37. USEFUL DOCUMENTATION FROM HCL • Domino cookbook • Domino upgrade guide • https://support.hcltechsw.com/csm?id=kb_article&sysp arm_article=KB0077811&sys_kb_id=bf5c8b72dbe2a41ca 45ad9fcd3961961
  38. 38. TIPS AND TRICKS FOR DEVELOPERS Approved for Test and Development Servers only!
  39. 39. AUTHENTICATION MECHANISMS • Mostly, we don’t really care… • However… • Customised Login/Logout • Simulating the same authentication scheme • Apps providing service for “weirdies” • Many options: • Basic Authentication • Session Authentication (Single Server vs Multi-server) • SAML, OAuth, IAM, etc. ==> Not today!
  40. 40. AUTHENTICATION: BASIC HCL Domino Server Browser Request: GET /path/database.nsf Response: 401 Unauthorized WWW-Authenticate: Basic realm=“/path” Request: GET /path/database.nsf Authorization: Basic SXQncyBiaWdnZXIgaW5zaWRlIQ== Response: 200 OK
  41. 41. AUTHENTICATION: SESSION HCL Domino Server Browser Request: GET /path/database.nsf Response: 200 OK Login Form (text/html) Request: POST names.nsf?Login Form Data with UserName + Password + RedirectTo Response: 200 OK Target Content + Authentication Cookie 401?
  42. 42. SINGLE SERVER VS. MULTI-SERVER • Single Server • Server creates a cookie “DomAuthSessId” • Server keeps a list of authenticated sessions • Cookie is only valid for single server • Multiple servers (SSO) • Server creates a cookie “LtpaToken” (customizable) • Token is hashed with the username and expiration time • Multiple Servers share a secret key to hash/verify the token. • Server doesn’t keep track of users (except for monitoring)
  43. 43. DIFFERENCES Basic Single Server Multi-Server Name of the Cookie: No cookie DomAuthSessId LtpaToken (Configurable) Expiration is kept… On Browser On Server On Cookie Timeout depends on… Browser Session Last request Cookie Creation tell Http Show Users None Accurate Inaccurate On HTTP Restart Continue Need Authentication Continue
  44. 44. SOME TIPS ON AUTHENTICATION • When Session Authentication is enabled, • Unauthenticated/unauthorized requests returns “200 OK” • 401/404 expected in modern web architecture • Tip: Override Session Authentication • Multi-server session cannot be extended, token cannot be canceled. • Tip: Remove LtpaToken cookie for manual logout • Tip: Keep the expiration time long enough • XPages Session ≠ Authentication Session • XPages session ~ SessionID cookie • Specific to the browser session
  45. 45. TIPS FOR XPAGES DEVELOPERS Always use Internet Sites! Additional settings Multiple domains More practical for testing Enabled from the server document Need site document for all protocols (e.g. IMAP, POP3, SMTP, etc.)
  46. 46. TIPS FOR XPAGES DEVELOPERS • Allowed methods and Domino Access Services • Relevant Internet Site Document Configuration • Important for RESTful developers
  47. 47. TIPS FOR XPAGES DEVELOPERS • Server-wide xsp.properties • Go to “[domino-data]properties” on the server • The sample file is the documentation for all properties.
  48. 48. TIPS FOR XPAGES DEVELOPERS • XPages app connecting to a remote server • XPages as a front-end application layer • Data in another NSF, even in another server • “Trusted Servers” will be useful! • It’s not for production • Low performance • Great to access real data from the production
  49. 49. TIPS FOR XPAGES DEVELOPERS • Debugging HTTP Thread • tell http debug thread on | off ==> Default level • tell http debug postdata on | off ==> for client POST data • tell http debug responsedata on | off ==> for server response data • Save some space! • tell http debug lastonly on | off ==> Keep only the last request! • For more options… • https://support.hcltechsw.com/kb_view.do?sysparm_article=KB0032210
  50. 50. TIPS FOR XPAGES DEVELOPERS • Use XPages Log File Reader from OpenNTF • https://www.openntf.org/p/xpages log file reader • Send your virtual kudos to Jakob Majkilde!
  51. 51. JVM CUSTOMIZATION • notes.ini parameters for JVM Memory • HTTPJVMMaxHeapSize ==> JVM heap for HTTP • JavaMaxHeapsize ==> JVM heap for the rest • Default values for Domino 8.5+ and 64-bit • HTTPJVMMaxHeapSize=1024M • JavaMaxHeapsize=256M
  52. 52. JVM CUSTOMIZATION • Add JVM arguments via notes.ini • Create a text file with JVM arguments • JavaOptionsFile=c:pathtojvm.txt • Very useful to customize JVM! • Testing different locales • Setting TLS protocols • Additional debugging • Tweak third party libraries
  53. 53. JVM CUSTOMIZATION • Modify Java security policy (like a pro!) • /[domino]/jvm/lib/security/java.policy ==> do not use! • /[user-home]/.java.policy ==> will persist! • What is [user-home]? • Linux: /local/notes (notes is the user for domino service) • Windows (Run as a service): C:WindowsSystem32configsystemprofile • Windows (Run as an app): C:UsersJANE.DOE • Technote: • https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0085173 • Reminder and Correction: • /[domino]/jvm/lib/security/java.pol ==> Obsolete as of R11+
  54. 54. SECURITY TIPS • Careful with the HTTPEnableConnectorHeaders • Normally, it should be “0” • It allows an attacker to impersonate any user! • Only for “behind the proxy” scenarios. • In case, Domino HTTP should be secured with Firewall. Image is from Wikipedia. Refer to Jesper Kiaer for more details. https://nevermind.dk/nevermind/blog.nsf/subject/security-hole-leaves-ibm-domino-server-wide-open---part-one
  55. 55. SECURITY TIPS • Use a different Domino domain for Test/development servers • Testing and UAT servers are wide open for breaches! • Open relay attacks • Insecure passwords for test users • Remote debugging (XPages/Agents) • Intel about production
  56. 56. Q&A TIME!
  57. 57. QUESTIONS? Use the GoToWebinar Questions Pane Please keep all questions related to the topics that our speakers are discussing!!! Unrelated Question => post at: http://openntf.slack.com/

×