Compact, Compress, De-Duplicate (DAOS)


Published on


Published in: Technology, Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • In Deutschland wurde am 3. August 1984 um 10:14 Uhr MEZ die erste Internet-E-Mail empfangen: Michael Rotert von der Universität Karlsruhe (TH) empfing unter seiner Adresse „rotert@germany“ eine Grußbotschaft von Laura Breeden („“) an der US-amerikanischen Plattform CSNET aus Cambridge (Massachusetts) zur elektronischen Kommunikation von Wissenschaftlern, die einen Tag zuvor (am 2. August 1984, 12:21 Uhr)[8] abgeschickt worden war. Eine Kopie dieser E-Mail wurde erst später an den Leiter des Projekts, Werner Zorn („zorn@germany“), geschickt.[9]
  • There are circumstances that compression conversion is explicitly avoid. Overall, it is to avoid the performance penalty of attempting a compression and failing back to uncompressed for file types that are already compressed. The common trick is applied to observe the file extensions and avoid these. Default list: zip,rar,jpg,gz,z,zz,lzo,zoo,rz,lzma,bz2 Ini to control: NONCOMPRESSABLE_FILES=<comma delimited replacement list> There is also a mechanism to avoid for specific item names. Default List: $JavaFile Init to control: NONCOMPRESSABLE_NOTE_ITEMS=<comma delimited replacement list - and do not remove $JavaFile - see below> For $JavaFile problem see technote:
  • The reason for my concern? We have been told that the DAOS store has lesser disk performance demands than say the Transaction Logs or the data directory, yet it appears that when a DAOS MGR RESYNC or similar operation is initiated, actually a very intensive audit of the DAOS area takes place. This is where the 40,000 file per directory comes in... Just doing an 'ls' or 'dir' of a very large directory like that takes a significant time, particularly when this area is stored on NFS or iSCSI disk. In the case of a NFS mounted DAOS directory (say on an enterprise-class NetApps file), that 40000 files per subcontainer is too much. When the DAOS MGR RESYNC starts, it enumerates the contents of all the subcontainers. Due to the nature of NFS this is an extremely intensive operation. For 1.2 million attachments in 30 subcontainers this takes quite some time on NFS (3 hours or so) and is a heavy hit on the NFS filer performance. During this time, the daos catalogue daos cat.nsf is "unavailable", and so mail routing cannot occur until directory enumeration is complete. This is only going to get worse as more attachments are added to the DAOS store. Whenever the DAOS is resync'd, mail cannot be routed until this initial phase is completed and it starts on the databases themselves. In my opinion, we would see much better performance on initial resync if file count per subcontainer can be restricted to 5000 files or less for situations such as NFS. So if there isn't a means of decreasing the files/directory ratio now, can one be created? Feedback number SMIE82TFFY created by Stuart McIntyre on 19.02.2010
  • Question How does the DAOS resync time window feature work.   Answer Brief explanation of how the Resync Time Window feature works The following two notes.ini parameters control when resync will be allowed to run. Note that these parameters do not launch resync automatically; they just control whether or not resync may run at a given time, and also tell resync when to stop. DAOS_RESYNC_START_TIME=12:00:00 AM DAOS_RESYNC_STOP_TIME=04:00:00 AM The format of the time is as follows: HH:MM:SS AM/PM The above example will allow resync to run between midnight and 4:00 AM. A program document could be created to launch resync just after midnight. If the catalog is SYNCHRONIZED, then the resync will not take place. However, if a resync is needed, then the resync process will begin. When the end time is reached, the resync threads will finish up the databases they are processing and exit. If resync finished processing all databases during the time window, then the catalog will go to the SYNCHRONIZED state. At this point there is still a little more work to be done to the catalog. Even though the time window is closed, this processing will be finished up. Therefore, there is a potential for resync to run past the end time, but the remaining work is low-impact and will not effect server response time. Note that until resync scans all databases on the system and puts them into the SYNCHRONIZED state, the DAOS catalog will remain in the RESYNCING state. This indicates that resync still has work to be done. It does not, however, indicate that resync is currently running. Resync Time Window FAQ: 1. Why does processing continues after set stop time, DAOS_RESYNC_STOP_TIME? One reason for the resync to continue is that each thread will finish the NSF it is processing, and this work may run past the specified stop time. The next night, when resync is run again, it will continue with the next databases to be processed. Another reason might be that after the nlo file reference counts for all the databases are completed, the remaining processing will be low-cost in terms of time, and will be allowed to complete. 2. What happens if DAOS_RESYNC_START_TIME and DAOS_RESYNC_STOP_TIME are set using the 24 hour clock? 24 hour clock times are not currently supported for this feature. The issue can be tracked under the SPR SRIO84NQTY. The following example shows the behavior of the time window with the 24-hour clock set. If you see this behavior, set the DAOS_RESYNC_START_TIME/DAOS_RESYNC_STOP_TIME to supply values based on the 12-hour clock. A. set config DAOS_RESYNC_START_TIME=15:00:00 B. set config DAOS_RESYNC_STOP_TIME=16:00:00 C. tell daosmgr resync results in an abort. sh config DAOS_RESYNC_START_TIME [602228:00009-01286] DAOS_RESYNC_START_TIME=15:00:00 sh config DAOS_RESYNC_STOP_TIME [602228:00009-01286] DAOS_RESYNC_STOP_TIME=16:00:00 tell daosmgr resync [376974:00002-00001] Current Time is not within resync time slot, aborting... tell daosmgr status [376974:00002-00001] 04/19/2010 15:36:42    DAOSMGR: Status started [376974:00002-00001] 04/19/2010 15:36:42   threadsShouldQuit = FALSE [376974:00002-00001] 04/19/2010 15:36:42   refcount = 1 [376974:00002-00001] 04/19/2010 15:36:42   threadsCreated = 3 [376974:00002-00001] 04/19/2010 15:36:42   [376974:00002-00001] 04/19/2010 15:36:42   basePath = /local1/notesdata/ [376974:00002-00001] 04/19/2010 15:36:42   baseStoragePath = /local1/notesdata/DAOSREP1/ [376974:00002-00001] 04/19/2010 15:36:42   storagePath = /local1/notesdata/DAOSREP1/ [376974:00002-00001] 04/19/2010 15:36:42   pruneInterval = 0 [376974:00002-00001] 04/19/2010 15:36:42   maxFilesPerSubcontainer = 40000 [376974:00002-00001] 04/19/2010 15:36:42   maxSubcontainers = 1000 [376974:00002-00001] 04/19/2010 15:36:42   minObjectSize = 4096 [376974:00002-00001] 04/19/2010 15:36:42   catalogState = NEEDS RESYNC [376974:00002-00001] 04/19/2010 15:36:42   catalogVersion = 3 [376974:00002-00001] 04/19/2010 15:36:42   catalogMinVersion = 5 3. Why does DAOSMGR report that a resync has completed during shutdown? A series of messages similar to those following indicates that the resync thread has completed processing and has terminated. It is not an indication that the resync process has completed its task of synchronizing the DAOS catalog. > quit 05/24/2010 08:53:17 AM  Cluster Database Directory Terminating 05/24/2010 08:53:17 AM  Cluster Database Directory shutdown ... 05/24/2010 08:53:18 AM PID=3888 TID=5 RESYNC Object rebuild finished: 0x0000 05/24/2010 08:53:18 AM PID=3888 TID=5 Unknown Failed rebuilding the object entries/index in the DAOS catalog: 0000 05/24/2010 08:53:18 AM  DAOS was unable to rebuild the list of external files while trying to resynchronize. 05/24/2010 08:53:18 AM  There was an error rebuilding the catalog. 05/24/2010 08:53:18 AM PID=3888 TID=5 RESYNC Catalog rebuild finished: 0x0AC5 05/24/2010 08:53:18 AM PID=3888 TID=5 RESYNC DAOS Resync 'QUICK' mode, exiting without updating the ref counts 05/24/2010 08:53:18 AM  The DAOS catalog cannot be resynchronized. DAOS deletions will be postponed. 05/24/2010 08:53:18 AM PID=3888 TID=5 RESYNC Resync catalog done. 05/24/2010 08:53:18 AM  DAOSMGR: Resync completed 05/24/2010 08:53:20 AM  DAOSMGR: DAOS Manager terminating 4. Why doesn't the console message for DAOS resync aborting include the window of time? If the time window is set and the current time is not within the window, a warning will report that the current time is not within the resync time window, but there will be no information on when the time window is. The same behavior will occur for the "tell daosmgr resync quick" command. The issue with the time window not being reported in this warning can be tracked under the SPR SRIO84NKUU. sh config DAOS_RESYNC_START_TIME [069C:0018-0C10] DAOS_RESYNC_START_TIME=08:00:00am sh config DAOS_RESYNC_STOP_TIME [069C:0018-0C10] DAOS_RESYNC_STOP_TIME=09:00:00am tell daosmgr resync [0E10:0002-0DE0] Current Time is not within resync time slot, aborting... [009C:0002-0098] 04/19/2010 11:14:13 AM  Admin Process: Searching Administration Requests database 04/19/2010 11:14:14 AM  Agent Manager: Error validating execution rights for agent 'AAA Test Validate' in database 'dct\\lsbeexpresults.nsf'. Agent signer 'Agent Signer/Agents', effective user 'Agent Signer/Agents'. Examine 'Programmability Restrictions' However, you can use "tell daosmgr status" to report the time. 05/24/2010 08:53:09 AM  resyncWindowStartTime = 08:00:00 PM 05/24/2010 08:53:09 AM  resyncWindowEndTime = 10:00:00 PM 5. How does an administrator start the resync outside the currently set window? You can accomplish this in two ways: 1. Reset the time window using "set config DAOS_RESYNC_START_TIME=<current time>" and "set config DAOS_RESYNC_STOP_TIME=<time to stop>" and restart the resync by issuing the "tell daosmgr resync" command. If necessary, reset the time window back to what it its original setting. 2. You can considering using the FORCE option, "tell daosmgr resync quick force" or "tell daosmgr resync force". It will force a resync outside of resync time window. However, this method is less desirable because there is a time cost associated with the FORCE commands. The resync FORCE will result in aborting all the work done to synchronize the catalog up until that point and will start all over. 6. Why is Resync aborted immediately when a Resync Time Window is set? This could happen if the stop and start time were mistakenly swapped. Consider the following time window: DAOS_RESYNC_START_TIME=10:00AM DAOS_RESYNC_STOP_TIME=8:00AM If resync is started at all after 10AM, it stops immediately because the time check shows that the current time is after the stop time of 8AM.
  • XPages Preload options New options designed to improve the startup performance of XPages applications have been added to this release. A preference can be set that preloads the XPages runtime on the Domino server and Notes client. Additionally, specific XPages applications may also be preloaded. The preferences are set as entries in the NOTES.INI file and have the following form: XPagesPreload=1 XPagesPreloadDB=myServer!!myDb.nsf/myXPage.xsp,myLocalDb.nsf/foo.xsp The first entry (XPagesPreload) causes an extensive list of XPages runtime Java classes to be preloaded, so they are available in memory when an application is opened by a user. The second preference (XPagesPreloadDB) allows a comma separated list of applications to be specified, with or without a specific XPage extension. Again, any required classes are loaded when the server or client starts up and are thus available as soon as any declared application is accessed by a user. The benefit of this may be more evident when running in remote mode on the Notes client, i.e. the XPages is running on the client but the application itself resides on a remote server. In this instance, the Java classes in the application (XPages, custom controls) need to be loaded across the network in order to be executed in the local XPD web container. Preloading such applications produces more responsive initial user experience.
  • Compact, Compress, De-Duplicate (DAOS)

    1. 1. Compact, Compress, De-Duplicate Ulrich Krause DanNotes, 23. – 24. November 2011, Comwell Klarskovgaard, Korsør Denmark
    2. 2. About: Ulrich Krause <ul><li>Administrator & Developer since 1993 </li></ul><ul><ul><li>Business Application Developer </li></ul></ul><ul><ul><li>[email_address] , Ratingen, Germany </li></ul></ul><ul><li>OpenNTF </li></ul><ul><ul><li>!!HELP!!, Cebulon, Trigger Happy </li></ul></ul><ul><ul><li>Various XPages Custom Controls </li></ul></ul><ul><li>IBM Champion 2011 </li></ul><ul><ul><li> </li></ul></ul><ul><li>Blog </li></ul><ul><li>Notes Forum ( </li></ul>
    3. 3. How did I get here?
    4. 4. How did I get here?
    5. 5. How did I get here? <ul><li>Overall Distance 839 km </li></ul><ul><li>Flight SK 626 </li></ul><ul><ul><li>Düsseldorf  Kopenhagen </li></ul></ul><ul><li>Train </li></ul><ul><ul><li>Kopenhagen  Korsor </li></ul></ul><ul><li>Car </li></ul><ul><ul><li>Korsor  Venue </li></ul></ul><ul><ul><li>Thanks to Tobias !! </li></ul></ul>
    6. 6. How is today‘s situation? <ul><li>More and bigger documents and attachments </li></ul><ul><li>More expensive storage is needed </li></ul><ul><li>The average size of an application increases </li></ul><ul><li>Duplicate attachments ( n times ) </li></ul><ul><li>Duration of standard tasks increases </li></ul><ul><li>More time for Backup </li></ul><ul><li>Network traffic increases </li></ul><ul><li>I/O and performance issues </li></ul>
    7. 7. What do we want to achieve? <ul><li>Leverage existing storage </li></ul><ul><li>Reduce size of applications </li></ul><ul><li>Avoid attachment redundancies </li></ul><ul><li>Speed-up standard tasks </li></ul><ul><li>Reduce backup time </li></ul><ul><li>Reduce network traffic </li></ul><ul><li>Reduce disc I / O </li></ul>
    8. 8. Application model ( simplified ) <ul><li>Design </li></ul><ul><li>Documents </li></ul><ul><ul><li>Text </li></ul></ul><ul><ul><li>Attachments </li></ul></ul><ul><ul><li>Embedded objects </li></ul></ul>
    9. 9. Compact Secrets <ul><li>DEBUG_ENABLE_COMPACT_8_5=1 </li></ul><ul><ul><li>W nn - Only compact databases which haven't been compacted successfully in the last nn days. </li></ul></ul><ul><ul><li>w - Exclude system databases (e.g., log.nsf, names.nsf). </li></ul></ul><ul><ul><li>X nn - Limit each database compaction to no more than nn minutes. (applies to in-place compaction only) </li></ul></ul><ul><ul><li>x nn - Limit total compaction time to nn minutes. </li></ul></ul><ul><ul><li># nn - Execute up to nn compactions in parallel using different threads. </li></ul></ul><ul><ul><ul><li>Don’t use more than 8 threads in parallel </li></ul></ul></ul><ul><ul><li>ODS performs a copy-style compact only on databases with an ODS smaller than the ODS provided with this parameter. </li></ul></ul><ul><ul><ul><li>ODS 48 would convert to ODS 51 </li></ul></ul></ul>
    10. 10. Compact Secrets (Sample) <ul><li>Load compact -# 8 mail </li></ul><ul><li>Compacts all applications in the mail subdir in 8 parallel Threads </li></ul>
    11. 11. Compact via application properties <ul><li>Compacting a database does not necessarily reduce its size </li></ul><ul><li>If the application requires structural changes (typically a change of ODS [on-disk structure]). then a copy-style compaction will be performed (equivalent to -L). </li></ul><ul><li>If the application is logged (Transactional Logging enabled), then a simple in-place compaction will be performed (equivalent to -b). </li></ul><ul><li>If the application is not logged (Transactional Logging disabled), an in-place with file size reduction compaction will be performed (equivalent to -B). </li></ul>
    12. 12. Bug Alert!!! <ul><li>In-place compact does not upgrade ODS when Create_R8_Databases is set in notes.ini </li></ul><ul><li>In Lotus Domino 8, a copy-style compact will automatically occur when a database needs a structural change, regardless of the compact options specified. However, you notice that databases compacted with the -b switch are not upgraded to ODS 48, even when Create_R8_Databases=1 is set in the notes.ini file. </li></ul><ul><li>Two notes.ini parameters are required to upgrade a database to ODS 48 using an in-place compact (compact -b): </li></ul><ul><ul><li>1. Create_R8_Databases=1 </li></ul></ul><ul><ul><li>2. Dont_Inplace_Compact_ODS43=1 </li></ul></ul>
    13. 13. Copy Style Compaction <ul><li>The compact -C (copy-style compaction) (1) creates a new empty application container on the same server with the same replica ID and a .tmp suffix, (2) copies all of the documents from the original application to the new application container. </li></ul><ul><li>After all documents have been populated in the new application, (3) the old .nsf application is deleted and the (4) .tmp file is renamed to the original filename. </li></ul>
    14. 14. Defrag <ul><li>File fragmentation saps Domino server I/O performance </li></ul><ul><li>Even a slightly fragmented file can cause a 50% decrease in non-cached read performance </li></ul><ul><li>Load compact –C does not allocate space needed before compacting the application </li></ul><ul><li>Applications are fragmented during copy-style compaction </li></ul><ul><li>Full Text Indexes are fragmented too </li></ul><ul><li>Even NLO files in DAOS repository are fragmented over the time </li></ul><ul><li>Use a tool to defrag your files speed up your Domino server ! </li></ul>
    15. 15. Defrag <ul><li>DefragNSF </li></ul><ul><ul><ul><li>Adam Osborne </li></ul></ul></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Domino Defrag </li></ul><ul><ul><ul><li>Andrew Luder, OpenNTF </li></ul></ul></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>Both products use the Microsoft defragmentation API </li></ul>
    16. 16. Defrag <ul><li>Sample: Lotus Software KnowledgeBase ( 5 * compact –c ) </li></ul><ul><li>Before </li></ul><ul><ul><li>68 fragments </li></ul></ul><ul><ul><li>7550 ms </li></ul></ul><ul><li>After </li></ul><ul><ul><li>1 fragments </li></ul></ul><ul><ul><li>125 ms </li></ul></ul>
    17. 17. Defrag <ul><li>Domino Defrag uses IBM Lotus Notes C API </li></ul><ul><li>Open Source </li></ul><ul><li>FREE </li></ul><ul><ul><li>There is also a professional support plan available </li></ul></ul><ul><li>Well documented </li></ul><ul><li>32/64 Bit support </li></ul><ul><li>New in Version 3: Free Space Defragmentation </li></ul><ul><ul><li>Uses jkdefragcmd.exe ( </li></ul></ul>
    18. 18. Domino Defrag <ul><li>Xpages based administration application, logs, charts </li></ul>Uses
    19. 19. FT Index and Defrag <ul><li>When the index is updated it just about always fragments the index files </li></ul><ul><li>Little benefit for lots of work! </li></ul><ul><li>Adam Osbourne </li></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul>
    20. 20. Design Note Compression
    21. 21. Design Note Compression <ul><li>New feature as of Domino 8 </li></ul><ul><li>Needs ODS 48 or above </li></ul><ul><ul><li>Create_R8_Databases=1 </li></ul></ul><ul><ul><li>or Create_R85_Databases=1 </li></ul></ul><ul><ul><li>load compact -c </li></ul></ul><ul><ul><li>load compact -c –n </li></ul></ul><ul><li>Check “Compress database design” </li></ul><ul><li>Reduces size of design by 55 – 60 % </li></ul><ul><li>Reduces disc I / O </li></ul>
    22. 22. Design Note Compression
    23. 23. Document Body Compression
    24. 24. Document Body Compression <ul><li>New feature as of Domino 8.0.1 </li></ul><ul><li>Reduces size of Rich Text up to 60% </li></ul><ul><li>Needs ODS 48 or above </li></ul><ul><li>load compact -c –v </li></ul><ul><li>Check „Compress document data“ </li></ul>
    25. 25. Document Body Compression
    26. 26. LZ1 Attachment Compression
    27. 27. LZ1 Attachment Compression <ul><li>As of release 6 </li></ul><ul><li>Replaces Huffmann algorithm </li></ul><ul><li>NOT active when creating a new applications. </li></ul><ul><li>Recompress using </li></ul><ul><ul><ul><li>load compact -c –ZU </li></ul></ul></ul><ul><ul><ul><li>In 6.5.6 also set DEBUG_ENABLE_LZ1_REPAIR=1 </li></ul></ul></ul><ul><li>To avoid problems with JAVA applets in <= 8.5 set </li></ul><ul><ul><ul><li>DEBUG_ENABLE_LZ1_HOST_TYPES=0 </li></ul></ul></ul><ul><ul><ul><li>fixed in 8.5.1 </li></ul></ul></ul>
    28. 28. LZ1 Attachment Compression <ul><li>There have been problems reported that the fulltext search does not work as expected using LZ1 compression. </li></ul><ul><li>To revert back to Huffmann use </li></ul><ul><ul><li>Load compact –c –ZD </li></ul></ul><ul><ul><li>In 6.5.6 also set DEBUG_ENABLE_LZ1_REPAIR=1 </li></ul></ul>
    29. 29. Avoiding compression conversions <ul><li>There are circumstances that compression conversion is explicitly avoid. </li></ul><ul><li>Overall, it is to avoid the performance penalty of attempting a compression and failing back to uncompressed for file types that are already compressed. </li></ul><ul><li>The common trick is applied to observe the file extensions and avoid these. </li></ul><ul><ul><li>NONCOMPRESSABLE_FILES =<comma delimited replacement list> </li></ul></ul><ul><ul><li>Default list: zip,rar,jpg,gz,z,zz,lzo,zoo,rz,lzma,bz2 </li></ul></ul>
    30. 30. LZ1 Attachment Compression Design Text Attachments
    31. 31. DAOS Estimator
    32. 32. DAOS Estimator <ul><li>Free tool to plan your DAOS rollout </li></ul><ul><ul><ul><li>Download: </li></ul></ul></ul>
    33. 33. DAOS Estimator (notes.ini) <ul><li>DAOS Estimator by default has a large gap in attachment size evaluation between 64k and 1 MB </li></ul><ul><ul><li>DAOSEST_BUCKETS=16,64,128,256,512,768,1024,2048,3072,4096 </li></ul></ul><ul><ul><li>The above line sets to: </li></ul></ul><ul><ul><li>16K, 64K, 128K, 256K, 512K, 768K, 1MB, 2MB, 3MB and 4MB </li></ul></ul><ul><li>Source: Lotus Software Knowledge Base </li></ul><ul><ul><ul><li> </li></ul></ul></ul>
    34. 34. LZ1 != HUFFMAN <ul><li>Db1 – LZ1 enabled </li></ul><ul><li>Db2 – LZ1 disabled </li></ul><ul><li>Result is 2 .NLO Files !! </li></ul>
    35. 35. D A O S
    36. 36. Detailled scientific study
    37. 37. DAOS <ul><li>Domino Attachment Object Service </li></ul><ul><li>Attachments are no longer stored in application </li></ul><ul><li>Attachments are stored as NLO (Notes Large Object ) </li></ul><ul><li>Document contains only a ticket </li></ul><ul><li>Separates static and dynamic data </li></ul><ul><li>Only one copy of a bit-like attachment </li></ul><ul><li>Transparent to users and 3rd-Party Products </li></ul>
    38. 38. Prerequisites <ul><li>Domino 8.5 ( ODS 51) </li></ul><ul><li>SHARED MAIL OFF </li></ul><ul><li>Transaction protocol </li></ul><ul><li>Create DAOS Repository </li></ul><ul><li>Activate DAOS on server </li></ul><ul><li>Activate DAOS on application </li></ul>
    39. 39. Switch OFF Shared Mail <ul><li>Serverdocument – Shared Mail </li></ul><ul><li>Field Shared Mail = None </li></ul><ul><li>„ Load object unlink shared.nsf“ </li></ul>
    40. 40. Transaction Protocol <ul><li>Server document – Transactional Logging </li></ul><ul><li>Required for DAOS – no discussion!! </li></ul><ul><li>Use a separate disk drive. </li></ul><ul><li>Create log file folder </li></ul><ul><li>Choose method </li></ul><ul><ul><ul><li>Circular </li></ul></ul></ul><ul><ul><ul><ul><ul><li>4GB limit </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Archive </li></ul></ul></ul><ul><ul><ul><ul><ul><li>No limit, backup will delete logs </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Linear </li></ul></ul></ul><ul><ul><ul><ul><ul><li>Like circular, NO size limit </li></ul></ul></ul></ul></ul><ul><li>CREATE_R85_LOG=1 </li></ul><ul><li>Restart server </li></ul>
    41. 41. DAOS Repository <ul><li>Use separate disk drive </li></ul><ul><li>Can be SAN, NAS </li></ul><ul><li>Each server has it‘s own repository </li></ul><ul><ul><li>Max.1.000 subdirectories </li></ul></ul><ul><ul><li>Max 40.000 .NLO / subdirectory </li></ul></ul><ul><ul><li>= 40.000.000 unique .NLOs </li></ul></ul><ul><ul><li>Daos.cfg ( in Domino Data Directory) </li></ul></ul><ul><ul><li>Daoscat.nsf </li></ul></ul><ul><li>.NLO files are encrypted with </li></ul><ul><ul><li>DAOS_ENCRYPT_NLO=0 </li></ul></ul><ul><li>Include repository in backup!! </li></ul>
    42. 42. DAOS Repository <ul><li>How do you change the number of DAOS NLO's per directory? </li></ul><ul><li>DAOS_MAX_FILES_PER_SUBCONTAINER=40000 </li></ul><ul><li>Requires a server restart </li></ul><ul><li>Will only affect the current DAOS subdirectory (older subdirectories are not changed). </li></ul><ul><li>This will also affect the maximum number of files in the DAOS store which is currently this values multiplied by 1000 (this value is not configurable currently). </li></ul><ul><ul><ul><li>1.000 * DAOS_MAX_FILES_PER_SUBCONTAINER = MAX_NLO </li></ul></ul></ul>
    43. 43. Activate DAOS on server <ul><li>Goto Serverdocument – DAOS </li></ul><ul><li>Set Minimum size of an object </li></ul><ul><ul><ul><li>recommended 64kB (= DEFAULT in 8.5.1 ) </li></ul></ul></ul><ul><ul><ul><li>Evaluate with DAOS Estimator </li></ul></ul></ul><ul><li>Set DAOS base path </li></ul><ul><ul><ul><li>Put it outside the data path! </li></ul></ul></ul><ul><li>Set Delete objects in repository </li></ul><ul><ul><ul><li>Depends on your backup strategy </li></ul></ul></ul><ul><ul><ul><li>0 Days = immediate delete after the refcount = 0 </li></ul></ul></ul><ul><ul><ul><li>Manually from console: TELL DAOSMGR PRUNE 0 </li></ul></ul></ul><ul><li>Restart the server </li></ul><ul><li>show server </li></ul>
    44. 44. Activate DAOS on applications <ul><li>Application  properties  „propeller hat“ for single application </li></ul><ul><li>Use Admin client for multiple applications </li></ul><ul><li>Valid for new attachments only! </li></ul><ul><li>To move existing attachments into the DAOS repository </li></ul><ul><ul><li>Load compact –C –DAOS ON <PathNSFs> </li></ul></ul>
    45. 45. Bug Alert !!! <ul><li>load compact -c -DAOS ON mail (w/o trailing backslash) enables DAOS on ALL applications, not only on those in mail subdirectory !! </li></ul><ul><li>Documented Lotus Software Knowledge Base #1410926 </li></ul><ul><li>SPR AHOE7XHM9J ( fixed in 8.5.2, no regression in 8.5.3 ) </li></ul>
    46. 46. Bug Alert !!! <ul><li>&quot;tell daosmgr status path/DatabaseName.nsf„ </li></ul><ul><ul><li>Notice the forward slash (/) instead of a back slash () </li></ul></ul><ul><li>A programming error was found but will not be corrected. It will be a permanent restriction. Use the correct syntax </li></ul><ul><li>LO50060: ENHANCEMENT: &quot;THIS DATABASE IS NOT REGISTERED WITH DAOS&quot; </li></ul>tell daosmgr status path/DatabaseName.nsf tell daosmgr status path/DatabaseName.nsf
    47. 47. The Result
    48. 48. Transaction Protocol Trap <ul><li>load compact –c –DAOS ON </li></ul><ul><li>NO attachment in DAOS repository </li></ul><ul><li>tell daosmgr dbsummary count = 0 </li></ul><ul><li>load compact -C -T -DAOS on </li></ul>
    49. 49. DAOS and replication <ul><li>DAOS is a server feature! </li></ul><ul><li>On a client replica, all attachments are stored in the application. </li></ul><ul><li>Replicas on a NON-DAOS server store attachments in the application </li></ul><ul><li>Each server in a cluster has it‘s own DAOS repository. </li></ul>
    50. 50. DAOS and replication (cont.) <ul><li>Client  Server (as of 8.5.1) </li></ul><ul><ul><li>„ known” attachments are no longer transferred to server = „smart replication“ </li></ul></ul><ul><ul><ul><li>Forward mail </li></ul></ul></ul><ul><ul><ul><li>Reply with history </li></ul></ul></ul><ul><li>Server  Server (as of 8.5.1) </li></ul><ul><ul><li>“ known” attachments are no longer replicated to target server </li></ul></ul><ul><li>8 new statistics </li></ul><ul><ul><li>show stat DAOS </li></ul></ul>
    51. 51. DAOS and replication (cont.) <ul><li>Domino 8.5 </li></ul><ul><li>Domino 8.5.1 </li></ul>WARNING!! : ADMINP_ACCELERATED_REPLICA_OVERRIDE=4 !!
    52. 52. DAOS and replication (last slide  )
    53. 53. DAOS and <ul><li>Remove RM_NO_LOG_OBJECTS_IN_MAILBOX from notes.ini. </li></ul><ul><li>Enable DAOS on all files </li></ul><ul><li>Enable DAOS on all mail files </li></ul><ul><li>Increases performance </li></ul>
    54. 54. DAOS and Encryption <ul><li>Increases number of NLO files! </li></ul><ul><ul><li>User A writes an encrypted mail with attachment to User B </li></ul></ul><ul><ul><li>User B writes an encrypted mail with same attachment to User A </li></ul></ul><ul><ul><li>Results in 4 NLO Files in repository </li></ul></ul><ul><li>Does not save space but separates static from dynamic data </li></ul>
    55. 55. DAOS and Quotas <ul><li>Technote: 1405456 </li></ul><ul><li>Depends on the quota enforcement setting that is in place </li></ul><ul><li>Check space used in file when adding a note </li></ul><ul><ul><li>Acts on the physical size of the application (without attachments) </li></ul></ul><ul><ul><li>Possibly a problem when switching DAOS off!! </li></ul></ul><ul><ul><ul><li>64GB limit </li></ul></ul></ul><ul><li>Check file size when extending the file </li></ul><ul><li>Check file size when adding a note </li></ul><ul><ul><li>Both acts on the logical size of the application (with attachments) </li></ul></ul>
    56. 56. Keep daoscat.nsf in sync <ul><li>DON‘T use DDM !! </li></ul><ul><li>Use program document </li></ul><ul><li>Tell daosmgr resync </li></ul><ul><li>SPR# DROO7ZNPZ8 Added a new DAOSMGR resync switch to perform a quick resync. This new switch updates the DIT and DOI but does not run through the applications or updates the NLO refcounts. ( as of 8.5.1 IF1 ) </li></ul>
    57. 57. „ Needs Resync“ <ul><li>DDM points to the first application that causes the DAOS catalog to go into „Needs Resync“ state. </li></ul><ul><li>If you cannot find a cause for the issue set </li></ul><ul><li>debug_threadid=1 console_log_enabled=1 daos_logging=<path/daos.txt>STATE_CHANGE </li></ul><ul><li>Many improvements in 8.5.2 that reduces the number of state changes (SPR# RCOR83NLQK) </li></ul>
    58. 58. DAOS resync time window feature <ul><li>The following two notes.ini parameters control when resync will be allowed to run </li></ul><ul><ul><ul><ul><li>DAOS_RESYNC_START_TIME=12:00:00 AM </li></ul></ul></ul></ul><ul><ul><ul><ul><li>DAOS_RESYNC_STOP_TIME=04:00:00 AM </li></ul></ul></ul></ul><ul><li>Note that these parameters do not launch resync automatically; they just control whether or not resync may run at a given time, and also tell resync when to stop. </li></ul><ul><li>A program document could be created to launch resync just after midnight. If the catalog is SYNCHRONIZED, then the resync will not take place. However, if a resync is needed, then the resync process will begin. When the end time is reached, the resync threads will finish up the databases they are processing and exit. </li></ul>
    59. 59. DAOS OFF <ul><li>What happens when DAOS is disabled / switched OFF? </li></ul><ul><li>In application properties </li></ul><ul><ul><li>No documents with attachments in the application </li></ul></ul><ul><ul><li>Application has documents with attachments </li></ul></ul><ul><ul><ul><li>Status in Admin Client changes to „Read Only“ </li></ul></ul></ul><ul><ul><ul><li>DAOS still works for already stored attachments but no new attachments are written to the repository for the specific application. </li></ul></ul></ul><ul><li>In server document </li></ul><ul><ul><li>Same as 1 </li></ul></ul>
    60. 60. DAOS and missing NLO <ul><li>NLO has been deleted on the OS level </li></ul><ul><li>TELL DAOSMGR LISTNLO MISSING <application.nsf> </li></ul><ul><li>Output to <datadir> by default </li></ul>
    61. 61. Daosmgr in Domino 8.5.2 <ul><li>daosmgr options have been enhanced in Domino 8.5.2 </li></ul><ul><li>You can now do a search for missing NLO on all applications in a subdirectory </li></ul><ul><ul><li>tell daosmgr listnlo missing mail </li></ul></ul>
    62. 62. DAOS and missing application <ul><li>NSF has been deleted on the OS level </li></ul><ul><li>Message on server startup </li></ul><ul><ul><ul><li>Informational - The DAOS catalog is not synchronized. Deletions will be postponed. Please run 'tell daosmgr resync' at the next convenient opportunity to re-synchronize . </li></ul></ul></ul><ul><li>Has the application been deleted by accident? </li></ul><ul><ul><li>NO </li></ul></ul><ul><ul><ul><li>TELL DAOSMGR RESYNC oder RESYNC FORCE </li></ul></ul></ul><ul><ul><li>YES </li></ul></ul><ul><ul><ul><li>Restore from backup </li></ul></ul></ul><ul><ul><ul><li>Create a new replica from a cluster mate </li></ul></ul></ul><ul><li>Same behaviour when moving an application (on OS level) </li></ul>
    63. 63. Can we save another 50% <ul><li>If we disable encryption, then we could put NLO for all servers into a single repository … </li></ul><ul><li>Yes, in theory </li></ul><ul><li>It‘s only experimental </li></ul><ul><li>Does not work because each server has its own daoscat.nsf. </li></ul><ul><li>Would cause problems with pruning </li></ul>
    64. 64. DAOS and Nagios ( Statistics ) <ul><li>Catalog.Status </li></ul><ul><li>Repository.Size </li></ul><ul><li>Plugin </li></ul>
    65. 65. DAOS and Statistics <ul><li>8 new statistics (8.5.1) </li></ul><ul><ul><li>show stat DAOS </li></ul></ul>8.5.1 Debug_NSF_Show_Allstats=1
    66. 66. DAOS and Statistics <ul><li>Windows: </li></ul><ul><ul><li>@echo off </li></ul></ul><ul><ul><li>setlocal </li></ul></ul><ul><ul><li>for /f &quot;tokens=*&quot; %%a in ('dir/w/s %1 ^|findstr &quot;Datei(en)&quot;') do ( </li></ul></ul><ul><ul><li>set info=%%a </li></ul></ul><ul><ul><li>) </li></ul></ul><ul><ul><li>echo %info% </li></ul></ul><ul><li>Linux: </li></ul><ul><ul><li>#!/bin/sh </li></ul></ul><ul><ul><li>REPOSITORY=/local/daos </li></ul></ul><ul><ul><li>OUTFILE=/local/daos.txt </li></ul></ul><ul><ul><li>fCount=`find $REPOSITORY -type f | wc –l` </li></ul></ul><ul><ul><li>fSize=`du -ksb $REPOSITORY` </li></ul></ul><ul><ul><li>IFS=”/” </li></ul></ul><ul><ul><li>array=($fSize) </li></ul></ul><ul><ul><li>fDate=$(date +”%d-%m-%Y”) </li></ul></ul><ul><ul><li>echo $fDate / $fCount / ${array[0]}>> $OUTFILE </li></ul></ul>
    67. 67. Use XML
    68. 68. DAOS and daos.cfg <ul><li>Contains information on file count and file size of .NLO files </li></ul><ul><li>Created on server startup when it has been deleted </li></ul><ul><li>Updated on server shutdown </li></ul><ul><li>Do not backup or restore!! </li></ul>
    69. 69. DAOS and daoscat.nsf <ul><li>Contains index and refcount of NLO </li></ul><ul><li>Contains index of all applications using DAOS </li></ul><ul><li>Does NOT contain any view </li></ul><ul><li>Re-Created on re-sync </li></ul><ul><li>Created on server startup when it has been deleted </li></ul><ul><li>It should not be replicated, compacted, indexed, or otherwise manipulated </li></ul>
    70. 70. DAOS and daoscat.nsf <ul><li>You receive an error when opening daoscat.nsf </li></ul><ul><ul><li>Error, special noteid not found </li></ul></ul><ul><ul><li>Database is not fully initialized yet. (It does not contain any views.) you will have to use 'Replicate...' on the 'File, Replication' menu to initialize it. </li></ul></ul><ul><li>The operation that incurred the error should not be performed </li></ul><ul><li>(&quot;Doctor, it hurts when I do this&quot; &quot;Then don't do that&quot;) </li></ul>
    71. 71. Admin Client ( Files tab ) <ul><li>The application now has sizes </li></ul><ul><ul><li>Logical size </li></ul></ul><ul><ul><li>Physical size </li></ul></ul><ul><ul><li>DAOS status </li></ul></ul><ul><ul><li>DAOS count </li></ul></ul><ul><ul><li>DAOS size </li></ul></ul><ul><li>Quotas use the logical size. </li></ul><ul><li>New columns regarding compression </li></ul>
    72. 72. Backup & Restore <ul><li>No change in backing up NSF files. </li></ul><ul><li>Additional backup job for NLO files </li></ul><ul><li>NLO files do not require any special agent for backup </li></ul><ul><li>IMPORTANT : the backup sequence </li></ul><ul><ul><li>server is up during backup </li></ul></ul><ul><ul><ul><li>NSF before NLO </li></ul></ul></ul><ul><ul><li>server is down during backup </li></ul></ul><ul><ul><ul><li>it’s your choice </li></ul></ul></ul><ul><li>IBM Technote # 1358548 - DAOS Backup and Restore </li></ul>
    73. 73. FT Indexes in a different location <ul><li>Migrate existing FT Indexes by bringing down server, moving all the indexes into matching subdirectory in new FTBasePath = and restart Domino Server </li></ul><ul><li>Works on the Notes Client as well. </li></ul><ul><ul><li>Run updall -f to rebuild all FT indexes </li></ul></ul>
    74. 74. FT Indexes in a different location <ul><li>You can move FTs to local drives, while the dbs are located on a network share/homeshare, e.g. for Citrix XenApp installations </li></ul><ul><li>If you set the notes.ini and don't move the indexes, a new index will be built from scratch in the new location and the old index will be deleted... No migration but fairly user friendly when it comes to cleanup.. </li></ul><ul><li>If for some reason the FTBasePath = set in the notes.ini does not exist or server does not have proper access to, the code defaults back to the same location as the database as it always did. </li></ul>
    75. 75. ODS Tuning <ul><li>Automatic Client ODS Update </li></ul><ul><ul><li>As of release 8.5.2 </li></ul></ul><ul><ul><li>Can be configured in „Desktop Policy </li></ul></ul><ul><ul><ul><li>Desktop Policy  Mail  MailSettings </li></ul></ul></ul><ul><ul><ul><li>Or via notes.ini NSF_UpdateODS=1 + (CREATE_R85_DATABASES=1) </li></ul></ul></ul><ul><ul><ul><li>NSF_UpdatedToODS=51 </li></ul></ul></ul><ul><ul><li>Applications that are in use will be upgraded on next client startup </li></ul></ul>
    76. 76. ODS Tuning <ul><li>Technote : Upgrading multiple local databases to a new ODS </li></ul><ul><ul><li>Unfortunately the admin help that is delivered with 8.5.2 does not include any information about this feature. </li></ul></ul><ul><ul><li> </li></ul></ul>
    77. 77. ODS Tuning <ul><li>ns6, ns7 … instead of .nsf </li></ul><ul><li>compact -c does not change the ODS !! </li></ul><ul><ul><li>Even, if you revert the suffix to .nsf </li></ul></ul><ul><ul><li>Technote: 1086780 </li></ul></ul><ul><ul><li>Create new copy or replica </li></ul></ul>Notespeek Download :
    78. 78. ODS Tuning <ul><li>Upgrading the ODS of templates </li></ul><ul><ul><li>load compact -c </li></ul></ul><ul><ul><ul><li>Upgrades all.nsf files, but not templates </li></ul></ul></ul><ul><ul><li>load compact –c *.ntf </li></ul></ul><ul><ul><ul><li>ERROR: &quot;Database compactor error: File does not exist.„ </li></ul></ul></ul><ul><ul><li>load compact –c cois_crm.ntf </li></ul></ul><ul><ul><ul><li>Works </li></ul></ul></ul><ul><ul><li>Workaround: use .ind files </li></ul></ul><ul><ul><ul><li>dir *.ntf /b > templates.ind </li></ul></ul></ul><ul><ul><ul><li>load compact -c templates.ind </li></ul></ul></ul>
    79. 79. Agent Manager Retry on Compact <ul><li>There is a new setting starting with 8.5.3 and 8.5.2 FP3 to retry to run an agent after the agent has been blocked out thru a compact previously. </li></ul><ul><li>SPR# KMUR63DF3V - Fix introduces an ini DEBUG_AMGR_ENABLE_RETRY_ON_COMPACT to allow an agent to run on a time interval once database compact is complete. Previously when a database was being compacted and an attempt was made to load a scheduled agent it would fail and the agent would be marked to not run again unless the user restarted the agent manager or the agent cache refreshed. This fix introduces a notes.ini variable to allow the agent in question to be retried on it's subsequent time interval. </li></ul>
    80. 80. JavaScript/CSS Aggregation <ul><li>Dynamically aggregates multiple Dojo modules, or multiple CSS into a single file. </li></ul><ul><li>This results in the following performance improvements: </li></ul><ul><ul><li>a decrease in requests sent from the browser to the server </li></ul></ul><ul><ul><li>an increase in user performance, particularly in the context of networks with high latency </li></ul></ul><ul><ul><li>an increase in the speed of JS/CSS parsing from the browser. </li></ul></ul><ul><ul><li>the freeing up of server connections to fulfill other requests. </li></ul></ul>Server level in xsp.resources.aggregate=true
    81. 81. JavaScript/CSS Aggregation
    82. 82. XPages Preload <ul><li>New notes.ini variable as of release 8.5.3 </li></ul><ul><ul><li>Server and Client </li></ul></ul><ul><ul><li>XPagesPreload=1 </li></ul></ul><ul><ul><ul><li>LoadsXPages runtime components at server startup </li></ul></ul></ul><ul><ul><li>XPagesPreloadDB =Server!!Db.nsf/XPage.xsp,myLocalDb.nsf </li></ul></ul><ul><ul><ul><li>Loads XPages runtime Java classes for 1 to n applications </li></ul></ul></ul>
    83. 83. Resources <ul><li>DAOS Wiki </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>DAOS Backup and Restore </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>DAOS Estimator </li></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><li>DAOS Technical Overview </li></ul><ul><ul><ul><li>$FILE/DAOSTechnicalOverview_NELotus.pdf </li></ul></ul></ul>
    84. 84. Har du nogen spørgsmål <ul><li>Mail: [email_address] </li></ul><ul><li>Blog: </li></ul><ul><li>Sametime: </li></ul><ul><li>Twitter: eknori </li></ul><ul><ul><li>Tak for </li></ul></ul><ul><ul><li>jeres opmærksomhed </li></ul></ul>