Autore: Jamie Magee
Professione: Domino Developer
Risolvere i problemi di performance
delle applicazioni Domino
MartinScott Consulting
Washington, DC, USA

Chi sono?
• Co‐founder of MartinScott Consulting
–Frequent speaker at Advisor, View, LotusSphere
• Developing Lotus apps since 1994
–Designing large apps so they scale well
–Speeding up slow applications
• Creator of:
–NoteMan Toolbar for Developers/Admins
• A more powerful “Document Properties” and more!
–WirelessMail for Domino email to any device
• Blog: www.JamieMagee.com

Due punti da chiarire…
1. Aggiungere Hardware piu’ potente NON sempre
risolve i problemi di un’applicazione con un
pessimo design
– Most performance issues are best resolved in the design
of the applications FIRST, then try hardware
2. La maggior parte dei miei suggerimenti sono validi
ANCHE per il web
– HTTP server is a Web “wrapper” around the Domino
server

Di cosa ci occuperemo …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

NotesUIDocument
• PIU’ VELOCE
(se ci sono multi fields)
5x
Faster
•Updates UI doc only once,
LotusScript after all fields are written

Database Properties
(See Designer Help
document “Properties
that improve database
performance”)

Elementi di design condivisi
(shared) nel Client Notes
• Shared fields/actions, subforms, script libraries, ecc.
• Client makes one request for each element
• Se ci sono multi elementi “shared” o un network lento, c'è un
compromesso fra riusabilita’ e performance
Domino Server
document Compromesso
Notes Client
form
Subform1
Subform2
Scriptlibrary1

Elementi di design condivisi
su Web
• In web applications, requesting and assembling
shared elements is done right on the server,
without network activity for each element
– No significant effect on performance!
Domino Server
document
Web client
form
HTML
Subform1
Subform2
Scriptlibrary1

Di cosa ci occuperemo …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Indicizzazione Viste
• Il maggior problema di performance della applicazioni Domino
e’ l’indicizzazione delle viste.
• Every time a document is saved, Domino updates the indexes
of all auto‐indexed views which select the document
– FileSave, @Command([FileSave]), doc.save, form.submit(), ecc.
• Affects all applications on the server
• Indexing performance is affected by…
• Number of documents in views
• Number of fields in a document
• Frequency of update
• Number of views
• Complexity of views (categorized, # columns, formulas, etc)

Views: Colonne ordinabili dall’utente
•Ogni opzione di ordinamento su una colonna genera un
nuovo indice.
•This view has a total of 9 indexes
R8 new column property…
W arning
W arning

Viste e campi Reader
• Vieste e multo lento se…
– Ci sono multi documenti (e.g., 10’000, 100’000 +), e
– User has Read access to a small number of docs
• Domino must “scan” the view until it finds one page (usually
30) of accessible docs, or the end of the view
Warning
Warning

Viste e campi Reader
• Per una Vieste simile a questo che è più veloce…
• Embedded view, single category = @UserName (@UserNameList
for roles and groups)

Di cosa ci occuperemo …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Usare I Cascading Style Sheets (CSS)
• Separates your style and data
• Smaller HTML pages = faster loading
• CSS, JS files are cached in the browser
• Make your CSS efficient and standardized

Perchè i Doctypes sono importanti
• With the default or incomplete Doctype, the page runs in
“Quirks Mode”
– This has extra parsing for older HTML
– This slows down the document parsing which slows the page
display
• “Standards Mode” is best
– This expects well formed HTML and XHTML
– This displays the page faster
• Not noticed on the server, but faster appearance to end user

Domino e Doctypes
• By default domino publishes an incomplete doctype
– 6.5.3 notes.ini parameter sets the doc type server‐wide
• DominoCompleteDoctype=n
– 0 = Domino default
» <!DOCTYPE HTML PUBLIC quot;‐//W3C//DTD HTML 4.01 Transitional//ENquot;>
– 1 = Standard
» <!DOCTYPE HTML PUBLIC quot;‐//W3C//DTD HTML 4.01 Transitional//ENquot;
quot;http://www.w3.org/TR/html4/loose.dtdquot;>
– 2 = Strict
» <!DOCTYPE HTML PUBLIC quot;‐//W3C//DTD HTML 4.01//ENquot;
quot;http://www.w3.org/TR/html4/strict.dtdquot;>
– R7.02 ‐ a field $$HTMLFrontMatter sets the doctype per page

Usare le Web Site Rules per
migliorare il caching
• Migliorare drasticamente la cache web dell’utente:
– The user’s browser will cache many files, but it still needs to check to
see if it has the current version each time
• This can cause 2‐3 extra seconds to each page load.
– To fix this, tell the user’s browser that certain files are good for “N”
days (icons, background images, etc)
• Admin must set it, but Developer must code it

Web Site Rule per il caching
Files in:
Domain/…/cachePath/
will be cached
304 = If-Modified-Since
header variable

Web Site Rule
per files compressi GZip
• Browsers can read GZip (GNU Zip) files
– for any external files (CSS, JS, HTML, etc.)
• Come si fa a dire Domino a dire a il browser cè
un file in formato GZip?
1. Compress your file in GZip format (www.gnu.org)
2. Name the file resource with a “gz/” prefix
3. Make a Web Site Rule…

Aggiungere Web Rule per GZip
http://Abc.com/web.nsf/gz/LoginScripts.gzip
will be served as GZip

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Server Document:
Maximum Concurrent Agents
• For agent‐intensive applications
• Allows more than one agent to run at the same time
• Monitor your resource utilization after such a change to
ensure your system has enough CPU and RAM

Web Concurrent Agents
• Allow more than one agent to run at the same time
on the Web
– Enabled – Allows more than one agent to run on the Web server at the
same time (asynchronously) = FASTER
– Disabled (default) – Run only one agent at a time (serially)

NOTES.INI on server
• VIEW_REBUILD_DIR=<path>
– Use a separate physical disk for view indexing
• Greatly improves overall performance on busy servers
• Updaters = [number]
– Run multiple update tasks to keep view indexes updated
– Set [number] = number of processors
• NSF DbCache_Maxentries = [number]
– Number of databases that can be cached at one time
– Default is 25 or NSF_Buffer_Pool_Size divided by 300 KB
– If ratio of Database.DbCache.Hits to InitialDbOpen is low, then consider
increasing [number]
– Console command “Dbcache disable” closes cached databases

Di cosa ci occuperemo …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

DDM (R7 and later)
• Domino Domain Monitor (DDM.nsf) contains super agent
statistical tools… “What agent is causing problems?”
– Time to run, Memory used
• Wrapped in a Domino DB for easy reporting

Agent Profiler (R7 and later)
• Profile an Agent to get detailed performance information
– Step 1 – mark the agent to be profiled
– Step 2 – Run the agent
– Step 3 – View

Agent Profiler – Output

FiddlerTool.com
Performance delle pagine web

Undocumented Notes Feature:
Watching NRPCs
• NRPC = Notes Remote Procedure Calls
– il protocollo di comunicazione fra il server Domino e il client Notes
– Notes‐to‐Domino talk
Notes Client Domino Server
•You can watch NRPC to debug slow performance…

Undocumented Notes Feature:
Trovare i colli di bottiglia
• Enable RPC watching on Notes client
– NOTES.INI file on Notes Client
• Client_Clock=1
• Debug_Console=1
• Debug_Outfile=<path to filename>
(optional)
– Restart Notes client
– (to DISABLE, remove from NOTES.INI
and restart Notes)

Watching Notes RPCs
(1-81) OPEN_DB:
(2-83) OPEN_COLLECTION: 429 ms. [32+626=658]
(3-83) READ_ENTRIES: 8354 ms. [46+64432=64478]
(4-91) CLOSE_COLLECTION: 0 ms. [12+0=12]
(5-92) DB_REPLINFO_GET: 438 ms. [14+32=46]
(6-92) DB_REPLINFO_GET: 169 ms. [14+32=46]
...

Capire RPCs
(20-121) OPEN_NOTE: 1045 ms. [28+3906=3934]
(Seq. #), RPC_NAME, time, [bytes_sent+bytes_received=total]
• Common RPCs and What They Indicate
RPC_Name Description
Open_Session Authenticate with the server and establish a session
Open_Database Find and open a database
Open_Note Get the contents of a note (data document, design element, or ACL info)
Update_Note Save a document
Open_Collection Open a view
Read_Entries Get a list of document information from a view or search
Find_By_Key Find a document via DBLookup or LotusScript GetDocumentByKey
Get_Special_Note_ID Get ACL information
Close_dB Close database session

Monitorare Notes RPCs
(1-14 [1]) OPEN_DB(CN=Vienna01/O=MSC!!mailjmagee2.nsf): (Connect to Vienna01/MSC: 0 ms) (Exch
names: 0 ms)
(OPEN_SESSION: 561 ms)
(2-16 [3]) GET_UNREAD_NOTE_TABLE: 571 ms. [134+290=424]
(2-17 [4]) DBGETREPLICAMATCHES: 1692 ms. [290+17452=17742]
(3-17 [5]) OPEN_NOTE(REP85257246:005DD81C-NTFFFF0010,03000400): 841 ms. [176+780=956]
(3-18 [6]) OPEN_DB(CN=Vienna01/O=MSC!!mailjmagee2.nsf): 621 ms. [48+1886=1934]
(4-18 [7]) GET_NAMED_OBJECT_ID($profile_015calendarprofile_): 641 ms. [134+290=424]
(4-18 [8]) READ REPLICATION HISTORY: 541 ms. [54+24=78]
(5-18 [9]) OPEN_NOTE(REP85257246:005DD81C-NT00002FE6,00400020): (6-19 [10])
POLL_DEL_SEQNUM: 571 ms. [60+64=12
4] (Cache entry not found)
(5-19 [10]) DB_REPLINFO_GET: 821 ms. [14+32=46]
1342 ms. [48+9050=9098]
(7-20 [11]) GET_NAMED_OBJECT_ID($profile_024archive database profile_): (6-20 [12]) SEARCH:
(Connect to Vienna
01/MSC: 1382 ms) 581 ms. [64+24=88]
(8-20 [13]) OPEN_NOTE(REP85257246:005DD81C-NT000010F2,00400020): (OPEN_SESSION: 531 ms)
520 ms. [48+214=262]
(9-21 [15]) OPEN_COLLECTION(REP85257246:005DD81C-NT0000073E,0040,4008): 1292 ms.
[70+782=852]
(7-21 [16]) OPEN_NOTE(REP85257246:005DD81C-NTFFFF0040,03000400): 551 ms. [110+28=138]
1061 ms. [48+1534=1582]

NRPC Parser open source tool
– Runs on NRPC output files to translate IDs and commands
• Much easier to find and understand problems
– Developed by Jamie Magee and Andrew Magerman
– Download from www.OpenNTF.org

Di cosa ci occuperemo …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Monitoraggio risorse
• Use operating system tools
–PerfMon (Windows)
• Start Control Panel Administrative Tools
Performance
–PerfMeter (UNIX)
–PerfMon / PEX (AS/400)
• Use during live use to see bottlenecks

Monitoraggio risorse
• Example: View index rebuild showing that more RAM and
faster drive would help, but more CPU would not help
performance

Monitoraggio risorse:
capire cosa migliorare
Limiting Area of Focus Improvement
Resource Potential
Disk Access •Reduce view complexity, number, and size
•Optimize cache settings in .INI file
•Take advantage of browser caching
Processor •Reduce view complexity, number, and size
•Optimize code implementation
•Optimize server .INI file settings
•Disable server screen saver
•Take advantage of browser caching
Memory •Increase server RAM
•Optimize .INI file memory settings
Network •Increase network capacity
•Take advantage of browser caching

Di cosa ci occuperemo …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Risorse – Riferimenti
• Lotus developerWorks performance zone:
• www.redbooks.ibm.com
– www.ibm.com/developerworks/lotus/performance
• Index to all IBM performance articles
– Any Domino book with “Performance” in the name
– Several other good ones
• MartinScott.com 30‐page performance article:
– http://MartinScott.com Resources Technical Articles
• NoteMan Toolbar www.NoteMan.com
• Andre Guirard’s blog: Best Practice Makes Perfect
– http://www‐10.lotus.com/ldd/bpmpblog.nsf
• Some ideas from Kevin Marshall and Andrew Magerman
• www.FastNSF.com Server performance tips

Grazie!
Questions?
How to contact me:
JMagee@MartinScott.com

Additional slides from a longer
version of this presentation…

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Getting a Handle to a View
• NotesDatabase.GetView(viewName)
– Can be slow (e.g., 1+ second)!
– Fastest implementation is when parameter is the true
view name – not the alias Tradeoff
• Using correct case!
– Using the view alias is easier to maintain, but slower
– Don’t get the same view again and again inside a loop
LotusScript and Java

Traversing a View or Doc Collection
• For traversing a NotesView:
– GetNthDocument(N) a little faster than GetFirstDoc/GetNextDoc
• For traversing NotesDocumentCollection:
– GetFirstDocument() with GetNextDocument(doc) FASTER than
GetNthDocument() when N is large
• When only reading data from documents
– @DbLookup/@DbColumn is faster than LotusScript lookup
LotusScript and Java

Reading Values on a Doc from a
View
Status = doc.Status(0) Status = doc.ColumnValues(2)
Status is the 3rd column of the view that
Status is the 3rd column of the view that
was used to get to this document
was used to get to this document
10%
Faster
LotusScript and Java

Looping
Do For y=1 to 15000
z=z+1 z=z+1
y=y+1
Loop While y<= 15000 Next
60%
Faster
LotusScript

Put the Smaller Bounds on the
Outer Loop
for q=0 to 5000 for y=0 to 2
for y=0 to 2
for y=0 to 2 for q=0 to 5000
for q=0 to 5000
z=z+x(q,y) z=z+x(q,y)
next next
next next
400%
Faster
LotusScript

Branching with IF Statements
if a=x and totalPrice()>0 if a=x then if totalPrice()>0
(has to compute totalPrice (only computes totalPrice if a=x)
even if a <> x)
Faster
LotusScript

Profile Docs for Config Data
• Profile Documents are cached
• Built‐in programmatic access:
– Formula: GetProfileField(…)
– LotusScript/Java: NotesDatabase.GetProfileDocument(…)
• But how do you manually access profile docs?
– End users: Profile Docs (open source) form from Andre Guirard’s blog at
www.ibm.com
• Shows profile docs in the parent db containing the form
– Geeks: NoteMan Toolbar (free) shows “view” of profile docs
• Shows profile docs in any database

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

View Indexing tactics…
–Use a separate physical disk for view indexing aid
(VIEW_REBUILD_DIR notes.ini variable)
–Delete unused views
• Determine which views are really being used …

Which views are not used?
Use the Admin Client
a. Purge suspect indexes
b. Allow user activity to resume for a week/month
c. Later, look for re‐created indexes (they are active views)
d. Remove unused views (Size=0) from design

View Complexity
A p p ly in g D iffe r e n t A ttr ib u te s to V ie w C o lu m n s
N u m b e r o f M in u te s to B u ild a V ie w w ith 5 C o lu m n s
M in u te s
25
P L A IN
20 F O R M U L A S (5 )
S O R T (1 )
S O R T (4 )
15 C AT E G O R IZ E (1 )
C AT E G O R IZ E (4 )
10
5
0
1 0 0 F ie ld s
1 0 0 ,0 0 0 D o c u m e n ts

Views: Time @Functions
• Avoid @Today, @Now in view formulas
– Forces the calculation to be done every time the view is opened
H o w m a n y s e c o n d s d o e s it ta k e to In d e x a V ie w ?
Seconds
35
F ie ld N a m e
30 @ To d a y
25 @ N ow
20
15
10
5
0
10 100 1000
N u m b e r o f D o c u m e n ts
T h e s e d a ta b a s e s c o n ta in o n ly 1 V ie w a n d 1 F o rm
T h e V ie w h a s o n ly o n e c o lu m n , a n d th e S e le c tio n F o rm u la is @ A ll

Views: Time @Functions
(continued)
• To show a “view” of documents more than N days old:
– A simple daily scheduled agent moves docs in/out of the “old
documents” folder (do not use a view)
• Putting a document in a folder does not modify it ‐‐ minimizes replication
activity and conflicts
– Or, daily agent modifies hard‐coded date in view formula
• NotesView.SelectionFormula
– Or, reduce view index refresh interval …

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Should I use WebQueryOpen?
• R6, R7 and R8 have shown tremendous improvement
regarding Open Agent performance.
– The big problems were from pre R6 days.
– Use R7 Agent Profiler to probe these agents.
• More on this later
– WebQueryOpen LotusScript agents not quite as fast as, but
offer more flexible programming functionality than, formulas.

WindowOnload Problem
• Developer uses the onload event to start Web apps on page
loads
– The problem: Onload does not fire until all page content has been
loaded, including images
– If you have any scripts that depend only on the DOM to be loaded, then
use a “domReady.js” as the kick off
• Get your copy of js_domready.js on DominoPerformance.com thanks to
Dwight Wilbanks
function FormOnLoad( ) {
~ your processes
}
var FormLoader = new domFunction( FormOnLoad,quot;quot;)

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Monitoring
Transactions and Users
• Server_Show_Performance=1 in notes.ini
– Post to the server console every 60 seconds
– Number of transactions the server performs per minute and the current
number of active users on the system
– Helpful in tracking system performance over a period of time
– Transactions include router, agent manager, replication, virus scanner,
and user actions
[01A8:0021-0254] 04/10/2008 04:30:47 PM 6 Transactions/Minute, 0 Notes Users
[01A8:0021-0254] 04/10/2008 04:31:47 PM 0 Transactions/Minute, 0 Notes Users
[01A8:0021-0254] 04/10/2008 04:32:47 PM 4 Transactions/Minute, 0 Notes Users

Configuring Server Settings
• Administrators own these settings, but developers must be
aware of them
• Both parties need to collaborate to determine best settings for
each server
• Check RAM, CPU, disk utilization before and after making such
changes so that you can determine if you have enough
hardware to benefit (and not make it worse!)

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Performance Debugging: User’s
perspective indications
Performance Symptom Possible Problem
Opening a view •Time functions in view formulas
•Readers restrictions on documents
•Index property not set to Automatic
Saving a document (submitting from a browser, •Time functions in view formulas
agent‐based saves of other docs) •Large active views with broad selection
formulas
•Excessive view categorization
Creating/opening/saving document •Large, continuous tables on forms
•Excessive DbLookup/DbColumn operations
•Inefficient WebQueryOpen/WebQueryClose
LotusScript events

What is this NoteID?…
(5-18 [9]) OPEN_NOTE(REP85257246:005DD81C-NT00002FE6,00400020): (6-19 [10])
POLL_DEL_SEQNUM: 571 ms.
• Use a free tool
like NoteMan to
look up the
NoteID 00002FE6
• This item is a
Calendar Profile doc,
and took .571
seconds of my DB
open time.

What We’ll Cover …
• Sviluppare per le performance
• Gestire l’indicizzazione delle viste
• Tecniche di ottimizzazione per il Web
• Configurare il server per le performance
• Individuare i problemi di performance
• Monitorare le risorse del server
• Altre informazioni

Resource Monitoring
Limited by
Processor
• Example: agent is bound by CPU.
More RAM would not help.

Key Points to Take Home
• Beware of code and design gotchas!
• Refine and minimize view indexing, remove unused views
• Employ web standards and take advantage of fast web service
techniques
• Developers and admins should collaborate on server
configuration settings
• DDM, Agent Profiling, and NRPC monitoring can tell you a lot
about your application code
• Resource monitoring reveals the “pulse” of the server, and
indicates hardware needs

Gli Speakers
Daniele Vistalli
Fabrizio Marchesano
Giorgio Blanghetti Gli Autori
Fabio Pignatti
Daniele Grillo
Enzo Stanzione Lui
Claudio Meregalli
Marianna Tomasatti
Giuseppe Grasso
Giuseppe Bucciarelli Davide Pannuto
Nico Vis
Jamie Magee
I Nostri Sponsor