© 2007 Wellesley Information Services. All rights reserved.Best Practices toWrite, Deploy, andMonitor ScheduledAgentsBill ...
2What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• S...
3Introduction• Who is the target audience? Lotus Notes developers who use server-based agents• What is this talk about? ...
4Who Am I?• Bill Buchan• Dual Principal Certified Lotus Professional (PCLP) inDomino v3, v4, v5, v6, v7• 10+ years senior ...
5Overview• This session: Is mostly slide-based Contains a few code examples Is a deep dive in terms of theory Summariz...
6What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• S...
7Agent Manager: Introduction• It’s been in Domino since version 3• It handles both scheduled and triggered agents• It hand...
8Agent Manager: Introduction (cont.)• It changes behavior for daytime and nighttime operation Defined in the server docum...
9Agent Manager: Security• Three types of agentsecurity: Do not allow restrictedoperations (default) Allow restricted ope...
10Agent Manager: Security (cont.)• What does this mean? Restrictions based on the code you write in your agent• Restricte...
11Agent Manager: Security (cont.)• Agent security was introduced on ND6 Beware• On a v5 upgrade to v6 or v7 upgrade, ALL ...
12Agent Manager: Agent Types• Scheduled agents Schedule a repeat time period Select either “All Servers” or aparticular ...
13What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• ...
14Scheduled Agents in LotusScript• Scheduled agents are: Pieces of code that you create which can run on the server orwor...
15Scheduled Agents: Time Limit• If the agent will take a long time, it should: Record its start time Find out how long t...
16DemoDemoBrief Overview ofAgentClass
17Scheduled Agents: Setting Frequency• The agent schedule gives you a number of choices The shortest time period is five ...
18Triggered Agents• Triggered agents are: Pieces of code that can run on servers or workstations, andoperate on certain t...
19What About Agent.RunOnServer?• In LotusScript, when you use “notesagent.RunOnServer”or “tell amgr run … ” Agent manager...
20What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• ...
21Tip: Memory Space• Scheduled agents share the server memory pool• Be careful of how much memory you consume Offences, s...
22Trap: Profile Documents• Profile documents are documents that do not appear inviews, and are easily accessible from Lotu...
23Tip: Execution Time• As mentioned earlier, you have a limited amount of timeto achieve your goal with a scheduled agent•...
24Trap: Garbage Collection• Custom Classes and Lists allow you to storeinformation in the memory in a structured manner T...
25Tip: “Trusted Servers”• Before Domino 6, scheduled agents could accessdatabases on the current server only• Now, if the ...
26Trap: Run on All Servers• You can set an agent to run on all servers that thedatabase exists on Very useful for distrib...
27Tip: Cache Open Views• Every time you open a view, it may be refreshed A very time-consuming operation• Use a “list” to...
28Trap: Timing and Replication Lag• On a distributed application: Users and agents update documents Replication is used ...
29Tip: Profile Those Agents!• Domino can “profile” your agent Agent Properties, “Profile this agent” Once it has run, ri...
30Tip: Profile Those Agents! (cont.)• The agent profiles are saved in profile documents Form = “$BEProfileR7” Subject = ...
31Trap: Remote Debugging• Remote debugging should allow you to debug youragent while running on the server I haven’t hear...
32Tip: Allowing Users to Manage Scheduled Agents• One common issue is allowing non-designers inproduction environments con...
33What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• ...
34Logging — Introduction• Scheduled Agents should have: Logging. This logging should provide: Run-time information When...
35Logging — Style• However: The NotesLog model creates a new document for everylog line This means that you either have ...
36• This logging achieves: Size Each log document is small Ease of use Errors are highlighted and easy to find• Option...
37Logging — Performance• Huge log databases are slow to open and consume lotsof space if replicated to all servers• Why no...
38Monitoring — Introduction• Scheduled agents are rarely monitored Bad practice Failures are often found first by the us...
39Monitoring — Requirements• All scheduled agents should then: Be aware of which servers they start on Be aware of the a...
40DemoDemoBrief Overview ofAgentClass withMonitoring
41What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• ...
42Resources• My object-orientated programming presentation www.billbuchan.com/web.nsf/htdocs/BBUN6MQECQ.htm• Steve McConn...
43Resources (cont.)• SearchDomino.com — LotusScript Learning Guide http://searchdomino.techtarget.com/originalContent/0,2...
447 Key Points to Take Home• Agent Manager is a harsh taskmaster• Scheduled Agents should be time-sensitive Run quickly a...
457 Key Points to Take Home (cont.)• Monitoring scheduled agents: Is not hard Aids environmental stability But be sensi...
46Your Turn!How to Contact Me:Bill BuchanBill@hadsl.com
Upcoming SlideShare
Loading in …5
×

Dev buchan best practices

981 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Dev buchan best practices

  1. 1. © 2007 Wellesley Information Services. All rights reserved.Best Practices toWrite, Deploy, andMonitor ScheduledAgentsBill BuchanHADSL
  2. 2. 2What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• Scheduled agents: Monitoring• Wrap-up
  3. 3. 3Introduction• Who is the target audience? Lotus Notes developers who use server-based agents• What is this talk about? It’s difficult to imagine a complex Notes application withoutsome form of scheduled agent However, these are often deployed quickly without muchthought as to how they can be managed and operated on areliable basis This session attempts to remedy this
  4. 4. 4Who Am I?• Bill Buchan• Dual Principal Certified Lotus Professional (PCLP) inDomino v3, v4, v5, v6, v7• 10+ years senior development consultancy forEnterprise customers Learn from my pain!• 5+ years code auditing• CEO of HADSL Developing best-practice tools
  5. 5. 5Overview• This session: Is mostly slide-based Contains a few code examples Is a deep dive in terms of theory Summarizes 10+ years of enterprise code auditing
  6. 6. 6What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• Scheduled agents: Monitoring• Wrap-up
  7. 7. 7Agent Manager: Introduction• It’s been in Domino since version 3• It handles both scheduled and triggered agents• It handles @Formula, Java, and LotusScript agents• It’s a very efficient place to run code Because it’s running on the server, it benefits from all theserver database, view, and document caches
  8. 8. 8Agent Manager: Introduction (cont.)• It changes behavior for daytime and nighttime operation Defined in the server document These are the defaults Should these be changed? I don’t recommend it …
  9. 9. 9Agent Manager: Security• Three types of agentsecurity: Do not allow restrictedoperations (default) Allow restricted operations Allow restricted operations with full administration rights
  10. 10. 10Agent Manager: Security (cont.)• What does this mean? Restrictions based on the code you write in your agent• Restricted operations include: File I/O: File operations, kill, chdir, etc. Network operations (in Java) Using a disk-based NotesLog Environment variables Encryption and signing(!) Embedded objects Calling C API-based routines
  11. 11. 11Agent Manager: Security (cont.)• Agent security was introduced on ND6 Beware• On a v5 upgrade to v6 or v7 upgrade, ALL agents bydefault are set to level 1 You have to find the agents that run restricted code, and“upgrade” them Either examine every agent or use automated tooling, such asTeamstudio Analyzer www.teamstudio.com
  12. 12. 12Agent Manager: Agent Types• Scheduled agents Schedule a repeat time period Select either “All Servers” or aparticular target server• Triggered agents From a client Before and after mail delivery After document creation After document is pasted• Remember: Agents can call other agents Useful for mixing languages …
  13. 13. 13What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• Scheduled agents: Monitoring• Wrap-up
  14. 14. 14Scheduled Agents in LotusScript• Scheduled agents are: Pieces of code that you create which can run on the server orworkstation at pre-defined intervals You can choose to run it on a particular server, or ALL serversthat this database exists on They are single-threaded They have a time limit If they exceed this time limit, they will be killed In this event, the “Terminate” code is executed You may have two instances of the same agent executing atthe same time … Bear this in mind during design
  15. 15. 15Scheduled Agents: Time Limit• If the agent will take a long time, it should: Record its start time Find out how long the task should run on this server Stop processing before this time period occurs Record its state so that it can restart This might be as little as marking each document as“processed” Log its progress, and allow you to see any issues• Or: Re-architect the solution to avoid this• This sounds difficult …
  16. 16. 16DemoDemoBrief Overview ofAgentClass
  17. 17. 17Scheduled Agents: Setting Frequency• The agent schedule gives you a number of choices The shortest time period is five minutes Remember, during the day there is, by default, only one agentmanager thread If you have hundreds of “five-minute” agents scheduled forfive minute intervals, the Agent Manager thread will beoverloaded …• If you need more frequent time periods, re-architect thesolution by using triggers Is this triggered by a mail-in document, document paste, etc.? Alternatively, use TriggerHappy Open source project, www.openntf.org Can trigger LotusScript agents on Extension Managerevents
  18. 18. 18Triggered Agents• Triggered agents are: Pieces of code that can run on servers or workstations, andoperate on certain triggers Before or after a document has been mailed to a database When a document is pasted into a database When a document has been updated• Agent manager has mechanisms to ensure that it doesNOT trigger too often Usually needs at least two minutes between each agent run Mail-in agents may not trigger enough: So if you have to rely on a mail-in database, create anothermechanism to pick up all “unprocessed” documents, suchas a status view
  19. 19. 19What About Agent.RunOnServer?• In LotusScript, when you use “notesagent.RunOnServer”or “tell amgr run … ” Agent manager appears to spawn a new agent thread The agent does not get terminated if it exceeds the agentrun-time on the server document The agent appears to run in its own memory space There is no connection to the agent You cannot tell it to quit• This means: Try not to use this technique in production If you do, be especially careful about: Making sure it terminates Logging all activity
  20. 20. 20What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• Scheduled agents: Monitoring• Wrap-up
  21. 21. 21Tip: Memory Space• Scheduled agents share the server memory pool• Be careful of how much memory you consume Offences, such as keeping large number of NotesDocumentobjects open at any time, should be avoided• If you have to “read” a large number of documents, dosome operation, then update a small number of them Read the comparison information and the NoteID into memory(Custom Classes and Lists are good for this) Do the operation When updating the document, use the NoteID to quicklyreopen and save the document
  22. 22. 22Trap: Profile Documents• Profile documents are documents that do not appear inviews, and are easily accessible from LotusScript and@Formula language Very useful for configuration settings, etc.• However, Agent Manager tends to cache these If the profile document contains rapidly changing information,there is a danger that your agent will not receive the mostup-to-date information• So: Don’t store rapidly changing information in a profile document
  23. 23. 23Tip: Execution Time• As mentioned earlier, you have a limited amount of timeto achieve your goal with a scheduled agent• If you are in danger of exceeding this, re-architect For instance, if this agent: Runs once per day Collects all “outstanding” documents Processes them Then have this agent: Run multiple times Store its last completion point and run from that point
  24. 24. 24Trap: Garbage Collection• Custom Classes and Lists allow you to storeinformation in the memory in a structured manner This leads to simpler code and faster execution• However: If you attempt to “clean up” heavily interlinked objects in yourcode, you may confuse the garbage collector It may crash Agent Manager, the server, or a client process• So: LotusScript’s garbage collector is pretty good Just let it happen naturally
  25. 25. 25Tip: “Trusted Servers”• Before Domino 6, scheduled agents could accessdatabases on the current server only• Now, if the “Trusted Servers” field is populated, youmay access databases on other servers• This means that you are no longer required to havereplicas of databases on every server in order to collectinformation from that server No more “replication storms” However, the link to the other server has to be reliable
  26. 26. 26Trap: Run on All Servers• You can set an agent to run on all servers that thedatabase exists on Very useful for distributing workload• Be careful to make wide-ranging changes on oneinstance of the database A worst practice story: 100K document database, 15 instances Agent set to update all documents — every HOUR oneach server Replication pushes out design elements first Result: 4.8 million new documents Database grew from 3GB  15GB!
  27. 27. 27Tip: Cache Open Views• Every time you open a view, it may be refreshed A very time-consuming operation• Use a “list” to keep all open views in a central locationDim allViews list as NotesViewSet allViews(“nabNames”) = dbNAB.getView(“($People)”)Set allViews(“Lookup”) = dbThis.GetView(“vLookup”)…Set doc = allViews(“Lookup”).getFirstDocument…
  28. 28. 28Trap: Timing and Replication Lag• On a distributed application: Users and agents update documents Replication is used to move documents around• Remember, when looking at document round-trip time,budget at least: Two replication cycles One Agent Manager cycle• You might lessen the agent cycle time to reducereplication conflicts
  29. 29. 29Tip: Profile Those Agents!• Domino can “profile” your agent Agent Properties, “Profile this agent” Once it has run, right click and select“View Profile Results”
  30. 30. 30Tip: Profile Those Agents! (cont.)• The agent profiles are saved in profile documents Form = “$BEProfileR7” Subject = AgentName + “Profile” Results in body text You can programmatically find and analyze these• Check out Teamstudio’s Profiler tool Can give you run-time analysis of your own functions Also, free tools include: Class Browser Call Tree Analysis, etc. http://blogs.teamstudio.com
  31. 31. 31Trap: Remote Debugging• Remote debugging should allow you to debug youragent while running on the server I haven’t heard of ANYONE getting this to work successfully• My advice: Test scheduled agents by constructing test harnesses you canrun manually• This means: The agents should have as little code in them as possible All your code should be in script libraries!
  32. 32. 32Tip: Allowing Users to Manage Scheduled Agents• One common issue is allowing non-designers inproduction environments control over agents Specifically, how often they run, on which servers, etc.• Rescheduling an agent usually means: Updating the template and refreshing the database design However, in larger environments, this may be impractical• One approach is to: Schedule the agent to run frequently on all servers Check a configuration document within the same database tosee if this agent should run at this time on this server Remember: Profile documents may cache and not shownew information for a significant period of time
  33. 33. 33What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• Scheduled agents: Monitoring• Wrap-up
  34. 34. 34Logging — Introduction• Scheduled Agents should have: Logging. This logging should provide: Run-time information When did the agent start, where, and why? How many operations did it process? Error information For instance, if a run-time error was triggered, whereand how was it triggered? If there was a data error,where did this occur? Did the agent then processbeyond this data error?
  35. 35. 35Logging — Style• However: The NotesLog model creates a new document for everylog line This means that you either have minimal logging, or ahuge log file• I recommend: You develop your own “ticker-tape” style log facility Similar to Notes log.nsf Color code different levels of error for easy debugging Count errors on the log view
  36. 36. 36• This logging achieves: Size Each log document is small Ease of use Errors are highlighted and easy to find• Optional enhancements Have a configuration document that shows only loggingdocuments up to a particular log levelLogging — Style (cont.)
  37. 37. 37Logging — Performance• Huge log databases are slow to open and consume lotsof space if replicated to all servers• Why not mail log documents to a central log database? Fast Your agent doesn’t have to open the log Easy It’s very easy to set up a mail-in database and send the logdocument via mail Secure You don’t have to grant author access for all users
  38. 38. 38Monitoring — Introduction• Scheduled agents are rarely monitored Bad practice Failures are often found first by the users• Business-critical agents should: Create a status document at the end of each run Possibly mail that status document to a central location Another agent should monitor those responses, and alertoperations if a status document is overdue
  39. 39. 39Monitoring — Requirements• All scheduled agents should then: Be aware of which servers they start on Be aware of the amount of time they can run for Be aware of their final status Report success/failure criteria Be able to report other statistical information• A central log database should then: Be aware of which databases, which servers, and whichscheduled agents are expected to report Raise alerts if these reports are overdue
  40. 40. 40DemoDemoBrief Overview ofAgentClass withMonitoring
  41. 41. 41What We’ll Cover …• Overview• Agent Manager introduction• Agent Manager: Scalability• Scheduled agents: Tips and traps• Scheduled agents: Monitoring• Wrap-up
  42. 42. 42Resources• My object-orientated programming presentation www.billbuchan.com/web.nsf/htdocs/BBUN6MQECQ.htm• Steve McConnell, Code Complete, Second Edition,(Microsoft Press, 2004). www.amazon.com/gp/product/0735619670• OpenNTF projects — OpenLog and Stubby www.OpenNTF.org• The Notes FAQ! www.keysolutions.com/NotesFAQ
  43. 43. 43Resources (cont.)• SearchDomino.com — LotusScript Learning Guide http://searchdomino.techtarget.com/originalContent/0,289142,sid4_gci1001826,00.html• Brian Benz and Rocky Oliver, Lotus Notes and Domino 6Programming Bible (Wiley, 2003). www.amazon.com/gp/product/0764526111• THE VIEW Mark Roden, “Simple Methods for Creating HTML-FormattedE-mail in Domino Web Applications and ScheduledAgents” (May/June 2003). www.eView.com
  44. 44. 447 Key Points to Take Home• Agent Manager is a harsh taskmaster• Scheduled Agents should be time-sensitive Run quickly and behave well Profile your agents to understand where they spend their time• Object-orientated techniques can: Help manage complexity Add standard code• “If you can’t measure it, you can’t manage it”
  45. 45. 457 Key Points to Take Home (cont.)• Monitoring scheduled agents: Is not hard Aids environmental stability But be sensitive to how much logging information yourapplication generates• Ensure that your agent security level is set correctly• Avoid “notesAgent.RunOnServer” in production code It may create unmanageable Agent Manager “zombie”processes …
  46. 46. 46Your Turn!How to Contact Me:Bill BuchanBill@hadsl.com

×