Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BP101 - 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments

2,632 views

Published on

Many common development techniques can cause dramatic effects when your application is rolled out over hundreds of servers. As a developer, you need a good understanding of certain parts of the infrastructure to build an application designed for wide-scale deployment. System administrators who review applications before deployment should know what to look for in the code to prevent problems when rolled out to production. This session takes a look at the area where Application Development and System Administration come together. You will hear about real-life problems, view examples of bad code as well as good code, and learn what you should consider when you have to develop or deploy an application which will be rolled out in a large-scale deployment, or how to "harden" your code to support large quantities of documents.

Published in: Technology
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, eBooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

BP101 - 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments

  1. 2. 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments <ul><ul><li>Martijn de Jong | ilionx </li></ul></ul>BP101
  2. 3. Agenda <ul><li>Speaker Introduction </li></ul><ul><li>Examples of Large Scale Environment Applications </li></ul><ul><li>Characteristics of a Large Scale Environment </li></ul><ul><li>10 Things to Consider... </li></ul><ul><li>Wrap-up </li></ul><ul><li>Q & A </li></ul>
  3. 4. Speaker Introduction <ul><li>So who am I? </li></ul><ul><ul><li>M.Sc. Electrical Engineering at the University of Delft, The Netherlands </li></ul></ul><ul><ul><li>Psychology & Ergonomics at the University of Stellenbosch, South Africa </li></ul></ul><ul><ul><li>Advanced Certified IBM Lotus® Notes® & Domino® 8 Application Developer & System Administrator and a Certified Lotus Instructor </li></ul></ul><ul><ul><li>2000: Lotus Notes trainer at IBM </li></ul></ul><ul><ul><li>2002: Started with Domino Development yet moved to System Administration job </li></ul></ul><ul><ul><li>2004: Lotus Consultant at ilionx </li></ul></ul><ul><ul><li>2006: Application Developer for the ABN AMRO Bank Messaging & Groupware Architecture Team </li></ul></ul><ul><ul><ul><li>500+ Domino servers </li></ul></ul></ul><ul><ul><ul><li>135.000+ Notes users </li></ul></ul></ul><ul><ul><ul><li>Single domain </li></ul></ul></ul><ul><li>I blog at http://www.socialsoftwareblog.nl </li></ul><ul><li>Contact me at: mdejong@ilionx.com / martdj@martdj.nl </li></ul>
  4. 5. Examples of Large Scale Environment Applications <ul><li>System to maintain & shape the Mail Infrastructure </li></ul>User's Mail files Domino Directory HRM Systems Information Store Users Admins Messaging Portal Mail file Tooling Mail file Upgrade Tool Server Logs
  5. 6. Examples of Large Scale Environment Applications <ul><li>System for requesting, monitoring & maintaining mobile devices </li></ul>Users Providers Support SQL Database Servers Collector Front Central External Parties Internal Support
  6. 7. Characteristics of a Large Scale Environment <ul><li>Many servers </li></ul><ul><li>Many users – Large Domino Directory </li></ul><ul><li>Large number of applications </li></ul><ul><li>Large amounts of documents in some applications </li></ul><ul><li>Many daily updates in some applications </li></ul><ul><li>Strict procedures </li></ul><ul><li>Separation between Developer & Infrastructure Teams </li></ul><ul><li>Some connections between servers have limited bandwidth </li></ul>
  7. 8. What does this mean for a developer? <ul><li>Compared to developing for smaller environments, developers have to give extra consideration to: </li></ul><ul><ul><li>System Administration costs </li></ul></ul><ul><ul><li>Efficient usage of system resources (disk space, cpu time, memory usage) </li></ul></ul><ul><ul><li>Network load </li></ul></ul><ul><ul><li>Security requirements </li></ul></ul>
  8. 9. 10 Things to Consider...
  9. 10. 10 Things to Consider...
  10. 11. 1. Consider the Deployment Effort <ul><li>Avoid multiple unique copies of the same database for one application </li></ul><ul><li>Replication Settings don't replicate. If you need them to be different from the default, set them with an agent </li></ul><ul><ul><li>Example code to set the Deletion Stub Cutoff Period Set repinfo = thisdb.ReplicationInfo ' Grab the preferred deletion stub cutoff date from a profile document CutoffPeriod = Clng(profdoc.GetItemValue(&quot;prfDelStubCutoff&quot;)(0)) If repinfo.CutoffInterval <> CutoffPeriod Then repinfo.CutoffInterval = CutoffPeriod repinfo.Save End If </li></ul></ul><ul><li>“ Don't allow simple search” also doesn't replicate </li></ul><ul><ul><li>Call thisdb.SetOption(76, True) ' sets don't allow simple search </li></ul></ul>
  11. 12. 1. Consider the Deployment Effort (2) <ul><li>It's easier to deploy an application on all servers than on 60% of the servers </li></ul><ul><li>It should be possible to deploy a new version of your application by replacing the design on the administration server. If this is not the case, think of a way to make it so </li></ul><ul><li>Deploying a replica of your application on a new server should just be a matter of creating a new replica. If any other actions are necessary, consider building extra code to automate them </li></ul><ul><li>Remember: The more time it takes for the administrators to deploy your application, the more time it will take you to verify their work! </li></ul>
  12. 13. 2. Design Flexibility <ul><li>Deploying design changes can be a lengthy process in larger organisations. Minimize the need by keeping your designs as flexible as possible </li></ul><ul><li>Examples </li></ul><ul><ul><li>Don't hard code mails in your design. Keep content in profile documents or use special memo documents </li></ul></ul>
  13. 14. 2. Design Flexibility <ul><li>Deploying design changes can be a lengthy process in larger organisations. Minimize the need by keeping your designs as flexible as possible </li></ul><ul><li>Examples </li></ul><ul><ul><li>Don't hard code mails in your design. Keep content in profile documents or use special memo documents </li></ul></ul><ul><ul><li>Don't use the Style Sheets design element. Keep your CSS code in documents </li></ul></ul><ul><ul><li>Run agents on -Any Server- and use profile documents to define the server(s) they should run on </li></ul></ul>
  14. 15. 3. How to run your agent where and when you'd want them to run <ul><li>Run agents on “-Any Server-” . Let them run as often as needed </li></ul><ul><li>Use a profile document for selection </li></ul><ul><li>Can be expanded by adding time slots for the agents </li></ul>
  15. 16. 3. How to run your agent where and when you'd want them to run <ul><li>Code example: Set profdoc = thisdb.GetProfileDocument(&quot;prfDb&quot;) ' Don't run the agent on servers that it shouldn't run on arrRunServerGroups = profdoc.GetItemValue(&quot;prfAgentServerGroups&quot;) [..] For i = 0 To Ubound(arrRunServerGroups) If Not Isnull(Arraygetindex(ExpandGroup(arrRunServerGroups(i), False), thisdb.Server)) Then blnServerMatched = True Exit For End If Next ' Exit if it's not in any of the groups If Not blnServerMatched Then Exit Sub ' Check if the server is in the exclude list NotRunServers = profdoc.GetItemValue(&quot;prfNonMFIServers&quot;) If Isnull(Arraygetindex(NotRunServers, thisdb.Server)) Then ' Next check if the agent should run today Call htoday.SetNow collectdays = profdoc.GetItemValue(&quot;prfMTCollectDays&quot;) If Not Isnull(Arraygetindex(collectdays,Weekday(htoday.DateOnly))) Then <Your agent code here> End If End If </li></ul>
  16. 17. 4. Special Signer IDs <ul><li>Many organisations require a special signer ID for agents that: </li></ul><ul><ul><li>Run unrestricted methods and operations </li></ul></ul><ul><ul><li>Send Mail </li></ul></ul><ul><li>Try to limit the number of these types of agents </li></ul><ul><ul><li>Example: Let all agents that need to send mail, create the mail and save it. Use one agent to send the mail </li></ul></ul>
  17. 18. 10 Things to Consider...
  18. 19. 5. Do you really need that view? <ul><li>In applications with many documents, views can take up big chunks of disk space and processing power of the server </li></ul><ul><li>It's better to build a few good user views than many different views which a user might want </li></ul><ul><li>Using an extra sortable column is better than building an extra view </li></ul><ul><li>99% of the time there's no need for a double sortable column </li></ul><ul><li>For lookup views, describe (in the comment field) which design elements use that view </li></ul><ul><li>Never use @Now/@Today in a view </li></ul><ul><ul><li>Alternative: See AD214: What Every Domino Developer Should Know About Performance </li></ul></ul>
  19. 20. 6. Do your agents run out of time? <ul><li>How long may your agents run: </li></ul><ul><ul><li>Agent running in the client: Indefinitely </li></ul></ul><ul><ul><li>Agent running on the server: Depends on type, trigger & time </li></ul></ul><ul><ul><ul><li>Before new mail arrives: Runs in the Router Task </li></ul></ul></ul>(Configuration Document)
  20. 21. 6. Do your agents run out of time? <ul><li>How long may your agents run: </li></ul><ul><ul><li>Agent running in the client: Indefinitely </li></ul></ul><ul><ul><li>Agent running on the server: Depends on type, trigger & time </li></ul></ul><ul><ul><ul><li>Before new mail arrives: Runs in the Router Task </li></ul></ul></ul><ul><ul><ul><li>WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task </li></ul></ul></ul>(Server document - Internet Protocols - Domino Web Engine)
  21. 22. 6. Do your agents run out of time? <ul><li>How long may your agents run: </li></ul><ul><ul><li>Agent running in the client: Indefinitely </li></ul></ul><ul><ul><li>Agent running on the server: Depends on type, trigger & time </li></ul></ul><ul><ul><ul><li>Before new mail arrives: Runs in the Router Task </li></ul></ul></ul><ul><ul><ul><li>WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task </li></ul></ul></ul><ul><ul><ul><li>Scheduled / triggered: Runs in the Amgr Task </li></ul></ul></ul>Simple / Formula Agents (Server Document - Basics)
  22. 23. 6. Do your agents run out of time? <ul><li>How long may your agents run: </li></ul><ul><ul><li>Agent running in the client: Indefinitely </li></ul></ul><ul><ul><li>Agent running on the server: Depends on type, trigger & time </li></ul></ul><ul><ul><ul><li>Before new mail arrives: Runs in the Router Task </li></ul></ul></ul><ul><ul><ul><li>WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task </li></ul></ul></ul><ul><ul><ul><li>Scheduled / triggered: Runs in the Amgr Task </li></ul></ul></ul>LotusScript / Java Agents (Server Document – Server Tasks – Agent Manager)
  23. 24. 6. Do your agents run out of time? <ul><li>How long may your agents run: </li></ul><ul><ul><li>Agent running in the client: Indefinitely </li></ul></ul><ul><ul><li>Agent running on the server: Depends on type, trigger & time </li></ul></ul><ul><ul><ul><li>Before new mail arrives: Runs in the Router Task </li></ul></ul></ul><ul><ul><ul><li>WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task </li></ul></ul></ul><ul><ul><ul><li>Scheduled / triggered: Runs in the Amgr Task </li></ul></ul></ul><ul><ul><ul><li>tell amgr run “<app name>” '<agent name>' : Runs in a separate Amgr Task Agent can run indefinitely... </li></ul></ul></ul><ul><ul><ul><li>“ Call notesagent.RunOnServer” : Server Task Agent can run indefinitely... </li></ul></ul></ul>
  24. 25. 6. Do your agents run out of time? <ul><li>Five ways to deal with agents running out of time </li></ul><ul><ul><li>Optimize them </li></ul></ul><ul><ul><ul><li>AD214: What Every Domino Developer Should Know About Performance </li></ul></ul></ul><ul><ul><li>Split up your agent </li></ul></ul><ul><ul><ul><li>For completely separate actions only </li></ul></ul></ul><ul><ul><li>Divide the workload in parts and run the agent multiple times </li></ul></ul><ul><ul><ul><li>Use a key field to recognize where the agent was cut off last </li></ul></ul></ul><ul><ul><ul><li>Run small batches divided on alphabet or DocumentUniqueID </li></ul></ul></ul><ul><ul><li>Divide the workload over multiple servers </li></ul></ul><ul><ul><ul><li>Make each server responsible for specific documents based on a field value or another key </li></ul></ul></ul><ul><ul><li>A combination </li></ul></ul>
  25. 26. 10 Things to Consider...
  26. 27. 7. Do you need all documents in all replicas? <ul><li>Sometimes a document is only relevant for a specific region, cluster or even a single server </li></ul><ul><li>Examples </li></ul><ul><ul><li>The Administration Requests database </li></ul></ul><ul><ul><li>The Database Catalog </li></ul></ul><ul><li>If your applications contains large quantities of these documents consider... </li></ul>Replication Formulas
  27. 28. 7. Do you need all documents in all replicas? (2) <ul><li>Any Server – Any Server replication formula has some problems (easily gets overruled by a server specific formula) </li></ul><ul><li>Better to use <Server Name> – Any Server replication formulas </li></ul><ul><li>Set these replication formulas with an agent (consider deployment effort) </li></ul><ul><li>Use @UserName for the current server </li></ul><ul><li>Disable replication of Replication Formulas </li></ul><ul><li>Deletion stubs, profile documents & agent data documents always replicate </li></ul><ul><li>Watch out for Domino 6.0.3 / 6.5.0 when using replication formulas </li></ul>
  28. 29. 7. Do you need all documents in all replicas? (3) <ul><li>Example of agent to set Replication Formula CreateRepFormula = True Set repinfo = thisdb.ReplicationInfo ' Check if a proper replication formula entry exists Set repentry = repinfo.GetEntry(&quot;-&quot;, thisdb.Server, False) If Not repentry Is Nothing Then If repentry.Views = &quot;&quot; Then If repentry.IsIncludeDocuments Then If repentry.Formula = | @IsMember(@UserName;Servers)| Then If repentry.IsIncludeACL Then If repentry.IsIncludeAgents Then If repentry.IsIncludeForms Then If Not repentry.IsIncludeFormulas Then CreateRepFormula = False End If [..] ' If it doesn't, create/modify it If CreateRepFormula Then Set repentry = repinfo.GetEntry(&quot;-&quot;, thisdb.Server, True) repentry.IsIncludeACL = True repentry.IsIncludeAgents = True repentry.IsIncludeDocuments = True repentry.IsIncludeForms = True repentry.IsIncludeFormulas = False repentry.Views = &quot;&quot; repentry.Formula = | @IsMember(@UserName;Servers)| repentry.Save End If </li></ul>
  29. 30. 8. Beware of deletion stubs <ul><li>How do deletion stubs expire: </li></ul><ul><ul><li>For default settings (90 days) deletion stubs are removed once every 30 days </li></ul></ul><ul><ul><li>On 'expiry day' all deletion stubs older than 90 days are removed </li></ul></ul>
  30. 31. 8. Beware of deletion stubs <ul><li>How do deletion stubs expire: </li></ul><ul><ul><li>For default settings (90 days) deletion stubs are removed once every 30 days </li></ul></ul><ul><ul><li>On 'expiry day' all deletion stubs older than 90 days are removed </li></ul></ul>
  31. 32. 8. Beware of deletion stubs (2) <ul><li>You know you have a problem with deletion stubs when: </li></ul><ul><ul><li>Creating a new copy of your database takes less than a minute and 5 MB of disk space </li></ul></ul><ul><ul><li>Creating a new replica of your database takes over an hour and 125 MB of disk space </li></ul></ul><ul><li>Cause: I copy-pasted code from another application which weekly removed all documents and recreated them instead of updating them </li></ul>
  32. 33. 8. Beware of deletion stubs (3) <ul><li>In case your documents have fields that need to be removed for certain cases, this code works: </li></ul><ul><ul><li>Sub EmptyDocument(doc As NotesDocument) Dim Allitems As Variant Dim i As Integer Allitems = doc.Items If Not Isempty(Allitems) Then For i = 0 To Ubound(Allitems) ' reason for the if then are docs with multiple fields of the same name If Not Allitems(i) Is Nothing Then Call Allitems(i).Remove Next End If End Sub </li></ul></ul><ul><li>(Only for extreme cases) </li></ul>
  33. 34. 9. Replication vs Mail vs Data Copy Agent <ul><li>To process documents on a specific server you can use either mail, replication or a Data Copy Agent to get it there </li></ul>
  34. 35. 10. Try to limit your replication traffic <ul><li>Replication Formulas can help </li></ul><ul><li>Don't save documents unless you have to Set doc = vw.GetFirstDocument While Not doc Is Nothing Set ndoc = vw.GetNextDocument(doc) ' Pick out the mail documents. Leave the rest for the process action agent If doc.GetItemValue(&quot;Form&quot;)(0) = &quot;Memo&quot; Then Call doc.ReplaceItemValue(&quot;Form&quot;, &quot;Action&quot;) Else If doc.GetItemValue(&quot;Form&quot;)(0) = &quot;Reply&quot; Then Call SendRejectMail(doc.GetItemValue(&quot;From&quot;)(0)) Call doc.ReplaceItemValue(&quot;Processed&quot;, &quot;1&quot;) End If End If Call doc.Save(False, False) Set doc = ndoc Wend </li></ul>Bad
  35. 36. 10. Try to limit your replication traffic <ul><li>Replication Formulas can help </li></ul><ul><li>Don't save documents unless you have to Dim ShouldSave As Boolean Set doc = vw.GetFirstDocument While Not doc Is Nothing Set ndoc = vw.GetNextDocument(doc) ShouldSave = False ' Pick out the mail documents. Leave the rest for the process action agent If doc.GetItemValue(&quot;Form&quot;)(0) = &quot;Memo&quot; Then Call doc.ReplaceItemValue(&quot;Form&quot;, &quot;Action&quot;) ShouldSave = True Else If doc.GetItemValue(&quot;Form&quot;)(0) = &quot;Reply&quot; Then Call SendRejectMail(doc.GetItemValue(&quot;From&quot;)(0)) Call doc.ReplaceItemValue(&quot;Processed&quot;, &quot;1&quot;) ShouldSave = True End If End If If ShouldSave Then Call doc.Save(False, False) Set doc = ndoc Wend </li></ul>
  36. 37. 10. Try to limit your replication traffic (2) <ul><li>Don't flag documents as a workaround for @Today view selection formulas </li></ul><ul><li>During the design phase of your application, try to make a good estimate of the expected number of daily document additions/updates of each type of document in your application </li></ul>
  37. 38. 10 Things to Consider...
  38. 39. Wrap-Up (2) <ul><li>When administrators and developers cooperate they can make better performing applications </li></ul><ul><li>Recommended Presentations </li></ul><ul><ul><li>AD214 - What Every Domino Developer Should Know About Performance </li></ul></ul><ul><ul><li>AD210 - Built to Last: IBM Lotus Domino Development Strategies </li></ul></ul><ul><ul><li>BP102 - A New Approach to Internationalizing IBM Lotus Domino Applications </li></ul></ul><ul><li>Code example database can be downloaded from: http://www.martdj.nl/martijn/bp101.zip </li></ul><ul><li>You can contact me at: mdejong@ilionx.com / martdj@martdj.nl </li></ul>
  39. 40. Q & A <ul><li>Q? </li></ul>
  40. 41. <ul><li>Thank You! </li></ul><ul><li>Please remember the session survey </li></ul>
  41. 42. Legal disclaimer <ul><li>© IBM Corporation 2008. All Rights Reserved. </li></ul><ul><li>The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. </li></ul><ul><li>References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. </li></ul><ul><li>All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. </li></ul><ul><li>IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. </li></ul><ul><li>Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. </li></ul>

×