071410 sun a_1515_feldman_stephen


Published on

Optimizing BbLearn 9.1 for High Performance

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

071410 sun a_1515_feldman_stephen

  1. 1. Best Practices for Optimizing Blackboard Learn Steve Feldman, sfeldman@blackboard.com
  2. 2. optimizing* The process of increasing speed (performance) and capacity (scalability) of a system and/or program.
  3. 3. What We’ll Cover •  Deploy for performance from the start. •  Optimizing the platform components. •  Continuous measurements are absolutely critical.
  4. 4. Scalable Deployments
  5. 5. Flexible and Scalable Application Deployment
  6. 6. Deployment: Large Address Space •  As of Blackboard Learn™ Release 9.1 all supported/ certified configurations include a 64-bit option. •  Pushing more processing to client and DB over the last few releases, but major memory management technique is to use more application caches. –  Memory stays persistent longer –  Less wasteful from a creation/destruction perspective, but puts greater demands on larger spaces. •  Most of our application testing focused on 4GB and 8GB JVM deployments on 6GB and 10GB OS spaces. –  Limited testing at 16GB and 32GB
  7. 7. Deployment: Resource Utilization via Virtualization •  Moore’s law is in full effect –  CPUs are getting faster with more cores –  Memory is in abundance and cheap –  Storage is grossly abundant •  Massive systems can be obtained at low cost, but cannot be saturated in stand-alone configurations. •  Virtualization offers this opportunity… –  Deploy with availability in mind –  Saturate system resources –  Save money and data center space
  8. 8. Deployment: Improving Page Responsiveness •  Gzip…Gzip…Gzip… –  All of our supported browsers handle gzip? –  Reduces payload •  Improves lower latency connections like Cable, DSL and Dial-up –  Minor overhead on the application layer (~2% to ~5%) •  Have the option to perform at the load-balancer layer –  Most Bb deployments do not enable Gzip at all •  Even when enabled, some proxies and software packages mess-up the Accept Encoding Header •  Optimize your images –  Page size really does matter –  Reduce the size without reducing the quality –  Smush.it, PunnyPng, OptiPng, RIOT and ImageOptim
  9. 9. Example Page Waterfall
  10. 10. Tools for Page Performance •  Analysis Tools –  Independent tools: Fiddler, HTTPWatch, WebPageTest.org, Web Inspector (Mac), IE Developer Tools –  Browser Plugins: Firebug/YSlow, Google Page Speed, WebPageTest Client –  JavaScript tools: dynatrace, IE Developer Tools, •  Optimization Tools –  Image optimization to reduced overall size and metadata: SmushIt, PNGOptimize, PNGCrusher, OptiPng and graphicsoptimization.com •  http://yuiblog.com/blog/2008/10/29/imageopt-1/ –  CSS optimization to reduce unused elements and compress CSS of unnecessary white spaces: CSS Optimizer, CSS Compressor and CleanCSS
  11. 11. Optimizing the Web Server •  The web server in the BbLearn configuration is nothing more than a gateway to the application container. –  When clusters were more relevant, the web server acted as a pseudo load-balancer. •  Not many opportunities for optimization other than –  KeepAlives –  Interfaces –  Compression •  It can become a bottleneck if not properly optimized –  Better to have high ceilings from an interface perspective
  12. 12. Optimizing the JVM •  Java hotspot offers standard –X and non-standard –XX options for performance and behavior. –  -X options are always guaranteed across releases and patches of Java. –  -XX options must be used with caution as they are subject to change with any release of Java. •  -XX options should be tested and measured using the production safe arguments. –  Be careful of leveraging options you see on ListServes and Google •  Read the release notes of Java for “performance” updates –  http://java.sun.com/javase/6/webnotes/ReleaseNotes.html
  13. 13. Regions of the JVM
  14. 14. Optimizing the JVM •  Cross-platform recommendation for using Concurrent Mark Sweep Collector –  Best optimized for 64-bit address –  Combine –XX:+UseConcMarkSweepGCwith –XX:+UseParNewGC •  Manually size New Space using –XX:NewSize and – XX:MaxNewSize options (1/4 to 1/3 total heap). –  Consider Survivor Space ratios 4 or lower. •  Be careful about sharing –XX non-standard options across customers. –  If you don’t understand what the option does and it’s not recommended by Blackboard, best choice is to not use it. •  PermSpace and B2s –  Every B2 you add, make sure to measure PermSpace usage via tools like JStat and VerboseGC
  15. 15. High Level Old Space Collection
  16. 16. Prototyping JVM Arguments •  Wrapper.conf exists on both UNIX and Windows configurations for handling JVM arguments. •  Options can be tested out without a PushConfigUpdates by simply adding an additional line with –  wrapper.java.additional.<insert numeric sequence>= –  Simply issue a “./ ServiceController services.appserver.restart” to restart the JVM without arguments
  17. 17. PrintGCStats
  18. 18. Tools for Java Instrumentation •  GC Log Analysis Tools: PrintGCStats, Jstat, GCViewer, •  Thread Analysis Tools: Samurai, JConsole, JavaVisualVM, •  Memory Analysis Tools: Eclipse Memory Analyzer, Jhat, •  Robust Commercial Tools: Quest Foglight, Oracle OEM, Dynatrace for Java, CA IntroScope •  Profiling Tools: Jprobe, YourKit, JProfiler
  19. 19. Optimizing the Database: SQL Server •  # of data files makes no difference on SQL Server for Data and Transaction •  Allow the data/transaction files to grow as big as they want within reason. –  What’s reason: 64GB –  http://msdn.microsoft.com/en-us/library/ms143432(sql.90).aspx •  TempDB is completely different story –  # of files = # of DB Threads –  Set first X files to a uniform size, set last file to same size with auto-extension ON –  Determine size need over time •  Separate volume for paging file
  20. 20. Optimizing the Database: SQL Server •  Be aware of MDOP: Max Degree of Parallelism –  Setting to unlimited can have a negative affect on query performance unintentionally. •  AWE can and does work on 64-bit systems •  Configure READ_COMMITTED_SNAPSHOT •  Two nuggets of information: –  Learn How to Use SQL DMVs –  Study SQL Server Wait Events and Tuning
  21. 21. Optimizing the Database: Oracle •  Balance I/Os across multiple data files (~2 to 8GB per file). •  REDO is critical to performance a session/query level. –  Be aware of how much REDO is being used over time. –  NOLOGGING will disable, be we rarely use NOLOGGING •  TEMP is very complex and used for managing transient data. –  One TEMP file is adequate –  If latency exists on TEMP, consider introducing TEMP file groups •  SGA is important, but PGA can be your best friend or your worst enemy with high concurrency.
  22. 22. Optimizing the Database: Oracle •  Oracle CBO can be your friend –  Must understand optimizer behavior –  Willingness to read Cost Execution Plans •  Using Wait Events and Cost Execution Plans for tuning initiatives –  Wait events are at a system, session and query level •  Importance of Statistics and Histograms –  CBO is just guessing without properly set statistics and histograms. –  CBO is dependent on your data.
  23. 23. Approach to Understanding DB Performance •  Capture statements using some pre-identified tool –  Oracle AWR and ASH (Session Level) –  SQL Server Performance Dashboard and Performance Warehouse •  Review Instance Level Information to gain a cursory view of “system” performance. •  Deep dive into SQL Analysis –  Focus on Logical I/Os as starting point. •  Physical I/Os are important, but more often logical –  Drill into CPU consumers followed by executions –  Isolate physical and structural for over usage and possible locking conditions –  Question why executions, but 0 rows processed –  Importance of both: Histograms and Statistics •  Know your data better than your DBAs
  24. 24. Please provide feedback for this session by emailing BbWorldFeedback@blackboard.com. The subject of the email should be title of this session: Best Practices for Optimizing Blackboard Learn