OpenDS_Jazoon2010

0 views
1,024 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
0
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

OpenDS_Jazoon2010

  1. 1. Building a high performance directory server in Java Lessons learned and tips from the OpenDS project. Ludovic Poitou Matthew Smith Sun Microsystems #118
  2. 2. AGENDA > Introduction to the OpenDS project > Architecture, Design Patterns and Tips > Experiences with Sun JVM > Conclusion 2
  3. 3. AGENDA > Introduction to the OpenDS project > Architecture, Design Patterns and Tips > Experiences with Sun JVM > Conclusion 3
  4. 4. The OpenDS Project > Released in Open Source in July 2006 – CDDL – Source code at https://opends.dev.java.net/ > Sponsored by Sun Microsystems > Written in Java by LDAP experts
  5. 5. What is it ? > OpenDS is effectively a Java based Server supporting the LDAPv3 protocol and services – Objet Oriented, Hierarchical Data Model – CRUD operations > It comes with its own embedded database – Based on Berkeley DB Java Edition – Not accessible from outside > It has all security, access controls, password management features to safely store the information about Users
  6. 6. What is it for ? > Generic object oriented data store > White pages and Email Address Book > Mostly the data store for Identities – For Authentication and Authorization – For profiles and personalization > The underlying infrastructure in all Enterprises – Leveraged by Web and Mail infrastructure products – Cornerstone of Identity Management products:  Access Management and Federation  Provisioning and De-provisioning tools
  7. 7. Who for ? > Telecom service providers, financial institutions use LDAP directories for customer related services (Portal) – Storing customers identities, phones, services associated – Building highly available services for 10 Millions, up to 200 Millions users > But OpenDS can be used as OS naming service, or for SMB – OpenSolaris, Solaris, Linux... – Coupled with SAMBA, as a Domain controler – Integrated with Kerberos – White Pages... > And being 100% pure Java, OpenDS can be embedded in other Java applications or Web applications – OpenSSO 7
  8. 8. OpenDS 2.2 > Released in December 2009 > LDAPv3 directory server fully standard compliant – Supports many LDAP standard and experimental extensions – Supports Multi Master Replication with 3 different levels of data consistency – Extensive security features > Improved performances, reliability over OpenDS 1.0 > Installs in 6 clicks and less than 3 minutes > Several GUI and CLI to manage, monitor the OpenDS server > Extensive documentation > Localized in 6 different languages
  9. 9. Performance characteristics > As for most servers, scalability is extremely important – Up to hundreds million of entries – Up to thousands connections – Maximize use of CPUs > What is the operations throughput ? > What is the average response time ? The maximum response time ? > Our basic test – 10 M entries, with an average size of 2.6K – 2 servers, with Multi-Master replication between them
  10. 10. Searchrate on Sun x4170 box
  11. 11. Modrate on a Sun x4170 box
  12. 12. AGENDA > Introduction to the OpenDS project > Architecture, Design Patterns and Tips > Experiences with Sun JVM > Conclusion 12
  13. 13. How to reach those results ? > 2 main aspects – Architecture and code – Run-time : JVM and Garbage Collector optimization > There is a strong relationship between code design and memory optimization
  14. 14. Architecture Overview
  15. 15. Patterns > Use of Asynchronous I/O – Exception for write disk transactions > Use of Immutable Objects – Intrinsic thread safety – Avoid need for defensive copies > Use of “Factories” over Constructors – Avoid creating an object – Ease optimization for common cases  Example : Most AttributeDescriptions have 0 options  Example : Attributes generally have 1 value – For immutable objects. 15
  16. 16. Patterns > Producers / Consumers – Queues – Thread Pool – Monitors > Strategies – Queue Strategies : ConcurrentLinkedQueue vs LinkedBlockingQueue 16
  17. 17. Anti-Patterns > String concatenation – Make sure to use a StringBuilder – Compiler now optimize simple “Aaa” + “Bbb” concatenation > Avoid very long methods (thousands of lines of code) > Avoid exposing the concrete representation of an object – Set vs LinkedHashSet. – Not a performance issue, but will require more work when optimizing code for performance later > Try to define only the methods you need. 17
  18. 18. Java Collections > Vector and Hashtable are synchronized for all methods – Pay the price even if not necessary > Some Java collection classes are not synchronized by default – ArrayList, LinkedList replace Vector – HashSet, HashMap replace Hashtable > To synchronize, wrap in a class > With a “static factory” – Collections.synchronizedList(new ArrayList()) > ConcurrentHashMap, for concurrency – But watch when using the iterator 18
  19. 19. Critical Sections > Try to minimise the code and time spent in the critical sections > But the throughput is limited by the time spent in the largest critical section – Example : LinkedBlockingQueue  200 000 operations on x64 processor  20 000 operations on the T2000 processor – We use it for the WorkQueue and Access Logs
  20. 20. Caching data > Using Caches reduce the disk access thus should provide better performances > But cache eviction add pressure to the GC – When modifying entries – When the cache is too small to hold all data > A cache is also a contention point > If you want to cache objects, make sure you cache those that will be reused. > Alternate possibility, use “thread local” cache. – But watch out for the cost (with 1000 threads?)
  21. 21. Server monitoring > Getting statistics for a server is mandatory > Beware of contention – Stats are updated frequently – But seldom read > A strategy could be to keep per thread statistics and collect them on demand – Not yet implemented in OpenDS !
  22. 22. AGENDA > Introduction to the OpenDS project > Architecture, Design Patterns and Tips > Experiences with tuning Sun JVM > Conclusion 22
  23. 23. Performance Tuning > When dealing with performances, you should consider the whole system – Java VM – OS – Hardware : CPU, Memory, Disks, Network... > In our case, we try to avoid disk I/Os – And try to cache as much of the database > We also want deterministic response times – Avoid any Full GC (Stop The World) – Make sure minor GC pauses are as small as possible 23
  24. 24. JVM Tuning for OpenDS > Super Size The Heap ! – We use 32GB Heaps, sometime up to 96GB – 2GB for the New Generation (or ¼ of heap if < 8GB) – -Xms32768M -Xmx32768M -Xmn2048M > Use CMS – -XX:+UseConcMarkSweepGC – -XX:+UseParNewGC > -XX:MaxTenuringThreshold=1 – Avoid copy of objects in New Gen 24
  25. 25. Some interesting JVM Options > -XX:CMSInitiatingOccupancyFraction=70 – Define the amount of occupancy in Old Gen before starting to collect – Larger = better throughput but higher full GC risk > -XX:+UseCompressedOops – For 64bits JVM, less than 32BG of heap – Will be the default in coming Java 6 updates > If running on processor with NUMA architecture – -XX:+UseNUMA > -XX:+AggressiveOpts – Enables aggressive JIT optimizations, not related to GC 25
  26. 26. The Garbage First (G1) GC > Introduced in the Java HotSpot VM in JDK 7. > An experimental version of G1 has also been released since Java SE 6 Update 14. > G1 is the long-term replacement for HotSpot's low-latency Concurrent Mark- Sweep (CMS) GC > Should be officially supported with Java SE 6 Update 21 26
  27. 27. G1 Characteristics > Future CMS Replacement – Server “Style” Garbage Collector – Parallel, Concurrent – Generational – Good Throughput – Compacting – Improved ease-of-use – Predictable (though not hard real-time) > The main stages consist of remembered set (RS) maintenance, concurrent marking, and evacuation pauses. 27
  28. 28. JVM Options With G1 > -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC > PauseTime (Hints, Goal with no promise, otherwise use Java Real Time ) – -XX:MaxGCPauseMillis=50 (target of 50 milliseconds) – -XX:GCPauseIntervalMillis=1000 (target of 1000 msecs) > Generation Size – -XX:+G1YoungGenSize=512m (for a 512 MB young gen) > Parallelism – -XX:+G1ParallelRSetUpdatingEnabled – -XX:+G1ParallelRSetScanningEnabled 28
  29. 29. OpenDS et G1 > Goal: Avoid any Full GC, best control of pauses > Collaboration between the HotSpot and the OpenDS teams – OpenDS is used as a “Large” reference application – Between 10 and 20 enhancements integrated in G1 following the tests – Performance with large heaps improved by a factor of 10 > We're still discovering it – When doing read operations, we see pauses between 10 and 20 ms with 32GB JVM – But we're still seeing Full GC when doing Write operations (More garbage, stresses more the Old Gen) – Hopefully this will be resolved in next builds 29
  30. 30. OpenDS G1 and Searches 30
  31. 31. AGENDA > Introduction to the OpenDS project > Architecture, Design Patterns and Tips > Experiences with Sun JVM > Conclusion 31
  32. 32. Summary > OpenDS – A open source LDAP directory server, 100% pure Java – Easy to install and use – Designed for high performance and high scalability > We saw some patterns and tips used in the OpenDS project > Knowing and understanding the JVM and GC is required to build high performance server – Tuning JVM and GC is an art – Performance engineering is a profession > Who said that Java is slow ?! 32
  33. 33. The Art of GC Tuning JavaOne Presentation: GC Tuning in HotSpot JVM http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-4887&yr=2009&track=javase 33
  34. 34. Now... > Give OpenDS a try – http://www.opends.org > Join our community: – Join/Login sur Java.net – http://opends.dev.java.net – Request a Role – Subscribe to the mailing lists – IRC: #opends on freenode.net > OpenDS is localized in several languages. It's community based, through online tools. An easy way to participate. 34
  35. 35. Ludovic Poitou blogs.sun.com/Ludo Sun Microsystems Ludovic.Poitou@sun.com Matthew Swift Sun Microsystems Matthew.Swift@sun.com

×