TF-LSD TIO performanse test setup

337 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
337
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Desire LDAP/TIO-index performance tabel 4 clients 8 clients 16 clients 32 clients 64 clients Filter-1 650 654 659 661 607 Filter-2 157 160 0 0 0 Filter-3 466 470 473 451 402 Filter-4 84 0 0 0 0 Filter-5 425 429 431 404 351 Filter-6 84 0 0 0 0 Filter-7 454 458 462 439 386 Filter-8 141 144 0 0 0 Filters: cn=henny cn=*henny* (&(cn=henny)(sn=bekker)) (&(cn=*henny*)(sn=*bekker*)) (|(cn=henny)(sn=bekker)) (|(cn=*henny*)(sn=*bekker*)) (&(!(cn=henny))(sn=bekker)) (&(!(cn=*henny*))(sn=*bekker*))
  • Lims LDAP/TIO-index performance table Filters: cn=henny cn=*henny* (&(cn=henny)(sn=bekker)) (&(cn=*henny*)(sn=*bekker*)) (|(cn=henny)(sn=bekker)) (|(cn=*henny*)(sn=*bekker*)) (&(!(cn=henny))(sn=bekker)) (&(!(cn=*henny*))(sn=*bekker*)) (&(!(cn=*Test*))(sn=*bekker*))
  • Filter: sn=bekker cn=*henny* (&(cn=henny)(sn=bekker)) (&(cn=*henny*)(sn=*bekker*)) (|(cn=henny)(sn=bekker)) (|(cn=*henny*)(sn=*bekker*)) (&(!(cn=henny))(sn=bekker)) (&(!(cn=*henny*))(sn=*bekker*)) (&(!(cn=*Test*))(sn=*bekker*)) (&(cn=*henny*)(|(sn=*bekker*)(sn=*becker*)))
  • Lims LDAP/TIO-index performance table Filters: sn=bekker cn=*henny* (&(cn=henny)(sn=bekker)) (&(cn=*henny*)(sn=*bekker*)) (|(cn=henny)(sn=bekker)) (|(cn=*henny*)(sn=*bekker*)) (&(!(cn=henny))(sn=bekker)) (&(!(cn=*henny*))(sn=*bekker*)) (&(!(cn=*Test*))(sn=*bekker*)) (&(cn=*henny*)(|(sn=*bekker*)(sn=*becker*))) (&(|(sn=*bekker*)(sn=*becker*))(cn=*henny*))
  •   4 clients 8 clients 16 clients 32 clients 64 clients   62895 87428 93632 93479 93433 Filter-1 61395 85551 92269 91523 93514 Filter-2 64842 89297 93349 93711 93566 Filter-3 37159 37255 37134 37347 37414 Filter-4 56439 77458 92602 92811 91950 Filter-5 35539 35624 35680 35768 35848 Filter-6 62112 86283 93722 93622 93841 Filter-7 35863 35988 35987 36060 36143 Filter-8 26308 25875 24214 25700 25900
  • TF-LSD TIO performanse test setup

    1. 2. LDAP TIO INDEX server performance test General setup for the LDAP TIO Index-server performance test Henny Bekker @ SURFnet
    2. 3. The LDAP-Index performance test.. <ul><li>The setup of the test lab </li></ul><ul><li>What software is used in the test </li></ul><ul><li>The general setup for the performance test </li></ul><ul><ul><li>The benchmark setup </li></ul></ul><ul><ul><li>The dataset, objectClasses and attributes </li></ul></ul><ul><ul><li>What tokenization </li></ul></ul><ul><li>What is tested and measured </li></ul><ul><ul><li>The filter setting </li></ul></ul><ul><ul><li>Latency and number of requests per second </li></ul></ul>
    3. 4. The setup of the test lab <ul><li>The TIO index client and server systems </li></ul><ul><ul><li>One dedicated server running out of the box RedHat v7.1 </li></ul></ul><ul><ul><ul><li>400-MHz Intel Pentium II </li></ul></ul></ul><ul><ul><ul><li>512 Mbyte RAM </li></ul></ul></ul><ul><ul><ul><li>100 Mbps/Full-Duplex Ethernet interface </li></ul></ul></ul><ul><ul><ul><li>Maxtor 10-Gbyte IDE disk </li></ul></ul></ul><ul><ul><li>4 dedicated client systems running RedHat v7.1 </li></ul></ul><ul><ul><ul><li>266-MHz Intel Pentium II (or better) </li></ul></ul></ul><ul><ul><ul><li>256 Mbyte RAM </li></ul></ul></ul><ul><ul><ul><li>100 Mbps/Full-Duplex Ethernet interface </li></ul></ul></ul><ul><ul><ul><li>Maxtor 4-Gbyte IDE disk </li></ul></ul></ul><ul><ul><li>Consoles connected using a ‘starview’ console switch </li></ul></ul><ul><li>The network setup </li></ul><ul><ul><li>A dedicated VLAN on a Cisco Catalyst 5000 switch </li></ul></ul><ul><ul><li>100-Mbps Fast Ethernet Full-Duplex </li></ul></ul>
    4. 5. What software is used in the test <ul><li>LDAP/TIO index servers </li></ul><ul><ul><li>LIMS V 1.01 (Catalogix) </li></ul></ul><ul><ul><li>Desire/LDAP-TIO index v1.0 (Desire/DAASI) </li></ul></ul><ul><ul><ul><li>MySQL v3.23.36 </li></ul></ul></ul><ul><ul><ul><li>OpenLDAP v1.2.10 </li></ul></ul></ul><ul><ul><ul><li>Apache v1.3.2 </li></ul></ul></ul><ul><ul><li>IDDS v4.5 </li></ul></ul><ul><li>TIO converter software </li></ul><ul><ul><li>Tags v1.0 </li></ul></ul><ul><ul><li>Ldif2tio & tiocollapse </li></ul></ul><ul><li>LDAP test software </li></ul><ul><ul><li>Ldapgun v0.25 </li></ul></ul><ul><ul><li>Ldapsearch v2.0.7-14 </li></ul></ul>
    5. 6. The general setup for the test <ul><li>The benchmark setup </li></ul><ul><ul><li>Plan the test (estimate load requirements) </li></ul></ul><ul><ul><ul><li>The index(es) must fit in main memory </li></ul></ul></ul><ul><ul><li>Designate PC’s for server and clients </li></ul></ul><ul><ul><li>Verify network performance </li></ul></ul><ul><ul><ul><li>tcpblast and iperf </li></ul></ul></ul><ul><ul><li>Execute verification test </li></ul></ul><ul><ul><ul><li>Using IDDS v4.5 </li></ul></ul></ul><ul><ul><ul><li>Ldapgun and ldapsearch </li></ul></ul></ul><ul><ul><ul><li>Use designated dataset </li></ul></ul></ul><ul><ul><li>Loading TIOs into index servers </li></ul></ul><ul><ul><li>Run ldap clients for 10 minutes to ‘warm up’ the index and system disk cache </li></ul></ul>
    6. 7. The general setup for the test (cont.) <ul><li>The dataset to be used </li></ul><ul><ul><li>A test-set of approximately 450-K entries </li></ul></ul><ul><ul><ul><li>SURFnet ±170-K entries </li></ul></ul></ul><ul><ul><ul><li>DFN ±120-K entries </li></ul></ul></ul><ul><ul><ul><li>Needs ±150-K entries (NO, SE) </li></ul></ul></ul><ul><ul><li>Used tokenization </li></ul></ul><ul><ul><ul><li>DNS: cn, o, ou, l, c, gn </li></ul></ul></ul><ul><ul><ul><li>RFC822: mail </li></ul></ul></ul><ul><ul><ul><li>Full: sn, objectclass </li></ul></ul></ul><ul><ul><li>Only objectClass=person entries </li></ul></ul><ul><li>Using anonymous bind </li></ul><ul><li>LDAPv3 subtree searches without chasing of referrals </li></ul>
    7. 8. The general setup for the test (cont.) <ul><li>Exemplary search requests on the FL-DSA & CAB </li></ul><ul><ul><li>The Dutch FL-DSA (1500K requests) </li></ul></ul><ul><ul><ul><li>1300K baseObject search on objectClass </li></ul></ul></ul><ul><ul><ul><li>60K onelevel search on objectClass </li></ul></ul></ul><ul><ul><ul><li>20K baseObject search on cn </li></ul></ul></ul><ul><ul><ul><li>10K onelevel search on ou </li></ul></ul></ul><ul><ul><ul><li>8K onelevel search on cn </li></ul></ul></ul><ul><ul><ul><li>5K onelevel search on o~ </li></ul></ul></ul><ul><ul><ul><li>4K onelevel search on o or ou </li></ul></ul></ul><ul><ul><ul><li>4K subtree search on uid </li></ul></ul></ul><ul><ul><ul><li>4K subtree search on cn </li></ul></ul></ul><ul><ul><ul><li>3K subtree search on cn~ or sn~ </li></ul></ul></ul><ul><ul><ul><li>3K subtree search on ou </li></ul></ul></ul><ul><ul><ul><li>3K subtree search on objectClass and cn </li></ul></ul></ul><ul><ul><ul><li>2K subtree search on cn or sn or uid </li></ul></ul></ul><ul><ul><ul><li>2K subtree search on sn </li></ul></ul></ul><ul><ul><ul><li>72K + 60 other different searches </li></ul></ul></ul>
    8. 9. The general setup for the test (cont.) <ul><ul><li>The end user LDAP server (750K requests) </li></ul></ul><ul><ul><ul><li>640K baseObject search on objectClass </li></ul></ul></ul><ul><ul><ul><li>53K onelevel search on objectClass </li></ul></ul></ul><ul><ul><ul><li>20K onelevel search on o </li></ul></ul></ul><ul><ul><ul><li>9K subtree search on cn or mail or sn </li></ul></ul></ul><ul><ul><ul><li>3K subtree search on cn or mail or sn or givenname </li></ul></ul></ul><ul><ul><ul><li>2K subtree search on cn </li></ul></ul></ul><ul><ul><ul><li>23K + 60 other different searches </li></ul></ul></ul><ul><ul><li>Very exotic filter settings for subtree searches </li></ul></ul><ul><ul><ul><li>(|(cn=kluck)(sn=kluck)(uid=kluck)(ou=kluck)) </li></ul></ul></ul><ul><ul><ul><li>(|(cn~=polak)(sn~=polak)(ou~=polak)) </li></ul></ul></ul><ul><ul><ul><li>(|(mail=nie*)(|(cn=nie*)(|(sn=nie*)(givenname=nie*)))) </li></ul></ul></ul>
    9. 10. What is tested and measured <ul><li>Defining the LDAP search filters </li></ul><ul><ul><li>From ‘simple’ to ‘extravagant’ (with lots of booleans). </li></ul></ul><ul><ul><li>Must be dealt with by both LDAP/TIO servers </li></ul></ul><ul><li>The filter settings: </li></ul><ul><ul><li>(cn=foo) </li></ul></ul><ul><ul><li>(cn=*foo*) </li></ul></ul><ul><ul><li>(&(cn=foo)(sn=bar)) </li></ul></ul><ul><ul><li>(&(cn=*foo*)(sn=*bar*)) </li></ul></ul><ul><ul><li>(|(cn=foo)(sn=bar)) </li></ul></ul><ul><ul><li>(|(cn=*foo*)(sn=*bar*)) </li></ul></ul><ul><ul><li>(&(!(cn=foo))(sn=bar)) </li></ul></ul><ul><ul><li>(&(!(cn=foo))(sn=*bar*)) </li></ul></ul><ul><ul><li>(&(!(cn=not_a_foo_name))(sn=*bar*)) </li></ul></ul><ul><ul><li>(&(cn=foo)(sn=*bar*)(mail=*foo*)(givenname=*bar*)) </li></ul></ul>
    10. 11. What is tested and measured (cont.) <ul><li>Failed subtree searches </li></ul><ul><ul><li>Lims: </li></ul></ul><ul><ul><ul><li>(&(!(cn~=*foo*))(sn~=*bar*)) </li></ul></ul></ul><ul><ul><ul><li>(&(!(cn=*foo*))(sn=*bar*)(mail=*bar*)) </li></ul></ul></ul><ul><ul><ul><li>(&(!(cn=*foo*))(&(|(cn=*bar*)(cn=*fiets*))(|(sn=*auto*)(sn=*trein*)))) </li></ul></ul></ul><ul><ul><ul><li>(|(cn~=foo)(sn~=bar)(ou~=bar)) </li></ul></ul></ul><ul><ul><li>Desire: </li></ul></ul><ul><ul><ul><li>(&(cn=*foo*)(|(sn=*bar*)(sn=*fiets*))) </li></ul></ul></ul><ul><ul><ul><li>(&(|(sn=*foo*)(sn=*bar*))(cn=*fiets*)) </li></ul></ul></ul><ul><ul><ul><li>(&(!(cn=*foo*))(&(|(cn=*bar*)(cn=*fiets*))(|(sn=*auto*)(sn=*trein*)))) </li></ul></ul></ul><ul><ul><ul><li>(&(!(cn~=*foo*))(sn~=*bar*)) </li></ul></ul></ul><ul><ul><li>No failures for IDDS </li></ul></ul>
    11. 12. What is tested and measured (cont.) <ul><li>Performance test results </li></ul><ul><ul><li>Number of requests per second for all 10 filters </li></ul></ul><ul><ul><ul><li>One single client (test run) </li></ul></ul></ul><ul><ul><ul><li>Four clients with multiple threads (max 128) </li></ul></ul></ul><ul><ul><ul><ul><li>Range: 4, 8 16, 32, 64, 128, 256 and 512 sessions </li></ul></ul></ul></ul><ul><ul><li>Latency tests </li></ul></ul><ul><ul><ul><li>Measured during test-run </li></ul></ul></ul><ul><ul><li>Hits and misses </li></ul></ul><ul><ul><ul><li>Every query must give one or more ‘hits’ ? </li></ul></ul></ul><ul><ul><ul><ul><li>Both implementations are indifferent to hits or misses </li></ul></ul></ul></ul><ul><ul><li>Measuring load on the index server </li></ul></ul><ul><ul><ul><li>Using ‘vmstat’ running at 10 sec. Interval </li></ul></ul></ul>
    12. 13. Desire LDAP/TIO-Index: Performance measurement
    13. 14. Desire: CPU load 4 to 64 simultaneous sessions
    14. 15. Lims LDAP/TIO Index: Performance measurement
    15. 16. Lims: CPU load 4 to 512 simultaneous sessions
    16. 17. IDDS: Performance measurement
    17. 18. IDDS: CPU load 4 to 512 simultaneous sessions
    18. 19. Lims: Heavy load performance measurement
    19. 20. Desire: CPU load 4 to 1024 simultaneous sessions
    20. 21. Latency tests <ul><li>Unloaded server with filter (&(cn=foo)(sn=bar)) </li></ul><ul><ul><li>Desire: 1000 queries in 630 seconds </li></ul></ul><ul><ul><li>Lims: 1000 queries in 8.95 seconds </li></ul></ul><ul><ul><li>IDDS: 1000 queries in 8,92 seconds </li></ul></ul><ul><li>Unloaded server with filter (&(cn=*foo*)(sn=*bar*)) </li></ul><ul><ul><li>Desire: 1000 queries in 3500 seconds </li></ul></ul><ul><ul><li>Lims: 1000 queries in 11,5 seconds </li></ul></ul><ul><ul><li>IDDS: 1000 queries in 11,0 seconds </li></ul></ul>
    21. 22. Latency tests (cont.) <ul><li>Heavy loaded server with filter (&(cn=foo)(sn=bar)) </li></ul><ul><ul><li>Desire: 1000 queries in 1976 seconds </li></ul></ul><ul><ul><li>Lims: 1000 queries in 17 seconds </li></ul></ul><ul><ul><li>IDDS: 1000 queries in 61 seconds </li></ul></ul><ul><li>Heavy loaded server with filter (&(cn=*foo*)(sn=*bar*)) </li></ul><ul><ul><li>Desire: 1000 queries in >7200 seconds </li></ul></ul><ul><ul><li>Lims: 1000 queries in 24 seconds </li></ul></ul><ul><ul><li>IDDS: 1000 queries in 69 seconds </li></ul></ul>
    22. 23. Latency tests (cont.) <ul><li>System status index server </li></ul><ul><ul><li>In all cases enough free memory without swapping </li></ul></ul><ul><ul><li>CPU for 100% occupied </li></ul></ul><ul><ul><ul><li>80% user-time; 20% system-time </li></ul></ul></ul><ul><ul><li>Lots of system interrupts and context-switches </li></ul></ul><ul><ul><ul><li>5000 interrupts per second </li></ul></ul></ul><ul><ul><ul><li>4000 context switches per second </li></ul></ul></ul><ul><ul><li>IDDS and Lims use threading </li></ul></ul><ul><ul><li>Desire does fork sub-processes </li></ul></ul>
    23. 24. Exchanging TIOs between Desire and Lims <ul><li>To be done … </li></ul>
    24. 25. Conclusion <ul><li>The study of exchanging TIOs between Desire and Lims need to be finalized. </li></ul><ul><li>The Lims LDAP/TIO-Index server performs better the the Desire LDAP/TIO-Index server with a factor of 100 </li></ul><ul><li>The Lims LDAP/TIO-Index server performs as good as Innosoft IDDS v4.51 </li></ul><ul><li>Complex LDAP filters with boolean operators or filtertypes like ‘approx’ are not supported by the Lims and Desire LDAP/TIO-Index server </li></ul>

    ×