What makes a LDAP server
running fast ?
Emmanuel Lécharny
Apache Software Foundation member
Chairman of MINA project
PMC of Apache Directory Project
IKTEK Owner (...
Latency numbers
every programmer should know !
(https://gist.github.com/hellerbarde/2843375)

Main memory reference .........
A request...
Search : From client to client
ASN/1
ASN/1 codec
FROM :
connection.bind( "uid=akarasulu,dc=example,dc=com", "password" );

TO :
0x30, 0x33,
0x02, 0x01, 0x01,
0...
Searching
Search, search, search

It's all about

Search
performance !
Check, please !
Search Request Checks
Checks done before the first entry is returned :
- Normalize the filter
- Check if the password shou...
The candidates
Selecting candidates

Candidates are references
to
entries
(in other words, they are just pointers...)
Selecting candidates

AND

OR

NOT

No index

∀

∀

∀

Index

∩

∪

∀

Remember : We don't actually fetch any entry !
Candidates & AND filter
Maximum = min(filters candidates)
Minimum = min(filters candidates)

Here, max = 72 and min = 72
Candidates & OR filter
Maximum = sum(filters candidates)
Minimum = sum(filters candidates)

Here, max = 8098 and min = 809...
Cost of creating candidates

Looking for the best index
+
Creating the set of references
=
20% of the search processing
Index

No index,
no

Gain
Filters

Build your search

Filters
with caution
The Cache
Cache

It's all about
Memory
Vs

Disk
latency
Reminder

Disk is from 4x to 80x slower !
Read 1 MB sequentially from memory ..... 250,000 ns

= 250 µs

Read 1 MB sequent...
Cache, the good

No Disk access
=>
Fast (very!)
Entry Cache

It caches Objects
Hash map
Or
Ordered data structure
Cache, the bad

Locks...
Algorithms...
Memory...
Cache, the ugly

L Has to be 'warm'
L Immutable objects
=>
A kind of copy is needed
45% of the search processing time
DISK
Backend

Storage !
Backend

Remember :
memory vs disk latency...
Memory
Of Price and Men

Memory : 64 GB = 1000$
vs
1 day of consulting to 'tune'
your servers for little gain = ???
Machines
(Olivia) Newton (John) Theory

Let's get physical,

physical
VM vs Bare metal

http://www.altechnative.net/2012/08/04/virtual-performance-part-1-vmware/
VM vs Bare metal

From 16h to 16 mins...
Most certainly IOs and/or disk access
(Spinning disks on a SAN)
Disk

Own your Disks !
Don't share them...

SSD is a winner !
SSD 1TB = 600$
HD 1TB = 100$
Network

Own your network !
Don't share it...
Network
Network
It's 4x to 40x times slower than memory :
Read 1 MB sequentially from memory ..... 250,000 ns

= 250 µs

Read 1 MB...
Network

Get a fast network !
Misceallenous
Authorization

AUTHz
Is
Not
Free.
Thanks!
Upcoming SlideShare
Loading in...5
×

What makes a LDAP server running fast ? An bit of insight about the various bottlenecks and solutions to avoid them

1,014

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,014
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

What makes a LDAP server running fast ? An bit of insight about the various bottlenecks and solutions to avoid them

  1. 1. What makes a LDAP server running fast ?
  2. 2. Emmanuel Lécharny Apache Software Foundation member Chairman of MINA project PMC of Apache Directory Project IKTEK Owner (www.iktek.com) www.iktek@com, elecharny@iktek.com
  3. 3. Latency numbers every programmer should know ! (https://gist.github.com/hellerbarde/2843375) Main memory reference ...................... 100 ns Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs Read 1 MB sequentially from SSD* ..... 1,000,000 ns = 1 ms Send 1 MB over 1 Gbps network ....... 10,000,000 ns = 10 ms Disk seek ........................... 10,000,000 ns = 10 ms Read 1 MB sequentially from disk .... 20,000,000 ns = 20 ms 3
  4. 4. A request...
  5. 5. Search : From client to client
  6. 6. ASN/1
  7. 7. ASN/1 codec FROM : connection.bind( "uid=akarasulu,dc=example,dc=com", "password" ); TO : 0x30, 0x33, 0x02, 0x01, 0x01, 0x60, 0x2E, // LDAPMessage ::=SEQUENCE { // messageID MessageID // CHOICE { ..., bindRequest BindRequest, ... // BindRequest ::= APPLICATION[0] SEQUENCE { 0x02, 0x01, 0x03, // version INTEGER (1..127), 0x04, 0x1F, // name LDAPDN, 'u', 'i', 'd', '=', 'a', 'k', 'a', 'r', 'a', 's', 'u', 'l', 'u', ',', 'd', 'c', '=', 'e', 'x', 'a', 'm', 'p', 'l', 'e', ',', 'd', 'c', '=', 'c', 'o', 'm', ( byte ) 0x80, 0x08, // authentication AuthenticationChoice // AuthenticationChoice ::= CHOICE { simple [0] OCTET STRING, // ... 'p', 'a', 's', 's', 'w', 'o', 'r', 'd' 7
  8. 8. Searching
  9. 9. Search, search, search It's all about Search performance !
  10. 10. Check, please !
  11. 11. Search Request Checks Checks done before the first entry is returned : - Normalize the filter - Check if the password should be reset - Check if the user is authenticated - Check the filter attributes - Find the backend This represents 9% of the initial search processing (for a search returning one entry). 11
  12. 12. The candidates
  13. 13. Selecting candidates Candidates are references to entries (in other words, they are just pointers...)
  14. 14. Selecting candidates AND OR NOT No index ∀ ∀ ∀ Index ∩ ∪ ∀ Remember : We don't actually fetch any entry !
  15. 15. Candidates & AND filter Maximum = min(filters candidates) Minimum = min(filters candidates) Here, max = 72 and min = 72
  16. 16. Candidates & OR filter Maximum = sum(filters candidates) Minimum = sum(filters candidates) Here, max = 8098 and min = 8098
  17. 17. Cost of creating candidates Looking for the best index + Creating the set of references = 20% of the search processing
  18. 18. Index No index, no Gain
  19. 19. Filters Build your search Filters with caution
  20. 20. The Cache
  21. 21. Cache It's all about Memory Vs Disk latency
  22. 22. Reminder Disk is from 4x to 80x slower ! Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs Read 1 MB sequentially from SSD* ..... 1,000,000 ns = 1 ms Read 1 MB sequentially from disk .... 20,000,000 ns = 20 ms
  23. 23. Cache, the good No Disk access => Fast (very!)
  24. 24. Entry Cache It caches Objects Hash map Or Ordered data structure
  25. 25. Cache, the bad Locks... Algorithms... Memory...
  26. 26. Cache, the ugly L Has to be 'warm' L Immutable objects => A kind of copy is needed 45% of the search processing time
  27. 27. DISK
  28. 28. Backend Storage !
  29. 29. Backend Remember : memory vs disk latency...
  30. 30. Memory
  31. 31. Of Price and Men Memory : 64 GB = 1000$ vs 1 day of consulting to 'tune' your servers for little gain = ???
  32. 32. Machines
  33. 33. (Olivia) Newton (John) Theory Let's get physical, physical
  34. 34. VM vs Bare metal http://www.altechnative.net/2012/08/04/virtual-performance-part-1-vmware/
  35. 35. VM vs Bare metal From 16h to 16 mins... Most certainly IOs and/or disk access (Spinning disks on a SAN)
  36. 36. Disk Own your Disks ! Don't share them... SSD is a winner ! SSD 1TB = 600$ HD 1TB = 100$
  37. 37. Network Own your network ! Don't share it...
  38. 38. Network
  39. 39. Network It's 4x to 40x times slower than memory : Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs Read 1 MB sequentially from SSD* ..... 1,000,000 ns = 1 ms Send 1 MB over 1 Gbps network ....... 10,000,000 ns = 10 ms But still : you can send up to 100 000 1kb entries per second Through 1Gbps network...
  40. 40. Network Get a fast network !
  41. 41. Misceallenous
  42. 42. Authorization AUTHz Is Not Free.
  43. 43. Thanks!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×