All About
Storeconfigs



       Brice Figureau
    masterzen (#puppet)
Who am I?
Puppet Community Developer
planetpuppet.org maintainer
Whatʼs Storeconfigs?
Storeconfigs stores to an RDBMS
uses Railsʼ Active Record
2 Possible Usages

Inventory      Export/Collect
Export/Collect
Node collaborations
Syntax: export

       @@resource {
         ...
       }
Syntax: collect


    Resource <<| query |>>
Collect: Query
•Resource Parameters
•Tags
•and, or
Tip


      Always tag
      exported
      resources
The BIG Picture

          master     collector
   DB                    2



                     collector
             ...
Simple example
Web server replication, deployment server
Simple example
Web server replication, web server part
Intermediate example
Building /etc/resolv.conf from exported snippets
Intermediate example
Building /etc/resolv.conf, aggregating snippets
Advanced Example
Tinydns master
Advanced Example
Tinydns master
Advanced Example
Tinydns slave
Limitations
Inner Workings
Inner Workings
1. Read all resources, tags for a node (slow)
Inner Workings
2. Compare with in-memory catalog (fast)
Inner Workings
3. Store/remove missing/extraneous
resources/tags (slow)
Inner Workings
DB schema
Performance Impact
Performance Impact
Collecting is fast
Performance Impact
Storeconfigs write can be slow
Performance Impact
1st run (1950 resources)
                  Off           On
         25


         19


         13


 ...
Performance Impact
Subsequent runs (1950 resources)
                Off          On
         8


         6


         4

...
First Run Impact
First Run Impact
Failure


err: Could not retrieve catalog: execution expired
Scaling: some rough math
1 HDD = 250 IOP/s
Scaling: some rough math
All nodes have 500 resources
4 params/ resources
4 tags/resources
=> 4500 writes
=> 1000 IOP
Scaling: some rough math
Master concurrency: 5
Scaling: some rough math
IOP/s left per node:

         250/5 = 50 IOP/s
Scaling: some rough math
Time per node

       1000 IOP = 20s
Scaling: some rough math
And for a 500 nodes setup...

        500/5 = 100 “runs”
Scaling: some rough math
With a sleep time of 30min

         30 min = 1800s
         1800s/20s = 90

      90 nodes per 3...
Scaling: some rough math
Houston we have a problem

 90 nodes when we needed 100
Scaling the back-end
Dedicated RDBMS server(s)
Scaling the back-end
Powerful I/O subsystem (RAID,BBU...)
Scaling the back-end
DB setup tuning
Scaling the back-end
MySQL, scaling for inserts
innodb_buffer_pool_size = 70% of physical RAM
Scaling the back-end
MySQL, scaling for inserts
innodb_log_file_size = up to 5% of physical RAM
Scaling the back-end
MySQL, scaling for inserts
innodb_flush_method = O_DIRECT
Scaling the back-end
MySQL, scaling for inserts
innodb_flush_log_at_trx_commit = 2
0.25 to the help!
0.25 to the help!
Async Storeconfigs - no more thundering herd
0.25: Async Storeconfigs

        puppetqd
   DB              Message Queue   master




                                  ...
0.25: Thin Storeconfigs
Only exported resources, tags and facts
0.25: Thin Storeconfigs
1st run (1950 resources)
                      Off              On
               6


             ...
Storeconfigs future?
More AR adapters
More AR tuning (pooling)
Non RDBMS back-ends
Non RDBMS back-ends
•Tokyo Cabinet/Tyrant
•Mongo DB
Questions ?
           Please speak loud and slowly and excuse me
      by advance if I donʼt understand a word of your qu...
All About Storeconfigs
All About Storeconfigs
Upcoming SlideShare
Loading in …5
×

All About Storeconfigs

12,652
-1

Published on

Storeconfigs is not a popular feature among Puppet admins, because most don’t know how to use it or fear performance issues. Attend this talk to know how to enhance your Puppet deployments with easy cross-nodes interactions and collaborations, while conserving system efficiency.

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,652
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
39
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

All About Storeconfigs

  1. All About Storeconfigs Brice Figureau masterzen (#puppet)
  2. Who am I?
  3. Puppet Community Developer planetpuppet.org maintainer
  4. Whatʼs Storeconfigs?
  5. Storeconfigs stores to an RDBMS uses Railsʼ Active Record
  6. 2 Possible Usages Inventory Export/Collect
  7. Export/Collect Node collaborations
  8. Syntax: export @@resource { ... }
  9. Syntax: collect Resource <<| query |>>
  10. Collect: Query •Resource Parameters •Tags •and, or
  11. Tip Always tag exported resources
  12. The BIG Picture master collector DB 2 collector 1 Exporter
  13. Simple example Web server replication, deployment server
  14. Simple example Web server replication, web server part
  15. Intermediate example Building /etc/resolv.conf from exported snippets
  16. Intermediate example Building /etc/resolv.conf, aggregating snippets
  17. Advanced Example Tinydns master
  18. Advanced Example Tinydns master
  19. Advanced Example Tinydns slave
  20. Limitations
  21. Inner Workings
  22. Inner Workings 1. Read all resources, tags for a node (slow)
  23. Inner Workings 2. Compare with in-memory catalog (fast)
  24. Inner Workings 3. Store/remove missing/extraneous resources/tags (slow)
  25. Inner Workings DB schema
  26. Performance Impact
  27. Performance Impact Collecting is fast
  28. Performance Impact Storeconfigs write can be slow
  29. Performance Impact 1st run (1950 resources) Off On 25 19 13 6 0 Time (seconds)
  30. Performance Impact Subsequent runs (1950 resources) Off On 8 6 4 2 0 Time (seconds)
  31. First Run Impact
  32. First Run Impact Failure err: Could not retrieve catalog: execution expired
  33. Scaling: some rough math 1 HDD = 250 IOP/s
  34. Scaling: some rough math All nodes have 500 resources 4 params/ resources 4 tags/resources => 4500 writes => 1000 IOP
  35. Scaling: some rough math Master concurrency: 5
  36. Scaling: some rough math IOP/s left per node: 250/5 = 50 IOP/s
  37. Scaling: some rough math Time per node 1000 IOP = 20s
  38. Scaling: some rough math And for a 500 nodes setup... 500/5 = 100 “runs”
  39. Scaling: some rough math With a sleep time of 30min 30 min = 1800s 1800s/20s = 90 90 nodes per 30 min
  40. Scaling: some rough math Houston we have a problem 90 nodes when we needed 100
  41. Scaling the back-end Dedicated RDBMS server(s)
  42. Scaling the back-end Powerful I/O subsystem (RAID,BBU...)
  43. Scaling the back-end DB setup tuning
  44. Scaling the back-end MySQL, scaling for inserts innodb_buffer_pool_size = 70% of physical RAM
  45. Scaling the back-end MySQL, scaling for inserts innodb_log_file_size = up to 5% of physical RAM
  46. Scaling the back-end MySQL, scaling for inserts innodb_flush_method = O_DIRECT
  47. Scaling the back-end MySQL, scaling for inserts innodb_flush_log_at_trx_commit = 2
  48. 0.25 to the help!
  49. 0.25 to the help! Async Storeconfigs - no more thundering herd
  50. 0.25: Async Storeconfigs puppetqd DB Message Queue master node
  51. 0.25: Thin Storeconfigs Only exported resources, tags and facts
  52. 0.25: Thin Storeconfigs 1st run (1950 resources) Off On 6 4 3 1 0 Time (seconds)
  53. Storeconfigs future?
  54. More AR adapters
  55. More AR tuning (pooling)
  56. Non RDBMS back-ends
  57. Non RDBMS back-ends •Tokyo Cabinet/Tyrant •Mongo DB
  58. Questions ? Please speak loud and slowly and excuse me by advance if I donʼt understand a word of your question. Being French has some advantages (like health care access or a cute if not stupid accent) it also has drawbacks, especially our educational system for learning foreign languages simply sucks :-D

×