Storeconfigs is not a popular feature among Puppet admins, because most don’t know how to use it or fear performance issues. Attend this talk to know how to enhance your Puppet deployments with easy cross-nodes interactions and collaborations, while conserving system efficiency.

1. All About
Storeconfigs
Brice Figureau
masterzen (#puppet)

2. Who am I?

4. Puppet Community Developer
planetpuppet.org maintainer

6. Whatʼs Storeconfigs?

7. Storeconfigs stores to an RDBMS
uses Railsʼ Active Record

8. 2 Possible Usages
Inventory Export/Collect

9. Export/Collect
Node collaborations

10. Syntax: export
@@resource {
...
}

11. Syntax: collect
Resource <<| query |>>

12. Collect: Query
•Resource Parameters
•Tags
•and, or

13. Tip
Always tag
exported
resources

14. The BIG Picture
master collector
DB 2
collector
1
Exporter

15. Simple example
Web server replication, deployment server

16. Simple example
Web server replication, web server part

17. Intermediate example
Building /etc/resolv.conf from exported snippets

18. Intermediate example
Building /etc/resolv.conf, aggregating snippets

19. Advanced Example
Tinydns master

20. Advanced Example
Tinydns master

21. Advanced Example
Tinydns slave

22. Limitations

23. Inner Workings

24. Inner Workings
1. Read all resources, tags for a node (slow)

25. Inner Workings
2. Compare with in-memory catalog (fast)

26. Inner Workings
3. Store/remove missing/extraneous
resources/tags (slow)

27. Inner Workings
DB schema

28. Performance Impact

29. Performance Impact
Collecting is fast

30. Performance Impact
Storeconfigs write can be slow

31. Performance Impact
1st run (1950 resources)
Off On
25
19
13
6
0
Time (seconds)

32. Performance Impact
Subsequent runs (1950 resources)
Off On
8
6
4
2
0
Time (seconds)

33. First Run Impact

34. First Run Impact
Failure
err: Could not retrieve catalog: execution expired

35. Scaling: some rough math
1 HDD = 250 IOP/s

36. Scaling: some rough math
All nodes have 500 resources
4 params/ resources
4 tags/resources
=> 4500 writes
=> 1000 IOP

37. Scaling: some rough math
Master concurrency: 5

38. Scaling: some rough math
IOP/s left per node:
250/5 = 50 IOP/s

39. Scaling: some rough math
Time per node
1000 IOP = 20s

40. Scaling: some rough math
And for a 500 nodes setup...
500/5 = 100 “runs”

41. Scaling: some rough math
With a sleep time of 30min
30 min = 1800s
1800s/20s = 90
90 nodes per 30 min

42. Scaling: some rough math
Houston we have a problem
90 nodes when we needed 100

43. Scaling the back-end
Dedicated RDBMS server(s)

44. Scaling the back-end
Powerful I/O subsystem (RAID,BBU...)

45. Scaling the back-end
DB setup tuning

46. Scaling the back-end
MySQL, scaling for inserts
innodb_buffer_pool_size = 70% of physical RAM

47. Scaling the back-end
MySQL, scaling for inserts
innodb_log_file_size = up to 5% of physical RAM

48. Scaling the back-end
MySQL, scaling for inserts
innodb_flush_method = O_DIRECT

49. Scaling the back-end
MySQL, scaling for inserts
innodb_flush_log_at_trx_commit = 2

50. 0.25 to the help!

51. 0.25 to the help!
Async Storeconfigs - no more thundering herd

52. 0.25: Async Storeconfigs
puppetqd
DB Message Queue master
node

53. 0.25: Thin Storeconfigs
Only exported resources, tags and facts

54. 0.25: Thin Storeconfigs
1st run (1950 resources)
Off On
6
4
3
1
0
Time (seconds)

55. Storeconfigs future?

56. More AR adapters

57. More AR tuning (pooling)

58. Non RDBMS back-ends

59. Non RDBMS back-ends
•Tokyo Cabinet/Tyrant
•Mongo DB

60. Questions ?
Please speak loud and slowly and excuse me
by advance if I donʼt understand a word of your question.
Being French has some advantages (like health care access or a cute if not stupid accent)
it also has drawbacks, especially our educational system for
learning foreign languages simply sucks :-D