Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What's new and what's next in Rudder

105 views

Published on

Rudder 4.1 was released in March 2017 with:

- an advanced feature to query external APIs and pull in node properties dynamically
the ability to add "key=value" tags to all Rules and Directives in order to categorize them
- a new API on relay servers to enable node-to-node file sharing and remote run in firewalled environments performance improvements
- a new plugin package format

Rudder 4.2 was released in September 2017 and includes the support for a new plugin that adds support for a new Windows DSC-based agent. Rudder 4.3 will include:

- Parameters for Technique Editor techniques
- ACLs on the API accounts
- Many architecture improvements

In parallel, new plugins are being developed:
- A plugin to integrate data from external APIs
- Monitoring integration with Centreon
- CMDB integration with iTop
- A reporting plugin for historized compliance

This talk will introduce these new features and show how to use them, hopefully getting you as excited as we are! Then, we will move on to explain about longer-term feature ideas we have for Rudder, and the general vision linked to future developments.

About Nicolas Charles

Nicolas is a tinkerer who likes when things just work, and tries his best to reach this goal. He started as a developer 15 years ago, and often had to reach out of this role to solve issues.

In 2010, he co-founded Normation, and he still enjoys fixing things in Rudder and at its users.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

What's new and what's next in Rudder

  1. 1. What’s new and what’s next in Rudder Nicolas CHARLES Co-founder and COO @nico_charles
  2. 2. 2 Agenda Rudder news since last camp What’s new in Rudder 4.1, 4.2 and 4.3 Future direction
  3. 3. 3 Rudder versions Currently supported versions ● 4.1.x – current ESR* version ● 4.2.x – current latest version – Will be supported for 3 months after next version is released (so until May 2018) ● 4.3.x – next version – Scheduled for release in February 2018 * ESR = Extended Support Release 2015 2016 2017 2018 Jun Jan Oct Mar Sep Feb 3.1 3.2 4.0 4.1 4.2 4.3
  4. 4. 4 Microsoft Powershell DSC Agent 4.2 Manage Windows Systems using Powershell DSC Native agent for Windows – Uses Microsoft Powershell DSC – Requires Powershell 4 or more
  5. 5. 5 Microsoft Powershell DSC Agent 4.2 Generic methods for DSC, Classic or both Agents
  6. 6. 6 New generation method: Directive by Directive ● Directives are not merged together anymore! ● New Policy Generation type: SEPARATED ● Mix audit and enforce mode for same Technique on a Node ● Several versions at the same time on a node 4.3
  7. 7. 7 New generation method: Directive by Directive ● Directives are not merged together anymore! ● Most techniques have been rewritten to support this feature – New version (but you can upgrade them one directive at a time) <POLICYGENERATION>separated</POLICYGENERATION> ● Generate one folder per directive – Path: TechniqueName/TechniqueVersion_DirectiveID 4.3
  8. 8. 8 New generation method: Directive by Directive ● Technically, RudderUniqueID placeholder ● In Technique bundle names/function names ● In resulting class to avoid collisions ● Hooks: One time action before and after Directives – For global actions – like getting the repositories PGP keys only once 4.3
  9. 9. 9 Techniques Parameters ● Defines parameters within the Technique Editor ● Better re-usability 4.3
  10. 10. 10 Techniques Parameters ● Defines parameters within the Technique Editor ● Define parameter during Directive creation 4.3
  11. 11. 11 Node properties ● Node properties can now be JSON values: datacenter = { "id": "FRA1", "name": "Colo 1, Paris", "location": "Paris, France", "dns_suffix": "paris.example.com" } ● Access properties in any directive field: ${node.properties[datacenter]} ${node.properties[datacenter][id]} ● Use default values: ${node.properties[datacenter][id] | default = "UK2" } ${node.properties[netbios_name] | default = ${rudder.node.hostname} } ${node.properties[datacenter][name] | default = """value with "quotes" if necessary""" } 4.1
  12. 12. 12 Node properties ● Import automatically properties on nodes from third-party REST application ● Datasource plugin ● Drive behaviour from external source ● Specific type of Node Property 4.1
  13. 13. 13 Node properties ● Import automatically properties on nodes from third-party REST application 4.1
  14. 14. 14 Node properties ● Import automatically properties on nodes from third-party REST application ● Can be global or on a node by node basis ● Add data in headers 4.1
  15. 15. 15 Node properties ● Import automatically properties on nodes from third-party REST application ● Extract from received JSON relevant information 4.1
  16. 16. 16 Node properties ● Import automatically properties on nodes from third-party REST application ● Customize update frequency 4.1
  17. 17. 17 Node properties ● Import automatically properties on nodes from third-party REST application ● Define what happens when the API doesn’t answer 4.1
  18. 18. 18 Node properties Agent searches for optional properties files /var/rudder/local/properties.d/*.json Add new properties or override existing properties defined on Rudder Example: Results in : "sysctls_postgresql":{"kernel.shmmax":"5368709120"} 4.1 On the node side "sysctls_postgresql": { "kernel.shmmax":"5368709120" } On the server side "sysctls_postgresql": { "kernel.shmall":"903330", "kernel.shmmax":"3700041320" } Override node properties locally
  19. 19. 19 JSON everywhere ncf methods variable_dict variable_dict_ from_fle Import JSON at runtime 4.1
  20. 20. 20 JSON everywhere 4.1 Merge JSON at runtime
  21. 21. 21 Tags everywhere! 4.1 Tags on Directives and Rules to classify and filter
  22. 22. 22 A new API on relay servers Central server Node Node Node TCP communication (port 5309) File metadata File contents Authentifcation + encryption (TLS) TCP/UDP communication (ports 443 and 514) Protocols: HTTPS, syslog Node Node Node Isolated network zone Relay server Inventory + Reports Confguration policy
  23. 23. 23 ... RELAY API A new API on relay servers Relay server Node Node UI REST API ... RELAY API Central server RELAY APIRELAY API Trigger agent runScenario 1: Trigger agent runs remotely, including via firewalls. 4.1
  24. 24. 24 ... RELAY API A new API on relay servers Relay server Node 1 Node 2 UI REST API ... RELAY API Central server RELAY APIRELAY API File shared with metadata Scenario 2: Share files from one node to another. In the same network or not (via relays). ... RELAY API Relay server sharedfle_to_node(‘node 2’, ‘db.sql’, ‘/var/share/db.sql’, ‘3 days’) sharedfle_from_node(‘node 1’, ‘db.sql’, ‘/var/share/db.sql’) ncf methods 4.1
  25. 25. 25 Hooks 4.1 Customize behaviour on the server
  26. 26. 26 Hooks 4.3 Extend inventory agent side ● Inventory runs hooks in /var/rudder/hooks.d or C:Program FilesRudderhooks.d ● Executable scripts, owned by current user or root, and not world writable ● Script must return valid JSON ● Added in inventory tag CUSTOM_PROPERTIES ● Available in Node Properties on the Rudder Server ● Can be used to create Groups ● Available in API
  27. 27. 27 Improved performance ● Improved UI performance ● New graph rendering library ● All Web resources are cached ● Compress all data from Web Interface ● Better Agent performance ● 40% faster in normal usage, up to 20 times faster with large policies ● Slightly faster policy generation 4.1
  28. 28. 28 Agent ● Lighter agent ● Perl is no longer packaged within the Agent ● SystemD support ● Timing on the CLI output ● Dropped the old cfengine network protocol 4.3
  29. 29. 29 Miscellaneous ● Groups of groups ● Node lifecycle ● Renaming of ncf generic method ● And a tool to automatically update the generic method call ● Same versioning for Rudder & ncf 4.3
  30. 30. 30 Plugins ● Branding: Customize Rudder UI ● Backgrounds and font colours ● Login page ● Logos ● Title text
  31. 31. 31 Plugins ● Precise ACLs on API ● Rights per token on any REST API endpoint ● Token expiration date ● Maps user permissions to tokens ● What can we define? ● AclPath : segments, separated by / – Segment is either a String (api, nodes, rules, etc) – Wildcard * , anywhere as a segment – Double wildcard ** , only at the end, matches anything ● HttpAction (GET, POST, PUT, DELETE) ● Anything that is not authorized is denied 4.3
  32. 32. 32 Plugins ● Examples ● ALLOW api/nodes/** , GET – Permits to read all in the nodes API – But no changes at all ● ALLOW api/nodes, GET – Permits to list nodes (including searches), but not the pending nodes ● ALLOW api/directives/7dd68892-6820-4f85-8e44-a7cc820dd06e , POST – Edit only directive with id 7dd68892-6820-4f85-8e44-a7cc820dd06e ● ALLOW api/directives/*/check, POST – Only permits to valid that a change is valid 4.3
  33. 33. 33 Plugins ● Centreon: Automatically configure monitoring on systems Node UI REST API ncf RELAY API Central server RELAY APIRELAY API 1 - Synchronize all nodes in Centreon Plugin 2 - Configure node 3 - Configure hosttemplate
  34. 34. 34 Plugins ● iTop: CMDB integration ● Export inventories to iTop ● Import properties from iTop – Drive policies from CMDB and external data ● Export Directives and Compliance – Measure impact of non-compliance
  35. 35. 35 Plugins ● Reporting ● Generate compliance reports over a period of time – In development progress
  36. 36. 36 Plugins ● Reporting ● Defines Rules/Groups/Nodes and a period ● Select what to display
  37. 37. 37 Bug classification
  38. 38. 38 Bug classification – 3 parameters
  39. 39. 39 Bug classification – 3 parameters ● User visibility: use case impacted by issue ● First impression – even before Rudder installation ● Getting started – during demo, first install or basic usage of simple Techniques ● Operational – usage of Technique Editor, advanced Techniques, Rudder settings ● Infrequent – complex configurations, third-party integration
  40. 40. 40 Bug classification – 3 parameters
  41. 41. 41 Bug classification – 3 parameters ● Severity: ● Critical – Prevent main usage of Rudder, can cause data loss – no workaround ● Major– Prevent usage of a part of Rudder – no easy workaround ● Minor – Something is misleading or with an easy workaround ● Trivial – No functional impact, but it would be nicer if it were fixed.
  42. 42. 42 Bug classification – 3 parameters
  43. 43. 43 Bug classification – 3 parameters ● Effort required: ● Small – This issue can be solved in less than a day ● Medium – It can be fixed in a reasonable amount of time ● Large – This issue is complex, needs some thoughts and time (about a week) ● Very large – This issue is so complex that we cannot estimate its duration (several weeks to months)
  44. 44. 44 Bug classification - Priority ● These information are reviewed, and a priority is computed from these ● From 0 (lower priority) to about 150 (the top priority) ● Weighted based on user visibility and severity ● Biased toward smallest effort and oldest bugs
  45. 45. 45 http://faq.rudder-project.org New FAQ
  46. 46. 46 Agenda The future
  47. 47. 47 Client – Server communication ● Two steps policy update ● Validation on the client side ● Ensure complete consistency ● Much faster policy generation
  48. 48. 48 Client – Server communication ● Drop syslog protocol ● Send reports via HTTPS ● Minimize impact of agent on nodes ● Improve performances and network usage
  49. 49. 49 Web Interface ● Customize Dashboard ● Customize columns in tables ● Improve Group page ● Improve search interface and group creation ● Manage Users within the UI
  50. 50. 50 Future plugins (planned + ideas) Sync data between Rudder servers Ideas of plugins Advanced access control (OrBAC) High availability for Rudder server “Ramp up” policies for progressive rollouts
  51. 51. 51 Rudder Ambassador Program ● Rudder Ambassador ● Program for exceptional Rudder contributors ● To be announced
  52. 52. 52 Rudder.io ● New Domain name ● Rudder.io ● Focus on the Rudder brand
  53. 53. What’s new and what’s next in Rudder Questions ? Nicolas CHARLES Co-founder and COO @nico_charles

×