Codecamp2015 pimp yourpipeline-saade-jens-1.1
•
0 likes•296 views
Central configuration management allows teams to manage configurations in a centralized repository. Configurations can be versioned, inherited in a hierarchy, and consumed by applications through a client that pulls configurations from the repository. This enables consistent configuration of environments like development, testing and production while allowing flexibility for individual configuration overrides.
Recommended
More Related Content
What's hot
What's hot (17)
Viewers also liked
Viewers also liked (10)
Similar to Codecamp2015 pimp yourpipeline-saade-jens-1.1
Similar to Codecamp2015 pimp yourpipeline-saade-jens-1.1 (20)
More from Codecamp Romania
More from Codecamp Romania (20)
Recently uploaded
Recently uploaded (20)
Codecamp2015 pimp yourpipeline-saade-jens-1.1
- 1. PimpYour Pipeline Central Configuration Management Jens Saade, youngculture April 25th 2015, Iasi CodeCamp Romania
- 3. ReferencesAbout youngculture Group Jens Saade Head Innovation & Technology E-Commerce Engineering Background (Retail, PIM, E-Commerce) Located in Zürich, CH Dev Process Architecture Innova2on Ini2a2ves
- 4. Central Configuration Repository Appliance in Continuous Delivery Automation & Configuration Intro youngculture E-Commerce
- 6. youngculture ECOM team delivers highly integrated E-Commerce Solutions based on the Hybris Platform.
- 7. Customized B2C Storefront Showcase
- 8. B2C & B2B Shops and PIM Systems
- 9. Delivering good quality in a highly complex project environment is only possible by aligning team, processes and infrastructure environments.
- 10. Con2nuous Delivery Configura2on Mgmt Test Automa2on Cloud Services Automated Provisioning Workspace (Confluence, JIRA) Prac2ce Meetups Integrated Communica2on (HipChat) System Monitoring HW/OS (NewRelic, Nagios) Applica2on Monitoring Logs (Elas2cSearch/ Kibana, Splunk) Performance (Dynatrace), Delivery Environment Environments CAM Collaborate AutomateMonitor TeamsSystems Projects
- 11. Automation
- 12. Continuous Delivery Familiar with the Concepts? Tried applying it in own Company? Part of your Culture?
- 13. AUTOMATION is hard work ... expensive to establish... and can get pretty complex BUT ... reduces monkey work ... speeds up the dev & delivery processes ... calms down stress level ...and raises confidence and quality.
- 14. Release anytime, reproducible Create Hybris Projects, Extension and Addons Setup Continously Build deployable artifacts Build Assess Software by Automated Testing Test Continuous Delivery Overview Install for testing, acceptance and production Deploy > Commit Stage > Acceptance Stage > Deployment Stage Feedback Feedback Feedback Establishing a Delivery Pipeline is a joint team effort! It involves lots of disciplines and evolves constantly.
- 15. CustomersDevelopers Product Owner Scrum Master Delivery Process: Automation is Key Test Execution Env Jenkins CI Artifactory Repository Git Repository Hybris Node YC Deploy Client Apache HTTPdSelenium Gatling JMeter Test Exec Hybris Test Commit Stage Acceptance Stage Deploy Stage Continuous Delivery Monitoring Node Stage/Production Hybris Node1 Hybris Cluster Apache HTTPd Varnish Apache HTTPd Varnish DB Cluster Load Balancer ElasticSeach Logstash Kibana Zabbix External Monitoring NewRelic Hybris Node 2 YC Deploy Client YC Deploy Client Hosting Environment TeamsInfrastructures Maintenance team
- 16. One essential problem left: Configurations
- 17. Ecosystem of Configurations Versioned Software Artifact Dev 1 Dev 2 Ops 1 Ops 2 QA Test specifc artifact versions for regressionQA on features Create new Conf Property Share his conf with Dev 2 Prepare Rollout Conf for tonight Enable feature for customer Req Eng Introduce new concept, define triggers A=1 B=true c=debug
- 18. 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties Node 1 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties Node 2 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties Node 3 1 Clustered Environment Dev(s) QA Systems Production UAT/Stage ... Hybris Configuration Stack (Properties based)
- 21. Diversity: Support complex environment setups != We need good concepts how to configure environments nearly equivalent but allow flexibility for individual deviations
- 22. „The most efficient way to manage configuration is to have a central service through which every application can get the configuration it needs.“ Jez Humble and Dave Farley „Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation“, Addison-Wesley 2010
- 23. Artifact Generic Configuration Machine Java VM Rolling out Configurations Configurations get packaged and rolled out JVMs are bootstrapped from packaged configurations Machine Java VM Machine Java VM Machine Java VM Environment Configuration
- 24. Turning from distributing (and manually tweaking) local configurations ... ... to a centralized environment that hosts configurations for self-service. Direction change: Push -> Pull
- 25. Artifact Machine Java VM Rolling out Configurations, alternative Rollout Software Artifact as usual, but keep it generic, just provide configuration IDs Isolate configurations from the deployment package and provide it centrally from the CCR Machine Java VM Machine Java VM Machine Java VM UUID Generic Configuration CCR Environment Configuration UUID UUID UUID UUID UUID Generic Configuration Environment Configuration
- 26. Hierachical Key-Value Store Configuration Configuration UUID3 UUID4 Machine Application Client Wrapper Central Configuration Repository (CCR) and Consuming Clients Agent pulls configuration properties via REST / Json Then injects properties to JVM System context MachineMachineMachine Machine Machine Machine Machine MachineMachineMachineMachine Machine Machine Machine Machine Machine REST / Json discoBit A P I Admin UI CCR
- 27. CD & CCR Applied @YC
- 28. Configuration Hierarchies Automated space UUID1 Shared space UUID2 UAT Environment UUID7 UAT node 1 UUID8 UAT Node 2 UUID9 QA: Nightly Build Test System UUID6 Personalized Conf Developer A UUID3 Personalized Conf Developer B UUID5 Ops 1 Ops 2 QA Dev 1 Dev 2 Req Eng Continuous Delivery Cycle
- 29. Customized (env1) UUID3 Customized (env2) UUID4 Machine Java VM discoBit Java Agent (UUID4) System Context Injection via Java Agent discoBit Agent pulls configuration properties via REST / Json Then injects properties to JVM System context Automated space UUID1 Shared space UUID2
- 33. jsaade@MacBook-Pro:~/hybris/hybris-commerce-suite-5.4.0.0/hybris/bin/platform$ ./hybrisserver.sh Launching a JVM... Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.bootstrap.DiscobitSettings <init> INFORMATION: [discobit] configuring server from system: http://127.0.0.1:8089 Apr 20, 2015 1:42:33 PM com.youngculture.ecom.hybris.discobit.HybrisDiscobitAgent execute INFORMATION: Reading discobit configuration from http://127.0.0.1:8089::bb2835fb-466c-4143-8081-ffcd60c49467 Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.bootstrap.DiscobitSettings <init> INFORMATION: [discobit] configuring server from system: http://127.0.0.1:8089 Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.bootstrap.DiscobitSettings <init> INFORMATION: [discobit] configuring server from system: http://127.0.0.1:8089 Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.DiscobitConnector authenticate INFORMATION: [discobit] authentication response: HTTP/1.1 200 OK Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.DiscobitConnector authenticate INFORMATION: [discobit] authentication cookie: [[version: 0][name: JSESSIONID][value: 65452FC130FE2F87E8655BF05D5F0A90][domain: 127.0.0.1][path: /][expiry: null]] Apr 20, 2015 1:42:34 PM com.youngculture.ecom.hybris.discobit.HybrisDiscobitAgent execute INFORMATION: injected 677 properties to system context WrapperManager: Initializing... Apr 20, 2015 1:42:39 PM org.apache.tomcat.util.digester.SetPropertiesRule begin WARNUNG: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'disableURLRewriting' to 'true' did not find a matching property. Apr 20, 2015 1:42:40 PM org.apache.tomcat.util.digester.SetPropertiesRule begin WARNUNG: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'disableURLRewriting' to 'true' did not find a matching property. Apr 20, 2015 1:42:40 PM org.apache.coyote.AbstractProtocol init INFORMATION: Initializing ProtocolHandler ["http-bio-9001"] Apr 20, 2015 1:42:40 PM org.apache.coyote.AbstractProtocol init INFORMATION: Initializing ProtocolHandler ["http-bio-9002"] Apr 20, 2015 1:42:45 PM org.apache.coyote.AbstractProtocol init INFORMATION: Initializing ProtocolHandler ["ajp-bio-8009"] Apr 20, 2015 1:42:45 PM org.apache.catalina.startup.Catalina load INFORMATION: Initialization processed in 5834 ms Apr 20, 2015 1:42:45 PM org.apache.catalina.mbeans.JmxRemoteLifecycleListener createServer INFORMATION: The JMX Remote Listener has configured the registry on port 9003 and the server on port 9004 for the Platform server Apr 20, 2015 1:42:45 PM org.apache.catalina.core.StandardService startInternal INFORMATION: Starting service Catalina
- 34. Key Take Aways
- 35. Key Take Aways • Simple, unified approach to avoid wide-spread configuration and their mutation • Language & product agnostic • Enabling other teams and colleagues to take controlled influence on configs in general • Needs a clear strategy concerning delivery cycle, security and infrastructure
- 36. Usecases • Configure applications in hosting enviroments, test farms, desktops • Developers share configurations while creating a solution • QA configures their testing systems individually • Dark launching of software changes • Feature toggling for indidual usage profiles
- 37. Pros • Central administration • Create multi-level configuration inheritance • Turn from a push-based model to a consumer pull-based self service • Allow fine grained access to (parts) of configurations for different stakeholder roles • Change properties without rollouts (trigger reload events instead) + + + + +
- 38. Cons (yes there are) • Accessibilty from environment to CCR needs to be resilient, fallbacks needed (e.g. local caching). • Configuring during runtime can have unforseen side effects in your application (check lifecycle). • Administrative access to configurations needs to be well thought. -‐ -‐ -‐
- 39. Where to go next? Potential Extensions • Tracking changes possible (who did what and when) • Add versioning by storing configs in an SCM for rollbacks ore regression testing • Stage changes for different rollout scenarios • Add hierarchical config structures like YAML
- 40. Questions?
- 41. Thanks! Mail: j.saade@youngculture.com Twitter: @jenssaade youngculture.com Hybris.com discoBit.com
- 42. Appendix
- 43. Continuous Delivery Hosting QA Testing Jenkins CI Embedding into Continuous Delivery Flow discoBit Application Instance 2 UUID 2 Application Instance 1 UUID1 Application Instance n UUID n Config Repository REST API Artifactory Distributable Artifact Acceptance Stage Deploy Stage deploybuild configure UI 1 2 3 Commit Stage env.properties project.properties advanced.properties
- 44. Continuous Delivery QA Testing Source (git) Jenkins Stacked Configurations in the Central Config Repo Commit Stage env.properties discoBit Application Instance 1 UUID3 Acceptance Stage Deploy Stage deploybuild configure Automated space UUID1 Shared space <extends> UUID2 UUID1 <extends> Customized (env2) Customized (env1) UUID3 UUID4 Devs’ Machine Application Instance 3 UUID4 Application Instance 2 UUID3 1 2 project.properties advanced.properties merge