Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amis conference soa deployment. the dirty tricks using bamboo, nexus and xl deploy


Published on

Build and deployment of SOA Composites for enterprise organizations can get complex. While building you have to take into account the architecture, development guidelines, other projects, platform release and component lifecycles. Next to this you have to work with the differences in deployment procedures for several versions of Oracle Fusion Middleware. In this session we will showcase some of the best practices and dirty tricks to create an effective and future proof build and deployment process for Oracle Middleware. We will showcase practical demos in for SOA components in Bamboo, Nexus and XLDeploy for a large scale enterprise application landscape.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Amis conference soa deployment. the dirty tricks using bamboo, nexus and xl deploy

  1. 1. OGh Oracle Fusion Middleware Experience 2016 bij FIGI Zeist Maarten Smeets, András Hetényi and Robbrecht van Amerongen, June-2016
  2. 2. Deployment Automation
  3. 3. Who are we? • Maarten Smeets – Senior Integration Consultant – Experience with Oracle SOA Suite since 2007 – Well certified (SOA, BPM, Java, SQL, PL/SQL) – Author of more than 100 blog articles • Robbrecht van Amerongen – Manager Continuous Delivery – Initiator of the Red Expert Alliance – Agile Master +31641010286 • András Hetényi – Continuous Delivery and Integration Consultant – Senior Oracle (SOA) developer
  4. 4. 4
  5. 5. 5
  6. 6. Enterprise SOA deployment means
  7. 7. How does it feel
  8. 8. People who break stuff “developers” “ops team” People who deal with the consequences Talk about DevOps
  9. 9. People who break stuff less and deal with the consequences “Devops team”
  10. 10. Non – technical tricks • Start small Show it in real software. Hijack a server and run a build engine • Show the build dashboard on a TV screen So you get questions from management and gain acceptance • Bribe your server admin Getthe admin on board and buy sweets, “real” coffee / tea
  11. 11. Non – technical tricks • Tool is fit for purpose Use specific tools for specific solution • Include legacy Work with legacy, not around it. Show it is working for all sorts of products • Make the project the owner of the tools first. Next find an organizational owner for Build en Deployment tools. Take care of lifecycle management of the tools • Communicate, communicate, communicate Project newsletter. Updates on the corporate intranet. Show it is there.
  12. 12. Acceptance The Continuous Delivery Pipeline For SOA Scrum team System 2 Scrum team System 1 Scrum team System n SVC Automated Provisioning omgeving Build Domain Specific Language Continuous Integration QA Automated deployments / testing Develop OS and Middleware DEV Test OS and Middleware Integration OS and Middleware ACCEPT OS and Middleware Production OS and Middleware PROD ArtifactRepository WebLogic Scripting Tool (WLST)
  13. 13. What to build / deploy • An artifact – Is deployed to a single environment – E.g. SOA, Service Bus, configuration, ADF – Configuration is not part of a service! • A functional unit (deployable unit) – Is the unit of version control – Always deployed as a single entity – Is a collection of artifacts – Is the input for the deployment process • A release – Is a collection of functional units – Has a specific version – Is deployed to acceptance and production – Is the complete set of software Artifact Functional Unit Release WebLogicScriptingTool(WLST)
  14. 14. Why Atlassian Bamboo? • Experience / implementations at different customers • The tool and vendor are regarded as best of breed-class • It is very strict by design, which is useful for sticking to the pipeline proces • From Atlassian, easily linked with Jira and Confluence • It is cheap AND has full support from Atlassian Tip: Scripts to bridge the gap between build server and CD platform should be CD tool independent!
  15. 15. Why Atlassian Bamboo?
  16. 16. Bamboo How does it look
  17. 17. Bamboo setup • Why no Bamboo deployment project? – Requires specifying a branch. Bamboo branches are used to specify a functional unit. Would require a deployment project per functional unit – A Bamboo release is a snapshot of a number of Bamboo artifacts Bamboo is not used as artifact store What would be the artifact? Which branch?
  18. 18. What to use for build / deploy scripts • WLST (essential part of WebLogic server) – Flexible, can be extended with Python modules and Java libs – Often used by deployment tools / plugins – By default uses Jython 2.2.1 (2007) • Ant (10g, 11g, still there in 12c) – Flexible – Poor dependency management can be fixed with Apache Ivy – Can become complicated • Maven (12c and later) – Not meant for scripting – Requires plugins WebLogic Scripting Tool (WLST)
  19. 19. Challenges building with Oracle and Maven / ANT • Scripts from hell – That one person who knows how it works – Tight coupling between scripts and deployment tooling (migration will be painful) • The incremental database and Maven – Cultural issues: it has to be done with SQLPlus – Technical issues: I want to be able to rebuild my database from scratch to create a new environment. I want to combine the increments of several sprints • Maven only for 12c+ – Maven for all artifact types? Rerunnable code (within sprintrelease)! Store database code per sprintrelease. Store the incremental set of sprintreleases to recreate an environment. Never do manual changes! KISS. Harder than you think! Standardize. Modular. Reduce the number of different products and component types. Share responsibility Keep all your code in the artifact repository. A release is just a list of artifacts
  20. 20. Artifact repository A single source of truth • Immutable – Once an artifact with a specific version is in the artifact repository, it does not change. A specific version is always the same code. • Traceability – Artifact to version control to build tooling traceability You can use the version for that. e.g. 1.0.[revision].[build number] • Build once deploy many – Apply the ‘Build once, deploy many’ pattern. Compilation does not always yield the same result • Proxy dependencies: Oracle Public Maven repository – Proxy the Oracle public Maven repository • Tip: Environments change. Do not store deployment plans / configuration plans with hardcoded endpoints!
  21. 21. Artifact repository Structure Functional unit Artifacts Versions Oracle Maven Repository is proxied
  22. 22. SOA testing • Unit tests – Tests the isolated unit. Dependencies are mocked – Data services (e.g. database) are not mocked • Functional tests – Tests the service including interactions with dependencies – No mocks • Tip: Tests should be independent of data. Test first prepares data, executes test, removes data Mocks
  23. 23. Deployment SOA Loose coupling • Dependencies create complexity – Dependency loops – Installation ordering in build code required – Can cause server start issues Avoid direct service artifact dependencies! • Use MDS for shared artifacts • Do not use SOA-Direct Service A Service B Cyclic dependency
  24. 24. SOA code quality
  25. 25. SOA code quality • SonarQube ojaudit plugin available, but no SOA audit rule-set (requires Java coding) • Code Compliancy Inspector (CCI) from AIA • Payload validation should be turned on in development / test! Do we have a set of SOA development standards we can agree on?
  26. 26. Software Delivery Process
  27. 27. OPS DEV Legacy Deployment Process Delivery DEV WLST SQL*Plus Unix shell TST INT FAT UAT PRD Sources Production releases about per quarter vs. bi-weekly sprint deliveries from the project teams
  28. 28. OPS DEV Gradual Automation DEV TST INT FAT UAT PRD WLST SQL*Plus Unix shell Production releases are every two weeks following the sprint deliveries from the project teams Sources
  29. 29. High-level Architecture Core
  30. 30. User Interfaces • Command Line Interface (CLI): – Jython application that the user can access remotely. • GUI – Browser client requiring Flash-plugin – Configure and perform deployments – View the release pipeline of the different applications – View and edit the repository – View reports – View or edit security settings (based on authorization)• REST API – Basic authorization – Can be accessed by URL http[s]://<xld_host>:<xld_port>/deployit/<Service> – All public services are exposed: • E.g.: Control, Server, User, Repository, Package, Deployment… – Example: • curl -i -X GET -H "Authorization:Basic YWRtaW46YWRtaW4xMjM=" ‘http://localhost:4516/deployit/repository/ci/Infrastructure/Lo cal/LocalWL’
  31. 31. Infrastructure Deployment definition Application Environment ReferencesPackage Deployable Deployable Deployable Deployable Deployable Deployable What Where How Middelware Plan Deployed Deployed Host Deployed Deployed Deployed DB Deployed Dictionaries Entire release. No increment!
  32. 32. Packaging • Creating a deployment package: – With the GUI: • Directly by creating items under an application version • Can be exported to DAR – With the CLI: • Through Jython objects representing the REST API services – With the XLDeploy Plugin for DAR Art1v1.0 Art3v1.2 Art4v1.1 manifest APP/Version deployables specification1 + URI1 specification2 specification3 + URI3 specification4 + URI4 Art1v1.0 Art3v1.2 Art4v1.1 POM Project/Version deployables specification1 + URI1 specification2 specification3 + URI3 specification4 + URI4
  33. 33. CI-s, types and properties Deployable: soa.CompositeSpec Deployed: soa.Composite Container: wls.Container wls.Container: wls.Domain wls.Cluster wls.Server
  34. 34. Refined Targeting Artifacts Container
  35. 35. Dictionary and Placeholders
  36. 36. – Specification: • Which deployables to which containers • How to configure them – Delta analysis: • Specification vs. current state of middleware • Results in Delta specification: – CREATE: First time deployment – MODIFY: Upgrading the item – DESTROY: Undeploy the item – NOOP: No change – Orchestration: • Delta spec into sub-specs, executed in isolation • Resuls in Deployment plan with sub-plans – Planning: • Adding Steps to each sub-plan needed to execute the deployment • Example: database restore point – Execute: • Execution of the complete deployment plan XL Deploy at Work
  37. 37. Mapping and Plan
  38. 38. Migration Scenarios • Adaptation to XL Deploy – Extend with DAR generation – Re-define the artifacts – Covered by the XL Deploy plug-ins – Significant effort from developers – XebiaLabs support applies • Adaptation to Legacy – Extend with DAR generation – Keep the artifacts except for placeholders – Development in XL Deploy – Minimal effort from developers – Not supported with the product, must be re-validated for each update
  39. 39. Generic SOA-OSB Services MDS BPM OSB Shared G.O. Data sources DB Adapter config JMS (CF,SRV,MDL,Q,T) config plan customization config plan config plan customization SCA SOA-OSB Artifacts SCA MDS BPM OSB Shared G.O. config plan config plan customization Manifest DS DBA CF DBA DS DS DS DBA DBA Q T Adaptation of Legacy to XL Deploy Artifact migration
  40. 40. Adaptation of XLDeploy to Legacy (Example of generic database scripts) Deployable dbconfig |– init.sql |– insert_refdata.sql |– insert_userdata.sql |– create_indexes.sql |– create_sequences.sql |– create_tables.sql |– grants | |– privileges.sql | |– roles.sql |– logging | |– loginit.sql | |– log_functions.sql |– patch.sql |– setup.sql |– update_packages.sql Deployable dbconfig |– 00-setup | |– insert_refdata.sql | |– insert_userdata.sql |– 00-setup.sql |– 01-patch | |– create_indexes.sql | |– create_sequences.sql | |– create_tables.sql | |– privileges.sql | |– roles.sql | |– update_packages.sql |– 01-patch.sql |– common | |– loginit.sql | |– log_functions.sql Wrapper Plugin custom.DbScript property: listOfScripts setup.sql patch.sql Deployable dbconfig |– init.sql |– insert_refdata.sql |– insert_userdata.sql |– create_indexes.sql |– create_sequences.sql |– create_tables.sql |– grants | |– privileges.sql | |– roles.sql |– logging | |– loginit.sql | |– log_functions.sql | |– patch.sql |– setup.sql |– update_packages.sql Plugin Plugin v.s.
  41. 41. Filling the Gaps: Rollback CREATE File Folder Ear War Package DESTROY File Folder Ear War DDL Package CREATE SOA BPM Package MODIFY 1.1-1.2 MODIFY 1.2-1.1 DESTROY Default
  42. 42. Orchestrating application components by custom rules. Example: Interrupt + Rules Filling the Gaps: Dependency Execute PRE Deploy SOA-OSB Execute POST SOA-OSB-Config Scripts PRE POST PAUSE SOA-OSB Services
  43. 43. 44
  44. 44. 45