Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building a CI/CD Pipeline for APIs


Published on

APIs play a key role in connecting applications and services to enable digital transformation of organizations. An organization can have multiple development and production environments with a WSO2 API Manager deployment and environment specific configurations. As the number of APIs used in an organization grows gradually, deploying and managing them across different environments become a tedious but critical task. A well-driven continuous integration and continuous deployment (CI/CD) pipeline makes this process easier by automating manually repetitive tasks.

This deck explores how WSO2 API Manager provides a clean and elegant solution to address automating API deployment via a platform-agnostic and version control-centric CI/CD process. Discussion topics include:
- Driving towards automation
- A dev-first approach for API deployment
- Technologies and tooling support
- Managing applications across environments
- Building CI/CD pipelines with WSO2 API Manager

Watch the webinar on-demand here:

Published in: Technology
  • Be the first to comment

Building a CI/CD Pipeline for APIs

  1. 1. Building a CI/CD Pipeline for APIs Harsha Kumara, Technical Lead, WSO2 Dushani Wellappili, Senior Software Engineer, WSO2
  2. 2. Discussion Topics • Driving towards automation • Automation challenges in API Management • Deciding CI/CD strategy • Technology and tooling support • Building simple CI/CD pipeline with WSO2 API Manager • Managing developer applications across environments
  3. 3. Importance of CI/CD in API Management
  4. 4. ● APIs are the de facto standard for connecting services, data, and apps ● APIs are the pumping fuel of digitally driven organizations ● Rapid phase of API development ● Smooth propagation of APIs between environments is critical ● Ensuring functionality upon updates across environments is important for the quality aspects
  5. 5. Why CI/CD in API Management?
  6. 6. ● Rapid development and deployment of APIs ● Less human interruptions ● Fast delivery to end users ● Time saving and efficiency gain ● Automated process provides greater management flexibility ● Detect issues earlier
  7. 7. Automation Challenges
  8. 8. ● Organizations are maintaining multiple deployment environments ● APIs associate with multiple policies and configurations ● Environment specific (endpoints) configurations ● Interference with multiple development teams
  9. 9. Which CI/CD Strategy Is Best for an Organization?
  10. 10. Building a CI/CD Strategy ● Organizations follows variant strategies ● First step of building a CI/CD process associates with ○ Organization culture ○ Governance structure and process ○ Team structure and dynamics
  11. 11. Selection of Tooling ● Technology and tooling plays a vital role in the CI/CD process implementation ● Selection of process automation tools ○ Jenkins ○ Travis CI ○ TeamCity ○ circleci ○ Drone ● Selection of source code repository ○ Github ○ Gitlab ○ Bitbucket
  12. 12. Support From API Management Vendor ● CI/CD approach of each API Management vendor is different ● API vendors offer ○ Tooling support ○ Support for interaction with source code repositories ○ Extensions to configure environment specific configurations ○ Plugins and extensions for pipeline tools ○ Standard REST APIs
  13. 13. Which Configs Are Changing Between Environments?
  14. 14. ● APIs contain environment specific configurations ● Part of configuration is changing between environments ● Environment specific configurations include ○ Backend endpoints ○ Credentials of backend services ○ Certificates of endpoints ○ Endpoint timeout settings ○ Gateway environments
  15. 15. CI/CD With WSO2 API Manager ● Selection of process orchestration tool such as Jenkins is a key requirement ● CI/CD process requires Source Code Management(SCM) system which acts as a Single Source of Truth for the pipeline ● apictl tool of API Manager provide the flexibility to integrate with these tools to migrate APIs between environments ● It is capable of handling environment specific configurations and can promote the API seamlessly to other environments via a single command
  16. 16. apictl
  17. 17. Features of apictl • Platform Agnostic • Inject environment related configurations • Command Line too • Easy configuration with many CI/CD tools
  18. 18. Simplistic View
  19. 19. CI/CD Process Overview APICTL
  20. 20. Configuring apictl
  21. 21. Preparing Environments • API Manager Command Line Tool should configure with deployment environments to import and export APIs Format: apictl add-env -e <environment-name> --registration <registration-endpoint> --apim <API-Manager-endpoint> --token <token-endpoint> --admin <admin-REST-API-endpoint> --api_list <API-listing-REST-API-endpoint> --app_list <application-listing-REST-API-endpoint> Sample: apictl add-env -n dev --registration https://localhost:9443/client-registration/v0.15/register --apim https://localhost:9443 --token https://localhost:8243/token --admin https://localhost:9443/api/am/admin/v0.15 --api_list https://localhost:9443/api/am/publisher/v0.15/apis --app_list https://localhost:9443/api/am/store/v0.15/applications
  22. 22. Login to API Manager Environments • Tool should require login to each environment to facilitate import and export APIs Sample: apictl login <Environment> -u <Username> -p <Password> Example: apictl login dev -u admin -p admin
  23. 23. Export APIs • User can list available APIs in a environment via apictl • Tool provide the flexibility to export APIs with following command Sample: apictl export-api -e <Env> -n <APINAME> -v <VERSION> -r <PROVIDER> -k Example: apictl export-api -e dev -n SwaggerPetstore -v 1.0.0 -r admin -k Sample: apictl apis list <flags> Example: apictl apis list -e dev -q provider:admin
  24. 24. Exported Archive Structure • Export command of the tool will export API in zip format which the user can extract and commit to a SCM repository <API>.zip ├── Docs │ └── FileContents ├── Image ├── Meta-information │ ├── api.json │ └── swagger.json ├── └── Sequences ├── fault-sequence ├── in-sequence └── out-sequence
  25. 25. Environment Specific Parameters • Tool allows to inject environment specific parameters through ‘api_params.yaml’ configuration Format: environments: # Define environments as a list - name: dev # Name of the environment gatewayEnvironments: # define gateway environments - "Production and Sandbox" # List gateways - "Another Gateway" certs: # define certificates for endpoints as a list endpoints: # Define endpoints for environment production: # Production endpoint url: '' # URL of the production backend config: retryTimeOut: 60 # Retries before suspension retryDelay: 90 # Retry Delay(ms) factor: 6 # Endpoint suspension factor
  26. 26. Sample: environments: - name: dev endpoints: production: url: '' sandbox: url: '' - name: prod endpoints: production: url: '' sandbox: url: ''
  27. 27. Importing With Environment Specific Parameters ● Tool can inject environment specific configurations in two ways ○ Specifying path to configuration ○ Creating configuration with name of ‘api_params.yaml’ in extracted folder of the API apictl import-api -f -e prod -k --params /home/user/env_params.yaml apictl import-api -f PhoneVerification_1.0 -e prod -k --update
  28. 28. Initializing APIs Using apictl • Tool provide the flexibility to initialize APIs using swagger or OpenAPI definition via URL or file API is created using base template defined in the ‘.wso2apictl/default_api.yaml’ or custom definition file specified with ‘--definition’ flag • User can configure environment variables template file which will resolve automatically through the tool id: providerName: admin apiName: $APINAME version: $APIVERSION
  29. 29. Format: apictl init <Proj Path> --oas <specification> --definition <template file> Sample: apictl init Petstore --oas apictl init Petstore --oas petstore.yaml
  30. 30. Building the CI/CD Pipeline
  31. 31. API First CI/CD Approach 1. API Developer writes the OpenAPI Specification and Test Scripts 2. Initialize an API Project using OpenAPI definition + API Definition Template + Environment Variables 3. Update API data in the API Project 4. Define Environment Specific Data in API Parameter File 5. Commit the API Project with Test Scripts to SCM Repository (Eg: Github) 6. Trigger a build pipeline of a CI tool using SCM polling (Eg: Trigger Jenkins Pipeline using a Github Webhook) Deploy API to Lower Environment Run Tests Deploy API to Upper Environment
  32. 32. API First Approach Based Automation 1 1 2 3 4 5 6
  33. 33. Demo
  34. 34. Migrating API Updates • apictl provides flexibility to create customized CI/CD approaches to migrate the updates done to APIs in lower environments to upper environments • Developer First Approach 1. API Developer checkout the API Project from SCM Repository 2. Manually update the API template files in API Project 3. Commit the changes to SCM Repository 4. Trigger a build pipeline of a CI tool using SCM polling (Eg: Trigger Jenkins Pipeline using a Github Webhook) Deploy Updated API to Lower Environment Run Tests Deploy Updated API to Upper Environment
  35. 35. Migrating API Updates • API Publisher Portal Based Approach 1. API Developer login to API Publisher Portal UI and update the API 2. Checkout the API Project from SCM Repository 3. Export the updated API zip file locally using apictl 4. Extract the zip and merge the changed files manually in API Project 5. Commit the changes to SCM Repository 6. Trigger a build pipeline of a CI tool using SCM polling (Eg: Trigger Jenkins Pipeline using a Github Webhook) Run Tests Deploy API to Upper Environment
  36. 36. Migrating API Updates • API Publisher Portal Based Approach 1. API Developer login to API Publisher Portal UI and update the API 2. Manually trigger a build pipeline of a CI tool (Eg: Trigger Jenkins Pipeline) Run Tests Export API from Lower Environment Merge Changes and Update API Project in SCM repository Deploy API to Upper Environment
  37. 37. API Publisher Portal Based Automation 1 2
  38. 38. Demo
  39. 39. Manage Developer Applications Across Environments
  40. 40. Manage Applications Across Environments
  41. 41. Export Applications • Export an application using the tool This command exports an Application named as MyApplication from Owner admin in dev environment. Application will be exported as a ZIP archive. • Exporting application with OAuth consumer key and secrets This command exports an Application named as MyApplication from Owner admin in dev environment with the consumer key and secret as specified in --withKeys flag. apictl export-app -n MyApplication -o admin -e dev apictl export-app -n MyApplication -o admin -e dev --withKeys
  42. 42. Import Applications • Import applications to another environment using the tool This will import the to the test environment. This will import all the consumer keys, subscribers into the next environment by default. Update flag will update an existing application if already exists in next environment, if not it will create a new one. If application already associated with consumer key/secret in the new environment for this application, existing keys in new environment will be untouched. • User can disable importing keys or subscriptions Note: By default the applications will be imported under current user executing the command, User can preserve the original user using --preserveOwner flag or change the ownership to a desired owner using --owner flag apictl import-app -f -e test --skipKeys --skipSubscriptions apictl import-app -f -e test --update
  43. 43. Join Our Slack Channel Link
  44. 44. Q&A
  45. 45. THANK YOU