Drone CI/CD 自動化測試及部署

Drone CI/CD

⾃動化測試及部署

About me
• Software Engineer in Mediatek

• Member of Drone CI/CD Platform

• Member of Gitea Platform

• Member of Gin Golang Framework

• Teacher of Udemy Platform: Golang + Drone
https://blog.wu-boy.com

課程⼤綱
• Docker 基礎介紹

• CI/CD 服務簡介

• Drone 架構介紹

• Drone 基礎使⽤

• Drone 外掛撰寫

VM 虛擬機好處
• 不同的作業系統環境

• 善⽤資源分配 (CPU, RAM, Storage)

• 透過 Hypervisor 管理 VM 快照

Container 容器好處
• 容器檔案非常⼩ (Ubuntu 才⼗幾 MB)

• 快速開啟 (不到⼀秒就可以起⼀個 Ubuntu)

• 透過 Docker
fi
le 客製化容器內容

• 快速實踐 Deployment + Operation

docker 基礎指令
• docker ps

• 列出正在跑的容器

• docker images

• 列出已經下載的 Images

• docker image prune -a -f

• 清除沒在使⽤的 Images (空間如果不夠)

• docker container prune -f

• 清除沒在跑的容器

• docker login -u xxxx

• 登入 docker registry (GitLab 內建, 預設 DockerHub)

• docker run -ti ubuntu /bin/bash

• 快速建立 ubuntu 容器並且登入 Bash Shell

• ⽤於測試及除錯

建立 Docker 服務
• docker run -d -p 8081:80 nginx



• -p port 對應關係 (host port : container port)

• -d 丟到背景執⾏

• —name 容器命名

• -e 環境變數

• -v 掛載硬碟

進入已存在容器
• docker exec -ti name /bin/bash

• docker ps 可以知道正在跑的容器

• 除錯常⽤

練習
• 在本機端架設底下服務

• redis (port 6380:6379)

• mysql (port 3307:3306)

• postgres (port 5433:5432)

• ⽤ Go 或 Node 測試連線

練習
• 架設 Go Module Proxy

• https://github.com/gomods/athens

• export GOPROXY=http://localhost:3010

• 掛載 local 端空間到 athens

FROM alpine:3.15

LABEL maintainer="Bo-Yi Wu <appleboy.tw@gmail.com>"

RUN apk add
-
-
no
-
cache ca
-
certif
i
cates
&
&

rm
-
rf /var/cache/apk
/
*

COPY release/linux/amd64/agent /bin/

ENTRYPOINT [ "/bin/agent" ]

編譯容器並上傳
• docker build -t appleboy/server:tag -f path .

• -t 組織名稱/容器名稱

• -f Docker
fi
le 檔案路徑

• tag: 版本 (⽤來 deploy production)

• docker push host:port/appleboy/server

• host: docker registry host (預設上傳到 docker hub)

• port: docker registry port

version: '3.1'

services:

prometheus:

image: prom/prometheus:v2.31.1

volumes:

- ./prometheus/:/etc/prometheus/

- ./data/prometheus:/prometheus

command:

- '
-
-
conf
i
g.f
i
le=/etc/prometheus/prometheus.yml'

- '
-
-
storage.tsdb.path=/prometheus'

- '
-
-
web.console.libraries=/usr/share/prometheus/console_libraries'

- '
-
-
web.console.templates=/usr/share/prometheus/consoles'

- '
-
-
storage.tsdb.retention.time=365d'

restart: always

grafana:

image: grafana/grafana:8.2.6

depends_on:

- prometheus

volumes:

- ./data/grafana:/var/lib/grafana

- ./grafana/provisioning/:/etc/grafana/provisioning/

env_f
i
le:

- ./grafana/conf
i
g.monitoring

restart: always

docker-compose 指令
• docker-compose up -d [service_name]

• -d 在背景執⾏

• docker-compose down -v

• -v 移除 volume 資料

• docker-compose logs -f [service_name]

• 看單⼀服務 Logs

• docker-compose pull

• 更新全部 Image (每次 deploy 都需要做此步驟)

• docker-compose exec db /bin/bash

• 進入容器看 DB 資料

練習
• 將 Go 服務搭配 redis + (mysql or postgres)

• redis

• mysql

• postgres

• ⽤ docker-compose 串起上⾯服務

部署流程

GitHub Flow vs Git Flow
https://blog.wu-boy.com/2017/12/github-
fl
ow-vs-git-
fl
ow/

GitHub Flow
14
Develop
Git Push
Git Tag
Develop
Git Push
Git Tag
Testing
Deploy
Deploy
Deploy
Production
Staging
Production
Testing
Deploy
Staging

GitHub Flow + Git Flow

in opensource
https://github.com/go-gitea/gitea

如何⾃動化上⾯的步驟?

Why?
• Complicated project setting

• Write the plugin (Java language)

• Maintenance?

• Learning Curve?

• Grow your team?

為什麼我不選

GitLab CI?

Why?
• GitLab Only

• Not Extensible

Drone CI
• Container native CI/CD platform

• Easy to install & maintain

• Isolate builds

• Simple YAML Con
fi
guration

• Integrates with several VCS Providers

• Rich set of of
fi
cial plugins (any container can be a plugin)

• Execute locally with simple command (drone exec)

• open source (https://github.com/harness/drone)

Drone CI/CD 平台真的免費嗎
https://blog.wu-boy.com/2021/08/drone-license/

Everything is a

Docker Container

Project CI/CD Flow
git clone testing deploy notify

Drone CI Infrastructure
Runner
Server
Step 1

git clone
Step 2

make build
Step 3

deploy app
work space
extra service
Runner

Apply GitHub Application
https:
/
/
github.com/settings/developers

https:
/
/
github.com/settings/applications/new

Drone Server and Runner

In docker-compose.yml

version: '3'

services:

drone
-
server:

image: drone/drone:2

ports:

- 8081
:
80

volumes:

- ./:/data

restart: always

environment:

- DRONE_SERVER_HOST=${DRONE_SERVER_HOST}

- DRONE_SERVER_PROTO=${DRONE_SERVER_PROTO}

- DRONE_RPC_SECRET=${DRONE_RPC_SECRET}

# GitHub Conf
i
g

- DRONE_GITHUB_SERVER=https:
/
/
github.com

- DRONE_GITHUB_CLIENT_ID=${DRONE_GITHUB_CLIENT_ID}

- DRONE_GITHUB_CLIENT_SECRET=${DRONE_GITHUB_CLIENT_SECRET}

- DRONE_LOGS_PRETTY=true

- DRONE_LOGS_COLOR=true
Create docker-compose.yml

DRONE_SERVER_HOST=

DRONE_SERVER_PROTO=

DRONE_RPC_SECRET=

DRONE_GITHUB_CLIENT_ID=

DRONE_GITHUB_CLIENT_SECRET=
Create .env
fi
le

drone
-
runner:

image: drone/drone
-
runner
-
docker:1

restart: always

depends_on:

- drone
-
server

ports:

- 3000
:
3000

volumes:

- /var/run/docker.sock:/var/run/docker.sock

environment:

- DRONE_RPC_HOST=${DRONE_SERVER_HOST}

- DRONE_RPC_PROTO=${DRONE_SERVER_PROTO}

- DRONE_RPC_SECRET=${DRONE_RPC_SECRET}

- DRONE_RUNNER_CAPACITY=2

- DRONE_RUNNER_NAME=my
-
f
i
rst
-
runner

kind: pipeline

type: docker

name: default

steps:

- name: greeting

image: alpine

commands:

- echo hello

- echo world

kind: pipeline

type: docker

name: linux_amd64

steps:

- name: build

image: golang:1.17

commands:

- make build_linux_amd64

build_linux_amd64
:

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build

-
v
-
a
-
o release/linux/amd64/helloworld

-
-
-

kind: pipeline

type: docker

name: linux_amd64

steps:

- name: build

image: golang:1.17

environment:

GOOS
:
linux

GOARCH
:
amd64

CGO_ENABLED
:
0

commands:


build:

go build
-
v-
a
-
o

release/${GOOS}/${GOARCH}/helloworld

kind: pipeline

type: docker

name: linux_amd64

steps:

- name: build

image: golang:1.17

commands:


-
-
-

kind: pipeline

type: docker

name: linux_i386

steps:

- name: build

image: golang:1.17

commands:

- make build_linux_i386
Multiple Pipeline

-
-
-

kind: pipeline

type: docker

name: linux_i386

steps:

- name: build

image: golang:1.17

environment:

GOOS
:
linux

GOARCH
:
386

CGO_ENABLED
:
0

commands:

Custom Environment

-
-
-

kind: pipeline

type: docker

name: linux_i386

environment:

GOOS
:
linux

GOARCH
:
386

CGO_ENABLED
:
0

steps:

- name: build

image: golang:1.17

commands:

Global Environment

Routing

將特定任務丟到特定主機

(GPU or High Memory)

kind: pipeline

type: docker

name: default

steps:

- name: build

image: golang

commands:

- go build

- go test

node:

keyA
:
valueA

keyB
:
valueB

DRONE_RUNNER_LABELS=keyA:valueA,keyB:valueB

kind: pipeline

type: docker

name: default

steps:

- name: backend

image: golang

commands:

- go build

- go test

- name: frontend

image: node

commands:

- npm install

- npm test

- name: notify

image: plugins/slack

depends_on:

- frontend

- backend

kind: pipeline

type: docker

name: default

steps:

- name: build

image: golang

commands:

- go build

- go test

when:

branch:

- master

- feature
/
*

kind: pipeline

type: docker

name: default

steps:

- name: build

image: golang

commands:

- go build

- go test

trigger:

branch:

- master

kind: pipeline

type: docker

name: default

steps:

- name: test

image: golang

volumes:

- name: cache

path: /go

commands:

- go get

- go test

- name: build

image: golang

volumes:

- name: cache

path: /go

commands:

- go build

volumes:

- name: cache

temp: {}

練習
• 任何程式語⾔進⾏ Lint + Testing + Build

• Go 語⾔使⽤ golangci-lint

• 測試 go test -v -cover

• 編譯 go build -o release/linux/amd64/out

Plugin
http://plugins.drone.io/

- name: publish

pull: always

image: plugins/docker:linux
-
amd64

settings:

auto_tag: true

auto_tag_suff
i
x: linux
-
amd64

cache_from: appleboy/gorush

daemon_off: false

dockerf
i
le: docker/Dockerf
i
le.linux.amd64

password:

from_secret: docker_password

repo: appleboy/gorush

username:

from_secret: docker_username

when:

event:

exclude:

- pull_request

PLUGIN_USERNAME=appleboy
PLUGIN_PASSWORD=password
PLUGIN_REPO=appleboy/gorush
PLUGIN_TAGS=1.0.0,1.0

Go Plugin vs Bash Plugin

(Any language you want)

kind: pipeline

type: docker

name: default

steps:

- name: webhook

image: acme/webhook

settings:

url: http:
/
/
hook.acme.com

method: post

body: |

hello world

#
!
/bin/sh

curl

-X ${PLUGIN_METHOD}

-
d ${PLUGIN_BODY}

${PLUGIN_URL}

FROM alpine:3.14

ADD script.sh /bin/

RUN chmod
+
x /bin/script.sh

RUN apk -Uuv add curl ca
-
certif
i
cates

ENTRYPOINT /bin/script.sh

$ docker build -t acme/webhook .
$ docker push acme/webhook
$ docker run --rm
-e PLUGIN_METHOD=post
-e PLUGIN_URL=http://hook.acme.com
-e PLUGIN_BODY=hello
acme/webhook
編譯

$ docker build -t acme/webhook .
$ docker push acme/webhook
$ docker run --rm
-e PLUGIN_METHOD=post
-e PLUGIN_URL=http://hook.acme.com
-e PLUGIN_BODY=hello
acme/webhook
測試

package main

import (

"net/http"

"os"

"strings"

)

func main() {

body
:
=
strings.NewReader(

os.Getenv("PLUGIN_BODY"),

)

_, err
:
=
http.NewRequest(

os.Getenv("PLUGIN_METHOD"),

os.Getenv("PLUGIN_URL"),

body,

)

if err
!
=
nil {

os.Exit(1)

}

} GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o webhook

FROM alpine:3.14

ADD webhook /bin/

RUN apk -Uuv add ca
-
certif
i
cates

ENTRYPOINT /bin/webhook

練習
• 任何程式語⾔進⾏ Plugin 撰寫

• 申請 dockerhub 帳號密碼

• 透過 Drone ⾃動打包 Image 並上傳

Deploy to Production Server

via SSH Command

- name: ssh commands

image: appleboy/drone
-
ssh

settings:

host: foo.com

username: root

key:

from_secret: ssh_key

passphrase: 1234

port: 22

script:

- mkdir abc/def/efg

- echo "you can't see the steps."

kind: pipeline

type: docker

name: default

steps:

- name: test

image: node

commands:

- npm install

- npm run test

- npm run bundle

- name: deploy

-
ssh

settings:
.
.
.

trigger:

event:

- promote

target:

- production

kind: pipeline

type: docker

name: deploy

steps:

- name: test

image: node

commands:

- npm install

- npm run test

- npm run bundle

- name: deploy

-
ssh

settings:
.
.
.

when:

event:

- promote

target:

- production

$ drone build promote octocat/hello-world 42 staging
$ drone build promote <repo> <number> <environment>
$ drone build promote octocat/hello-world 42 production

Drone CI/CD 自動化測試及部署

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Drone CI/CD 自動化測試及部署

Similar to Drone CI/CD 自動化測試及部署 (20)

More from Bo-Yi Wu

More from Bo-Yi Wu (20)

Recently uploaded

Recently uploaded (20)

Drone CI/CD 自動化測試及部署