Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Everything as a Code / Александр Тарасов (Одноклассники)

355 views

Published on

РИТ++ 2017, Root Conf
Зал Пекин + Шанхай, 5 июня, 11:00

Тезисы:
http://rootconf.ru/2017/abstracts/2627.html

Процесс разработки не начинается и не заканчивается на написании кода программного продукта. Мы пишем документацию, придумываем, как это всё оттестировать, и заботимся о том, чтобы доступность приложения была на высоком уровне.

Мы все делаем привычные вещи привычным для нас способом. Порой выполняя много ручной и неэффективной работы. Но что, если есть другой, радикальный подход. Можно ли формализовать свою деятельность и переложить её в код? Какие практики и инструменты для этого использовать?

В докладе будет представлен личный опыт автора по автоматизации различных элементов разработки ПО.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Everything as a Code / Александр Тарасов (Одноклассники)

  1. 1. Everything as a Code Непривычно о привычном
  2. 2. @aatarasoff @aatarasoff @aatarasoff
  3. 3. Everything as a Code Непривычно о привычном
  4. 4. No warranty guarantee
  5. 5. You think you are creator 5
  6. 6. 6 … but matrix has you
  7. 7. 7
  8. 8. Выйти за пределы... 8
  9. 9. ...поняв, что всё есть код 9
  10. 10. Как вы представляете себе код? 10
  11. 11. @Configuration @EnableConfigurationProperties(OneServerProperties.class) public class OneServerConfiguration implements ApplicationContextAware { ApplicationContext applicationContext; @Autowired OneServerProperties properties; @Bean public HttpServer httpServer() throws IOException { HttpServer httpServer = new RelativePathHttpServer(); for (String beanName : context.getBeans(HttpController.class)) { httpServer.addRequestHandlers(context.getBean(beanName)); } return httpServer; } } 11 Наверное как-то так
  12. 12. >>,[>>,]<<[ [<<]>>>>[ <<[>+<<+>-] >>[>+<<<<[->]>[<]>>-] <<<[[-]>>[>+<-]>>[<<<+>>>-]] >>[[<+>-]>>]< ]<<[>>+<<-]<< ]>>>>[.>>] 12 ...или может быть так?
  13. 13. Что такое код? ● Коллекция инструкций ● Человекочитаемый формат ○ plain text ● Может быть понят и “проигран” ○ после компиляции ○ интерпретатором 13
  14. 14. Да я тоже пишу код! 14
  15. 15. Вооружи себя 15
  16. 16. Настройка рабочего окружения 16
  17. 17. Install apps Code Checkout Configure workspace First Blood 17
  18. 18. Install apps Code Checkout Configure workspace First Blood brew install 18
  19. 19. Install apps Code Checkout Configure workspace First Blood brew install pip install 19
  20. 20. Install apps Code Checkout Configure workspace First Blood git 20
  21. 21. Install apps Code Checkout Configure workspace First Blood customize *.properties 21
  22. 22. Install apps Code Checkout Configure workspace First Blood customize *.propertiestemplate .gradle 22
  23. 23. Install apps Code Checkout Configure workspace First Blood customize *.properties install certificates 23
  24. 24. Install apps Code Checkout Configure workspace First Blood first build 24
  25. 25. Install apps Code Checkout Configure workspace First Blood mass build first deploy 25
  26. 26. ansible-playbook -i local-inv setup.yml --tags configure 26 Интерпретатор
  27. 27. ansible-playbook -i local-inv setup.yml --tags configure 27 Конфигурация
  28. 28. ansible-playbook -i local-inv setup.yml --tags configure 28 Алгоритм
  29. 29. # checkout repositories from git - include: checkout.yml # configure your local environment - include: configure.yml # add useful mappings to your hosts file - include: hosts.yml # add gradle support - include: gradle.yml # clean and build all projects - include: build.yml # delpoy all services to dev - include: deploy.yml 29 Алгоритм
  30. 30. ansible-playbook -i local-inv setup.yml --tags configure 30 Входные параметры
  31. 31. - name: checkout services git: repo: "{{ git.root }}/{{ item.name }}.git" dest: "{{ work_dir }}/{{ item.name }}" update: yes with_items: - "{{ services }}" tags: - services 31 Массовый checkout/pull
  32. 32. - name: checkout services git: repo: "{{ git.root }}/{{ item.name }}.git" dest: "{{ work_dir }}/{{ item.name }}" update: yes with_items: - "{{ services }}" tags: - services Переменные 32
  33. 33. - name: checkout services git: // with_items: - "{{ services }}" services: - { name: one-instant-messenger , sourceDir: src } - { name: one-discussions , sourceDir: src } - { name: one-attachment , sourceDir: src, testDir: test, local: true } Циклы 33
  34. 34. services: - { name: one-instant-messenger, sourceDir: src } - { name: one-discussions, sourceDir: src } - { name: one-attachment, sourceDir: src, testDir: test } Коллекции и структуры данных 34
  35. 35. - name: create gradle build file template: src: build.gradle.j2 dest: "{{ work_dir }}/build.gradle" mode: 0644 - name: create gradle settings file template: src: settings.gradle.j2 dest: "{{ work_dir }}/settings.gradle" mode: 0644 Шаблоны 35
  36. 36. {% if services is not none %}{% for service in services %} if (project.name == '{{ service.name }}') { {% if 'sourceDir' in service %} main.java.srcDir '{{ service.sourceDir }}' {% endif %}{% if 'testDir' in service %} test.java.srcDir '{{ service.testDir }}' {% endif %} } Условные операторы 36
  37. 37. {% if services is not none %}{% for service in services %} if (project.name == '{{ service.name }}') { {% if 'sourceDir' in service %} main.java.srcDir '{{ service.sourceDir }}' {% endif %}{% if 'testDir' in service %} test.java.srcDir '{{ service.testDir }}' {% endif %} } Опять циклы 37
  38. 38. {% if services is not none %}{% for service in services %} if (project.name == '{{ service.name }}') { {% if 'sourceDir' in service %} main.java.srcDir '{{ service.sourceDir }}' {% endif %}{% if 'testDir' in service %} test.java.srcDir '{{ service.testDir }}' {% endif %} } Переменные 38
  39. 39. - stat: path={{ username }}.key register: certkey - name: generate private key shell: openssl genrsa -out {{ username }}.key -aes256 4096 when: not certkey.stat.exists Идемпотентность 39
  40. 40. Install apps Code Checkout Configure workspace Multiplie Use 40 Петля обратной связи git
  41. 41. Что получили ● Автоконфигурация рабочего пространства ○ повторяемая ○ немутабельная ● Можно дать скрипт новичку ● Можно и нужно пользоваться этим каждый день 41
  42. 42. Код есть код 42
  43. 43. Искусство сборки 43
  44. 44. Compile code Unit tests Package Publishing 44
  45. 45. Compile code Unit tests Package compiler Publishing 45
  46. 46. Compile code Unit tests Package javacresource processing Publishing 46
  47. 47. Compile code Unit tests Package javacresources copyingdependency resolving Publishing 47
  48. 48. Compile code Unit tests Package junit Publishing 48
  49. 49. Compile code Unit tests Package jar Publishing 49
  50. 50. Compile code Unit tests Package jar npm, deb, ... Publishing 50
  51. 51. Compile code Unit tests Package jar npm, so, ... docker image Publishing 51
  52. 52. Compile code Unit tests Package ivy Publishing 52
  53. 53. Compile code Unit tests Package maven, ivy maven Publishing 53
  54. 54. Compile code Unit tests Package Publishing maven, ivylocal or dev deploydocker registry 54
  55. 55. Системы сборки ● Ant + Ivy ● Maven ● Gradle ● Docker ● npm ● ... 55
  56. 56. FROM golang:1.7-alpine MAINTAINER aatarasoff@gmail.com VOLUME /data WORKDIR /data RUN apk update && apk upgrade && apk add git bash RUN go get github.com/aatarasoff/apistress && go install github.com/aatarasoff/apistress CMD [ "apistress" ] Dockerfile. Наследование 56
  57. 57. FROM golang:1.7-alpine MAINTAINER aatarasoff@gmail.com VOLUME /data WORKDIR /data RUN apk update && apk upgrade && apk add git bash RUN go get github.com/aatarasoff/apistress && go install github.com/aatarasoff/apistress CMD [ "apistress" ] Dockerfile. Инструкции 57
  58. 58. FROM golang:1.7-alpine MAINTAINER aatarasoff@gmail.com ARG maindir=/data VOLUME $maindir WORKDIR $maindir RUN apk update && apk upgrade && apk add git bash RUN go get github.com/aatarasoff/apistress && go install github.com/aatarasoff/apistress CMD [ "apistress" ] Dockerfile. Переменные 58
  59. 59. docker build … && docker push … 59
  60. 60. publishing { publications { mavenJava(MavenPublication) { artifactId 'spring-one-nio-autoconfigure' from components.java artifact sourceJar { classifier "sources" } } } } Gradle. DSL 60
  61. 61. compile(ivyDependencies(projectDir, 'runtime')) def ivyDependencies(ivyPath, conf) { def dep = [] def ivyModule = new XmlParser().parse(file("${ivyPath}/ivy.xml")) ivyModule.dependencies.dependency.each dep.add([group: (null == it.@org ? 'ru.odnoklassniki' : it.@org), name: it.@name, version: it.@rev, configuration: (it.@conf =~ /->(w+)/)[0][1]]) } return dep } Gradle. Код как он есть 61
  62. 62. <macrodef name="docker-build-image" description="Build docker image"> <attribute name=" buildcommand" default="build -t @{repo}/@{image}:@{tag} ."/> <sequential> <exec executable="docker"> <arg line=" @{buildcommand} "/> </exec> </sequential> </macrodef> И даже в Ant-е есть жизнь 62
  63. 63. ./gradlew build test ant-docker-build-image publish 63
  64. 64. Фреймворки для автоматизации ● Ant + Ivy ● Maven ● Gradle ● Docker ● npm ● ... 64
  65. 65. Что получили ● Сборка это код ○ XML ○ DSL ○ Groovy ○ etc ● Системы сборки не только для сборки 65
  66. 66. Ликвидация багов 66
  67. 67. Unit tests API tests Stress tests UI tests 67
  68. 68. Unit tests API tests Stress tests UI tests TDD 68
  69. 69. goss --vars vars.yaml validate 69
  70. 70. port: tcp:5601: listening: true ip: - 0.0.0.0 service: mesosd: enabled: true running: true goss.yml 70
  71. 71. port: tcp:5601: listening: true ip: - 0.0.0.0 service: mesosd: enabled: true running: true goss.yml 71
  72. 72. port: tcp:5601: listening: true ip: - 0.0.0.0 service: mesosd: enabled: true running: true goss.yml 72
  73. 73. Unit tests API tests Stress tests UI tests BDD Specs 73
  74. 74. def "Return error code 400, if User-ID header is not presented"() { given: def request = post("/rent") when: def result = this.mvc.perform(request) then: result.andExpect(status().isBadRequest()) .andDo(document("rent-user-id-is-absent")) } Дружелюбный BDD 74
  75. 75. def "Return error code 400, if User-ID header is not presented"() { given: def request = post("/rent") when: def result = this.mvc.perform(request) then: result.andExpect( status().isBadRequest()) .andDo(document("rent-user-id-is-absent")) } Простые проверки 75
  76. 76. Unit tests API tests Stress tests UI tests JMeter, wrk, vegeta 76
  77. 77. Unit tests API tests Stress tests UI tests JMeter production 77
  78. 78. > config | run command > echo $? {0,1} 0 - success 1 - error 78 Экспресс-тест
  79. 79. { "baseUrl": "http://host:9000/rent-service", "tests": [ { "rps": 10, "duration": 5, "target": { "method": "GET", "path": "/rent", "headers": [ ... ] }, "sla": { "latency": 1000, "successRate": 99.999 } }, ... ] } config.json 79
  80. 80. { "baseUrl": "http://host:9000/rent-service", "tests": [ { "rps": 10, "duration": 5, "target": { "method": "GET", "path": "/rent", "headers": [ ... ] }, "sla": { "latency": 1000, "successRate": 99.999 } }, ... ] } config.json 80
  81. 81. { "baseUrl": "http://host:9000/rent-service", "tests": [ { "rps": 10, "duration": 5, "target": { "method": "GET", "path": "/rent", "headers": [ ... ] }, "sla": { "latency": 1000, "successRate": 99.999 } }, ... ] } config.json 81
  82. 82. cat config.json | docker run -i apistress -config=stdin 82 Где-то мы такое видели https://github.com/aatarasoff/apistress
  83. 83. Requests [total, rate] 50, 10.20 Duration [total, attack, wait] 5.022872793s, 4.899943287s, 122.929506ms Latencies [mean, 50, 95, 99, max] 143.772484ms, ..., 290.101831ms Bytes In [total, mean] 4842, 96.84 Bytes Out [total, mean] 950, 19.00 Success [ratio] 100.00% Status Codes [code:count] 200:50 Error Set: Test#1 83
  84. 84. Requests [total, rate] 50, 10.20 Duration [total, attack, wait] 5.022872793s, 4.899943287s, 122.929506ms Latencies [mean, 50, 95, 99, max] 143.772484ms, ..., 290.101831ms Bytes In [total, mean] 4842, 96.84 Bytes Out [total, mean] 950, 19.00 Success [ratio] 100.00% Status Codes [code:count] 200:50 Error Set: Test#1 84
  85. 85. attacker := vegeta.NewAttacker() var metrics vegeta.Metrics for res := range attacker.Attack(targeter, rate, duration) { metrics.Add(res) } metrics.Close() if metrics.Success*100 < test.SLA.SuccessRate { os.Exit(1) } if metrics.Latencies.P99() > SLA.Latency*time.Millisecond.Nanoseconds() { os.Exit(1) } Немного go-кода 85
  86. 86. attacker := vegeta.NewAttacker() var metrics vegeta.Metrics for res := range attacker.Attack( targeter, rate, duration) { metrics.Add(res) } metrics.Close() if metrics.Success*100 < test.SLA.SuccessRate { os.Exit(1) } if metrics.Latencies.P99() > SLA.Latency*time.Millisecond.Nanoseconds() { os.Exit(1) } Майним метрики 86
  87. 87. attacker := vegeta.NewAttacker() var metrics vegeta.Metrics for res := range attacker.Attack(targeter, rate, duration) { metrics.Add(res) } metrics.Close() if metrics.Success*100 < test.SLA.SuccessRate { os.Exit(1) } if metrics.Latencies.P99() > SLA.Latency*time.Millisecond.Nanoseconds() { os.Exit(1) } Не слишком ли много ошибок? 87
  88. 88. attacker := vegeta.NewAttacker() var metrics vegeta.Metrics for res := range attacker.Attack(targeter, rate, duration) { metrics.Add(res) } metrics.Close() if metrics.Success*100 < test.SLA.SuccessRate { os.Exit(1) } if metrics.Latencies.P99() > SLA.Latency*time.Millisecond.Nanoseconds() { os.Exit(1) } Уложились ли по времени? 88
  89. 89. > cat config.json | docker run -i apistress -config=stdin > echo $? 0 89 Код возврата
  90. 90. Requests [total, rate] 200, 10.05 Duration [total, attack, wait] 23.04784s, 19.899754743s, 3.148093811s Latencies [mean, 50, 95, 99, max] 3.023677499s, ..., 11.832287083s Bytes In [total, mean] 6874, 34.37 Bytes Out [total, mean] 1349, 6.75 Success [ratio] 35.50% Status Codes [code:count] 0:129 200:71 Error Set: Get http://host:9000/rent-service/rent: EOF Get http://host:9000/rent-service/rent: http: server closed idle connection ... Test#2 90
  91. 91. > cat config.json | docker run -i apistress -config=stdin > echo $? 1 91 Код возврата
  92. 92. Unit tests API tests Stress tests UI tests Selenium, Selenide 92
  93. 93. Что получили ● Все тестовые сценарии в коде ● Можно встроить в процесс сборки или доставки ПО ● Если хотите, то можно генерировать отчёты для разбора полётов 93
  94. 94. Эксперименты 94
  95. 95. Code Branches 95 if / switch
  96. 96. Code Branches 96 if / switch DI
  97. 97. Code Branches 97 if / switch DI API v2
  98. 98. ISupplier<String, Ctx> srcsetSupplier = (ctx) -> { if (configuration. isDoubleDensityAvatarsEnabled(user.getModel())) { String link = imageSrc.getModel(); return linkBuilder.createSourceSetLink(link); } return ""; }; Очень простой эксперимент 98
  99. 99. Code Branches One server or partition 99 //by partition app.photos.doubleDensityAvatarsEnabled: 0 Step#1
  100. 100. Code Branches One server or partition Part of servers or partitions 100 //by partition app.photos.doubleDensityAvatarsEnabled: 0,4,7-9 Step#1 Step#2
  101. 101. Code Branches One server or partition Part of servers or partitions All servers or partitions 101 //by partition app.photos.doubleDensityAvatarsEnabled: ALL Step#1 Step#2 Step#3
  102. 102. //step#1 app.photos.doubleDensityAvatarsEnabled: 0 //step#2 app.photos.doubleDensityAvatarsEnabled: 0,4,7-9 //step#3 app.photos.doubleDensityAvatarsEnabled: ALL 102 Очень простой эксперимент
  103. 103. No one One server or partition Part of servers or partitions All servers or partitions 103 monitor Step#1 Step#2 Step#3Step#0
  104. 104. 104 Что хотим? ansible-playbook -i {dev,test,prod}-env exp.yml --tags stepN
  105. 105. 105 Абстракция - name: update properties uri: url: "http://{{ pms_host }}/api/conf/update" method: POST user: "{{ username }}" password: "{{ password }}" force_basic_auth: yes body: applicationName: "{{ application_name }}" propertyName: "{{ item.value.name }}" propertyValue: "{{ item.value.value }}" body_format: json status_code: 200 headers: Content-Type: "application/json" with_dict: "{{ properties }}"
  106. 106. 106 Координаты - name: update properties uri: url: "http://{{ pms_host }}/api/conf/update" method: POST user: "{{ username }}" password: "{{ password }}" force_basic_auth: yes body: applicationName: "{{ application_name }}" propertyName: "{{ item.value.name }}" propertyValue: "{{ item.value.value }}" body_format: json status_code: 200 headers: Content-Type: "application/json" with_dict: "{{ properties }}"
  107. 107. 107 Вкатываем настройки - name: update properties uri: url: "http://{{ pms_host }}/api/conf/update" method: POST user: "{{ username }}" password: "{{ password }}" force_basic_auth: yes body: applicationName: "{{ application_name }}" propertyName: "{{ item.value.name }}" propertyValue: "{{ item.value.value }}" body_format: json status_code: 200 headers: Content-Type: "application/json" with_dict: "{{ properties }}"
  108. 108. 108 Проверяем корректность - name: update properties uri: url: "http://{{ pms_host }}/api/conf/update" method: POST user: "{{ username }}" password: "{{ password }}" force_basic_auth: yes body: applicationName: "{{ application_name }}" propertyName: "{{ item.value.name }}" propertyValue: "{{ item.value.value }}" body_format: json status_code: 200 headers: Content-Type: "application/json" with_dict: "{{ properties }}"
  109. 109. 109 Step#0 - hosts: local vars: application_name: odnoklassniki-web props: doubleDensityAvatarsEnabled: name: "app.photos.doubleDensityAvatarsEnabled" value: "" roles: - { name: pms, properties: "{{ props }}" }
  110. 110. 110 Step#0 - hosts: local vars: application_name: odnoklassniki-web props: doubleDensityAvatarsEnabled: name: "app.photos.doubleDensityAvatarsEnabled" value: "" roles: - { name: pms, properties: "{{ props }}" }
  111. 111. 111 Step#1 - hosts: local vars: application_name: odnoklassniki-web props: doubleDensityAvatarsEnabled: name: "app.photos.doubleDensityAvatarsEnabled" value: "0" roles: - { name: pms, properties: "{{ props }}" }
  112. 112. 112 Step#2 - hosts: local vars: application_name: odnoklassniki-web props: doubleDensityAvatarsEnabled: name: "app.photos.doubleDensityAvatarsEnabled" value: "0,4,7-9" roles: - { name: pms, properties: "{{ props }}" }
  113. 113. 113 Step#3 - hosts: local vars: application_name: odnoklassniki-web props: doubleDensityAvatarsEnabled: name: "app.photos.doubleDensityAvatarsEnabled" value: "ALL" roles: - { name: pms, properties: "{{ props }}" }
  114. 114. 114 exp.yml --- - include: step0.yml tags: - step0 - cleanup - include: step1.yml tags: step1 - include: step2.yml tags: step2 - include: step3.yml tags: - step3 - complete
  115. 115. Что получили ● Эксперименты хранятся в git- репозитории ● Можно применить любой шаг в любой момент времени на любой среде ● Можно встроить в пайплайн доставки 115
  116. 116. Кододокументация 116
  117. 117. Analyst, PM Developer Tester Docs Word, PDF... 117
  118. 118. Analyst, PM Developer Tester Docs Word, PDF... Code + Tests 118
  119. 119. Analyst, PM Developer Tester Docs Word, PDF... Code + Tests Test cases 119
  120. 120. Analyst, PM Developer Tester Docs :( Word, PDF... Code + Tests Test cases 120
  121. 121. Analyst, PM Developer Tester Docs :) Markdown/Asciidoctor Docs :) 121
  122. 122. = Hippo Rent Service This is documentation for Open API of our hippo renting service == Methods === Rent ==== Request specification ===== Headers //тут опишем http-заголовки ===== Example //а здесь будут примеры вызова ==== Response specification ===== Response fields //здесь описание полей ответа ===== Example //ну и пример того, что вообще ожидать в ответе Вот такой документ 122
  123. 123. ./gradlew ... asciidoc publishDocs 123 Компиляция документа
  124. 124. def "Rent a hippo"() { given: def request = post("/rent").header("User-ID", "aatarasoff") when: def result = this.mvc.perform(request) then: result.andExpect(status().isOk()) .andDo(document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) )) } …и снова тесты 124
  125. 125. def "Rent a hippo"() { given: def request = post("/rent").header("User-ID", "aatarasoff") when: def result = this.mvc.perform(request) then: result.andExpect(status().isOk()) .andDo(document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) )) } А не документация ли это? 125
  126. 126. document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ) Имя сниппета 126
  127. 127. document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ) Тестируем заголовки 127
  128. 128. document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ) Тестируем поля ответа 128
  129. 129. generated-snippets rent-hippo curl-request.adoc http-request.adoc http-response.adoc httpie-request.adoc request-headers.adoc response-fields.adoc Получаем сниппеты 129
  130. 130. = Hippo Rent Service This is documentation for Open API of our hippo renting service == Methods === Rent ==== Request specification ===== Headers include::{snippets}/rent-hippo/request-headers.adoc[] ===== Example include::{snippets}/rent-hippo/http-request.adoc[] ==== Response specification ===== Response fields include::{snippets}/rent-hippo/response-fields.adoc[] ===== Example include::{snippets}/rent-hippo/http-response.adoc[] Вставляем их в документ 130
  131. 131. = Hippo Rent Service This is documentation for Open API of our hippo renting service == Methods === Rent ==== Request specification ===== Headers include::{snippets}/rent-hippo/request-headers.adoc[] ===== Example include::{snippets}/rent-hippo/http-request.adoc[] ==== Response specification ===== Response fields include::{snippets}/rent-hippo/response-fields.adoc[] ===== Example include::{snippets}/rent-hippo/http-response.adoc[] 131
  132. 132. 132
  133. 133. Что получили? ● Документация как код ○ лежит в репозитории с кодом ○ встроена в цикл сборки ○ рендерится в html, pdf и т.д. ○ почти всегда актуальна ● Синергия с тестами 133
  134. 134. Инфракод 134
  135. 135. Hardware Containers Application PaaS Mesos/Kubernetes/Private cloud 135
  136. 136. Hardware + OS System Libs PaaS Application 136
  137. 137. Hardware + OS System Libs PaaS Application Ansible 137
  138. 138. Hardware + OS System Libs PaaS Application Ansible Puppet/Chef 138
  139. 139. Ansible. Inventory [datacenter] api-server-1 api-server-2 api-server-3 [datacenter:vars] magicvar = 42 139
  140. 140. Ansible. Playbook - hosts: datacenter roles: - role: docker - role: rsyslog 140
  141. 141. ansible-playbook -i dc1 bootstrap.yml 141 Без комментариев
  142. 142. Hardware + OS System Libs PaaS Application Ansible 142
  143. 143. Hardware + OS System Libs PaaS Application Ansible Puppet/Chef 143
  144. 144. Hardware + OS System Libs PaaS Application Manifest 144
  145. 145. Docker compose services: zk: image: zookeeper network_mode: bridge ports: - 2181:2181 environment: ZK_CONFIG: tickTime=2000,initLimit=10,clientPort=2181 ZK_ID: 1 145
  146. 146. Docker compose services: zk: image: zookeeper network_mode: bridge ports: - 2181:2181 environment: ZK_CONFIG: tickTime=2000,initLimit=10,clientPort=2181 ZK_ID: 1 146
  147. 147. Конфигурация сервиса services: zk: image: zookeeper network_mode: bridge ports: - 2181:2181 environment: ZK_CONFIG: tickTime=2000,initLimit=10,clientPort=2181 ZK_ID: 1 147
  148. 148. Mesos/Marathon { "id": "/api/rent-service", "cpus": 1, "mem": 1024, "instances": 3, "container": { "docker": { "image": "rent-service:0.0.1", "portMappings": [ { "containerPort": 8080 } ] } } } 148
  149. 149. curl -X POST ... http://marathon/v2/apps?force=true 149 Современный деплоймент
  150. 150. Конфигурация приложений https://your_repo/rent-service-config/routes.yml routes: payment: path: /payment-service/** serviceId: payment-service 150
  151. 151. Ещё конфигурация ● Zookeeper ● Consul ● Vault ● configo ● ... 151
  152. 152. configo 152 docker run -e CONFIGO_SOURCE_0='{"type": "http", "format": "yaml", "url": "https://my.server.com/common.yaml"}' rent-service //внутри приложения getEnvironmentVariable("MY_ENV_VAR") https://github.com/bsideup/configo
  153. 153. configo 153 docker run -e CONFIGO_SOURCE_0='{"type": "http", "format": "yaml", "url": "https://my.server.com/common.yaml"}' -e CONFIGO_SOURCE_1='{"type" : "consul", "address": "consul.master:8500", "prefix" : "common"}' rent-service //внутри приложения getEnvironmentVariable("MY_ENV_VAR") https://github.com/bsideup/configo
  154. 154. Что получили? ● Инфрастуктура может быть легко описана в виде кода ● Деплоймент и конфигурация приложений в виде конфигов и манифестов 154
  155. 155. Неубиваемый CI 155
  156. 156. 156 Install Master Configure Slaves
  157. 157. 157 Install Master Configure Slaves Ansible
  158. 158. 158 Install Master Configure Slaves Ansible
  159. 159. Jenkins Docker Cloud plugin <——— Хост с докером <——— Сколько контейнеров можно запустить 159
  160. 160. Автоконфигурация <clouds> {% for group in ['build', 'test', 'production'] %} {% for node in groups[group + '-slaves'] %} <com.github.kostyasha.yad.DockerCloud plugin="yet-another-docker-plugin@0.1.0-rc31"> <name>{{ node }}</name> ... <templates> <com.github.kostyasha.yad.DockerSlaveTemplate> <id>mycloud-template</id> <dockerContainerLifecycle> <image>{{ group }}-jenkins-slave</image> ... </templates> <connector> <serverUrl>tcp://{{ node.hostname }}:2375</serverUrl> <apiVersion>1.20</apiVersion> </connector> </com.github.kostyasha.yad.DockerCloud> {% endfor %} {% endfor %} </clouds> 160
  161. 161. Код доставки 161
  162. 162. //checkout and definition stage node('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git" // Mark build 'stage' stage 'Build' sh ('./gradlew clean build final') } //next steps
  163. 163. //checkout and definition stage node('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git" // Mark build 'stage' stage 'Build' sh ('./gradlew clean build final') } //next steps
  164. 164. //checkout and definition stage node('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git" // Mark build 'stage' stage 'Build' sh ('./gradlew clean build final') } //next steps
  165. 165. //checkout and definition stage node('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git" // Mark build 'stage' stage 'Build' sh ('./gradlew clean build final') } //next steps
  166. 166. //checkout and definition stage node('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git" // Mark build 'stage' stage 'Build' sh ('./gradlew clean build final') } //next steps
  167. 167. //checkout and definition stage node('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git" // Mark build 'stage' stage 'Build' sh ('./gradlew clean build final') } //next steps
  168. 168. //deploy artifact to test node('test') { sh('ansible-galaxy install -r requirements.yml') ansiblePlaybook( credentialsId: 'ansible', installation: 'ansible', playbook: 'deploy.yml', inventory: 'test' ) }
  169. 169. //deploy artifact to test node('test') { sh('ansible-galaxy install -r requirements.yml') ansiblePlaybook( credentialsId: 'ansible', installation: 'ansible', playbook: 'deploy.yml', inventory: 'test' ) }
  170. 170. //deploy artifact to test node('test') { sh('ansible-galaxy install -r requirements.yml') ansiblePlaybook( credentialsId: 'ansible', installation: 'ansible', playbook: 'deploy.yml', inventory: 'test' ) }
  171. 171. //deploy artifact to test node('test') { sh('ansible-galaxy install -r requirements.yml') ansiblePlaybook( credentialsId: 'ansible', installation: 'ansible', playbook: 'deploy.yml', inventory: 'test' ) }
  172. 172. //deploy artifact to test node('test') { sh('ansible-galaxy install -r requirements.yml') ansiblePlaybook( credentialsId: 'ansible', installation: 'ansible', playbook: 'deploy.yml', inventory: 'test' ) } jiraComment ( issueKey: issue_id, body: "Artifact has been deployed" )
  173. 173. node('build') { def repos = fetchRepos(project) for (repo in repos) { build(repo) } } def fetchRepos(String project) { def url = new URL("https://repo/projects/${project}/repos?limit=1000") def conn = url.openConnection() conn.setRequestMethod("GET") def responseCode = conn.getResponseCode() final slurper = new groovy.json.JsonSlurper() def repos = slurper.parse(conn.getInputStream()).values for (repo in repos) { if (repo.slug.contains('one-')) result << repo.slug } return result }
  174. 174. node('build') { def repos = fetchRepos(project) for (repo in repos) { build(repo) } } def fetchRepos(String project) { def url = new URL("https://repo/projects/${project}/repos?limit=1000") def conn = url.openConnection() conn.setRequestMethod("GET") def responseCode = conn.getResponseCode() final slurper = new groovy.json.JsonSlurper() def repos = slurper.parse(conn.getInputStream()).values for (repo in repos) { if (repo.slug.contains('one-')) result << repo.slug } return result }
  175. 175. node('build') { def repos = fetchRepos(project) for (repo in repos) { build(repo) } } def fetchRepos(String project) { def url = new URL("https://repo/projects/${project}/repos?limit=1000") def conn = url.openConnection() conn.setRequestMethod("GET") def responseCode = conn.getResponseCode() final slurper = new groovy.json.JsonSlurper() def repos = slurper.parse(conn.getInputStream()).values for (repo in repos) { if (repo.slug.contains('one-')) result << repo.slug } return result }
  176. 176. Микросервисы 178
  177. 177. 179 Install Master Configure Slaves Create meta job Ansible
  178. 178. 180 Install Master Configure Slaves Create meta job Ansible cURL
  179. 179. 181 Install Master Configure Slaves Create meta job Create pipelines
  180. 180. jobs.each { job -> pipelineJob("${basePath}/${job}") { //define SCM definition { cps { script(readFileFromWorkspace('pipeline-template.groovy')) sandbox() } } } } JobDSL plugin 182
  181. 181. jobs.each { job -> pipelineJob("${basePath}/${job}") { //define SCM definition { cps { script(readFileFromWorkspace('pipeline-template.groovy')) sandbox() } } } } JobDSL plugin 183
  182. 182. jobs.each { job -> pipelineJob("${basePath}/${job}") { //define SCM definition { cps { script(readFileFromWorkspace('pipeline-template.groovy')) sandbox() } } } } JobDSL plugin 184
  183. 183. 185 Install Master Configure Slaves Create meta job Create pipelines git
  184. 184. ansible-playbook -i jenkins-for-my-team jenkins.yml 186 Это последний раз
  185. 185. Что получили? ● Пайплайн как код ● Неубиваемый CI ○ без бэкапов ○ всё хранится как код ○ разворачивается за X минут 187
  186. 186. Development Testing Deployment Post Deployment 188 > ./gradlew build test
  187. 187. Development Testing Deployment Post Deployment 189 > ./gradlew integrationTest publishDocs
  188. 188. Development Testing Deployment Post Deployment 190 > ansible-playbook -i env deploy.yml
  189. 189. Development Testing Deployment Post Deployment 191 > ansible-playbook -i prod exp.yml
  190. 190. Development Testing Deployment Post Deployment 192 Delivery Pipeline
  191. 191. 193
  192. 192. Позитивные выводы ● Почти любой процесс можно формализовать, представить в виде кода и автоматизировать ● Мы все пишем код, хотя можем думать, что это не так ● Рано или поздно всё превращается в код 194
  193. 193. Trade-offs ● Необходимы как разовые “капиталовложения”, так и постоянные затраты ресурсов ● Могут потребоваться изменения в архитектуре ● Требует дисциплины и более высокой квалификации специалистов 195
  194. 194. Спасибо, что выбрали красную
  195. 195. @aatarasoff @aatarasoff @aatarasoff QA

×