Coscup

  • 198 views
Uploaded on

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
198
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Better node.js software
  • 2. release practice using open
  • 3. source tool
  • 4. ????????
  • 5. Software release to production is such a hard practice, especially considering the business
  • 6. impact and the quality of software . I will demonstrate how to use open source tools to
  • 7. enhance a node.js web software release process and ensure the better release quality.
  • 8. The open source tools I will mention will includes Jenkins, Git, Ubuntu dpkg system, node.js
  • 9. language , ruby , rake, jshint , nodeunit, expresso , YETI, Selenium, and mongodb
  • 10. ?, ??????? 100+ ???? = =
  • 11. Web Software release best
  • 12. practice
  • 13. Agenda
  • 14. Who am I ?
  • 15. Release engineering in Y! Shopping
  • 16. Process
  • 17. Testing
  • 18. Deploy Tools
  • 19. Configuration Management
  • 20. Who am I ?
  • 21. thegiive
  • 22. • A open source lover
  • 23. • Release manager of Taiwan Yahoo! Shopping site
  • 24. • Continuous study on software development, release engineering and software quality
  • 25. • Fall in love with Ruby, VIM, Puppet and Rake
  • 26. • Trying to do more work rather than write more blog
  • 27. Release engineering in Y! Shopping
  • 28. Common image about release
  • 29. engineer
  • 30. Yahoo! Shopping release manager
  • 31. integration hub for more complex software development teams, sitting at the cross between
  • 32. development, product management, quality assurance and other engineering efforts, also known
  • 33. as DevOps
  • 34. DevOps
  • 35. Release???1 : ???
  • 36. ??????????????
  • 37. • RD Team ???? , PHP Team , .Net Team , Search Team
  • 38. • User Team ???? , ?? Team , ?? Team , Search team
  • 39. Release???1 : ???
  • 40. ?????
  • 41. Release???2 : bug????
  • 42. From Book "Beautiful Testing" Chaptor 2
  • 43. TestROI : ????????
  • 44. • ACPB : Average Cost of a Production Bug
  • 45. • ACTB : Average Cost of a Test Bug
  • 46. • TBN : Total Bug Number
  • 47. • TDBC : Total Defect Bug Cost
  • 48. Taiwan Shopping ? ACPB?????? Test ROI ???
  • 49. Test ROI ??????????? testing ????
  • 50. Release???2 : bug????
  • 51. Release???3 : User????
  • 52. • ??RD?bug?QA????????SE??????RM?????
  • 53. • User ???? monitor script , ????????????
  • 54. ???? Release Manager , not release engineer
  • 55. Release???4 : ??????
  • 56. PHP Team
  • 57. • Dev process : Scrum
  • 58. • Source Code Management : Subversion
  • 59. • Language : PHP and partial C++
  • 60. • OS : Linux
  • 61. .Net Team
  • 62. • Dev process : Waterfall
  • 63. • Source Code Management : VSS
  • 64. • Language : ASP.Net and ASP
  • 65. • OS : Windows
  • 66. ????????????? ... ?????? 100??
  • 67. release ?release 2000 ??
  • 68. item
  • 69. ??????
  • 70. Release quality ??
  • 71. Y axies stands for ehnancement number / release number
  • 72. Throughput ??
  • 73. Y axies stands for
  • 74. release item / release number
  • 75. User ?? , ???????
  • 76. ??Release Quality ??????
  • 77. • 2011 Q4 24? release ??, ? 12??(02:00 ~ 06:00) release
  • 78. • ???? user ?? manager ?????
  • 79. ??????????????? , ??????? release ???????, ????? 6???
  • 80. ?? release qulaity ???user ???????????
  • 81. • 2012 Q2 11? release ??, ? 0??(02:00 ~ 06:00) release
  • 82. • 2012 Q2? 1??07:30 release
  • 83. ????????
  • 84. Process
  • 85. Communication : No Surprise !
  • 86. • ???????????, ?????????????????????? briefing and retrospective , ??????? 18%
  • 87. • Communication ???? 9???
  • 88. ???????
  • 89. ?????????
  • 90. • ????
  • 91. • ???
  • 92. • double confirm ??????????
  • 93. ??
  • 94. • ????
  • 95. • ?? 30min ??, 30???????????? 20000 NTD
  • 96. ??????
  • 97. • ?????? sharing information ?????
  • 98. ????????
  • 99. ?????????
  • 100. • Information syncing : 40%
  • 101. • Discussion : 30%
  • 102. • decision making : 30%
  • 103. ?????
  • 104. • Information sync : 5% => ?????????
  • 105. • Discussion : 15%
  • 106. • decision making : 10%
  • 107. Convention over configuration
  • 108. Release??
  • 109. • Release ????? ???????????
  • 110. • ??????? release ??
  • 111. • ????????? release
  • 112. • Release ???????? release briefing
  • 113. • Release ??? Code Freeze
  • 114. • ???? cost , ?????? retrospective
  • 115. ??????????? sync ??
  • 116. • ???? Release Plan
  • 117. • ????????? Plan ??
  • 118. ◦ Item ????,???? , ????
  • 119. ◦ Item ?????? , ???? , ????
  • 120. ◦ ??????? item ????????????
  • 121. release briefing ?????
  • 122. • Information sync : 5% => information double confirm
  • 123. • Discussion : 15% => ???? sync , ??????????
  • 124. • decision making : 10% => ??sync??????????????
  • 125. ???????
  • 126. • ???????? ??? release????? ????email
  • 127. • ????????????????? , ????????????
  • 128. • ????? , ???? surprise , ???????
  • 129. Testing
  • 130. Release Manager ???
  • 131. • Unit Test : nodeunit , YETI
  • 132. • Automation : Selenium
  • 133. • CI ( Continuous Integration ) : Jenkins
  • 134. ????????????????
  • 135. Unit Test ?????
  • 136. ?? code ???? , 100% RD ???
  • 137. Automation
  • 138. • ??????? SLA (Service-level agreement)
  • 139. • ???? ?? automation coverage rate ??? SLA ? 100%
  • 140. • ?? release ?????? automation
  • 141. • Presentation layer ? coverage rate ???? 100% , ???????
  • 142. Continuous Integration
  • 143. • ??????? Unit Test and Automation , ???????
  • 144. • CI ????????????? Unit Test , Automation
  • 145. • ?????? CI ? auto build ??
  • 146. Deploy Tool
  • 147. ???SE/Ops ??? Deploy Tool
  • 148. ??: ???? Yinst/Igor
  • 149. Use os level package system
  • 150. • Debian/Ubuntu : dpkg
  • 151. • RedHat/Centos/Fedora : rpm
  • 152. • FreeBSD : port
  • 153. • Windows : MSI
  • 154. Dont use language level package system , ????????? relay on ???????? server
  • 155. • Webservr : apache, nginx
  • 156. • Cache : memcached , redis
  • 157. • SQL : MySQL , PosgreSQL , MongoDB
  • 158. ????? Deploy Script
  • 159. ?????????? script ? deploy code
  • 160. 1 apt-get install npm
  • 161. 2 apt-get install pip
  • 162. 3 apt-get install rubygem
  • 163. 4 apt-get install nginx #web server
  • 164. 5 apt-get install memcached #cache server
  • 165. 6 npm install -g express # Web framework
  • 166. 7 npm install -d
  • 167. 8 pip install pykml
  • 168. 9 gem install amqp # node.js has no rabbitmq client , so use ruby XD
  • 169. 10 apt-get install rabbitmq-server # rabbitmq server
  • 170. 11 wget http://mydev.server/mycode_package.tgz # get my code from dev server
  • 171. 12 tar zxcf mycode_package.tgz # unpack the code package
  • 172. • ???? hybrid ?? package system , package dependency ?????
  • 173. • SE/Ops team ?? language package system ??? , ???? RD ????
  • 174. ?? Deploy Script
  • 175. ????? package system , ?????? package version
  • 176. 1 apt-get install nginx=0.1.19-1
  • 177. 2 apt-get install memcached=1.4.13-0ubuntu2
  • 178. 3 apt-get install node-express=2.5.9-1
  • 179. 4 apt-get install pykml=0.0.3
  • 180. 5 apt-get install ruby-amqp=0.9.5-1.
  • 181. 6 apt-get install rabbitmq-server=2.8.4-1
  • 182. 7 apt-get install mycode_package=1.2.3 # ???? code ?? package
  • 183. ??
  • 184. • SE/Ops team ??????? deploy ???
  • 185. • CI server ????
  • 186. ???? pkg version
  • 187. Production System ? ???? pkg system ? depentency
  • 188. How to pack your package
  • 189. • ?? code ????? (Code,Config,template)
  • 190. • ??? code ??? prodcut ????? (??,??????,???,????)
  • 191. • ????????? (??lib ,??lib ,?? config , ??config )
  • 192. • ???????????? ( 100? , 10? , 1? , ???? , ???? )
  • 193. ???? Framework ???
  • 194. Change Management
  • 195. ??? code ???????
  • 196. • mycode-1.2.3.zip : ?????? change ???? ???? readme
  • 197. PKG list : ?? SE/Ops ?????
  • 198. • Upgrade : functionA-config-1.2.3-2012.07.08
  • 199. • Upgrade : Member-template-2.3.2-2012.07.11
  • 200. • Donwgrade : functionC-code-3.2.1-2012.03.05
  • 201. • Upgrade : ShoppingCart-lib-100W-1.2.3-2012.07.20
  • 202. • Upgrade : WS-stock-lib-200W-3.2.1-2012.07.21
  • 203. Package pack script ???
  • 204. Script best practice
  • 205. • ???
  • 206. • ? package ???? source code version ???
  • 207. • Version ??????
  • 208. • Package ????? package server
  • 209. • Readme ???????? svn log
  • 210. Package Version 2.3.1-2012.08.12
  • 211. [thegiive v123] add some slides change
  • 212. [RD2 v121] change function a
  • 213. Configuration Management
  • 214. debconf
  • 215. ???? config package
  • 216. • mypkg-config.2.2.3-2012.07.17
  • 217. • ?????? config ? ????? readme
  • 218. ?????? config ??
  • 219. 1 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 220. 2 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
  • 221. 3 echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 222. 4 debian_frontend=noninteractive dpkg-reconfigure
  • 223. package ??? postinst
  • 224. 1 db_get mypkg/use_memached
  • 225. 2 echo $RET #??? debconf ?????
  • 226. 3 do_config_change($RET)
  • 227. ??????SE/Ops?????
  • 228. Cache Server ???SE?????? disable cache
  • 229. 1 echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
  • 230. 2 debian_frontend=noninteractive dpkg-reconfigure
  • 231. WS1.coscup.org ??????SE???? WS2.coscup.org
  • 232. 1 echo "mypkg mypkg/ws_host string ws2.coscup.org"|debconf-set-selections
  • 233. 2 debian_frontend=noninteractive dpkg-reconfigure
  • 234. ???? title , SE ????? XD
  • 235. 1 echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
  • 236. 2 debian_frontend=noninteractive dpkg-reconfigure
  • 237. • ????RD ???SE?Daily work?????? RD , SE ??????
  • 238. • ?? incident ??? , SE ????config ???? , RD??? SE?????
  • 239. ?? mobile login server ??
  • 240. • ?? mobile loging server ?????????????????
  • 241. • ?????????? RD ???????RD,SE,??????
  • 242. • ???? config change , ????? mobile login ?? Web?? login ??
  • 243. Debconf ????...???
  • 244. ????????deploy script
  • 245. 1 # pkg
  • 246. 2 export debian_frontend=noninteractive
  • 247. 3 apt-get install -y nginx=0.1.19-1
  • 248. 4 apt-get install -y memcached=1.4.13-0ubuntu2
  • 249. 5 apt-get install -y node-express=2.5.9-1
  • 250. 6 apt-get install -y pykml=0.0.3
  • 251. 7 apt-get install -y ruby-amqp=0.9.5-1.
  • 252. 8 apt-get install -y rabbitmq-server=2.8.4-1
  • 253. 9 apt-get install -y mycode_package=1.2.3
  • 254. 10 # config
  • 255. 11 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 256. 12 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
  • 257. 13 echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 258. 14 dpkg-reconfigure
  • 259. • ??????? script ? deploy ?????????? , VM , Cloud instance
  • 260. • script ???? non-interactive mode ? disable ??????? step
  • 261. • ?? CI server ?? auto build , auto config
  • 262. deploy script ?? version
  • 263. control
  • 264. ?? tag ??? deploy , rollback
  • 265. 7??1? release ? tag ? v2012_07_01 , 7??2? release ? tag ? v2012_07_02
  • 266. 1 # ???1? release
  • 267. 2 git tag v2012_07_01
  • 268. 3 exec_deploy_script
  • 269. 4 # ???2? release
  • 270. 5 git tag v2012_07_02
  • 271. 6 exec_deploy_script
  • 272. ? v2012_07_02 release ??????? rollback ??????(v2012_07_01)
  • 273. 1 git checkout v2012_07_01
  • 274. 2 exec_deploy_script
  • 275. ?? git diff ?????? change
  • 276. ????7??2? release ?????
  • 277. 1 git diff v2012_07_01 v2012_07_02
  • 278. 2 #git diff output
  • 279. 3 .....
  • 280. 4 -apt-get install -y mycode_package=1.2.3
  • 281. 5 +apt-get install -y mycode_package=1.3.1
  • 282. 6 -echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 283. 7 +echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
  • 284. 8 .....
  • 285. 9 -echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 286. 10 +echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
  • 287. 11 .....
  • 288. • package ????? , config management ????change ???
  • 289. • ?? RD ??????? diff ???????????
  • 290. ?? git branch ?????
  • 291. • ?? git ??? branch ???????? deploy script
  • 292. • ???????? alpha , beta , staging , production ?? branch
  • 293. ??? alpha ???????? beta ??? git merge
  • 294. 1 git checkout beta
  • 295. 2 git merge alpha
  • 296. 3 ...solve confict...
  • 297. 4 git commit
  • 298. One More Thing
  • 299. Deploy System : Puppet
  • 300. • Open source deploy system written by Ruby
  • 301. • Client Server ????????? pure client
  • 302. • Config is language, built by Ruby DSL !!!
  • 303. • ??? template system ???? perfect configuration manage System
  • 304. • ???OS ? package system , ???????? OS ????
  • 305. • From Wikipedia ????: Google, Zynga, Sun, Wikipedia, Dell, Twitter...
  • 306. Puppet Node config
  • 307. In module
  • 308. 1 class ntpd {
  • 309. 2 package { "ntp": ensure => installed, }
  • 310. 3 file { "/etc/ntp.conf": owner => root, group => root,
  • 311. 4 require => package["ntp"] }
  • 312. 5 @service { "ntpd": ensure => running, enable => true,
  • 313. 6 subscribe => File["/etc/ntp.conf"] }
  • 314. 7 }
  • 315. 8 class ci{ package { "jenkins" : ensure => installed,} }
  • 316. 9 class Webservr{ package { "nginx" : ensure => installed,} }
  • 317. In main config
  • 318. 1 node basenode { include ntpd }
  • 319. 2 node ci.coscup.org inherits basenode {
  • 320. 3 include ci
  • 321. 4 }
  • 322. 5 node web.example.com inherits basenode {
  • 323. 6 include webserver
  • 324. 7 }
  • 325. Puppet Package config
  • 326. ?? Package ??
  • 327. 1 package {vim: ensure => installed }
  • 328. 2 package { "apache2": ensure => 2.2.22-1ubuntu1 }
  • 329. 3 package { mysql: ensure => installed,
  • 330. 4 provider => msi,
  • 331. 5 source => N:/packages/mysql-xxx.msi,
  • 332. 6 install_options => { INSTALLDIR => C:mysql-5.5 }, }
  • 333. ??????? Deploy Script
  • 334. • Debian / Ubuntu : Apt-get
  • 335. • RedHat / Fedora / CentOS : yum
  • 336. • FreeBSD : port
  • 337. • MacOS : macport
  • 338. • Windows : msi
  • 339. • Gentoo : portage
  • 340. Puppet Template System
  • 341. Configure Management ??? : ERB
  • 342. ??????? config file
  • 343. 1 $server_name = domain.coscup.org
  • 344. 2 $server_port = 80
  • 345. 3 file {"/usr/conf/nginx.conf": mode => "664",
  • 346. 4 content => template("nginx/config.erb"), }
  • 347. template
  • 348. 1 server {
  • 349. 2 listen <%= $server_port%>;
  • 350. 3 server_name <%= $server_name %>;
  • 351. 4 .....
  • 352. 5 }
  • 353. Pure ruby config menas
  • 354. 1 Given I run "ssh localhost -p 2222" interactively
  • 355. 2 And I type "apache2ctl configtest"
  • 356. 3 And the exit status should be 0
  • 357. • We can write unit test ... so we can TDD
  • 358. • We can write rspec and cucumber ... so we can BDD
  • 359. • We can commit our config in Hudson... so we can CI our config
  • 360. ??? Deploy Script ??
  • 361. 1 class web{ package { "nginx": ensure => 0.1.19-1 } }
  • 362. 2 class cache{ package { "memcached" : ensure => 1.4.13-0ubuntu2 }}
  • 363. 3 class kml_lib{ package { "pykml" : ensure =>0.0.3 }}
  • 364. 4 class message_quene{
  • 365. 5 package { "ruby-amqp" : ensure => 0.9.5-1 }
  • 366. 6 package { "rabbitmq-server": ensure => 2.8.4-1 }
  • 367. 7 }
  • 368. 8 class code_package{
  • 369. 9 include kml_lib
  • 370. 10 include message_quene
  • 371. 11 package { "node-express" : ensure => 2.5.9-1 }
  • 372. 12 package { "mycode" : ensure => 1.2.3 }
  • 373. 13 }
  • 374. 14 node "w1.coscup.com" {
  • 375. 15 include web
  • 376. 16 include cache
  • 377. 17 include code_package
  • 378. 18 }
  • 379. Q&A
  • 380. Puppet Config
  • 381. ?? Command
  • 382. 1 exec { remove file : command => rm /tmp/pup/etd , path => /bin }
  • 383. ?? Windows Command
  • 384. 1 exec { wincommand: command => C:Windows..powershell.exe -executionpolicy remotesigned -fi
  • 385. ?? file
  • 386. 1 file{/tmp/pup: ensure => directory}
  • 387. 2 file{ /tmp/pup/et: ensure => present, content => look at me,
  • 388. 3 require => File[/tmp/pup]}
  • 389. 4 file { "/etc/ntp.conf": owner => root, group => root,
  • 390. 5 mode => 0644, source => "puppet:///ntpd/ntp.conf",
  • 391. 6 require => package["ntp"]
  • 392. 7 }