Coscup
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Coscup

on

  • 391 views

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

Statistics

Views

Total Views
391
Views on SlideShare
391
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Coscup Presentation Transcript

  • 1. Better node.js software
  • 2. release practice using open
  • 3. source tool
  • 4. ????????
  • 5. Software release to production is such a hard practice, especially considering the business
  • 6. impact and the quality of software . I will demonstrate how to use open source tools to
  • 7. enhance a node.js web software release process and ensure the better release quality.
  • 8. The open source tools I will mention will includes Jenkins, Git, Ubuntu dpkg system, node.js
  • 9. language , ruby , rake, jshint , nodeunit, expresso , YETI, Selenium, and mongodb
  • 10. ?, ??????? 100+ ???? = =
  • 11. Web Software release best
  • 12. practice
  • 13. Agenda
  • 14. Who am I ?
  • 15. Release engineering in Y! Shopping
  • 16. Process
  • 17. Testing
  • 18. Deploy Tools
  • 19. Configuration Management
  • 20. Who am I ?
  • 21. thegiive
  • 22. • A open source lover
  • 23. • Release manager of Taiwan Yahoo! Shopping site
  • 24. • Continuous study on software development, release engineering and software quality
  • 25. • Fall in love with Ruby, VIM, Puppet and Rake
  • 26. • Trying to do more work rather than write more blog
  • 27. Release engineering in Y! Shopping
  • 28. Common image about release
  • 29. engineer
  • 30. Yahoo! Shopping release manager
  • 31. integration hub for more complex software development teams, sitting at the cross between
  • 32. development, product management, quality assurance and other engineering efforts, also known
  • 33. as DevOps
  • 34. DevOps
  • 35. Release???1 : ???
  • 36. ??????????????
  • 37. • RD Team ???? , PHP Team , .Net Team , Search Team
  • 38. • User Team ???? , ?? Team , ?? Team , Search team
  • 39. Release???1 : ???
  • 40. ?????
  • 41. Release???2 : bug????
  • 42. From Book "Beautiful Testing" Chaptor 2
  • 43. TestROI : ????????
  • 44. • ACPB : Average Cost of a Production Bug
  • 45. • ACTB : Average Cost of a Test Bug
  • 46. • TBN : Total Bug Number
  • 47. • TDBC : Total Defect Bug Cost
  • 48. Taiwan Shopping ? ACPB?????? Test ROI ???
  • 49. Test ROI ??????????? testing ????
  • 50. Release???2 : bug????
  • 51. Release???3 : User????
  • 52. • ??RD?bug?QA????????SE??????RM?????
  • 53. • User ???? monitor script , ????????????
  • 54. ???? Release Manager , not release engineer
  • 55. Release???4 : ??????
  • 56. PHP Team
  • 57. • Dev process : Scrum
  • 58. • Source Code Management : Subversion
  • 59. • Language : PHP and partial C++
  • 60. • OS : Linux
  • 61. .Net Team
  • 62. • Dev process : Waterfall
  • 63. • Source Code Management : VSS
  • 64. • Language : ASP.Net and ASP
  • 65. • OS : Windows
  • 66. ????????????? ... ?????? 100??
  • 67. release ?release 2000 ??
  • 68. item
  • 69. ??????
  • 70. Release quality ??
  • 71. Y axies stands for ehnancement number / release number
  • 72. Throughput ??
  • 73. Y axies stands for
  • 74. release item / release number
  • 75. User ?? , ???????
  • 76. ??Release Quality ??????
  • 77. • 2011 Q4 24? release ??, ? 12??(02:00 ~ 06:00) release
  • 78. • ???? user ?? manager ?????
  • 79. ??????????????? , ??????? release ???????, ????? 6???
  • 80. ?? release qulaity ???user ???????????
  • 81. • 2012 Q2 11? release ??, ? 0??(02:00 ~ 06:00) release
  • 82. • 2012 Q2? 1??07:30 release
  • 83. ????????
  • 84. Process
  • 85. Communication : No Surprise !
  • 86. • ???????????, ?????????????????????? briefing and retrospective , ??????? 18%
  • 87. • Communication ???? 9???
  • 88. ???????
  • 89. ?????????
  • 90. • ????
  • 91. • ???
  • 92. • double confirm ??????????
  • 93. ??
  • 94. • ????
  • 95. • ?? 30min ??, 30???????????? 20000 NTD
  • 96. ??????
  • 97. • ?????? sharing information ?????
  • 98. ????????
  • 99. ?????????
  • 100. • Information syncing : 40%
  • 101. • Discussion : 30%
  • 102. • decision making : 30%
  • 103. ?????
  • 104. • Information sync : 5% => ?????????
  • 105. • Discussion : 15%
  • 106. • decision making : 10%
  • 107. Convention over configuration
  • 108. Release??
  • 109. • Release ????? ???????????
  • 110. • ??????? release ??
  • 111. • ????????? release
  • 112. • Release ???????? release briefing
  • 113. • Release ??? Code Freeze
  • 114. • ???? cost , ?????? retrospective
  • 115. ??????????? sync ??
  • 116. • ???? Release Plan
  • 117. • ????????? Plan ??
  • 118. ◦ Item ????,???? , ????
  • 119. ◦ Item ?????? , ???? , ????
  • 120. ◦ ??????? item ????????????
  • 121. release briefing ?????
  • 122. • Information sync : 5% => information double confirm
  • 123. • Discussion : 15% => ???? sync , ??????????
  • 124. • decision making : 10% => ??sync??????????????
  • 125. ???????
  • 126. • ???????? ??? release????? ????email
  • 127. • ????????????????? , ????????????
  • 128. • ????? , ???? surprise , ???????
  • 129. Testing
  • 130. Release Manager ???
  • 131. • Unit Test : nodeunit , YETI
  • 132. • Automation : Selenium
  • 133. • CI ( Continuous Integration ) : Jenkins
  • 134. ????????????????
  • 135. Unit Test ?????
  • 136. ?? code ???? , 100% RD ???
  • 137. Automation
  • 138. • ??????? SLA (Service-level agreement)
  • 139. • ???? ?? automation coverage rate ??? SLA ? 100%
  • 140. • ?? release ?????? automation
  • 141. • Presentation layer ? coverage rate ???? 100% , ???????
  • 142. Continuous Integration
  • 143. • ??????? Unit Test and Automation , ???????
  • 144. • CI ????????????? Unit Test , Automation
  • 145. • ?????? CI ? auto build ??
  • 146. Deploy Tool
  • 147. ???SE/Ops ??? Deploy Tool
  • 148. ??: ???? Yinst/Igor
  • 149. Use os level package system
  • 150. • Debian/Ubuntu : dpkg
  • 151. • RedHat/Centos/Fedora : rpm
  • 152. • FreeBSD : port
  • 153. • Windows : MSI
  • 154. Dont use language level package system , ????????? relay on ???????? server
  • 155. • Webservr : apache, nginx
  • 156. • Cache : memcached , redis
  • 157. • SQL : MySQL , PosgreSQL , MongoDB
  • 158. ????? Deploy Script
  • 159. ?????????? script ? deploy code
  • 160. 1 apt-get install npm
  • 161. 2 apt-get install pip
  • 162. 3 apt-get install rubygem
  • 163. 4 apt-get install nginx #web server
  • 164. 5 apt-get install memcached #cache server
  • 165. 6 npm install -g express # Web framework
  • 166. 7 npm install -d
  • 167. 8 pip install pykml
  • 168. 9 gem install amqp # node.js has no rabbitmq client , so use ruby XD
  • 169. 10 apt-get install rabbitmq-server # rabbitmq server
  • 170. 11 wget http://mydev.server/mycode_package.tgz # get my code from dev server
  • 171. 12 tar zxcf mycode_package.tgz # unpack the code package
  • 172. • ???? hybrid ?? package system , package dependency ?????
  • 173. • SE/Ops team ?? language package system ??? , ???? RD ????
  • 174. ?? Deploy Script
  • 175. ????? package system , ?????? package version
  • 176. 1 apt-get install nginx=0.1.19-1
  • 177. 2 apt-get install memcached=1.4.13-0ubuntu2
  • 178. 3 apt-get install node-express=2.5.9-1
  • 179. 4 apt-get install pykml=0.0.3
  • 180. 5 apt-get install ruby-amqp=0.9.5-1.
  • 181. 6 apt-get install rabbitmq-server=2.8.4-1
  • 182. 7 apt-get install mycode_package=1.2.3 # ???? code ?? package
  • 183. ??
  • 184. • SE/Ops team ??????? deploy ???
  • 185. • CI server ????
  • 186. ???? pkg version
  • 187. Production System ? ???? pkg system ? depentency
  • 188. How to pack your package
  • 189. • ?? code ????? (Code,Config,template)
  • 190. • ??? code ??? prodcut ????? (??,??????,???,????)
  • 191. • ????????? (??lib ,??lib ,?? config , ??config )
  • 192. • ???????????? ( 100? , 10? , 1? , ???? , ???? )
  • 193. ???? Framework ???
  • 194. Change Management
  • 195. ??? code ???????
  • 196. • mycode-1.2.3.zip : ?????? change ???? ???? readme
  • 197. PKG list : ?? SE/Ops ?????
  • 198. • Upgrade : functionA-config-1.2.3-2012.07.08
  • 199. • Upgrade : Member-template-2.3.2-2012.07.11
  • 200. • Donwgrade : functionC-code-3.2.1-2012.03.05
  • 201. • Upgrade : ShoppingCart-lib-100W-1.2.3-2012.07.20
  • 202. • Upgrade : WS-stock-lib-200W-3.2.1-2012.07.21
  • 203. Package pack script ???
  • 204. Script best practice
  • 205. • ???
  • 206. • ? package ???? source code version ???
  • 207. • Version ??????
  • 208. • Package ????? package server
  • 209. • Readme ???????? svn log
  • 210. Package Version 2.3.1-2012.08.12
  • 211. [thegiive v123] add some slides change
  • 212. [RD2 v121] change function a
  • 213. Configuration Management
  • 214. debconf
  • 215. ???? config package
  • 216. • mypkg-config.2.2.3-2012.07.17
  • 217. • ?????? config ? ????? readme
  • 218. ?????? config ??
  • 219. 1 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 220. 2 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
  • 221. 3 echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 222. 4 debian_frontend=noninteractive dpkg-reconfigure
  • 223. package ??? postinst
  • 224. 1 db_get mypkg/use_memached
  • 225. 2 echo $RET #??? debconf ?????
  • 226. 3 do_config_change($RET)
  • 227. ??????SE/Ops?????
  • 228. Cache Server ???SE?????? disable cache
  • 229. 1 echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
  • 230. 2 debian_frontend=noninteractive dpkg-reconfigure
  • 231. WS1.coscup.org ??????SE???? WS2.coscup.org
  • 232. 1 echo "mypkg mypkg/ws_host string ws2.coscup.org"|debconf-set-selections
  • 233. 2 debian_frontend=noninteractive dpkg-reconfigure
  • 234. ???? title , SE ????? XD
  • 235. 1 echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
  • 236. 2 debian_frontend=noninteractive dpkg-reconfigure
  • 237. • ????RD ???SE?Daily work?????? RD , SE ??????
  • 238. • ?? incident ??? , SE ????config ???? , RD??? SE?????
  • 239. ?? mobile login server ??
  • 240. • ?? mobile loging server ?????????????????
  • 241. • ?????????? RD ???????RD,SE,??????
  • 242. • ???? config change , ????? mobile login ?? Web?? login ??
  • 243. Debconf ????...???
  • 244. ????????deploy script
  • 245. 1 # pkg
  • 246. 2 export debian_frontend=noninteractive
  • 247. 3 apt-get install -y nginx=0.1.19-1
  • 248. 4 apt-get install -y memcached=1.4.13-0ubuntu2
  • 249. 5 apt-get install -y node-express=2.5.9-1
  • 250. 6 apt-get install -y pykml=0.0.3
  • 251. 7 apt-get install -y ruby-amqp=0.9.5-1.
  • 252. 8 apt-get install -y rabbitmq-server=2.8.4-1
  • 253. 9 apt-get install -y mycode_package=1.2.3
  • 254. 10 # config
  • 255. 11 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 256. 12 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
  • 257. 13 echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 258. 14 dpkg-reconfigure
  • 259. • ??????? script ? deploy ?????????? , VM , Cloud instance
  • 260. • script ???? non-interactive mode ? disable ??????? step
  • 261. • ?? CI server ?? auto build , auto config
  • 262. deploy script ?? version
  • 263. control
  • 264. ?? tag ??? deploy , rollback
  • 265. 7??1? release ? tag ? v2012_07_01 , 7??2? release ? tag ? v2012_07_02
  • 266. 1 # ???1? release
  • 267. 2 git tag v2012_07_01
  • 268. 3 exec_deploy_script
  • 269. 4 # ???2? release
  • 270. 5 git tag v2012_07_02
  • 271. 6 exec_deploy_script
  • 272. ? v2012_07_02 release ??????? rollback ??????(v2012_07_01)
  • 273. 1 git checkout v2012_07_01
  • 274. 2 exec_deploy_script
  • 275. ?? git diff ?????? change
  • 276. ????7??2? release ?????
  • 277. 1 git diff v2012_07_01 v2012_07_02
  • 278. 2 #git diff output
  • 279. 3 .....
  • 280. 4 -apt-get install -y mycode_package=1.2.3
  • 281. 5 +apt-get install -y mycode_package=1.3.1
  • 282. 6 -echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 283. 7 +echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
  • 284. 8 .....
  • 285. 9 -echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 286. 10 +echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
  • 287. 11 .....
  • 288. • package ????? , config management ????change ???
  • 289. • ?? RD ??????? diff ???????????
  • 290. ?? git branch ?????
  • 291. • ?? git ??? branch ???????? deploy script
  • 292. • ???????? alpha , beta , staging , production ?? branch
  • 293. ??? alpha ???????? beta ??? git merge
  • 294. 1 git checkout beta
  • 295. 2 git merge alpha
  • 296. 3 ...solve confict...
  • 297. 4 git commit
  • 298. One More Thing
  • 299. Deploy System : Puppet
  • 300. • Open source deploy system written by Ruby
  • 301. • Client Server ????????? pure client
  • 302. • Config is language, built by Ruby DSL !!!
  • 303. • ??? template system ???? perfect configuration manage System
  • 304. • ???OS ? package system , ???????? OS ????
  • 305. • From Wikipedia ????: Google, Zynga, Sun, Wikipedia, Dell, Twitter...
  • 306. Puppet Node config
  • 307. In module
  • 308. 1 class ntpd {
  • 309. 2 package { "ntp": ensure => installed, }
  • 310. 3 file { "/etc/ntp.conf": owner => root, group => root,
  • 311. 4 require => package["ntp"] }
  • 312. 5 @service { "ntpd": ensure => running, enable => true,
  • 313. 6 subscribe => File["/etc/ntp.conf"] }
  • 314. 7 }
  • 315. 8 class ci{ package { "jenkins" : ensure => installed,} }
  • 316. 9 class Webservr{ package { "nginx" : ensure => installed,} }
  • 317. In main config
  • 318. 1 node basenode { include ntpd }
  • 319. 2 node ci.coscup.org inherits basenode {
  • 320. 3 include ci
  • 321. 4 }
  • 322. 5 node web.example.com inherits basenode {
  • 323. 6 include webserver
  • 324. 7 }
  • 325. Puppet Package config
  • 326. ?? Package ??
  • 327. 1 package {vim: ensure => installed }
  • 328. 2 package { "apache2": ensure => 2.2.22-1ubuntu1 }
  • 329. 3 package { mysql: ensure => installed,
  • 330. 4 provider => msi,
  • 331. 5 source => N:/packages/mysql-xxx.msi,
  • 332. 6 install_options => { INSTALLDIR => C:mysql-5.5 }, }
  • 333. ??????? Deploy Script
  • 334. • Debian / Ubuntu : Apt-get
  • 335. • RedHat / Fedora / CentOS : yum
  • 336. • FreeBSD : port
  • 337. • MacOS : macport
  • 338. • Windows : msi
  • 339. • Gentoo : portage
  • 340. Puppet Template System
  • 341. Configure Management ??? : ERB
  • 342. ??????? config file
  • 343. 1 $server_name = domain.coscup.org
  • 344. 2 $server_port = 80
  • 345. 3 file {"/usr/conf/nginx.conf": mode => "664",
  • 346. 4 content => template("nginx/config.erb"), }
  • 347. template
  • 348. 1 server {
  • 349. 2 listen <%= $server_port%>;
  • 350. 3 server_name <%= $server_name %>;
  • 351. 4 .....
  • 352. 5 }
  • 353. Pure ruby config menas
  • 354. 1 Given I run "ssh localhost -p 2222" interactively
  • 355. 2 And I type "apache2ctl configtest"
  • 356. 3 And the exit status should be 0
  • 357. • We can write unit test ... so we can TDD
  • 358. • We can write rspec and cucumber ... so we can BDD
  • 359. • We can commit our config in Hudson... so we can CI our config
  • 360. ??? Deploy Script ??
  • 361. 1 class web{ package { "nginx": ensure => 0.1.19-1 } }
  • 362. 2 class cache{ package { "memcached" : ensure => 1.4.13-0ubuntu2 }}
  • 363. 3 class kml_lib{ package { "pykml" : ensure =>0.0.3 }}
  • 364. 4 class message_quene{
  • 365. 5 package { "ruby-amqp" : ensure => 0.9.5-1 }
  • 366. 6 package { "rabbitmq-server": ensure => 2.8.4-1 }
  • 367. 7 }
  • 368. 8 class code_package{
  • 369. 9 include kml_lib
  • 370. 10 include message_quene
  • 371. 11 package { "node-express" : ensure => 2.5.9-1 }
  • 372. 12 package { "mycode" : ensure => 1.2.3 }
  • 373. 13 }
  • 374. 14 node "w1.coscup.com" {
  • 375. 15 include web
  • 376. 16 include cache
  • 377. 17 include code_package
  • 378. 18 }
  • 379. Q&A
  • 380. Puppet Config
  • 381. ?? Command
  • 382. 1 exec { remove file : command => rm /tmp/pup/etd , path => /bin }
  • 383. ?? Windows Command
  • 384. 1 exec { wincommand: command => C:Windows..powershell.exe -executionpolicy remotesigned -fi
  • 385. ?? file
  • 386. 1 file{/tmp/pup: ensure => directory}
  • 387. 2 file{ /tmp/pup/et: ensure => present, content => look at me,
  • 388. 3 require => File[/tmp/pup]}
  • 389. 4 file { "/etc/ntp.conf": owner => root, group => root,
  • 390. 5 mode => 0644, source => "puppet:///ntpd/ntp.conf",
  • 391. 6 require => package["ntp"]
  • 392. 7 }