Coscup
Upcoming SlideShare
Loading in...5
×
 

Coscup

on

  • 341 views

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

Statistics

Views

Total Views
341
Views on SlideShare
341
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Coscup Coscup Presentation Transcript

  • Better node.js software
  • release practice using open
  • source tool
  • ????????
  • Software release to production is such a hard practice, especially considering the business
  • impact and the quality of software . I will demonstrate how to use open source tools to
  • enhance a node.js web software release process and ensure the better release quality.
  • The open source tools I will mention will includes Jenkins, Git, Ubuntu dpkg system, node.js
  • language , ruby , rake, jshint , nodeunit, expresso , YETI, Selenium, and mongodb
  • ?, ??????? 100+ ???? = =
  • Web Software release best
  • practice
  • Agenda
  • Who am I ?
  • Release engineering in Y! Shopping
  • Process
  • Testing
  • Deploy Tools
  • Configuration Management
  • Who am I ?
  • thegiive
  • • A open source lover
  • • Release manager of Taiwan Yahoo! Shopping site
  • • Continuous study on software development, release engineering and software quality
  • • Fall in love with Ruby, VIM, Puppet and Rake
  • • Trying to do more work rather than write more blog
  • Release engineering in Y! Shopping
  • Common image about release
  • engineer
  • Yahoo! Shopping release manager
  • integration hub for more complex software development teams, sitting at the cross between
  • development, product management, quality assurance and other engineering efforts, also known
  • as DevOps
  • DevOps
  • Release???1 : ???
  • ??????????????
  • • RD Team ???? , PHP Team , .Net Team , Search Team
  • • User Team ???? , ?? Team , ?? Team , Search team
  • Release???1 : ???
  • ?????
  • Release???2 : bug????
  • From Book "Beautiful Testing" Chaptor 2
  • TestROI : ????????
  • • ACPB : Average Cost of a Production Bug
  • • ACTB : Average Cost of a Test Bug
  • • TBN : Total Bug Number
  • • TDBC : Total Defect Bug Cost
  • Taiwan Shopping ? ACPB?????? Test ROI ???
  • Test ROI ??????????? testing ????
  • Release???2 : bug????
  • Release???3 : User????
  • • ??RD?bug?QA????????SE??????RM?????
  • • User ???? monitor script , ????????????
  • ???? Release Manager , not release engineer
  • Release???4 : ??????
  • PHP Team
  • • Dev process : Scrum
  • • Source Code Management : Subversion
  • • Language : PHP and partial C++
  • • OS : Linux
  • .Net Team
  • • Dev process : Waterfall
  • • Source Code Management : VSS
  • • Language : ASP.Net and ASP
  • • OS : Windows
  • ????????????? ... ?????? 100??
  • release ?release 2000 ??
  • item
  • ??????
  • Release quality ??
  • Y axies stands for ehnancement number / release number
  • Throughput ??
  • Y axies stands for
  • release item / release number
  • User ?? , ???????
  • ??Release Quality ??????
  • • 2011 Q4 24? release ??, ? 12??(02:00 ~ 06:00) release
  • • ???? user ?? manager ?????
  • ??????????????? , ??????? release ???????, ????? 6???
  • ?? release qulaity ???user ???????????
  • • 2012 Q2 11? release ??, ? 0??(02:00 ~ 06:00) release
  • • 2012 Q2? 1??07:30 release
  • ????????
  • Process
  • Communication : No Surprise !
  • • ???????????, ?????????????????????? briefing and retrospective , ??????? 18%
  • • Communication ???? 9???
  • ???????
  • ?????????
  • • ????
  • • ???
  • • double confirm ??????????
  • ??
  • • ????
  • • ?? 30min ??, 30???????????? 20000 NTD
  • ??????
  • • ?????? sharing information ?????
  • ????????
  • ?????????
  • • Information syncing : 40%
  • • Discussion : 30%
  • • decision making : 30%
  • ?????
  • • Information sync : 5% => ?????????
  • • Discussion : 15%
  • • decision making : 10%
  • Convention over configuration
  • Release??
  • • Release ????? ???????????
  • • ??????? release ??
  • • ????????? release
  • • Release ???????? release briefing
  • • Release ??? Code Freeze
  • • ???? cost , ?????? retrospective
  • ??????????? sync ??
  • • ???? Release Plan
  • • ????????? Plan ??
  • ◦ Item ????,???? , ????
  • ◦ Item ?????? , ???? , ????
  • ◦ ??????? item ????????????
  • release briefing ?????
  • • Information sync : 5% => information double confirm
  • • Discussion : 15% => ???? sync , ??????????
  • • decision making : 10% => ??sync??????????????
  • ???????
  • • ???????? ??? release????? ????email
  • • ????????????????? , ????????????
  • • ????? , ???? surprise , ???????
  • Testing
  • Release Manager ???
  • • Unit Test : nodeunit , YETI
  • • Automation : Selenium
  • • CI ( Continuous Integration ) : Jenkins
  • ????????????????
  • Unit Test ?????
  • ?? code ???? , 100% RD ???
  • Automation
  • • ??????? SLA (Service-level agreement)
  • • ???? ?? automation coverage rate ??? SLA ? 100%
  • • ?? release ?????? automation
  • • Presentation layer ? coverage rate ???? 100% , ???????
  • Continuous Integration
  • • ??????? Unit Test and Automation , ???????
  • • CI ????????????? Unit Test , Automation
  • • ?????? CI ? auto build ??
  • Deploy Tool
  • ???SE/Ops ??? Deploy Tool
  • ??: ???? Yinst/Igor
  • Use os level package system
  • • Debian/Ubuntu : dpkg
  • • RedHat/Centos/Fedora : rpm
  • • FreeBSD : port
  • • Windows : MSI
  • Dont use language level package system , ????????? relay on ???????? server
  • • Webservr : apache, nginx
  • • Cache : memcached , redis
  • • SQL : MySQL , PosgreSQL , MongoDB
  • ????? Deploy Script
  • ?????????? script ? deploy code
  • 1 apt-get install npm
  • 2 apt-get install pip
  • 3 apt-get install rubygem
  • 4 apt-get install nginx #web server
  • 5 apt-get install memcached #cache server
  • 6 npm install -g express # Web framework
  • 7 npm install -d
  • 8 pip install pykml
  • 9 gem install amqp # node.js has no rabbitmq client , so use ruby XD
  • 10 apt-get install rabbitmq-server # rabbitmq server
  • 11 wget http://mydev.server/mycode_package.tgz # get my code from dev server
  • 12 tar zxcf mycode_package.tgz # unpack the code package
  • • ???? hybrid ?? package system , package dependency ?????
  • • SE/Ops team ?? language package system ??? , ???? RD ????
  • ?? Deploy Script
  • ????? package system , ?????? package version
  • 1 apt-get install nginx=0.1.19-1
  • 2 apt-get install memcached=1.4.13-0ubuntu2
  • 3 apt-get install node-express=2.5.9-1
  • 4 apt-get install pykml=0.0.3
  • 5 apt-get install ruby-amqp=0.9.5-1.
  • 6 apt-get install rabbitmq-server=2.8.4-1
  • 7 apt-get install mycode_package=1.2.3 # ???? code ?? package
  • ??
  • • SE/Ops team ??????? deploy ???
  • • CI server ????
  • ???? pkg version
  • Production System ? ???? pkg system ? depentency
  • How to pack your package
  • • ?? code ????? (Code,Config,template)
  • • ??? code ??? prodcut ????? (??,??????,???,????)
  • • ????????? (??lib ,??lib ,?? config , ??config )
  • • ???????????? ( 100? , 10? , 1? , ???? , ???? )
  • ???? Framework ???
  • Change Management
  • ??? code ???????
  • • mycode-1.2.3.zip : ?????? change ???? ???? readme
  • PKG list : ?? SE/Ops ?????
  • • Upgrade : functionA-config-1.2.3-2012.07.08
  • • Upgrade : Member-template-2.3.2-2012.07.11
  • • Donwgrade : functionC-code-3.2.1-2012.03.05
  • • Upgrade : ShoppingCart-lib-100W-1.2.3-2012.07.20
  • • Upgrade : WS-stock-lib-200W-3.2.1-2012.07.21
  • Package pack script ???
  • Script best practice
  • • ???
  • • ? package ???? source code version ???
  • • Version ??????
  • • Package ????? package server
  • • Readme ???????? svn log
  • Package Version 2.3.1-2012.08.12
  • [thegiive v123] add some slides change
  • [RD2 v121] change function a
  • Configuration Management
  • debconf
  • ???? config package
  • • mypkg-config.2.2.3-2012.07.17
  • • ?????? config ? ????? readme
  • ?????? config ??
  • 1 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 2 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
  • 3 echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 4 debian_frontend=noninteractive dpkg-reconfigure
  • package ??? postinst
  • 1 db_get mypkg/use_memached
  • 2 echo $RET #??? debconf ?????
  • 3 do_config_change($RET)
  • ??????SE/Ops?????
  • Cache Server ???SE?????? disable cache
  • 1 echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
  • 2 debian_frontend=noninteractive dpkg-reconfigure
  • WS1.coscup.org ??????SE???? WS2.coscup.org
  • 1 echo "mypkg mypkg/ws_host string ws2.coscup.org"|debconf-set-selections
  • 2 debian_frontend=noninteractive dpkg-reconfigure
  • ???? title , SE ????? XD
  • 1 echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
  • 2 debian_frontend=noninteractive dpkg-reconfigure
  • • ????RD ???SE?Daily work?????? RD , SE ??????
  • • ?? incident ??? , SE ????config ???? , RD??? SE?????
  • ?? mobile login server ??
  • • ?? mobile loging server ?????????????????
  • • ?????????? RD ???????RD,SE,??????
  • • ???? config change , ????? mobile login ?? Web?? login ??
  • Debconf ????...???
  • ????????deploy script
  • 1 # pkg
  • 2 export debian_frontend=noninteractive
  • 3 apt-get install -y nginx=0.1.19-1
  • 4 apt-get install -y memcached=1.4.13-0ubuntu2
  • 5 apt-get install -y node-express=2.5.9-1
  • 6 apt-get install -y pykml=0.0.3
  • 7 apt-get install -y ruby-amqp=0.9.5-1.
  • 8 apt-get install -y rabbitmq-server=2.8.4-1
  • 9 apt-get install -y mycode_package=1.2.3
  • 10 # config
  • 11 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 12 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
  • 13 echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 14 dpkg-reconfigure
  • • ??????? script ? deploy ?????????? , VM , Cloud instance
  • • script ???? non-interactive mode ? disable ??????? step
  • • ?? CI server ?? auto build , auto config
  • deploy script ?? version
  • control
  • ?? tag ??? deploy , rollback
  • 7??1? release ? tag ? v2012_07_01 , 7??2? release ? tag ? v2012_07_02
  • 1 # ???1? release
  • 2 git tag v2012_07_01
  • 3 exec_deploy_script
  • 4 # ???2? release
  • 5 git tag v2012_07_02
  • 6 exec_deploy_script
  • ? v2012_07_02 release ??????? rollback ??????(v2012_07_01)
  • 1 git checkout v2012_07_01
  • 2 exec_deploy_script
  • ?? git diff ?????? change
  • ????7??2? release ?????
  • 1 git diff v2012_07_01 v2012_07_02
  • 2 #git diff output
  • 3 .....
  • 4 -apt-get install -y mycode_package=1.2.3
  • 5 +apt-get install -y mycode_package=1.3.1
  • 6 -echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
  • 7 +echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
  • 8 .....
  • 9 -echo "mypkg mypkg/title string coscup"|debconf-set-selections
  • 10 +echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
  • 11 .....
  • • package ????? , config management ????change ???
  • • ?? RD ??????? diff ???????????
  • ?? git branch ?????
  • • ?? git ??? branch ???????? deploy script
  • • ???????? alpha , beta , staging , production ?? branch
  • ??? alpha ???????? beta ??? git merge
  • 1 git checkout beta
  • 2 git merge alpha
  • 3 ...solve confict...
  • 4 git commit
  • One More Thing
  • Deploy System : Puppet
  • • Open source deploy system written by Ruby
  • • Client Server ????????? pure client
  • • Config is language, built by Ruby DSL !!!
  • • ??? template system ???? perfect configuration manage System
  • • ???OS ? package system , ???????? OS ????
  • • From Wikipedia ????: Google, Zynga, Sun, Wikipedia, Dell, Twitter...
  • Puppet Node config
  • In module
  • 1 class ntpd {
  • 2 package { "ntp": ensure => installed, }
  • 3 file { "/etc/ntp.conf": owner => root, group => root,
  • 4 require => package["ntp"] }
  • 5 @service { "ntpd": ensure => running, enable => true,
  • 6 subscribe => File["/etc/ntp.conf"] }
  • 7 }
  • 8 class ci{ package { "jenkins" : ensure => installed,} }
  • 9 class Webservr{ package { "nginx" : ensure => installed,} }
  • In main config
  • 1 node basenode { include ntpd }
  • 2 node ci.coscup.org inherits basenode {
  • 3 include ci
  • 4 }
  • 5 node web.example.com inherits basenode {
  • 6 include webserver
  • 7 }
  • Puppet Package config
  • ?? Package ??
  • 1 package {vim: ensure => installed }
  • 2 package { "apache2": ensure => 2.2.22-1ubuntu1 }
  • 3 package { mysql: ensure => installed,
  • 4 provider => msi,
  • 5 source => N:/packages/mysql-xxx.msi,
  • 6 install_options => { INSTALLDIR => C:mysql-5.5 }, }
  • ??????? Deploy Script
  • • Debian / Ubuntu : Apt-get
  • • RedHat / Fedora / CentOS : yum
  • • FreeBSD : port
  • • MacOS : macport
  • • Windows : msi
  • • Gentoo : portage
  • Puppet Template System
  • Configure Management ??? : ERB
  • ??????? config file
  • 1 $server_name = domain.coscup.org
  • 2 $server_port = 80
  • 3 file {"/usr/conf/nginx.conf": mode => "664",
  • 4 content => template("nginx/config.erb"), }
  • template
  • 1 server {
  • 2 listen <%= $server_port%>;
  • 3 server_name <%= $server_name %>;
  • 4 .....
  • 5 }
  • Pure ruby config menas
  • 1 Given I run "ssh localhost -p 2222" interactively
  • 2 And I type "apache2ctl configtest"
  • 3 And the exit status should be 0
  • • We can write unit test ... so we can TDD
  • • We can write rspec and cucumber ... so we can BDD
  • • We can commit our config in Hudson... so we can CI our config
  • ??? Deploy Script ??
  • 1 class web{ package { "nginx": ensure => 0.1.19-1 } }
  • 2 class cache{ package { "memcached" : ensure => 1.4.13-0ubuntu2 }}
  • 3 class kml_lib{ package { "pykml" : ensure =>0.0.3 }}
  • 4 class message_quene{
  • 5 package { "ruby-amqp" : ensure => 0.9.5-1 }
  • 6 package { "rabbitmq-server": ensure => 2.8.4-1 }
  • 7 }
  • 8 class code_package{
  • 9 include kml_lib
  • 10 include message_quene
  • 11 package { "node-express" : ensure => 2.5.9-1 }
  • 12 package { "mycode" : ensure => 1.2.3 }
  • 13 }
  • 14 node "w1.coscup.com" {
  • 15 include web
  • 16 include cache
  • 17 include code_package
  • 18 }
  • Q&A
  • Puppet Config
  • ?? Command
  • 1 exec { remove file : command => rm /tmp/pup/etd , path => /bin }
  • ?? Windows Command
  • 1 exec { wincommand: command => C:Windows..powershell.exe -executionpolicy remotesigned -fi
  • ?? file
  • 1 file{/tmp/pup: ensure => directory}
  • 2 file{ /tmp/pup/et: ensure => present, content => look at me,
  • 3 require => File[/tmp/pup]}
  • 4 file { "/etc/ntp.conf": owner => root, group => root,
  • 5 mode => 0644, source => "puppet:///ntpd/ntp.conf",
  • 6 require => package["ntp"]
  • 7 }