• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Coscup
 

Coscup

on

  • 323 views

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

License by http://creativecommons.org/licenses/by-nc-nd/2.5/tw/

Statistics

Views

Total Views
323
Views on SlideShare
323
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Coscup Coscup Presentation Transcript

    • Better node.js software
    • release practice using open
    • source tool
    • ????????
    • Software release to production is such a hard practice, especially considering the business
    • impact and the quality of software . I will demonstrate how to use open source tools to
    • enhance a node.js web software release process and ensure the better release quality.
    • The open source tools I will mention will includes Jenkins, Git, Ubuntu dpkg system, node.js
    • language , ruby , rake, jshint , nodeunit, expresso , YETI, Selenium, and mongodb
    • ?, ??????? 100+ ???? = =
    • Web Software release best
    • practice
    • Agenda
    • Who am I ?
    • Release engineering in Y! Shopping
    • Process
    • Testing
    • Deploy Tools
    • Configuration Management
    • Who am I ?
    • thegiive
    • • A open source lover
    • • Release manager of Taiwan Yahoo! Shopping site
    • • Continuous study on software development, release engineering and software quality
    • • Fall in love with Ruby, VIM, Puppet and Rake
    • • Trying to do more work rather than write more blog
    • Release engineering in Y! Shopping
    • Common image about release
    • engineer
    • Yahoo! Shopping release manager
    • integration hub for more complex software development teams, sitting at the cross between
    • development, product management, quality assurance and other engineering efforts, also known
    • as DevOps
    • DevOps
    • Release???1 : ???
    • ??????????????
    • • RD Team ???? , PHP Team , .Net Team , Search Team
    • • User Team ???? , ?? Team , ?? Team , Search team
    • Release???1 : ???
    • ?????
    • Release???2 : bug????
    • From Book "Beautiful Testing" Chaptor 2
    • TestROI : ????????
    • • ACPB : Average Cost of a Production Bug
    • • ACTB : Average Cost of a Test Bug
    • • TBN : Total Bug Number
    • • TDBC : Total Defect Bug Cost
    • Taiwan Shopping ? ACPB?????? Test ROI ???
    • Test ROI ??????????? testing ????
    • Release???2 : bug????
    • Release???3 : User????
    • • ??RD?bug?QA????????SE??????RM?????
    • • User ???? monitor script , ????????????
    • ???? Release Manager , not release engineer
    • Release???4 : ??????
    • PHP Team
    • • Dev process : Scrum
    • • Source Code Management : Subversion
    • • Language : PHP and partial C++
    • • OS : Linux
    • .Net Team
    • • Dev process : Waterfall
    • • Source Code Management : VSS
    • • Language : ASP.Net and ASP
    • • OS : Windows
    • ????????????? ... ?????? 100??
    • release ?release 2000 ??
    • item
    • ??????
    • Release quality ??
    • Y axies stands for ehnancement number / release number
    • Throughput ??
    • Y axies stands for
    • release item / release number
    • User ?? , ???????
    • ??Release Quality ??????
    • • 2011 Q4 24? release ??, ? 12??(02:00 ~ 06:00) release
    • • ???? user ?? manager ?????
    • ??????????????? , ??????? release ???????, ????? 6???
    • ?? release qulaity ???user ???????????
    • • 2012 Q2 11? release ??, ? 0??(02:00 ~ 06:00) release
    • • 2012 Q2? 1??07:30 release
    • ????????
    • Process
    • Communication : No Surprise !
    • • ???????????, ?????????????????????? briefing and retrospective , ??????? 18%
    • • Communication ???? 9???
    • ???????
    • ?????????
    • • ????
    • • ???
    • • double confirm ??????????
    • ??
    • • ????
    • • ?? 30min ??, 30???????????? 20000 NTD
    • ??????
    • • ?????? sharing information ?????
    • ????????
    • ?????????
    • • Information syncing : 40%
    • • Discussion : 30%
    • • decision making : 30%
    • ?????
    • • Information sync : 5% => ?????????
    • • Discussion : 15%
    • • decision making : 10%
    • Convention over configuration
    • Release??
    • • Release ????? ???????????
    • • ??????? release ??
    • • ????????? release
    • • Release ???????? release briefing
    • • Release ??? Code Freeze
    • • ???? cost , ?????? retrospective
    • ??????????? sync ??
    • • ???? Release Plan
    • • ????????? Plan ??
    • ◦ Item ????,???? , ????
    • ◦ Item ?????? , ???? , ????
    • ◦ ??????? item ????????????
    • release briefing ?????
    • • Information sync : 5% => information double confirm
    • • Discussion : 15% => ???? sync , ??????????
    • • decision making : 10% => ??sync??????????????
    • ???????
    • • ???????? ??? release????? ????email
    • • ????????????????? , ????????????
    • • ????? , ???? surprise , ???????
    • Testing
    • Release Manager ???
    • • Unit Test : nodeunit , YETI
    • • Automation : Selenium
    • • CI ( Continuous Integration ) : Jenkins
    • ????????????????
    • Unit Test ?????
    • ?? code ???? , 100% RD ???
    • Automation
    • • ??????? SLA (Service-level agreement)
    • • ???? ?? automation coverage rate ??? SLA ? 100%
    • • ?? release ?????? automation
    • • Presentation layer ? coverage rate ???? 100% , ???????
    • Continuous Integration
    • • ??????? Unit Test and Automation , ???????
    • • CI ????????????? Unit Test , Automation
    • • ?????? CI ? auto build ??
    • Deploy Tool
    • ???SE/Ops ??? Deploy Tool
    • ??: ???? Yinst/Igor
    • Use os level package system
    • • Debian/Ubuntu : dpkg
    • • RedHat/Centos/Fedora : rpm
    • • FreeBSD : port
    • • Windows : MSI
    • Dont use language level package system , ????????? relay on ???????? server
    • • Webservr : apache, nginx
    • • Cache : memcached , redis
    • • SQL : MySQL , PosgreSQL , MongoDB
    • ????? Deploy Script
    • ?????????? script ? deploy code
    • 1 apt-get install npm
    • 2 apt-get install pip
    • 3 apt-get install rubygem
    • 4 apt-get install nginx #web server
    • 5 apt-get install memcached #cache server
    • 6 npm install -g express # Web framework
    • 7 npm install -d
    • 8 pip install pykml
    • 9 gem install amqp # node.js has no rabbitmq client , so use ruby XD
    • 10 apt-get install rabbitmq-server # rabbitmq server
    • 11 wget http://mydev.server/mycode_package.tgz # get my code from dev server
    • 12 tar zxcf mycode_package.tgz # unpack the code package
    • • ???? hybrid ?? package system , package dependency ?????
    • • SE/Ops team ?? language package system ??? , ???? RD ????
    • ?? Deploy Script
    • ????? package system , ?????? package version
    • 1 apt-get install nginx=0.1.19-1
    • 2 apt-get install memcached=1.4.13-0ubuntu2
    • 3 apt-get install node-express=2.5.9-1
    • 4 apt-get install pykml=0.0.3
    • 5 apt-get install ruby-amqp=0.9.5-1.
    • 6 apt-get install rabbitmq-server=2.8.4-1
    • 7 apt-get install mycode_package=1.2.3 # ???? code ?? package
    • ??
    • • SE/Ops team ??????? deploy ???
    • • CI server ????
    • ???? pkg version
    • Production System ? ???? pkg system ? depentency
    • How to pack your package
    • • ?? code ????? (Code,Config,template)
    • • ??? code ??? prodcut ????? (??,??????,???,????)
    • • ????????? (??lib ,??lib ,?? config , ??config )
    • • ???????????? ( 100? , 10? , 1? , ???? , ???? )
    • ???? Framework ???
    • Change Management
    • ??? code ???????
    • • mycode-1.2.3.zip : ?????? change ???? ???? readme
    • PKG list : ?? SE/Ops ?????
    • • Upgrade : functionA-config-1.2.3-2012.07.08
    • • Upgrade : Member-template-2.3.2-2012.07.11
    • • Donwgrade : functionC-code-3.2.1-2012.03.05
    • • Upgrade : ShoppingCart-lib-100W-1.2.3-2012.07.20
    • • Upgrade : WS-stock-lib-200W-3.2.1-2012.07.21
    • Package pack script ???
    • Script best practice
    • • ???
    • • ? package ???? source code version ???
    • • Version ??????
    • • Package ????? package server
    • • Readme ???????? svn log
    • Package Version 2.3.1-2012.08.12
    • [thegiive v123] add some slides change
    • [RD2 v121] change function a
    • Configuration Management
    • debconf
    • ???? config package
    • • mypkg-config.2.2.3-2012.07.17
    • • ?????? config ? ????? readme
    • ?????? config ??
    • 1 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
    • 2 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
    • 3 echo "mypkg mypkg/title string coscup"|debconf-set-selections
    • 4 debian_frontend=noninteractive dpkg-reconfigure
    • package ??? postinst
    • 1 db_get mypkg/use_memached
    • 2 echo $RET #??? debconf ?????
    • 3 do_config_change($RET)
    • ??????SE/Ops?????
    • Cache Server ???SE?????? disable cache
    • 1 echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
    • 2 debian_frontend=noninteractive dpkg-reconfigure
    • WS1.coscup.org ??????SE???? WS2.coscup.org
    • 1 echo "mypkg mypkg/ws_host string ws2.coscup.org"|debconf-set-selections
    • 2 debian_frontend=noninteractive dpkg-reconfigure
    • ???? title , SE ????? XD
    • 1 echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
    • 2 debian_frontend=noninteractive dpkg-reconfigure
    • • ????RD ???SE?Daily work?????? RD , SE ??????
    • • ?? incident ??? , SE ????config ???? , RD??? SE?????
    • ?? mobile login server ??
    • • ?? mobile loging server ?????????????????
    • • ?????????? RD ???????RD,SE,??????
    • • ???? config change , ????? mobile login ?? Web?? login ??
    • Debconf ????...???
    • ????????deploy script
    • 1 # pkg
    • 2 export debian_frontend=noninteractive
    • 3 apt-get install -y nginx=0.1.19-1
    • 4 apt-get install -y memcached=1.4.13-0ubuntu2
    • 5 apt-get install -y node-express=2.5.9-1
    • 6 apt-get install -y pykml=0.0.3
    • 7 apt-get install -y ruby-amqp=0.9.5-1.
    • 8 apt-get install -y rabbitmq-server=2.8.4-1
    • 9 apt-get install -y mycode_package=1.2.3
    • 10 # config
    • 11 echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
    • 12 echo "mypkg mypkg/ws_host string ws1.coscup.org"|debconf-set-selections
    • 13 echo "mypkg mypkg/title string coscup"|debconf-set-selections
    • 14 dpkg-reconfigure
    • • ??????? script ? deploy ?????????? , VM , Cloud instance
    • • script ???? non-interactive mode ? disable ??????? step
    • • ?? CI server ?? auto build , auto config
    • deploy script ?? version
    • control
    • ?? tag ??? deploy , rollback
    • 7??1? release ? tag ? v2012_07_01 , 7??2? release ? tag ? v2012_07_02
    • 1 # ???1? release
    • 2 git tag v2012_07_01
    • 3 exec_deploy_script
    • 4 # ???2? release
    • 5 git tag v2012_07_02
    • 6 exec_deploy_script
    • ? v2012_07_02 release ??????? rollback ??????(v2012_07_01)
    • 1 git checkout v2012_07_01
    • 2 exec_deploy_script
    • ?? git diff ?????? change
    • ????7??2? release ?????
    • 1 git diff v2012_07_01 v2012_07_02
    • 2 #git diff output
    • 3 .....
    • 4 -apt-get install -y mycode_package=1.2.3
    • 5 +apt-get install -y mycode_package=1.3.1
    • 6 -echo "mypkg mypkg/use_memached boolean true"|debconf-set-selections
    • 7 +echo "mypkg mypkg/use_memached boolean false"|debconf-set-selections
    • 8 .....
    • 9 -echo "mypkg mypkg/title string coscup"|debconf-set-selections
    • 10 +echo "mypkg mypkg/title string coscup.org"|debconf-set-selections
    • 11 .....
    • • package ????? , config management ????change ???
    • • ?? RD ??????? diff ???????????
    • ?? git branch ?????
    • • ?? git ??? branch ???????? deploy script
    • • ???????? alpha , beta , staging , production ?? branch
    • ??? alpha ???????? beta ??? git merge
    • 1 git checkout beta
    • 2 git merge alpha
    • 3 ...solve confict...
    • 4 git commit
    • One More Thing
    • Deploy System : Puppet
    • • Open source deploy system written by Ruby
    • • Client Server ????????? pure client
    • • Config is language, built by Ruby DSL !!!
    • • ??? template system ???? perfect configuration manage System
    • • ???OS ? package system , ???????? OS ????
    • • From Wikipedia ????: Google, Zynga, Sun, Wikipedia, Dell, Twitter...
    • Puppet Node config
    • In module
    • 1 class ntpd {
    • 2 package { "ntp": ensure => installed, }
    • 3 file { "/etc/ntp.conf": owner => root, group => root,
    • 4 require => package["ntp"] }
    • 5 @service { "ntpd": ensure => running, enable => true,
    • 6 subscribe => File["/etc/ntp.conf"] }
    • 7 }
    • 8 class ci{ package { "jenkins" : ensure => installed,} }
    • 9 class Webservr{ package { "nginx" : ensure => installed,} }
    • In main config
    • 1 node basenode { include ntpd }
    • 2 node ci.coscup.org inherits basenode {
    • 3 include ci
    • 4 }
    • 5 node web.example.com inherits basenode {
    • 6 include webserver
    • 7 }
    • Puppet Package config
    • ?? Package ??
    • 1 package {vim: ensure => installed }
    • 2 package { "apache2": ensure => 2.2.22-1ubuntu1 }
    • 3 package { mysql: ensure => installed,
    • 4 provider => msi,
    • 5 source => N:/packages/mysql-xxx.msi,
    • 6 install_options => { INSTALLDIR => C:mysql-5.5 }, }
    • ??????? Deploy Script
    • • Debian / Ubuntu : Apt-get
    • • RedHat / Fedora / CentOS : yum
    • • FreeBSD : port
    • • MacOS : macport
    • • Windows : msi
    • • Gentoo : portage
    • Puppet Template System
    • Configure Management ??? : ERB
    • ??????? config file
    • 1 $server_name = domain.coscup.org
    • 2 $server_port = 80
    • 3 file {"/usr/conf/nginx.conf": mode => "664",
    • 4 content => template("nginx/config.erb"), }
    • template
    • 1 server {
    • 2 listen <%= $server_port%>;
    • 3 server_name <%= $server_name %>;
    • 4 .....
    • 5 }
    • Pure ruby config menas
    • 1 Given I run "ssh localhost -p 2222" interactively
    • 2 And I type "apache2ctl configtest"
    • 3 And the exit status should be 0
    • • We can write unit test ... so we can TDD
    • • We can write rspec and cucumber ... so we can BDD
    • • We can commit our config in Hudson... so we can CI our config
    • ??? Deploy Script ??
    • 1 class web{ package { "nginx": ensure => 0.1.19-1 } }
    • 2 class cache{ package { "memcached" : ensure => 1.4.13-0ubuntu2 }}
    • 3 class kml_lib{ package { "pykml" : ensure =>0.0.3 }}
    • 4 class message_quene{
    • 5 package { "ruby-amqp" : ensure => 0.9.5-1 }
    • 6 package { "rabbitmq-server": ensure => 2.8.4-1 }
    • 7 }
    • 8 class code_package{
    • 9 include kml_lib
    • 10 include message_quene
    • 11 package { "node-express" : ensure => 2.5.9-1 }
    • 12 package { "mycode" : ensure => 1.2.3 }
    • 13 }
    • 14 node "w1.coscup.com" {
    • 15 include web
    • 16 include cache
    • 17 include code_package
    • 18 }
    • Q&A
    • Puppet Config
    • ?? Command
    • 1 exec { remove file : command => rm /tmp/pup/etd , path => /bin }
    • ?? Windows Command
    • 1 exec { wincommand: command => C:Windows..powershell.exe -executionpolicy remotesigned -fi
    • ?? file
    • 1 file{/tmp/pup: ensure => directory}
    • 2 file{ /tmp/pup/et: ensure => present, content => look at me,
    • 3 require => File[/tmp/pup]}
    • 4 file { "/etc/ntp.conf": owner => root, group => root,
    • 5 mode => 0644, source => "puppet:///ntpd/ntp.conf",
    • 6 require => package["ntp"]
    • 7 }