Your SlideShare is downloading. ×
Development Setup of B-Translator
Development Setup of B-Translator
Development Setup of B-Translator
Development Setup of B-Translator
Development Setup of B-Translator
Development Setup of B-Translator
Development Setup of B-Translator
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Development Setup of B-Translator

112

Published on

B-Translator helps to get feedback about l10n (translations of the programs). It tries to collect very small translation contributions from a wide crowd of people and to dilute them into something …

B-Translator helps to get feedback about l10n (translations of the programs). It tries to collect very small translation contributions from a wide crowd of people and to dilute them into something useful. It is developed as a Drupal7 profile and the code is hosted on GitHub. Here I describe the development setup and process that I use for this project. Most of the the tips are project specific, however some of them can be used on any Drupal project.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
112
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Development Setup of B­Translator Author: Dashamir Hoxha <dashohoxha@gmail.com> Date: 2013-07-08 17:54:13 CEST HTML generated by org-mode 6.33x in emacs 23 B-Translator helps to get feedback about l10n (translations of the programs). It tries to collect very small translation contributions from a wide crowd of people and to dilute them into something useful. It is developed as a Drupal7 profile and the code is hosted on GitHub. Here I describe the development setup and process that I use for this project. Most of the the tips are project specific, however some of them can be used on any Drupal project. Table Of Contents • Installation • Changing the webserver • Re-installing the application • Making a clone for development • Making a backup of the application • Accessing the code of the application from outside chroot • Pushing commits to github • The commit workflow • Making a local git clone of the dev application • Working with a dev-test-live workflow Installation Installation for development is the same as the installation for a production server, inside a chroot. The steps are like this: mkdir /var/chroot cd /var/chroot/ git clone https://github.com/dashohoxha/B-Translator.git nohup nice B-Translator/install/install.sh btr & tail -f nohup.out chroot btr/ /tmp/install/config.sh chroot btr/ rm -rf /tmp/install reboot ## it is advisable to reboot the host after this installation Changing The Webserver The webserver that is used for production is NGINX because it is more responsive in high load, has build in cache, etc. The configuration of NGINX is also tweaked for production (has DoS protection, short timeout, etc.) For development Apache2 can be more suitable. It can be started and stopped like this: chroot /var/chroot/btr/
  • 2. /var/www/btranslator/profiles/btranslator/dev/apache2.sh start /var/www/btranslator/profiles/btranslator/dev/apache2.sh stop Important: When apache2 is started, the services nginx, memcached and php5-fpm are stopped. Don't forget to modify the init script /etc/init.d/chroot-btr like this: #CHROOT_SERVICES="cron php5-fpm memcached mysql nginx" CHROOT_SERVICES="cron mysql apache2" The script apache2.sh cannot make this modification automatically because it runs inside the chroot, and the init script is outside the chroot (on the host system). So it has to be done manually. I also add something like this on /etc/hosts, so that I can access it on the browser with a domain name: 127.0.1.1 l10n.org.xx dev.l10n.org.xx This modification has to be done manually as well. Re­Installing The Application It can be done with the script dev/reinstall.sh: chroot /var/chroot/btr/ /var/www/btranslator/profiles/btranslator/dev/reinstall.sh It will rebuild the Drupal directory with drush make and install the btranslator profile with drush site-install, and then do all the rest of configurations just like they are done during installation. Normally there is no need to reinstall the application, unless we want to test the installation profile and the installation scripts. Another kind of re-installation, which touches only the database of Drupal (btranslator) and nothing else, can be done with the script dev/reinstall-db.sh: chroot /var/chroot/btr/ /var/www/btranslator/profiles/btranslator/dev/reinstall-db.sh It is useful for testing the features. Usually, when features are un-installed, things are not undone properly. In order to leave out a feature, it should not be installed since the beginning. So, it is important to test different combinations of them to see which one works better for us. Making A Clone For Development Inside the chroot I make a clone for development like this: chroot /var/chroot/btr/ cd /var/www/btranslator/profiles/btranslator/ dev/clone.sh dev It creates a new application with root /var/www/btranslator_dev/ and with DB named
  • 3. btranslator_dev. It also creates the drush alias @dev, and modifies the configuration of the webserver so that the cloned application can be accessed at dev.l10n.org.xx. Caution: The root directory and the DB of the clone will be erased, if they exist. Other clones like this can be created for testing etc. To cleanup (remove/erase) a clone, we can use clone_rm.sh like this: chroot /var/chroot/btr/ cd /var/www/btranslator/profiles/btranslator/ dev/clone_rm.sh dev By the way, we can also modify a little bit the configuration of a development copy of the application (in order to help us not confuse a development copy with a live or testing one), with the script dev/config.php: chroot /var/chroot/btr/ cd /var/www/ drush php-script btranslator/profiles/btranslator/config.php dev1 drush @dev php-script btranslator/profiles/btranslator/config.php dev2 It will set site_name to 'B-Translator (dev1)', will make site email something like 'user+dev1@gmail.com', will enable email re-routing, display the devel menu on the footer region, etc. It is not required, but sometimes may be useful. Making A Backup Of The Application Sometimes, when testing things on Drupal (installing/uninstalling modules etc.) things get messy and it is not possible anymore to revert to the state that you were before starting the test. In this case the only way to get safely to a previous stable state is by restoring a backup (or installing from the scratch and repeating all the configurations). A snapshot of the application is just like a full backup with a time stamp. It saves the state of the application at a certain time, both the code (the whole Drupal directory) and the database. It can be done like this: chroot /var/chroot/btr/ cd /var/www/ ln -s btranslator_dev/profiles/btranslator/ B-Translator B-Translator/dev/snapshot.sh make B-Translator/dev/snapshot.sh make @dev These will create the files snapshot-btranslator-20130602.tgz and snapshot- btranslator_dev-20130602.tgz. They can be restored like this: B-Translator/dev/snapshot.sh restore --file=snapshot-btranslator- 20130602.tgz B-Translator/dev/snapshot.sh restore --file=snapshot-btranslator_dev- 20130602.tgz B-Translator/dev/snapshot.sh restore @dev --file=snapshot-btranslator- 20130602.tgz B-Translator/dev/snapshot.sh restore @dev --file=snapshot-btranslator_dev- 20130602.tgz
  • 4. As you may notice, a snapshot of @dev can also be restored on the main application, and the other way around. However, in many cases a backup/restore of the database is all that is needed, and it is more efficient. It can be done withdrush sql-dump and drush sql-query like this: drush sql-dump > btranslator.sql drush sql-dump @dev > btranslator_dev.sql drush sql-query --file=$(pwd)/btranslator.sql drush sql-query --file=$(pwd)/btranslator_dev.sql drush @dev sql-query --file=$(pwd)/btranslator.sql drush @dev sql-query --file=$(pwd)/btranslator_dev.sql Accessing The Code Of The Application From Outside Chroot In order to access easily the code of the application from outside chroot, I create a symbolic link like this: cd /var/chroot/ ln -s btr/var/www/btranslator_dev/profiles/btranslator/ dev Now I can go to /var/chroot/dev/ and start emacs or any other tools. This way I don't have to install emacs or any other development tools inside the chroot and can use the best of development tools that my host environment can offer me. Pushing Commits To Github The copy of the application on /var/www/btranslator_dev/profiles/btranslator/ (as well as the one on/var/www/btranslator/profiles/btranslator/) are actually clones of the git repository at https://github.com/dashohoxha/B-Translator so we can pull from it and push to it. Pulling (to get up-to-date) can be done by everybody, however pushing requires a username and password (the ones that are used to access the account at GitHub). The Commit Workflow For small or straight-forward changes I can also work directly on the master branch, then commit, and then push to github. However I usually use a bit more complicated workflow. First I create and checkout a dev branch. When the work is done I merge this branch to master and then delete it. Finally push the commit(s) to github. git checkout -d dev ### create a branch and switch to it [work-commit-work-comit] git checkout master ### switch back to master git pull ### get any latest commits from github git merge dev [--squash] git push ### send commits to github git branch -D dev ### erase the branch Usually there are no commits comming from github, since I am the only developper (unless I
  • 5. have worked and commited from some other location). So, when I merge without –squash this usually results in fast-forward merge, which means that all the commits that I have done on the branch dev are automatically transferred to the branch master. However sometimes there may be dirty commits on the dev branch, which means that there may be incomplete commits, or commits that reverse what was done on the previous commits etc. When I wish to reorganise commits and make them cleaner, I use the –squash option, which collects all the changes on the dev branch and leaves them on the master sandbox as local modifications (uncommitted). Then I can redo the commits on a cleaner or more logical way. Afterwards the dev branch will be deleted and the old commits will be lost. Making A Local Git Clone Of The Dev Application Sometimes it is not easy or suitable to test modifications on the @dev application (/var/www/btranslator_dev). For example this is the case when I have to test install/uninstall, enable/disable modules, features, etc. In this case I test them on the main application instance (/var/www/btranslator). Since both applications are clones of the GitHub repository, it is easy to push commits from btranslator_dev to github and to pull them from github to btranslator. However, sometimes it is better to test modifications and make sure that they work, before pushing them to github. This can be done if the code on btranslator is a git clone of the code on btranslator_dev (instead of being a clone from github). The script dev/git-clone-dev.sh makes just this. It replaces the code of btranslator with a git clone of the dev branchfrom btranslator_dev. Then the workflow is like this: 1. Work and commit on the branch dev of btranslator_dev 2. Pull on btranslator and test. 3. Repeat steps 1 and 2 until the modification that we are making is OK. 4. Push changes upwards to github, like this: cd /var/www/btranslator_dev/profiles/btranslator git checkout master git merge dev [--squash] git push git branch -D dev git checkout -b dev So, after merging to master and pushing to github, we delete the branch dev and create a new one. 5. Make a git pull on btranslator and sync it with btranslator_dev cd /var/www/btranslator/profiles/btranslator git pull Working With A Dev­Test­Live Workflow All the work that is described on the sections above is about development and local testing that is done on a working copy (sandbox) of a chroot installation. This is usually installed on my personal machine (that I use for development).
  • 6. At some point, all the modifications have to be transferred to a public server, where the application is in "production", performing "live". On that public server there is the same chroot environment as in the development server. The synchronisation of the application can be done via git push and pull. However drush rsync and drush sql-sync offer another option for synchronisation. For more details see: drush help rsync drush help sql-sync drush topic docs-aliases These commands use drush aliases, which allow also remote execution of drush commands. On my development environment I have created the file /etc/drush/remote.aliases.drushrc.php, which has a content like this: <?php $aliases['live'] = array ( 'root' => '/var/www/btranslator', 'uri' => 'http://l10n.org.al', 'remote-host' => 'l10n.org.al', 'remote-user' => 'root', 'ssh-options' => '-p 2201 -i /root/.ssh/id_rsa', 'path-aliases' => array ( '%profile' => 'profiles/btranslator', '%data' => '/var/www/btranslator_data', '%pofiles' => '/var/www/PO_files', '%exports' => '/var/www/exports', '%downloads' => '/var/www/downloads', ), 'command-specific' => array ( 'sql-sync' => array ( 'simulate' => '1', ), 'rsync' => array ( 'simulate' => '1', ), ), ); $aliases['test'] = array ( 'parent' => '@live', 'root' => '/var/www/btranslator', 'uri' => 'http://www2.l10n.org.al', 'remote-host' => 'www2.l10n.org.al', 'command-specific' => array ( 'sql-sync' => array ( 'simulate' => '0', ),
  • 7. 'rsync' => array ( 'simulate' => '0', ), ), ); It defines the aliases live and test. The test/stage application is almost identical to the live/production one, however it is not for public use. The idea is to test there first any updates/upgrades of the application, in order to make sure that they don't break any things, before applying them to the real live application. In my case it is placed on a different server, however it can also be placed on the same server as the live application (just make a clone of the main application with dev/clone.sh test). When everything is set up correctly, the synchronisation can be done as simply as this: drush rsync @live @test drush sql-sync @live @test drush rsync @live @dev drush sql-sync @live @dev Note: Synchronising this way from @test to @live or from @dev to @live, usually is a HUGE mistake, but the simulate option on the config file will make sure that it fails. For drush commands to work remotely, ssh daemon has to be running on the remote server, inside the chroot environment. By default it is not installed, but it can be installed with the script dev/install-sshd.sh. This script will also take care to change the ssh port to 2201, in order to avoid any conflicts with any existing daemon on the host environment, and also for increased security. For remote access to work correctly, the public/private key ssh access should be set up and configured as well. For more detailed instructions on how to do it see: http://dashohoxha.blogspot.com/2012/08/how-to-secure-ubuntu-server.html

×