The document discusses various shell and command line tools to improve productivity for developers, including: - Bash basics like keyboard navigation, piping, I/O redirection - Tools like GNU Screen for multitasking and Vim for text editing - Phing for project automation and deployment - Benchmarking tools like Apache Bench and Siege It provides examples of using these tools and resources for further learning shell scripting and command line skills. The overall agenda is around becoming a more efficient "Bash Ninja" through mastering the shell environment.

1. Becoming A Bash Ninja New York PHP March 24, 2009 Brian Dailey [email_address] http://realm3.com/

2. Becoming A Bash Ninja Agenda Bash Basics Keyboard Navigation Shell Shortcuts Piping Shell Tools GNU Screen vim phing Benchmarking (ab/siege)

3. About Me 1998 – Introduced to MS-DOS 5.0 Avid reader of ”help” command. Windows 95 ”Required”!?! Why? The command line is perfect! Only switched to GUI reluctantly. 2001-2004: The Dark Depths of Java First introduction to vi in 2001 Redhat 7.2, Knoppix, DamnSmallLinux 2004 – PHP 2006 – Ubuntu 6.06 Arrives, immediately embraced.

4. Why Learn To Use The Shell?

5. Advantages You're probably already using it. You'll be able to work from nearly any *nix based computer (including servers). Huge set of tools available to make your life easier. It is concise and expressive. Increased productivity.* *Productivity can be arbitrarily defined as ”gettin' 'er done, quickly.”

6. Disadvantages It can be intimidating. Steep learning curve.

7. Development Productivity Figure out what you do often. Automate it. Determine which tasks take the most of your time. Can you optimize use of this time? (Yes, that's a simplification.)

8. Productivity Waste!

9. What's Bash? Command line environment Scripting environment right at your fingertips All the magic happens right here Fine print: Debian/Ubuntu shell actually uses ”dash”, not bash.

10. Bash Doesn't mean you have to run Linux locally! Set up a development server and use PuTTY. Install Cygwin inside Windows. Or use ports of GNU Utilities: http://unxutils.sourceforge.net/ Use Mac Terminal. Use a Live CD.

11. Shell Basics: Keyboard Navigation Getting from one point to another can be difficult at first. Example: mysql> SELECT user.id, user.username, user.email, user.address, user.city, user.state, user.name, userfile.file_name, userfile.metadata, userfile.other_stuff FOM user JOIN userfiles ON user.id = userfile.user_id AND user.id BETWEEN 1 AND 100 AND user.created BETWEEN '2008-10-10' AND '2009-10-10' AND user.is_active = 1 AND user.name LIKE 'Frodo%' LIMIT 10;

12. Shell Basics: Keyboard Navigation ↑↓ - Scroll through previous commands Ctrl+U - Clear all before cursor Ctrl+K - Clear all after cursor Ctrl+A - <Home> Ctrl+E - <End> Alt+B - Move cursor back one ”word” (Similar to Ctrl+ ← ) Alt+F - Move cursor forward one ”word” (Ctrl+ -> ) Alt+Bckspc - Delete previous ”word” (Also, Ctrl+W) Alt+. - Recall last command argument Tab - Autocomplete commands, files, folders, and more. Ctrl+L - Clear the screen Ctrl+_ - Undo CTRL+R - Recall previous commands Esc, T - Swap last two words These shortcuts are nearly universal (bash, MySQL) due to use of the GNU Readline library. Exhaustive list is available here: http://onlyubuntu.blogspot.com/2007/03/bash-shell-keyboard-shortcuts-for-linux.html

13. Shell Basics: Keyboard Navigation Get used to using Tab for auto-completion! $ svn ci -m”Fixed issue.” fil[tab] ↓ $ svn ci -m”Fixed issue.” filename.php [Tab][Tab] shows all available matches: $ svn ci -m”Fixed issue.” f[tab][tab] filename.php frodo_baggins.php friendly_fellow.php Enable programmable bash completion for further efficiency. $ svn c[tab][tab] cat checkout cl co copy changelist ci cleanup commit cp

14. Shell Basics: Argument Shortcuts !! - Run the last command (same as ↑ +return) !my - Runs last cmd starting with ”my”, e.g., ”mysql -uroot -p db” !my:p - Find last cmd starting with ”my” and print it out (don't execute) vi !$ - Replaces !$ with last argument in previous command vi !* - Insert all arguments from previous command A very useful (and more comprehensive) list is available here: http://mail.linux.ie/pipermail/ilug/2006-May/087799.html

15. Shell Basics: I/O Redirect Concept can be thought of as ”pipes” and ”streams” Negates the need for an application to write to a file, only for another application to read it. Image Source: Wikipedia

16. Shell Basics: I/O Redirect Basics Basic example of standard output: # format an XML file and feed it out to formatted.xml $ xmllint feed.xml --format > formatted.xml # same as above, but append to existing file. $ xmllint feed.xml --format >> formatted.xml Standard input: $ mysql -uuser -p db1 < schema.sql Or both: $ mysql -uuser -p db1 < schema.sql > log.txt Can also redirect stderr (errors). Use the pipe! $ history | grep svn

17. Shell Basics: Piping Examples of pipe chaining... Colorize Subversion diff output: $ svn diff | colordiff | less -R Search files for child classes, do not include test files, sort them alphabetically, and paginate. $ grep -r -n -H '^class.*extends.*{$' * | grep -v test | sort | more Cat a text file containing a list of files and execute git add on each one. $ cat files.txt | xargs git add

18. Shell Basics: Pipe Filters less – Interactive text pager with search capability. sort – Sort lines (alphabetically, numerically, etc) grep – Regular expression searches uniq – Detect (report or omit) repeated lines sed – Stream EDitor awk – powerful pattern scanning and text processing language xargs – build and execute commands from input There are a lot more of these tools!

19. Shell Tricks: Pipe Use sed (Stream EDitor) to filter text from git, and automatically add all untracked files. $ git status | \ # display text from ”Untracked” to end of output sed -n '/Untracked/,/$d/p' | \ # only display lines starting with ”#” followed by a tab sed -n '/#\t/p' | \ # Strip out beginning ”#” and whitespace sed 's/#\s*//' | \ # Add the files to the git repo xargs git add But how are you going to remember this command?

20. Shell Basics: alias Allows you to map a command to a more complicated statement. Examples: # List files in friendly format alias l='ls -l' # Colorize svn diff, use less -R to accept colors alias sdv='svn diff | colordiff | less -R' # Do not allow updates or deletes without WHERE clause. alias mysql='mysql --i-am-a-dummy' Arguments are appended to the command. Save to /etc/profile (or .bash_profile, .bashrc) to automatically load in new sessions.

21. Shell Basics: alias Limitations # Usage: fun [name] alias fun='echo $1 is having fun!' $ fun Brian is having fun! Brian Alias is only useful as the beginning of a command. Arguments don't work well, can't do loops, etc. Answer? Functions and/or shell scripts!

22. Shell Tricks: Functions Look up PHP command arguments: phpargs() { # $1 is the first argument passed # get php page from manual mirror curl -s http://us3.php.net/$1 | \ # pull all text in target div sed -n '/<div class=&quot;methodsynopsis dc-description&quot;>/,/<\/div>/p' | \ # strip out HTML tags sed 's/<[^>]*>//g' | \ # strip out line feed tr -d &quot;\n&quot; # add ending line echo } Then from the command line: $ phpargs in_array bool in_array ( mixed $needle, array $haystack [, bool $strict ] ) The only way to learn scripts is to write them!

23. Shell Trick: Finding Files Use ”find” for quick, local searches (includes all subdirectories): # Get rid of all those Windows executables! $ find /var/www -name '*.exe' -exec rm {} \; # Find all files modified in the past 24 hours. $ find /var/www -mtime -1 Use locate (or slocate) to search filesystem using a file index: # run updatedb nightly to update a file index # search for filename.php, filter out svn matches. $ locate filename.php | grep -v svn

24. bash: Resources Comprehensive Introduction to the shell http://www.linuxcommand.org/learning_the_shell.php Bash keyboard shortcuts http://onlyubuntu.blogspot.com/2007/03/bash-shell-keyboard-shortcuts-for-linux.html Bash arguments http://www.deadman.org/bash.html Bash programmable completion: http://www.debian-administration.org/articles/316 http://freshmeat.net/projects/bashcompletion/ IBM Bash By Tutorial Series http://www.ibm.com/developerworks/library/l-bash.html http://www.ibm.com/developerworks/library/l-bash2.html http://www.ibm.com/developerworks/library/l-bash3.html

25. Next Up: Shell Tools Editing text files Multitasking Automated deployment, testing, etc. Benchmarking

26. Shell Tools: GNU Screen

27. Shell Tools: GNU Screen What does it do? Command-prompt multi-tasking (multiplex) Detachable workstation If you're accidentally disconnected, easily reconnect without losing anything. Start new screen ”php”: screen -S php Reconnect later: screen -r php Customize your ”status” line so you'll always know what server you're on. Configuration in ~/.screenrc Share screen with another user. Training sessions Code reviews

28. Shell Tools: GNU Screen Screen allows you to run multiple shells in one terminal. Command key followed by command (usually Ctrl+A, command) 0/9 – Switches between windows n – Switches to the next available window p – Switches to the previous available A – Changes window session name K – Kills a window session c – Creates a new window [ – Then use arrows to scroll up and down terminal, select and copy text. d – Detach window M – Monitor window for activity ” – List current windows and names ? – Help

29. GNU screen: Resources GNU Screen http://magazine.redhat.com/2007/09/27/a-guide-to-gnu-screen/ Screen Configuration Examples http://softpanorama.org/Utilities/Screen/screenrc_examples.shtml

30. Shell Tools: Vim Most of the usual IDE tools are here: Syntax highlighting Code Completion Error Highlighting Debugging and Profiling (with xdebug plugin) Integration with versioning (CVS, SVN, git) Automatic creation of phpDoc ( /*** )

31. vim: Syntax Highlighting

32. vim: Code Completion

33. Vim: Debugging

34. vim Tricks Abbreviations :abbr teh the :abbr vd var_dump :abbr fn function(

35. vim: What Makes It Different? . (Repeat last command) Build Powerful Macros Command line still accessible Highly customizable Some demonstrations follow...

36. vim: Resources Andrei Zmievski's excellent vim+PHP guide: http://gravitonic.com/talks/ Include's Andrei's .vim files, plugins, etc. Getting Started with vim and PHP: http://realm3.com/articles/getting_started_with_vim_and_php ” Why, oh WHY, do those #?@! nutheads use vi?” http://www.viemu.com/a-why-vi-vim.html A Slightly Advanced Introduction to Vim http://linuxgazette.net/152/srinivasan.html Integrating xdebug http://tech.blog.box.net/2007/06/20/how-to-debug-php-with-vim-and-xdebug-on-linux/ vim Shortcuts Cheatsheet http://rayninfo.co.uk/vimtips.html

37. phing: Project Build Tool Styled after Java's ”ant” XML configuration file Cross-platform Automated > Manual Configuration Deployment Testing

38. phing: Example Generate documentation: <?xml version=&quot;1.0&quot;?> <project name=&quot;myproject.com&quot; default=&quot;main&quot; basedir=&quot;.&quot;> <target name=&quot;phpdoc&quot;> <phpdoc title=&quot;Project API Documentation&quot; output=&quot;HTML:Smarty:PHP&quot; sourcecode=&quot;no&quot; destdir=&quot;${docs}&quot; quiet=&quot;true&quot;> <fileset dir=&quot;.&quot;> <include name=&quot;**/*.php&quot; /> </fileset> </phpdoc> </target> </project>

39. phing: Example Run PHP_Codesniffer: <?xml version=&quot;1.0&quot;?> <project name=&quot;myproject.com&quot; default=&quot;main&quot; basedir=&quot;.&quot;> <target name=&quot;sniffcode&quot;> <phpcodesniffer standard=&quot;Zend&quot; file=&quot;app_controller.php&quot; allowedFileExtensions=&quot;php php5 ctp&quot;> <fileset dir=&quot;.&quot;> <include name=&quot;app_controller.php&quot; /> </fileset> </phpcodesniffer> </target> </project>

40. phing: Example Deploy via SSH: <?xml version=&quot;1.0&quot;?> <project name=&quot;realm3.com&quot; default=&quot;main&quot; basedir=&quot;.&quot;> <property name=&quot;live.dir&quot; value=&quot;/path/to/myproject/&quot; /> <property name=&quot;project.tar&quot; value=&quot;myproject.tgz&quot; /> <target name=&quot;deploy&quot; depends=&quot;clean&quot;> <echo msg=&quot;Copying gz to the server.&quot; /> <exec command=&quot;scp ${tarfile} me@myserver.com:/var/projects/myproject/&quot; dir=&quot;.&quot; /> <echo msg=&quot;Unzipping on the server.&quot; /> <exec command=&quot;ssh me@myserver.com 'cd {$live.dir}; \ tar -xzvf ${project.tar}; rm ${project.tar}'&quot; dir=&quot;.&quot; /> <delete file=&quot;${tarfile}&quot;/> </target> </project>

41. phing Now chain everything together. <?xml version=&quot;1.0&quot;?> <project name=&quot;myproject.com&quot; default=&quot;main&quot; basedir=&quot;.&quot;> <target name=&quot;phpdoc&quot;>[snip]</target> <target name=&quot;sniffcode&quot;>[snip]</target> <target name=&quot;deploy&quot;>[snip]</target> <target name=&quot;main&quot; depends=&quot;phpdoc,sniffcode,deploy&quot; > </target> </project> On the command line just run code sniffer: $ phing sniffcode Or to do it all: $ phing

42. phing: Why use it? Standardize documentation (all developers generating similar documentation) Standardize and automate deployment Run pre-deployment checks Prevent mistakes. Configure files with ”live” settings

43. phing: Resources User Guide http://phing.info/docs/guide/current/ Shell Scripts no More! http://nsslive.net/2009/02/16/shell-scripts-no-more/ Phing: Building With PHP http://www.slideshare.net/hozn/phing-building-with-php

44. Shell Tools: Benchmarking Get a quick and dirty idea of how your code might perform under a heavy load. ab - Apache HTTP server benchmarking tool siege - Siege is an http regression testing and benchmarking utility. It was designed to let web developers measure the performance of their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP and HTTPS protocols. It allows the user hit a web server with a configurable number of concurrent simulated users. Those users place the webserver &quot;under siege.&quot; According to Paul M. Jones, siege is more accurate. http://paul-m-jones.com/?p=413

45. Shell Tools: siege $ seige http://www.example.com ** SIEGE 2.66 ** Preparing 15 concurrent users for battle. The server is now under siege... HTTP/1.1 200 0.75 secs: 2912 bytes ==> / [...snip...] Lifting the server siege... done. Transactions: 32 hits Availability: 78.05 % Elapsed time: 24.30 secs Data transferred: 0.09 MB Response time: 2.55 secs Transaction rate: 1.32 trans/sec Throughput: 0.00 MB/sec Concurrency: 3.35 Successful transactions: 32 Failed transactions: 9 Longest transaction: 8.51 Shortest transaction: 0.56

46. Again, Why learn all of these tools? Automate repetitive tasks. Make complex tasks simpler. Ensure best practices are followed. Don't re-invent the wheel.

47. Questions?

48. Contact & Credits [realm3.com] brian /at/ realm3.com twitter.com/brian_dailey (917) 512-3594 Credit to: Andrew Yochum http://plexpod.com Gennady Feldman http://www.gena01.com