Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Unix Master

1,013 views

Published on

Unix Basics for Bioinformaticians

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Unix Master

  1. 1. Unix for Bioinformaticians A survival guide Paolo Marcatili - Programmazione 08-09
  2. 2. Agenda <ul><li>Unix </li></ul><ul><li>Folders </li></ul><ul><li>Files </li></ul><ul><li>Processes </li></ul><ul><li>Redirection </li></ul>Paolo Marcatili - Programmazione 08-09
  3. 3. Unix He can smell your fear Paolo Marcatili - Programmazione 08-09
  4. 4. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  5. 5. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  6. 6. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  7. 7. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  8. 8. What is Unix ? <ul><li>Operating system </li></ul><ul><li>stable, </li></ul><ul><li>multi-user, </li></ul><ul><li>multi-tasking </li></ul><ul><li>for servers, </li></ul><ul><li>desktops </li></ul><ul><li>and laptops. </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  9. 9. Types of Unix <ul><li>Solaris </li></ul><ul><li>OS-X </li></ul><ul><li>Linux! </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  10. 10. Unix Operating System Unix Paolo Marcatili - Programmazione 08-09
  11. 11. What’s in Unix? <ul><li>Files </li></ul><ul><li>Processes </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  12. 12. Unix Filesystem Unix Paolo Marcatili - Programmazione 08-09
  13. 13. Terminal <ul><li>It makes the difference! </li></ul><ul><li>Powerful </li></ul><ul><li>Transparent </li></ul><ul><li>User-unfriendly </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  14. 14. Our Task Today Yes, I can! Paolo Marcatili - Programmazione 08-09
  15. 15. Human Immunoglobulins Task Paolo Marcatili - Programmazione 08-09
  16. 16. Human Immunoglobulins Task Paolo Marcatili - Programmazione 08-09
  17. 17. The data <ul><li>We have 3 files </li></ul><ul><li>Fasta Sequences of </li></ul><ul><li>Heavy (heavy.fasta) </li></ul><ul><li>Lambda (lambda.fasta) </li></ul><ul><li>Kappa (kappa.fasta) </li></ul>Task Paolo Marcatili - Programmazione 08-09
  18. 18. The Task <ul><li>Look data </li></ul><ul><li>Correct errors </li></ul><ul><li>Do some analysis </li></ul><ul><li>Make a single fasta file </li></ul>Task Paolo Marcatili - Programmazione 08-09
  19. 19. Folders: read and write Paolo Marcatili - Programmazione 08-09
  20. 20. Change Directory <ul><li>To know where you are </li></ul><ul><li>>pwd </li></ul><ul><li>/home/bioinfoSMFN </li></ul><ul><li>Let’s move! </li></ul><ul><li>>cd /home/bioinfoSMFN/Desktop/task </li></ul><ul><li>Or </li></ul><ul><li>>cd Desktop/ </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  21. 21. Folder content <ul><li>Let’s check what’s in the folder: </li></ul><ul><li>>ls </li></ul><ul><li>Better </li></ul><ul><li>>ls -la </li></ul><ul><li>Cheat: parameters are useful! </li></ul><ul><li>>ls -lart </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  22. 22. Wildcards <ul><li>Extremely useful! </li></ul><ul><li>>ls -la *heavy* </li></ul><ul><li>>ls -la heavy.?asta </li></ul><ul><li>*= 0,1,2,…. occurences of whatever </li></ul><ul><li>?= exactly 1 occurence of whatever </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  23. 23. Commands <ul><li>To know more about a command: </li></ul><ul><li>>man ls </li></ul><ul><li>>whatis ls </li></ul><ul><li>>apropos ls </li></ul><ul><li>Or Google! </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  24. 24. Unix Permissions Folders Paolo Marcatili - Programmazione 08-09
  25. 25. Folders - summary <ul><li>Command Meaning </li></ul><ul><li>ls list files and directories </li></ul><ul><li>ls -a list all files and directories </li></ul><ul><li>mkdir make a directory </li></ul><ul><li>cd directory change to named directory </li></ul><ul><li>cd change to home-directory </li></ul><ul><li>cd ~ change to home-directory </li></ul><ul><li>cd .. change to parent directory </li></ul><ul><li>pwd path of the current directory </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  26. 26. Files: read and write Paolo Marcatili - Programmazione 08-09
  27. 27. Backup original data <ul><li>Create a directory </li></ul><ul><li>>mkdir backup </li></ul><ul><li>Copy all the files in the directory </li></ul><ul><li>>cp heavy.fasta backup/ </li></ul><ul><li>>cp heavy.fasta heavy_copy.fasta </li></ul><ul><li>>mv heavy_copy.fasta backup/heavy_backup.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  28. 28. Read a file <ul><li>Let’s see… </li></ul><ul><li>>cat heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  29. 29. Read a file <ul><li>Let’s see… </li></ul><ul><li>>cat heavy.fasta </li></ul><ul><li>Mmh… </li></ul><ul><li>>less heavy.fasta </li></ul><ul><li>Hint: q to quit… </li></ul><ul><li>but if you are stuck try ctrl+c </li></ul>Files Paolo Marcatili - Programmazione 08-09
  30. 30. Look for text <ul><li>If we want to look for something </li></ul><ul><li>while in less: </li></ul><ul><li>/ </li></ul><ul><li>91979410 (enter) </li></ul>Files Paolo Marcatili - Programmazione 08-09
  31. 31. Edit a file <ul><li>Let’s remove the first H sequence </li></ul><ul><li>>vi heavy.fasta </li></ul><ul><li>>nano heavy.fasta </li></ul><ul><li>For those who live in 2008 </li></ul><ul><li>>gedit heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  32. 32. Look for text <ul><li>How many Igs in each file? </li></ul><ul><li>We can count the lines! </li></ul><ul><li>>wc heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  33. 33. Look for text <ul><li>How many Igs in each file? </li></ul><ul><li>We can count the lines! </li></ul><ul><li>>wc heavy.fasta </li></ul><ul><li>Lines > proteins!!! </li></ul>Files Paolo Marcatili - Programmazione 08-09
  34. 34. Grep <ul><li># of proteins = # of “>” in a file </li></ul><ul><li>>grep “>” heavy.fasta </li></ul><ul><li>>grep -c “>” heavy.fasta </li></ul><ul><li>Do it for all the files and write the result </li></ul><ul><li>(I think they are too many…) </li></ul>Files Paolo Marcatili - Programmazione 08-09
  35. 35. Grep <ul><li>>grep -c V-region heavy.fasta </li></ul><ul><li>>grep -c v-region heavy.fasta </li></ul><ul><li>>grep -ci v-region heavy.fasta </li></ul><ul><li>Ok, better! </li></ul>Files Paolo Marcatili - Programmazione 08-09
  36. 36. Files: sumary <ul><li>cp file1 file2 copy file1 and call it file2 </li></ul><ul><li>mv file1 file2 move or rename file1 to file2 </li></ul><ul><li>rm file remove a file </li></ul><ul><li>cat file display a file </li></ul><ul><li>less file display a file a page at a time </li></ul><ul><li>head file display the first few lines </li></ul><ul><li>tail file display the last few lines </li></ul><ul><li>grep 'key' file search a file for keywords </li></ul><ul><li>wc file number of lines </li></ul>Files Paolo Marcatili - Programmazione 08-09
  37. 37. Processes Paolo Marcatili - Programmazione 08-09
  38. 38. Run a process <ul><li>Process = execution of some instructions </li></ul><ul><li>Usually the instructions are in a file. </li></ul><ul><li>e.g. less -> /usr/bin/less </li></ul><ul><li>So </li></ul><ul><li>>/usr/bin/less heavy.fasta </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  39. 39. Run - It’s simple <ul><li>Ok, so let’s try </li></ul><ul><li>>./loop.pl </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  40. 40. Run - It’s simple <ul><li>Ok, so let’s try </li></ul><ul><li>>./loop.pl </li></ul><ul><li>Ok, and now? </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  41. 41. Controlling processes <ul><li>Ctrl+z -> go to sleep </li></ul><ul><li>Now you’ve got the control again! </li></ul><ul><li>But it’s not still dead… </li></ul><ul><li>bg </li></ul><ul><li>Ctrl+C </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  42. 42. Controlling processes <ul><li>Or: </li></ul><ul><li>>./loop.pl </li></ul><ul><li>Ctrl+z </li></ul><ul><li>>ps </li></ul><ul><li>>top ( q to exit) </li></ul><ul><li>>kill the_number_that_you_have_just_read </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  43. 43. Redirection Control the force Paolo Marcatili - Programmazione 08-09
  44. 44. Inputs and outputs Keyboard Mouse Tablet Kernel Display Printer File Redirection Paolo Marcatili - Programmazione 08-09
  45. 45. Write into a file <ul><li>>Cat > lista.txt </li></ul><ul><li>Heavy 29061 </li></ul><ul><li>Ctrl+d </li></ul><ul><li>And </li></ul><ul><li>>less lista.txt </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  46. 46. Append to a file <ul><li>>Cat >> lista.txt </li></ul><ul><li>Kappa 7476 </li></ul><ul><li>Ctrl+d </li></ul><ul><li>And </li></ul><ul><li>>less lista.txt </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  47. 47. The big one! <ul><li>>cat heavy.fasta > bigone.fasta </li></ul><ul><li>>cat kappa >> bigone.fasta </li></ul><ul><li>>cat lambda.fasta >> bigone.fasta </li></ul><ul><li>>grep -c “>” bigone.fasta </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  48. 48. Extract headers <ul><li>>grep “>” bigone.fasta > headers.head </li></ul><ul><li>>sort headers.head > headers_sort.head </li></ul><ul><li>Do we need the first file? </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  49. 49. Extract headers <ul><li>>grep “>” bigone.fasta > headers.head </li></ul><ul><li>>sort headers.head > headers_sort.head </li></ul><ul><li>Do we need the first file? No! </li></ul><ul><li>>grep “>” bigone.fasta | sort > headers_sorted.head </li></ul><ul><li>| is called pipe, and it’s something </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  50. 50. Redirect everything! <ul><li>>grep &quot;>&quot; biggone.fasta 2> err.log </li></ul><ul><li>>less err.log </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  51. 51. Redirect - summary <ul><li>command > file redirect std output to a file </li></ul><ul><li>command >> file append std output to a file </li></ul><ul><li>command < file redirect std input from a file </li></ul><ul><li>cmd1 | cmd2 pipe the output of cmd1 to the input of cmd2 </li></ul><ul><li>cat f1 f2 > f0 concatenate f1 and f2 to f0 </li></ul><ul><li>sort sort data </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  52. 52. Quiz <ul><li>What does this command perform? >grep -v &quot;>&quot; heavy.fasta </li></ul><ul><li>How many proteins in each file? </li></ul><ul><li>How many residues in each file? (hint: wc -m counts the # of char in a line) </li></ul><ul><li>Average length of proteins in each file? </li></ul><ul><li>Average content of Prolines of VH, VL and VK? (hint: look at grep parameters) </li></ul>Paolo Marcatili - Programmazione 08-09

×