Unix for Bioinformaticians A survival guide Paolo Marcatili - Programmazione 08-09
Agenda <ul><li>Unix </li></ul><ul><li>Folders </li></ul><ul><li>Files </li></ul><ul><li>Processes </li></ul><ul><li>Redire...
Unix He can smell your fear Paolo Marcatili - Programmazione 08-09
What is  Unix ? Unix Paolo Marcatili - Programmazione 08-09
What is  Unix ? Unix Paolo Marcatili - Programmazione 08-09
What is  Unix ? Unix Paolo Marcatili - Programmazione 08-09
What is  Unix ? Unix Paolo Marcatili - Programmazione 08-09
What is  Unix ? <ul><li>Operating system </li></ul><ul><li>stable,  </li></ul><ul><li>multi-user,  </li></ul><ul><li>multi...
Types of Unix <ul><li>Solaris </li></ul><ul><li>OS-X </li></ul><ul><li>Linux! </li></ul>Unix Paolo Marcatili - Programmazi...
Unix Operating System Unix Paolo Marcatili - Programmazione 08-09
What’s in Unix? <ul><li>Files  </li></ul><ul><li>Processes </li></ul>Unix Paolo Marcatili - Programmazione 08-09
Unix Filesystem Unix Paolo Marcatili - Programmazione 08-09
Terminal <ul><li>It makes the difference! </li></ul><ul><li>Powerful </li></ul><ul><li>Transparent </li></ul><ul><li>User-...
Our Task Today Yes, I can! Paolo Marcatili - Programmazione 08-09
Human Immunoglobulins Task Paolo Marcatili - Programmazione 08-09
Human Immunoglobulins Task Paolo Marcatili - Programmazione 08-09
The data <ul><li>We have 3 files </li></ul><ul><li>Fasta Sequences of </li></ul><ul><li>Heavy (heavy.fasta) </li></ul><ul>...
The Task <ul><li>Look data </li></ul><ul><li>Correct errors </li></ul><ul><li>Do some analysis </li></ul><ul><li>Make a si...
Folders: read and write Paolo Marcatili - Programmazione 08-09
Change Directory <ul><li>To know where you are </li></ul><ul><li>>pwd </li></ul><ul><li>/home/bioinfoSMFN </li></ul><ul><l...
Folder content <ul><li>Let’s check what’s in the folder: </li></ul><ul><li>>ls </li></ul><ul><li>Better </li></ul><ul><li>...
Wildcards <ul><li>Extremely useful! </li></ul><ul><li>>ls -la *heavy* </li></ul><ul><li>>ls -la heavy.?asta </li></ul><ul>...
Commands <ul><li>To know more about a command: </li></ul><ul><li>>man ls </li></ul><ul><li>>whatis ls </li></ul><ul><li>>a...
Unix Permissions Folders Paolo Marcatili - Programmazione 08-09
Folders - summary <ul><li>Command  Meaning </li></ul><ul><li>ls    list files and directories </li></ul><ul><li>ls -a    l...
Files: read and write Paolo Marcatili - Programmazione 08-09
Backup original data <ul><li>Create a directory </li></ul><ul><li>>mkdir backup </li></ul><ul><li>Copy all the files in th...
Read a file <ul><li>Let’s see… </li></ul><ul><li>>cat heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
Read a file <ul><li>Let’s see… </li></ul><ul><li>>cat heavy.fasta </li></ul><ul><li>Mmh… </li></ul><ul><li>>less heavy.fas...
Look for text <ul><li>If we want to look for something </li></ul><ul><li>while in less: </li></ul><ul><li>/ </li></ul><ul>...
Edit a file <ul><li>Let’s remove the first H sequence </li></ul><ul><li>>vi heavy.fasta </li></ul><ul><li>>nano heavy.fast...
Look for text <ul><li>How many Igs in each file? </li></ul><ul><li>We can count the lines! </li></ul><ul><li>>wc heavy.fas...
Look for text <ul><li>How many Igs in each file? </li></ul><ul><li>We can count the lines! </li></ul><ul><li>>wc heavy.fas...
Grep <ul><li># of proteins = # of “>” in a file </li></ul><ul><li>>grep “>” heavy.fasta </li></ul><ul><li>>grep -c “>” hea...
Grep <ul><li>>grep -c V-region heavy.fasta </li></ul><ul><li>>grep -c v-region heavy.fasta </li></ul><ul><li>>grep -ci v-r...
Files: sumary <ul><li>cp file1 file2  copy file1 and call it file2 </li></ul><ul><li>mv file1 file2  move or rename file1 ...
Processes Paolo Marcatili - Programmazione 08-09
Run a process <ul><li>Process = execution of some instructions </li></ul><ul><li>Usually the instructions are in a file. <...
Run - It’s simple <ul><li>Ok, so let’s try </li></ul><ul><li>>./loop.pl </li></ul>Processes Paolo Marcatili - Programmazio...
Run - It’s simple <ul><li>Ok, so let’s try </li></ul><ul><li>>./loop.pl </li></ul><ul><li>Ok, and now? </li></ul>Processes...
Controlling processes <ul><li>Ctrl+z  -> go to sleep </li></ul><ul><li>Now you’ve got the control again! </li></ul><ul><li...
Controlling processes <ul><li>Or: </li></ul><ul><li>>./loop.pl </li></ul><ul><li>Ctrl+z   </li></ul><ul><li>>ps </li></ul>...
Redirection Control the force Paolo Marcatili - Programmazione 08-09
Inputs and outputs Keyboard Mouse Tablet Kernel Display Printer File Redirection Paolo Marcatili - Programmazione 08-09
Write into a file <ul><li>>Cat > lista.txt </li></ul><ul><li>Heavy 29061 </li></ul><ul><li>Ctrl+d </li></ul><ul><li>And </...
Append to a file <ul><li>>Cat >> lista.txt </li></ul><ul><li>Kappa 7476 </li></ul><ul><li>Ctrl+d </li></ul><ul><li>And </l...
The big one! <ul><li>>cat heavy.fasta > bigone.fasta </li></ul><ul><li>>cat kappa >> bigone.fasta </li></ul><ul><li>>cat l...
Extract headers <ul><li>>grep “>” bigone.fasta > headers.head </li></ul><ul><li>>sort headers.head > headers_sort.head </l...
Extract headers <ul><li>>grep “>” bigone.fasta > headers.head </li></ul><ul><li>>sort headers.head > headers_sort.head </l...
Redirect everything! <ul><li>>grep &quot;>&quot; biggone.fasta 2> err.log </li></ul><ul><li>>less err.log </li></ul>Redire...
Redirect - summary <ul><li>command > file  redirect std output to a file </li></ul><ul><li>command >> file  append std out...
Quiz <ul><li>What does this command perform? >grep -v &quot;>&quot; heavy.fasta </li></ul><ul><li>How many proteins in eac...
Upcoming SlideShare
Loading in …5
×

Unix Master

918 views
861 views

Published on

Unix Basics for Bioinformaticians

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
918
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
31
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Unix Master

  1. 1. Unix for Bioinformaticians A survival guide Paolo Marcatili - Programmazione 08-09
  2. 2. Agenda <ul><li>Unix </li></ul><ul><li>Folders </li></ul><ul><li>Files </li></ul><ul><li>Processes </li></ul><ul><li>Redirection </li></ul>Paolo Marcatili - Programmazione 08-09
  3. 3. Unix He can smell your fear Paolo Marcatili - Programmazione 08-09
  4. 4. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  5. 5. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  6. 6. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  7. 7. What is Unix ? Unix Paolo Marcatili - Programmazione 08-09
  8. 8. What is Unix ? <ul><li>Operating system </li></ul><ul><li>stable, </li></ul><ul><li>multi-user, </li></ul><ul><li>multi-tasking </li></ul><ul><li>for servers, </li></ul><ul><li>desktops </li></ul><ul><li>and laptops. </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  9. 9. Types of Unix <ul><li>Solaris </li></ul><ul><li>OS-X </li></ul><ul><li>Linux! </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  10. 10. Unix Operating System Unix Paolo Marcatili - Programmazione 08-09
  11. 11. What’s in Unix? <ul><li>Files </li></ul><ul><li>Processes </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  12. 12. Unix Filesystem Unix Paolo Marcatili - Programmazione 08-09
  13. 13. Terminal <ul><li>It makes the difference! </li></ul><ul><li>Powerful </li></ul><ul><li>Transparent </li></ul><ul><li>User-unfriendly </li></ul>Unix Paolo Marcatili - Programmazione 08-09
  14. 14. Our Task Today Yes, I can! Paolo Marcatili - Programmazione 08-09
  15. 15. Human Immunoglobulins Task Paolo Marcatili - Programmazione 08-09
  16. 16. Human Immunoglobulins Task Paolo Marcatili - Programmazione 08-09
  17. 17. The data <ul><li>We have 3 files </li></ul><ul><li>Fasta Sequences of </li></ul><ul><li>Heavy (heavy.fasta) </li></ul><ul><li>Lambda (lambda.fasta) </li></ul><ul><li>Kappa (kappa.fasta) </li></ul>Task Paolo Marcatili - Programmazione 08-09
  18. 18. The Task <ul><li>Look data </li></ul><ul><li>Correct errors </li></ul><ul><li>Do some analysis </li></ul><ul><li>Make a single fasta file </li></ul>Task Paolo Marcatili - Programmazione 08-09
  19. 19. Folders: read and write Paolo Marcatili - Programmazione 08-09
  20. 20. Change Directory <ul><li>To know where you are </li></ul><ul><li>>pwd </li></ul><ul><li>/home/bioinfoSMFN </li></ul><ul><li>Let’s move! </li></ul><ul><li>>cd /home/bioinfoSMFN/Desktop/task </li></ul><ul><li>Or </li></ul><ul><li>>cd Desktop/ </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  21. 21. Folder content <ul><li>Let’s check what’s in the folder: </li></ul><ul><li>>ls </li></ul><ul><li>Better </li></ul><ul><li>>ls -la </li></ul><ul><li>Cheat: parameters are useful! </li></ul><ul><li>>ls -lart </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  22. 22. Wildcards <ul><li>Extremely useful! </li></ul><ul><li>>ls -la *heavy* </li></ul><ul><li>>ls -la heavy.?asta </li></ul><ul><li>*= 0,1,2,…. occurences of whatever </li></ul><ul><li>?= exactly 1 occurence of whatever </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  23. 23. Commands <ul><li>To know more about a command: </li></ul><ul><li>>man ls </li></ul><ul><li>>whatis ls </li></ul><ul><li>>apropos ls </li></ul><ul><li>Or Google! </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  24. 24. Unix Permissions Folders Paolo Marcatili - Programmazione 08-09
  25. 25. Folders - summary <ul><li>Command Meaning </li></ul><ul><li>ls list files and directories </li></ul><ul><li>ls -a list all files and directories </li></ul><ul><li>mkdir make a directory </li></ul><ul><li>cd directory change to named directory </li></ul><ul><li>cd change to home-directory </li></ul><ul><li>cd ~ change to home-directory </li></ul><ul><li>cd .. change to parent directory </li></ul><ul><li>pwd path of the current directory </li></ul>Folders Paolo Marcatili - Programmazione 08-09
  26. 26. Files: read and write Paolo Marcatili - Programmazione 08-09
  27. 27. Backup original data <ul><li>Create a directory </li></ul><ul><li>>mkdir backup </li></ul><ul><li>Copy all the files in the directory </li></ul><ul><li>>cp heavy.fasta backup/ </li></ul><ul><li>>cp heavy.fasta heavy_copy.fasta </li></ul><ul><li>>mv heavy_copy.fasta backup/heavy_backup.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  28. 28. Read a file <ul><li>Let’s see… </li></ul><ul><li>>cat heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  29. 29. Read a file <ul><li>Let’s see… </li></ul><ul><li>>cat heavy.fasta </li></ul><ul><li>Mmh… </li></ul><ul><li>>less heavy.fasta </li></ul><ul><li>Hint: q to quit… </li></ul><ul><li>but if you are stuck try ctrl+c </li></ul>Files Paolo Marcatili - Programmazione 08-09
  30. 30. Look for text <ul><li>If we want to look for something </li></ul><ul><li>while in less: </li></ul><ul><li>/ </li></ul><ul><li>91979410 (enter) </li></ul>Files Paolo Marcatili - Programmazione 08-09
  31. 31. Edit a file <ul><li>Let’s remove the first H sequence </li></ul><ul><li>>vi heavy.fasta </li></ul><ul><li>>nano heavy.fasta </li></ul><ul><li>For those who live in 2008 </li></ul><ul><li>>gedit heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  32. 32. Look for text <ul><li>How many Igs in each file? </li></ul><ul><li>We can count the lines! </li></ul><ul><li>>wc heavy.fasta </li></ul>Files Paolo Marcatili - Programmazione 08-09
  33. 33. Look for text <ul><li>How many Igs in each file? </li></ul><ul><li>We can count the lines! </li></ul><ul><li>>wc heavy.fasta </li></ul><ul><li>Lines > proteins!!! </li></ul>Files Paolo Marcatili - Programmazione 08-09
  34. 34. Grep <ul><li># of proteins = # of “>” in a file </li></ul><ul><li>>grep “>” heavy.fasta </li></ul><ul><li>>grep -c “>” heavy.fasta </li></ul><ul><li>Do it for all the files and write the result </li></ul><ul><li>(I think they are too many…) </li></ul>Files Paolo Marcatili - Programmazione 08-09
  35. 35. Grep <ul><li>>grep -c V-region heavy.fasta </li></ul><ul><li>>grep -c v-region heavy.fasta </li></ul><ul><li>>grep -ci v-region heavy.fasta </li></ul><ul><li>Ok, better! </li></ul>Files Paolo Marcatili - Programmazione 08-09
  36. 36. Files: sumary <ul><li>cp file1 file2 copy file1 and call it file2 </li></ul><ul><li>mv file1 file2 move or rename file1 to file2 </li></ul><ul><li>rm file remove a file </li></ul><ul><li>cat file display a file </li></ul><ul><li>less file display a file a page at a time </li></ul><ul><li>head file display the first few lines </li></ul><ul><li>tail file display the last few lines </li></ul><ul><li>grep 'key' file search a file for keywords </li></ul><ul><li>wc file number of lines </li></ul>Files Paolo Marcatili - Programmazione 08-09
  37. 37. Processes Paolo Marcatili - Programmazione 08-09
  38. 38. Run a process <ul><li>Process = execution of some instructions </li></ul><ul><li>Usually the instructions are in a file. </li></ul><ul><li>e.g. less -> /usr/bin/less </li></ul><ul><li>So </li></ul><ul><li>>/usr/bin/less heavy.fasta </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  39. 39. Run - It’s simple <ul><li>Ok, so let’s try </li></ul><ul><li>>./loop.pl </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  40. 40. Run - It’s simple <ul><li>Ok, so let’s try </li></ul><ul><li>>./loop.pl </li></ul><ul><li>Ok, and now? </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  41. 41. Controlling processes <ul><li>Ctrl+z -> go to sleep </li></ul><ul><li>Now you’ve got the control again! </li></ul><ul><li>But it’s not still dead… </li></ul><ul><li>bg </li></ul><ul><li>Ctrl+C </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  42. 42. Controlling processes <ul><li>Or: </li></ul><ul><li>>./loop.pl </li></ul><ul><li>Ctrl+z </li></ul><ul><li>>ps </li></ul><ul><li>>top ( q to exit) </li></ul><ul><li>>kill the_number_that_you_have_just_read </li></ul>Processes Paolo Marcatili - Programmazione 08-09
  43. 43. Redirection Control the force Paolo Marcatili - Programmazione 08-09
  44. 44. Inputs and outputs Keyboard Mouse Tablet Kernel Display Printer File Redirection Paolo Marcatili - Programmazione 08-09
  45. 45. Write into a file <ul><li>>Cat > lista.txt </li></ul><ul><li>Heavy 29061 </li></ul><ul><li>Ctrl+d </li></ul><ul><li>And </li></ul><ul><li>>less lista.txt </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  46. 46. Append to a file <ul><li>>Cat >> lista.txt </li></ul><ul><li>Kappa 7476 </li></ul><ul><li>Ctrl+d </li></ul><ul><li>And </li></ul><ul><li>>less lista.txt </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  47. 47. The big one! <ul><li>>cat heavy.fasta > bigone.fasta </li></ul><ul><li>>cat kappa >> bigone.fasta </li></ul><ul><li>>cat lambda.fasta >> bigone.fasta </li></ul><ul><li>>grep -c “>” bigone.fasta </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  48. 48. Extract headers <ul><li>>grep “>” bigone.fasta > headers.head </li></ul><ul><li>>sort headers.head > headers_sort.head </li></ul><ul><li>Do we need the first file? </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  49. 49. Extract headers <ul><li>>grep “>” bigone.fasta > headers.head </li></ul><ul><li>>sort headers.head > headers_sort.head </li></ul><ul><li>Do we need the first file? No! </li></ul><ul><li>>grep “>” bigone.fasta | sort > headers_sorted.head </li></ul><ul><li>| is called pipe, and it’s something </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  50. 50. Redirect everything! <ul><li>>grep &quot;>&quot; biggone.fasta 2> err.log </li></ul><ul><li>>less err.log </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  51. 51. Redirect - summary <ul><li>command > file redirect std output to a file </li></ul><ul><li>command >> file append std output to a file </li></ul><ul><li>command < file redirect std input from a file </li></ul><ul><li>cmd1 | cmd2 pipe the output of cmd1 to the input of cmd2 </li></ul><ul><li>cat f1 f2 > f0 concatenate f1 and f2 to f0 </li></ul><ul><li>sort sort data </li></ul>Redirection Paolo Marcatili - Programmazione 08-09
  52. 52. Quiz <ul><li>What does this command perform? >grep -v &quot;>&quot; heavy.fasta </li></ul><ul><li>How many proteins in each file? </li></ul><ul><li>How many residues in each file? (hint: wc -m counts the # of char in a line) </li></ul><ul><li>Average length of proteins in each file? </li></ul><ul><li>Average content of Prolines of VH, VL and VK? (hint: look at grep parameters) </li></ul>Paolo Marcatili - Programmazione 08-09

×