Jennifer Shelton, profile picture
Uploaded byJennifer Shelton
PDF, PPTX4,311 views

Lecture1: NGS Analysis on Beocat and an introduction to Perl programming for Bioinformatics 2014

This document provides an introduction to using the command line interface on the Beocat computing cluster. It discusses how to log in via SSH, navigate directories using commands like pwd, ls, cd, and create/delete directories and symbolic links. The document also covers using relative paths, editing files with nano, and moving files with mv.

Embed presentation

Download as PDF, PPTX
Files, directories, editing and pipes NGS Analysis on Beocat and an introduction to Perl programming for Bioinformatics 2014! ! Jennifer Shelton
Before class Please read through the following pages and install the software listed on these pages onto your laptop before coming to class:! ! https://github.com/i5K-KINBRE-script-share/FAQ/blob/master/ UsingBeocat.md! ! https://github.com/i5K-KINBRE-script-share/FAQ/blob/master/ BeocatEditingTransferingFiles.md
Logging in • Use the program “ssh” an OpenSSH SSH client (remote login program) to log into Beocat! • You will not see text as you type your password $ ssh EID@beocat.cis.ksu.edu password:
Terminal Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
Terminal • We are now connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
Terminal • We are now connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
Terminal • We are now connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). • A read-eval-print loop (REPL) is a command-line interface that reads a command from the user, executes it, prints the result, and waits for another command. Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
Terminal • We are now connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). • A read-eval-print loop (REPL) is a command-line interface that reads a command from the user, executes it, prints the result, and waits for another command. Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
Terminal • We are now connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). • A read-eval-print loop (REPL) is a command-line interface that reads a command from the user, executes it, prints the result, and waits for another command. • A graphical user interface (GUI) is a graphical user interface, usually controlled by using a mouse. Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
Shell • shell: A command-line interface such as Bash (the Bourne-Again Shell) or the Microsoft Windows DOS shell that allows a user to interact with the operating system. shell User Software carpentry v.5 http://software-carpentry.org/v5/gloss.html! Software carpentry v.4 http://software-carpentry.org/v4/shell
Shell shell User $ ps -p $$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash
Shell shell User $ ps -p $$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash “process status” program
Shell shell User $ ps -p $$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash “process status” program PID parameter
Shell shell User $ ps -p $$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash Current process “process status” program PID parameter
Shell shell User $ ps -p $$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash Current process “process status” program PID parameter Name of the current shell
Shell shell User $ whoami bioinfo
Shell shell User $ whoami bioinfo “whoami” program
Shell shell User $ whoami bioinfo “whoami” program User ID
Files and directories $ pwd /homes/bioinfo
Files and directories $ pwd /homes/bioinfo “pwd” or print working directory program
Files and directories $ pwd /homes/bioinfo “pwd” or print working directory program Current working directory
Files and directories $ pwd /homes/bioinfo “pwd” or print working directory program root / Current working directory
Files and directories $ pwd /homes/bioinfo “pwd” or print working directory program root / tmp homes bin Current working directory
Files and directories $ pwd /homes/bioinfo “pwd” or print working directory program root / tmp homes bin user1 bioinfo user2 Current working directory
Files and directories $ ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
Files and directories $ ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
Files and directories $ ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
Files and directories $ ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
Relative paths $ ls /homes/bioinfo $ ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
Relative paths $ ls /homes/bioinfo $ ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
Relative paths $ ls /homes/bioinfo $ ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
Relative paths $ ls /homes/bioinfo $ ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
Relative paths $ ls /homes/bioinfo $ ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
Navigate and create directories $ cd ~/pipeline_datasets/RNA-SeqAlign2Ref $ ls sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1-iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* $ pwd /homes/bioinfo/pipeline_datasets/RNA-SeqAlign2Ref $ mkdir test $ ls test… “cd” change directories! “mkdir” make directories
Navigate and create directories “touch” creates files! “rm” deletes files! or use cyberduck
Navigate and create directories “touch” creates files! “rm” deletes files! “nano” is a commandline file editor! or use cyberduck! ! Software carpentry v.5 http://software-carpentry.org/v5/gloss.html! Software carpentry v.4 http://software-carpentry.org/v4/shell
Navigate and create directories “touch” creates files! “rm” deletes files! “nano” is a commandline file editor! or use cyberduck! ! Software carpentry v.5 http://software-carpentry.org/v5/gloss.html! Software carpentry v.4 http://software-carpentry.org/v4/shell
Move files or directories $ mv ~/pipeline_datasets/test.txt ~/test.txt $ ls ~ test.txt… “mv” move files or directories to a new location
Unix wildcards and head/tail $ ls ~/pipeline_datasets/RNA-SeqAlign2Ref/*.fastq pipeline_datasets/RNA-SeqAlign2Ref/Galaxy5-brain_2.fastq* pipeline_datasets/RNA-SeqAlign2Ref/Galaxy4-brain_1.fastq* pipeline_datasets/RNA-SeqAlign2Ref/Galaxy3-adrenal_2.fastq* pipeline_datasets/RNA-SeqAlign2Ref/Galaxy2-adrenal_1.fastq* $ head ~/pipeline_datasets/RNA-SeqAlign2Ref/*.fastq ==> pipeline_datasets/RNA-SeqAlign2Ref/Galaxy2-adrenal_1.fastq <== @ERR030881.107 HWI-BRUNOP16X_0001:2:1:13663:1096#0/1 ATCTTTTGTGGCTACAGTAAGTTCAATCTGAAGTCAAAACCAACCAATTT + 5.544,444344555CC?CAEF@EEFFFFFFFFFFFFFFFFFEFFFEFFF… “*” any character 0 or 1 times (can be used with most basic Unix commands)! “head” prints first 4 lines of a file “tail” prints the last
Common bioinformatics file formats @ERR030881.107 HWI-BRUNOP16X_0001:2:1:13663:1096#0/1 ATCTTTTGTGGCTACAGTAAGTTCAATCTGAAGTCAAAACCAACCAATTT + 5.544,444344555CC?CAEF@EEFFFFFFFFFFFFFFFFFEFFFEFFF Fastq: sequence data with quality scores. Four lines per entry header line, sequence, second header or +, base quality scores. http://en.wikipedia.org/wiki/FASTQ_format >Locus_1_Transcript_2/3_Confidence_0.333_Length_600 CCCCCCTTCAGTTCCCTTAAAGCACAGCCCAGGGAAACCTCCTCACAGTTTTCATCCAGC CACGGGCCAGCATGTCTGGGGGCAAATACGTAGACTCGGAGGGACATCTCTACACCGTTC CCATCCGGGAACAGGGCAACATCTACAAGCCCAACAACAAGGCCATGGCAGACGAGC Fasta: sequence data. Header line that begins with “>”, sequence (generally wrapped). http://www.ncbi.nlm.nih.gov/ BLAST/blastcgihelp.shtml
Common bioinformatics file formats !HWUSI-EAS1794_0001_FC61KOJ:5:110:7624:5467#0 99 Locus_126_Transcript_1 6319 1 50M = 6478 209 GCTTGTGGCAT IIIIIIIIIIII HWUSI-EAS1794_0001_FC61KOJ:5:110:7624:5467#0 147 Locus_126_Transcript_1 6478 1 50M = 6319 -209 GACGTTCGTGAT IHIIHHIIIIII Sam: sequence alignment. Tab delimited file with eleven required feilds. http://samtools.github.io/hts-specs/SAMv1.pdf Bam: binary version of a sam file. Read header MAPQ Target header! Read seq Read quality
Pipes Standard! input Stdin ! Software carpentry v.4 http://software-carpentry.org/v4/shell
Pipes Standard! input Stdin Standard! input Stdin “|” passes output from some kinds of programs as input to other programs to chain together steps! “>” tells the shell to print the output to a file rather than display on the screen ! Software carpentry v.4 http://software-carpentry.org/v4/shell
Pipes ! $ cd ~/pipeline_datasets/RNA-SeqAlign2Ref $ wc -l *.fastq > lines wc lines ! Software carpentry v.4 http://software-carpentry.org/v4/shell
Pipes ! $ wc -l *.fastq | sort > lines wc sort lines ! Software carpentry v.4 http://software-carpentry.org/v4/shell
Pipes ! $ wc -l *.fastq | sort | head -1 > lines lines wc sort head -1 ! Software carpentry v.4 http://software-carpentry.org/v4/shell
Pipes and grep ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. A filter transforms a stream of input into a stream of output ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. A filter transforms a stream of input into a stream of output ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters Any program that reads lines of text from standard input, and writes lines of text to standard output, can work with every other ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep This programming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters Any program that reads lines of text from standard input, and writes lines of text to standard output, can work with every other ! $ wc -l *.fastq | sort | head -1 > lines
Pipes and grep $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
Pipes and grep “|” passes output from some kinds of programs as input to other programs to chain together steps $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
Pipes and grep “|” passes output from some kinds of programs as input to other programs to chain together steps “-” tells samtools program to use the output from the previous step as input $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
Pipes and grep “|” passes output from some kinds of programs as input to other programs to chain together steps “-” tells samtools program to use the output from the previous step as input “>” tells the shell to print the output to a file rather than display on the screen $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
Pipes and grep “|” passes output from some kinds of programs as input to other programs to chain together steps “-” tells samtools program to use the output from the previous step as input “>” tells the shell to print the output to a file rather than display on the screen “grep” searches for patterns in a file. The “-c” parameter tells greps to count lines with the pattern (in this case we can count contigs in a fasta). $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
Pipes with samtools ! $ /homes/bioinfo/bioinfo_software/samtools/samtools https://www.biostars.org/p/43677/! ! http://samtools.sourceforge.net/pipe.shtml
Review Unix ps -p $$ process status for the process id of the current shell pwd print working directory ln -s create link with the -s parameter for symbolic ls list directory contents .. one directory up from the current working directory . current working directory ~ home directory * wildcard cd change directories mkdir make directories mv moves files or directories head prints first four lines of a file tail prints last four lines of a file | chains programs together grep searches for patterns wget non-interactive network downloader
Review NGS samtools cat concatenate BAMs samtools flagstat simple stats samtools view SAM<->BAM conversion samtools sort Sort alignments by leftmost coordinates samtools rmdup Remove potential PCR duplicates

More Related Content

PPT
Happy porting x86 application to android
byOwen Hsu
 
PDF
Badge Hacking with Nerves Workshop - ElixirConf 2016 - Justin Schneck and Fra...
byGregMefford
 
PDF
Advanced debugging  techniques in different environments
byAndrii Soldatenko
 
PDF
Tracing Software Build Processes to Uncover License Compliance Inconsistencies
byShane McIntosh
 
PPT
Composer - Package Management for PHP. Silver Bullet?
byKirill Chebunin
 
PDF
Composer the right way - SunshinePHP
byRafael Dohms
 
PDF
Identifying Hotspots in the PostgreSQL Build Process
byShane McIntosh
 
PDF
Identifying Hotspots in Software Build Processes
byShane McIntosh
 
Happy porting x86 application to android
byOwen Hsu
 
Badge Hacking with Nerves Workshop - ElixirConf 2016 - Justin Schneck and Fra...
byGregMefford
 
Advanced debugging  techniques in different environments
byAndrii Soldatenko
 
Tracing Software Build Processes to Uncover License Compliance Inconsistencies
byShane McIntosh
 
Composer - Package Management for PHP. Silver Bullet?
byKirill Chebunin
 
Composer the right way - SunshinePHP
byRafael Dohms
 
Identifying Hotspots in the PostgreSQL Build Process
byShane McIntosh
 
Identifying Hotspots in Software Build Processes
byShane McIntosh
 

What's hot

PDF
What Can Reverse Engineering Do For You?
byAmanda Rousseau
 
PDF
[COSCUP 2021] LLVM Project: The Good, The Bad, and The Ugly
byMin-Yih Hsu
 
PDF
Browser Serving Your We Application Security - ZendCon 2017
byPhilippe Gamache
 
PPTX
Adding Security to Your Workflow with InSpec (MAY 2017)
byMandi Walls
 
PDF
OSDC 2017 | Building Security Into Your Workflow with InSpec by Mandi Walls
byNETWAYS
 
PDF
Advanced Eclipse Workshop (held at IPC2010 -spring edition-)
byBastian Feder
 
PDF
Submit PHP: Standards in PHP world. Михайло Морозов
byBinary Studio
 
PPTX
Modern javascript localization with c-3po and the good old gettext
byAlexander Mostovenko
 
PDF
Make your application expressive
byChristian Varela
 
PDF
Divorcing System
byShawn Sorichetti
 
PPT
How To Start Up With Php In Ibm I
byAlex Frenkel
 
PDF
Building robust and friendly command line applications in go
byAndrii Soldatenko
 
PPT
ZFConf 2012: Dependency Management в PHP и Zend Framework 2 (Кирилл Чебунин)
byZFConf Conference
 
KEY
Misultin 0.7.1 Presentation
byRoberto Ostinelli
 
PDF
Debugging concurrency programs in go
byAndrii Soldatenko
 
KEY
ICSE2011_SRC
byShane McIntosh
 
PDF
Dependency Management with Composer
byJordi Boggiano
 
What Can Reverse Engineering Do For You?
byAmanda Rousseau
 
[COSCUP 2021] LLVM Project: The Good, The Bad, and The Ugly
byMin-Yih Hsu
 
Browser Serving Your We Application Security - ZendCon 2017
byPhilippe Gamache
 
Adding Security to Your Workflow with InSpec (MAY 2017)
byMandi Walls
 
OSDC 2017 | Building Security Into Your Workflow with InSpec by Mandi Walls
byNETWAYS
 
Advanced Eclipse Workshop (held at IPC2010 -spring edition-)
byBastian Feder
 
Submit PHP: Standards in PHP world. Михайло Морозов
byBinary Studio
 
Modern javascript localization with c-3po and the good old gettext
byAlexander Mostovenko
 
Make your application expressive
byChristian Varela
 
Divorcing System
byShawn Sorichetti
 
How To Start Up With Php In Ibm I
byAlex Frenkel
 
Building robust and friendly command line applications in go
byAndrii Soldatenko
 
ZFConf 2012: Dependency Management в PHP и Zend Framework 2 (Кирилл Чебунин)
byZFConf Conference
 
Misultin 0.7.1 Presentation
byRoberto Ostinelli
 
Debugging concurrency programs in go
byAndrii Soldatenko
 
ICSE2011_SRC
byShane McIntosh
 
Dependency Management with Composer
byJordi Boggiano
 

Viewers also liked

PPTX
Workshop NGS data analysis - 1
byMaté Ongenaert
 
PDF
Introduction to Next-Generation Sequencing (NGS) Technology
byQIAGEN
 
PPTX
Crispr/cas9 101
bySuk Namgoong
 
PDF
Introduction to next generation sequencing
byVHIR Vall d’Hebron Institut de Recerca
 
PPTX
Workshop NGS data analysis - 2
byMaté Ongenaert
 
PPTX
ENCODE project: brief summary of main findings
byMaté Ongenaert
 
PDF
QIAseq Technologies for Metagenomics and Microbiome NGS Library Prep
byQIAGEN
 
PPTX
Next Gen Sequencing (NGS) Technology Overview
byDominic Suciu
 
PDF
Computational infrastructure for NGS data analysis
bycursoNGS
 
PPTX
Workshop NGS data analysis - 3
byMaté Ongenaert
 
PPTX
Ngs ppt
byArcha Dave
 
PDF
NGS technologies - platforms and applications
byAGRF_Ltd
 
PPT
Spsl by sasidhar 3 unit
bySasidhar Kothuru
 
PPTX
Integrative transcriptomics to study non-coding RNA functions
byMaté Ongenaert
 
PDF
Advanced NGS Data Analysis & Interpretation- BGW + IVA: NGS Tech Overview Web...
byQIAGEN
 
PPTX
Genomics Facts: Did You Know?
byInsideDNA
 
Workshop NGS data analysis - 1
byMaté Ongenaert
 
Introduction to Next-Generation Sequencing (NGS) Technology
byQIAGEN
 
Crispr/cas9 101
bySuk Namgoong
 
Introduction to next generation sequencing
byVHIR Vall d’Hebron Institut de Recerca
 
Workshop NGS data analysis - 2
byMaté Ongenaert
 
ENCODE project: brief summary of main findings
byMaté Ongenaert
 
QIAseq Technologies for Metagenomics and Microbiome NGS Library Prep
byQIAGEN
 
Next Gen Sequencing (NGS) Technology Overview
byDominic Suciu
 
Computational infrastructure for NGS data analysis
bycursoNGS
 
Workshop NGS data analysis - 3
byMaté Ongenaert
 
Ngs ppt
byArcha Dave
 
NGS technologies - platforms and applications
byAGRF_Ltd
 
Spsl by sasidhar 3 unit
bySasidhar Kothuru
 
Integrative transcriptomics to study non-coding RNA functions
byMaté Ongenaert
 
Advanced NGS Data Analysis & Interpretation- BGW + IVA: NGS Tech Overview Web...
byQIAGEN
 
Genomics Facts: Did You Know?
byInsideDNA
 

Similar to Lecture1: NGS Analysis on Beocat and an introduction to Perl programming for Bioinformatics 2014

PPT
Shell_Scripting.ppt
byKiranMantri
 
PPT
cisco
byedomaldo
 
PPTX
Introduction to-linux
bykishore1986
 
PDF
2018-Summer-Tutorial-Intro-to-Linux.pdf
bysanjeevkuraganti
 
PPTX
Introduction to-linux
byrowiebornia
 
ODP
Lamp ppt
bypoornima sugumaran
 
PDF
Linux CLI
byInternational Institute of Tropical Agriculture
 
PDF
Basic linux commands for bioinformatics
byBonnie Ng
 
PDF
Basic shell commands by Jeremy Sanders
byDevanand Gehlot
 
ODP
NYPHP March 2009 Presentation
bybrian_dailey
 
PPTX
Introduction-to-Linux.pptx
bySharanShrinivasan1
 
PPTX
Introduction-to-Linux.pptx
byDavidMaina47
 
PPTX
Introduction khgjkhygkjiyhgikjyhgikygkii
bycmdept1
 
PDF
Bash shell programming in linux
byNorberto Angulo
 
PPT
3. intro
byHarsh Shrimal
 
PPT
Unix
bySudharsan S
 
PDF
Unleash your inner console cowboy
byKenneth Geisshirt
 
PPT
BITS: Introduction to Linux - Text manipulation tools for bioinformatics
byBITS
 
PPT
Karkha unix shell scritping
bychockit88
 
PDF
Linux intro 3 grep + Unix piping
byGiovanni Marco Dall'Olio
 
Shell_Scripting.ppt
byKiranMantri
 
cisco
byedomaldo
 
Introduction to-linux
bykishore1986
 
2018-Summer-Tutorial-Intro-to-Linux.pdf
bysanjeevkuraganti
 
Introduction to-linux
byrowiebornia
 
Lamp ppt
bypoornima sugumaran
 
Linux CLI
byInternational Institute of Tropical Agriculture
 
Basic linux commands for bioinformatics
byBonnie Ng
 
Basic shell commands by Jeremy Sanders
byDevanand Gehlot
 
NYPHP March 2009 Presentation
bybrian_dailey
 
Introduction-to-Linux.pptx
bySharanShrinivasan1
 
Introduction-to-Linux.pptx
byDavidMaina47
 
Introduction khgjkhygkjiyhgikjyhgikygkii
bycmdept1
 
Bash shell programming in linux
byNorberto Angulo
 
3. intro
byHarsh Shrimal
 
Unix
bySudharsan S
 
Unleash your inner console cowboy
byKenneth Geisshirt
 
BITS: Introduction to Linux - Text manipulation tools for bioinformatics
byBITS
 
Karkha unix shell scritping
bychockit88
 
Linux intro 3 grep + Unix piping
byGiovanni Marco Dall'Olio
 

More from Jennifer Shelton

PPTX
RNA-Seq transcriptome analysis of Gonium pectorale cell cycle.
byJennifer Shelton
 
PPTX
RNA-Seq transcriptome analysis of Gonium pectorale cell cycle
byJennifer Shelton
 
PDF
Structural Variation Detection
byJennifer Shelton
 
PDF
Bionano genome maps_feb2014
byJennifer Shelton
 
PDF
Param selection phase1summary_v2
byJennifer Shelton
 
PPTX
Applied Bioinformatics Journal Club Pacbio RNA-Seq
byJennifer Shelton
 
PDF
Bioinformatic core facilities discussion
byJennifer Shelton
 
PPTX
Hub gene selection_ds
byJennifer Shelton
 
PDF
Bioinformatic jc 08_14_2013_formal
byJennifer Shelton
 
PDF
Using BioNano Maps to Improve an Insect Genome Assembly​
byJennifer Shelton
 
PDF
Bng presentation draft
byJennifer Shelton
 
PDF
Multi-k-mer de novo transcriptome assembly and assembly of assemblies using 4...
byJennifer Shelton
 
PPTX
RNASeq DE methods review Applied Bioinformatics Journal Club
byJennifer Shelton
 
PDF
Summary slides by Prabhakar Chalise of the Oberg et al. 2012 article "Technic...
byJennifer Shelton
 
PDF
Journal club slides to discuss "Differential analysis of gene regulation at t...
byJennifer Shelton
 
PDF
Translocation detection in lung cancer using mate-pair sequencing and iVIGS
byJennifer Shelton
 
RNA-Seq transcriptome analysis of Gonium pectorale cell cycle.
byJennifer Shelton
 
RNA-Seq transcriptome analysis of Gonium pectorale cell cycle
byJennifer Shelton
 
Structural Variation Detection
byJennifer Shelton
 
Bionano genome maps_feb2014
byJennifer Shelton
 
Param selection phase1summary_v2
byJennifer Shelton
 
Applied Bioinformatics Journal Club Pacbio RNA-Seq
byJennifer Shelton
 
Bioinformatic core facilities discussion
byJennifer Shelton
 
Hub gene selection_ds
byJennifer Shelton
 
Bioinformatic jc 08_14_2013_formal
byJennifer Shelton
 
Using BioNano Maps to Improve an Insect Genome Assembly​
byJennifer Shelton
 
Bng presentation draft
byJennifer Shelton
 
Multi-k-mer de novo transcriptome assembly and assembly of assemblies using 4...
byJennifer Shelton
 
RNASeq DE methods review Applied Bioinformatics Journal Club
byJennifer Shelton
 
Summary slides by Prabhakar Chalise of the Oberg et al. 2012 article "Technic...
byJennifer Shelton
 
Journal club slides to discuss "Differential analysis of gene regulation at t...
byJennifer Shelton
 
Translocation detection in lung cancer using mate-pair sequencing and iVIGS
byJennifer Shelton
 

Recently uploaded

PPTX
Pig- piggy bank in Big Data Analytics.ppt.pptx
byVarshini M
 
PPTX
Searching in PubMed andCochrane_Practical Presentation.pptx
bySystematic Reviews Network (SRN)
 
PPTX
Overview of how to Create a Model in Odoo 18
byCeline George
 
PDF
Projecte de la porta d'i5B: Els animals marins
byeduardrubio1
 
PDF
Unit-III pdf (Basic listening Skill, Effective Writing Communication & Writin...
bySayyadTarannum
 
PPTX
How to use search_read method in Odoo 18
byCeline George
 
PPTX
TAMIS & TEMS - HOW, WHY and THE STEPS IN PROCTOLOGY
byJohn Thanakumar
 
PDF
Types of eggs and Egg membranes (Developmental Zoology)
bySudhandraKarthi
 
PPTX
Expected Revenue Report In Odoo 18 CRM
byCeline George
 
PDF
Biology Practical Class 12th 2025 PDF(2)-2-52.pdf
bySCPRPWomenCollegeChi
 
PPTX
3 G8_Q3_L3_ (Cartoon as Representation in Opinion Editorial Article).pptx
byNorafeSahagun
 
PDF
DHA/HAAD/MOH/DOH OPTOMETRY MCQ PYQ. .pdf
byAnmol Singh
 
PDF
Micro Economics for class 12 and class 11
bysurajbajaj4116
 
PPTX
Basics in Phytochemistry, Extraction, Isolation methods, Characterisation etc.
byD. Y. Patil College of Pharmacy, Kolhapur.
 
PPTX
Planning Assessment for Outcome-based Education
byAdri Jovin
 
PPTX
META-ANALYSIS INTERPRETATION, PUBLICATION BIAS AND GRADE ASSESSMENT.pptx
bySystematic Reviews Network (SRN)
 
PPTX
Accounting Skills Paper-II (Registers of PACs and Credit Co-operative Societies)
byDr. Sushil Bansode
 
PDF
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
PPTX
PARENTAL ROUTES OF DRUGS ADMINISTRATION .pptx
byAneetaSharma15
 
PPTX
4 G8_Q3_L4 (Evaluating opinion editorials for textual evidence and quality).pptx
byNorafeSahagun
 
Pig- piggy bank in Big Data Analytics.ppt.pptx
byVarshini M
 
Searching in PubMed andCochrane_Practical Presentation.pptx
bySystematic Reviews Network (SRN)
 
Overview of how to Create a Model in Odoo 18
byCeline George
 
Projecte de la porta d'i5B: Els animals marins
byeduardrubio1
 
Unit-III pdf (Basic listening Skill, Effective Writing Communication & Writin...
bySayyadTarannum
 
How to use search_read method in Odoo 18
byCeline George
 
TAMIS & TEMS - HOW, WHY and THE STEPS IN PROCTOLOGY
byJohn Thanakumar
 
Types of eggs and Egg membranes (Developmental Zoology)
bySudhandraKarthi
 
Expected Revenue Report In Odoo 18 CRM
byCeline George
 
Biology Practical Class 12th 2025 PDF(2)-2-52.pdf
bySCPRPWomenCollegeChi
 
3 G8_Q3_L3_ (Cartoon as Representation in Opinion Editorial Article).pptx
byNorafeSahagun
 
DHA/HAAD/MOH/DOH OPTOMETRY MCQ PYQ. .pdf
byAnmol Singh
 
Micro Economics for class 12 and class 11
bysurajbajaj4116
 
Basics in Phytochemistry, Extraction, Isolation methods, Characterisation etc.
byD. Y. Patil College of Pharmacy, Kolhapur.
 
Planning Assessment for Outcome-based Education
byAdri Jovin
 
META-ANALYSIS INTERPRETATION, PUBLICATION BIAS AND GRADE ASSESSMENT.pptx
bySystematic Reviews Network (SRN)
 
Accounting Skills Paper-II (Registers of PACs and Credit Co-operative Societies)
byDr. Sushil Bansode
 
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
PARENTAL ROUTES OF DRUGS ADMINISTRATION .pptx
byAneetaSharma15
 
4 G8_Q3_L4 (Evaluating opinion editorials for textual evidence and quality).pptx
byNorafeSahagun
 

Lecture1: NGS Analysis on Beocat and an introduction to Perl programming for Bioinformatics 2014

  • 1.
    Files, directories, editingand pipes NGS Analysis on Beocat and an introduction to Perl programming for Bioinformatics 2014! ! Jennifer Shelton
  • 2.
    Before class Please readthrough the following pages and install the software listed on these pages onto your laptop before coming to class:! ! https://github.com/i5K-KINBRE-script-share/FAQ/blob/master/ UsingBeocat.md! ! https://github.com/i5K-KINBRE-script-share/FAQ/blob/master/ BeocatEditingTransferingFiles.md
  • 3.
    Logging in • Usethe program “ssh” an OpenSSH SSH client (remote login program) to log into Beocat! • You will not see text as you type your password $ ssh EID@beocat.cis.ksu.edu password:
  • 4.
    Terminal Software carpentry v.5http://software-carpentry.org/v5/gloss.html
  • 5.
    Terminal • We arenow connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
  • 6.
    Terminal • We arenow connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
  • 7.
    Terminal • We arenow connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). • A read-eval-print loop (REPL) is a command-line interface that reads a command from the user, executes it, prints the result, and waits for another command. Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
  • 8.
    Terminal • We arenow connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). • A read-eval-print loop (REPL) is a command-line interface that reads a command from the user, executes it, prints the result, and waits for another command. Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
  • 9.
    Terminal • We arenow connected to Beocat using a command-line interface (CLI). A CLI is an interface based on typing commands, usually at a read-eval-print loop (REPL). • A read-eval-print loop (REPL) is a command-line interface that reads a command from the user, executes it, prints the result, and waits for another command. • A graphical user interface (GUI) is a graphical user interface, usually controlled by using a mouse. Software carpentry v.5 http://software-carpentry.org/v5/gloss.html
  • 10.
    Shell • shell: Acommand-line interface such as Bash (the Bourne-Again Shell) or the Microsoft Windows DOS shell that allows a user to interact with the operating system. shell User Software carpentry v.5 http://software-carpentry.org/v5/gloss.html! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 11.
    Shell shell User $ ps -p$$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash
  • 12.
    Shell shell User $ ps -p$$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash “process status” program
  • 13.
    Shell shell User $ ps -p$$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash “process status” program PID parameter
  • 14.
    Shell shell User $ ps -p$$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash Current process “process status” program PID parameter
  • 15.
    Shell shell User $ ps -p$$ PID TTY TIME CMD 63825 ttys002 0:00.04 -bash Current process “process status” program PID parameter Name of the current shell
  • 16.
  • 17.
  • 18.
  • 19.
    Files and directories $pwd /homes/bioinfo
  • 20.
    Files and directories $pwd /homes/bioinfo “pwd” or print working directory program
  • 21.
    Files and directories $pwd /homes/bioinfo “pwd” or print working directory program Current working directory
  • 22.
    Files and directories $pwd /homes/bioinfo “pwd” or print working directory program root / Current working directory
  • 23.
    Files and directories $pwd /homes/bioinfo “pwd” or print working directory program root / tmp homes bin Current working directory
  • 24.
    Files and directories $pwd /homes/bioinfo “pwd” or print working directory program root / tmp homes bin user1 bioinfo user2 Current working directory
  • 25.
    Files and directories $ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
  • 26.
    Files and directories $ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
  • 27.
    Files and directories $ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
  • 28.
    Files and directories $ln -s /homes/bioinfo/pipeline_datasets/ ./ $ ls pipeline_datasets@ $ ls pipeline_datasets/RNA-SeqAlign2Ref/ sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1- iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* “ln” or link program with the -s parameter for symbolic! “ls” list directory contents RNA-SeqAlign2Ref AssembleT pipeline_datasets sample_read_list.txt*! Galaxy5-brain_2.fastq*! Galaxy4-brain_1.fastq*! Galaxy3-adrenal_2.fastq*! Galaxy2-adrenal_1.fastq*! Galaxy1- iGenomes_UCSC_hg19_c hr19_gene_annotation.gtf*! hg19.fa* notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt notes.txt
  • 29.
    Relative paths $ ls /homes/bioinfo $ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
  • 30.
    Relative paths $ ls /homes/bioinfo $ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
  • 31.
    Relative paths $ ls /homes/bioinfo $ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
  • 32.
    Relative paths $ ls /homes/bioinfo $ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
  • 33.
    Relative paths $ ls /homes/bioinfo $ls ../../bin ls ln rm mkdir… $ ls ../bioinfo/bioinfo_software cufflinks@ tophat@ samtools@… $ ls ~/pipeline_datasets Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq*… root / tmp homes bin user1 bioinfo user2 “ls” list directory contents! .. one directory up from the current working directory! . current working directory! ~ home directory
  • 34.
    Navigate and createdirectories $ cd ~/pipeline_datasets/RNA-SeqAlign2Ref $ ls sample_read_list.txt* Galaxy5-brain_2.fastq* Galaxy4-brain_1.fastq* Galaxy3-adrenal_2.fastq* Galaxy2-adrenal_1.fastq* Galaxy1-iGenomes_UCSC_hg19_chr19_gene_annotation.gtf* hg19.fa* $ pwd /homes/bioinfo/pipeline_datasets/RNA-SeqAlign2Ref $ mkdir test $ ls test… “cd” change directories! “mkdir” make directories
  • 35.
    Navigate and createdirectories “touch” creates files! “rm” deletes files! or use cyberduck
  • 36.
    Navigate and createdirectories “touch” creates files! “rm” deletes files! “nano” is a commandline file editor! or use cyberduck! ! Software carpentry v.5 http://software-carpentry.org/v5/gloss.html! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 37.
    Navigate and createdirectories “touch” creates files! “rm” deletes files! “nano” is a commandline file editor! or use cyberduck! ! Software carpentry v.5 http://software-carpentry.org/v5/gloss.html! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 38.
    Move files ordirectories $ mv ~/pipeline_datasets/test.txt ~/test.txt $ ls ~ test.txt… “mv” move files or directories to a new location
  • 39.
    Unix wildcards andhead/tail $ ls ~/pipeline_datasets/RNA-SeqAlign2Ref/*.fastq pipeline_datasets/RNA-SeqAlign2Ref/Galaxy5-brain_2.fastq* pipeline_datasets/RNA-SeqAlign2Ref/Galaxy4-brain_1.fastq* pipeline_datasets/RNA-SeqAlign2Ref/Galaxy3-adrenal_2.fastq* pipeline_datasets/RNA-SeqAlign2Ref/Galaxy2-adrenal_1.fastq* $ head ~/pipeline_datasets/RNA-SeqAlign2Ref/*.fastq ==> pipeline_datasets/RNA-SeqAlign2Ref/Galaxy2-adrenal_1.fastq <== @ERR030881.107 HWI-BRUNOP16X_0001:2:1:13663:1096#0/1 ATCTTTTGTGGCTACAGTAAGTTCAATCTGAAGTCAAAACCAACCAATTT + 5.544,444344555CC?CAEF@EEFFFFFFFFFFFFFFFFFEFFFEFFF… “*” any character 0 or 1 times (can be used with most basic Unix commands)! “head” prints first 4 lines of a file “tail” prints the last
  • 40.
    Common bioinformatics fileformats @ERR030881.107 HWI-BRUNOP16X_0001:2:1:13663:1096#0/1 ATCTTTTGTGGCTACAGTAAGTTCAATCTGAAGTCAAAACCAACCAATTT + 5.544,444344555CC?CAEF@EEFFFFFFFFFFFFFFFFFEFFFEFFF Fastq: sequence data with quality scores. Four lines per entry header line, sequence, second header or +, base quality scores. http://en.wikipedia.org/wiki/FASTQ_format >Locus_1_Transcript_2/3_Confidence_0.333_Length_600 CCCCCCTTCAGTTCCCTTAAAGCACAGCCCAGGGAAACCTCCTCACAGTTTTCATCCAGC CACGGGCCAGCATGTCTGGGGGCAAATACGTAGACTCGGAGGGACATCTCTACACCGTTC CCATCCGGGAACAGGGCAACATCTACAAGCCCAACAACAAGGCCATGGCAGACGAGC Fasta: sequence data. Header line that begins with “>”, sequence (generally wrapped). http://www.ncbi.nlm.nih.gov/ BLAST/blastcgihelp.shtml
  • 41.
    Common bioinformatics fileformats !HWUSI-EAS1794_0001_FC61KOJ:5:110:7624:5467#0 99 Locus_126_Transcript_1 6319 1 50M = 6478 209 GCTTGTGGCAT IIIIIIIIIIII HWUSI-EAS1794_0001_FC61KOJ:5:110:7624:5467#0 147 Locus_126_Transcript_1 6478 1 50M = 6319 -209 GACGTTCGTGAT IHIIHHIIIIII Sam: sequence alignment. Tab delimited file with eleven required feilds. http://samtools.github.io/hts-specs/SAMv1.pdf Bam: binary version of a sam file. Read header MAPQ Target header! Read seq Read quality
  • 42.
    Pipes Standard! input Stdin ! Software carpentryv.4 http://software-carpentry.org/v4/shell
  • 43.
    Pipes Standard! input Stdin Standard! input Stdin “|”passes output from some kinds of programs as input to other programs to chain together steps! “>” tells the shell to print the output to a file rather than display on the screen ! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 44.
    Pipes ! $ cd ~/pipeline_datasets/RNA-SeqAlign2Ref $wc -l *.fastq > lines wc lines ! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 45.
    Pipes ! $ wc -l*.fastq | sort > lines wc sort lines ! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 46.
    Pipes ! $ wc -l*.fastq | sort | head -1 > lines lines wc sort head -1 ! Software carpentry v.4 http://software-carpentry.org/v4/shell
  • 47.
    Pipes and grep ! $wc -l *.fastq | sort | head -1 > lines
  • 48.
    Pipes and grep Thisprogramming model called pipes and filters. ! $ wc -l *.fastq | sort | head -1 > lines
  • 49.
    Pipes and grep Thisprogramming model called pipes and filters. ! $ wc -l *.fastq | sort | head -1 > lines
  • 50.
    Pipes and grep Thisprogramming model called pipes and filters. A filter transforms a stream of input into a stream of output ! $ wc -l *.fastq | sort | head -1 > lines
  • 51.
    Pipes and grep Thisprogramming model called pipes and filters. A filter transforms a stream of input into a stream of output ! $ wc -l *.fastq | sort | head -1 > lines
  • 52.
    Pipes and grep Thisprogramming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters ! $ wc -l *.fastq | sort | head -1 > lines
  • 53.
    Pipes and grep Thisprogramming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters ! $ wc -l *.fastq | sort | head -1 > lines
  • 54.
    Pipes and grep Thisprogramming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters Any program that reads lines of text from standard input, and writes lines of text to standard output, can work with every other ! $ wc -l *.fastq | sort | head -1 > lines
  • 55.
    Pipes and grep Thisprogramming model called pipes and filters. A filter transforms a stream of input into a stream of output A pipe connects two filters Any program that reads lines of text from standard input, and writes lines of text to standard output, can work with every other ! $ wc -l *.fastq | sort | head -1 > lines
  • 56.
    Pipes and grep $cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
  • 57.
    Pipes and grep “|”passes output from some kinds of programs as input to other programs to chain together steps $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
  • 58.
    Pipes and grep “|”passes output from some kinds of programs as input to other programs to chain together steps “-” tells samtools program to use the output from the previous step as input $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
  • 59.
    Pipes and grep “|”passes output from some kinds of programs as input to other programs to chain together steps “-” tells samtools program to use the output from the previous step as input “>” tells the shell to print the output to a file rather than display on the screen $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
  • 60.
    Pipes and grep “|”passes output from some kinds of programs as input to other programs to chain together steps “-” tells samtools program to use the output from the previous step as input “>” tells the shell to print the output to a file rather than display on the screen “grep” searches for patterns in a file. The “-c” parameter tells greps to count lines with the pattern (in this case we can count contigs in a fasta). $ cd ~/pipeline_datasets/sam_bam ! $ /homes/bioinfo/bioinfo_software/samtools/samtools cat brain_rep_1_tophat2_out/accepted_hits.bam adrenal_rep_1_tophat2_out_1/accepted_hits.bam | /homes/bioinfo/ bioinfo_software/samtools/samtools flagstat - > alignment_stats.txt ! $ grep -c ">" ../RNA-SeqAlign2Ref/hg19.fa
  • 61.
    Pipes with samtools ! $/homes/bioinfo/bioinfo_software/samtools/samtools https://www.biostars.org/p/43677/! ! http://samtools.sourceforge.net/pipe.shtml
  • 62.
    Review Unix ps -p$$ process status for the process id of the current shell pwd print working directory ln -s create link with the -s parameter for symbolic ls list directory contents .. one directory up from the current working directory . current working directory ~ home directory * wildcard cd change directories mkdir make directories mv moves files or directories head prints first four lines of a file tail prints last four lines of a file | chains programs together grep searches for patterns wget non-interactive network downloader
  • 63.
    Review NGS samtools catconcatenate BAMs samtools flagstat simple stats samtools view SAM<->BAM conversion samtools sort Sort alignments by leftmost coordinates samtools rmdup Remove potential PCR duplicates