2011-04-26_01-velvet-curtain-presentation

Velvet / Curtain
Matthias Haimel

EBI is an Outstation of the European Molecular Biology Laboratory.



2 25.04.11 Velvet / Curtain

Overview
• De Bruijn Graph
• Velvet
• Theory
• Practice
• Data formats and quality
• Velvet
• Simulation data
• Multiple insert lengths
• Curtain
• Theory
• Practice


De Bruijn graph
• A concept in combinatorial mathematics
• In combinatorics, de bruijn graph is usually fully connected
• http://en.wikipedia.org/wiki/De_Bruijn_graph
• de bruijn sequence
• Related concept
• Path through graph

• Velvet
• de Bruijn inspired graph structure


De Bruijn graph (Velvet)
• Representation of
• a sequence based on short words (k-mers)
• overlaps between words
• K-mer: word of length k
• K=5
GCCTTCCA
• k-1 overlap

GCCTT GCCTT GCCTT
CCTTC CCTTC CCTTC
CTTCC CTTCC
TTCCA
...
GCCTTCCA GCCTTCCA GCCTTCCA


De Bruijn graph (Velvet)
GCCTTCCAATTT
GCCTTCAAATTT

C A
CTTC TTCC .....
CAATT
T
CCT TC
G CT
C AATTT
A A
CTTC TTCA ..... AAATT


De Bruijn graph representations (Velvet)
TTCA
ATTC TCAG
Error free, no repeat,
no polymorphism

Repeat > kmer length

SNP, variant, < kmer length

Structural variant, inversion
Structural variant, deletion…
…


Example
TAGTCGAGGCTTTAGATCCGATGAGGCTTTAGAGACAG
AGTCGAG CTTTAGA CGATGAG CTTTAGA
GTCGAGG TTAGATC ATGAGGC GAGACAG
GAGGCTC ATCCGAT AGGCTTT GAGACAG
AGTCGAG TAGATCC ATGAGGC TAGAGAA
TAGTCGA CTTTAGA CCGATGA TTAGAGA
CGAGGCT AGATCCG TGAGGCT AGAGACA
TAGTCGA GCTTTAG TCCGATG GCTCTAG
TCGACGC GATCCGA GAGGCTT AGAGACA
TAGTCGA TTAGATC GATGAGG TTTAGAG
GTCGAGG TCTAGAT ATGAGGC TAGAGAC
AGGCTTT ATCCGAT AGGCTTT GAGACAG
AGTCGAG TTAGATT ATGAGGC AGAGACA
GGCTTTA TCCGATG TTTAGAG
CGAGGCT TAGATCC TGAGGCT GAGACAG
AGTCGAG TTTAGATC ATGAGGC TTAGAGA
GAGGCTT GATCCGA GAGGCTT GAGACAG


Example

Read: GTCGAGG

GTCG
(1x)


Example

Read: GTCGAGG

GTCG TCGA
(1x) (1x)


Example

Read: GTCGAGG

GTCG TCGA CGAG
(1x) (1x) (1x)


Example

Read: GTCGAGG

GTCG TCGA CGAG GAGG
(1x) (1x) (1x) (1x)


Example

New read: CGAGGCT

GTCG TCGA CGAG GAGG
(1x) (1x) (2x) (1x)


Example

Read: CGAGGCT

GTCG TCGA CGAG GAGG
(1x) (1x) (2x) (2x)


Example

Read: CGAGGCT

GTCG TCGA CGAG GAGG AGGC
(1x) (1x) (2x) (2x) (1x)


Example

Read: CGAGGCT

GTCG TCGA CGAG GAGG AGGC GGCT
(1x) (1x) (2x) (2x) (1x) (1x)


Example

New read: TCGACGC

(1x) (2x) (2x) (2x) (1x)


Example

Read: TCGACGC

(1x) (2x) (2x) (2x) (1x)

CGAC GACG ACGC
(1x) (1x) (1x)


Example

etc…
GATT
(1x)

TGAG ATGA GATG CGAT CCGA TCCG ATCC GATC AGAT
(9x) (8x) (5x) (6x) (7x) (7x) (7x) (8x) (8x)

AGAA
(1x)

GCTC CTCT TCTA CTAG
(2x) (1x) (2x) (2x)

TAGT AGTC GTCG TCGA CGAG GAGG AGGC GGCT TAGA AGAG GAGA AGAC GACA ACAG
(3x) (7x) (9x) (10x) (8x) (16x) (16x) (11x) (16x) (9x) (12x) (9x) (8x) (5x)
GCTT CTTT TTTA TTAG
(8x) (8x) (8x) (12x)
CGAC GACG ACGC
(1x) (1x) (1x)


Example

After simplification…

GATT
AGAT

GATCCGATGAG AGAA
GCTCTAG
TAGTCGA CGAG

GAGGCT GGCT TAGA AGAGA AGACAG
GCTTTAG
CGACGC


Example

Tips removed…

AGAT

GATCCGATGAG
GCTCTAG
TAGTCGA CGAG

GCTTTAG


De Bruijn graph biology extensions (Velvet)
• Handling of reverse strand
• DNA is read in two directions
• Paired-end data
• Handling small differences, which are “uninteresting”
• Errors in sequencing technology
• Memory
• regularly use 80, 100GB real memory
• easily get to 1TB real memory requirements


Read variety
• Short reads ~75bp
• Illumina / Solexa
• SOLiD (colour space)
• Long reads 500-1000 bp
• 454 read
• Sanger capillary reads
• Paired-end reads
• Short reads
• short insert length
• Mate pair reads
• Short reads
• long insert length


Paired-End

Mate Pair


Short paired-end / mate pair reads

?

Velvet expect Illumina paired-end orientation: (L-> <-R)

L R paired-end


Short paired-end / mate pair reads

Illumina mate-pair orientation: (<-L R->)
L R
mate pair

reverse complement

L R paired-end


Velvet algorithms
• Remove Bubbles
• Tour Bus

• Velvet parameters
• -max_branch_length
• -max_divergence
• -max_gap_count


Example

AGAT

GATCCGATGAG
GCTCTAG
TAGTCGA CGAG

GCTTTAG

GCTC CTCT TCTA CTAG
(2x) (1x) (2x) (2x)

GGCT TAGA
(11x) (16x)
GCTT CTTT TTTA TTAG
(8x) (8x) (8x) (12x)


Example

Bubbles removed… by TourBus

AGAT

GATCCGATGAG

TAGTCGA CGAG

GAGGCT GGCT GCTTTAG TAGA AGAGA AGACAG


Example

Final simplification…

AGATCCGATGAG

TAGTCGAG GAGGCTTTAGA AGAGACAG


Example
TAGTCGAGGCTTTAGATCCGATGAGGCTTTAGAGACAG

Final simplification…

AGATCCGATGAG

TAGTCGAG GAGGCTTTAGA AGAGACAG

One possible walk through the graph ...
TAGTCGAG
GAGGCTTTAGA
AGATCCGATGAG
GAGGCTTTAGA
AGAGACAG


N50
• Total

• N90

• N50

• N10


N50
• Total
• 4,295,113bp
• N90
• 439bp

• N50
• 3,119bp

• N10
• 13,519bp


N50
• N50 is the length of the smallest contig
• contains the fewest (largest) contigs
• combined length represents at least 50% of the assembly
• N10
• > 10 % of the largest contigs

http://www.broadinstitute.org/crd/wiki/index.php/N50


Velvet practical: Part 1
• Compile
• Single end (ERX001300)
• K-mer length
• Coverage cut-offs
• Whole genome sequence as input???
• Staphylococcus aureus MRSA252


Velvet algorithms
• Long read information
• Rock Band

• -long_mult_cutoff


Velvet algorithms
• Paired-end information
• Pebble

• -min_pair_count

Once all distances and variance computed,
Simple greedy extension from main contigs out


Paired-end in Velvet
• Hugely improves quality of assembly
• Insert length greater than repeat
• greater than the length of the most common genomic repeat
• Mixed insert length improves results
• Short: helps for local assembly
• Long: get over repeats
• Large genomes
• Very memory intensive
• Calculation intensive


Data formats and quality
• Fasta • Fastq
• .fasta • .fastq
• .fa • .fq
• ? • ?
Header

>read_1 @SEQ_ID
TATAATATTTAT... GATTTGGGGTTCAAAGC
Sequence +
!''*((((***+))%%%

Quality


FASTQ paired
@SRR022863.1.F
ATATAGATGTACATAAATTAGTTGAAGTATATGAACG
+
.F .R IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIAIIII
/1 /2 @SRR022863.1.R
TTCACCCATTTTATCCATGATTTTGTTCTTTCTCTTC
+
IIIIIHIIIIIIII3III.,IIII&II6II-))&'I0

@SRR022863.1.F @SRR022863.1.R
ATATAGATGTACATAAATTAGT... TTCACCCATTTTATCCATGATTTTGTT...
+ +
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIAIIII IIIIIHIIIIIIII3III.,IIII&II6II-))&'I0
@SRR022863.2.F @SRR022863.2.R
TTATGAATTATTAATAAGTGCT... CATAAAAAAAGAAAATGTACTCTTTAC...
+ +
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII IIII)0&A,%.&9$8I4+A;I)4II)&%-I$I%#)II


Quality score
• Velvet does NOT use quality score!!!
• Error correction of de Bruijn graph
• p
• the probability that the corresponding base call is incorrect

• Phred quality score
• 10 -> 1 in 10
• 40 -> 1 in 10,000.

• Odds ratio
• earlier versions of solexa pipeline
• differs mainly at lower levels


Quality encoding
• !''*((((***+))%%%
• One value per base
• Integer mapping based on ASCII encoding
• probability of incorrect base call

• Sanger format • Illumina 1.5+
• Phred score • Phred score
• ASCII 33 – 126 -> 0 – 93 • ASCII 59 – 126 -> -5 – 62
• Rarely exceeds 60 • Only 2 – 40 expected
• ! = 33 -> 0 • ! = 33 -> (does not exist)
• b = 66 -> 33 • b = 66 -> 2


Quality encoding
• wikipedia


Quality trimming

Good / Bad ?

Quality score

Bp position in Read


Quality trimming
• Fixed length trimming
• Cut-off at position x
• Adaptive trimming
• Quality score cut-off
• Minimum sequence length
• Sliding window
• Window size
• Quality score cut-off
• Use average quality value of window


• Paired-end (SRX008042)
• Explore parameters
• Set cut-offs
• Analyse quality score (SRX008042)
• Trimming reads


Velvet modules
• Columbus (since Velvet 1.0)
• use reference sequence
• assist with alignment information
• local re-sequencing
• structural variants


Velvet modules
• Oases
• De novo transcriptome assembler
• uses preliminary Velvet assembly
• clusters contigs into loci
• construct transcript isoforms using paired-end / long read
information
• confidence score: describes uniqueness of a transcript in a locus


Read Simulation - Why?
• Controlling the data
• Contamination
• Coverage distribution
• Sequencing errors
• Genome size
• Insert length
• Insert length distribution


Read Simulation - Why?
• Make results comparable
• Assemblers
• Parameters
• Algorithms
• Assembly strategies
• Genome specific “features”
• Robust
• Introduce errors
• Simulate SNPs


Real data vs. simulation

Mario Caccamo


• Velvet
• Long Reads
• Hybrid Assembly
• Mixed insert length libraries


Curtain
• assembly pipeline
• Paired-end assembly for large genomes
• Group related Contigs
• Uses velvet to assemble groups of related reads
• Iterative approach


Curtain

Genome assembly Pipeline

Curtain
Contigs

Map Group Fill
Assemble Collect
Reads Contigs Bins


Curtain
Curtain Contigs
Map Group Fill
AssembleCollect
Reads Contigs Bins

• Set of input Contigs
• Use established assemblers
• Velvet unpaired
• Cortex
• SGA
• ...


Curtain
Curtain Contigs
Map Group Fill AssembleCollect
Reads Contigs Bins

• Map reads to input contigs
• SAM file support
• bwa
• maq


Curtain
Curtain Contigs
Map Group Fill AssembleCollect
Reads Contigs Bins

• Group Contigs using Paired-end information

1 2 3 4 5

bin mapping read & read pair


Curtain
Curtain Contigs
Map Group Fill
Reads Contigs Bins AssembleCollect

• Assemble each bin
• Run velvet using paired-end information
• bin specific parameters
• Run each bin individually velvet
• Highly parallelizable
• Collect results
• Start next iteration ………………….

Results


Curtain
• Low memory footprint
• Scalable for large genomes
• Make use of cluster
• Available
• www.ebi.ac.uk/egt
• http://code.google.com/p/curtain/
• Future announcements
• http://groups.google.com/group/curtain-assembler
• Future work
• Long read support


Curtain practical
• Run Curtain for Staphylococcus
• Simulation data


Thanks ...


2011-04-26_01-velvet-curtain-presentation

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

2011-04-26_01-velvet-curtain-presentation