SlideShare a Scribd company logo
Hands-On Training in Methylation Sequencing Analysis
Time Topic Speaker
09:00~10:30 WGBS Dataset QC & Mapping 張益峰 博士
l 測試資料簡介(Human IMR90 以及 H1 的
WGBS Dataset)
l 執行 short read QC
l meth-pipe 簡介(網站、下載、安裝)
l 執行 meth-pipe 進行 short read mapping 和
mapped raw data 處理
l 計算 bisulfide conversion rate
l 計算 HMR / DMR / PMD
l 產生 BED 格式的檔案
10:45~11:00 Q & A / 休息
11:00~12:30
Basic WGBS Analysis and UCSC Genome
Browser
林依璿 博士
l BEDtools 的簡介
l BEDtools 實做(一):HMR / DMR /
PMD 在Chromosome X 上的分佈
l BEDtools 實做(二):CpG Island 的 DNA
methylation 計算
l BEDtools 實做(三):比較 male (H1) &
female (IMR90) 在 Chromosome X 上的
methylation 差異
l 用 R 以及 ggplot2 package 呈現以上的
methylation 差異
l UCSC Genome Browser 介紹和使用操作
l UCSC Genome Browser 裡面的 MethBase
Public TrackHub 介紹和使用操作
12:30~13:00 Q & A / 午餐
This slide is available in http://www.slideshare.net/YiFengChang
1
Required Software in Your Laptop
• Linux console
• Putty:
http://the.earth.li/~sgtatham/putty/lat
est/x86/putty.exe
• SCP/SFTP/FTP client
• Winscp:
http://winscp.net/download/winscp55
6.zip
• PDF viewer
• http://get.adobe.com/tw/reader/
2
http://120.126.44.231/ycl6/20141219_Ian.pptx
http://120.126.44.231/ycl6/20141219_NRPB2014.pdf
NCHC ALPS1 for NRPB Users• Login node: alps1.nchc.org.tw
• Computing nodes
• 48 (we can use 4) x 48 cores 128GB RAM
• 1 x 64 cores 1TB RAM
• Storages
• Users home: 200GB (Temp Account: 1GB)
• /work3: 42TB
• /work5: 200TB
This slide is available in http://www.slideshare.net/YiFengChang
Queue Name Test[註一] 4G 16G 48G 128G 192core 384core 1T
使用記憶體上限 (GB) 2 4 16 48 100 100/48core 100/48core 1024
使用核心數上限 1 2 8 24 48 192 384 64
建議核心使用上限 1 1 6 20 40 192 384 60
工作優先權[註二] 90 85 80 50 30 20 20 10
什麼是 queue?
Queue 就是針對工作所設立的虛擬的運算單元,一個 queue 可以負擔一個運算
工作。如果一臺機器被指定兩個 queue,意味這台機器可能同時運行兩個工作。
Queue 的設計是用來管理計算資源 。
•所有的工作必須透過 queuing system 執行
•目前提供Test, 4G, 16G, 48G, 128G,192core,384core, 1T 共有 五 種 queue
•工作優先順序以Test> 4G>16G>48G>128G>192core>384core> 1T
•記憶體使用大於上限者,將會被中斷運算
•可先使用1T queue試運算,決定記憶體用量後再選擇使用何種 queue
•使用queue時的限制與規則如下:
註一 ,test queue一人只能送一個job,執行時間為10分鐘
註二 ,數字愈大優先權愈高
網中心提供之Queuing system
國網中心提供 IBM Load Sharing Facility (LSF)
目前可以使用的 Queue
3
Submit a Job
s00yao25@alps1:~> cd
s00yao25@alps1:~> bsub -q 4G -o stdout -e stderr "ls"
Job <422673> is submitted to queue <4G>.
s00yao25@alps1:~> cat stdout
s00yao25@alps1:~> bjobs
JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME
422673 s00yao2 RUN 4G alps1 2*alps1-25 ls Dec 18 23:16
s00yao25@alps1:~> bjobs
No unfinished job found
4
Check Job Status
5
Kill a Job
>bkill JOBID
# chang queue priority of pending jobs
> btop
6
Effect and Problems of Bisulfite
Treatment of DNA
7
Krueger, F., Kreck, B., Franke, A. & Andrews, S.R. DNA methylome analysis using short bisulfite sequencing data. Nat
Methods 9, 145-51 (2012).
Mapping bisulfite reads to 4 possible bisulfite strands (OT/CTOT/OB/CTOB) is
equivalent to mapping the bisulfite read and its reverse complementary
read to both Top/Bottom strands of the original reference sequence.
How to Align BS Reads Against Reference Genome?
8
Krueger, F. & Andrews, S.R. Bismark: A flexible aligner and methylation caller for Bisulfite-Seq
applications. Bioinformatics (2011).
Bock, C. Analysing and interpreting DNA
methylation data. Nat Rev Genet 13, 705-19 (2012)
Y=C or T
TCGA TCGT ACGT ATGA
Multiple hits
TTGT ATGT
Multiple hits
TCGA ATGA
Why RMAPBS/RMAPBS-PE
9
http://smithlabresearch.org/manuals/rmap_manual.pdf
Analysis Pipeline
10
Allele-specific Methylated Regions
amrfinder allelicmeth
Differential Methylation Region
dmr
Large Hypo/Hyper-Methylation Domains
pmd
Hypo/Hyper-Methylation Regions
hmr hyperhmr pmr
Methylation Calling
methcounts + error correction
Bisulfite Conversion Rate
bsrate
Remove Duplicate Reads
duplicate-remover
Mapping
rmapbs rmapbs-pe
Quality Trimming
fastq_masker
Cross-species Comparison of Methylomes
liftOver
Calculating Methylation Ratio for Regions
bigWigAverageOverBed roimethstat Bwtools
Generate Methylation BED file
Bedtools bedGraphToBigWig
fastx toolkit: http://hannonlab.cshl.edu/fastx_toolkit/
MethPipe: http://smithlabresearch.org/software/methpipe/
Bedtools: https://github.com/arq5x/bedtools2
Programs from UCSC Genome Browser:
http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64
bwtool: https://github.com/CRG-Barcelona/bwtool/wiki
H1 (male): human embryonic stem cells (107GB)
IMR90 (female): fetal lung fibroblasts (154GB)
http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE16256
11
Convert SRA to Fastq (DO NOT RUN)
sra-toolkit: https://github.com/ncbi/sratoolkit
> fastq-dump --split-3 SRR018975.sra
> ls
SRR018975.fastq
12
Quality Trimming (DO NOT RUN)
#e.g. SRR018975.fastq.gz
> for f in *.gz; # read all gzip files one by one
do
b=`basename $f .gz`; # SRR018975.fastq
echo $f
bsub -q 4G -o $f.stdout -e $f.stderr "
gzip -dc $f| # read gzip file
fastq_masker -q 30 -Q33| # mask low quality reads as Ns
split -dl 6000000 - $b- "; # split fastq file into smaller ones
done
> ls
SRR018975.fastq-00
SRR018975.fastq-01
SRR018975.fastq-02
…
13
Mapping
bsub -q 4G -o rmapbs.stdout -e rmapbs.stderr "
/work3/NRPB1219/bin/rmapbs-pe 
-c /work3/NRPB1219/methpipe-data/data/genome 
-o /home/s00yao00/Output/test.mr 
-m 3 -L 400 -C AGATCGGAAGAGC:GCTCTTCCGATCT
/work3/NRPB1219/methpipe-data/data/snippet_1.fq 
/work3/NRPB1219/methpipe-data/data/snippet_2.fq"
#head /home/s00yao00/Output/test.mr
chr22 379487 379588 FREDDYKRUEGER_0001:1:1:160:969#0/2 2 +
AAGTAATATATATGTTTTGGGATTAGTGAATTTAAGTTAGTATTAAGAATTTTATTATTATTTTTTTATTATATTTTAGAGAGTTATTTTTTTATTTTTAA
B_c^cbbPfbfbLc]][[Xcbcc`cbd`bbcbdZUdbfOffffdffdSSTTQbZbbb`df[bbbbbdffYffffffbbbZbZX^^afffffcff
chr22 379487 379588 FREDDYKRUEGER_0001:1:1:160:969#0/1 3 +
AAGTAATATTTATGTTTTGGGATTAGTGAATTTAAGTTAGTATTATGAATTTTATTATTATTTTTTTATTATATTTTAGAGAGTTATTTTTTTCTTTTTAA
`dd``addddbcUcccdcfffaf_ZdddOVM_[bOZdb`dbffZ`ObObffaeff^ffdfffffffaffdWf_bdebdaU[^[ffa_f_b`OUK^`UZT
chr22 568970 569071 FREDDYKRUEGER_0001:1:1:249:1215#0/2 0 -
TTTTATTTGATGGATAATATTAAGAAATTTGTAGTATTGTTTTGGAATTTTTTGTGAGGGATAAATAAATAGAATATAGTAGTATTGTTTTATAATTTTTT
BcXe^cecccgbabfadadcf^cgggegTggefgeggggggggggggggggggggggggegaggg^gggbggggggggggggegggggggggfggfcdggg
chr22 310957 311058 FREDDYKRUEGER_0001:1:1:303:856#0/2 4 +
TTGTAATTATGTTGATTTTATGTGTAGTTATTGATGTTGTTGTATGGTAGTTTATGGTTTTTTAGGAATTTAGAATTTGAGTTTTATTTTTGTTTTATAGT
Z]]`W[J`]fdPcefbf^fgggggWcggaeaedSSOQTcggdgcgggcdgffdeaccaddcadfbacfaffcaaecadggbdggggcgcgggggccg
chr22 568970 569071 FREDDYKRUEGER_0001:1:1:249:1215#0/1 2 +
AAAGGATTGTAGAATAGTGTTATTGTGTTTTGTTTGTTTGTTTTTTATAAAGGATTTTAGAGTAGTGTTGTAGATTTTTTAGTGTTGTTTATTAGATAGGT
ggagcfggdgggbegffgfgggggggggfgggggggggggggfggggfcf^d^Pfggggebafbbgfgfgge^^ggggggccggfegefggfggeP^ZW[
chr22 581983 582084 FREDDYKRUEGER_0001:1:1:359:1280#0/2 2 +
GAGGTAATTTAGAGTGTTGTTTTTGGTTTTTGAGGGTTTGTTTTTTAAATAGGATATTATTATATTGTTACGATAGTTTGAATGTTTGTTCGTGATAAATG
Bffeeegg_fgeegggeeeeagcggggfgaggggggggggggggggggggfgffdeggggggfgfgggffgggffgggggfggfgggggbgbggggggcf
chr22 310918 311019 FREDDYKRUEGER_0001:1:1:303:856#0/1 0 +
TATATAGAGTTAGGTTTTATAGTTTATTTTTTTATTATTTTGTAATTTTGTTAATTTTATGTTTAGTTATTGATGTTTTTGTATGGTAGTTTATGGTTTTT
faf_f_ggeggcg^ggggbgdggggcgggggggdggdggggggg^ggggggdg[ggggdggcgdccggagggNggcdgggcbYddTcKRcde[ddYgcggg
chr22 581975 582076 FREDDYKRUEGER_0001:1:1:359:1280#0/1 1 +
TATTTTATAAGGTAATTTAGAGTGTTGTTTTTGGTTTTTGAGGGTTTGTTTTTTAAATAGGATATTATTATATTGTTACGATAGTTTGAATGTTTGTTCGT
hggggfggghggegggggcg]gggggggggggggggggggefgeeggggggfggggfgggceggggggegggggggfeaeaggdgggWgegegecgegfg
chr22 578161 578262 FREDDYKRUEGER_0001:1:1:871:393#0/2 3 +
TTAGAATAGTGTTGTTTGTATTCGAGTGTTTGTTTTTTATATAGGATTATAGAATATTTTTACGAGTGTTCGAATGTTTGTTTTTTAGATAGGATTTTGGA
^ac]eL]]R_JeeeLegggVbggeeeeVeefceegaggggggggefgdgggggfggggdgggggggggggggggggggegegggggggfgggggggggg
chr22 578102 578203 FREDDYKRUEGER_0001:1:1:871:393#0/1 0 +
AAGAAAGGATTTTAGAATAGTGTTGTTGTGTTTTGAGTGTTTGTTATTTTTGAATGATTTTAGATTATTGTTGTTGGTATTCGAGTGTTTGTTTTTTATAT
cce^aVeaaecdeefOadfegcgg_gdggagggghgggggdggfbgggggbg_fgWgeggWgLgggggeggcgdePa`fffafOe`_egggdg_g_gae
14
http://120.126.44.231/ycl6/20141219_Ian.pptx
http://120.126.44.231/ycl6/20141219_NRPB2014.pdf
Sorting mr file
bsub -q 16G -o stdout -e stderr "
LC_ALL=C sort -S 14G -k 1,1 -k 2,2n -k 3,3n -k 6,6 
-o /work3/s00yao25/h1.chrX.mr.sorted_start 
/work3/NRPB1219/h1.chrX.mr"
15
Remove Duplicates
export PATH=$PATH:/pkg/biology/methpipe/methpipe-3.3.1/bin/
bsub -q 16G -o stdout -e stderr "
duplicate-remover -S /work3/s00yao25/h1.chrX_dremove_stat.txt 
-o /work3/s00yao25/h1.chrX.mr.dremove 
/work3/s00yao25/h1.chrX.mr.sorted_start "
Successfully completed.
Resource usage summary:
CPU time : 167.80 sec.
Max Processes : 3
Max Threads : 4
16
TOTAL READS IN: 24350707
GOOD BASES IN: 1987943796
TOTAL READS OUT: 22884736
GOOD BASES OUT: 1867152730
DUPLICATES REMOVED: 1465971
READS WITH DUPLICATES: 1219174
Estimating bisulfite conversion rate
bsub -q 16G -o stdout -e stderr "
bsrate -c /work3/NRPB1219/hg18 
-o /home/s00yao25/Output/h1.chrX.bsrate 
/work3/s00yao25/h1.chrX.mr.dremove"
17
# head –n 16 /home/s00yao25/Output/h1.chrX.bsrate
OVERALL CONVERSION RATE = 0.980192
POS CONVERSION RATE = 0.980204 96942555
NEG CONVERSION RATE = 0.980179 96821402
BASE PTOT PCONV PRATE NTOT NCONV NRATE BTHTOT BTHCONV BTHRATE ERR ALL ERRRATE
1 1798190 1762518 0.98016 1796291 1760655 0.98016 3594481 3523173 0.98016 36327 3630808 0.01001
2 1654252 1617801 0.97797 1649805 1613025 0.97771 3304057 3230826 0.97784 41299 3345356 0.01235
3 1646403 1615036 0.98095 1644710 1613525 0.98104 3291113 3228561 0.98099 48231 3339344 0.01444
4 1699787 1666286 0.98029 1695105 1662078 0.98052 3394892 3328364 0.98040 50697 3445589 0.01471
5 1663363 1631006 0.98055 1658397 1626045 0.98049 3321760 3257051 0.98052 52464 3374224 0.01555
6 1720978 1687130 0.98033 1716036 1682351 0.98037 3437014 3369481 0.98035 45366 3482380 0.01303
7 1677561 1644979 0.98058 1677119 1644343 0.98046 3354680 3289322 0.98052 53873 3408553 0.01581
8 1714426 1681206 0.98062 1714378 1681339 0.98073 3428804 3362545 0.98068 34491 3463295 0.00996
9 1702891 1668424 0.97976 1700092 1665742 0.97980 3402983 3334166 0.97978 34861 3437844 0.01014
10 1681522 1648092 0.98012 1680471 1647068 0.98012 3361993 3295160 0.98012 45776 3407769 0.01343
11 1664207 1631036 0.98007 1664386 1631083 0.97999 3328593 3262119 0.98003 46055 3374648 0.01365
12 1651326 1618334 0.98002 1649370 1616514 0.98008 3300696 3234848 0.98005 44139 3344835 0.01320
Computing single-site methylation levels
# sorting… again
bsub -q 16G -o stdout -e stderr "
LC_ALL=C sort -S 14G -k 1,1 -k 3,3n -k 2,2n -k 6,6 
-o /work3/s00yao25/h1.chrX.mr.sorted_end_first 
/work3/s00yao25/h1.chrX.mr.dremove"
# methylation calling
bsub -q 16G -o stdout -e stderr "
methcounts -c /work3/NRPB1219/hg18 
-o /work3/s00yao25/h1.chrX.meth 
/work3/s00yao25/h1.chrX.mr.sorted_end_first"
#extract CpG sites
bsub -q 16G -o stdout -e stderr "
symmetric-cpgs 
-o /work3/s00yao25/h1.chrX_CpG.meth h1.chrX.meth"
18
chrX 0 + CHH 0 0
chrX 4 + CHH 0 0
chrX 5 + CHH 0 0
chrX 6 + CHH 0 0
chrX 10 + CHH 0 0
chrX 11 + CHH 0 0
chrX 12 + CHH 0 0
chrX 16 + CHH 0 0
chrX 17 + CHH 0 0
chrX 18 + CHH 0 0
chrX 152 + CpG 0 0
chrX 232 + CpG 0 0
chrX 330 + CpG 0 0
chrX 334 + CpG 0 0
chrX 336 + CpG 0 0
chrX 364 + CpG 0 0
chrX 366 + CpG 0 0
chrX 374 + CpG 0 0
chrX 376 + CpG 0 0
meth ratio read count
Computation of methylation level statistics
bsub -q 16G -o stdout -e stderr "
levels -o /home/s00yao25/Output/h1.chrX.levels 
/work3/s00yao25/h1.chrX.meth"
19
Hypomethylated (hmr), hypermethylated (hypermr), and
partial methylated (pmr) regions
bsub -q 16G -o stdout -e stderr "
hmr -o /work3/s00yao25/h1.chrX.hmr /work3/s00yao25/h1.chrX_CpG.meth"
bsub -q 16G -o stdout -e stderr "
hmr -partial -o /work3/s00yao25/h1.chrX.pmr /work3/s00yao25/h1.chrX_CpG.meth"
bsub -q 16G -o stdout -e stderr "
pmd -o /work3/s00yao25/h1.chrX.pmd /work3/s00yao25/h1.chrX_CpG.meth"
20
chrX 2727656 2728600 HYPO0 18 +
chrX 2731108 2731952 HYPO1 14 +
chrX 2732390 2733303 HYPO2 23 +
chrX 2740632 2740962 HYPO3 9 +
chrX 2756524 2758153 HYPO4 139 +
chrX 2817685 2817980 HYPO5 8 +
chrX 2855757 2857708 HYPO6 127 +
chrX 2890571 2890884 HYPO7 9 +
chrX 3004371 3004626 HYPO8 9 +
chrX 3238227 3238677 HYPO9 9 +
# of CpG
Differential Methylation Analysis
bsub -q 16G -o stdout -e stderr "
methdiff -o /work3/s00yao25/h1.imr90.chrX.methdiff
/work3/NRPB1219/h1.chrX_CpG.meth /work3/NRPB1219/imr90.chrX_CpG.meth"
21
chrX 2709681 + CpG 0.749276 7 2 12 7
chrX 2709727 + CpG 0.917633 4 1 9 12
chrX 2709774 + CpG 0.894737 3 1 6 10
chrX 2709871 + CpG 0.742424 0 16 0 48
chrX 2709890 + CpG 0.857575 3 20 3 47
chrX 2709982 + CpG 0.999354 10 2 7 19
chrX 2710014 + CpG 0.704043 3 6 3 10
chrX 2710023 + CpG 0.600782 4 3 4 4
chrX 2710146 + CpG 0.523077 1 2 8 14
chrX 2710155 + CpG 0.234026 3 3 17 9
Probability
Sample A
Un-meth
Sample A
Meth
Sample B
Un-meth
Sample B
Meth
Differential methylated region
(DMR)
bsub -q 16G -o stdout -e stderr "
dmr /work3/s00yao25/h1.imr90.chrX.methdiff /work3/NRPB1219/h1.chrX.hmr
/work3/NRPB1219/imr90.chrX.hmr DMR_h1_lt_imr90 DMR_imr90_lt_h1"
22
==> DMR_h1_lt_imr90 <==
chrX 2727656 2728600 X:18 10 +
chrX 2731108 2731952 X:15 4 +
chrX 2732390 2733303 X:37 8 +
chrX 2740632 2740962 X:9 0 +
chrX 2758131 2758153 X:3 0 +
chrX 2817685 2817980 X:9 0 +
chrX 2855757 2855890 X:1 1 +
chrX 2890571 2890884 X:9 4 +
chrX 3004371 3004626 X:9 0 +
chrX 3238227 3238677 X:24 0 +
==> DMR_imr90_lt_h1 <==
chrX 2825454 2826947 X:37 17 +
chrX 2857708 2857760 X:2 0 +
chrX 3272822 3273033 X:13 3 +
chrX 3275527 3275594 X:1 0 +
chrX 3287038 3289160 X:36 9 +
chrX 3643168 3643374 X:7 0 +
chrX 4016033 4022054 X:47 29 +
chrX 4028369 4042000 X:79 54 +
chrX 4051286 4059878 X:52 39 +
chrX 4079778 4087714 X:45 26 +
Number of significant differential methylated CpG
Meth. level lower in H1 than IMR90 Meth. level lower in IMR90 than H1

More Related Content

What's hot

LSFMM 2019 BPF Observability
LSFMM 2019 BPF ObservabilityLSFMM 2019 BPF Observability
LSFMM 2019 BPF Observability
Brendan Gregg
 
Performance Wins with BPF: Getting Started
Performance Wins with BPF: Getting StartedPerformance Wins with BPF: Getting Started
Performance Wins with BPF: Getting Started
Brendan Gregg
 
YOW2020 Linux Systems Performance
YOW2020 Linux Systems PerformanceYOW2020 Linux Systems Performance
YOW2020 Linux Systems Performance
Brendan Gregg
 
Performance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux KernelPerformance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux Kernel
lcplcp1
 
BPF Internals (eBPF)
BPF Internals (eBPF)BPF Internals (eBPF)
BPF Internals (eBPF)
Brendan Gregg
 
20170602_OSSummit_an_intelligent_storage
20170602_OSSummit_an_intelligent_storage20170602_OSSummit_an_intelligent_storage
20170602_OSSummit_an_intelligent_storage
Kohei KaiGai
 
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)
Brendan Gregg
 
bcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challengesbcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challenges
IO Visor Project
 
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
Kohei KaiGai
 
pgconfasia2016 plcuda en
pgconfasia2016 plcuda enpgconfasia2016 plcuda en
pgconfasia2016 plcuda en
Kohei KaiGai
 
UM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of SoftwareUM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of Software
Brendan Gregg
 
Hadoop pig
Hadoop pigHadoop pig
Hadoop pig
Wei-Yu Chen
 
Debugging Ruby
Debugging RubyDebugging Ruby
Debugging Ruby
Aman Gupta
 
Debugging Ruby Systems
Debugging Ruby SystemsDebugging Ruby Systems
Debugging Ruby Systems
Engine Yard
 
Extreme Scripting July 2009
Extreme Scripting July 2009Extreme Scripting July 2009
Extreme Scripting July 2009
Ian Foster
 
Kernel Recipes 2017: Using Linux perf at Netflix
Kernel Recipes 2017: Using Linux perf at NetflixKernel Recipes 2017: Using Linux perf at Netflix
Kernel Recipes 2017: Using Linux perf at Netflix
Brendan Gregg
 
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceeBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to Userspace
SUSE Labs Taipei
 
Linux kernel tracing superpowers in the cloud
Linux kernel tracing superpowers in the cloudLinux kernel tracing superpowers in the cloud
Linux kernel tracing superpowers in the cloud
Andrea Righi
 
GPU Accelerated Data Science with RAPIDS - ODSC West 2020
GPU Accelerated Data Science with RAPIDS - ODSC West 2020GPU Accelerated Data Science with RAPIDS - ODSC West 2020
GPU Accelerated Data Science with RAPIDS - ODSC West 2020
John Zedlewski
 
Prashant de-ny-project-s1
Prashant de-ny-project-s1Prashant de-ny-project-s1
Prashant de-ny-project-s1
Prashant Ratnaparkhi
 

What's hot (20)

LSFMM 2019 BPF Observability
LSFMM 2019 BPF ObservabilityLSFMM 2019 BPF Observability
LSFMM 2019 BPF Observability
 
Performance Wins with BPF: Getting Started
Performance Wins with BPF: Getting StartedPerformance Wins with BPF: Getting Started
Performance Wins with BPF: Getting Started
 
YOW2020 Linux Systems Performance
YOW2020 Linux Systems PerformanceYOW2020 Linux Systems Performance
YOW2020 Linux Systems Performance
 
Performance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux KernelPerformance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux Kernel
 
BPF Internals (eBPF)
BPF Internals (eBPF)BPF Internals (eBPF)
BPF Internals (eBPF)
 
20170602_OSSummit_an_intelligent_storage
20170602_OSSummit_an_intelligent_storage20170602_OSSummit_an_intelligent_storage
20170602_OSSummit_an_intelligent_storage
 
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)
 
bcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challengesbcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challenges
 
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
 
pgconfasia2016 plcuda en
pgconfasia2016 plcuda enpgconfasia2016 plcuda en
pgconfasia2016 plcuda en
 
UM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of SoftwareUM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of Software
 
Hadoop pig
Hadoop pigHadoop pig
Hadoop pig
 
Debugging Ruby
Debugging RubyDebugging Ruby
Debugging Ruby
 
Debugging Ruby Systems
Debugging Ruby SystemsDebugging Ruby Systems
Debugging Ruby Systems
 
Extreme Scripting July 2009
Extreme Scripting July 2009Extreme Scripting July 2009
Extreme Scripting July 2009
 
Kernel Recipes 2017: Using Linux perf at Netflix
Kernel Recipes 2017: Using Linux perf at NetflixKernel Recipes 2017: Using Linux perf at Netflix
Kernel Recipes 2017: Using Linux perf at Netflix
 
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceeBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to Userspace
 
Linux kernel tracing superpowers in the cloud
Linux kernel tracing superpowers in the cloudLinux kernel tracing superpowers in the cloud
Linux kernel tracing superpowers in the cloud
 
GPU Accelerated Data Science with RAPIDS - ODSC West 2020
GPU Accelerated Data Science with RAPIDS - ODSC West 2020GPU Accelerated Data Science with RAPIDS - ODSC West 2020
GPU Accelerated Data Science with RAPIDS - ODSC West 2020
 
Prashant de-ny-project-s1
Prashant de-ny-project-s1Prashant de-ny-project-s1
Prashant de-ny-project-s1
 

Similar to 20141219 workshop methylation sequencing analysis

Troubleshooting common oslo.messaging and RabbitMQ issues
Troubleshooting common oslo.messaging and RabbitMQ issuesTroubleshooting common oslo.messaging and RabbitMQ issues
Troubleshooting common oslo.messaging and RabbitMQ issues
Michael Klishin
 
Reproducible Computational Pipelines with Docker and Nextflow
Reproducible Computational Pipelines with Docker and NextflowReproducible Computational Pipelines with Docker and Nextflow
Reproducible Computational Pipelines with Docker and Nextflow
inside-BigData.com
 
BC-Cancer ChimeraScan Presentation
BC-Cancer ChimeraScan PresentationBC-Cancer ChimeraScan Presentation
BC-Cancer ChimeraScan Presentation
Elijah Willie
 
Node Interactive Debugging Node.js In Production
Node Interactive Debugging Node.js In ProductionNode Interactive Debugging Node.js In Production
Node Interactive Debugging Node.js In Production
Yunong Xiao
 
100 bugs in Open Source C/C++ projects
100 bugs in Open Source C/C++ projects 100 bugs in Open Source C/C++ projects
100 bugs in Open Source C/C++ projects
Andrey Karpov
 
sun solaris
sun solarissun solaris
sun solaris
Subur Haryawan
 
|QAB> : Quantum Computing, AI and Blockchain
|QAB> : Quantum Computing, AI and Blockchain|QAB> : Quantum Computing, AI and Blockchain
|QAB> : Quantum Computing, AI and Blockchain
Kan Yuenyong
 
Trying and evaluating the new features of GlusterFS 3.5
Trying and evaluating the new features of GlusterFS 3.5Trying and evaluating the new features of GlusterFS 3.5
Trying and evaluating the new features of GlusterFS 3.5
Keisuke Takahashi
 
DevoxxUK: Optimizating Application Performance on Kubernetes
DevoxxUK: Optimizating Application Performance on KubernetesDevoxxUK: Optimizating Application Performance on Kubernetes
DevoxxUK: Optimizating Application Performance on Kubernetes
Dinakar Guniguntala
 
Stress your DUT
Stress your DUTStress your DUT
Stress your DUT
Redge Technologies
 
Introduction to LAVA Workload Scheduler
Introduction to LAVA Workload SchedulerIntroduction to LAVA Workload Scheduler
Introduction to LAVA Workload Scheduler
Nopparat Nopkuat
 
More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...
More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...
More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...
Databricks
 
Presto anatomy
Presto anatomyPresto anatomy
Presto anatomy
Dongmin Yu
 
Pasteur deep seq_analysis_theory_2016
Pasteur deep seq_analysis_theory_2016Pasteur deep seq_analysis_theory_2016
Pasteur deep seq_analysis_theory_2016
Christophe Antoniewski
 
Picking Mushrooms after Cppcheck
Picking Mushrooms after CppcheckPicking Mushrooms after Cppcheck
Picking Mushrooms after Cppcheck
Andrey Karpov
 
Oracle cluster installation with grid and iscsi
Oracle cluster  installation with grid and iscsiOracle cluster  installation with grid and iscsi
Oracle cluster installation with grid and iscsi
Chanaka Lasantha
 
Bioinfo ngs data format visualization v2
Bioinfo ngs data format visualization v2Bioinfo ngs data format visualization v2
Bioinfo ngs data format visualization v2
Li Shen
 
[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기
NAVER D2
 
20211119 ntuh azure hpc workshop final
20211119 ntuh azure hpc workshop final20211119 ntuh azure hpc workshop final
20211119 ntuh azure hpc workshop final
Meng-Ru (Raymond) Tsai
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
Workhorse Computing
 

Similar to 20141219 workshop methylation sequencing analysis (20)

Troubleshooting common oslo.messaging and RabbitMQ issues
Troubleshooting common oslo.messaging and RabbitMQ issuesTroubleshooting common oslo.messaging and RabbitMQ issues
Troubleshooting common oslo.messaging and RabbitMQ issues
 
Reproducible Computational Pipelines with Docker and Nextflow
Reproducible Computational Pipelines with Docker and NextflowReproducible Computational Pipelines with Docker and Nextflow
Reproducible Computational Pipelines with Docker and Nextflow
 
BC-Cancer ChimeraScan Presentation
BC-Cancer ChimeraScan PresentationBC-Cancer ChimeraScan Presentation
BC-Cancer ChimeraScan Presentation
 
Node Interactive Debugging Node.js In Production
Node Interactive Debugging Node.js In ProductionNode Interactive Debugging Node.js In Production
Node Interactive Debugging Node.js In Production
 
100 bugs in Open Source C/C++ projects
100 bugs in Open Source C/C++ projects 100 bugs in Open Source C/C++ projects
100 bugs in Open Source C/C++ projects
 
sun solaris
sun solarissun solaris
sun solaris
 
|QAB> : Quantum Computing, AI and Blockchain
|QAB> : Quantum Computing, AI and Blockchain|QAB> : Quantum Computing, AI and Blockchain
|QAB> : Quantum Computing, AI and Blockchain
 
Trying and evaluating the new features of GlusterFS 3.5
Trying and evaluating the new features of GlusterFS 3.5Trying and evaluating the new features of GlusterFS 3.5
Trying and evaluating the new features of GlusterFS 3.5
 
DevoxxUK: Optimizating Application Performance on Kubernetes
DevoxxUK: Optimizating Application Performance on KubernetesDevoxxUK: Optimizating Application Performance on Kubernetes
DevoxxUK: Optimizating Application Performance on Kubernetes
 
Stress your DUT
Stress your DUTStress your DUT
Stress your DUT
 
Introduction to LAVA Workload Scheduler
Introduction to LAVA Workload SchedulerIntroduction to LAVA Workload Scheduler
Introduction to LAVA Workload Scheduler
 
More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...
More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...
More Algorithms and Tools for Genomic Analysis on Apache Spark with Ryan Will...
 
Presto anatomy
Presto anatomyPresto anatomy
Presto anatomy
 
Pasteur deep seq_analysis_theory_2016
Pasteur deep seq_analysis_theory_2016Pasteur deep seq_analysis_theory_2016
Pasteur deep seq_analysis_theory_2016
 
Picking Mushrooms after Cppcheck
Picking Mushrooms after CppcheckPicking Mushrooms after Cppcheck
Picking Mushrooms after Cppcheck
 
Oracle cluster installation with grid and iscsi
Oracle cluster  installation with grid and iscsiOracle cluster  installation with grid and iscsi
Oracle cluster installation with grid and iscsi
 
Bioinfo ngs data format visualization v2
Bioinfo ngs data format visualization v2Bioinfo ngs data format visualization v2
Bioinfo ngs data format visualization v2
 
[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기
 
20211119 ntuh azure hpc workshop final
20211119 ntuh azure hpc workshop final20211119 ntuh azure hpc workshop final
20211119 ntuh azure hpc workshop final
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
 

Recently uploaded

Competition and Regulation in Professions and Occupations – ROBSON – June 202...
Competition and Regulation in Professions and Occupations – ROBSON – June 202...Competition and Regulation in Professions and Occupations – ROBSON – June 202...
Competition and Regulation in Professions and Occupations – ROBSON – June 202...
OECD Directorate for Financial and Enterprise Affairs
 
Pro-competitive Industrial Policy – LANE – June 2024 OECD discussion
Pro-competitive Industrial Policy – LANE – June 2024 OECD discussionPro-competitive Industrial Policy – LANE – June 2024 OECD discussion
Pro-competitive Industrial Policy – LANE – June 2024 OECD discussion
OECD Directorate for Financial and Enterprise Affairs
 
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...
Suzanne Lagerweij
 
Pro-competitive Industrial Policy – OECD – June 2024 OECD discussion
Pro-competitive Industrial Policy – OECD – June 2024 OECD discussionPro-competitive Industrial Policy – OECD – June 2024 OECD discussion
Pro-competitive Industrial Policy – OECD – June 2024 OECD discussion
OECD Directorate for Financial and Enterprise Affairs
 
Disaster Management project for holidays homework and other uses
Disaster Management project for holidays homework and other usesDisaster Management project for holidays homework and other uses
Disaster Management project for holidays homework and other uses
RIDHIMAGARG21
 
Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...
Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...
Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...
OECD Directorate for Financial and Enterprise Affairs
 
Artificial Intelligence, Data and Competition – LIM – June 2024 OECD discussion
Artificial Intelligence, Data and Competition – LIM – June 2024 OECD discussionArtificial Intelligence, Data and Competition – LIM – June 2024 OECD discussion
Artificial Intelligence, Data and Competition – LIM – June 2024 OECD discussion
OECD Directorate for Financial and Enterprise Affairs
 
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
OECD Directorate for Financial and Enterprise Affairs
 
XP 2024 presentation: A New Look to Leadership
XP 2024 presentation: A New Look to LeadershipXP 2024 presentation: A New Look to Leadership
XP 2024 presentation: A New Look to Leadership
samililja
 
IEEE CIS Webinar Sustainable futures.pdf
IEEE CIS Webinar Sustainable futures.pdfIEEE CIS Webinar Sustainable futures.pdf
IEEE CIS Webinar Sustainable futures.pdf
Claudio Gallicchio
 
ASONAM2023_presection_slide_track-recommendation.pdf
ASONAM2023_presection_slide_track-recommendation.pdfASONAM2023_presection_slide_track-recommendation.pdf
ASONAM2023_presection_slide_track-recommendation.pdf
ToshihiroIto4
 
The remarkable life of Sir Mokshagundam Visvesvaraya.pptx
The remarkable life of Sir Mokshagundam Visvesvaraya.pptxThe remarkable life of Sir Mokshagundam Visvesvaraya.pptx
The remarkable life of Sir Mokshagundam Visvesvaraya.pptx
JiteshKumarChoudhary2
 
Why Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdf
Why Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdfWhy Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdf
Why Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdf
Ben Linders
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
 
原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样
原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样
原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样
gpww3sf4
 
The Intersection between Competition and Data Privacy – COLANGELO – June 2024...
The Intersection between Competition and Data Privacy – COLANGELO – June 2024...The Intersection between Competition and Data Privacy – COLANGELO – June 2024...
The Intersection between Competition and Data Privacy – COLANGELO – June 2024...
OECD Directorate for Financial and Enterprise Affairs
 
BRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdf
BRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdfBRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdf
BRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdf
Robin Haunschild
 
The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...
The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...
The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...
OECD Directorate for Financial and Enterprise Affairs
 
The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...
The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...
The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...
OECD Directorate for Financial and Enterprise Affairs
 
The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...
The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...
The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...
OECD Directorate for Financial and Enterprise Affairs
 

Recently uploaded (20)

Competition and Regulation in Professions and Occupations – ROBSON – June 202...
Competition and Regulation in Professions and Occupations – ROBSON – June 202...Competition and Regulation in Professions and Occupations – ROBSON – June 202...
Competition and Regulation in Professions and Occupations – ROBSON – June 202...
 
Pro-competitive Industrial Policy – LANE – June 2024 OECD discussion
Pro-competitive Industrial Policy – LANE – June 2024 OECD discussionPro-competitive Industrial Policy – LANE – June 2024 OECD discussion
Pro-competitive Industrial Policy – LANE – June 2024 OECD discussion
 
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...
 
Pro-competitive Industrial Policy – OECD – June 2024 OECD discussion
Pro-competitive Industrial Policy – OECD – June 2024 OECD discussionPro-competitive Industrial Policy – OECD – June 2024 OECD discussion
Pro-competitive Industrial Policy – OECD – June 2024 OECD discussion
 
Disaster Management project for holidays homework and other uses
Disaster Management project for holidays homework and other usesDisaster Management project for holidays homework and other uses
Disaster Management project for holidays homework and other uses
 
Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...
Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...
Artificial Intelligence, Data and Competition – ČORBA – June 2024 OECD discus...
 
Artificial Intelligence, Data and Competition – LIM – June 2024 OECD discussion
Artificial Intelligence, Data and Competition – LIM – June 2024 OECD discussionArtificial Intelligence, Data and Competition – LIM – June 2024 OECD discussion
Artificial Intelligence, Data and Competition – LIM – June 2024 OECD discussion
 
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
Competition and Regulation in Professions and Occupations – OECD – June 2024 ...
 
XP 2024 presentation: A New Look to Leadership
XP 2024 presentation: A New Look to LeadershipXP 2024 presentation: A New Look to Leadership
XP 2024 presentation: A New Look to Leadership
 
IEEE CIS Webinar Sustainable futures.pdf
IEEE CIS Webinar Sustainable futures.pdfIEEE CIS Webinar Sustainable futures.pdf
IEEE CIS Webinar Sustainable futures.pdf
 
ASONAM2023_presection_slide_track-recommendation.pdf
ASONAM2023_presection_slide_track-recommendation.pdfASONAM2023_presection_slide_track-recommendation.pdf
ASONAM2023_presection_slide_track-recommendation.pdf
 
The remarkable life of Sir Mokshagundam Visvesvaraya.pptx
The remarkable life of Sir Mokshagundam Visvesvaraya.pptxThe remarkable life of Sir Mokshagundam Visvesvaraya.pptx
The remarkable life of Sir Mokshagundam Visvesvaraya.pptx
 
Why Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdf
Why Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdfWhy Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdf
Why Psychological Safety Matters for Software Teams - ACE 2024 - Ben Linders.pdf
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
 
原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样
原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样
原版制作贝德福特大学毕业证(bedfordhire毕业证)硕士文凭原版一模一样
 
The Intersection between Competition and Data Privacy – COLANGELO – June 2024...
The Intersection between Competition and Data Privacy – COLANGELO – June 2024...The Intersection between Competition and Data Privacy – COLANGELO – June 2024...
The Intersection between Competition and Data Privacy – COLANGELO – June 2024...
 
BRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdf
BRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdfBRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdf
BRIC_2024_2024-06-06-11:30-haunschild_archival_version.pdf
 
The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...
The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...
The Intersection between Competition and Data Privacy – OECD – June 2024 OECD...
 
The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...
The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...
The Intersection between Competition and Data Privacy – KEMP – June 2024 OECD...
 
The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...
The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...
The Intersection between Competition and Data Privacy – CAPEL – June 2024 OEC...
 

20141219 workshop methylation sequencing analysis

  • 1. Hands-On Training in Methylation Sequencing Analysis Time Topic Speaker 09:00~10:30 WGBS Dataset QC & Mapping 張益峰 博士 l 測試資料簡介(Human IMR90 以及 H1 的 WGBS Dataset) l 執行 short read QC l meth-pipe 簡介(網站、下載、安裝) l 執行 meth-pipe 進行 short read mapping 和 mapped raw data 處理 l 計算 bisulfide conversion rate l 計算 HMR / DMR / PMD l 產生 BED 格式的檔案 10:45~11:00 Q & A / 休息 11:00~12:30 Basic WGBS Analysis and UCSC Genome Browser 林依璿 博士 l BEDtools 的簡介 l BEDtools 實做(一):HMR / DMR / PMD 在Chromosome X 上的分佈 l BEDtools 實做(二):CpG Island 的 DNA methylation 計算 l BEDtools 實做(三):比較 male (H1) & female (IMR90) 在 Chromosome X 上的 methylation 差異 l 用 R 以及 ggplot2 package 呈現以上的 methylation 差異 l UCSC Genome Browser 介紹和使用操作 l UCSC Genome Browser 裡面的 MethBase Public TrackHub 介紹和使用操作 12:30~13:00 Q & A / 午餐 This slide is available in http://www.slideshare.net/YiFengChang 1
  • 2. Required Software in Your Laptop • Linux console • Putty: http://the.earth.li/~sgtatham/putty/lat est/x86/putty.exe • SCP/SFTP/FTP client • Winscp: http://winscp.net/download/winscp55 6.zip • PDF viewer • http://get.adobe.com/tw/reader/ 2 http://120.126.44.231/ycl6/20141219_Ian.pptx http://120.126.44.231/ycl6/20141219_NRPB2014.pdf
  • 3. NCHC ALPS1 for NRPB Users• Login node: alps1.nchc.org.tw • Computing nodes • 48 (we can use 4) x 48 cores 128GB RAM • 1 x 64 cores 1TB RAM • Storages • Users home: 200GB (Temp Account: 1GB) • /work3: 42TB • /work5: 200TB This slide is available in http://www.slideshare.net/YiFengChang Queue Name Test[註一] 4G 16G 48G 128G 192core 384core 1T 使用記憶體上限 (GB) 2 4 16 48 100 100/48core 100/48core 1024 使用核心數上限 1 2 8 24 48 192 384 64 建議核心使用上限 1 1 6 20 40 192 384 60 工作優先權[註二] 90 85 80 50 30 20 20 10 什麼是 queue? Queue 就是針對工作所設立的虛擬的運算單元,一個 queue 可以負擔一個運算 工作。如果一臺機器被指定兩個 queue,意味這台機器可能同時運行兩個工作。 Queue 的設計是用來管理計算資源 。 •所有的工作必須透過 queuing system 執行 •目前提供Test, 4G, 16G, 48G, 128G,192core,384core, 1T 共有 五 種 queue •工作優先順序以Test> 4G>16G>48G>128G>192core>384core> 1T •記憶體使用大於上限者,將會被中斷運算 •可先使用1T queue試運算,決定記憶體用量後再選擇使用何種 queue •使用queue時的限制與規則如下: 註一 ,test queue一人只能送一個job,執行時間為10分鐘 註二 ,數字愈大優先權愈高 網中心提供之Queuing system 國網中心提供 IBM Load Sharing Facility (LSF) 目前可以使用的 Queue 3
  • 4. Submit a Job s00yao25@alps1:~> cd s00yao25@alps1:~> bsub -q 4G -o stdout -e stderr "ls" Job <422673> is submitted to queue <4G>. s00yao25@alps1:~> cat stdout s00yao25@alps1:~> bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 422673 s00yao2 RUN 4G alps1 2*alps1-25 ls Dec 18 23:16 s00yao25@alps1:~> bjobs No unfinished job found 4
  • 6. Kill a Job >bkill JOBID # chang queue priority of pending jobs > btop 6
  • 7. Effect and Problems of Bisulfite Treatment of DNA 7 Krueger, F., Kreck, B., Franke, A. & Andrews, S.R. DNA methylome analysis using short bisulfite sequencing data. Nat Methods 9, 145-51 (2012). Mapping bisulfite reads to 4 possible bisulfite strands (OT/CTOT/OB/CTOB) is equivalent to mapping the bisulfite read and its reverse complementary read to both Top/Bottom strands of the original reference sequence.
  • 8. How to Align BS Reads Against Reference Genome? 8 Krueger, F. & Andrews, S.R. Bismark: A flexible aligner and methylation caller for Bisulfite-Seq applications. Bioinformatics (2011). Bock, C. Analysing and interpreting DNA methylation data. Nat Rev Genet 13, 705-19 (2012) Y=C or T TCGA TCGT ACGT ATGA Multiple hits TTGT ATGT Multiple hits TCGA ATGA
  • 10. Analysis Pipeline 10 Allele-specific Methylated Regions amrfinder allelicmeth Differential Methylation Region dmr Large Hypo/Hyper-Methylation Domains pmd Hypo/Hyper-Methylation Regions hmr hyperhmr pmr Methylation Calling methcounts + error correction Bisulfite Conversion Rate bsrate Remove Duplicate Reads duplicate-remover Mapping rmapbs rmapbs-pe Quality Trimming fastq_masker Cross-species Comparison of Methylomes liftOver Calculating Methylation Ratio for Regions bigWigAverageOverBed roimethstat Bwtools Generate Methylation BED file Bedtools bedGraphToBigWig fastx toolkit: http://hannonlab.cshl.edu/fastx_toolkit/ MethPipe: http://smithlabresearch.org/software/methpipe/ Bedtools: https://github.com/arq5x/bedtools2 Programs from UCSC Genome Browser: http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64 bwtool: https://github.com/CRG-Barcelona/bwtool/wiki
  • 11. H1 (male): human embryonic stem cells (107GB) IMR90 (female): fetal lung fibroblasts (154GB) http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE16256 11
  • 12. Convert SRA to Fastq (DO NOT RUN) sra-toolkit: https://github.com/ncbi/sratoolkit > fastq-dump --split-3 SRR018975.sra > ls SRR018975.fastq 12
  • 13. Quality Trimming (DO NOT RUN) #e.g. SRR018975.fastq.gz > for f in *.gz; # read all gzip files one by one do b=`basename $f .gz`; # SRR018975.fastq echo $f bsub -q 4G -o $f.stdout -e $f.stderr " gzip -dc $f| # read gzip file fastq_masker -q 30 -Q33| # mask low quality reads as Ns split -dl 6000000 - $b- "; # split fastq file into smaller ones done > ls SRR018975.fastq-00 SRR018975.fastq-01 SRR018975.fastq-02 … 13
  • 14. Mapping bsub -q 4G -o rmapbs.stdout -e rmapbs.stderr " /work3/NRPB1219/bin/rmapbs-pe -c /work3/NRPB1219/methpipe-data/data/genome -o /home/s00yao00/Output/test.mr -m 3 -L 400 -C AGATCGGAAGAGC:GCTCTTCCGATCT /work3/NRPB1219/methpipe-data/data/snippet_1.fq /work3/NRPB1219/methpipe-data/data/snippet_2.fq" #head /home/s00yao00/Output/test.mr chr22 379487 379588 FREDDYKRUEGER_0001:1:1:160:969#0/2 2 + AAGTAATATATATGTTTTGGGATTAGTGAATTTAAGTTAGTATTAAGAATTTTATTATTATTTTTTTATTATATTTTAGAGAGTTATTTTTTTATTTTTAA B_c^cbbPfbfbLc]][[Xcbcc`cbd`bbcbdZUdbfOffffdffdSSTTQbZbbb`df[bbbbbdffYffffffbbbZbZX^^afffffcff chr22 379487 379588 FREDDYKRUEGER_0001:1:1:160:969#0/1 3 + AAGTAATATTTATGTTTTGGGATTAGTGAATTTAAGTTAGTATTATGAATTTTATTATTATTTTTTTATTATATTTTAGAGAGTTATTTTTTTCTTTTTAA `dd``addddbcUcccdcfffaf_ZdddOVM_[bOZdb`dbffZ`ObObffaeff^ffdfffffffaffdWf_bdebdaU[^[ffa_f_b`OUK^`UZT chr22 568970 569071 FREDDYKRUEGER_0001:1:1:249:1215#0/2 0 - TTTTATTTGATGGATAATATTAAGAAATTTGTAGTATTGTTTTGGAATTTTTTGTGAGGGATAAATAAATAGAATATAGTAGTATTGTTTTATAATTTTTT BcXe^cecccgbabfadadcf^cgggegTggefgeggggggggggggggggggggggggegaggg^gggbggggggggggggegggggggggfggfcdggg chr22 310957 311058 FREDDYKRUEGER_0001:1:1:303:856#0/2 4 + TTGTAATTATGTTGATTTTATGTGTAGTTATTGATGTTGTTGTATGGTAGTTTATGGTTTTTTAGGAATTTAGAATTTGAGTTTTATTTTTGTTTTATAGT Z]]`W[J`]fdPcefbf^fgggggWcggaeaedSSOQTcggdgcgggcdgffdeaccaddcadfbacfaffcaaecadggbdggggcgcgggggccg chr22 568970 569071 FREDDYKRUEGER_0001:1:1:249:1215#0/1 2 + AAAGGATTGTAGAATAGTGTTATTGTGTTTTGTTTGTTTGTTTTTTATAAAGGATTTTAGAGTAGTGTTGTAGATTTTTTAGTGTTGTTTATTAGATAGGT ggagcfggdgggbegffgfgggggggggfgggggggggggggfggggfcf^d^Pfggggebafbbgfgfgge^^ggggggccggfegefggfggeP^ZW[ chr22 581983 582084 FREDDYKRUEGER_0001:1:1:359:1280#0/2 2 + GAGGTAATTTAGAGTGTTGTTTTTGGTTTTTGAGGGTTTGTTTTTTAAATAGGATATTATTATATTGTTACGATAGTTTGAATGTTTGTTCGTGATAAATG Bffeeegg_fgeegggeeeeagcggggfgaggggggggggggggggggggfgffdeggggggfgfgggffgggffgggggfggfgggggbgbggggggcf chr22 310918 311019 FREDDYKRUEGER_0001:1:1:303:856#0/1 0 + TATATAGAGTTAGGTTTTATAGTTTATTTTTTTATTATTTTGTAATTTTGTTAATTTTATGTTTAGTTATTGATGTTTTTGTATGGTAGTTTATGGTTTTT faf_f_ggeggcg^ggggbgdggggcgggggggdggdggggggg^ggggggdg[ggggdggcgdccggagggNggcdgggcbYddTcKRcde[ddYgcggg chr22 581975 582076 FREDDYKRUEGER_0001:1:1:359:1280#0/1 1 + TATTTTATAAGGTAATTTAGAGTGTTGTTTTTGGTTTTTGAGGGTTTGTTTTTTAAATAGGATATTATTATATTGTTACGATAGTTTGAATGTTTGTTCGT hggggfggghggegggggcg]gggggggggggggggggggefgeeggggggfggggfgggceggggggegggggggfeaeaggdgggWgegegecgegfg chr22 578161 578262 FREDDYKRUEGER_0001:1:1:871:393#0/2 3 + TTAGAATAGTGTTGTTTGTATTCGAGTGTTTGTTTTTTATATAGGATTATAGAATATTTTTACGAGTGTTCGAATGTTTGTTTTTTAGATAGGATTTTGGA ^ac]eL]]R_JeeeLegggVbggeeeeVeefceegaggggggggefgdgggggfggggdgggggggggggggggggggegegggggggfgggggggggg chr22 578102 578203 FREDDYKRUEGER_0001:1:1:871:393#0/1 0 + AAGAAAGGATTTTAGAATAGTGTTGTTGTGTTTTGAGTGTTTGTTATTTTTGAATGATTTTAGATTATTGTTGTTGGTATTCGAGTGTTTGTTTTTTATAT cce^aVeaaecdeefOadfegcgg_gdggagggghgggggdggfbgggggbg_fgWgeggWgLgggggeggcgdePa`fffafOe`_egggdg_g_gae 14 http://120.126.44.231/ycl6/20141219_Ian.pptx http://120.126.44.231/ycl6/20141219_NRPB2014.pdf
  • 15. Sorting mr file bsub -q 16G -o stdout -e stderr " LC_ALL=C sort -S 14G -k 1,1 -k 2,2n -k 3,3n -k 6,6 -o /work3/s00yao25/h1.chrX.mr.sorted_start /work3/NRPB1219/h1.chrX.mr" 15
  • 16. Remove Duplicates export PATH=$PATH:/pkg/biology/methpipe/methpipe-3.3.1/bin/ bsub -q 16G -o stdout -e stderr " duplicate-remover -S /work3/s00yao25/h1.chrX_dremove_stat.txt -o /work3/s00yao25/h1.chrX.mr.dremove /work3/s00yao25/h1.chrX.mr.sorted_start " Successfully completed. Resource usage summary: CPU time : 167.80 sec. Max Processes : 3 Max Threads : 4 16 TOTAL READS IN: 24350707 GOOD BASES IN: 1987943796 TOTAL READS OUT: 22884736 GOOD BASES OUT: 1867152730 DUPLICATES REMOVED: 1465971 READS WITH DUPLICATES: 1219174
  • 17. Estimating bisulfite conversion rate bsub -q 16G -o stdout -e stderr " bsrate -c /work3/NRPB1219/hg18 -o /home/s00yao25/Output/h1.chrX.bsrate /work3/s00yao25/h1.chrX.mr.dremove" 17 # head –n 16 /home/s00yao25/Output/h1.chrX.bsrate OVERALL CONVERSION RATE = 0.980192 POS CONVERSION RATE = 0.980204 96942555 NEG CONVERSION RATE = 0.980179 96821402 BASE PTOT PCONV PRATE NTOT NCONV NRATE BTHTOT BTHCONV BTHRATE ERR ALL ERRRATE 1 1798190 1762518 0.98016 1796291 1760655 0.98016 3594481 3523173 0.98016 36327 3630808 0.01001 2 1654252 1617801 0.97797 1649805 1613025 0.97771 3304057 3230826 0.97784 41299 3345356 0.01235 3 1646403 1615036 0.98095 1644710 1613525 0.98104 3291113 3228561 0.98099 48231 3339344 0.01444 4 1699787 1666286 0.98029 1695105 1662078 0.98052 3394892 3328364 0.98040 50697 3445589 0.01471 5 1663363 1631006 0.98055 1658397 1626045 0.98049 3321760 3257051 0.98052 52464 3374224 0.01555 6 1720978 1687130 0.98033 1716036 1682351 0.98037 3437014 3369481 0.98035 45366 3482380 0.01303 7 1677561 1644979 0.98058 1677119 1644343 0.98046 3354680 3289322 0.98052 53873 3408553 0.01581 8 1714426 1681206 0.98062 1714378 1681339 0.98073 3428804 3362545 0.98068 34491 3463295 0.00996 9 1702891 1668424 0.97976 1700092 1665742 0.97980 3402983 3334166 0.97978 34861 3437844 0.01014 10 1681522 1648092 0.98012 1680471 1647068 0.98012 3361993 3295160 0.98012 45776 3407769 0.01343 11 1664207 1631036 0.98007 1664386 1631083 0.97999 3328593 3262119 0.98003 46055 3374648 0.01365 12 1651326 1618334 0.98002 1649370 1616514 0.98008 3300696 3234848 0.98005 44139 3344835 0.01320
  • 18. Computing single-site methylation levels # sorting… again bsub -q 16G -o stdout -e stderr " LC_ALL=C sort -S 14G -k 1,1 -k 3,3n -k 2,2n -k 6,6 -o /work3/s00yao25/h1.chrX.mr.sorted_end_first /work3/s00yao25/h1.chrX.mr.dremove" # methylation calling bsub -q 16G -o stdout -e stderr " methcounts -c /work3/NRPB1219/hg18 -o /work3/s00yao25/h1.chrX.meth /work3/s00yao25/h1.chrX.mr.sorted_end_first" #extract CpG sites bsub -q 16G -o stdout -e stderr " symmetric-cpgs -o /work3/s00yao25/h1.chrX_CpG.meth h1.chrX.meth" 18 chrX 0 + CHH 0 0 chrX 4 + CHH 0 0 chrX 5 + CHH 0 0 chrX 6 + CHH 0 0 chrX 10 + CHH 0 0 chrX 11 + CHH 0 0 chrX 12 + CHH 0 0 chrX 16 + CHH 0 0 chrX 17 + CHH 0 0 chrX 18 + CHH 0 0 chrX 152 + CpG 0 0 chrX 232 + CpG 0 0 chrX 330 + CpG 0 0 chrX 334 + CpG 0 0 chrX 336 + CpG 0 0 chrX 364 + CpG 0 0 chrX 366 + CpG 0 0 chrX 374 + CpG 0 0 chrX 376 + CpG 0 0 meth ratio read count
  • 19. Computation of methylation level statistics bsub -q 16G -o stdout -e stderr " levels -o /home/s00yao25/Output/h1.chrX.levels /work3/s00yao25/h1.chrX.meth" 19
  • 20. Hypomethylated (hmr), hypermethylated (hypermr), and partial methylated (pmr) regions bsub -q 16G -o stdout -e stderr " hmr -o /work3/s00yao25/h1.chrX.hmr /work3/s00yao25/h1.chrX_CpG.meth" bsub -q 16G -o stdout -e stderr " hmr -partial -o /work3/s00yao25/h1.chrX.pmr /work3/s00yao25/h1.chrX_CpG.meth" bsub -q 16G -o stdout -e stderr " pmd -o /work3/s00yao25/h1.chrX.pmd /work3/s00yao25/h1.chrX_CpG.meth" 20 chrX 2727656 2728600 HYPO0 18 + chrX 2731108 2731952 HYPO1 14 + chrX 2732390 2733303 HYPO2 23 + chrX 2740632 2740962 HYPO3 9 + chrX 2756524 2758153 HYPO4 139 + chrX 2817685 2817980 HYPO5 8 + chrX 2855757 2857708 HYPO6 127 + chrX 2890571 2890884 HYPO7 9 + chrX 3004371 3004626 HYPO8 9 + chrX 3238227 3238677 HYPO9 9 + # of CpG
  • 21. Differential Methylation Analysis bsub -q 16G -o stdout -e stderr " methdiff -o /work3/s00yao25/h1.imr90.chrX.methdiff /work3/NRPB1219/h1.chrX_CpG.meth /work3/NRPB1219/imr90.chrX_CpG.meth" 21 chrX 2709681 + CpG 0.749276 7 2 12 7 chrX 2709727 + CpG 0.917633 4 1 9 12 chrX 2709774 + CpG 0.894737 3 1 6 10 chrX 2709871 + CpG 0.742424 0 16 0 48 chrX 2709890 + CpG 0.857575 3 20 3 47 chrX 2709982 + CpG 0.999354 10 2 7 19 chrX 2710014 + CpG 0.704043 3 6 3 10 chrX 2710023 + CpG 0.600782 4 3 4 4 chrX 2710146 + CpG 0.523077 1 2 8 14 chrX 2710155 + CpG 0.234026 3 3 17 9 Probability Sample A Un-meth Sample A Meth Sample B Un-meth Sample B Meth
  • 22. Differential methylated region (DMR) bsub -q 16G -o stdout -e stderr " dmr /work3/s00yao25/h1.imr90.chrX.methdiff /work3/NRPB1219/h1.chrX.hmr /work3/NRPB1219/imr90.chrX.hmr DMR_h1_lt_imr90 DMR_imr90_lt_h1" 22 ==> DMR_h1_lt_imr90 <== chrX 2727656 2728600 X:18 10 + chrX 2731108 2731952 X:15 4 + chrX 2732390 2733303 X:37 8 + chrX 2740632 2740962 X:9 0 + chrX 2758131 2758153 X:3 0 + chrX 2817685 2817980 X:9 0 + chrX 2855757 2855890 X:1 1 + chrX 2890571 2890884 X:9 4 + chrX 3004371 3004626 X:9 0 + chrX 3238227 3238677 X:24 0 + ==> DMR_imr90_lt_h1 <== chrX 2825454 2826947 X:37 17 + chrX 2857708 2857760 X:2 0 + chrX 3272822 3273033 X:13 3 + chrX 3275527 3275594 X:1 0 + chrX 3287038 3289160 X:36 9 + chrX 3643168 3643374 X:7 0 + chrX 4016033 4022054 X:47 29 + chrX 4028369 4042000 X:79 54 + chrX 4051286 4059878 X:52 39 + chrX 4079778 4087714 X:45 26 + Number of significant differential methylated CpG Meth. level lower in H1 than IMR90 Meth. level lower in IMR90 than H1