Genomes On Rails

Slide 1: Genomes on Rails has_many :sequences

Slide 2: Hello

Slide 3: ➊ Previously ➋ Production ➌ Process

Slide 4: ➊ Previously

Slide 5: The human genome 15 years to decode 3 billion letters

Slide 6: $3 billion

Slide 7: $3 billion ++

Slide 8: Race for the prize

Slide 11: Open data

Slide 12: Open source

Slide 13: Perl

Slide 14: Lots of Perl

Slide 15: Lots of Perl ~4500 modules

Slide 16: Onwards!

Slide 17: 40 species

Slide 21: Map evolutionary space

Slide 22: Compare genomes

Slide 23: compare species Compare genomes

Slide 24: compare species Compare genomes compa re indi viduals

Slide 25: More Perl ~1500 modules

Slide 29: Quantum leap!

Slide 30: 1000 personal genomes

Slide 31: beyond 23andme 1000 personal genomes

Slide 32: Hypertension

Slide 33: Diabetes

Slide 34: Coronary heart disease

Slide 35: Bipolar disorder

Slide 36: Malaria

Slide 37: ➋ Production

Slide 38: Register projects Register samples Sample prep Sequencing Analysis

Slide 42: Change!

Slide 43: Flexible data capture

Slide 44: Virtual fields

Slide 45: Sample Name Organism Concentration

Slide 46: class Sample < ActiveRecord::Base has_many :descriptors has_many :descriptor_values end

Slide 47: Key value pairs

Slide 48: Faster than you’d think

Slide 51: Change!

Slide 52: V1 V2 Sample Sample Name Name Organism Organism Concentration Concentration Origin Quality metric

Slide 55: Rationalize!

Slide 56: V1 V2 Sample Sample Name Name Organism Organism Concentration Concentration Origin Quality metric

Slide 57: Mapping!

Slide 58: V1 V3 Sample Sample Name Name Organism Species Concentration Concentration Origin Origin Quality metric

Slide 59: Pipeline management

Slide 60: Workflow Task 1 Task 2 Task 3 Name Name Name Operator Serial number Passed Instrument Kit

Slide 64: Throughput!

Slide 66: 320Tb 450 CPU

Slide 67: 320Tb 450 CPU Archive

Slide 68: 75 Tb

Slide 73: pilot study!

Slide 74: Multiple apps

Slide 75: Multiple instances

Slide 76: Loosely coupled

Slide 77: Loose coupling is hard

Slide 78: Deployment

Slide 79: Maintenance

Slide 80: Monitoring

Slide 81: Hard to maintain separation

Slide 82: Support novel science

Slide 83: Single code base

Slide 84: nginx reverse proxy

Slide 85: fairnginx

Slide 86: Mongrel

Slide 87: Fast deployment

Slide 88: Automate everything

Slide 90: Play well with others! Interoperability!

Slide 91: Legacy databases

Slide 92: RESTful services

Slide 93: Generate API stubs

Slide 95: SCALE!

Slide 96: Trillionics

Slide 97: 2 X

Slide 98: 150Tb per week

Slide 99: Over 6 months

Slide 100: More hardware

Slide 101: 400 additional nodes

Slide 102: additional 360 Tb

Slide 103: Towards a Virtual Institute

Slide 104: Lots of data

Slide 105: Lots of data, lots of people

Slide 106: Lots of data, lots of people, lots of compute

Slide 107: Lots of data, lots of people, lots of compute, lots of uses

Slide 108: Lots of data, lots of people, lots of compute, lots of uses, lots and lots and lots and lots...

Slide 109: ➌ Process

Slide 110: Concept Requirements Development Product

Slide 111: takes too lon g Concept Requirements Development Product

Slide 112: takes too lon g Concept Requirements Development Product the se change

Slide 113: Plan Development REVIEW Concept What we need Get ready

Slide 114: Focused

Slide 115: Project owner is key

Slide 116: Weekly releases

Slide 117: More flexible

Slide 118: Less time

Slide 119: Better transparency

Slide 120: Less software

Slide 121: Sequencing informatics

Slide 122: Thank you

Slide 123: GREENISGOOD.CO.UK