Into The Wonderful

Slide 1: Into the Wonderful Towards a Virtual Institute

Slide 2: you are here

Slide 3: Data

Slide 4: Lots of data

Slide 5: Lots of data, lots of people

Slide 6: Lots of data, lots of people, lots of compute

Slide 7: Lots of data, lots of people, lots of compute, lots of uses

Slide 8: Lots of data, lots of people, lots of compute, lots of uses, lots and lots and lots and lots...

Slide 9: Trillionics

Slide 10: A platform for science

Slide 12: 1 Get 2 Select 3 Work 4 Save

Slide 13: 1 Get 2 Select 3 Work 4 Save

Slide 14: 1 Get 2 Select 3 Work 4 Save

Slide 15: 1 Get 2 Select 3 Work 4 Save

Slide 16: Work is the killer app get here quickly

Slide 17: Work = publications

Slide 18: Problematic for complex data

Slide 19: 1 Get 2 Select 3 Work 4 Save

Slide 20: 1 Get: flat files / databases 2 Select 3 Work 4 Save

Slide 21: 1 Get: flat files / databases 2 Select: scripts / directories 3 Work 4 Save

Slide 22: 1 Get: flat files / databases 2 Select: scripts / directories 3 Work: interesting 4 Save

Slide 23: 1 Get: flat files / databases 2 Select: scripts / directories 3 Work: interesting 4 Save: flat files / databases

Slide 24: Get Filter Work Save

Slide 25: Get Filter Work Save

Slide 26: Get Filter Work Save

Slide 27: Get Filter Work Save

Slide 28: Get Filter Work Save

Slide 29: Get Filter Work Save

Slide 30: Get Filter Work Save

Slide 31: Get Filter Work Save

Slide 32: Filter Save Get Work Get Filter Work Save

Slide 33: Filter Save Work Get Get Work Get Filter Work Save

Slide 34: Filter Save Work Get Get Work Get Filter Work Save

Slide 36: Virtualise

Slide 37: Get Save

Slide 38: Data platform Get Save

Slide 39: Data platform Get Save Work

Slide 40: Data platform Get Save Work App platform

Slide 41: Data accessible via services

Slide 42: Applications accessible via services

Slide 43: Data platform Get / Save Work Projects / SNP calling App platform

Slide 44: Distribute

Slide 45: Data platform Hintxon Get / Save San Diego Work App platform

Slide 46: Distributed storage Virtualised services Application programming interfaces Getters Filters Savers Work

Slide 47: Distributed storage Virtualised services Application programming interfaces Getters Filters Savers Work Work Work Work Work Work Work Work Work Work Work Work Work Work Work Work Work

Slide 48: A distributed mindset

Slide 49: map/reduce

Slide 50: 1. map

Slide 51: @a = [ 1, 2, 3 ] @result = [] for each $value in @a push @result, map($value) end sub map($incoming) return ($incoming * 10) end

Slide 52: 2. reduce

Slide 53: reduce(@result) sub reduce($r) <transform $r> end

Slide 54: independent

Slide 55: of array size! independent

Slide 56: of array size! independent of ea ch other!

Slide 57: independent distribute across virtual machines!

Slide 58: Prerequisites

Slide 59: Open data easy to get a t data

Slide 60: soft ware as a service Open APIs

Slide 61: Beyond SQL

Slide 62: Accessibility

Slide 63: East coast 24/7 Accessibility Dow n the co rridor West coast

Slide 64: Reliability

Slide 65: Build for flux

Slide 66: Authentication

Slide 67: Privacy

Slide 68: Less software

Slide 69: Distribute everything

Slide 70: Replicate everything Speed. Redundancy.

Slide 71: Will it scale?

Slide 72: Oh yes

Slide 73: New York Times

Slide 74: 11 million TIFs

Slide 75: 24 hours $500

Slide 76: Google, Yahoo! Amazon

Slide 77: We are here

Slide 78: We need to start now

Slide 79: 2 X

Slide 80: 150 Tb/week

Slide 81: We need to start now as in, like, ye sterday

Slide 82: Petabyte journal club foomongers.org.uk

Slide 83: Thank you

Slide 84: GREENISGOOD.CO.UK