For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/sept-2017-alliance-vitf-courtney
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Dr. Patrick Courtney, MBA, of tec-connection and the Standards in Laboratory Automation (SiLA) Consortium delivers the presentation "The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications" at the Embedded Vision Alliance's September 2017 Vision Industry and Technology Forum. In his presentation, Courtney covers the following topics:
▪ Motivation: the need and the market
▪ Big applications today: NGS case study
▪ Improvement curve: Carlson’s curve and what this means
▪ The next applications for imaging
"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection
1. Page 1
The Reverse Factory
Embedded Vision in High-Volume
(and Value) Laboratory Applications
Patrick Courtney
patrick.courtney@tec-connection.com
Embedded Vision Alliance
Hamburg 6th September 2017
V2.6
3. Page 3
Fred Sanger (1918-2013)
• Nobel Prize 1958
• Protein sequencing
• Human insulin
Image credit: MRC Laboratory of Molecular Biology
4. Page 4
Structure of DNA 1953
Crick and Watson
Nobel 1962
Friedrich Miescher 1869
5. Page 5
Fred Sanger (1918-2013)
• Nobel Prize #2
• DNA sequencing 1980
1st generation sequencing
C T G A
Image credit: MRC Laboratory of Molecular Biology
separationbyelectricfield
6. Page 6
Synopsis
• Motivation: the need and the market
• Laboratory as a factory in reverse
• Enabled by science and technology (including imaging)
• Big applications today: NGS case study
• End applications: ourselves and our world, family, food
• How it works: chemistry, optics, software
• Role of imaging in delivering performance
• Improvement curve: Carlson’s curve and what this means
• Cost, speed, growing the market, new applications
• The next applications for imaging
• Scientific & technological trends
• There are still plenty of opportunities
7. Page 7
Laboratory as a factory in reverse:
from sample to information
petrochemicals
Industrial
biomedical
research
pharmaceutical
forensics environmental
materials
research
food & drink
consumer
goods
from well behaved to heterogenous; from solid into liquid form
clinical
Life sciences Physical sciences
9. Page 9
Clinical applications of genomics
• Screening
• Diagnosis
• for cancer, infection
• Treatment
• for selection, progress, follow up
• example: breast cancer BRCA1
• Emerging area
• counselling and reproduction
cisncancer pharmainfo.net
10. Page 10
Applications expanding beyond medicine
• Next generation sequencing is now used very widely
• Family
• Food
• Flu
• Forensics
• Fish
• High volume applications of NGS: all that touches on life
11. Page 11
Family: self, ancestry, genealogy
• Self
• Inheritance
• Health risk?
• Regulation
• FDA and terms of use
• (and our pets)
consumer
goods
12. Page 12
Flu: Tracing infection Zika 2016
• 4-40 entry points from April
Grubaugh, Nathan D., et al. "Genomic epidemiology
reveals multiple introductions of Zika virus into the
United States." Nature (2017) 546, pp401-405.
13. Page 13
Aircraft safety: bird strike data - what when and why
Lapwing
Kestrel
Galah
environmental
14. Page 14
Elements of an NGS (next generation) system
• DNA strand
• Flow cell
• Chemistry
• Optics
• Laser
• Camera
• Software
https://www.youtube.com/watch?v=9YxExTSwgPM sequencing 5min
https://www.youtube.com/watch?v=pfZp5Vgsbw0 flow cell 2min
illumina
15. Page 15
Role of imaging: the flow cell
Each image 3-4Mp, 120k images per 36 cycle run = 350Gb
16. Page 16
Role of imaging: the optical path
illumina
fluorescence
17. Page 17
Role of imaging: how it works
8 lanes x 100 tiles. 70bp -> 28k images / lane
300k clusters per tile. 3Gb totalillumina
20. Page 20
Further improvements (1)
Problem: 4 colour channels per image
Solution: from 4 channels to 2 channels
illumina
21. Page 21
Further improvements (1)
Leads to other problems
• But …. 2 colour chemistry can overcall high confidence G bases
The sequence below shows this effect:
@1:11101:2930:2211 1:N:0
ATTTATTATTAATTAAATATTAATAATAAATAGATCGGAAGAGCACACGTCTGAACTCCAGTCACTAGCTTAGCGCGTATGCCGTCGTCGGCGTGCAAAAAAAAAGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
+AAAAAEEEEEEEEEEEE6EEEEEAEEEEEEEEEA/EE<EEEAEE/EAEEAEEEE6</EEEEEA/<//<///A/A//////</E<//////E///A/</A/<<A////A/E<EEEEEEEAEEE/EEEAEAEAEAE6/AEAEE<AAEAEE
It’s easier to see if you visualise the quality scores for this sequence
single_seq_quality
https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/
22. Page 22
Further improvements (2)
Problem: cluster density issues
• Cluster density can be demanding
• Especially for some samples
Krueger F, Andrews SR, Osborne CS (2011) Large Scale Loss of Data in Low-Diversity Illumina Sequencing Libraries Can Be Recovered by
Deferred Cluster Calling. PLOS ONE 6(1): e16607. https://doi.org/10.1371/journal.pone.0016607
24. Page 24
Further improvements (2)
Solution: patterned flow cells
• From random spots to fixed positions
• Simplification of analysis
• Increase in density and reliability
• So more data in less time, cost
illumina
25. Page 25
Further improvements (3)
Problem: better use of flow cell
• Solution: use two surface imaging
• Challenging imaging and focussing
• End up with an optical head with 6 linear cameras
Illumina US8143599
26. Page 26
Nature 507, 294–295 (20 March 2014)
Improvement: Carlson’s curve and what this means
27. Page 27
Improvement: Carlson’s curve and what this means
• average of 5x in 2 year (2.4x faster than Moore)
• with a peak of 1000x in 2 years
• 100k x better over 14 years vs 34 years
Ben Moore, in gnuplot by grendel|khan. - Own work, Public Domain, https://commons.wikimedia.org/w/index.php?curid=31006154
Illumina Hiseq2000s BGI Hong Kong (128 units)
Moore’s
Carlson’s
2001
peak Carlson
• Enabled by many
technologies
100k is 57 and 217
28. Page 28
Or put it another way, if computing had improved as fast….
IBM PC XT 1983 = 34 years ago
So for the technology we have now
the IBM PC would have been introduced in 2003
The same year of Nemo…
….or a car would cost €0.20… or a $1000 flight, 1 cent
29. Page 29
Market size and trends
• Lab instruments market
• $40bn (instruments/service)
• Segments and growth rate
• Oncology, infection, reproduction, agriculture, forensic, consumer
• Currently $3bn growing 30% CAGR to $12bn by 2022
• Market capitalisation:
• illumina (cap $28bn) make profit of $1.7bn on sales of $2.5bn
• Learnings for the vision supply chain:
• Rewards fall to the users, and system integrator
• Components suppliers get small % units sales
• Driver: cost per genome, not raw speed
• But someone has to learn the application and design the system
Grand view research; Macquarie (USA) Research 2014
Consumer genomics
Agri-genomics Forensics
Metagenomics, drug development
Immune system monitoring
Reproductive health
Clinical Investigation
Oncology
30. Page 30
Market trends and remaining opportunity
• Remaining potential for clinical applications
• On the cost reduction from $3.000M to $1000
• Moving WGS (Whole Genome Sequencing) into the doctors practice
• How many units? How many physicians? 10M
• Remaining potential for all other applications
• How much DNA is there out there?
33. Page 33
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors
• Optics
• Algorithms
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust
34. Page 34
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors
• Optics
• Algorithms
Evolution of the microscope
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust
35. Page 35
Evolution of the microscope since c.1670
Expanding the market
• Drivers: quality, productivity by ease of use and automation
modern microscope imaging plate readervan Leeuwenhoek benchtop microscope
Individual cells: fluorescence brightfield
foldscope
plate of cells
36. Page 36
Automated cell counters: from $60k to $5k
Expanding the market
Beckmann, Invitrogen, SigmaAldrich
automated
manual
37. Page 37
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors, Optics
• Algorithms
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust
38. Page 38
Improved scientific knowledge: Nobel prizes for the lab
• The Nobel Prize in Chemistry 2008
• Osamu Shimomura, Martin Chalfie and Roger Y. Tsien†
• for the discovery & development of green fluorescent protein GFP
• New labels (antibodies, nanoparticles…)
• The Nobel Prize in Chemistry 2014
• Eric Betzig, Stefan W. Hell and William E. Moerner
• for the development of super-resolved fluorescence microscopy
• New imaging modes (Raman [1930], IR, spectroscopy…)
39. Page 39
What is the resolution revolution ?
and why imaging is (still) important
40. Page 40
What is the resolution revolution ?
and why imaging is (still) important
Ernst Abbe stated a limit
on resolving power (1873)
By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637
Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)
41. Page 41
What is the resolution revolution ?
and why imaging is (still) important
Ernst Abbe stated a limit
on resolving power (1873)
By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637
Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)
Resolution scheme: adopted from Thorley et al., Super-resolution Microscopy: A Comparison of Commercially Available Options, Fluorescence Microscopy Super-Resolution and Other Novel Techniques, Academic Press, 2014
43. Page 43
Super-resolution: how it works (1)
http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html
44. Page 44
New reagents: Brainbow labelling
and why imaging is (still) important
Lichtman et al., Nature Reviews Neuroscience 2008
45. Page 45
Building brainbow from fluorescent proteins
• Motivation: to map all the connections in the brain
• What this means for the imaging supply chain:
• better faster smarter cameras
• multichannel, multifocal xyz-t-λ
Lawson Kurtz et al. / Duke University
46. Page 46
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New reagents
• New technology
• Sensors, optics
• Algorithms
• Robotics
• Lab environment
• Drivers: faster, easy to use, more specific, sensitive, robust
multiple
sensors
AndrewAlliance
48. Page 48
What the lab really looks like: it’s a messy place
A long way from
lean processes,
from industry 4.0
If the DNA is the
”job description
for the cell”, this
is what actually
happens when it
meets the world
Cancer research makes for a messy bench. … @WorldwideCancer
49. Page 49
Applications tomorrow: watching the lab
• Klavins Lab
• “Aquarium”
• See TEDx talk on synthetic “programming” biology
https://www.youtube.com/watch?v=jL0cG4NJGd4
50. Page 50
Actions on future applications
• Future imaging (super-microscopes)
• Lab (factory) of the future: 20-100 cameras per lab
• Hospital of the future: role of imaging in the lab
• Take home message:
• imaging has proven value but still only present at a very low level
• Role of EU programmes
51. Page 53
Bringing it all together:
The Healthcare Lighthouse vision
Laboratory
Care
Surgery
Rehabilitation
euRobotics topic groups on medical and laboratory robotics
52. Acknowledgements
• Almost too many to mention, but I’ll try
– DNA sequencing: Illumina, HPA, Qiagen
– Microscopy: PerkinElmer, Sartorius, Stefan Hell
– Cell counting: Luna, Roche, Jenoptik
– Smartlab: Deutsche Messe
– AndrewAlliance, EU, euRobotics
54. Page 58
How much DNA is there out there?
6x1030
microbes
on earth
55. Page 60
Ebola and the most expensive tent in the world
Dr Sam Collins
Prof Ian Goodfellowactually an ion torrent machine
56. Page 61
Super-resolution: how it works (1)
http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html
57. Page 62
Super-resolution: how it works (2)
Localisation is
more precise
than resolving
In effect: trade
time for space
Role of imaging
Actually, several techniques http://www.practicallyscience.com/category/bio/cellbio/