"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection

The Reverse Factory
Embedded Vision in High-Volume
(and Value) Laboratory Applications
Patrick Courtney
patrick.courtney@tec-connection.com
Embedded Vision Alliance
Hamburg 6th September 2017
V2.6

Information-rich image sets in the hands of users

Fred Sanger (1918-2013)
• Nobel Prize 1958
• Protein sequencing
• Human insulin
Image credit: MRC Laboratory of Molecular Biology

Structure of DNA 1953
Crick and Watson
Nobel 1962
Friedrich Miescher 1869

Fred Sanger (1918-2013)
• Nobel Prize #2
• DNA sequencing 1980
1st generation sequencing
C T G A
Image credit: MRC Laboratory of Molecular Biology
separationbyelectricfield

Synopsis
• Motivation: the need and the market
• Laboratory as a factory in reverse
• Enabled by science and technology (including imaging)
• Big applications today: NGS case study
• End applications: ourselves and our world, family, food
• How it works: chemistry, optics, software
• Role of imaging in delivering performance
• Improvement curve: Carlson’s curve and what this means
• Cost, speed, growing the market, new applications
• The next applications for imaging
• Scientific & technological trends
• There are still plenty of opportunities

Laboratory as a factory in reverse:
from sample to information
petrochemicals
Industrial
biomedical
research
pharmaceutical
forensics environmental
materials
research
food & drink
consumer
goods
from well behaved to heterogenous; from solid into liquid form
clinical
Life sciences Physical sciences

petrochemicals
Industrial
biomedical
research
pharmaceutical
forensics
materials
research
food & drink
consumer
goods
clinical
Life sciences Physical sciences
Pharma
R&D
$50bn
Clinical
Testing
$60bn
Lab
instruments
$40bn
forensic
testing
$20bn
food
testing
$11bn
environmental
Increasing use of imaging
Laboratory as a factory in reverse:
from sample to information

Clinical applications of genomics
• Screening
• Diagnosis
• for cancer, infection
• Treatment
• for selection, progress, follow up
• example: breast cancer BRCA1
• Emerging area
• counselling and reproduction
cisncancer pharmainfo.net

Applications expanding beyond medicine
• Next generation sequencing is now used very widely
• Family
• Food
• Flu
• Forensics
• Fish
• High volume applications of NGS: all that touches on life

Family: self, ancestry, genealogy
• Self
• Inheritance
• Health risk?
• Regulation
• FDA and terms of use
• (and our pets)
consumer
goods

Flu: Tracing infection Zika 2016
• 4-40 entry points from April
Grubaugh, Nathan D., et al. "Genomic epidemiology
reveals multiple introductions of Zika virus into the
United States." Nature (2017) 546, pp401-405.

Aircraft safety: bird strike data - what when and why
Lapwing
Kestrel
Galah
environmental

Elements of an NGS (next generation) system
• DNA strand
• Flow cell
• Chemistry
• Optics
• Laser
• Camera
• Software
https://www.youtube.com/watch?v=9YxExTSwgPM sequencing 5min
https://www.youtube.com/watch?v=pfZp5Vgsbw0 flow cell 2min
illumina

Role of imaging: the flow cell
Each image 3-4Mp, 120k images per 36 cycle run = 350Gb

Role of imaging: the optical path
illumina
fluorescence

Role of imaging: how it works
8 lanes x 100 tiles. 70bp -> 28k images / lane
300k clusters per tile. 3Gb totalillumina

Further improvements (1)
Problem: 4 colour channels per image
illumina

Problem: 4 colour channels per image
Solution: from 4 channels to 2 channels
illumina

Leads to other problems
• But …. 2 colour chemistry can overcall high confidence G bases
The sequence below shows this effect:
@1:11101:2930:2211 1:N:0
ATTTATTATTAATTAAATATTAATAATAAATAGATCGGAAGAGCACACGTCTGAACTCCAGTCACTAGCTTAGCGCGTATGCCGTCGTCGGCGTGCAAAAAAAAAGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
+AAAAAEEEEEEEEEEEE6EEEEEAEEEEEEEEEA/EE<EEEAEE/EAEEAEEEE6</EEEEEA/<//<///A/A//////</E<//////E///A/</A/<<A////A/E<EEEEEEEAEEE/EEEAEAEAEAE6/AEAEE<AAEAEE
It’s easier to see if you visualise the quality scores for this sequence
single_seq_quality
https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/

Problem: cluster density issues
• Cluster density can be demanding
• Especially for some samples
Krueger F, Andrews SR, Osborne CS (2011) Large Scale Loss of Data in Low-Diversity Illumina Sequencing Libraries Can Be Recovered by
Deferred Cluster Calling. PLOS ONE 6(1): e16607. https://doi.org/10.1371/journal.pone.0016607

Solution: patterned flow cells
50nm spots spacing 250nmillumina

Solution: patterned flow cells
• From random spots to fixed positions
• Simplification of analysis
• Increase in density and reliability
• So more data in less time, cost
illumina

Problem: better use of flow cell
• Solution: use two surface imaging
• Challenging imaging and focussing
• End up with an optical head with 6 linear cameras
Illumina US8143599

Nature 507, 294–295 (20 March 2014)
Improvement: Carlson’s curve and what this means

Improvement: Carlson’s curve and what this means
• average of 5x in 2 year (2.4x faster than Moore)
• with a peak of 1000x in 2 years
• 100k x better over 14 years vs 34 years
Ben Moore, in gnuplot by grendel|khan. - Own work, Public Domain, https://commons.wikimedia.org/w/index.php?curid=31006154
Illumina Hiseq2000s BGI Hong Kong (128 units)
Moore’s
Carlson’s
2001
peak Carlson
• Enabled by many
technologies
100k is 57 and 217

Or put it another way, if computing had improved as fast….
IBM PC XT 1983 = 34 years ago
So for the technology we have now
the IBM PC would have been introduced in 2003
The same year of Nemo…
….or a car would cost €0.20… or a $1000 flight, 1 cent

Market size and trends
• Lab instruments market
• $40bn (instruments/service)
• Segments and growth rate
• Oncology, infection, reproduction, agriculture, forensic, consumer
• Currently $3bn growing 30% CAGR to $12bn by 2022
• Market capitalisation:
• illumina (cap $28bn) make profit of $1.7bn on sales of $2.5bn
• Learnings for the vision supply chain:
• Rewards fall to the users, and system integrator
• Components suppliers get small % units sales
• Driver: cost per genome, not raw speed
• But someone has to learn the application and design the system
Grand view research; Macquarie (USA) Research 2014
Consumer genomics
Agri-genomics Forensics
Metagenomics, drug development
Immune system monitoring
Reproductive health
Clinical Investigation
Oncology

Market trends and remaining opportunity
• Remaining potential for clinical applications
• On the cost reduction from $3.000M to $1000
• Moving WGS (Whole Genome Sequencing) into the doctors practice
• How many units? How many physicians? 10M
• Remaining potential for all other applications
• How much DNA is there out there?

The next generation: Oxford nanopore

What’s next? Following the trends

Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors
• Optics
• Algorithms
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust

• New labels
• New technology
• Sensors
• Optics
• Algorithms
Evolution of the microscope
• Robotics

Evolution of the microscope since c.1670
Expanding the market
• Drivers: quality, productivity by ease of use and automation
modern microscope imaging plate readervan Leeuwenhoek benchtop microscope
Individual cells: fluorescence brightfield
foldscope
plate of cells

Automated cell counters: from $60k to $5k
Expanding the market
Beckmann, Invitrogen, SigmaAldrich
automated
manual

• New labels
• New technology
• Sensors, Optics
• Algorithms
• Robotics

Improved scientific knowledge: Nobel prizes for the lab
• The Nobel Prize in Chemistry 2008
• Osamu Shimomura, Martin Chalfie and Roger Y. Tsien†
• for the discovery & development of green fluorescent protein GFP
• New labels (antibodies, nanoparticles…)
• The Nobel Prize in Chemistry 2014
• Eric Betzig, Stefan W. Hell and William E. Moerner
• for the development of super-resolved fluorescence microscopy
• New imaging modes (Raman [1930], IR, spectroscopy…)

What is the resolution revolution ?
and why imaging is (still) important

Ernst Abbe stated a limit
on resolving power (1873)
By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637
Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)

Ernst Abbe stated a limit
on resolving power (1873)
By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637
Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)
Resolution scheme: adopted from Thorley et al., Super-resolution Microscopy: A Comparison of Commercially Available Options, Fluorescence Microscopy Super-Resolution and Other Novel Techniques, Academic Press, 2014

Super-resolution: breaking the Abbe limit
Betzig et al., Science, 2006, 313, 1642-1645
to 20nm
to 50nm

Super-resolution: how it works (1)
http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html

New reagents: Brainbow labelling
Lichtman et al., Nature Reviews Neuroscience 2008

Building brainbow from fluorescent proteins
• Motivation: to map all the connections in the brain
• What this means for the imaging supply chain:
• better faster smarter cameras
• multichannel, multifocal xyz-t-λ
Lawson Kurtz et al. / Duke University

• New reagents
• New technology
• Sensors, optics
• Algorithms
• Robotics
• Lab environment
multiple
sensors
AndrewAlliance

Smartlab at LabVolution Hannover, May 2017

What the lab really looks like: it’s a messy place
A long way from
lean processes,
from industry 4.0
If the DNA is the
”job description
for the cell”, this
is what actually
happens when it
meets the world
Cancer research makes for a messy bench. … @WorldwideCancer

Applications tomorrow: watching the lab
• Klavins Lab
• “Aquarium”
• See TEDx talk on synthetic “programming” biology
https://www.youtube.com/watch?v=jL0cG4NJGd4

Actions on future applications
• Future imaging (super-microscopes)
• Lab (factory) of the future: 20-100 cameras per lab
• Hospital of the future: role of imaging in the lab
• Take home message:
• imaging has proven value but still only present at a very low level
• Role of EU programmes

Bringing it all together:
The Healthcare Lighthouse vision
Laboratory
Care
Surgery
Rehabilitation
euRobotics topic groups on medical and laboratory robotics

Acknowledgements
• Almost too many to mention, but I’ll try
– DNA sequencing: Illumina, HPA, Qiagen
– Microscopy: PerkinElmer, Sartorius, Stefan Hell
– Cell counting: Luna, Roche, Jenoptik
– Smartlab: Deutsche Messe
– AndrewAlliance, EU, euRobotics

How much DNA is there out there?
6x1030
microbes
on earth

Ebola and the most expensive tent in the world
Dr Sam Collins
Prof Ian Goodfellowactually an ion torrent machine

Super-resolution: how it works (2)
Localisation is
more precise
than resolving
In effect: trade
time for space
Role of imaging
Actually, several techniques http://www.practicallyscience.com/category/bio/cellbio/

"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to "The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection

Similar to "The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection (20)

More from Edge AI and Vision Alliance

More from Edge AI and Vision Alliance (20)

Recently uploaded

Recently uploaded (20)

"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection