Practical level introduction for ophthalmic device design. How in the future, multimodal measurement devices will be replacing unimodal devices with simple decision tree indices. Some examples of embedded cameras, measurement illumination, stimulus presentation illumination are shown. Alternative download link: https://www.dropbox.com/s/lt76ohoeusopkoo/practicalConsiderations_embeddedOphthalmicDevices.pdf?dl=0

1. Practical
Considerations
in the design of
Embedded
Ophthalmic
Devices
Petteri Teikari, PhD
Singapore Eye Research Institute (SERI)
Visual Neurosciences group
http://petteri-teikari.com/
Version “Tue 5 June 2018“
Illustrationby
WieslawWalkuski

2. The Vision
Various different user
segments exist withdifferent
technical requirements
And different
price points.
Make amodulardesign
allowingeasycustomization
of the multimodaldevice.
https://www.vision­dimension.com/en/
machine­vision/

3. de facto
“Standard”
of portable future
Have a single
portable device for
both structural and
functional measures
of the visual,
system including
both retina and
cornea.
https://theophthalmologist.com/issues/01
16/the-eye-exams-quantum-leap/
http://dx.doi.org/10.1167/tvst.6.4.16
Test 6: Pupillometry Pupil
reactions were assessed using
simultaneous OCT capture of
the anterior segments including
the iris plane. Each eye was
stimulated independently and
sequentially with a single, bright,
250-ms flash of white light. B-
scan recordings are captured at
regular intervals of 350 ms prior
to stimulation and 4000 ms
post-stimulation. Measurements
of the pupil circumference could
subsequently be calculated to
identify pupil abnormalities and
relative afferentpupillarydefects

4. Multimodal
In ophthalmic context, what
you could think to use as new
and existingbiomarkers for
retinopathies,
and forretinal-derived
measures of
neurodegenerative disease
like Alzheimer’s and
Parkinson’s

5. Fundus
Camera
”ROI”-capabilityand
multiframetechniques
forbetterimagequality
Multi/hyperspectralor
not?
Wearable version already
demonstrated →
http://web.media.mit.edu/~tswedish/projects/eyeSelfie.html

6. Pupillometry
Quantifiesthe
functional“health”ofthe
retinacomparedto
fundusimagingthat
captures thestructural
health.
Functionandstructure
oftendonotgohand-in-
handwithretinopathies.
Our OpenPupil electronic system
consists of: a Raspberry Pi 2 B+,
with a NoIR camera, two 850nm
IR LEDs, a 16 neopixel ring, USB
wireless dongle, microSD card . Our
hardware system consists of: a 3D
printed eyepiece, a lazercut
enclosure, aGoogleCardboard lens
Our software system consists of: a
python flask server, a supervisor
instance, a C library and program for
imagecollection andprocessing
https://github.com/kaiwhata/OpenPupil
https://www.shapeways.com/shops/openpupil

7. Eye
Movements
Similar “end-to-end”
pipeline as for
pupillometry (and
even the same
hardware)
One can “easily”
integrate various
measures to a single
VR headset
https://github.com/pupil-labs/hmd-eyeshttps://pupil-labs.com/vr-ar/
Orloskyetal.(2017) https://doi.org/10.1109/TVCG.2017.2657018

8. Electro-
physiology
Relatively simple to
develop and integrate
to VR-type headset
http://dx.doi.org/10.1515/bmt-2015-0042
68.6 mm x 53.4 mm
Bleduino
Rathersmalland
goodforprototyping
withBluetooth LE
Digital Instrumentation amplifier
AD8253 with Evaluation Kit to
speed up development
ArduinoUno
Shieldsverybulky
A filter setting of 1–200 Hz appears most sensitive to
detect glaucomatous damage if using a two-global-
flash mfERG: High frequencies of 100–300 Hz also
contain information that differentiates glaucoma from
normal and thus should be included in the analysis.
A 50 Hz notch filter allows grossly contaminated
waveforms to be analyzed in a meaningful manner.
With a 50 Hz filter, glaucoma patients still differed
significantly from normal.
Mini-Ganzfeld

9. Magneto-
retinography
With Diamond
Magnetometry
(instead of costly SQUID
magnetometers used in
magnetoencephalography
[MEG] for example).
At some point eventually allowing
non-contact electrical
measurements when the price goes
even well below $150k per
instrument?
Today, Matthew Dale and Gavin
Morley at the University of
Warwick in the U.K. say that
diamond sensors are poised to
revolutionize the way physicians
use magnetic field measurements
in diagnostic medicine. They map
out the state of the art in this area
and say that the business
opportunityissignificant.
https://arxiv.org/abs/1705.01994
https://www.technologyreview.com/s/607871/how-diamond-sensors-are-set-to-revolutionize-medical-diagnostics/
There are around 100 SQUID MEG systems installed worldwide, at a cost of over $1M each. The MCG
market should be much larger if the instrumentation was affordable and portable, because MCG has been
shown to be superior to ECG and hence other non-invasive approaches for the diagnosis of coronary
artery disease (CAD) [Kwongetal.2013, Fenici et al. 2005 and 2013]. CAD is the most common type of heart
diseaseandistheleadingcauseofdeathintheUnitedStatesinbothmenandwomen.
Several companies have tried and failed to commercialize SQUID-based MCG, held back by the cost
of a cryogen-based system. We estimate that 100,000 MCG systems could be sold if the functionality
were the same as existing SQUID systems and the price was below $150k. This is based on there being
over 100,000hospitalsinChina,India,theEU,JapanandtheUSA.
Diamond magnetometers are at technology readiness level (TRL) 7: the technology has been
demonstrated and is moving towards being put on sale. However, this has not yet reached the sensitivity
neededfor MCG,soanMCGsystembasedondiamondisatTRL4-5(technologydevelopment).

10. Electro-
physiology
Correlating function
with structure
Documenta Ophthalmologica April 2017, Volume134, Issue 2, pp 111–128|
Comparingthree differentmodesof
electroretinographyinexperimental
glaucoma: diagnosticperformance and
correlationtostructure
Laura Wilsey,Sowjanya Gowrisankaran,Grant Cull,
Christy Hardin.Claude F. Burgoyne,Brad Fortune
https://doi.org/10.1007/s10633-017-9578-x
ArqBrasOftalmol. 2017Mar-Apr;80(2):118-121.
doi:10.5935/0004-2749.20170028
Structure-functionalcorrelation
usingadaptiveoptics,OCT,and
microperimetryinacaseofoccult
macular dystrophy.
VianaKÍ, MessiasA, SiqueiraRC, RodriguesMW, Jorge R
Optical coherencetomography(1-3), adaptiveoptics(A-H), and
microperimetry (2-4) imagesof a patientwith occultmaculardystrophy.
(Arrows) Lossof continuityof theouterphotoreceptorlayerin thecentral
fovealregion.(yellowasterisk)Reduced ring photoreceptor densityin the
fovealregion.(2) Reductionof ringsensitivityin thecentral foveal region.
(Red Asterisk)Reduced photoreceptor densityin acentralfoveal region.(4)
Reduced central sensitivityinthefovea.

11. Visual Evoked
Potentials
Essentially a EEG
headset with reduced
electrode count and
active dry electrodes
ThenGoggle,aPortableBrain-Computer Interfacefor AssessmentofVisualFunctionandglaucoma
diagnosis–Nakanishietal.(2017) http://dx.doi.org/10.1001/jamaophthalmol.2017.0738
REINVENT:Alow-cost,virtual
realitybrain-computerinterface
forseverestrokeupperlimb
motorrecovery
Spiceret al. (2017)
https://doi.org/10.1109/VR.2017.7892338
AVirtual-RealityBased
NeurofeedbackGame
Frameworkfor Depression
RehabilitationusingPervasive
Three-ElectrodeEEGCollector
Caietal. (2017)
https://doi.org/10.1145/3127404.3127433
HTC Vive Modified With Neurable
ReadsYourMind At SIGGRAPH
https://youtu.be/47WHqDNckI8
https://www.extremetech.com/extreme/254816-eeg-
virtual-reality-matrix-just-around-corner
A feasibilitystudyonSSVEP-basedinteractionwith
motivatingandimmersivevirtual andaugmented reality
Josef Faller, Brendan Z. Allison, ClemensBrunner, ReinholdScherer, Dieter Schmalstieg, GertPfurtscheller, ChristaNeuper
(Submitted on15Jan 2017)
https://arxiv.org/abs/1701.03981

12. Visual Fields
Very common measurement
even though it can be
stressful for the
patient with high noise
in this psychophysical
measurement (reduced by
using log units)
Similarly, visual field
measurement fail to
detect early changes in
glaucoma as the brain
can compensate for the
neurodegeneration of
RGCs
ARVO 2017 poster for deep
learning in visual field
assessment
2846—B0449Adeep-learningbased
automaticglaucomaidentification.
Serife SedaS. Kucur1
, M. Abegg2
, S. Wolf2
, R. Sznitman1
.
1
ARTORGCenter, Universityof Bern, Bern, Switzerland;
2
DepartmentofOpthalmology, Inselspital Bern, Bern, Switzerland https://doi.org/10.1016/j.ophtha.2017.06.028
VisualFieldTestingwithHead-
MountedPerimeter‘imo’
Matsumoto et al. (2016)
https://doi.org/10.1371/journal.pone.0161974
The perimeter imo has completely isolated
optical systems for the right and left eyes.
Stimulus presentation is also independently
performedfor eacheye.
Visualfieldexaminationmethodusing
virtual realityglassescomparedwith
the Humphrey perimeter
Tsapakis et al. (2017)
doi: 10.2147/OPTH.S131160
Effect ofcognitive demand on
functionalvisualfield performancein
seniordriverswithglaucoma
Gangeddula, Viswa, et al (2017)
https://doi.org/10.3389/fnagi.2017.00286

13. Visual
Fields
Deeplearninghereas
wellfortrajectory
prediction
8 September 2016
Visionary technology
Innovative optical concept offers simple,
affordable, fast glaucoma screening test
The new Viewi optical concept developed by Cambridge Consultants shows how it
could be possible for patients to monitor any effect on their vision in the comfort of
their own homes. At the moment, patients typically have an annual optometrist or
hospital check-up using a visual field analyser. Flashing lights at varying points of the
visual field test sensitivity – with the patient pressing a button each time they see a
light. The novel Viewi technology performs the same test but at a fraction of the
cost – around £20 rather than £20,000 for the clinical device.
The innovative Viewi concept has been hailed as an important advance by optics
expert Chris Dainty, a professor at University College London Institute of
Ophthalmology and Moorfields Eye Hospital.
https://doi.org/10.1101/293621
“Using unfiltered real-world datasets, deep
learning networks show an impressive ability to
not only learn spatio-temporal HVF changes
but also to generate predictions for future
HVFs up to 5.5 years, given only a single
HVF. “

14. Visual
Function
Towards assessing real-
life function,
simultaneously for
diagnosis and disease
progression purposes
Using virtual reality to cause a
subject to correct for perceived
motion has revealed that
glaucoma patients’ reactions are
more erratic than those of healthy
individuals
Diniz-Filho,etal.(2015)
doi: 10.1016/j.ophtha.2015.02.010
ChristopherKent,Senior EditorPUBLISHED 6JULY2015
Virtual Reality:A NewFrontierinEye
Care?
https://www.reviewofophthalmology.com/article/virtual-reality-a-new-frontier-in-eye-care
Daga,Fábio B., et al. "Wayfinding and Glaucoma: A Virtual
Reality Experiment." InvestigativeOphthalmology&Visual
Science58.9(2017):3343-3349. doi: 10.1167/iovs.17-21849
The VEHuNT consisted of a cave automatic virtual environment (CAVE) used to present an
immersiveVR environmentto studywayfinding tasks.
Usingvirtual/augmentedrealitytosimulate
visualimpairments
ByDr.PeteRJones
http://www.ucl.ac.uk/~smgxprj/projects.html

15. Multimodal
Rationale
Modeling
Background
Welldeeplearning
modelinpractice

16. Multimodal
Model
Single new index, or
single new device
won’t really be
enough for complex
pathologies.
Need new ways to
interpret the data as
well if linear
decision trees can’t
explain your data
HollyF.Ainsworthetal.(2017):
The use of causal inference techniques to integrate omics and GWAS data has the potential
to improve biological understanding of the pathways leading to disease. Our study
demonstrates the suitability of various methods for performing causal inference under
several biologically plausible scenarios
EwenCallaway(2017):
“Biologists are likely to find that larger studies turn up more and more genetic variants
– or “hits” - that have minuscule influences on disease” - Jonathan Pritchard, Stanford
University
EmergingFrontiersof
Neuroengineering:A
NetworkScienceof
BrainConnectivity
Danielle S. Bassett, AnkitN. Khambhati, and ScottT. Grafton
Annual Review ofBiomedical Engineering Vol. 19:327-352
(Volume publicationdate June 2017)
https://doi.org/10.1146/annurev-bioeng-071516-044511
Constructing connectomes from magnetic resonance
imaging (MRI) data. To generate human connectomes
with MRI, an anatomic scan delineating gray matter is
partitioned into a setof nodes.Thisscan iscombined with
either diffusion scans of white matter structural
connections or time series of brain activity measured by
functional MRI, resulting in a weighted connectivity
matrix.
Noteson"ArtificialIntelligencein
BioscienceSymposium2017"
Organized byBenevolentAI in Londpn,UK
https://www.slideshare.net/PetteriTeikariPhD/notes-on-artificial-intel
ligence-in-bioscience-symposium-2017

17. Multimodal
Model
Personalized
precision medicine
based on multiple
measures with EHR
mining.
Example for diagnostics
Example for management
Example for treatment
In 2015, aresearch groupatMountSinai Hospitalin New York wasinspiredtoapplydeeplearning to
the hospital’s vast database of patient records. This data set features hundreds of variables on
patients, drawn from their test results, doctor visits, and so on. The resulting program, which the
researchers named Deep Patient, was trained using data from about 700,000 individuals, and
whentestedonnewrecords,itprovedincrediblygoodatpredictingdisease.
https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/
http://doi.org/10.1038/srep26094 https://github.com/greenelab/deep-review/issues/63
https://syncedreview.com/2017/02/26/deep-pa
tient-improving-prognosis-with-electronic-h
ealth-records-by-deep-learning/ on-demand.gputechconf.com

18. Multimodal
Model
Personalized
precision medicine
based on multiple
measures with EHR
mining.
Example for diagnostics
Example for management
Example for treatment
Thefuture ofhealth diagnostics. Currentdiagnosticsarebased on a “snapshot”in timeand limited datapoints.Inthefuture, largedatasetsacquired over
timethroughconstantmonitoring will beanalyzed to establishbaselinesand trends,enabling preventativeinterventions. Partof thisfigurereusesa drawing
previouslypublished in Swedish etal,2015.Copyright©2017ACM, Inc. Adapted fromSwedish T,Roesch K,LeeIK, Rastogi K, BernsteinS, RaskarR. EyeSelfie:
Self directed eyealignmentusing reciprocaleyeboximaging. ACM TransGraph. 2015;34(4):58.39
KarinRoesch, TristanSwedish, and RameshRaskar (2017): “Automated retinalimaging andtrend analysis– atool forhealthmonitoring”
https://dx.doi.org/10.2147/OPTH.S116265

19. Multimodal
Model
Personalized
precision medicine
based on multiple
measures with EHR
mining.
Example for diagnostics
Example for management
Example for treatment
Classificationofadvanced
stagesofParkinson’s
disease:translation into
stratifiedtreatments
JournalofNeuralTransmission August
2017, Volume124, Issue 8, pp1015–1027
RejkoKrüger, Jochen Klucken, Daniel Weiss, Lars Tönges, Pierre Kolber,
Stefan Unterecker, Michael Lorrain, Horst Baas,Thomas Müller, Peter
Riederer https://doi.org/10.1007/s00702-017-1707-x
PrecisionMedicineinPediatric
Oncology:Translating
GenomicDiscoveriesinto
OptimizedTherapies
American Association forCancerResearch June9,2017
ThaiHoaTran, AvanthiTayiShah and Mignon L. Loh
https://doi.org/10.1158/1078-0432.CCR-16-0115
Relative frequency of genomic alterations in neuroblastoma at
diagnosiscompared with relapse

20. “Auxiliary
Measurements”
Some “helper”
measurements like
pupil center could
actually be useful
for clinical
purposes as well.

21. Case:
Pupillometry#1
Basicallyincluded
alreadyin many
existingophthalmic
devices.
Whatisthe value ofa
stand-alone device in
the end? - not much?
With OCT Carrasco-Zevalloetal. (2015) - cited by 13
With Adaptive Opticsretinal imaging
Sahin etal.(2012) - cited by 26

22. Case:
Pupillometry#2
These designsjust
need abit of
algorithmsand an
illumination for PLR,
and itisthere.
Corneal topography devices
Boxer etal. (2000) | cited by 62
Scheimpflugimaging
Baradaran-Rafiietal. (2017) - cited by 1

23. Case:
Pupillometry#3
Basicallythese
deviceshavethe
capabilitytodo
chromatic
pupillometryif
modifiedforit.
RETEval Portable ERG
Katoetal.(2015) | cited by14
Eye Trackers
https://pupil-labs.com/pupil/

24. Case:
EyeMovements
Eye blinkrate and slow
eye movements
(SEMs)can be used to
monitorvigilance /
alertness.
Thinkofdriver/airlinepilot/
oranyotherfatigue
management scenariowith
real-time drowsiness
estimate
https://doi.org/10.1016/j.inffus.2017.11.005
EEGandIMU sensorsareusedto
detect 5levelsof drowsinesswith95%
ofaccuracy.
EEGandocular correlatesof
circadianmelatoninphaseandhuman
performancedecrementsduring
sleeploss
ChristianCajochen, SatBir S.Khalsa, JamesK.Wyatt,
CharlesA.Czeisler,and Derk-JanDijk 01SEP1999
https://doi.org/10.1152/ajpregu.1999.277.3.R640
Theaimofthisstudywastoquantifytheassociations
betweensloweyemovements(SEMs),eyeblinkrate,
wakingelectroencephalogram(EEG)powerdensity,
neurobehavioralperformance,andthecircadianrhythmof
plasmamelatonin
Slow eyelid closure asameasure of
driverdrowsinessand itsrelationship to
performance
Melinda L.Jackson, Susan Raj, RodneyJ. Croft,
Amie C. Hayley, Luke A. Downey, Gerard A.
Kennedy&Mark E. Howard
Traffic InjuryPrevention Volume17,2016- Issue3
https://doi.org/10.1080/15389588.2015.1055327
https://abcnews.go.com/Technology
/blink-eye-dozing-driving/story?i
d=17870880

25. Mechanical
Considerations

26. Eyepiece
You maywant toput the
same hardwarefor various
mechanical platforms
- Handheldversion
- Clip-on
- For VR-type headset
- Optometrists’ glasses
Thus,make the eyecup/piece
modular sothatthe same can be
used inall designs, or alternatively
we cancustomize easily eyecups
for same mechanical connection
Manget al.(2016)
https://doi.org/doi.org/10.1
117/12.2212799
Kohanet al.(2018)
https://doi.org/doi.org/10.1
117/12.2212799
Mobileglasseslight-pupileye-tracker device
(GET)manufacturedbyTobii(Danderyd,
Sweden).ImagefromTobii.
researchgate.net(Beheetal.2013)

27. Eyepiece
Connector
Threadadapter?
“MechanicalAPI”forthe
hardwareallowingany
sortofeyepiecetobe
usedwiththegiven
“standardized”coupling
ring
Ideaof step-up/downadaptersusedfor
photographicfilters
https://www.bhphotovideo.com/explora/photography/hands-
review/wisdom-step-and-step-down-rings
Couplingring with samesizethreadsonboth sides.
i.e. othersidewouldconnecttothe eyepiece andthe otherto the hardware
VenidiceSL-1026800k DimmableMacro LED Ring Flash

28. “Mechanical
API”
Applythe same API forall
the otherdevicesthat you
possiblywant tohookup
withthe platform
e.g. Industrialcameras useC-mount
objectives that have off-the-shelffilter
accessories as well
https://www.edmundoptics.com.sg/optomechanics/tube-system/c-mount-com
ponents/c-mount-double-thread-rings/
+https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_ID=8284
https://www.ptgrey.com/blackfly-32-mp-mono-gige-poe-sony-imx265-2
LensMount:C-mount
PointGreyprogramfor
RaspberryPiusinghardware
encoding
https://github.com/trbotha/PointGrey
_RPi
http://www.lmscope.com/produkt22/Kompak
tkameras_Camcorder_am_Mikroskop_en.shtm
l

29. Auxiliary
Eyecup
Sensors
Modular eyepiece would
allow the use of auxiliary
sensors (if electric contacts
are there?) simply by
changingtheeyepiece
Like pulseoximetry,skin
conductance.or non-contact
skin temperature.
MaximMAX30102
[withArduinolibrary #1,
#2,#3,andtutorial]
TMP006.
https://www.zdnet.com/article/tis-contact-less-temperature
-sensor-opens-up-possibilities/
https://www.mouser.sg/new/Texas-Instruments/ti-tmp006/
Skinconductancesensors(non-cumbersome)
AffectivaQSensor
http://www.prweb.com/releases/2010/11/prweb4
745334.htm

30. Vascular
Dysregulation
& ANS
Short recordingofoximetrynot
maybethe most usefulinmost
cases,soit couldbe anoptional
feature.
Alsoitwouldbe interestedifthere
wasareal-time ”automatic
qualitymeasurement” ofthe
attentionorjust stresslevel of the
subject.Useful for example in
cognitive pupillometry.
ŌURA ring
Apple
Watch
Fitbit Charge HR / PurePulse
Optical heartrate sensor
on Samsungphones
http://dx.doi.org/10.1155/2015/798958,cited
by2articles
Conclusions. In conclusion, these alterations in blood pressure
variability and coupling with heart rate suggest modified autonomic
regulation due to a vascular dysfunction in patients suffering from
glaucoma. The importance of the vascular influence for the
pathogenesisofglaucomaisagain emphasizedbythis study.
Further studies need to show if the method is valuable to identify
systemic autonomic dysfunction in glaucoma. Patients with
systemic autonomic dysfunction might be at higher risk for
progression of the disease due to a higher susceptibility of the optic
nerve to fluctuations in intraocular pressure or ocular perfusion
pressure.
Orsimply “makeconnected” with
wristsensors:
September12,2016
http://www.wareable.com/fitness-trackers/heart-rate-variability-explained
http://www.salu.ca/
←TheSilentKiller,May3, 2016

31. Measurement
Camera
Considerations

32. Camera
Options
Variouslow-costdevicedesigns
havebeenbuilt ontopofthe
RaspberryPiand its camera
module.
Youcouldmakethe cameraunit
alsomodularsothatyou could
popin another cameramodule
withahigherframe rate and/or
higherspatialresolution for
example
PCB ~25x25mm
Camera 8x8x8 mm
http://www.raspiworld.com/viewtopic.php?t=13
A PracticalDeviceforMeasuringthe
LuminanceDistribution
ThijsKruisselbrink ,MyriamAries, AlexanderRosemann
https://doi.org/10.26607/ijsl.v19i1.76 (2017)
“Raspberry Pi 2 model B, was used to control the
camera, carry out the computations and communicate
the results using a Wi-Fi dongle. The camera
functionality was accounted for by the Raspberry Pi
Camera Board version 1.3 with a CMOS sensor (3.60
mm, f/2.9) with a maximum resolution of 2592 x 1944
pixels, comparable to cameras in smartphones. A
miniature equisolid-angle fisheye lens, suitable for the
Raspberry Pi Camera Board, with a measured angle of
view of 187° (3mm, f/0.4) was used on top of the camera
sensortoprovideahemisphericalimage.”
A lotof phone
cameralenskits have
alow-costandlow-
qualityfisheyelens
forquickproof-of-
concept prototyping
forexample.

33. Raspberry
Module as
a Scientific
Camera
RaspberryPi3v2camera
modulegood inexpensive option
for homemonitoring,remote care
in thecommunity,etc.
Layingthe foundation to use RaspberryPi3 V2 camera
module imagery for scientific andengineeringpurposes
Mary A.Pagnutti; RobertE.Ryan; GeorgeJ.Cazenavette; MaxwellJ.Gold; Ryan Harlan; EdwardLeggett; JamesF.Pagnutti;
InnovativeImaging&ResearchCorp.(UnitedStates)
J.ofElectronicImaging,26(1),013014(2017).doi:10.1117/1.JEI.26.1.013014
The Raspberry Pi credit-card-sized
computer supports several
accessories, including a camera
module containing the Sony IMX219
sensor. This computer and camera
configuration is of particular interest
since it can provide raw-data
format imagery thatcan beusedfora
multitude of applications, including
computer vision, biophotonics,
medical testing, patient-centric
healthcare with IoT, remote sensing,
astronomy, improved image quality,
high dynamic range (HDR) imaging,
and security monitoring. This paper
evaluates the characteristics of the
Raspberry Pi V2.1 camera based on
the Sony IMX219 sensor and the
radiometric performance of its raw-
data format imagery, so the system
can be effectively used for scientific
imagingandengineeringpurposes.
Turning the camera on and taking images can cause changes in output due to sensor
warming. The plots show that after approximately a 200 frame warm-up period, data values
reachsteadystate.
RaspberryPi camera mean-variancecurvesfor
ISO100, 200, and 400.DN = digital number (as
in pixel intensity)
RaspberryPiCameraV2
spectral response.

34. Raspberry
Module
derivatives
Somepeoplebundlethe camera
withuseful add-ons.And you
couldthinkofdoingyour own as
wellifyoufindituseful foryour
application (andeven publish
that?)
InfraredNightVisionSurveillanceCamera+
2-PcsInfraredLights forRaspberryPi
$21.99
This camera module features with 5MP (2592×1944
pixels) and 70 degrees viewing angle. It is compatible
withbothRaspberryPimodelAandmodelB.
OV5647 sensor, the sensor itself has a native resolution
of5megapixel,andhasafixedfocuslensonboard
The CSIbusiscapableofextremelyhighdatarates,and
it exclusively carries pixel data to the BCM2835
processor
Supports 1080 p @ 30 fps, 720 p @ 60 fps and 640
x480p60/90videorecording
https://www.sainsmart.com/products/infrared-night-vis
ion-surveillance-camera-2-pcs-infrared-lights-for-ras
pberry-pi

35. Raspberry
Pi in
Ophthalmic
Imaging
Somepeoplebundlethe camera
withuseful add-ons.And you
couldthinkofdoingyour own as
wellifyoufindituseful foryour
application (andeven publish
that?)
https://www.raspberrypi.org/blog/an-affordable-ocular-fundus-camera/
Journal of Ophthalmology
Volume 2017, Article ID 4526243
https://doi.org/10.1155/2017/4526243
Research Article
A Portable,Inexpensive,
NonmydriaticFundus
CameraBasedonthe
Raspberry Pi®Computer
BaileyY. Shen1
and Shizuo Mukai2
1
Department ofOphthalmology, Illinois Eyeand EarInfirmary, UniversityofIllinois at
Chicago, 1855 WestTaylor Street, Chicago, IL 60612, USA
2
Retina Service, Massachusetts Eye and EarInfirmary, Harvard Medical School, 243
Charles Street, Boston, MA 02114-3096, USA
The Supplementary Instructionsavailableonlineat
https://doi.org/10.1155/2017/4526243 provide a
step-by-step description ofhow tobuild our
prototype camera. Alsoavailable online isa
SupplementaryPartsList, whichprovidesacost
breakdown.
Total cost ofcamera:$185.20

36. Camera
Upgrade
Whatifyouneed heavierlifting
thatRaspberryPicangive you.
“Intellgentimaging” forexample
withthe NVIDIATegra/Jetson
embeddedGPUs.
Make sure then thatyour system
accommodatethismodules?
https://devtalk.nvidia.com/default/topic/1005570/jetson-tx2/best-cameras-for-jetson/
Ingeneral,ifyourapplicationisextremelyintensiveandyouarefightingfor everylastdropofperformance(highFPS,high
resolution,etc)andyouneedlowlevelcontrolofthecamerayouwanttobeusingaCSIcamera,ifyour applicationisfairly
light(i.e.youdon'tneedsuper highFPS,resolutionetc)then USBcamerasarefine,easier touse,andhavebetter
supportfromthemanufacturers.
Somepopular USBCameramanufacturers/suppliers:
● Flir(wasPointGreyResearch),http://www.flir.com.hk/home/ -12+weekleadtimeonmostmodels
● Basler,https://www.baslerweb.com/en/ -17+weekleadtimeonmostmodels
● iDS,https://en.ids-imaging.com/home.html-Never usedthembutlook nice
● Ximea,https://www.ximea.com/ -Greatcameras,moreexpensive
● TheImagingSource,https://www.theimagingsource.com/ -drivers/APIarenotaswelldocumentedastheothers
● AlliedVision,https://www.alliedvision.com-never usedthem,mostlydifferentinterfacesbuthavesomeUSBmodels
● LeopardImaging,https://leopardimaging.com/-nVidiaPartner,basiccamerasthoughnowherenearfeaturesof
above
● Therearemanymanymore...Justgoogle"MachineVisionCamera"
● Theother optionisgenericwebcamswhichcanbeuseddirectlybyOpenCVwithoutanycustomdriversetcbutgive
almostzerofeaturesandcontrolonexposure,gain,framerate,etc
Somepopular CSIBusmanufacturerswithTX1support:
● LeopardImaging,https://leopardimaging.com/
● e-ConSystems,https://www.e-consystems.com/

37. Cameras
Ximea
Smallformfactor withvarious
camerasfor alot ofapplications
withseeminglygoodAPIsupport
aswellhttps://www.ximea.com/
https://www.ximea.com/en/products/xilab-application-specific-custom-oem/Embedded-vision-cameras-xiX
ExamplesofsolutionsandcomponentsforEmbedded vision systems fordeeplearning tasks

38. Ximea
Modularity for
their cameras
as well
Smallformfactor withvarious
camerasfor alot ofapplications
withseeminglygoodAPIsupport
aswellhttps://www.ximea.com/
The xiC is an extremely diversified and highly modular camera family designed for the Sony
Pregius™ sensor series. It offers multiple choices of combining sensors and interfaces. Even
though it isaboard stack, for systemintegratorsit hasallthebenefitsofasingleboarddesign and is
incrediblysmall.
The small xiX cameras are based on the same board-level concept and stream images to
the host computer via 2 lanes on a PCI Express Gen2 bus. Together with minimal latencies and
CPU load, the cameras are a perfect fit for embedded vision and multi-camera applications.
Thanks to flat flex cabling, the board-level and semi-housed variants allow integration in tight
spacesandcloseproximitybetween cameras.
Integrated CanonEF-mount controlwithdynamic apertureandfocuscontrol
Remotelyusablesensor boards- longdistance reach
OEMtypeofproduct-specialconditionsmayapply
https://www.ximea.com/files/brochures/xiX%20Infographic.pdf

39. Cameras
E-Con Systems
Lowercost thanXimea,
interestingoptionsaswell for
ophthalmicapplications
HIGH-DYNAMICRANGE
This camera can be directly interfaced to camera connector(J22) on the NVIDIA® Jetson TX2
developer Kit. e-CAM20_CUTX2 is exposed as a standard V4L2 device and customers can use
theV4L2APIsto accessthiscameraandcontrolthesame.
● Houses e-CAM20_CU0230_MOD -2MPCameraModulewithS-mountlensholder
● HighDynamicRange(HDR)andLowlightperformance
● High-speed4-laneMIPICSI-2interfacetoconnectwiththeCPU
● PlugsintoNVIDIA®JetsonTX2/TX1developerkit
https://www.youtube.com/watch?v=ca9fE-RKwBE
https://doi.org/10.1155/2017/8209270

40. Cameras
E-Con Systems
Lowercost thanXimea,
interestingoptionsaswell for
ophthalmicapplications
CustomBayerarray
including IRpixel (at the
expenseofgreenresolution)
ThisKeyFeaturesofOV4682 RGB+ IRCameraModule.
● Omnivision OV4682RGBIR-1/3"OpticalformatCMOSImagesensor
● Fixedfocus,S-Mount(M12P0.5) holderfor Interchangablelens
● e-CAM40_CUTK1-MIPIInterfaceFor NVIDIAJetsonTK1cando330fpsatVGAresolution(640x480px)
https://www.youtube.com/watch?v=W9WpEPj_kLo
Closeupofthecornealreflection

41. Cameras
E-Con Systems
Lowercost thanXimea,
interestingoptionsaswell for
ophthalmicapplications
Autofocuslenseswithliquid
optics iffixedfocusistoohard in
yourapplication
This camera is based on 1/3 inch AR0330 CMOS Image sensor from ON Semiconductor® with
2.2 µm pixel and A-Pix™ technology and an integrated high performance image signal processor
(ISP). ThisS-Mount(M12)liquidlenscameraenables autofocuswith nomechanical moving
part like VCM (Voice Coil Motors) will increase the lifetime and has an ability to focus and refocus
inamatterofmilliseconds.
● Housese-CAM31_MI0330_MOD-3.4 MP CameraModulewithS-mountlensholder
● High-speed4-laneMIPICSI-2interfacetoconnectwiththeCPU
● PlugsintoNVIDIA®JetsonTX2andTX1Developer Kit
TARGETED APPLICATIONS
» BuildingDrones
» AutonomousRoboticSystems
» UnmannedAerialVehicles(UAVs)
» MobileMedicalImaging
» IntelligentVideoAnalytics(IVA)
https://www.youtube.com/watch?v=iwhEsWas3nQ

42. Smartphone
Camera
modulesimproving
aswell

43. Wavelength
Sensitivity
Maybe youwanttomake it
more sensitive ontheinfrared
range.Checkthe datasheetsfor
spectralsensitivity.
Youcan alsomake the“camera
faster” byultrafastlightingpulses
ifyoucan synchronizethe lighting
withthe camera(I/O,GPIOon
the cameraside),thusreducing
motion blurin practice.
e.g. At60fps,maximumframe
exposureis~16.7 msof which
wecouldkeep the IR
illuminationononly forthefirst
1 msforexample
(if thetimerhasenough bitsfor
PWM)
https://www.raspberrypi.org/products/pi-noir-camera-v2/

44. Trigger I/O
For most of thescientific
paradigms(e.g. experimental
stimulusdefinedinPsychoPy),
youwanttosynchronize the
measurements(timestamp)with
the stimulusdeliverysoyoucan
for example analyze the epochs
(inSSVEPforexample,see
nGoggle)
AschematicshowingaTSI-IOBOB2connectedtoanArduinoinacustomcamerasystem.
ScientificCamerasforMicroscopy: FastFrameRate
https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=7485

45. Design
Optomechanical
Considerations

46. Optical
Design
Softwarelicenses areexpensive
Dependingonyourdesign
workflow,youprobablywant
interoperabilitybetweenoptical
designandoptomechanical CAD
software
Open-source BEAMIVusedbydeMeloin hisM.Sc.Thesisfor
mobilefunduscamera (2017). http://hdl.handle.net/10362/27916
Zemax used bySamaneigoet al.(2012) forA
Portable,ScalableRetinalImaging System
Wuetal. (2017)usedbothZemax andLightToolstodesign the
opticalsystemoftheirbinocularfundus camera
https://doi.org/10.1080/24699322.2017.1379143

47. Mechanical
design jointly
with
optical design
Zemax defactostandardfor
imagingoptical designwith
optomechanical integration
(LensMechanix),
withLightTools offeringthe
CAD integration(Solidworks),
whichisused morewithnon-
imagingopticsdesign
https://www.zemax.com/products/lensmechanix
https://www.synopsys.com/optical-solutions/lighttools.html

48. Example of
design
OpticalDesignofaRetinal Image
Acquisition Device forMobile
DiabeticRetinopathy
Assessment
DavidSimõesdeMelo;
NOVAUniversityofLisbon
September, 2017
Mechanical prototype designed
using Solid Works in red with the
rack and pinion assembly in gray
andthesmartphonein black.
● Optical Table
● Optical Post Holders, ThorLabsand Edmund Optics;
● S-LAH64CNC-Polished Aspheric Lens, 40.0 mm Focal Length, 50 mm ø, ThorLabs;
● N-BK7 Best-Form Lens, 40mm Focal Length, 25.4mm ø, Thorlabs
● Microsoft Lumia360;
● N-BK7 Plano-Convex Lens, 38.1 mm Focal Length, 25.4mm ø, VIS-NIRCoated, Edmund
Optics;
● Adjustable Iris;

49. Measurement
Illumination
Considerations

50. Pupil IR
Wavelength
Optimization
for Eye
tracking
Thedifference of 850nm vs.
950nm vs. 1050nm for iris
colors.
Have all wavelength in some
IR illuminationmodel andadd
auto-illuminationfeature?
SPEED: SPectral eyevidEodatabase
AnaGebejes, PauliFält, RomanBednarik, MarkkuHauta-Kasari
University of Eastern Finland,Joensuu,Finland
UbiComp '16 https://doi.org/10.1145/2968219.2968335
Ahyperspectralimagingsystemforthe
evaluationofthe human irisspectralreflectance
LucaDi Cecilia; FrancescoMarazzi;Luigi Rovati
Univ. ofModenaandReggioEmilia, Italy
SPIE BiOS, 2017,doi: 10.1117/12.2252184
Hyperspectral opticalimagingofhuman
irisin vivo: characteristicsof
reflectancespectra
Jose M. Medina,LuísM. Pereira,Helder T.Correia,and Sérgio M. C.
Nascimento,UniversityofMinho,Portugal
Journal of Biomedical Optics16(7),076001 (July 2011)
Reflectance factor as a function of the wavelength
measured with the hyperspectral system. Reflectance
data correspond to (a) dark (orange), (b) light
pigmented iris (cyan)
Our study provides evidence for hyperspectral imaging
being suitable for the characterization of melanin and the
noninvasive diagnosis ofocular diseases and iris color.
Iris reflectances in the visible/near-infrared spectral region.
Light blue iris (blue line), hazelnut-green iris (green
line)and darkbrown iris (brown line).
(right) RGB image generated form the liquid crystal
tunable filter (LCTF) spectral image; Bottom – spectral
signatures of the points selected from the spectral image.
These are mean spectra from a 10x10 pixel areas sampled
from thefeatures ofinterestmarked on the RGB image.
Inordertomake the life the easiestforthe eye tracking
algorithm.We wanttohave the maximumintensity
difference between the irisandpupil tomake the life ofthe
algorithm the easiest.
Brown irises typicallygive the best imagequality

51. Machine
Vision
illumination
GOAL Theintensity
differencebetweenthe
irisandpupilaslargeas
possiblethroughthe
wholeboundary
http://www.ni.com/white-paper/6901/en/

52. Pupil
Illumination
Angle upward
illumination
reduces
shadows
Good ideato add channels for
IRLEDdriver(ifnotalready
present)?
Soin theory “autoillumination
algorithms” are possible for
optimal iris-pupil bounder
differentiation

53. Pupil Auto-
Illumination
Inspiration from
fundus imaging
Inothertrackreal-time
thecontrastofpupil-iris
boundaryandoptimize
lightingaccordingto
that
Linetal.(2017), in other words how sharp are
vessels
And this could be of course later expanded to
more sophisticated features (e.g.
Shao etal. (2017)), and later to deep learning in
the machine reducing operator-dependent
artifacts
https://social.technet.microsoft.com/wiki/content
s/articles/27382.windows-phone-how-to-provide-cus
tom-autofocus-and-white-balance-control-on-window
s-phone-8.aspx
AReal-timeContinuousAutomatic Focus
AlgorithmforDigitalCameras

54. Measurement
Filtering the
Illuminant

55. Camera
Filters
Polarizationfilter
forremoval of
polarizedspecular
reflections(i.e.Purkinje
Images)as done by
Lealetal.(2012)
Easiertodetectpupil→
boundaries
Atworst, the reflection(s)
can mask abig portionof
the pupil.
Aslam et al.(2009)
http://doi.org/10.1098/rs
if.2008.0530
The first photo is taken with the polarizer filter on the camera rotated to be parallel to the polarizer on the light (A), and a
second photo with the polarizer rotated 90 degrees to be perpendicular, and thus cross-polarized (B). Then in Nuke,
subtract the two images (A-B) giving you the specular difference. You will need to desaturate this as well for dielectric
materials. The final image shows the result of adding the cross-polarized "diffuse only" image with the derived specular
image(B+C).http://docs.sharktacos.com/photography/xpol.html

56. Camera
Filters
Polarizationfilter
Simple polarization filter
makestheobjectto
appearmore evenly
illuminated, whichiseasier
foryourmachine vision
then.
Thesepolarizedfilter
sheetscanbeeasily
cutwithscissorsor a
papercutter.
Thickness:0.21mm
Transmittance:43%
Units10< $3.14/pc
http://www.3dlens.com/shop/linearpolarizer.php
http://slideplayer.com/slide/10832585/
http://www.dentistryiq.com/articles/2017/03/cross-polarized-reflective-light-den
tal-photography-the-silver-bullet-of-tooth-shade-selection.html
Dichromaticreflectionmodel.
Sunetal.(2017)
http://dx.doi.org/10.3390/s17010215

57. Illuminant
Filters
Interferencefilters
For sometasks theLED
spectrummightbe toobroad. In
orderto“monochromatize” your
light,youneed abandpass
interferencefilterthat can beabit
priceyforlow-cost devices.
488nm Interferencefilter fromEdmund
opticswouldbeoptimalformelanopic
orcornealmelanopsin sensitivity
PCB-levelholders foroff-the-shelffilters?
Schott Advanced Optics| Optical Interference Filters
https://www.schott.com/advanced_optics/english/products/optical-components/optical-filters/interference-filters/index.html

58. “Silent
Substitution”
LED
illumination
Vexample ofhow togeta
multispectral camerabyfiltering
the illuminants,orsimplyby
selectingmanydifferent LEDs
Fillinthe wholevisual spectrum,
optimized forphotoreceptors.
CustomLED module would allow
easyuseinanimal researchtoo.
HyperCam costs approximately $800 and is based on a FL3-U3-13Y3M-
C USB 3.0 camera from Point Grey with CMOS image sensor that is
sensitive from 350-1080 nm with peak quantum efficiency at 560 nm. The
camera uses 17 different spectral bands that are created using
narrow-band, off-the-shelf LEDs configured in a ring arrangement. The
wavelengths for these LEDs vary from 450 to 990 nm, and were selected
to cover the camera's sensitivity range.
https://www.vision-systems.com/articles/print/volume-20/issue-11/depa
rtments/snapshots/december-snapshots.html

59. Illumination
Ophthalmic
Optics
Constraints
Differentstructures
havedifferent
spectralproperties

60. Ocular
Media
Crystalline lens
and cornea
vandeKraats and van Norren (2007): “Optical densityof the aging human ocularmedia in
the visible and theUV”. J. Opt. Soc. Am. A/ Vol. 24, No. 7/ July2007
https://doi.org/10.1364/JOSAA.24.001842 | Cited by124 articles
PetteriTeikari, Raymond P. Najjar, Kenneth Knoblauch, Dominique Dumortier, Pierre-
Loïc Cornut, Philippe Denis, Howard M. Cooper and ClaudeGronfier "Refined flicker
photometrytechniqueto measureocular lens density" J. Opt. Soc. Am. A Vol. 29, Issue
11, pp. 2469-2478 (2012) https://doi.org/10.1364/JOSAA.29.002469
Human crystalline
lensgetsmore
yellowwith age

61. Macular
Pigment
Across the macula, macular protective pigment (MPP) distribution
takes the form of a mountain, peaking centrally at the foveola and
declining to nil at an eccentricity of 7°. L – lutein, Z – zeaxanthin.
optometricmanagement.com
Recovered spectra for one normal subject. The four spectra recovered by NMF. The
fourth spectrum denotes the MP spectrum (solid line). The two peaks between 450
and 500 nm are the classic bifid spectrum previously reported (Hammond etal.2005) in
vitro.Thesecondarypeaksat 425nmhavealso beenreported invitro. - Fawzi etal. (2011)
Clinical imagingof macularpigment optical densityand
spatialdistribution
Christopher M Putnam | College ofOptometry,University of Missouri-St Louis
Clin Exp Optom, 100: 333–340. doi: 10.1111/cxo.12500
Spectralis optical coherence tomography (OCT)
provides a cross-section of the central retina of a
healthy human subject. The layers of the retina have
been identified with arrows. The vertical
distribution pattern of macular pigment (MP) is
identified primarily within the photoreceptor
axons that comprise the outer plexiform layer (OPL),
the inner plexiform layer (IPL) and Henle fibre layer
within the macula. Lesser concentrations of macular
pigment have also been identified at the level of the
retinal pigment epithelium (RPE) and photoreceptor
outer segments.
Macular pigment is deposited preferentially in the fovea in the Henle fiber layer which
consists of the foveal cones’ axons, and in the parafovea, macular pigment is also located in
the inner plexiform layers of the retina (Snodderly, Auran &Delori,1984;
Trieschmann,etal., 2008).
Macular pigment optical densitymap of oneeye included in
the study. - VerônicaCastroLimaet al. (2013)

62. Irisand
Ocular
Wall
“In special cases such as with intraocular straylight measuruments (
Ijspeertetal. 1990; vanden Berg etal.2009; Michael etal.2009), the
transmural (ocular wall) and iris transmittance need to be explicitly
addressed in contrast of assuming them to be light-tight structures (
van den Berg et al. 1990). The translucency of iris and the ocular wall are
exploited by ophthalmologists when performing diaphanoscopy (cf
Greenwood 1913), in which a light guide is positioned against the sclera
andthepupil isseentoglowfromwithin (La Heyetal.1993).
Vanden Berg etal.1991 estimated the irises of blue-eyed individuals to
attenuate the red light only 0.72 log units and the green for 1.48 log units,
whereas the corresponding attenuation values were 2.27 for red and 2.64
log unitsforgreen lightin brown-eyed individuals.
In addition to the translucency of the iris and the surrounding ocular wall,
fundal reflections (Vos1963; van deKraatsand vanNorren2008) might
contribute to the pigmentation-related differences. Furthermore, the iris
pigmentation have been shown to correlate directly with choroidal
pigmentation (Weiteretal. 1985) and to be reduced with aging (
Schmidtand Peisch 1986).”
SPEED:SPectraleyevidEo database
AnaGebejes,PauliFält, RomanBednarik, MarkkuHauta-Kasari
University ofEastern Finland,Joensuu,Finland
UbiComp '16 https://doi.org/10.1145/2968219.2968335
Ahyperspectralimagingsystemforthe
evaluationofthehuman irisspectralreflectance
Luca Di Cecilia;FrancescoMarazzi;Luigi Rovati
Univ. ofModenaandReggioEmilia, Italy
SPIE BiOS,2017,doi: 10.1117/12.2252184
Hyperspectral opticalimagingofhuman
irisin vivo: characteristicsof
reflectancespectra
JoseM.Medina, LuísM. Pereira,Helder T.Correia,andSérgio M.C.
Nascimento,UniversityofMinho, Portugal
Journal of Biomedical Optics16(7),076001 (July 2011)
Reflectance factor as a function of the wavelength
measured with the hyperspectral system. Reflectance
data correspond to (a) dark (orange), (b) light
pigmented iris (cyan)
Our study provides evidence for hyperspectral imaging
being suitable for the characterization of melanin and the
noninvasive diagnosis ofoculardiseases and iris color.
Iris reflectances in thevisible/near-infrared spectral region.
Light blue iris (blue line), hazelnut-green iris (green
line)and darkbrown iris (brown line).
(right) RGB image generated form the liquid crystal
tunable filter (LCTF) spectral image; Bottom – spectral
signatures of the points selected from the spectral image.
These are mean spectra from a 10x10 pixel areas sampled
fromthefeatures ofinterest marked on theRGB image.

63. Sclera
Vogel et al. (1991):Optical Properties ofHuman Sclera, and TheirConsequences for
Transscleral Laser Applications. LasersinSurgeryand Medicine11:331340(1991)

64. Eyelid
M. J. Moseley, S. C. Bayliss and A. R. Fielder (1988) Light transmission
through the eyelid:doi:10.1111/j.1475-1313.1988.tb01043.x
Spectral transmittance of arbitrary unit amounts of hemoglobin,
melanin, and bilirubin used for predicting the spectral transmittance of
eyelidskin.
Bierman etal. (2011): Measuring and predicting eyelid spectral
transmittance.J.of BiomedicalOptics,16(6),
MillisecondFlashesofLightPhase Delay theHuman Circadian
Clockduring Sleep JamieM.Zeitzer,RyanA. Fisicaro, Norman F. Ruby,H.
Craig Heller.StanfordUniversity.Journal of Biological Rhythms201429(5): 370-
376. doi: 10.1177/0748730414546532
“Confirmation that the flashes penetrated the
eyelids is presented by the occurrence of an
evokedresponseintheEEG.“

65. Stimulus
presentation
Electromechanical
considerationswhen
selectingLEDs

66. LED
illumination
For ERG,SSVEP,
Pupillometryand various
photoreceptionstudies,
youwanttoprovide
illumination asthe stimulus
The cheapest setup with
SMDRGB(W)LEDs
CreeCLQ6A-TKW-C1L1R1H1QBB7935AA3
Wavelength/ColourTemperature:624nm,535nm,475nm,5700K
LuminousFlux/RadiantFlux:14lm,30lm,8.2lm,25lm
Package/Case:PLCC-8
Unitprice$0.942/100pcs
Dimensions:5.2mmx5x1.1mm
https://www.mouser.sg/ProductDetail/Cree-Inc/CLQ6A-TKW-C1
L1R1H1QBB7935AA3?qs=BZBei1rCqCCs2mRhgHvO8g%3d%3d

67. RGB LEDs 7 matches found with the higher outputs
https://www.mouser.sg/Optoelectronics/LED-Lighting/LED-Emitters/_/N-8
usfd?P=1y962ekZ1y962e7Z1y961e7Z1z0z7ptZ1y9fnu7Z1yyjzvjZ1y95alhZ1yzrdo
0
CREE CLQ6A-TKW-
C1L1R1H1QBB7935AA3
https://www.mouser.sg/Produc
tDetail/Cree-Inc/CLQ6A-TKW-C
1L1R1H1QBB7935AA3?qs=sGAEpiM
ZZMtKo6xK7%252bVI1jd2MmfBQ0t
Q9nz6Da7%252b9xH%252b%252bB9
Z83G0Qw%3d%3d
Luminous Intensity:
3000 mcd, 7030 mcd, 1824 mcd, 5860 mcd
624 nm, 535 nm, 475 nm, 5700 K
http://www.cree.com/
led-components/media
/documents/ds-CLQ6A-
TKW.pdf
If
- Forward Current:
100 mA
Vf
- Forward Voltage:
2.1 V, 3 V, 3.1 V, 2.9 V

68. RGB LEDs
HighPower
Expensive formassproduction
but ifyou make yourPCBrings
the samesize,easytodo
plug’n’playreplacement ifhigh
intensities neededinresearch
settings
Optek/TT ElectronicsOVSPRGBCR4
LuminousIntensity: 9000 mcd (9x),14000 mcd (7x),3550 mcd (3.5x)
Wavelength/ColourTemperature:625 nm, 525 nm,465nm
LuminousFlux/Radiant Flux:14 lm,27 lm, 57 lm
Vf -ForwardVoltage:2.3 V, 3.4V, 3.4V
If -ForwardCurrent:250 mA
LED Size:6 mm x 6 mm x 1.5 mm
https://www.mouser.sg/ProductDetai
l/Optek-TT-Electronics/OVSPRGBCR4?
qs=VHdrSJkwcuYnu6VZIPrX7Q%3d%3d

69. ‘Standardize’
PCB area for
the LED
module
Plug’n’playmodel for easy
change ofexperimental
lights.
Youcan populate the
same PCB withdifferent
LEDwavelengthsfor
different purposes
(thumb)-screw or probably
some other quick-snapon
connector mechanism?

70. Standardize
LED sizes?
Need toonlydoonce the
PCB designswithplaces
forthe SMDLEDs,and
then theycan be
populated withdesired
colors.
Ordocouple ofversionsat
most withdifferent
package sizes
KingbrightAPG0603PBC-TT-5MAV
467nm0201 0.6x0.3mmSMDLED
Notthebrightest
Only115optionsfor 0201-0402packages
PLCC-4 has302optionswith23RGBLEDs
with21BLUELEDs(464-475nm)
PLCC-6has70options
PLCC-4
https://www.mouser.sg/leds-smd/

71. Going too
Small?
Harderand more
expensive then to
manufacture.
But ifyou standardizeonly
connectorson PCB,easy
touse different layouts
https://electronics.stackexchange.com/questions/30671/esti
mating-assembly-cost
In2017, 0402is completely standard
and no contract manufacturer will blink
aneye,whetherinprototype ormass
production quantities.
https://electronics.stackexchange.com/questions/31460
4/is-0603-vs-0402-assembly-mass-production-cost-optim
ization-still-relevant
https://www.embedded
.com/design/prototyp
ing-and-development/
4429499/Applying-Des
ign-for-Manufacturin
g--DFM--to-PCB-devel
opment-

72. LED Drivers?
Youneed todrive the
LEDssomehow
i.e.todeterminethe
currentthroughLEDs
Sowhat isthe maxcurrent
youneed,whatsortof
frequencyyou mayneed.
The better the dynamic range, the shorter pulses you
can generate, and/or higher modulation frequencies

73. Example LED
Drivers
How about other
features?
One driverper circular
PCB,sothere should be
enoughchannels?
Dutycycle control with
programmableforward
current(If
)?
The LEDsare'chainable' byconnectingthe output ofone chip
intothe inputof another -see the datasheet for diagramsand
pinouts. Toallow the entire chip tobe integrated intoa6-pin
package, thereis asingledatalinewith avery timing-specific
protocol.
Sincethe protocol is verysensitive to timing, it requiresa
real-time microconroller such asanAVR, Arduino, PIC, mbed,
etc. It cannot be used witha Linux-based
microcomputer or interpreted microcontroller such asthe
netduinoor Basic Stamp. TheLEDsbasicallyhave aWS2811
inside, but fixed at the800KHz'high speed'setting.
Our wonderfully-written
Neopixel libraryfor Arduinosupportsthesepixels! Asit requires
hand-tuned assemblyit isonlyfor AVRcores but othersmay
have ported thischip driver code sopleasegoogle around. An
8MHz or faster processor isrequired.
544Matchesleft withthesamespeed(at least 800 kHz)
https://www.mouser.sg/Optoelectronics/Drivers/LED-Lighting-Drivers/_/N-7zi53?P=1z0z7pt&Rl=7zi53Zer
avZ1y9l30tZ1yydjbaSGT

74. Example LED
Drivers
The high-powerRGB led
hasmaximum forward
currentat250mA, and the
Cree one at 100mA.
Highest forwardvoltage
now is3.4V(need aboost
converterifyourbattery
voltage isbelow this)
This filtering leaves us with 187 options
(when trimmingbulky packagesaway aswell)
https://www.mouser.sg/Optoelectronics/Drivers/LED-Lighting-Drivers/_/N-7zi53?P=1z0z7pt&Rl=7zi53ZerbyZ1z0xsyoZ1ya7ku2Z1z0y1pq
Z1z0jilrSGT7zi53ZerbbZ1z0wbyxZ1yxbqmiSGT7zi53ZeravZ1y9l30tZ1yydjbaSGT&Ns=Pricing%7c0
Enabling Multiple
channels leaves us only
17 drivers now
(MostlyTexasIntruments)
https://www.mouser.sg/Optoelectronics/Drivers/LED-Lighting-Drivers
/_/N-7zi53?P=1z0z7ptZ1yzvta6Z1yzvta3Z1yzxao1Z1yx7p9yZ1yzvta4&Rl=7z
i53ZerbyZ1z0xsyoZ1ya7ku2Z1z0y1pqZ1z0jilrSGT7zi53ZerbbZ1z0wbyxZ1yxb
qmiSGT7zi53ZeravZ1y9l30tZ1yydjbaSGT&Ns=Pricing%7c0

75. Example LED
Drivers
TexasIntruments
http://www.ti.com/power-management/led-driver/indicator-rgbw/overview.html
http://www.ti.com/power-management/led-driver/products.html?pqs
=paqs&familyid=480#p2823=250;5000

76. Stimulus
presentation
LED Wavelength
Considerations
forphotoreception
studies

77. Custom Light
Channels
needed
Atleastthese
forhuman
photoreceptionstudies
Foragoodresearch tool (atleastthe“Pro”version)couldcontainone
sensorpereachphotoreceptor:
● Rod sensitivity V’(λ)
● S-cone sensitivity “blue cones”
● M-conesensitivity “green cones”
● L-conesensitivity “red cones”
● “Melanopic” sensitivity “blue cones”
=5channels
● Photopic sensitivity V(λ) is alinear
combination of M and L cone
sensitivities
● Mesopic can becomputed
from V’(λ) and V(λ), but it is a
nonlinear function depending on
the intensity
CORNEAL
RETINAL

78. Custom Light
Channels
needed
Mightaswell
these
Atleastasan
option
Formoreexoticsensitivities,youcouldleaveopenhardwaredesign so
thatphotopigmentintermediatescouldbeadded
● MetarhodopsinII (~380nm) that couldplay role with BlueLight Hazard
(Grimm et al. 2000, 2001), and in photoreversal(Hubbard and Kropf1958)
● M(476 nm) and Estates(453nm)
of melanopsin (in vitro mouse,
Emanuel et al. 2016; purified pigment,
Matsuyamaet al. 2012). R-stateat 471
nm.
=3 channels

79. Melanopsin E-
state?
Clinical
significance?
Zhuetal.(2007)
Findingsofmelanopsin-
mediation potentiation of
PLR
After “supersaturating” light
stimulus (1 minute of 18 W/m2
,
white light from an unspecified
halogen light source) the
following 1 minute blue
“probe” light (470 nm) pulse,
either at half-saturating
irradiance IR 50 or at 70%-
saturating irradiance IR 70 , was
shown to elicit a potentiated
PLR rather than a depressed
PLR.
AnalogoustoM’
photoproductoffly
rhodopsin that
buildsupatextremelight
intensities
Franceschinietal.(1981)
Signaling E-state athighintensities?

80. Stimulus
presentation
Optical
Considerations

81. Withbig boylike
Cree youhave a
lotof LED
OPTICS
options
https://www.mouser.sg/new/ledil/ledillenscree/
https://www.led-profes
sional.com/products/le
d-optics
https://www.led-professional.com/resources-1/articles/how-silicones-are-evolving-to-meet
-the-growing-needs-of-the-led-lighting-industry-by-dow-corning

82. Diffusers
Probablysomeapplicationsare
happierwithlower transmissions
andsmallerhomogeneityoflight
frominexpensivefilters,when
somemightwanttopayforthe
topquality?
Thinkhow tomakethediffusers
easilychangeable too?
Cheapest probablytokeepslots
in theenclosurefor somerigid
diffuser tothrow in,ratherthan a
thread-basedfilter?
https://www.edmundoptics.com/optics/windows-diffusers
/optical-diffusers/Holographic-Diffusers/
From $135
https://www.luminitco.com/
https://www.luminitco.com/downloads/data-sheets
https://www.edmundoptics.com/optics/win
dows-diffusers/optical-diffusers/White-
Diffusing-Glass/
http://visionlux.net/product/diffuser-polarizer/
Malaysia

83. Visual Field
Sections
Full-field
Central vs.Peripheral
Quadrant stimulation
Multifocal
So if you want 17 spectral bands
for 101 different visual field
locations, it might get tricky
place all those LEDs to a very
small PCB.
However such “silent
substitution” device might get
feasible for quadrant design, and
especially for full-field
stimulus in which the light needs
to be well mixed (i.e. diffused)

84. Central vs.
Peripheral
http://doi.org/10.1167/16.8.5

85. Quadrants
http://doi.org/10.1038/srep33373

86. Multifocal
http://doi.org/10.1038/srep45847