High-level design specifications for a portable diagnostic device for ophthalmic structure + function assessment. How to integrate fundus / OCT imaging with a simultaenous functional measurement such as pupillometry (PLR), eye movements, electroretinography (ERG), visual evoked potentials and flicker vessel response. You might want to think how to use the same device, for individual "self-calibration" as well. Use for example heterochromatic flicker photometry for measuring macular pigment and crystalline lens density psychophysically, or use Purkinje imaging to measure ocular media transmittance. Additionally, you would like to measure ECG / PPG for cardiac gating of vascular imaging, and for artifact correction in pupillometry. Especially if you are interested in nonlinear dynamics of the pupillary response. Alternative download: https://www.dropbox.com/s/3n5o6xzca0m22qm/multimodal_portable_diagnostics.pdf?dl=0

1. Multimodal
PortableOphthalmic
Structure+Function
Diagnostics
Petteri Teikari, PhD
Singapore Eye Research Institute (SERI)
Visual Neurosciences group
http://petteri-teikari.com/
Version “Fri 31 August 2018“

2. “Executive Summary”
Forgetabout
thebulky
desktop
instruments

3. Modular Wearable System
Single
camera
for both fundus
imagingand
video-basedeye
movementsand
pupillometry
http://www.yankodesign.com/2016/04/21/e-r-meets-v-r/
Designer:Jonathan Stewart

4. Other Biosignals fromear,bridgeand face?

5. Other Biosignals fortheirintrinsicvalueoras
“correctivesignals”formainsignals
Optimallyrequiredcorrectivesignals asthereseemstobeclear
“artifactualANSmodulation”Onoratietal. 2015
ofpupilsize
●
Respiration,breathingrate (Borgdorf1975; Calcagninietal. 2000; Francoet al.2014)
●
Heartrate, R-Rintervals forHRV (Hungetal. 2006; Bäretal. 2009; Francoetal.2014; Parketal. 2018)
Withthesameheartratemeasuringefortyoucanmeasureoxygen
saturationwithPPG.Andoptionallyyoucouldmeasure:
●
tympanictemperature
●
skinconductance(ANS)
●
bloodpressure.

6. Imaging
E-Conwith
TX1/TX2
Wouldbenice,
butmaybethe
USBoptions
worksas astart

7. SENSOR RGB+IR Array Omnivision
OmnivisionOV4682https://www.ovt.com/sensors/OV4682

8. CAMERA with that sensor

9. CAMERA 40fps still with Raspberry Pi
e-CAM40_CUMI4682_MOD
VolumePricestartsfrom
US$89
RaspberryPiZero
https://devtalk.nvidia.com/default/topic/1005570/jetson-tx2/best-cameras-for-jetson/
In general, if your application isextremelyintensiveand you are fghtingfor everylast drop of performance (high FPS, highresolution, etc)
and youneed low level control ofthe camerayou want tobe usinga CSI camera, if your application isfairlylight (i.e. youdon't need super
high FPS, resolution etc) then USB camerasarefne, easier touse, and have better support from the manufacturers.
Somepopular USB Cameramanufacturers/suppliers:
●
Flir (wasPoint GreyResearch), http://www.fir.com.hk/home/ - 12+ weeklead time on most models
●
Basler, https://www.baslerweb.com/en/ -17+ week lead time on most models
●
iDS, https://en.ids-imaging.com/home.html - Never used them but looknice
●
Ximea, https://www.ximea.com/ - Great cameras, more expensive
●
TheImaging Source, https://www.theimagingsource.com/ -drivers/API are not aswell documented asthe others
●
Allied Vision, https://www.alliedvision.com - never used them, mostlydiferentinterfacesbut have someUSB models
●
Leopard Imaging, https://leopardimaging.com/ -nVidiaPartner, basic camerasthough nowhere near featuresof above
●
e-Con Systems, https://www.e-consystems.com/
●
There aremanymanymore... Just google "Machine VisionCamera"
●
Theother option isgenericwebcamswhich can be used directlybyOpenCVwithout anycustom driversetc but give almost zero
featuresand control on exposure, gain, framerate, etc
Wouldrathergowiththe TX1/TX2 (devkit1kSGD),orwithacameramodulethatcanbestarted
todevelopwithRaspberryPiandthenupgradedatsomepoint
higherframeratesandcanuseheavier algorithms→

10. SENSOR RGB+IR Array ON Semiconductor
AR0237RGB-IR: CMOSImageSensor,2.1MP,1/2.7",RGB-IR
http://www.onsemi.com/PowerSolutions/product.do?id=AR0237%20RGB-IR
Features
●
Superior low-lightperformance
●
DR-PIX(TM)technologywithDualConversionGain
●
FullHDsupportatupto1080p60fpsforsuperiorvideoperformance
●
Linear orhighdynamicrangecapture
●
On-chipphase-lockedloop(PLL)oscillator
●
SupportslineinterleavedT1/T2readouttoenableHDRprocessinginISP chip
●
Supportfor externalmechanicalshutter
●
Integratedposition-basedcolor andlensshadingcorrection
●
Slavemodeforpreciseframe-ratecontrol
●
Stereo/3Dcamerasupport
●
Statisticsengine
●
Four-laneserialhigh-speedpixelinterface(HiSPi),diferentialsignaling(SLVS
andHiVcm),orparalleldatainterface
●
Autoblacklevelcalibration
●
High-speedconfgurablecontextswitching
●
Temperaturesensor
●
Support4 x4 RGB-IRpixelpattern
https://www.mouser.sg/Search/Refne.aspx?Keyword=AR0237

11. SENSOR RGB+IR Array imec
https://www.imec-int.com/en/articles/imec-introduces-new-snapshot-multispectral-image-sensor-that-combines-color-and-near-infrared-imaging
imecPacksNewFastEyeTracking Technologyinto ErgonomicDesign for High-Quality
AR/VR Experiencesfor healthcare applications such as
early detection of neurodegenerative diseases
https://www.imec-int.com/en/articles/imec-packs-new-fast-
eye-tracking-technology-into-ergonomic-design-for-high-qu
ality-ar-vr-experiences
Same”shop”foreye movement,
andhyperspectralimaginggear
https://www.imec-int.com/en/hyperspectral-imaging
Hyperspectral imaging camera demo-kits including imec HSI
sensor(s), camera housing, lens, tripod mounts, refectance
calibration tile, lighting solution, storage box and imec HSImager™
hyperspectral imaging software license are available from imec
directlytoease your applicationwork:

12. Illumination
RGB+IR
ForbothFUNDUSand
EYEMOVEMENTS

13. RGB(W) LED sharing the optics
e.g.6x Optek/TTElectronicsRGBLED OVSPRGBCR4
LuminousIntensity: 9000 mcd(9x), 14000 mcd (7x),3550 mcd (3.5x)
Old RGB Intensities: 700-1000 mcd, 1500-2200 mcd, 700-1000 mcd
Wavelength/ColourTemperature:625 nm, 525 nm,465 nm
LuminousFlux/Radiant Flux:14 lm, 27lm, 57 lm
Vf - Forward Voltage:2.3 V, 3.4 V, 3.4V
If - ForwardCurrent:250 mA
LED Size:6 mm x6 mm x1.5mm
LED Engin RGBW
LED SMD-8
LZ4-04MDPB-0000
Cree941-CLQ6ATKWAA3
PLCC8SMDRGBW

14. IR LED as separate chip
Unfortunately no RGB+IR LED?
Optek
OP180
OP280 Series
https://sg.rs-online.com/web/c/displays-optoelectronics/leds-led-accessories/ir-leds/?applied-dimensions=
4293786449,4293786418,4294870025,4294867640

15. Slightly tricky with trans-pars-planar illumination
Illuminationwith 565 nmLED(Fig. 6c.
M565L3,Thorlabs) withanarc-shaped
mask.
An aperture was placed at the pupil conjugate
plane to restrict efective imaging pupil size to 2.5
mm for best imaging resolution24,25
, as well as to
reject scattering light from the sclera. A lens was
positioned behind the camera viewfnder and a
cross that was illuminated by an LED lamp
was placed in front of the lens to serve as a
fxation target, so that the testing subjects
could fx their eyes by looking into the camera
through the lensesand look at thecross
Benquan Wanget al.(2018)
https://doi.org/10.1038/s41598-018-27112-x

16. Some space needed on the sides
for all those lights
“Pars planar”
beam RGB+IR
RGB
STIMULUS
Light
~850nm
Need ICA for
spectralunmixing to
reducethespectral
crosstalk?

17. Advanced
Imaging
Functional
vessel
response,
and
“multispectral”
imaging

18. Functional Retinal Vessel Response
LaurenceS.Lim; LiengH.Ling; Peng GuanOng; WallaceFoulds; E.ShyongTai; TienYinWong
Singapore Eye Research Institute, SingaporeNational Eye Centre, Duke - National University ofSingapore, Singapore
Department ofOphthalmology, Yong LooLin School ofMedicine, National UniversityofSingapore, Singapore
The Dynamic Vessel Analyzer (DVA) is a commercially available system consisting of a
fundus camera, video camera, real-time monitor, and a personal computer with analysis
software. The DVA measures retinal vessel dilatation in response to difuse luminance ficker
(Polak et al. 2000).
In conclusion, we demonstrated that reduced retinal arteriolar and venular dilatory
responses to fickering light, are associated with a higher likelihood of DR progression at 1
year in adultpatientswith diabetes. Furtherstudy mayhelptodefne the clinicalvalue ofthe
DVA in predicting DR progression, or in determining its utility as a tool for monitoring
therapeutic efcacy.
Br J Ophthalmol. 2000 Nov;84(11):1285-90.
Evaluation of the Zeiss retinal vessel analyser.
Polak K, Dorner G, Kiss B, Polska E, Findl O, Rainer G, Eichler HG, Schmetterer Leopold.
Relatedarticles
http://sci-hub.tw/10.1167/iovs.17-22461
http://doi.org/10.1167/iovs.17-22461
TheDiameterofRetinalVessels IsAfectedbyTransient
IschemiaoftheArm in NormalPersons

19. Gate the vessel imaging with HR+Respiration
A schematic representation of the neurovascular unit (NVU) showing cellular
elements regulatingcerebral blood fow along the vascular tree. - Kisler etal. (2017):
Cerebral blood fow regulatoo aod oeurovascular dysfuoctoo io Alzheimer disease
Each line represents the mean of 12 retinal vessel diameters of all
12 participants average ficker profles for arteries (top panel) and
veins (bottom panel) for each ficker duration (5, 7, 10, and 20
seconds). Vertical gray line denotes ficker onset; horizontal gray
line denotes baseline diameter normalized to 100%.
Heitmar aod Summers (2015): “Reliability of Usiog Retoal Vascular Fractal
Dimeosioo as a Biomarker io the Diabetc Retoopathy Detectoo”
BiopacDTU200
MRIGatingSystemfor TwoSignals
If youwantto quantifythefunctionalvesseldiameter change,
badideatodofundusvideography withno gating

20. “Multispectral” with optimized illuminants
Imageformationmodel
https://doi.org/10.1364/AO.50.00H177
microaneurysms
hardexudates
arteriesvs.veins
GREEN Optimized A Optimized B
https://sci-hub.tw/10.2352/J.ImagingSci.Technol
.2011.55.3.030509

21. Construct your pathology-specifc illuminants
●
Have same-sized
PCBsmanufactured,
●
Try to use same
size SMDLEDs
●
Selectthenfrom
Mouser/ Digikey/
Farnell /etc. your
desiredLEDspectral
characteristics
●
Preparemechanically
also to usesome
interference
fltersforsome
narrow bands
●
You mightwantto
have multiple
“easilyfltered”
LED chips

22. Construct your pathology-specifc illuminants
●
Have same-sized
PCBsmanufactured,
●
Try to use same
size SMDLEDs
●
Selectthenfrom
Mouser/ Digikey/
Farnell /etc. your
desiredLEDspectral
characteristics
●
Preparemechanically
also to usesome
interference
fltersforsome
narrow bands
●
You mightwantto
have multiple
“easilyfltered”
LED chips
https://www.mouser.sg/Optoelectronics/LED-Lighting/LED-Emitters/_/N-8usfd?P=1z0z7ptZ1z0yshjZ1y
zvpeqZ1z0yrqcZ1z0yra5Z1z0j42b
λ
?

23. Osram 587nm
PLCC-4, 3 lm
Construct your pathology-specifc illuminants
●
Have same-sized
PCBsmanufactured,
●
Try to use same
size SMDLEDs
●
Selectthenfrom
Mouser/ Digikey/
Farnell /etc. your
desiredLEDspectral
characteristics
●
Preparemechanically
also to usesome
interference
fltersforsome
narrow bands
●
You mightwantto
have multiple
“easilyfltered”
LED chips
2
Yellow LEDenough
probably
570nm CWL, 10nm
FWHM, 12.5mm
Stock#67-767
$75.00
λ
?

24. Quantify Hemoglobin on a Fundus photo
2

25. Multispectral meets Flicker Response
http://doi.org/10.1167/iovs.04-0100 |Citedby60
Blood fow (Fonh
) at the neuroretinal rim of the optic disc was
continuously monitored by laser Doppler fowmetry before and during
exposuretoa15-Hz,30°feldgreenluminanceficker.
Scatterplot showing RFonh
data in OD obtained from patients with ocular
hypertension (OHT) or early open-angle glaucoma (EOAG), as a function
ofcorresponding neuroretinalrimarea.
The efectof acutely elevated intraocular
pressure onthefunctionalandbloodfow
responsesof therat retinatoficker
stimulation
BingyaoTan; BenjaminMacLellan; ErikMason;
KostadinkaK.Bizheva (2017)

26. Multispectral, Blood Flow meets IOP
http://doi.org/10.1167/iovs.18-23872
Touselaserspeckle
fowgraphy(LSFG;LSFG-NAVI
device,SoftcareCo.,Ltd.,Fukutsu,Japan
)to
assessbloodfow(BF)inthe
opticnervehead(ONH)tissue
andchoroidduringelevated
intraocular pressure(IOP).
HemodynamicsduringIOP
elevationdifer inthechoroid
andONHtissue.LSFGenables
highlyreproducible
assessmentofthedynamic
autoregulationofocular BFin
theONHtissue.
We consider that our protocol should aid
the study of diseases involving
dysfunctional autoregulation, such as
glaucoma, diabetic retinopathy, and
age-relatedmaculardegeneration(AMD).
http://www.softcare-ltd.co.jp/510k_clearance.html

27. Functional
Studies
With the same
hardware.
The “might as
well” section to
get the most
out the same
hardware

28. Critical Flicker Frequency (CFF) Fusion
RobL.P.vander Veenetal.(2009)
https://doi.org/10.1111/j.1475-1313.2008.00618.x
CFF
Prerequisiteforheterochromatic
ficker fusion test
LifeSciencesbyLafayette
InstrumentCompany
Flicker FusionSystem
PocketCFFTester™
MeasurementDevices:BernellCorporation
https://www.bernell.com/product/cscf/Measurement_Devices
https://homes.cs.washington.e
du/~rkarkar/pubs/workshop-u
bicomp2016-mhsi.pdf

29. Heterochromatic Flicker Photometry (HFP)
12 November 2015
Stimulusedgeefectsinthe measurement of macularpigment using heterochromaticficker photometry
William E. Smollon;BillyR. Wooten;Billy R. Hammond
https://doi.org/10.1117/1.JBO.20.11.115004
MPSII, ElektronTechnology
www.elektron-healthcare.com
HFP requires patients to make ficker matches using
two wavelengths of light, one of which (blue;
465nm) is absorbed by the macular pigment and
another (green; 530nm) which is not. Flicker
matches are made at both the central and peripheral
pointin the retina.
Choice of two test modes: Standard (central, The
1°central target is surrounded by a 30° white area)
and Detailed (both central and peripheral).

30. Heterochromatic Flicker Photometry (HFP)
Clip-ON Accessory?
ANNULUS DISC
Homogenize the light distrubution with
Luminit’sLED Light ShapingDifuser

31. Biosignals

32. Tech exists there, like Fitbit, Ouraring, anearlobe clip, or swim goggle based system
https://doi.org/10.1109/BSN.2015.7299350 | https://doi.org/10.1016/j.future.2017.04.036

33. Respiration withpiezoelectricsensor?
The piezoelectricsensor idea is old there with new tweaks like the pitch from
A*StartCentral, and there are apparently couple of commercially available sensors
on the market, e.g. PiezoelectricRespiration(PZT) Sensor (95eur), or low-cost
estimationpossible with accelerometersaswell.
Nellcor™ SpO₂Adhesive Sensors| Medtronic
Thisstillokayforresearch,
butnot reallyforscreening
https://doi.org/10.1016/j.sna.2018.06.038
RESPIRE: aspectral
kurtosis-based method
to extract respiration
rate from wearable PPG
signals
https://doi.org/10.1109/CHA
SE.2017.64
Validatesomesoftwarealgorithmatthelabusingthe
hardwaresensing,and inthewild usetheRESPIRE
or somethingsimilarfrom thePPG

34. Blood Pressure tootrickyfromthe head
Ifthe SaluPulse+
workswellitisan
interesting,allowing
wearableblood
pressuremonitoring
withHRV,O2
saturationandactivity
levels.
Andifthatparticular
deviceisnotgood,I
assumethatthereisa
hugemarketforeasy-
to-wearblood
pressurewearable
Wrist watchstyle blood pressure monitor
DHLee -USPatent9,943,263,2018-GooglePatents
https://doi.org/10.1515/teme-2017-0113

35. Blood Flow
Theoreticaldevelopmentsand clinical experiments ofmeasuring bloodfowvolume(BFV)
atarteriovenousfstula(AVF) using aphotoplethysmography (PPG) sensor
Pei-Yu Chiang, Paul C.-P. Chao, Chih-YuYang,Der-Cherng Tarng
MicrosystemTechnologiespp1–17|Technical Paper |09April 2018
The mathematical equation is derived under the practical perspective, aiming
at applying to small-size, portable, inexpensive and easy-to-use PPG sensor, in
order to replace bulky, expensive Doppler machines which is the commonly
used instruments for accessing AVF noninvasively nowadays.

36. Skin Conductance notvalidatedforfacial skin?
Skin conductance,useful
if youwant toconsult for
Scientologists or just
wanttoappeal to more
cognitive audiences and
spur further collaboration
with somenew people?
Otherwise, forget aboutit
See e.g. this paper on
ANS andemotion with
skinconductance and
pupil.
https://store.plux.info/42-professional-sensors

37. Skin Temperature Ifyouwouldhave some 3rd
party sensoron
the wrist of the subject,theycould bealertedto
doocularmeasurement by themselves athome,same
true for actigraphy-based alerting
https://doi.org/10.1371/journal.pone.0061142
RelevanceofaMobileInternet PlatformforCapturing
Inter- and IntrasubjectVariabilitiesinCircadian
CoordinationDuringDaily Routine:PilotStudy
FrancisLévietal.
MedInternetRes2018;20(6):e204.DOI: 10.2196/jmir.9779
Home-Basede-HealthPlatformforMultidimensional
TelemonitoringofSymptoms,BodyWeight,Sleep,and
CircadianActivity: RelevanceforChronomodulated
AdministrationofIrinotecan,Fluorouracil-Leucovorin,
and Oxaliplatinat Home—ResultsFromaPilotStudy
FrancisLévietal
DOI:10.1200/CCI.17.00125JCO
ClinicalCancerInformatics
publishedonlineFebruary22,2018

38. Circadian changes of the eye
If you have really motivated subjects on “constanr routineprotocol” youcould alertthem todo
self-image eitherusingthe fundus or OCT
Circadianmacular volume changes
inthe healthyhumanchoroid
American Journal of Ophthalmology
Volume 159, Issue 2, February2015, Pages365-371.e2
https://doi.org/10.1016/j.ajo.2014.11.002
Gerald Seidel, SilkeHausberger, SereinaAnnik Herzog,StefanPalkovits, EvaMariaPöschl,
Werner Wackernagel, MartinWeger
Department of Ophthalmology,Medical Universityof Graz/Austria
Institute forMedical Informatics,StatisticsandDocumentation,Medical Universityof
Graz/Austria
Self-aligned, mobile, non-mydriatic Fundus
Photography. The user is presented with an alignment
dependent fxation cue on a ray-based display. Once
correctly aligned, a self-acquired retinal image is
captured. This retinal image can be used for health,
securityor HMD calibration. Ilustration: LauraPiraino
https://dx.doi.org/10.1167%2Ftvst.6.4.16

39. Raspberry Pi as DAQ
Biosignalsplux(1000eur)had "AuxiliaryPorts: 1digitalI/O+
1ground" witha PythonAPIso youshouldbeabletotakea
wirefromthe GPIOoftheRaspberryPi ofthepupillometer
totheinputofthebox,
andsyncthemsothatyouhavesharedtimecodeasthese
guysfromStockholmhaddonewiththeir BiosignalPi.
Sensors 2015, 15(1),93-109; https://doi.org/10.3390/s150100093
BiosignalPI,anAfordableOpen-Source
ECGandRespirationMeasurementSystem
TheRPIComputeModel; 67.6×30mm
boardasDDR2SODIMM

40. Existing Biomarkers
Of course one should use this, or
at least validate this with existing
clinical workflows

41. Virtual Clinic Workfow
AachalKotechaPhD,etalA,BMJOpen.2015Dec15;5(12):e009463.
http://dx.doi.org/10.1136/bmjopen-2015-009463
https://www.kingsfund.org.uk/sites/default/fles/media/Aachal_Kotecha.pdf
examination inlessthanfour minutes. http://www.saccadio.com/?page_id=25119
https://youtu.be/ntEKT-C_km4

42. Add-ons
Makesurethatyou
canaddnew
modules,andhave
triggerI/Oporton
thesideofmain
platform
Seethe
previoussets
fromthis
“series”↓
ERG http://doi.org/10.1007/978-1-4939-2330-4_25
andmakeitmultifocalelectrogramfornicestructure+function
testing
Clinical Electrophysiology | Visually EvokedPotential (VEP)
MarleeM.Spaford, https://slideplayer.com/slide/4763504/
AdHawkMEMSEyetracker
”Our camera-free eyetracking system enablestruly
mobile datacapture and pavesthewayfor anew generation
ofhighlyimmersive AR/VRexperiences.”
→
https://www.tomshardware.com/news/intel-invests-4.6m-adhawk-eye-tracking,35737.html

43. https://www.slideshare.net/PetteriTeikariPhD/practical-considerations-in-the-design-of-embedded-ophthalmic-devices-100648266

44. https://www.slideshare.net/PetteriTeikariPhD/portable-retinal-imaging-and-medical-diagnostics

45. https://www.slideshare.net/PetteriTeikariPhD/portable-diagnostics-for-visual-function

47. https://www.slideshare.net/PetteriTeikariPhD/developing-a-portable-erg-device-97140760

48. https://www.slideshare.net/PetteriTeikariPhD/portable-multispectral-fundus-camera

49. https://www.slideshare.net/PetteriTeikariPhD/purkinje-imaging-for-crystalline-lens-density-measurement