Download as PDF, PPTX
Uploaded byYoung Min Song
Biomimetics : Compound eyes
The document discusses advancements in biomimetics, focusing on the design and integration of flexible, stretchable electronic systems inspired by natural compound eyes. It compares the structure and functionality of artificial and biological eyes and highlights recent research trends in the development of compound eye cameras for various applications including surveillance. The findings suggest promising future directions for optical device applications based on nature-inspired designs and technologies.
More Related Content
Biomimetics : Compound eyes
- 1. Biomimetics : Compoundeyes Young Min Song Assistant Professor Department of Electronic Engineering Pusan National University http://sites.google.com/site/youngminsong81 1
- 2. A Future forElectronics: Stretchy, Curvy, Bio-Integrated Past Current Future Industrial Personal Bio-Integr. / Bio-Insp. PNAS 106, 10875 (2009). Science 327, 1603 (2010).
- 3.
- 4.
- 5. Bio-integration: examples Transient Electronics Science337, 1640 (2012) 5
- 6. Eyes in animalkingdom Camera-type eye, single lens system Human Fly Bird Ant Fish Shrimp Compound eye (Arthropods eye) : 80% of animal species
- 7. Anatomy of Eyes Camera-typeEye (single lens system) Compound Eye (apposition type) Microlens Lens Retina Optic Nerve Rhabdom Screening Optic Nerve pigment Ommatidium 7
- 8. Artificial (camera) vs.biological (human eye) imaging • Small field of view, high resolution imaging • Complex multi-component lens systems to achieve focal imaging plane with small aberrations • Planar CCD detectors Double Gauss focusing lens CCD detector Light receptors (hemispherical) • High field of view, high resolution imaging lens • Simple lens system • Curved (hemispherical) detectors (retina)
- 9. Imaging With aSingle Lens Planar Camera Ray Tracing 40 20 lens 0 -20 -40 - Planar (commecial camera) - Hemispherical (human eye) - Parabola (ideal) -60 -40 -20 0 Distance (mm) 20 40
- 10. Mimicking the humaneye form hemispherical PDMS transfer element compressed interconnect ~1 cm ~1 cm radially stretch PDMS form Si focal plane array and release from underlying wafer substrate adhesive cure adhesive; flop over substrate compressible interconnect integrate optics & interconnect to control electronics to complete the device transfer focal plane array onto PDMS Si device island (photodetector & pn diode) hemispherical focal plane array Nature 454, 748 (2008) 10
- 11. Mimicking the humaneye Eyeball camera mounted on PCB Hemispherical detector 1 cm 1 cm With single lens Image 5 5 mm 10 12 0 5 5 0 5 Nature 454, 748 (2008) (axis scale: mm) Others: Hawk eye, zooming, etc. 11
- 12. Anatomy of Eyes Camera-typeEye (single lens system) Compound Eye (apposition type) Microlens Lens Retina Optic Nerve Rhabdom Screening Optic Nerve pigment Ommatidium 12
- 13. Research Trends Europe –CURVACE (Curved Artificial Compound Eyes) : 2009-2013, Collaborative project (EPFL, ISF Fraunhofer, etc. ) the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission, under FETOpen grant number: 237940 Japan – TOMBO (thin observation modules by bound optics) : 2000-present, Osaka Univ., etc. US – UCB, UIUC, Harvard Univ., Ohio Univ., etc. : 2000~present, Optic components/systems Science (2006) 13
- 14. Compound Eye Camera CompoundEye Challenge Microlens Rhabdom Ommatidium Optic Nerve Screening pigment Requirement – Full set of microlens/photoreceptor units with hemispherical geometry 14
- 15. Approach – StretchableOptical/Electrical Subsystem Optical subsystem Elastomeric microlens array Combine, stretch Electrical subsystem Stretchable photodiode array Hemispherical Compound eye camera Y. M. Song et al., Nature 497, 95 (2013) 15
- 16. Optical Design Flat ∆φ ∆φ ∆Φ n0 L0 f r Deformed L H rs R n d β Inter-ommatidialangle (∆Φ) ρL0 2Rβ ∆Φ = ,ρ= R 2rs ∆φ0 n0 = 1.0 (air) n = 1.43 (PDMS) > Acceptance angle (∆φ) ∆φ = d rn , f= f n-1 16
- 17. Polymeric Microlens Arrays Mechanicalmodeling PDMS membrane FEM Strain (%) 50 25 0 Optical design Aluminum mold Target FOV ~160° ∆Φ = 11°, ∆φ = 9.7° L0 r = 0.4 mm, dpost = 0.8 mm, L0 = 0.92 mm r f dpost d Mechanical design h t f = 1.35 mm, h = 0.4 mm, t = 0.55 mm d = 0.16 mm
- 18. Electrical Subsystem (Photodiode/BlockingDiode) 2nd 1st Encapsulation metal layer 2nd PI layer metal layer 1st PI layer N+ doped Imaging pixel P+ doped N+ doped Blocking diode Photodiode 200 μm
- 19. Integration of Optical/ElectricalSubsystem Microlens array Integrated form of lens/pixel arrays (flat state) Photodetector array 5 mm 19
- 20. Hemispherical Deformation Fluidic chamber PD/BDarray Compound eye camera PDMS Inlet Outlet Flat Deformed 2 mm Y. M. Song et al., Nature 497, 95 (2013) 20
- 21. Compound Eye Cameras Natural Artificial Blackmatrix Microlens array PD/BD array 2 cm Compound eye cameras mounted on PCB Thin film contact pads Black support Integrated form 21
- 22. Operating principle Image from scanning 10x 10 scanning Image from activated PDs ‘+’ image at each microlens Central portion of a camera (8x8 array) 22
- 23. Measurement setup - 10x 10 scanning for high resolution imaging 23
- 24. Measurement Representative output images z y 30° 60° 90° z x y 30° 60° 90° x y 30° 60° 90° z Modeling z x x y 30° 60° 90° -10 x 10 scanning for high resolution imaging Y. M. Song et al., Nature 497, 95 (2013) 24
- 25. Imaging with WideField of View Object movement Left (- 50°) Center (0°) Right (50°) Laser spot illumination 0° 20° 40° 60° 80° 20° 40° 60° 80° x z y Y. M. Song et al., Nature 497, 95 (2013) 25
- 26. Depth of fieldexperiment Camera 40° - 40° DA = 12 mm DB = 12 mm DA = 12 mm DB = 22 mm DA = 12 mm DB = 32 mm Y. M. Song et al., Nature 497, 95 (2013) 26
- 27. Applications and futureworks Surveillance, Military, etc. http://paulmader.blogspot.com/ Novel imaging systems - Apposition type - Superposition type (refractive, reflective, neural) - Polarization, color, etc. 27
- 28. Night-active insects –Moth, Mosquito, etc. Imaging type Apposition (daylight) Additional nanostructures Moth eye Superposition (night active) 500 nm Hierarchical micro/nano structure 28
- 29. Subwavelength Structures (SWSs) Gratingsin m n1 sin m i n2 n2 Equation -2 -1 0 λ 1 Effective medium theory W 2 h Λ neff Λ -2 -1 1 0 1 0 n2 2 1 n1,eff λ … n4,eff Λ -1 0 0 λ Λ 0 n2 1 Zeroth order grating (ZOG) m 0 Reflectance @ normal incidence n2 n1 R n2 n1 2 Antireflective subwavelength structures 29
- 30. Previous works /Challenges From nature To optical materials Moth eye 500 nm Opt. Lett. 26, 1642 (2001) Nano Lett. 9, 279 (2009) Key Challenges Ideal geometry (period, height, shape, packing density) Optical device applications (PVs, LEDs, etc.) 30
- 31. Ideal geometry ofSWSs Cone Broadband AR: (1) Shorter period (2) Taller height Parabola Moth eye 500 nm - Difficult to integration (3) Shape (4) Packing density Index discontinuity 4.0 4.0 Flat surface SWS (parabola) SWS (cone) nGaAs = 3.7 2.5 2.0 Air 1.5 3.0 100 % 95 % 90 % 85 % 80 % GaAs substrate 2.0 nair = 1.0 Refractive index 2.5 3.5 Air 3.0 GaAs substrate Refractive index 3.5 1.5 1.0 1.0 0 100 200 Height (nm) 300 400 0 100 200 300 400 Height (nm) 31
- 32. Ideal geometry ofSWSs Cone shape Parabola shape 800 Reflectance Height (nm) 700 0% 600 4.0% 500 8.0% 2.0% 400 12% 2.0% 16% 10% 300 2.0% 200 > 20% 10% 10% 100 500 1000 1500 2000 2500 3000 500 Wavelength (nm) 1000 1500 2000 2500 3000 Wavelength (nm) Optical modeling: Rigorous Coupled-Wave Analysis(RCWA) Method Y. M. Song et al., Small 6, 984 (2010) 32
- 33. Parabola shape SWSs Approach– Lens-like shape transfer Interference lithography PR patterns Reflowed PR patterns Parabola-shaped SWS Photoresist Substrate Y. M. Song et al., Small 6, 984 (2010) Period : 300nm 33
- 34. Reflectance characteristics ofSWS Reflectance measurement results Reflectance (%) 50 Normal incidence 40 Bulk GaAs GaAs substrate with and without SWS 30 20 10 500 1000 1500 2000 Wavelength (nm) Reflectance (%) 12 Cone Parabola 10 8 Bulk GaAs SWS GaAs 6 4 2 500 1000 1500 Wavelength (nm) 2000 34
- 35. Optical device applications Gratingequation (reflection) sin r , m m n sin i Photovoltaic devices n = 1.0 Λ≈ λ -1 Absorbing materials m = +1 n ~ 3.5 Back reflector - Higher order diffraction - Reflection minima θr,m : m-th order reflected diffraction angle θi : incidence angle m : diffraction order λ : incident wavelength Λ : grating period n : refractive index of incident medium Light emitting diodes/materials Transparent glasses/materials n = 1.0 Λ≈ λ -1 m = +1 0 Active medium n ~ 1.5 n ~ 3.5 - Higher order diffraction - Total internal reflection Multiple internal reflection 35
- 36. Optical device applications Transparent glasses/materials Lightemitting diodes/materials Photovoltaic devices 800 2 Height o 12.71% 400 13.31% 13.92% 300 1 0 -1 14.52% 200 100 nm, 300 nm, 500 nm, 99 12.10% 500 Z (um) Height (nm) i = 20 o Transmittance (%) Cell eff. 11.50% 600 100 i = 0 Cell efficiency Transmittance 700 98 200 nm 400 nm flat surface 97 96 95 94 93 92 91 100 100 200 300 400 500 600 700 800 Period Period (nm) -2 -0.5 0.0 0.5 -0.5 0.0 0.5 90 300 400 X (um) Y. M. Song et al., Appl. Phys. Lett. 97, 093110 (2010) Y. M. Song et al., Opt. Express 19, A157(2011) 600 700 Wavelength Bare glass Y. M. Song et al., Opt. Lett. 35, 276 (2010) Y. M. Song et al., Sol. Mat. 101, 73 (2012) 500 800 Wavelength (nm) Oneside SWS Bothside SWS Y. M. Song et al., Opt. Express 18, 13063 (2010) K. Choi et al., Adv. Mater. (2010) Y. M. Song et al., ‘Antireflective nanostructures for optical device applications’ 36
- 37. Thank you! Nature Bio-inspiration ‘Beyond biology’ Contact Information Young Min Song ysong@pusan.ac.kr 051-510-3120, 010-2992-8182 http://sites.google.com/site/youngminsong81 37
Editor's Notes
- #3 Future: mount/laminate electronics stretchy, curvy interface. Tissue like form of electronics - Epidermis for health monitoring Brain machine interface Inspiration from biology
- #4 As you know, recently, curved or flexible electronic devices are very big issues in electronics market. And, some of these products such as galaxy round or g-flex are now commercially available. On the other hand, in research fields, stretchable electronics is of great importance for various future applications. One of the examples is biomedical applications. If you want to attach the electronic devices to our body or to implant inside our body, then the device should be stretchable, because our body and organ has curvy and stretchy surfaces. So, many different types of stretchable devices, such as stretchable LEDs, batteries, and other electronic sensors are already reported.
- #5 As you know, recently, curved or flexible electronic devices are very big issues in electronics market. And, some of these products such as galaxy round or g-flex are now commercially available. On the other hand, in research fields, stretchable electronics is of great importance for various future applications. One of the examples is biomedical applications. If you want to attach the electronic devices to our body or to implant inside our body, then the device should be stretchable, because our body and organ has curvy and stretchy surfaces. So, many different types of stretchable devices, such as stretchable LEDs, batteries, and other electronic sensors are already reported.
- #6 As you know, recently, curved or flexible electronic devices are very big issues in electronics market. And, some of these products such as galaxy round or g-flex are now commercially available. On the other hand, in research fields, stretchable electronics is of great importance for various future applications. One of the examples is biomedical applications. If you want to attach the electronic devices to our body or to implant inside our body, then the device should be stretchable, because our body and organ has curvy and stretchy surfaces. So, many different types of stretchable devices, such as stretchable LEDs, batteries, and other electronic sensors are already reported.
- #7 In animal kingdom, there are around ten different types of eye and they can be largely categorized by two: one is the camera-type eye, which can be easily found in human, mamalian, bird, and fishes. It consists of single lens and the photo-receptor arrays at the backside of the eyeball. The other one is compound eye, which consists of bundles of microlenses. And these can be easily found in insect’s eye or in crustaceans’ eye like shrimp or lobster something like that. And, more generally, most of the arthropods (in Korean, 절지동물) has this type of eyes, so sometimes it is called Arthropods eye. Because the arthropods possess 80% of all the existing animal species in the world, the study of this type of eye optics is pretty much valuable to develop new type of imaging systems and optic components.
- #11 In the previous slide, I mentioned that there are two major types of eye. In this camera type eye, it has a single lens and the photo-receptor arrays, as known as retina at the backside of the eyeball. On the other hand, in compound eye, it consists of an array of individual imaging unit, (in the biological system), as known as an ommatidium. It consists of microlens and photocensitive part known as rhabdom, and these units are distributed out over the surface of hemisphere. When you build an eye with this type of design, It allow the creature that simultaneously look in all directions at once. And it also provide the interesting capability (for imaging system) such as high sensitivity to motion and infinite depth of field. So, our goal was to make the man-made version of this insect’s eye and to explore these interesting imaging characteristics.
- #12 In the previous slide, I mentioned that there are two major types of eye. In this camera type eye, it has a single lens and the photo-receptor arrays, as known as retina at the backside of the eyeball. On the other hand, in compound eye, it consists of an array of individual imaging unit, (in the biological system), as known as an ommatidium. It consists of microlens and photocensitive part known as rhabdom, and these units are distributed out over the surface of hemisphere. When you build an eye with this type of design, It allow the creature that simultaneously look in all directions at once. And it also provide the interesting capability (for imaging system) such as high sensitivity to motion and infinite depth of field. So, our goal was to make the man-made version of this insect’s eye and to explore these interesting imaging characteristics.
- #16 So, Our team also introduced stretchable electronics for this compound eye systems. As you can see here, first we prepared two different subsystems. In optical subsystem, it consists of elastomeric microlens arrays on a thin base membrane. This can be simply made by molding of polymeric materials like a PDMS. Electrical subsystem is very thin and stretchable silicon photodetector arrays, which is fabricated on a silicon on insulator wafer and it is transferred to thin polymeric membrane. And then, these two systems are bonded each other with a precise alignment process, after that, by using some mechanical tools, this integrated system can be deformed from flat to full hemispherical shape like this. Interesting point is that, in the electrical subsystem, each imaging pixel is interconnected with adjacent pixels by using serpentine shaped metal lines (not a straight line). So, during the deformation process, this metal lines are stretched out like a spring which enables very large deformation of this system without any electrical and mechanical failure. (So, the basic structure is here, but, for the proper operation, we need to consider some important factors in terms of optics, electronics as well as mechanics.)
- #17 (For the proper operation, we need to carefully design the geometries of optical system.) In optical design, there are two important parameters. First one is inter-ommatidial angle that can be defined by the angle between adjecent ommatidia. The other one is acceptance angle of each ommatidium, which corresponds to the focal length of microlens and the diameter of photodetector. The basic rule is that the acceptance angle should be smaller than the inter-ommatidial angle to prevent the overlapping of the visual field of each ommatidium. If not, the captured image will be pretty much blurred. So, to satisfy this condition,we adjusted several parameters, such as initial length between each ommatidia, radius of curvature of this hemisphere. Focal length of this microlens and lateral dimension of photodetector is also important to determine the acceptance angle. (Refractive index of this polymeric material is also important.)
- #18 So, based on this rule, we made a polymeric microlens arrays. This is the picture of the aluminum mold and the resulting PDMS microlens arrays. this aluminum mold with concave-shaped structures were made simply by micromachining. In our current version, we aimed 160 degree field of view, in this case, inter-ommatidial angle is 11 degree, and the acceptance angle is around 10 degree, which satisfy the condition that I mentioned in a previous slide. And the corresponding design parameters are listed here.
- #19 Now we should prepare the electrical subsystem. This figure shows an exploded schematic illustration of a single unit cell, which consists of thin doped silicon membrane, metal interconnection lines and polymeric layers. Each cell involves single photodiode as well as a blocking diode for preventing the electrical crosstalk from the adjacent pixels. And you can see here, each pixel is interconnected with other devices both in row and column with serpentine shaped metal lines encapsulated with polymeric materials. Because we used thin silicon membrane of a silicon on insulator wafer, the total thickness of this electrical subsystem is only few micrometers, which allows very large deformation. I am not gonna go into details on fabrication, but if you have question, please ask me after this presentation.
- #20 Alright, now we have an optical and electrical subsystem. So, by using a transfer printing method with a precise alignment process, this photodetector array can be transferred to the backside of this microlens arrays. This is the integrated form of these two systems in a flat state. This is transparent while this color is black and brown, because it contains only very thin silicon membrane. (so there are no light absorptive medium at the background). You can see here, each imaging pixel is positioned at the bottom center of the corresponding microlenses.
- #21 For the hemispherical deformation, the integrated system is sealed in a customized fluidic chamber that consists of inlet/outlet like this. So, if we fill the water inside the chamber and close the input port and extract the water from this output port, then due to the pressure change, this flat membrane deformed to hemispherical form like this. After that we can hold this shape by using some hemispherical supporting rod made of pdms (with some adhesive). This picture shows compound eye camera that we made. It involves 180 ommatidia with full hemispherical geometry which enable an extremely wide field of view. And as you can see here, some of the metal lines are largely stretched.
- #22 For the imaging test, we need to consider several other things. First of all, in a real insect’s eye, they have black pigments between each ommatidia to prevent the stray light from the adjacent ommatidia. (In a biological term, it is called screening pigment) So, we prepared similar stuffs, that is black matrix with perforated hole arrays and we stretched this sheet and wrapped on our compound eye camera. And also, we used black supporting rod instead of clear one to prevent the back scattered light (at these areas). And we used thin film contact pads to connect our camera to a printed circuit board. And, finally, this pcb is connected to a control box with a ribbon cable to record a data set for image construction.
- #23 Alright so, I want to briefly introduce the operating principle of our camera. This is central portion of our camera and this is an object with ‘+’ shaped line art pattern. As you know, each microlens produces a small image of the object at specific positions, but only if a portion of these image overlaps with the active area of photodiode, then it generates photocurrent like this. Because the current version of our camera has only limited number of ommatidia, this camera produce a sampled image like that. In order to improve the resolution, we used a scanning method with 10 x 10 moving steps, in x and y direction, as a result, we could obtain this kind of image. (Actually, all of these are simulation results and I will show you the experimental results later.)
- #24 Here is the measurement setup. This is our camera which is mounted on a rotation stage for scanning process. The light is illuminated from here with some specific mask pattern and it is projected to our camera.
- #25 These are the representative output images captured by our compound eye camera. These two are experimental results and these are modeling results. And these are original line art patterns that we used as a mask. As I mentioned before, In order to improve the resolution, I used a scanning method with 10 x 10 moving steps, in x and y direction, as a result, We could obtain this kind of image. And, the resulting images are rendered on a hemispherical plane and as you can see here, the shape and size of these images are quite well matched with the modeling results. Actually, this modeling is conducted by ray tracing method with some of basic parameters that we calculated in the previous slide. (But, still there are some loss of resolution at several parts, which is caused by the parasitic scatting within the camera.)
- #26 One of the powerful mode of compound eye is the wide field of view. These images show an example of this characteristics. When you put the object at center, left and right with a 50 degree rotation angle, and take a picture, then you can see that all three resulting images show comparable clarity, without anomalous aberrations. And this laser spot illumination also shows the uniformity in sizes, shapes, and positions of these spots over the entire viewing angles. (from zero to 80 degrees)
- #27 Another important feature is infinite depth-of-field (due to the optic geometry), which means that everything is in focus independent the distance away from the camera. For this experiment, We prepared two object with different patterns (triangle and circle) at different positions like this. And, if this object is fixed and this object moves away from the camera, then as expected, these circular pattern size decreases whereas triangle has same size. But, interesting point is that this image is still in focus. So, again, we can say that this type of camera can simultaneously take the pictures of multiple objects at widely different angular positions and even different distances, without any tuning of optic systems.
- #29 Our current camera is inspired by this apposition compound eyes which can be found in daylight insects. On the other hand, in some night active insects, the structures are little bit different with this and each photorecepter receives light from so many microlenses. This type of compound eye is more sensitive to light, while it has lower image resolution. This is really interesting point. And, another important point is that it has nanostructures on the surface of microlens arrays.
- #36 And, second challenge is that how we can apply these nanostructure on various optical devices. And what is the design rule for the specific applications. Based on the grating equation again, the m-th order diffraction angle is strongly related to the refractive index of incident medium. This is very important factor to determine the design rules. For example, in general, we know that smaller period is good for broadband antireflection. But, in photovoltaic device or photodetector, it is not. In case of solar cell or other light absorbing medium, by controlling the period, we can arbitarilly induce the first order transmitted diffractions while minimizing the surface reflection. In this case, this higher order diffraction will increase the light path length thereby will increase the absorption efficiency. This is quite different with the conventional grating structure, because they has high surface reflection. This kind of interesting effects can also be found in LED and transparent glasses. In case of LEDs, the grating period should be much smaller than the optical wavelength, because the light is coming out from inside to outside and the refractive index of these materials are very high. In case of transparent glasses, the light is coming from outside, but it experiences multiple internal reflection, these structure should has smaller period than optical wavelength.
- #37 So based on the theoretical modeling and expectation, I fabricated solar cells, LEDs, and highly transparent glasses, with well designed nanostructures. And all of these experimental results showed remarkable enhancements in optical efficiency. I don’t wanna go into details, but I wanna mentioned that this interesting nanostructure can be applied to any type of optoelectronic devices and optic components.