Sensory                                                                            Information                            ...
2012 AFOSR SPRING REVIEW          PORTFOLIO OVERVIEW         Program manager:                                   Willard La...
Program Trends and Strategy:                                TOPIC AREA OVERVIEW                                           ...
Grantees’ Self-Organized                 Research Workshops (2011)Neural Oscillations and Speech Perception.     24 May. H...
Sensory Information Systems Program                          Primary Coordination         NSF                             ...
Sensory Information Systems                   Program -- People Dr. Jennifer Talley,                       2012           ...
RECENT HIGHLIGHTS    OPSIN MAP                                                                                    DR. KEY ...
Recent TransitionsTO: Navy:    Bio-inspired method to classify acoustic sources, e.g., vehicles,humans, marine animals etc...
Autonomous Steering:                    Transition to Army MAST Quadrotor implements autonomous 3D navigation using wide f...
Bio-Emulation Progress        From 6.1 at U. Queensland, U. Bristol, & UMBC          1. Discovery:                        ...
Nano-Rod Deposition Recipe Emulates            Biological Achromatic Phase Retarder                            Y          ...
Auditory Modeling for Acoustic              Analysis – Extramural Grants•   L. Atlas (U. Washington):    New approach to m...
Pointillistic Speech Coding                                   G. Kidd, Boston Univ., 2011                                 ...
Transaural Synthesis: A New Technical                  Advance in Auditory Measurement                                    ...
Acoustic Source Separation:                 Recovering Multiple Speech Signals from a Single Channel                      ...
How do Human Listeners Sort               Overlapping Speech Information ?Scientific Question:                            ...
Coherent Modulation Analysis:                    Dr. Les Atlas, University of Washington                                  ...
Sensori-motor Control of Natural                   Flight and Navigation Fundamental Question:What underlying principles d...
Sensori-motor Control of Natural                 Flight and Navigation                    = AFOSR BIO-NAVIGATION FUNDING I...
International and 6.2                  CoordinationU.K.                   AFRL-DSTL Working Group                         ...
Span of Natural Flier Research:                    A Few Program Participants  Vertebrates                                ...
Antennae in Flight Control                Dr. Sanjay Sane, Tata Institute, Bangalore, India                               ...
Wings are Sensory Receptors                          - - - A New Research Effort                            CORIOLIS  MECH...
The Mode Sensing Hypothesis                       for Biological Flight ControlProposed by 3 AFOSR scientists:            ...
Each Visual System Cell is Tuned to a                       Preferred Axis of Rotation                                    ...
Other Motion Sensor Tuning Differs from                       Compound Eye Tuning  Plots derived from single-unit recordin...
Wing Sensorimotor Activation                     - - A New Program Initiative           Natural Fliers’ Wing Joints      (...
Discovering Bat Wing Musculature                Dynamics during Flight                                                    ...
SUMMARY:           Transformational Impacts & OpportunitiesHearing protection:• Massive improvements in high-noise attenua...
Questions?       Thank you for your attention.Willard Larkin, Program Manager, AFOSR/NL                 703-696-7793      ...
Questions?       Thank you for your attention.Willard Larkin, Program Manager, AFOSR/NL                 703-696-7793      ...
DISTRIBUTION A: Approved for public release; distribution is unlimited.                                                   ...
Robber Fly                                                         No Ocelli                                              ...
Dynamics of Bat Wing Musculature           S. Swartz, T. Roberts, Brown Univ., 2011                              Large bat...
Plagiopatagiales Muscles                                                                       Muscles active during upstr...
4000                          3000         Frequency (Hz)                          2000                          1000     ...
Upcoming SlideShare
Loading in …5
×

Larkin - Sensory Information Systems - Spring Review 2012

991 views

Published on

Dr. Willard Larkin presents an overview of his program - Sensory Information Systems - at the AFOSR 2012 Spring Review.

Published in: Technology, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
991
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Larkin - Sensory Information Systems - Spring Review 2012

  1. 1. Sensory Information Systems Program 6 March 2012 Willard Larkin Program Manager AFOSR/RSL Integrity  Service  Excellence Air Force Research Laboratory15 February 2012 DISTRIBUTION A: Approved for public release; distribution is unlimited. 1
  2. 2. 2012 AFOSR SPRING REVIEW PORTFOLIO OVERVIEW Program manager: Willard Larkin BRIEF DESCRIPTION OF PORTFOLIO:• Auditory modeling for acoustic analysis• Biological polarization optics & vision• Sensori-motor control of bio- flight & navigation SUB-AREAS IN PORTFOLIO: Sensory Information Systems (3003/L) DISTRIBUTION A: Approved for public release; distribution is unlimited. 2
  3. 3. Program Trends and Strategy: TOPIC AREA OVERVIEW 40%Advanced Auditory Modeling:Scientific Question: How does the auditory brain parse acousticlandscapes, bind sensory inputs, adapt its filters, hear through noise anddistortion? Could autonomous listening devices emulate neurology to match orexceed human auditory analysis, e.g., to detect and identify speech targets innoise and reverberation? 12%Polarization Vision & Optics:Scientific Question: How do natural photoreceptors detect and how doanimal brains interpret polarization information? How is it used for nocturnalnavigation or recognition of obscured targets? Can these unique bio-opticalsystems be emulated? AFOSR 47% BioNavigation ResearchSensorimotor Control of Flight & Navigation: InitiativeScientific Question: How does neural control make natural, low-ReynoldsNo. flight autonomous, efficient, and robust? Discover principles ofmultisensory fusion, distributed sensors and actuators. Develop control lawsfor emulation in MAVs. Strategy: Forge useful connections between math and biology DISTRIBUTION A: Approved for public release; distribution is unlimited. 3
  4. 4. Grantees’ Self-Organized Research Workshops (2011)Neural Oscillations and Speech Perception. 24 May. Hosted by NYU.Science & Applications for Bio-Inspired Navigation and Control. 23-25 May. Hosted by AFRL/RW.Bio-Optics and Photoreception. 27-30 June. Hosted by U. Bristol. Convened Funded by the Rank Prize Foundation by grantees;Encoding for Auditory Representations. No AFOSR 22-23 August. Hosted by U. Washington. workshop fundingInformational Masking & Binaural Hearing. 17-19 Nov. Hosted by Boston U.Brain Rhythms and Cortical Computation. 18 Nov. Hosted by NYU. DISTRIBUTION A: Approved for public release; distribution is unlimited. 4
  5. 5. Sensory Information Systems Program Primary Coordination NSF ONR B. Tuller T. McKenna M.Steinberg ARO AFRL/RI AFRL/RH AFRL/MN WRAMCE. Schmeisser A. Noga D. Luginbuhl R. Wehling D. Brungart N. Iyer DISTRIBUTION A: Approved for public release; distribution is unlimited. 5
  6. 6. Sensory Information Systems Program -- People Dr. Jennifer Talley, 2012 Dr. Eric Thompson, Biology Judith A. Resnik Medal Acoustics Dr. Pramod Varshney,Dr. Talley (Ph.D. 2010) was Syracuse U. Dr. Thompson (Ph.D. 2009) recruited from AFOSR- was recruited from sponsored AFOSR-sponsoredproject at Case Western U. projects at Boston Univ. to to insect flight lab at psychoacoustics lab at AFRL/Eglin AFRL/RH DISTRIBUTION A: Approved for public release; distribution is unlimited. 6
  7. 7. RECENT HIGHLIGHTS OPSIN MAP DR. KEY DISMUKES AFOSR Polarization Biology Sub-program Keynote Speaker for New Genomic featured in AFRL – Wright State Analysis reveals J. Royal Society Internat. Symposium maximum-likelihood Special Issue on phylogenetic model Aviation Psychology forPhotoreceptor Opsins DISTRIBUTION A: Approved for public release; distribution is unlimited. 7
  8. 8. Recent TransitionsTO: Navy: Bio-inspired method to classify acoustic sources, e.g., vehicles,humans, marine animals etc., using cortical auditory model. Dr. Sam Pascarelle,Advanced Acoustic Concepts, Inc.TO: AFRL-- Eglin: Measurements and data on biological wide-field-of-viewoptical systems to enable 6.2 and 6.3 efforts in vision-based guidance andnavigation. D. Stavenga, U. Groningen, N. Strausfeld, U. Arizona, M. Wehling, et al.TO: Bloedel Hearing Institute: New, patented method to improveauditory coding in cochlear implants. Developed by Dr. Les Atlas, U. Wash. Dr. JayRebenstein will develop commercial applications.TO: AFIT: Techniques in electrophysiology and neuroanatomy formechanical engineering projects to emulate flapping wing flight. Dr. MarkWillis, Case Western U., Dr. Anthony Palazotto, AFITTO: DARPA: System for adaptive, autonomous control of roboticmovement, based upon hierarchical neural model of biological control.Roger Quinn &. Roy Ritzmann, Case West. U.; G. Pratt, DARPA. DISTRIBUTION A: Approved for public release; distribution is unlimited. 8
  9. 9. Autonomous Steering: Transition to Army MAST Quadrotor implements autonomous 3D navigation using wide field-of-view optic flow Inspiration: • Insect Compound Eye Research Eglin AFB 6.1 (M. Wehling, et al.) • Insect Visual Target Detection Modeling Tanner Research (P. Shoemaker.) U. Adelaide (D. O’Carroll) • Insect Flight Behavior Modeling U. MD. (J. S. Humbert)Microautonomous Systems and Technology http://www.avl.umd.edu/ DISTRIBUTION A: Approved for public release; distribution is unlimited. 9
  10. 10. Bio-Emulation Progress From 6.1 at U. Queensland, U. Bristol, & UMBC 1. Discovery: 2. Optical Modeling:Polarization photoreceptor imparts -invariant 90o phase delay “Nature’s Perfect Waveplate” Key property: Intrinsic birefringence of microvilli elements, in balance with form birefringence, yields wavelength- invariant phase delay. Honeycomb structure of photoreceptor’s optical retarder membrane 3. Emulation DISTRIBUTION A: Approved for public release; distribution is unlimited. 10
  11. 11. Nano-Rod Deposition Recipe Emulates Biological Achromatic Phase Retarder Y Deposition Angles Alternate in Ta2O5 Nano-rod Layers X 174 nm Upright nanorods Nano-rod array (red layers) exhibits birefringence S0 ,S3 S1 , S2 Tilted nanorods (yellow layers) All 4 Stokes parameters remain nearly constant in a multi-layer structureYi-Jun Jen, Nature Comm. 21 June 2011 DISTRIBUTION A: Approved for public release; distribution is unlimited. 11
  12. 12. Auditory Modeling for Acoustic Analysis – Extramural Grants• L. Atlas (U. Washington): New approach to modulation representation and filtering.• E. Bleszynski (Monopole Research): Math model of bone- & tissue-conducted sound Mathematical and Psychoacoustic• M. Elhilali (Johns Hopkins U.): Research to Cortical model for acoustic scene segregation Understand & Emulate• O. Ghitza (Boston U.): the Human Listener’s Theory of speech parsing via brain rhythms Auditory Capabilities• W. Hartmann (Mich. State U.): and to Enhance these Sound localization Capabilities where• G. Kidd (Boston U.): Possible. Informational masking & speech segregation• R. Kumaresan & P. Cariani (U. Rhode Is. & Harvard): 4 NEW TECHNICAL Spectro-temporal codes for auditory signal representation ADVANCES• D. Wang (Ohio State U.): Computational auditory scene analysis DISTRIBUTION A: Approved for public release; distribution is unlimited. 12
  13. 13. Pointillistic Speech Coding G. Kidd, Boston Univ., 2011 A sparse, pointillistic rendition ofNew Technique Isolates the word “SHOES,” overlaid on theInformational & Energetic Masking conventional spectrogramBackground:• What listeners can hear in real,multisource acoustic environments ismore constrained by informationalmasking, than by direct, energeticinterference of one sound by another.• AFOSR seeks techniques to study& suppress informational masking.• Sparse pointillistic coding leavesspeech remarkably intelligible,eliminates energy overlaps, isolates Each time/frequency slice of speech has beeneffects of informational “cross-talk.” replaced with a matching cosine fragment.• Briefed to AFRL/RI in Dec. 2011. Red color scales the intensity. DISTRIBUTION A: Approved for public release; distribution is unlimited. 13
  14. 14. Transaural Synthesis: A New Technical Advance in Auditory Measurement W. Hartmann, MSU. NOISE SPECTRUM MEASUREMENTSScientific Challenge: IN EAR CANAL How does binaural hearing disclose the locus of sound CONVENTIONAL METHOD in real 3D environments?Transaural Synthesis Method:• Eliminates inter-aural crosstalk, compareslocalization judgments with precisely knownacoustic parameters in the ear canals.• Tests auditory theory by manipulatingInteraural phase and amplitude, comparingreal sounds with “virtual” sounds.• Tests in real rooms & environments, SYNTHESIS METHOD with varied reverberation – no headphones.• Self-compensates for probe-tube positions,enables precise reproducability. DISTRIBUTION A: Approved for public release; distribution is unlimited. 14
  15. 15. Acoustic Source Separation: Recovering Multiple Speech Signals from a Single Channel Why Phase Matters: Research Challenge: A B Devise a factorization method to: • Recover both magnitude and phase Original Sources for each speech component superimposed in a single channel Mixed Sources • Avoid trial-and-error basis selection. Separation via New Complex Factorization Phase Recovered SPEECH RECOVERY INDEX 40 % Conventional Non-Negative 35 Factorization Phase Ignored 30 25 20 Brian King New Complex Matrix Factorization method U. Washington 15 Ph.D. Candidate 10 solves superposition and phase problems, & ORIG. NMF CMF eliminates ad hoc basis selection,RH Summer Student 5 & Improves speech recognition scores 0 King & Atlas, IEEE Trans on Audio, Speech andA: Approved for Processing (Nov. is unlimited. DISTRIBUTION Language public release; distribution 2011) 15
  16. 16. How do Human Listeners Sort Overlapping Speech Information ?Scientific Question: WORD IDENTIFICATION PERFORMANCEWhen each ear receives adistinct message, can theauditory brain extract keyinformation from both?Surprising Result:Manipulation of word predictability indichotic listening strongly supports amodel of attention-switching, rejectshypothesis of divided attention. (Level of noise masking a keyword) N. Iyer, B. Simpson, Principal Investigators, 711 th HPW/AFRL/RHCB Battlespace Acoustics DISTRIBUTION A: Approved for public release; distribution is unlimited. 16
  17. 17. Coherent Modulation Analysis: Dr. Les Atlas, University of Washington Maximum of Estimated ModulatorMathematical challenge: 800Estimate and separate the linear 700time-invariant and frequency-invariantcomponents of an acoustic source. 600 500 0.80mm 0.58mm 400Approach: 300 0.94mmTheory of Complex Modulation Filtering,devised by Prof. Les Atlas (originally for 200 5 10 15 20 25analysis of speech signals.) Hole Number→Increasing WearUnexpected Application:A simple, one-microphone set-up detects hole qualitywhile drilling in airframe composite material. DISTRIBUTION A: Approved for public release; distribution is unlimited. 17
  18. 18. Sensori-motor Control of Natural Flight and Navigation Fundamental Question:What underlying principles drivebiology’s design of actuation andsensing architectures? Motivating Observations from Insect Research :• Sensors are “noisy,” redundant, distributed in non-orthogonal coordinates.• Inputs fuse across modalities prior to activating flight muscles.• No conventional distinctions between estimate/control or inner/outer loop• Sensors differ radically in bandwidth & temporal response, e.g., vision lags Insect Lab, Eglin AFB mechanoreception. Dr. Jennifer Talley DISTRIBUTION A: Approved for public release; distribution is unlimited. 18
  19. 19. Sensori-motor Control of Natural Flight and Navigation = AFOSR BIO-NAVIGATION FUNDING INITIATIVET. Daniel ( U. Washington): R. Olberg ( Union College): Wing mechanosensor functions. Dragonfly flight to target capture S. Sane ( Tata Institute):J. Evers (AFRL/RW): Insect multisensory integration Natural 3D flight dynamics P. Shoemaker (Tanner Res.):M. Frye ( UCLA): Visual detection of small targets Higher-order motion detection S. Sterbing ( U. MD):S. Humbert ( U. Maryland): Wing sensors in bat flight control Modeling sensorimotor control G. Taylor (Oxford):H. Krapp ( Imp. College London): Raptor pursuit strategies in 3D Neural basis of visual steering E. Warrant (Lund U.):P. Krishnaprasad ( U. MD): Nocturnal navigation Modeling formation flight control M. Wehling ( AFRL/RW):S. Reppert ( U. Mass): Neural analysis of optic flow. Clock-compensated navigation M. Willis (Case Western):R. Ritzmann (Case Western): Visual / olfactory target tracking Adaptive locomotion control DISTRIBUTION A: Approved for public release; distribution is unlimited. 19
  20. 20. International and 6.2 CoordinationU.K. AFRL-DSTL Working Group U.S. Biologically-Motivated Micro-Air-Vehicles “STATE OF THE ART REVIEW”G. Taylor Georgia Tech M. Willis 15-18 June, 2010H. Krapp S. Humbert Organizers: M. Wehling, AFRL. P. Biggins, Dstl 30 Participants from UK, US, Industry, Academia, & Gov. Presentations: https://livelink.ebs.afrl.af.mil/livelink/llisapi.dll?func=ll&objId=24091294 &objAction=browse&viewType=1 T. DanielJ. Niven DISTRIBUTION A: Approved for public release; distribution is unlimited. 20
  21. 21. Span of Natural Flier Research: A Few Program Participants Vertebrates InvertebratesHawks & Falcons Megalopta genalis (Nocturnal Bee) Bombus terrestrisEcholocating Bats Dragonfly Hawkmoth DISTRIBUTION A: Approved for public release; distribution is unlimited. 21
  22. 22. Antennae in Flight Control Dr. Sanjay Sane, Tata Institute, Bangalore, India 1 Precise control of antennae position is critical for aerial flight maneuvers 2 Control system is closed-loop, with sensory latency of 15 msec. 3 Antennae integrate mechanosensoryHawkmoth, Daphnis nerii & photoreceptor inputs 4 Antennae position in flight is governed by Bohm’s bristles only at pedicel.AOARD grant via T. Erstfeld DISTRIBUTION A: Approved for public release; distribution is unlimited. 22
  23. 23. Wings are Sensory Receptors - - - A New Research Effort CORIOLIS MECHANICAL REGIONS INPUT 4-NEURON COHERENCE INPUT POWER SPECTRUMNEURONRESPONSEWing Campaniform Neurons Respond to Mechanical Forces Hawkmoth, Manduca sexta Hypothesis: Wings not only drive flight, but also detect inertial moments. - - - strain receptors modulate wing shape and position. T. Daniel (biology) and K. Morgensen (Aeronautics) U. Washington. DISTRIBUTION A: Approved for public release; distribution is unlimited. 23
  24. 24. The Mode Sensing Hypothesis for Biological Flight ControlProposed by 3 AFOSR scientists: H. Krapp G. TaylorTheory: S. Humbert IMPERIALBiological sensors and actuators compose MARYLAND COLLEGE LONDON OXFORDa suite of “matched filters,” tuned to salientpatterns (“modes”) of self-motion – H.G. Krapp, G. K. Taylor, and J.Sorganized to combine multiple inputs into Humbert (in press) “The Mode-actuation signals encoded in modal coordinates. Sensing Hypothesis: Matching Sensors, Actuators and Flight Dynamics.”Challenge from AFOSR:• Test the theory in neurobiology and flight behavior• Devise mathematical emulation suitable for MAVs DISTRIBUTION A: Approved for public release; distribution is unlimited. 24
  25. 25. Each Visual System Cell is Tuned to a Preferred Axis of Rotation Calliphora vicina Lobular Plate Tangential Cells (Compound Eye System) 10 Vertical System Cells 3 HS cells10 VScells • Sensory Integration begins by selectively merging these inputs. • The resulting “matched filters” for self-motion are robust and innate.H. Krapp, Imperial College, London DISTRIBUTION A: Approved for public release; distribution is unlimited. 25
  26. 26. Other Motion Sensor Tuning Differs from Compound Eye Tuning Plots derived from single-unit recordings during controlled rotational stimulation Theory: Neck motor neurons (red) • Short-latency mechanosensors first LPTCs (blue) detect body rotation, then feed forward to induce compensatory head roll via neck motor system … Ocellar system (polarization vision) • Long-latency visual system acts on residual optic flow feedback from incomplete mechanical compensation. • Behavioral data fit preliminary model for multimodal sensory integration.Data from U. Cambridge, (M. Parsons)& Imperial College, London (H. Krapp) DISTRIBUTION A: Approved for public release; distribution is unlimited. 26
  27. 27. Wing Sensorimotor Activation - - A New Program Initiative Natural Fliers’ Wing Joints ( ) with potential to be controlled during flight Insect Bird Bat Bat wing has Intrinsic, non-joint muscles Plagiopatagiales MusclesResearch Questions:• Discover joint coordination timing• Determine functional redundancy• Do joint muscles control force or position?• Discover what the intrinsic muscles do. DISTRIBUTION A: Approved for public release; distribution is unlimited. 27
  28. 28. Discovering Bat Wing Musculature Dynamics during Flight Intrinsic Muscles Active only During Wing Upstroke:Large bat (1.2kg) in low speed flight has precise control via wing skeletal muscles and wing intramembranous (intrinsic) musclesResearchTechniques:• Electromyography during flight• Thermal videography for metabolic load) Time (s)• Selective, reversible muscle paralysis• 3D X-ray mapping of skeletal motion … A Discovery from AFOSR MURI DISTRIBUTION A: Approved for public release; distribution is unlimited. 28
  29. 29. SUMMARY: Transformational Impacts & OpportunitiesHearing protection:• Massive improvements in high-noise attenuation.Advanced auditory modeling:• Mathematics for coherent modulation analysis• Neural-Inspired analyses to parse acoustic scenesOptical processing:• Polarization vision and signaling adapted from biology• Achromatic 1/4 wave optical retarders• Emulating compound eye in new optical devicesAutonomous flight control:• Adaptive airfoils based upon bio-sensory mechanisms• Steering based upon neural autonomous systems• Discover sensorimotor basis of formation flight DISTRIBUTION A: Approved for public release; distribution is unlimited. 29
  30. 30. Questions? Thank you for your attention.Willard Larkin, Program Manager, AFOSR/NL 703-696-7793 Willard.Larkin@afosr.af.mil DISTRIBUTION A: Approved for public release; distribution is unlimited. 30
  31. 31. Questions? Thank you for your attention.Willard Larkin, Program Manager, AFOSR/NL 703-696-7793 Willard.Larkin@afosr.af.mil DISTRIBUTION A: Approved for public release; distribution is unlimited. 31
  32. 32. DISTRIBUTION A: Approved for public release; distribution is unlimited. 32
  33. 33. Robber Fly No Ocelli Halteres No VS cells 5-18 HS cellsNew Lab for Insect VisionSpectral & PolarizationElectroretinographyAFRL/RWG PI: Martin Wehling DISTRIBUTION A: Approved for public release; distribution is unlimited. 33
  34. 34. Dynamics of Bat Wing Musculature S. Swartz, T. Roberts, Brown Univ., 2011 Large bat (1.2kg) in low speed flight maintains precise control via wing skeletal muscles and wing intramembranous muscles Techniques: Electromyography during flight Thermal videography (to measure metabolic load) Selective, reversible muscle paralysis 3D X-ray mapping of skeletal motion DISTRIBUTION A: Approved for public release; distribution is unlimited. 34
  35. 35. Plagiopatagiales Muscles Muscles active during upstroke: 1.5 gm EMG Telemetryfor in-flight muscle activity Time (s) DISTRIBUTION A: Approved for public release; distribution is unlimited. 35
  36. 36. 4000 3000 Frequency (Hz) 2000 1000 0 0.2 0.25 0.3 0.2 0.25 0.3 Time (s) Time (s)DISTRIBUTION A: Approved for public release; distribution is unlimited. 36

×