PROJECT: Neural Recording LaboratoryPRODUCT: Research LaboratoryROLE: Architect/Doctoral ResearcherOVERVIEWIn January 2006 I joined the laboratory of Dr. StephenHelms Tillery, who only months before had accepted atenure track position in the department ofBioengineering at Arizona State University. Our goalwas to build a laboratory to conduct world-classresearch into the neural basis of sensor and motorfunction in the brain.The opportunity was rare and exciting, but we truly started from scratch. My first glimpse of thelab was of completely empty room. The Boss told me what he wanted and said, “Build it.”MY ROLEThis was my most complex and comprehensive accomplishment to date. I was responsible forprocurement, installation, configuration, software development, and hardware development foreverything in the lab. I was given broad budgetary discretion for purchases. If something wasrequired and it didn’t exist, I designed it in Solidworks then fabricated it with machine tools orrapid prototyping machines. If software was required, I developed it in C/C++, Python, Matlab,Simulink, etc... Over the next 2 years the following core pieces of the laboratory setup weredeveloped: 1. 6-AXIS INDUSTRIAL ROBOT: A robot arm was required to present objects to research subjects. Developed custom real-time control routines in C++ using manufacturer SDK. Integrated pneumatic end effector tool changer system with 6-DoF Force/Torque sensor. Developed custom front end GUI using LabView. 2. VIRTUAL REALITY SIMULATION: Our experiments required subjects to execute tasks in a virtual reality environment. This was developed in the Python programming language in using Vizard development software. VR control was integrated into overall system using LabView. 3. 3-D MOTION CAPTURE: We tracked and analyzed the detailed kinematics of our subjects’ hand during the experimental task using an active marker (LED) system from Phasespace. Developed custom marker arrays and core software routines to acquire maker data in real time to drive animations in the VR simulation. 4. NEUROPHYSIOLOGICAL RECORDING SETUP: We recorded the activity of single neurons in the sensory cortex of the brain during our experiments. The Plexon system was integrated in the overall system using LabView software
5. SYSTEM CONTROL SOFTWARE: Developed a hybrid PC/Real-Time application in LabView to unify all components of the system, including sensory I/O, robot commands and VR control. The end product was a unified control GUI from which all aspects of the experiment could be monitored and controlled. 6. MONKEYS!!: Truly the most unpredictable and challenging aspect of the this work; an education within an education. Learned to handle, train and work with two male Rhesus macaque monkeys to obtain neurophysiological data.OUTCOMEAfter nearly four and a half years of continuous work, I completed my experimental work anddoctoral dissertation, graduating in May of 2010. My legacy is a neurophysiological recordinglaboratory that is uniquely capable, flexible and reconfigurable for many kinds of neuralexperimentation protocols.Accomplishing the PhD made me a better engineer and taught me how to carry out research.What I now seek is the opportunity to unify the myriad skills acquired in my work and educationso far into a single goal requiring multidisciplinary skills.
EXPERIMENTAL SETUP 1. Monkeys were trained in a novel Reach-to-Grasp task in which all visual cues were presented in a Virtual Reality simulation. 2. Grasp object of different sizes were presented in the workspace behind the VR presentation and hand position and digit kinematics were tracked using an active marker motion capture system. 3. The firing activity of single neurons in sensory cortex was recorded while the monkeys completed either a physical task (object present in the workspace) or a randomly inserted virtual task (object presented just out of reach) 4. This approach permitted manipulation of the actual and expected sensory outcome of the task.
SMORG ROBOT AND ASSOCIATED HARDWARE. A b3 B b2 b1 CA. THE 6-AXIS INDUSTRIAL ROBOT. Mounted on a custom platform and controlled using customsoftware. Dedicated signal and air channels routed through the robot enabled feedback from a6-DOF F/T sensor, object touch sensors and control of a pneumatic tool changer.B. THE ROBOT END EFFECTOR. The F/T sensor (b2) was mounted directly to the robot endeffector (b1). The master plate of the tool changer (b3) was mounted to the F/T sensor using acustom interface plate. Air lines originating from ports on the robot controlled the lockingmechanism of the master plate.C. GRASP OBJECT ASSEMBLY. The object was mounted to a six-inch standoff post that mountedto a tool plate. Touch sensors were mounted flush with the object surface and wires wererouted to the object interior for protection. Power and signal lines were routed through a pass-through connector (not visible), through the robot interior to an external connector on the robotbase.
CAPTURING HAND KINEMATICS. A B C DOur experiments required knowledge of detailed hand kinematics – of monkeys! This figurehighlights just some of the hardware and electronics development I conducted to accomplishthis task.A. COMMERCIALLY AVAILABLE DATA GLOVES feature numerous integrated bend sensors tocapture the posture of the digits and palm but were prohibitively expensive and difficult tocustomize to the monkey hand.B. EARLY PROTOTYPE OF THE CUSTOM MONKEY GLOVE. Bend sensors and electronics wereremoved from a gaming glove and reconfigured to the monkey hand.C. A WIRELESS VERSION OF THE MONKEY GLOVE. This more advanced version featured 5 bendsensors, 2-axis roll/pitch sensing, wireless bluetooth transmisstion and a rechargeable battery.Electronics were encased in epoxy for protection.D. AN ALTERNATIVE STRATEGY FOR PASSIVE MOTION CAPTURE. Cubemarkers with finger attachment clips were developed to utilize largermarkers for a novel camera sensing technique. This approachcaptured only crude measures of hand postureACTIVE MARKER LEDS. We eventually settled on an active markerLED system from Phasespace, show at right.
GENERAL SMORG LAB PICTURES TESTING ROOM Motion capture cameras surround the subject seating area. A 3D monitor placed overhead reflects the VR environment into a mirror directly ahead. The robot presented grasp objects in the workspaceHARDWARE AND ELECTRONICS These experimentsrequired the integration of much hardware andelectronicsCUSTOM HARDWARE These experiments alsorequired the development of custom hardware,such as the grasp objects seen here.
PROJECT: Experimental Device DevelopmentPRODUCT: Actuated Ankle Foot Orthosis (AAFO)ROLE: Project ManagerOVERVIEWAdvensys, LLC is an early stage startup company developingadaptive neuromorphic systems for “advancing humanmobility.” In 2004 the company was awarded a competitivePhase I contract by the US Army to develop aneuromorphically controlled lower leg orthosis capable ofproviding ambulatory support for soldiers with injuriessustained in combat. The company was tasked withdeveloping a prototype electronic control system based onneural pattern networks, management of AAFO hardwaredevelopment and experimental assessment of the integratedsystem. The duration of Phase I prototype development,integration and testing was just 6 months.MY ROLEI was hired by the President and co-founder of the companyas the Program Manager. My duties included severalsignificant subtasks of the overall project: 7. NUMERICAL MODELING OF NEURAL PATTERN GENERATOR NETWORK: a biologically inspired oscillating signal generator patterned after spinal locomotor circuitry of the lamprey. The activity of this network could be entrained by an external periodic signal and would form the basis of our real-time control system. 8. PROTOTYPE ELECTRONICS DEVELOPMENT: Following numerical simulation and characterization, the neural pattern generator was implemented in breadboard electronics using RC circuit models of neural membrane dynamics. 9. REAL-TIME CONTROL SYSTEM DEVELOPMENT: A real-time numerical controller was developed using Matlab, Simulink and Real-Time Workshop. The controller was driven by a hip angle sensor and drove the ankle “push-off” of the AAFO.OUTCOMEPhase I was completed on time and under budget. The device was demonstrated to seniorArmy program officials in a live demonstration and feedback was overwhelmingly positive.Advensys was invited to apply for Phase II funding, and was subsequently awardedapproximately $1.5 million over 2 years.
NUMERICAL MODELING OF NEURAL PATTERN GENERATOR NETWORK THE UNIT PATTERN GENERATOR (UPG) CONCEPT: A bi-laterally symmetric network whose connectivity is based on the known architecture of the lamprey spinal cord. The three spinal neural classes that form the kernel of the uPG are: E- excitatory, L-inhibitory, and C- crossed inhibitory. The E, L, and C neurons represent the core of the network and the M neurons are the “output” units of the network.SIMULATION RESULTS OF NEURALNETWORK ENTRAINMENT IN RESPONSETO INJECTED CURRENT INPUT. Bothsides of the network oscillate atdefault frequency before entrainment,and then assume the frequency ofthe injected current.
PROTOTYPE ELECTRONICS DEVELOPMENT PROTOTYPE UNIT PATTERN GENERATOR (UPG) ELECTRONICS: Unit Pattern Generator network showing neurons implemented in analog hardware.ELECTRONICS TESTING:uPG network testing
REAL-TIME CONTROL SYSTEM DEVELOPMENT !CONTROL SYSTEM DEVELOPMENT: Simulation and controller development were in Matlab,Simulink and Real-Time Workshop (RTW). Compiled, executable RTW code was ported to theHardware Computer via an ethernet link. The Hardware Computer executed the real-timecontrol system code, including sampling analog sensor input (analog-to-digital) from externalhardware and asserting all analog commands (digital-to-analog) to hardware through the PCIDAS1200 A/D, D/A card. Two Break-Out Boards (BOB) provided numbered conductorconnection points for system I/O. Motor commands, were sent from the BOB to the AmplifierBoard, which directly drove the motor.