The document describes a proposed patient positioning system for maskless head and neck radiotherapy using a soft robot. The system uses a Kinect camera for vision-based sensing of patient head position. A soft robot consisting of an inflatable air bladder and pneumatic valves would manipulate the patient's head to correct for any motion during treatment. Preliminary results show the system was able to control 1 degree of freedom of motion (flexion/extension) of a mannequin head using proportional valve control and Kinect vision feedback to a control system. Further work is needed to validate the system for actual use in radiotherapy treatment.

1. A PATIENT
POSITIONING
SYSTEM FOR
MASKLESS HEAD
AND NECK
RADIOTHERAPY
Electrical Engineering Department, UT Dallas, TX
4/23/2015
1
Presented by Olalekan Ogunmolu. April 23,
2015

2. Olalekan Ogunmolu: Bio Snippet
o Semester of PhD Enrolment: Fall 2014 GPA:
4.00/4.00
o Advisor: Nicholas Gans, Assistant Professor
o MSc (in Engineering) in Control Systems 2011 -
2012
The University of Sheffield, England, UK
MS Thesis: “Autonomous Navigation of a Rotorcraft Unmanned Aerial Vehicle Using
Machine Vision”, The University of Sheffield, South Yorkshire, England, Sept. 2012.
Advisor: Tony J. Dodd, Professor of Autonomous Systems Engineering, ACSE Dept.,
UoS.
o Research Thrusts
o Vision-based Control: Features tracking, Active Appearance Models, Neural
Networks
o Control Systems and Automation: Classical Control, Non-linear Systems,
Switched Systems
o Publication
o Olalekan Ogunmolu et. al, “A Real-Time Soft Robotic Patient Positioning System for
Maskless Head and Neck Cancer Radiotherapy”, IEEE Conference on Automation
and Systems Engineering [Submitted], Gothenburg, Sweden, August 2015Electrical Engineering Department, UT Dallas, TX 4/23/2015
2

3. Overview of Presentation
 A first-stage systematic examination of a novel method
at automating patient positioning systems during cancer
head and neck cancer radiotherapy (RT)
 Visual-servoing control using a radio-transparent soft
robot on the flexion/extension cranial motion of a patient
during mask-less radiotherapy
 Proof-of-concept validation of one degree of freedom
patient’s motion control with designed soft robot system
Electrical Engineering Department, UT Dallas, TX 4/23/2015
3

4. Background and Motivation
 Head and Neck (H&N) Cancers contribute nearly 3% to all cancer
developments in the United States [Jemal, A.]
 Very dangerous to benign tissues/organs nearby malign cancerous formations in
H&N region
 Treatment often involves intensity modulated radiotherapy (IMRT)
 Clinical studies have shown that small perturbations cause high
sensitivity to IMRT treatment dose [Xing., L]
 State-of-the-art robotic positioning treatments assume immobilized
patient motion on a 6D couch which is unrealistic
Electrical Engineering Department, UT Dallas, TX 4/23/2015
4

5. Recent Research Results
 Frameless and Maskless Cranial SRS, Cervino et al
 Anthropomorphic head phantoms employed in checking the
accuracy of a 3D surface imaging system (AlignRT System)
 Compared results from an infra-red optical tracking system with
the AlignRT software system
 For different couch angles, the difference between phantom
positions recorded by the two systems were within 1mm
displacement and 1° rotation
Electrical Engineering Department, UT Dallas, TX 4/23/2015
5

6. Recent Research Approaches
 Recent Research Thrusts – 6D Motion Frameless SRS
Approach, Wiersma et al
Electrical Engineering Department, UT Dallas, TX 4/23/2015
6

7. Typical Frame-based Radiotherapy
Electrical Engineering Department, UT Dallas, TX 4/23/2015
7

8. Research Overview
 Proof-of-concept study and experiment demonstrating a 1-DOF
flexion/extension control of patient cranial motion during H&N Cancer RT
 Testbed is a Mannequin head lying in a supine position on an inflatable air
bladder (IAB)
 Soft-robot consists of the IAM, two two-port SMC Pnematics Co.
proportional valves, and silicone tubes for conveying air from a pressurized
air canister
 A Kinect RGB-D camera is employed for head motion sensing and feedback
to a classical control network implemented on an NI myRIO hardware
 Work in partnership with the Drs. Xuejun Gu and Steve Jiang of the
Radiation Oncology Department of UT Southwestern, Dallas, TX, USA
Electrical Engineering Department, UT Dallas, TX 4/23/2015
8

9. Recent Research Results
 Average max. volunteer head motion in the head mold during the
20 minutes interval in any direction was 0.7mm (range 0.4 –
1.1mm)
 Patient motion due to couch motion was less than 0.2mm
 Drawbacks
 Minimal immobilization provided by head mold results in poor
positioning
 Relies heavily on patient cooperation to achieve immobilization
 Inter-fractional motions often ignored during pre-treatments
Electrical Engineering Department, UT Dallas, TX 4/23/2015
9

10. Research Aims and Objectives
 Aims
 Accurate and automatic patient positioning system (pre-
treatment)
 In-treatment automatic and accurate patient positioning with
patient drift compensation
 Objectives
 Surface-image control of the cranial flexion/extension motion of a
patient during simulated H&N RT (pre-treatment)
 Use of radio-transparent soft robot system for
positioning/manipulation tasks
Electrical Engineering Department, UT Dallas, TX 4/23/2015
10

11. Current System Set-up
Electrical Engineering Department, UT Dallas, TX 4/23/2015
11
Kinect RGBD Camera Sensor
Hair packed with medic neck
pillow to reduce effect of
infrared wavelengths
scattering and minimize
dropped pixels
Inflatable Air Bladder (IAB)
Mannequin
Head
NI myRIO microcontroller
Current Source (Air Flow)
Regulator Circuit
Inlet/Outlet Silicone
Tubes
Torso Ball Joint
Simulator
Inlet Proportional Flow
Control Valve
Data Processing System
24V DC Power
Supply

12. Vision Sensing System
 Kinect RGB-D Camera employed for position-based visual
servoing: Better depth image and alignment; Skeleton tracking
 Real-time Human Pose Recognition in Parts from Single Depth
Images. Jamie Shotton, et. al, CVPR 2011, Best Paper Award
 Real-time 3D face-tracking based on active appearance model
constrained by depth data. Nikolai Smolyanski et. al, Image and
Vision Computing, 2014, MS SDK v1.5.2
 Online Resources: OpenNI, ofxKinect
 OpenNI Process
 Generates 640 × 480 image at 30 fps with depth resolution of 40
centimeters
Electrical Engineering Department, UT Dallas, TX 4/23/2015
12

13. Kinect Depth Imaging
 Works with low light levels; Color and texture invariant
 Subtraction of background simplified
 Easy synthesis of realistic depth images of objects
 Cheap computational cost of building a large training dataset
 3 trees, 20 deep, 300k training images per tree, 2000 training example
pixels per image, 2000 candidate features θ, and 50 candidate
thresholds τ
Electrical Engineering Department, UT Dallas, TX 4/23/2015
13

14. Depth Constrained 3D Face Tracking
 MS Kinect SDK v1.5.2
 Uses depth data from the Kinect RGB-D Commodity camera to
enable 3D tracking using an active appearance method
 image difference patterns corresponding to changes in each
model parameter are learned and used to modify a model
estimate
 Resolves the monocular tracking problem by fitting an energy
term into the 2D+3D AAM fitting that minimizes a distance
between 3D face model vertices and depth data coming from a
RGBD camera.
Electrical Engineering Department, UT Dallas, TX 4/23/2015
14

15. Depth Constrained 3D Face Tracking
 Process Flow in Face Tracking System
 Find a face rectangle in a video frame using a face detector
 Use a neural network to find five points inside the face area – eye
centers, mouth corners, and tip of the nose
 Precompute scale of tracked face from the five points un-
projected to 3D camera space and scale 3D camera space
appropriately.
 Initialize next frame’s 2D face shape based on the
correspondences found by a robust local feature matching
between that frame and the previous frame.
 Results
 With the depth constrained 2D+3D AAM fitting, we found good
position-estimation results on a human subject when object is at a
distance of 1 to 2.5m from the Kinect System
Electrical Engineering Department, UT Dallas, TX 4/23/2015
15

16. Face Tracking Results
 Generalization errors and hence incorrect position estimation errors with
respect to the mannequin head due to inconsistency in depth data
 An ongoing investigation
 So we mostly relied on the OpenNI depth map centroids which we
computed for our position estimation
Electrical Engineering Department, UT Dallas, TX 4/23/2015
16
Tracking of a Human Face Tracking of a Mannequin Head

17. Modeling of Soft Robot System
 Modeling procedure: Overview
 Collect Data Set, 𝒁 𝑵
, of input signal, 𝑢 𝑡 , and output measurement,
𝑦(𝑡), respectively where
 𝒁 𝑵
= {𝑢 1 , 𝑦 1 , ⋯ , 𝑢 𝑁 , 𝑦 𝑁 }, 1 ≤ 𝑡 ≤ 𝑁 (1)
 Obtain a continuous-time parametric model structure similar to a one-
step ahead predictor
 𝑦 𝑛
= −𝑎 𝑛−1 𝑦 𝑛−1
− ⋯ − −𝑎0 𝑦 + 𝑏 𝑚 𝑢 𝑚
+ 𝑏 𝑚−1 𝑢 𝑚−1
+ ⋯ + 𝑏1 𝑢 + 𝑏0 𝑢 (2)
 Introduce vectors
𝜽 = 𝑎 𝑛−1, ⋯ , 𝑎0 𝑏 𝑚−1, ⋯ , 𝑏0 and 𝝓 𝒕 = −𝑦 𝑛−1
⋯ − 𝑦 − 𝑦 𝑢 𝑚
⋯ 𝑢
(3)
Since the model output depends on past data (2), we call the estimated value
𝒚 𝑡 𝜃 . Therefore, 𝒚 𝑡 𝜃 = 𝝓 𝑻
𝑡 𝜽 .
 Identification Goal: identify the best model, χ, in the set guided by the frequency
distribution analysis
 Choice of Excitation Input Signal: Sawtooth Waveform
 Integral and differential of a sawtooth waveform preserves the sawtooth
waveform with only phase and amplitude shifts
Electrical Engineering Department, UT Dallas, TX 4/23/2015
17

18. Modeling of Soft Robot System
 Spectrum contains both even and odd harmonics of the fundamental
frequency i.e. it contains all integer harmonics
 𝑥 𝑠𝑡ℎ 𝑡 =
𝐴
2
−
𝐴
𝜋 𝑘=1
8800 sin 2𝜋𝑘𝑓𝑡
𝑘
(4)
where A is the amplitude, A = 180mA and f is the signal frequency, 𝑓 =
10𝐻𝑧
Fig 1. Excitation Signal and Corresponding Head Position (Kinect
Measurement)
Electrical Engineering Department, UT Dallas, TX 4/23/2015
18

19. Data Pre-Processing
 Data Pre-processing
 We removed means and linear trends in collected data to minimize
disturbances above interest to desired system dynamics, mitigate
measurement outliers and non-continuous records in collected data
 Means removal
 𝒖 𝒂𝒗𝒆(𝒕) = 𝒖 𝑡 − 𝒖(𝑡) ; 𝒚 𝒂𝒗𝒆 𝑡 = 𝒚 𝑡 − 𝒚(𝑡); (5)
where 𝒖 𝒂𝒗𝒆(𝒕) and 𝒚 𝒂𝒗𝒆(𝒕) are the respective averages of the input and output
signals;
𝒖(𝒕) =
1
𝑁
𝑡=1
𝑛
𝑢 𝑡 𝑎𝑛𝑑 𝒚 (𝒕) =
1
𝑁
𝑡=1
𝑛
𝑦 𝑡 ;
 n is the discrete data length and N is the total data length
 Data Pre-processing
 Data detrending
 𝒖 𝒅 = 𝒖 𝒂𝒗𝒆 − 𝑨𝜽 𝒖, 𝒚 𝒅 = 𝒚 𝒂𝒗𝒆 − 𝑨𝜽 𝒚, (6)
 𝜽 𝒖 𝑎𝑛𝑑 𝜽 𝒚 are solutions to the least square fit equations
 𝑨 𝑻
𝑨 𝜽 𝑢 = 𝑨 𝑻
𝒖, 𝑨 𝑻
𝑨 𝜽 𝒚 = 𝑨 𝑇
𝒚
Electrical Engineering Department, UT Dallas, TX 4/23/2015
19

20. Data Pre-Processing
 and
 𝑨 𝑻
=
1 1 ⋯ 1 1
1
𝑁
1
𝑁
⋯
1
𝑁
1
𝑁
(7)
Electrical Engineering Department, UT Dallas, TX 4/23/2015
20
 Fig 2. Data Preprocessing: Means Removal

21. Cross-Correlation Analysis
 The cross-correlation function provides an estimate of the
system impulse response and is defined as:
 𝜓 𝑡 = 𝑡=𝜏+1
𝑁 𝑢 𝑡 −𝜏 − 𝑢 𝑦 𝑡 −𝜏 −𝑦(𝑡)
√ 𝑡=1
𝑁 𝑢 𝑡 − 𝑢
2
√ 𝑡=1
𝑁 𝑦 𝑡 −𝑦(𝑡)
2, 𝜏 = 0, ±1, Λ, ± 𝑁 − 1 (8)
 The cross-correlation function (CCF) is given by the convolution of the
system impulse response and the process auto-correlation function
(Wiener-Hopf equation)
 𝜓 𝑢𝑦(𝜏)=∫ ℎ 𝜈 𝔼 [𝑢 𝑡 𝑢(𝑡 + 𝜏 − 𝜈)d𝜈 =∫ ℎ 𝜈 𝜓 𝑢𝑢(𝜏 − 𝜈)d𝜈
 The cross correlation function between the output and test input is proportional to
the system impulse response when the input is white noise
 𝒖 𝑡 = 𝒖 𝒘 𝑡
1
𝑭(𝑧−1)
where 𝒖 𝒘 𝑡 is a zero mean white input sequence and
𝑭 𝑧−1
is an autoregressive model filter thus defined: 𝑭 𝑧−1
= 1 + 𝜆1 𝑧−1
+ Λ𝜆 𝜆 𝑧−𝜆
Electrical Engineering Department, UT Dallas, TX 4/23/2015
21

22. Cross-Correlation Analysis
 We estimate the parameters 𝜆𝑗 𝑗 = 1, 2, Λ, 𝜆 by fitting an AR model to 𝑢(𝑡) using a
least squares algorithm.
 The estimates of the filter, 𝑭 𝒛−𝟏
, are used to filter the input and output signals as
follows
𝑢 𝑤 𝑡 = 𝑭 𝒛−𝟏
𝑢 (𝑡)
𝑦 𝑤 𝑡 = 𝑭 𝒛−𝟏
𝑦 (𝑡)
(9)
 By estimating the filter 𝑭 𝒛−𝟏 𝑤𝑖𝑡ℎ 𝜆 = 20, the estimate of the cross-
correlation function was generated. The auto-correlation function tells of the
quality of the filter 𝑭 𝒛−𝟏 and is defined as
 𝜙11 𝜏 = 𝑡=𝜏+1
𝑁 𝒖 𝒘 𝑡 − 𝑢 𝑤 𝑡 [ 𝑢 𝑤 𝑡−𝜏 − 𝑢 𝑤]
𝑡=1
𝑁 [ 𝑢 𝑤 𝑡 − 𝑢 𝑤]2 , 𝜏 = 0, ±1, Λ, ± 𝑁 − 1
(10)
Electrical Engineering Department, UT Dallas, TX 4/23/2015
22

23. Correlation Analysis
Electrical Engineering Department, UT Dallas, TX 4/23/2015
23
Fig 3. ACF of detrended excitation signal

24. Correlation Analysis
Electrical Engineering Department, UT Dallas, TX 4/23/2015
24
Fig. 4. CCF between pre-whitened input and
output

25. Correlation of Residuals
 To determine the model structure of the system, we used the original
detrended data
 We chose a linear, second-order grey-box model set whose quality is
measurable by a mean-square error (MSE)
 Our choice, garnered from erstwhile analysis, ensures cost of model is not
too high in solving for 𝜽 𝑁;
 a high − order complex model is more difficult to use for simulation and
control design. If it is not marginally better than a simpler model, it may
not be worth the higher price [Llung, §16.8]
Electrical Engineering Department, UT Dallas, TX 4/23/2015
25

26. Sub-model Selection & Model Validation
4/23/2015Electrical Engineering Department, UT Dallas, TX
26
 A control system will perform well with an optimal linear sub-
model, tolerate disturbances and nonlinearities.
 We pick the linear frequency range: 0.00232 rad/sec to 6.85
rad/sec to represented the model of the soft robot system
 We carried out the canonical correlation to gain insight into the
desired model characteristics
 The correlation is measured between the measured head
position, 𝒚(𝑡), and estimated position by the auto-regressive
model we chose, i.e. the residuals
 𝝐 𝑡, 𝜽 𝑁 = 𝒚 𝑡 − 𝒚 𝑡 𝜽 𝑁
 The prediction errors (model error model) in the obtained model are
computed as a frequency response from the input to the residuals

27. Model Validation
4/23/2015Electrical Engineering Department, UT Dallas, TX
27
 Fig. 6. Correlation analysis of residuals between output, 𝒚(𝒕), and its estimate,
𝒚(𝒕)
Fig.5.Bodeplotofinputandoutput
Fig.6.Correlationfunctionofresiduals
Fig. 7. Estimates of standard variation of model from validation data set

28. Model Validation
4/23/2015Electrical Engineering Department, UT Dallas, TX
28
 The confidence interval compares the estimate with the estimated standard deviation
from the validation dataset
 A 99% confidence region (yellow bands) encloses the model response informing us
we have a reliable model [Llung (1999), §16.6]
 Comparing Model Structure
 Evaluation of different model structures and comparing quality of offered models
 Best fit:
 a second-order process model with delay and a RHP zero
 𝐺 𝑠 =
−0.006(𝑠 −1.7137)
(𝑠+0.01)(𝑠+0.1028)
𝑒−2𝑠
(11)
 The model has an 87.35% fit to original data with a mean square error of
0.05498𝑚𝑚2
and a final prediction error of 1.672.
 The open-loop step response of the identified transfer function is shown in Fig. 8.

29. Soft Robot Step Response
Electrical Engineering Department, UT Dallas, TX 4/23/2015
29
Fig. 8. Open Loop Step Response of Identified System

30. Control Analysis
 We see that the system is non-minimum phase with very slow transient
response.
 We require a controller that will increase the response time, guarantee
closed-loop stability whilst balancing robustness and controller
aggressiveness.
 Approximating the delay with the second-order Pade function,
𝐻 𝑠 =
𝑠2 − 3𝑠 + 3
𝑠2 + 3𝑠 + 3
 we introduce a PI controller,
𝐺𝑐 𝑠 = 3.79 +
0.0344
s
 nested within a PID controller, 𝐺 𝑝𝑖𝑑 = 3.4993 +
0.054765
𝑠
+ 55.8988𝑠, as in Fig. 9.
Electrical Engineering Department, UT Dallas, TX 4/23/2015
30

31. Close Loop Diagram and Step
Responses
4/23/2015Electrical Engineering Department, UT Dallas, TX
31
Fig 10. Closed Loop Step Response of Simulated
System
Fig. 9. Block Diagram of Control Network
Fig. 11. Experimental Results: Constant
Set-point

32. Experimental Results: Video
 Video 1. Experimental Results: Set-points Tracking
Electrical Engineering Department, UT Dallas, TX 4/23/2015
32

33. Conclusions and Future Work
 Deviations from desired positions during H&N Cancer RT cause dose variations
and degenerate treatment efficacy
 We have presented and demonstrated accurate control of cranial
flexion/extension motion of a patient during maskless H&N RT
 Our soft robot system can accurately track desired trajectory within 14 seconds
after start-up with the aid of a PID/PI feedforward network
 Future efforts include
 Extending results to deformable motions of the upper torso, and H&N.
 Improved bladder control: Adaptive Control, Gain Scheduling e.t.c.
 Incorporation of multi-bladders to accommodate multi-axis positioning
 Benefits
 Comprehensive and accurate control of the patient’s position
 Elimination of anatomical deformations as a result of positioning errors
Electrical Engineering Department, UT Dallas, TX 4/23/2015
33

34. References
 Cervino, L. I., et al. Frame-less and mask-less cranial stereotactic
radiosurgery: a feasibility study. 2010, Physics In Medicine And Biology
55(7): 1863-1873.
 Jemal A, Siegel R, Xu J, Ward E. Cancer statistics, 2010. CA: A Cancer
Journal for Clinicians2010; 60(5):277–300.
 L. Llung, System Identification Theory for the User, 2nd Edition, Upper
Saddle River, NJ, USA. Prentice Hall, 1999.
 Xing, L. Dosimetric effects of patient displacement and collimator and
gantry angle misalignment on intensity modulated radiation therapy.
Radiotherapy & Oncology, 2000. 56(1): p. 97 - 108
Electrical Engineering Department, UT Dallas, TX 4/23/2015
34