This document provides an overview of convolutional neural networks (CNNs) and describes a research study that used a two-dimensional heterogeneous CNN (2D-hetero CNN) for mobile health analytics. The study developed a 2D-hetero CNN model to assess fall risk using motion sensor data from 5 sensor locations on participants. The model extracts low-level local features using convolutional layers and integrates them into high-level global features to classify fall risk. The 2D-hetero CNN was evaluated against feature-based approaches and other CNN architectures and performed ablation analysis.
This presentation is Part 2 of my September Lisp NYC presentation on Reinforcement Learning and Artificial Neural Nets. We will continue from where we left off by covering Convolutional Neural Nets (CNN) and Recurrent Neural Nets (RNN) in depth.
Time permitting I also plan on having a few slides on each of the following topics:
1. Generative Adversarial Networks (GANs)
2. Differentiable Neural Computers (DNCs)
3. Deep Reinforcement Learning (DRL)
Some code examples will be provided in Clojure.
After a very brief recap of Part 1 (ANN & RL), we will jump right into CNN and their appropriateness for image recognition. We will start by covering the convolution operator. We will then explain feature maps and pooling operations and then explain the LeNet 5 architecture. The MNIST data will be used to illustrate a fully functioning CNN.
Next we cover Recurrent Neural Nets in depth and describe how they have been used in Natural Language Processing. We will explain why gated networks and LSTM are used in practice.
Please note that some exposure or familiarity with Gradient Descent and Backpropagation will be assumed. These are covered in the first part of the talk for which both video and slides are available online.
A lot of material will be drawn from the new Deep Learning book by Goodfellow & Bengio as well as Michael Nielsen's online book on Neural Networks and Deep Learning as well several other online resources.
Bio
Pierre de Lacaze has over 20 years industry experience with AI and Lisp based technologies. He holds a Bachelor of Science in Applied Mathematics and a Master’s Degree in Computer Science.
https://www.linkedin.com/in/pierre-de-lacaze-b11026b/
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
This presentation is Part 2 of my September Lisp NYC presentation on Reinforcement Learning and Artificial Neural Nets. We will continue from where we left off by covering Convolutional Neural Nets (CNN) and Recurrent Neural Nets (RNN) in depth.
Time permitting I also plan on having a few slides on each of the following topics:
1. Generative Adversarial Networks (GANs)
2. Differentiable Neural Computers (DNCs)
3. Deep Reinforcement Learning (DRL)
Some code examples will be provided in Clojure.
After a very brief recap of Part 1 (ANN & RL), we will jump right into CNN and their appropriateness for image recognition. We will start by covering the convolution operator. We will then explain feature maps and pooling operations and then explain the LeNet 5 architecture. The MNIST data will be used to illustrate a fully functioning CNN.
Next we cover Recurrent Neural Nets in depth and describe how they have been used in Natural Language Processing. We will explain why gated networks and LSTM are used in practice.
Please note that some exposure or familiarity with Gradient Descent and Backpropagation will be assumed. These are covered in the first part of the talk for which both video and slides are available online.
A lot of material will be drawn from the new Deep Learning book by Goodfellow & Bengio as well as Michael Nielsen's online book on Neural Networks and Deep Learning as well several other online resources.
Bio
Pierre de Lacaze has over 20 years industry experience with AI and Lisp based technologies. He holds a Bachelor of Science in Applied Mathematics and a Master’s Degree in Computer Science.
https://www.linkedin.com/in/pierre-de-lacaze-b11026b/
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Here, we have implemented CNN network in FPGA by incorporating a novel technique of convolution which includes pipelining technique as well as parallelism (by optimizing) between the two.
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
Once-for-All: Train One Network and Specialize it for Efficient Deploymenttaeseon ryu
안녕하세요 딥러닝 논문읽기 모임 입니다! 오늘 소개 드릴 논문은 Once-for-All: Train One Network and Specialize it for Efficient Deployment 라는 제목의 논문입니다.
모델을 실제로 하드웨어에 Deploy하는 그 상황을 보고 있는데 이 페이퍼에서 꼽고 있는 가장 큰 문제는 실제로 트레인한 모델을 Deploy할 하드웨어 환경이 너무나도 많다는 문제가 하나 있습니다 모든 디바이스가 갖고 있는 리소스가 다르기 때문에 모든 하드웨어에 맞는 모델을 찾기가 사실상 불가능하다는 문제를 꼽고 있고요
각 하드웨어에 맞는 옵티멀한 네트워크 아키텍처가 모두 다른 상황에서 어떻게 해야 될건지에 대한 고민이 일반적 입니다. 이제 할 수 있는 접근중에 하나는 각 하드웨어에 맞게 옵티멀한 아키텍처를 모두 다 찾는 건데 그게 사실상 너무나 많은 계산량을 요구하기 때문에 불가능하다라는 문제를 갖고 있습니다 삼성 노트 10을 예로 한 어플리케이션의 requirement가 20m/s로 그 모델을 돌려야 된다는 요구사항이 있으면은 그 20m/s 안에 돌 수 있는 모델이 뭔지 accuracy가 뭔지 이걸 찾기 위해서는 파란색 점들을 모두 찾아야 되고 각 점이 이제 트레이닝 한번을 의미하게 됩니다 그래서 사실상 다 수의 트레이닝을 다 해야지만 그 중에 뭐가 최적인지 또 찾아야 합니다. 실제 Deploy해야 되는 시나리오가 늘어나면 이게 리니어하게 증가하기 때문에
각 하드웨어에 맞는 그런 옵티멀 네트워크를 찾는게 사실상 불가능합니다.
그래서 이제 OFA에서 제안하는 어프로치는 하나의 네트워크를 한번 트레이닝 하고 나면 다시 하드웨어에 맞게 트레이닝할 필요 없이 그냥 각 환경에 맞게 가져다 쓸 수 있는 서브네트워크를 쓰면 된다 이게 주로 메인으로 사용하고 있는 어프로치입니다.
오늘 논문 리뷰를 위해 펀디멘탈팀 김동현님이 자세한 리뷰를 도와주셨습니다 많은 관심 미리 감사드립니다!
Introduction to computer vision with Convoluted Neural NetworksMarcinJedyk
Introduction to computer vision with Convoluted Neural Networks - going over history of CNNs, describing basic concepts such as convolution and discussing applications of computer vision and image recognition technologies
Convolutional Neural Networks : Popular Architecturesananth
In this presentation we look at some of the popular architectures, such as ResNet, that have been successfully used for a variety of applications. Starting from the AlexNet and VGG that showed that the deep learning architectures can deliver unprecedented accuracies for Image classification and localization tasks, we review other recent architectures such as ResNet, GoogleNet (Inception) and the more recent SENet that have won ImageNet competitions.
This is a presentation on Handwritten Digit Recognition using Convolutional Neural Networks. Convolutional Neural Networks give better results as compared to conventional Artificial Neural Networks.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Here, we have implemented CNN network in FPGA by incorporating a novel technique of convolution which includes pipelining technique as well as parallelism (by optimizing) between the two.
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
Once-for-All: Train One Network and Specialize it for Efficient Deploymenttaeseon ryu
안녕하세요 딥러닝 논문읽기 모임 입니다! 오늘 소개 드릴 논문은 Once-for-All: Train One Network and Specialize it for Efficient Deployment 라는 제목의 논문입니다.
모델을 실제로 하드웨어에 Deploy하는 그 상황을 보고 있는데 이 페이퍼에서 꼽고 있는 가장 큰 문제는 실제로 트레인한 모델을 Deploy할 하드웨어 환경이 너무나도 많다는 문제가 하나 있습니다 모든 디바이스가 갖고 있는 리소스가 다르기 때문에 모든 하드웨어에 맞는 모델을 찾기가 사실상 불가능하다는 문제를 꼽고 있고요
각 하드웨어에 맞는 옵티멀한 네트워크 아키텍처가 모두 다른 상황에서 어떻게 해야 될건지에 대한 고민이 일반적 입니다. 이제 할 수 있는 접근중에 하나는 각 하드웨어에 맞게 옵티멀한 아키텍처를 모두 다 찾는 건데 그게 사실상 너무나 많은 계산량을 요구하기 때문에 불가능하다라는 문제를 갖고 있습니다 삼성 노트 10을 예로 한 어플리케이션의 requirement가 20m/s로 그 모델을 돌려야 된다는 요구사항이 있으면은 그 20m/s 안에 돌 수 있는 모델이 뭔지 accuracy가 뭔지 이걸 찾기 위해서는 파란색 점들을 모두 찾아야 되고 각 점이 이제 트레이닝 한번을 의미하게 됩니다 그래서 사실상 다 수의 트레이닝을 다 해야지만 그 중에 뭐가 최적인지 또 찾아야 합니다. 실제 Deploy해야 되는 시나리오가 늘어나면 이게 리니어하게 증가하기 때문에
각 하드웨어에 맞는 그런 옵티멀 네트워크를 찾는게 사실상 불가능합니다.
그래서 이제 OFA에서 제안하는 어프로치는 하나의 네트워크를 한번 트레이닝 하고 나면 다시 하드웨어에 맞게 트레이닝할 필요 없이 그냥 각 환경에 맞게 가져다 쓸 수 있는 서브네트워크를 쓰면 된다 이게 주로 메인으로 사용하고 있는 어프로치입니다.
오늘 논문 리뷰를 위해 펀디멘탈팀 김동현님이 자세한 리뷰를 도와주셨습니다 많은 관심 미리 감사드립니다!
Introduction to computer vision with Convoluted Neural NetworksMarcinJedyk
Introduction to computer vision with Convoluted Neural Networks - going over history of CNNs, describing basic concepts such as convolution and discussing applications of computer vision and image recognition technologies
Convolutional Neural Networks : Popular Architecturesananth
In this presentation we look at some of the popular architectures, such as ResNet, that have been successfully used for a variety of applications. Starting from the AlexNet and VGG that showed that the deep learning architectures can deliver unprecedented accuracies for Image classification and localization tasks, we review other recent architectures such as ResNet, GoogleNet (Inception) and the more recent SENet that have won ImageNet competitions.
This is a presentation on Handwritten Digit Recognition using Convolutional Neural Networks. Convolutional Neural Networks give better results as compared to conventional Artificial Neural Networks.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
1. An Introduction to
Convolutional Neural Networks:
Overview, Implementation, and Example
Shuo Yu and Hsinchun Chen, AI Lab
University of Arizona
Updated April, 2020
1
8/1/2023
2. Acknowledgments
• Many of the images, results, and other materials are from:
• Deep Learning (2016), Ian Goodfellow, Yoshua Bengio, and Aaron Courville
• Lee Giles and Alex Ororbia, Penn State University
• Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature, May 28, 2015.
2
8/1/2023
3. Outline
• Introduction
• Academic Basis
• Building Blocks
• Convolutional Layer
• Non-linear Layer
• Pooling Layer
• Implementation
• Build a CNN with Keras in Python
• Research Example: 2D-hetero CNN for Mobile Health Analytics
• Research Design
• Evaluation and Results
3
8/1/2023
5. Representation Learning & Deep Learning
• Representation Learning is a set of methods that allows a machine to
be fed with raw data and to automatically discover the
representations needed for detection or classification.
• Deep Learning methods are representation learning methods with
multiple levels of representation, obtained by composing simple but
non-linear modules that each transform the representation at one
level (starting with the raw input) into a representation at a higher,
slightly more abstract level. With the composition of enough such
transformations, very complex functions can be learned.
• E.g., in image recognition, pixels edges motifs parts
objects; in speech recognition, sounds phones phonemes
syllables words phrases sentences
• Many DL methods build upon Deep Neural Networks (e.g., CNN).
5
6. Convolutional Neural Network (CNN)
6
8/1/2023
• Convolutional Neural Networks (CNN) is the most successful Deep
Learning method used to process multiple arrays, e.g., 1D for signals,
2D for images, 3D for video.
• CNN consists of a list of Neural Network layers that transform the
input data into an output (class/prediction).
• Great success in ImageNet competition in 2012 and later.
• Efficient use of GPUs (NVIDIA), ReLUs (10-20 layers, billions of
connections), and dropout for regularization.
• Development of specialized CNN chips (by NVIDIA, Intel, Samsung,
etc.) for real-time applications in smartphones, cameras, robots, self-
driving cars, etc.
7. Convolutional Neural Network (CNN)
• Convolutional Neural Networks, or Convolutional Networks, or CNNs, or
ConvNets
• For processing data with a grid-like or array topology
• 1-D grid: time-series data, sensor signal data
• 2-D grid: image data
• 3-D grid: video data
• CNNs include four key ideas
related to natural signals:
• Local connections
• Shared weights
• Pooling
• Use of many layers
7
8/1/2023
8. Convolutional Neural Network (CNN)
• Convolutional Neural Networks are inspired by mammalian visual
cortex.
• The visual cortex contains a complex arrangement of cells, which are sensitive
to small sub-regions of the visual field, called a receptive field.
• These cells act as local filters over the input space and are well-suited to
exploit the strong spatially local correlation present in natural images.
• Two basic cell types:
• Simple cells respond maximally to specific edge-like patterns within their receptive field.
• Complex cells have larger receptive fields and are locally invariant to the exact position
of the pattern.
8
11. CNN Architecture
• Intuition: Neural network with specialized connectivity structure
• Stacking multiple layers of feature extractors, low-level layers extract local features,
and high-level layers extract learn global patterns.
• There are a few distinct types of layers:
• Convolutional Layer: detecting local features through filters (discrete convolution)
• Non-linear Layer: normalization via Rectified Linear Unit (ReLU)
• Pooling Layer: merging similar features
11
12. Building-blocks for CNNs
12
Each sub-region yields a
feature map, representing
its feature.
Images are segmented into
sub-regions.
Feature maps are trained
with neurons and
weights.
Feature maps of a larger
region are combined.
Shared weights
13. (1) Convolutional Layer
• The core layer of CNNs
• Convolutional layer consists of a set of filters, wk,l
• Each filter covers a spatially small portion of the input data, Zi,j
• Each filter is convolved across the dimensions of the input data, producing a
multidimensional feature map.
• As we convolve the filter, we are computing the dot product between the parameters of the
filter and the input.
• Deep Learning algorithm: During training, the network corrects errors and filters
are learned, e.g., in Keras, by adjusting weights based on Stochastic Gradient
Descent, SGD (stochastic approximation of GD using a randomly selected subset
of the data).
• The key architectural characteristics of the convolutional layer is local
connectivity and shared weights.
13
14. Convolutional Layer: Local Connectivity
• Neurons in layer m are only connected to 3 adjacent
neurons in the m-1 layer.
• Neurons in layer m+1 have a similar connectivity
with the layer below.
• Each neuron is unresponsive to variations outside of
its receptive field with respect to the input.
• Receptive field: small neuron collections which process
portions of the input data
• The architecture thus ensures that the learnt feature
extractors produce the strongest response to a
spatially local input pattern.
14
15. Convolutional Layer: Shared Weights
• We show 3 hidden neurons belonging to the same feature
map (the layer right above the input layer).
• Weights of the same color are shared—constrained to be
identical.
• Replicating neurons in this way allows for features to be
detected regardless of their position in the input.
• Additionally, weight sharing increases learning efficiency
by greatly reducing the number of free parameters being
learnt.
15
16. (2) Non-linear Layer
• Intuition: Increase the nonlinearity of the entire architecture without
affecting the receptive fields of the convolution layer
• A layer of neurons that applies the non-linear activation function,
such as,
• 𝒇 𝒙 = 𝐦𝐚𝐱(𝟎, 𝒙) - Rectified Linear Unit (ReLU);
fast and most widely used in CNN
• 𝑓 𝑥 = tanh 𝑥
• 𝑓 𝑥 = | tanh 𝑥 |
• 𝑓 𝑥 = (1 + 𝑒−𝑥
)−1
- sigmoid
16
17. (3) Pooling Layer
• Intuition: to progressively reduce the spatial size of the representation to reduce
the amount of parameters and computation in the network, and hence to also
control overfitting
• Pooling partitions the input image into a set of non-overlapping rectangles and,
for each such sub-region, outputs the maximum value of the features in that
region.
17
Input
18. Building-blocks for CNNs
18
Each sub-region yields a
feature map, representing
its feature.
Images are segmented into
sub-regions.
Feature maps are trained
with neurons and
weights.
Feature maps of a larger
region are combined.
Shared weights
19. Other Layers
• The convolution, non-linear, and pooling layers are typically used as a
set. Multiple sets of the above three layers can appear in a CNN design.
• Input -> Conv. -> Non-linear -> Pooling -> Conv. -> Non-linear -> Pooling -> …->
Output
• Recent CNN architectures have 10-20 such layers.
• After a few sets, the output is typically sent to one or two fully
connected layers.
• A fully connected layer is a ordinary
neural network layer as in other neural networks.
• Typical activation function is the sigmoid function.
• Output is typically class (classification)
or real number (regression).
19
8/1/2023
20. Other Layers
• The final layer of a CNN is determined by the research task.
• Classification: Softmax Layer
𝑃 𝑦 = 𝑗 𝒙 =
𝑒𝒘𝑗⋅𝒙
𝑘=1
𝐾
𝑒𝒘𝑘⋅𝒙
• The outputs are the probabilities of belonging to each class.
• Regression: Linear Layer
𝑓 𝒙 = 𝒘 ⋅ 𝒙
• The output is a real number.
20
8/1/2023
23. Python CNN Implementation
• Prerequisites:
• Python 3.5+ (https://www.python.org/)
• TensorFlow (https://www.tensorflow.org/)
• Keras (https://keras.io/)
• Keras is a high-level neural networks API, written in
Python and capable of running on top of
TensorFlow, CNTK, or Theano.
• Recommended:
• NumPy
• Scikit-Learn
• NLTK
• SciPy
23
8/1/2023
24. Build a CNN in Keras
• The Sequential model is used to build a linear stack of layers.
• The following code shows how a typical CNN is built in Keras.
import numpy as np
import keras
from keras.models import Sequential
from keras.layers import Dense, Flatten
from keras.layers import Conv2D,
MaxPooling2D
from keras.optimizers import SGD
Note:
Dense is the fully connected layer;
Flatten is used after all CNN layers and
before a fully connected layer;
Conv2D is the 2D convolution layer;
MaxPooling2D is the 2D max pooling layer;
SGD is stochastic gradient descent algorithm.
24
8/1/2023
25. Build a CNN in Keras
(continued)
model = Sequential()
# We create an empty Sequential model and add layers onto it.
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(100,
100)))
# We add a Conv2D layer with 32 filters, 3x3 each, followed by a detector layer ReLU.
# This is the first layer we add to the model, so we need to specify the shape of the input. In this
case we assume our input is a 100x100 matrix.
model.add(MaxPooling2D(pool_size=(2, 2)))
# We add a MaxPooling2D layer with a 2x2 pooling size.
25
8/1/2023
26. Build a CNN in Keras
(continued)
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
# We can add more Conv2D and MaxPooling2D layers onto the model.
model.add(Flatten())
# After all the desired CNN layers are added, add a Flatten layer.
model.add(Dense(256, activation='sigmoid'))
# Add a fully connected layer followed by a detector layer with the sigmoid function.
model.add(Dense(10, activation='softmax')
# A softmax layer is added to achieve multiclass classification. In this example we have 10
classes.
26
8/1/2023
27. Build a CNN in Keras
(continued)
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
# Default SGD training parameters for correcting errors for filters
model.compile(loss='categorical_crossentropy', optimizer=sgd)
# Compile the model and use categorical crossentropy as the loss function, sgd as the optimizer
model.fit(x_train, y_train, batch_size=32, epochs=10)
# Fit the model with x_train and y_train, batch_size and epochs can be set to other
values
score = model.evaluate(x_test, y_test, batch_size=32)
# Evaluate model performance using x_test and y_test
27
8/1/2023
29. Introduction
• We developed two-dimensional heterogeneous convolutional neural networks
(2D-hetero CNN), a motion sensor-based system for fall risk assessment using
convolutional neural networks (CNN).
• Five sensor systems (chest, left/right thigh, left/right foot) for clinical tests
• Comprehensive assessment for gait and balance features
• CNNs are powerful in extracting low-level local features as well as integrating
them into high-level global features.
• Feature-less; avoid feature engineering that is labor intensive, ad hoc,
and inconclusive.
29
8/1/2023
30. Research Design: 2D-hetero CNN for Fall Risk
Assessment
30
Data Collection
Sensor Attachment
Data Preprocessing
Signal Segmentation
Model Design
2D-hetero CNN
Data Augmentation
Walking Test Evaluation
8/1/2023
31. Research Design – Data Collection
• Twenty-two (22) subjects were recruited at a Neurology Clinic.
• 12 with high fall risks, 10 with low fall risks
• 5 tri-axial accelerometers attached to each subject
• Sampling rate: 25 Hz
• 25 sampling points per second; sufficient for capturing gait
cycles
• Chest, left/right thigh, left/right foot (as shown in Fig. 4)
• To capture body and lower extremity movement (left/right)
• 10-meter ground walking tests were conducted to collect data for
gait and balance.
31
Fig. 4. Sensor Locations
Fig. 3. Shape and Size of
SilverLink Sensors
8/1/2023
32. Research Design – 2D-hetero CNN
32
Left/right thigh
6 x 100
Chest
3 x 100
Left/right foot
6 x 100
10 @ 2 x 96
3 x 5 conv.
stride (3, 1)
10 @ 2 x 96
3 x 5 conv.
stride (3, 1)
3 x 5 conv.
10 @ 1 x 96
10 @ 2 x 24
1 x 4 pool.
10 @ 2 x 24
1 x 4 pool.
1 x 4 pool.
10 @ 1 x 24
20 @ 1 x 20
2 x 5 conv.
20 @ 1 x 20
2 x 5 conv.
1 x 5 conv.
20 @ 1 x 20
20 @ 1 x 5
1 x 4 pool.
20 @ 1 x 5
1 x 4 pool.
1 x 4 pool.
20 @ 1 x 5 20 @ 3 x 5
Flatten
⋮
300
Fully
connected Softmax
classifier
2
Stage 1: Cross-Axial Convolution Stage 2: Cross-Locational Convolution Stage 3: Integration
Note: The notation “x @ y x z” denotes x feature maps with height y and width z.
8/1/2023
33. Research Design – 2D-hetero CNN
• We partitioned the data into three parts based on sensor locations.
• Chest, left/right thigh, left/right foot
• Stage 1: Cross-Axial Convolution
• Convolve among the three axes of a single sensor
• Extract features among axes within a sensor
• Stage 2: Cross-Locational Convolution
• Convolve between sensors on left/right thighs and left/right feet
• Extract balance features between the left and the right
• Stage 3: Integration
• Integrate extracted features to provide final inference on fall risk assessment
33
8/1/2023
34. Research Design – 2D-hetero CNN
Some Technical Details:
• A Rectified Linear Unit (ReLU) layer is added after each convolutional layer for
model non-linearity.
• A dropping layer is added after each pooling layer and the densely connected
layer to avoid over-fitting.
• Dataset split:
• Training (60%), validation (20%), test (20%)
• The validation set is used for model selection.
• The test set is used for reporting performance.
• As the model training process can get into local maxima, we train the model
for five times and report the average performance.
34
8/1/2023
35. Evaluation
• Benchmark 1: Feature-based fall risk assessment
• Most widely used approach for fall risk assessment
• Stride variability (SVAR), acceleration root mean square (ARMS), walking speed (SPD)
• Benchmark 2: CNN models with alternative architectures
• 2D homogeneous CNN (2D-homo CNN) as applied in image recognition tasks
• 1D CNN (1D-CNN) as applied in activity recognition and ECG classification tasks
• Benchmark 3: Ablation analysis
• 2D heterogeneous CNN with cross-axial convolutions only (2D-axis CNN)
• 2D heterogeneous CNN with cross-locational convolutions only (2D-loc CNN)
35
8/1/2023
36. Results
• Our proposed 2D-hetero CNN significantly outperformed all three
sets of benchmark systems.
• Showing the advantage of 2D-hetero CNN over traditional feature-based and
1D/2D CNN methods.
8/1/2023 36
0.00
0.20
0.40
0.60
0.80
1.00
Precision Recall Specificity F-measure
2D-hetero CNN SVAR ARMS SPD
0.00
0.20
0.40
0.60
0.80
1.00
Precision Recall Specificity F-measure
2D-hetero CNN 2D-homo CNN 1D CNN
0.00
0.20
0.40
0.60
0.80
1.00
Precision Recall Specificity F-measure
2D-hetero CNN 2D-axial CNN 2D-loc CNN
37. Conclusions
• We developed a 2D-hetero CNN to support fall risk assessment
based on motion sensor data, collected at a Parkinson Clinic.
• A novel CNN architecture with cross-axial and cross-locational
convolutions was proposed to optimize in our application context
of fall risk assessment.
• Our model achieved F-measure of 0.962, significantly
outperforming the benchmarks.
37
8/1/2023