Random Valued Impulse Noise Removal in Colour Images using Adaptive Threshold...IDES Editor
To remove random valued impulse noise from
colour images, an efficient impulse detection and filtering
scheme is presented. The locally adaptive threshold for
impulse detection is derived from the pixels of the filtering
window. The restoration of the noisy pixel is done on the basis
of brightness and chromaticity information obtained from the
neighbouring pixels in the filtering window. Experimental
results demonstrate that the proposed scheme yields much
superior performance in comparison with other colour image
filtering methods.
발표자: 배재성(KAIST 석사과정)
발표일: 2018.10.
최근 딥러닝을 이용한 방법은 다양한 음성 인식 과제에서 괄목할 만한 성과를 내고 있습니다. 특히 Convolutional Neural Network (CNN)을 이용한 방식은 지역적인 특징 (local feature)들을 효과적으로 잡아낼 수 있기 때문에 비교적 짧은 시간 의존도를 가지는 음성 키워드 인식이나 음소 단위 인식과 같은 과제들에서 활발히 사용되고 있습니다. 그러나 CNN은 낮은 레벨의 특징들 간의 공간적 관계성을 고려하지 않는다는 한계점이 있습니다. 이를 극복하기 위해 캡슐 네트워크 구조를 도입하여 음성 스펙트로그램에서 추출된 특징들의 공간적 관계성을 고려하고자 하였습니다. 구글 음성 단어 데이터셋에서 CNN과 그 성능을 비교해 보았으며, 깨끗한 환경과 잡음 환경 모두에서 주목할만한 성능 향상을 이끌어 냈습니다.
Random Valued Impulse Noise Removal in Colour Images using Adaptive Threshold...IDES Editor
To remove random valued impulse noise from
colour images, an efficient impulse detection and filtering
scheme is presented. The locally adaptive threshold for
impulse detection is derived from the pixels of the filtering
window. The restoration of the noisy pixel is done on the basis
of brightness and chromaticity information obtained from the
neighbouring pixels in the filtering window. Experimental
results demonstrate that the proposed scheme yields much
superior performance in comparison with other colour image
filtering methods.
발표자: 배재성(KAIST 석사과정)
발표일: 2018.10.
최근 딥러닝을 이용한 방법은 다양한 음성 인식 과제에서 괄목할 만한 성과를 내고 있습니다. 특히 Convolutional Neural Network (CNN)을 이용한 방식은 지역적인 특징 (local feature)들을 효과적으로 잡아낼 수 있기 때문에 비교적 짧은 시간 의존도를 가지는 음성 키워드 인식이나 음소 단위 인식과 같은 과제들에서 활발히 사용되고 있습니다. 그러나 CNN은 낮은 레벨의 특징들 간의 공간적 관계성을 고려하지 않는다는 한계점이 있습니다. 이를 극복하기 위해 캡슐 네트워크 구조를 도입하여 음성 스펙트로그램에서 추출된 특징들의 공간적 관계성을 고려하고자 하였습니다. 구글 음성 단어 데이터셋에서 CNN과 그 성능을 비교해 보았으며, 깨끗한 환경과 잡음 환경 모두에서 주목할만한 성능 향상을 이끌어 냈습니다.
Deep Learning Based Voice Activity Detection and Speech EnhancementNAVER Engineering
발표자: 김준태 (KAIST 박사과정)
발표일: 2018.10
Voice activity detection (VAD) and speech enhancement (SE) are important front-end technologies for noise robust speech recognition system.
From incoming noisy signal, VAD detects the speech signal only and SE removes the noise signal while conserving the speech signal.
For VAD and SE, this presentation will cover the traditional methods, deep learning based methods, and our papers as follows:
1. J. Kim and M. Hahn, "Voice Activity Detection Using an Adaptive Context Attention Model," in IEEE Signal Processing Letters, vol. 25, no. 8, pp. 1181-1185, Aug. 2018.
2. J. Kim and M. Hahn, "Speech Enhancement Using a Two Step Network," submitted to IEEE Signal Processing Letters, 2018.
Also, this presentation will briefly introduce some experimental results in real-world environment (far-field, noisy environment), conducted on the embedded board.
For VAD,
Traditional VAD methods.
Deep learning based VAD methods.
Paper presentation: J. Kim and M. Hahn, "Voice Activity Detection Using an Adaptive Context Attention Model," in IEEE Signal Processing Letters, vol. 25, no. 8, pp. 1181-1185, Aug. 2018.
End point detection based on VAD.
Experimental results of DNN-EPD on embedded board in real-world environment.
For SE,
Traditional SE methods.
Deep learning based SE methods.
Paper presentation: J. Kim and M. Hahn, "Speech Enhancement Using a Two Step Network," submitted to IEEE Signal Processing Letters, 2018.
Experimental results in real-world environment.
#6 PyData Warsaw: Deep learning for image segmentationMatthew Opala
Deep learning techniques ignited a great progress in many computer vision tasks like image classification, object detection, and segmentation. Almost every month a new method is published that achieves state-of-the-art result on some common benchmark dataset. In addition to that, DL is being applied to new problems in CV.
In the talk we’re going to focus on DL application to image segmentation task. We want to show the practical importance of this task for the fashion industry by presenting our case study with results achieved with various attempts and methods.
Image Compression Using Wavelet Packet TreeIDES Editor
Methods of compressing data prior to storage and
transmission are of significant practical and commercial
interest. The necessity in image compression continuously
grows during the last decade. The image compression includes
transform of image, quantization and encoding. One of the
most powerful and perspective approaches in this area is
image compression using discrete wavelet transform. This
paper describes a new approach called as wavelet packet tree
for image compression. It constructs the best tree on the basis
of Shannon entropy. This new approach checks the entropy of
decomposed nodes (child nodes) with entropy of node, which
has been decomposed (parent node) and takes the decision of
decomposition of a node. In addition, authors have proposed
an adaptive thresholding for quantization, which is based on
type of wavelet used and nature of image. Performance of the
proposed algorithm is compared with existing wavelet
transform algorithm in terms of percentage of zeros and
percentage of energy retained and signals to noise ratio.
This slide deck introduces Deep Learning concepts, such gradient descent, back propagation, activation functions, and CNNs. Basic knowledge of vectors, matrices, and Android, as well as elementary calculus (derivatives), are strongly recommended in order to derive the maximum benefit from this session.
Deep learning lecture - part 1 (basics, CNN)SungminYou
This presentation is a lecture with the Deep Learning book. (Bengio, Yoshua, Ian Goodfellow, and Aaron Courville. MIT press, 2017) It contains the basics of deep learning and theories about the convolutional neural network.
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...Marina Santini
In this lecture, we talk about two different discriminative machine learning methods: decision trees and k-nearest neighbors. Decision trees are hierarchical structures.k-nearest neighbors are based on two principles: recollection and resemblance.
This presentation focuses on Deep Learning (DL) concepts, such as neural networks, backprop, activation functions, and Convolutional Neural Networks, followed by a TypeScript-based code sample that replicates the Tensorflow playground. Basic knowledge of matrices is helpful for this session.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Multilabel Classification by BCH Code and Random ForestsIDES Editor
Multilabel classification deals with problems in
which an instance can belong to multiple classes. This
paper uses error correcting codes for multilabel
classification. BCH code and Random Forests learner are
used to form the proposed method. Thus, the advantage of
the error-correcting properties of BCH is merged with the
good performance of the random forests learner to
enhance the multilabel classification results. Three
experiments are conducted on three common benchmark
datasets. The results are compared against those of
several exiting approaches. The proposed method does
well against its counterparts for the three datasets of
varying characteristics.
Deep Learning Based Voice Activity Detection and Speech EnhancementNAVER Engineering
발표자: 김준태 (KAIST 박사과정)
발표일: 2018.10
Voice activity detection (VAD) and speech enhancement (SE) are important front-end technologies for noise robust speech recognition system.
From incoming noisy signal, VAD detects the speech signal only and SE removes the noise signal while conserving the speech signal.
For VAD and SE, this presentation will cover the traditional methods, deep learning based methods, and our papers as follows:
1. J. Kim and M. Hahn, "Voice Activity Detection Using an Adaptive Context Attention Model," in IEEE Signal Processing Letters, vol. 25, no. 8, pp. 1181-1185, Aug. 2018.
2. J. Kim and M. Hahn, "Speech Enhancement Using a Two Step Network," submitted to IEEE Signal Processing Letters, 2018.
Also, this presentation will briefly introduce some experimental results in real-world environment (far-field, noisy environment), conducted on the embedded board.
For VAD,
Traditional VAD methods.
Deep learning based VAD methods.
Paper presentation: J. Kim and M. Hahn, "Voice Activity Detection Using an Adaptive Context Attention Model," in IEEE Signal Processing Letters, vol. 25, no. 8, pp. 1181-1185, Aug. 2018.
End point detection based on VAD.
Experimental results of DNN-EPD on embedded board in real-world environment.
For SE,
Traditional SE methods.
Deep learning based SE methods.
Paper presentation: J. Kim and M. Hahn, "Speech Enhancement Using a Two Step Network," submitted to IEEE Signal Processing Letters, 2018.
Experimental results in real-world environment.
#6 PyData Warsaw: Deep learning for image segmentationMatthew Opala
Deep learning techniques ignited a great progress in many computer vision tasks like image classification, object detection, and segmentation. Almost every month a new method is published that achieves state-of-the-art result on some common benchmark dataset. In addition to that, DL is being applied to new problems in CV.
In the talk we’re going to focus on DL application to image segmentation task. We want to show the practical importance of this task for the fashion industry by presenting our case study with results achieved with various attempts and methods.
Image Compression Using Wavelet Packet TreeIDES Editor
Methods of compressing data prior to storage and
transmission are of significant practical and commercial
interest. The necessity in image compression continuously
grows during the last decade. The image compression includes
transform of image, quantization and encoding. One of the
most powerful and perspective approaches in this area is
image compression using discrete wavelet transform. This
paper describes a new approach called as wavelet packet tree
for image compression. It constructs the best tree on the basis
of Shannon entropy. This new approach checks the entropy of
decomposed nodes (child nodes) with entropy of node, which
has been decomposed (parent node) and takes the decision of
decomposition of a node. In addition, authors have proposed
an adaptive thresholding for quantization, which is based on
type of wavelet used and nature of image. Performance of the
proposed algorithm is compared with existing wavelet
transform algorithm in terms of percentage of zeros and
percentage of energy retained and signals to noise ratio.
This slide deck introduces Deep Learning concepts, such gradient descent, back propagation, activation functions, and CNNs. Basic knowledge of vectors, matrices, and Android, as well as elementary calculus (derivatives), are strongly recommended in order to derive the maximum benefit from this session.
Deep learning lecture - part 1 (basics, CNN)SungminYou
This presentation is a lecture with the Deep Learning book. (Bengio, Yoshua, Ian Goodfellow, and Aaron Courville. MIT press, 2017) It contains the basics of deep learning and theories about the convolutional neural network.
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...Marina Santini
In this lecture, we talk about two different discriminative machine learning methods: decision trees and k-nearest neighbors. Decision trees are hierarchical structures.k-nearest neighbors are based on two principles: recollection and resemblance.
This presentation focuses on Deep Learning (DL) concepts, such as neural networks, backprop, activation functions, and Convolutional Neural Networks, followed by a TypeScript-based code sample that replicates the Tensorflow playground. Basic knowledge of matrices is helpful for this session.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Multilabel Classification by BCH Code and Random ForestsIDES Editor
Multilabel classification deals with problems in
which an instance can belong to multiple classes. This
paper uses error correcting codes for multilabel
classification. BCH code and Random Forests learner are
used to form the proposed method. Thus, the advantage of
the error-correcting properties of BCH is merged with the
good performance of the random forests learner to
enhance the multilabel classification results. Three
experiments are conducted on three common benchmark
datasets. The results are compared against those of
several exiting approaches. The proposed method does
well against its counterparts for the three datasets of
varying characteristics.
IJERA (International journal of Engineering Research and Applications) is International online, ... peer reviewed journal. For more detail or submit your article, please visit www.ijera.com
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
論文紹介:Learning With Neighbor Consistency for Noisy LabelsToru Tamaki
Ahmet Iscen, Jack Valmadre, Anurag Arnab, Cordelia Schmid, "Learning With Neighbor Consistency for Noisy Labels" CVPR2022
https://openaccess.thecvf.com/content/CVPR2022/html/Iscen_Learning_With_Neighbor_Consistency_for_Noisy_Labels_CVPR_2022_paper.html
An introduction to Deep Learning concepts, with a simple yet complete neural network, CNNs, followed by rudimentary concepts of Keras and TensorFlow, and some simple code fragments.
Restricting the Flow: Information Bottlenecks for Attributiontaeseon ryu
101번째 영상,
펀디멘탈팀 김준호 님의
Restricting the Flow: Information Bottlenecks for Attribution
논문 리뷰 입니다
Explanable ai, xai와 관련된 페이퍼 입니다! 관련되어 관심있으신 분들이 많은 도움이 되시길 바랍니다! attribution map을 이용하여 결과물에 영향을 준 네트워크의 gradient를 직접 추적하여 비주얼 explanation을 추적하는 방식입니다! 펀디멘탈팀 김준호님이 밑바닥부터 자세한 리뷰를 도와주셨습니다!
오늘도 많은 관심과 사랑 감사합니다!
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
PhDThesis, Dr Shen Furao
1. 1/39
An Algorithm for Incremental
Unsupervised Learning and
Topology Representation
Shen Furao
Hasegawa Lab
Department of Computational
Intelligence and Systems Science
3. 3/39
Introduction
Clustering: Construct decision boundaries
based on unlabeled data.
Topology learning: find a topology
structure that closely reflects the topology
of the data distribution
Online incremental learning: Adapt to new
information without corrupting previously
learned information
4. 4/39
Vector Quantization
Targets
To minimize the average distortion through a
suitable choice of codewords
Application
Data compression, speech recognition
Separate the data set to Voronoi regions, find the
centroid of the Voronoi regions
LBG method (Linde, Buzo & Gray, 1980)
Dependence on initial starting conditions
Tendency to result in local minima
5. 5/39
Adaptive incremental LBG
(Shen & Hasegawa, 2005)
To solve the problem caused by poorly chosen
initial conditions
independent of initial conditions
With fixed number of codewords, to find a suitable
codebook to minimize the distortion error MQE.
It can work better than or same as ELBG (Patane &
Russo, 2001)
With fixed distortion error, to minimize the number
of codewords and find a suitable codebook.
Meaning: To get the same reconstruction quality for
different vector set, the codebook will have different size
and thus can save plenty of storage.
6. 6/39
Test Image
Lena (512*512*8) is
separated to 4*4 blocks. Such
blocks are the input vectors.
There are totally 16384
vectors.
Peak Signal to Noise Ratio
(PSNR) is used to evaluate the
resulting images after the
quantization process.
2552
PSNR 10 log10
1
N
i 1
( f (i ) g (i )) 2
N Lena (512*512*8)
7. 7/39
Improvement I:
Incrementally inserting codewords
The optimal
solution of k-
clustering
problem can
be reachable
from the (k-
1)-clustering
problem.
8. 8/39
Improvement II:
Distance measure function
Within cluster
distance must be
significantly less
than between
cluster distance.
l
d ( x, c) ( ( xi ci ) 2 ) p
i 1
p log10 q 1
9. 9/39
Improvement III:
Delete and insert codeword
Delete codeword
with lowest local
distortion error
Insert codeword
near the codeword
with highest local
distortion error
10. 10/39
Experiment 1
PSNR
Number of
codewords LBG (Linde Mk (Lee et ELBG(Pata
AILBG
et al.,1980) al., 1997) ne, 2001)
256 31.60 31.92 31.94 32.01
512 32.49 33.09 33.14 33.22
1024 33.37 34.42 34.59 34.71
Meaning: With the same number of codewords, proposed
method can get highest PSNR, i.e., with the same compression
ratio, proposed method can get best reconstruction quality.
11. 11/39
Experiment 2
Number of codewords
PSNR ELBG (Patane,
AILBG
2001)
31.94 256 244
33.14 512 488
34.59 1024 988
Meaning:
• With a predefined reconstruction quality, proposed method can
find a good codebook with reasonable number of codewords.
13. 13/39
Results of experiment 3
PSNR Number of codewords
(dB) Gray21 Lena Boat
28.0 9 22 54
30.0 12 76 199
33.0 15 454 1018
Meaning:
1. For different images, with the same PSNR, number of codewords will be different.
2. Proposed method can be used to set up an image database with same
reconstruction quality (PSNR)
14. 14/39
Unsupervised learning
Clustering
K-means (King, 1967), ELBG (Patane, 2001), Global k-means (Likas, 2003),
AILBG (Shen, 2005)
Determine the number of clusters k in advance
data sets consisting only of isotropic clusters
Single-link (Sneath, 1973), complete-link (King, 1967), CURE (Guha, 1998)
Computation overload, much memory space
Unsuitable for large data sets or online data
Topology Learning: Reflects topology of high-dimension data distribution
SOM (Kohonen, 1982): predetermined structure and size
CHL+NG (Martinetz, 1994): a priori decision about the network size
GNG (Fritzke, 1995): permanent increase in the number of nodes
Online Learning
GNG-U (Frutzke, 1998): destroy learned knowledge
LLCS (Hamker, 2001): supervised learning
15. 15/39
Self-organizing incremental neural
network (Shen & Hasegawa, 2005)
1. To process the on-line non-stationary data.
2. To do the unsupervised learning without any priori
condition such as:
• suitable number of nodes
• a good initial codebook
• how many classes there are
3. Report a suitable number of classes
4. Represent the topological structure of the input probability
density.
5. Separate the classes with some low-density overlaps
6. Detect the main structure of clusters polluted by noises
16. 16/39
The Proposed algorithm
First Layer Second Layer
Input Growing First Growing Second
pattern Network Output Network Output
Insert Delete
Classify
Node Node
17. 17/39
Algorithms
Insert new nodes
Criterion: nodes with high errors serve as a criterion to
insert a new node
error-radius is used to judge if the insert is successful
Delete nodes
Criterion: remove nodes in low probability density
regions
Realize: delete nodes with no or only one direct topology
neighbor
Classify
Criterion: all nodes linked with edges will be one cluster
18. 18/39
First-layer Second-layer
Input signals==
Initialize multiple of
Input signal Within-class
Insertion
Find winner Judge if insertion
and second winner is successful
Delete overlap and
Y Between-class noise nodes
Insertion
N N Input signals==
Connect winner multiple of LT
and second winner
Y
Update weight of First-layer Y
winner and neighbor
N
Output results
19. 19/39
Experiment
Environment
I II III IV V VI VII
A 1 0 1 0 0 0 0
B 0 1 0 1 0 0 0
C 0 0 1 0 0 1 0
D 0 0 0 1 1 0 0
E1 0 0 0 0 1 0 0
E2 0 0 0 0 0 1 0
Original Data Set E3 0 0 0 0 0 0 1
20. 20/39
Experiment:
Stationary environment
Original Data Set GNG (Fritzke, 1995)
21. 21/39
Experiment:
Stationary environment
Proposed method: first layer Proposed method: final results
33. 33/39
Application: Handwritten
character recognition
Optical Recognition of Handwritten Digits
database (optdigits) (UCI repository, 1996)
10 classes (handwritten digits) from a total of 43
people
30 contributed to the training set, 3823 samples
Different 13 to the test set, 1797 samples
Dimension of the samples is 64
Method:
Train: A separate SOINN to describe each class of data
Test: Classify an unknown data point according to
whichever model gives the best match (nearest
neighbor)
37. 37/39
Journal papers (2003~2005)
1. Shen Furao & Osamu Hasegawa, “An adaptive incremental LBG
for vector quantization,” Neural Networks, accepted.
2. Shen Furao & Osamu Hasegawa, “An incremental network for on-
line unsupervised classification and topology learning,” Neural
Networks, accepted.
3. Shen Furao & Osamu Hasegawa, Fractal image coding with
simulated annealing search, Journal of Advanced Computational
Intelligence and Intelligent Informatics, Vol.9, No.1, pp.80-88,
2005.
4. Shen Furao & Osamu Hasegawa, A fast no search fractal image
coding method, Signal Processing: Image Communication, vol.19,
pp.393-404, (2004)
5. Shen Furao & Osamu Hasegawa, A growing neural network for
online unsupervised learning, Journal of Advanced Computational
Intelligence and Intelligent Informatics, Vol.8, No.2, pp.121-129,
(2004)
38. 38/39
Refereed International
Conference (2003~2005)
1. Shen Furao, Youki Kamiya & Osamu Hasegawa, “An incremental neural network for online
supervised learning and topology representation,” 12th International Conference on Neural
Information Processing (ICONIP 2005), Taipei, Taiwan, October 30 - November 2, 2005, accepted.
2. Shen Furao & Osamu Hasegawa, “An incremental k-means clustering algorithm with adaptive
distance measure,” 12th International Conference on Neural Information Processing (ICONIP
2005), Taipei, Taiwan, October 30 - November 2, 2005, accepted.
3. Shen Furao & Osamu Hasegawa, “An on-line learning mechanism for unsupervised classification
and topology representation,” IEEE Computer Society International Conference on Computer
Vision and Pattern Recognition (CVPR 2005), San Diego, CA, USA, June 21-26, 2005.
4. Shen Furao & Osamu Hasegawa, “An incremental neural network for non-stationary unsupervised
learning,” 11th International Conference on Neural Information Processing (ICONIP 2004), Calcutta,
India, November 22-25, 2004.
5. Shen Furao & Osamu Hasegawa, “An effective fractal image coding method without search,” IEEE
International Conference on Image Processing (ICIP 2004), Singapore, October 24-27, 2004.
6. Youki Kamiya, Shen Furao & Osamu Hasegawa, “Non-stop learning : a new scheme for continuous
learning and recognition,” Joint 2nd SCIS and 5th ISIS, Keio University, Yokohama, Japan,
September 21-24, 2004.
7. Osamu Hasegawa & Shen Furao, “A self-structurizing neural network for online incremental
learning,” CD-ROM SICE Annual Conference in Sapporo, FAII-5-2, August 4-6, 2004.
8. Shen Furao & Osamu Hasegawa, “A self-organized growing network for on-line unsupervised
learning,” 2004 International Joint Conference on Neural Networks (IJCNN 2004), Budapest,
Hungary, CD-ROM ISBN 0-7803-8360-5, Vol.1, pp.11-16, 2004.
9. Shen Furao & Osamu Hasegawa, “A fast and less loss fractal image coding method using
simulated annealing,” 7th Joint Conference on Information Science (JCIS 2003), Cary, North
Carolina, USA, September 26-30, 2003.