Deep learning is a machine learning technique that uses artificial neural networks with multiple hidden layers to learn representations of data by increasing the level of abstraction from lower to higher layers. It has proven effective for multimedia data mining tasks like image tagging and caption generation. Deep neural networks can extract meaningful patterns from high-dimensional input using convolutional and recurrent layers, whereas shallow networks are limited. While deep learning has achieved good results, supervised approaches require large labeled datasets.

1. Multimedia Data Mining
using Deep Learning
Presented by
Bhagyashree V. Barde

2. Contents
Introduction
Data Mining
Multimedia mining
•Text Mining
•Image Mining
Shallow artificial Neural Networks models
Deep artificial Neural Network models
Conclusion

3. Introduction
• There is lot of multi-media data such as image,
text, audio, video, etc.
• To extract meaningful information from that
data we use various techniques.
• One of them is Deep learning.
• Deep learning is a new area of machine
learning

4. Data Mining
• Data:
It is a collection of numbers, words, measurements,
observations or even just descriptions of things.
• Information:
Organized data that has meaning.
• Knowledge:
Information can be converted into knowledge about
historical patterns and
future trends.
• Data Mining:
The task of discovering interesting patterns from

5. Contd..
Fig. knowledge discovery process

6. Contd..
1.Data cleaning: Noise data and irrelevant data are removed from the
collection.
2.Data integration: Multiple data sources, often heterogeneous, may be
combined in a common source.
3.Data selection: The data relevant to the analysis is decided on and retrieved
from the data collection.
4.Data transformation: The selected data is transformed into forms appropriate
for the mining procedure.
5.Data mining: It is the crucial step in which clever techniques are applied to
extract patterns potentially useful.
6.Pattern evaluation: Strictly interesting patterns representing knowledge are
identified based on given measures.
7.Knowledge representation: Discovered knowledge is visually represented to
the user.

7. Data Mining Techniques
• Characterization
• Discrimination
• Classification and Prediction
• Cluster analysis
• Outlier analysis
• Association analysis
• Evolution analysis

8. Multimedia Mining
Fig. Categories of Multimedia Data Mining
Multimedia data
mining
Static media Dynamic media
Image
mining
Audio
Mining
Video
Mining
Text
Mining

9. Contd..
• Text mining
• Image mining
• Video Mining
• Audio mining
It is a technique by which content of an audio
signal can be searched, analyzed, etc.

10. Converting Un-structured data to structured data
• Data resides in fixed
field within a record or
file is called structured
data.
• Unstructured data
means pixel
representation for an
image, audio, video
and character
representation for text
Data Mining tool
Structured
data
Unstruct
ured
data

11. Multimedia data mining process
Raw data
Training set
Model
Data collection
Data Pre-
processing
1.Data cleaning
2.Feature
extraction
Machine learning

12. Architecture for multimedia data mining
Input Multimedia
contents
Text Image Audio Video
Spatiotemporal
segmentation
Feature extraction
Evaluation of resultFinding the similar patterns

13. Contd..
• Input
• Multimedia content : It is selection stage which require user to
select the databases or subset of fields .
• Spatiotemporal segmentation : It is useful for object
segmentation. It is nothing but moving objects in image
sequences in the video.
• Feature extraction : It is preprocessing step.
• Finding similar patterns : It include some approaches of
finding similar pattern contain classification, clustering, etc.

14. Text Mining
• Text Mining is to
process unstructured
information, extract
meaningful
information from the
text.
Fig. Text mining process

15. Contd..
• Text preprocessing
Tokenization
Stop word removal
Stemming
• Text transformation
• Feature selection
• Text mining methods
• Evaluation

16. Techniques used in text
mining
• Information Extraction : Analyze unstructured text
and then finding relationships within text.
• Categorization : Assign one or more category to text
document.
• Clustering : It find groups of documents with similar
content
• Visualization : It improve and simplify the discovery
of relevant information
• Summarization : It reduce the length and detail of a
document.

17. Image Mining
• Image Mining is an extended branch of data mining
that is concerned with the process of knowledge
discovery concerning images.
• Image Mining deals with the extraction of image
patterns from a large collection of images.

18. Contd..
Query Image
Feature
Extraction
Query
Image
Features
Image
collection
Similarity
Matching
Query Image
Features
Feature
Extraction
Feature
Databases
Fig. Content Based Image Retrieval System Architecture

19. Shallow artificial neural network
• Many learning schemes use shallow artificial neural
network.
• Shallow artificial neural network has only one hidden
layer.
Fig. General symbol of neuron

20. Contd..
• If ANN becomes complicated then it tends to
be slow and are prone to over fitting.
• It starts to capture noise instead of
relationships between image.
• They are often incapable to extract meaningful
patterns from high-dimensional input .

21. What is Deep learning?
• Deep learning is a machine learning
technique .
• Deep learners are a type of artificial neural
networks with multiple layers.
• Multiple layers learn representations by
increasing the level of abstraction from one
layer to another.

22. Contd..
• Multimedia data mining has been used for
image tagging.
• Tagging has become a standard mechanism on
the Internet for annotating multimedia data and
search engines rely on tags to retrieve
multimedia data.
• Image caption generation is the process of
generating a descriptive sentence of an image.
• For that we are using Deep learning.

23. Deep artificial neural network
• It consists of multiple hidden layer.
• It works for decision making.
• They take an array of numbers that can
represent pixels and run a series of functions
on that array .
• It gives one or more number as output.
• The outputs are usually a prediction of some
object that you are trying to guess from input.

24. Contd..
• In deep neural network there are multiple hidden
layer with lowest layer takes the raw data like images,
text, sound, etc.
• Then each neurons stores some information about the
data they encounter.
• Each neuron in the layer sends information up to the
next layers of neurons .
• So the higher you go up, the more abstract features
you learn.

25. Convolutional neural network
• It is a type of feed forward artificial neural
network.
• Variations of multilayer perceptron which
are designed to use minimal amounts of
preprocessing.
• It take fixed size input and generate fixed
size output.

26. Recurrent neural networks
• Recurrent networks has connections that feed back
from the output to the input layer and also input layer
feed back to themselves.
• It allows loops.
• It use their internal memory to process arbitrary
sequences of inputs.
• It can handle arbitrary input, output length.
• It required much more data to give better result.
• It is more complex model.

27. Conclusion
• Deep learning has proven to be suitable for problems
where shallow learners didn’t provide satisfactory
results.
• The combination of convolutional network and RNN
has yielded very promising results in many domains.
• one drawback is that these methods mostly used
supervised approaches.

28. Thank you