Key frame extraction is an essential technique in the computer vision field. The extracted key frames should brief the salient events with an excellent feasibility, great efficiency, and with a high-level of robustness. Thus, it is not an easy problem to solve because it is attributed to many visual features. This paper intends to solve this problem by investigating the relationship between these features detection and the accuracy of key frames extraction techniques using TRIZ. An improved algorithm for key frame extraction was then proposed based on an accumulative optical flow with a self-adaptive threshold (AOF_ST) as recommended in TRIZ inventive principles. Several video shots including original and forgery videos with complex conditions are used to verify the experimental results. The comparison of our results with the-state-of-the-art algorithms results showed that the proposed extraction algorithm can accurately brief the videos and generated a meaningful compact count number of key frames. On top of that, our proposed algorithm achieves 124.4 and 31.4 for best and worst case in KTH dataset extracted key frames in terms of compression rate, while the-state-of-the-art algorithms achieved 8.90 in the best case.
Secure IoT Systems Monitor Framework using Probabilistic Image EncryptionIJAEMSJORNAL
In recent years, the modeling of human behaviors and patterns of activity for recognition or detection of special events has attracted considerable research interest. Various methods abounding to build intelligent vision systems aimed at understanding the scene and making correct semantic inferences from the observed dynamics of moving targets. Many systems include detection, storage of video information, and human-computer interfaces. Here we present not only an update that expands previous similar surveys but also a emphasis on contextual abnormal detection of human activity , especially in video surveillance applications. The main purpose of this survey is to identify existing methods extensively, and to characterize the literature in a manner that brings to attention key challenges.
TRANSFER LEARNING WITH CONVOLUTIONAL NEURAL NETWORKS FOR IRIS RECOGNITIONijaia
Iris is one of the common biometrics used for identity authentication. It has the potential to recognize persons with a high degree of assurance. Extracting effective features is the most important stage in the iris recognition system. Different features have been used to perform iris recognition system. A lot of them are based on hand-crafted features designed by biometrics experts. According to the achievement of deep learning in object recognition problems, the features learned by the Convolutional Neural Network (CNN) have gained great attention to be used in the iris recognition system. In this paper, we proposed an effective iris recognition system by using transfer learning with Convolutional Neural Networks. The proposed system is implemented by fine-tuning a pre-trained convolutional neural network (VGG-16) for features extracting and classification. The performance of the iris recognition system is tested on four public databases IITD, iris databases CASIA-Iris-V1, CASIA-Iris-thousand and, CASIA-Iris-Interval. The results show that the proposed system is achieved a very high accuracy rate.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Exploring visual and motion saliency for automatic video object extractionMuthu Samy
Sybian Technologies Pvt Ltd
Final Year Projects & Real Time live Projects
JAVA(All Domains)
DOTNET(All Domains)
ANDROID
EMBEDDED
VLSI
MATLAB
Project Support
Abstract, Diagrams, Review Details, Relevant Materials, Presentation,
Supporting Documents, Software E-Books,
Software Development Standards & Procedure
E-Book, Theory Classes, Lab Working Programs, Project Design & Implementation
24/7 lab session
Final Year Projects For BE,ME,B.Sc,M.Sc,B.Tech,BCA,MCA
PROJECT DOMAIN:
Cloud Computing
Networking
Network Security
PARALLEL AND DISTRIBUTED SYSTEM
Data Mining
Mobile Computing
Service Computing
Software Engineering
Image Processing
Bio Medical / Medical Imaging
Contact Details:
Sybian Technologies Pvt Ltd,
No,33/10 Meenakshi Sundaram Building,
Sivaji Street,
(Near T.nagar Bus Terminus)
T.Nagar,
Chennai-600 017
Ph:044 42070551
Mobile No:9790877889,9003254624,7708845605
Mail Id:sybianprojects@gmail.com,sunbeamvijay@yahoo.com
Measuring the Effects of Rational 7th and 8th Order Distortion Model in the R...IOSRJVSP
One of the biggest and important issues in the video watermarking is the distortion and attacks. The attacks and distortion affect the digital watermarking. Watermarking is an embedding process. With the help of watermarking, we insert the data into the digital objects. There are few methods are available for authentication of data, securing/protection of data. The watermarking technique also provides the data security, copyright protection and authentication of the data. Watermarking provides a comfortable life to authorized users. In my proposed work, we are working on distorted watermarked video. The distortion is present on the watermarked video is rational 7 th and 8 th order distortion model. In this paper, firstly we are embedding the watermark information into the original video and after that work on the distortion model which may be come into the watermarked video. We are also calculating the PSNR (Peak signal to noise ratio), SSIM (Structural similarity index measure), Correlation, BER (Bit Error Rate) and MSE (Mean Square Error) parameters for distorted watermarked video. We are showing the relationship between correlation and SSIM with BER, MSE and PSNR.
Our paper on homogeneous motion discovery oriented reference frame for high efficiency video coding talks about the idea of segmenting the current frame into cohesive motion regions made of blocks and then using these regions to form a motion compensated prediction. This prediction when used as an additional reference frame for the current frame, shows encouraging savings in bit rate over standalone HEVC reference coder.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Secure IoT Systems Monitor Framework using Probabilistic Image EncryptionIJAEMSJORNAL
In recent years, the modeling of human behaviors and patterns of activity for recognition or detection of special events has attracted considerable research interest. Various methods abounding to build intelligent vision systems aimed at understanding the scene and making correct semantic inferences from the observed dynamics of moving targets. Many systems include detection, storage of video information, and human-computer interfaces. Here we present not only an update that expands previous similar surveys but also a emphasis on contextual abnormal detection of human activity , especially in video surveillance applications. The main purpose of this survey is to identify existing methods extensively, and to characterize the literature in a manner that brings to attention key challenges.
TRANSFER LEARNING WITH CONVOLUTIONAL NEURAL NETWORKS FOR IRIS RECOGNITIONijaia
Iris is one of the common biometrics used for identity authentication. It has the potential to recognize persons with a high degree of assurance. Extracting effective features is the most important stage in the iris recognition system. Different features have been used to perform iris recognition system. A lot of them are based on hand-crafted features designed by biometrics experts. According to the achievement of deep learning in object recognition problems, the features learned by the Convolutional Neural Network (CNN) have gained great attention to be used in the iris recognition system. In this paper, we proposed an effective iris recognition system by using transfer learning with Convolutional Neural Networks. The proposed system is implemented by fine-tuning a pre-trained convolutional neural network (VGG-16) for features extracting and classification. The performance of the iris recognition system is tested on four public databases IITD, iris databases CASIA-Iris-V1, CASIA-Iris-thousand and, CASIA-Iris-Interval. The results show that the proposed system is achieved a very high accuracy rate.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Exploring visual and motion saliency for automatic video object extractionMuthu Samy
Sybian Technologies Pvt Ltd
Final Year Projects & Real Time live Projects
JAVA(All Domains)
DOTNET(All Domains)
ANDROID
EMBEDDED
VLSI
MATLAB
Project Support
Abstract, Diagrams, Review Details, Relevant Materials, Presentation,
Supporting Documents, Software E-Books,
Software Development Standards & Procedure
E-Book, Theory Classes, Lab Working Programs, Project Design & Implementation
24/7 lab session
Final Year Projects For BE,ME,B.Sc,M.Sc,B.Tech,BCA,MCA
PROJECT DOMAIN:
Cloud Computing
Networking
Network Security
PARALLEL AND DISTRIBUTED SYSTEM
Data Mining
Mobile Computing
Service Computing
Software Engineering
Image Processing
Bio Medical / Medical Imaging
Contact Details:
Sybian Technologies Pvt Ltd,
No,33/10 Meenakshi Sundaram Building,
Sivaji Street,
(Near T.nagar Bus Terminus)
T.Nagar,
Chennai-600 017
Ph:044 42070551
Mobile No:9790877889,9003254624,7708845605
Mail Id:sybianprojects@gmail.com,sunbeamvijay@yahoo.com
Measuring the Effects of Rational 7th and 8th Order Distortion Model in the R...IOSRJVSP
One of the biggest and important issues in the video watermarking is the distortion and attacks. The attacks and distortion affect the digital watermarking. Watermarking is an embedding process. With the help of watermarking, we insert the data into the digital objects. There are few methods are available for authentication of data, securing/protection of data. The watermarking technique also provides the data security, copyright protection and authentication of the data. Watermarking provides a comfortable life to authorized users. In my proposed work, we are working on distorted watermarked video. The distortion is present on the watermarked video is rational 7 th and 8 th order distortion model. In this paper, firstly we are embedding the watermark information into the original video and after that work on the distortion model which may be come into the watermarked video. We are also calculating the PSNR (Peak signal to noise ratio), SSIM (Structural similarity index measure), Correlation, BER (Bit Error Rate) and MSE (Mean Square Error) parameters for distorted watermarked video. We are showing the relationship between correlation and SSIM with BER, MSE and PSNR.
Our paper on homogeneous motion discovery oriented reference frame for high efficiency video coding talks about the idea of segmenting the current frame into cohesive motion regions made of blocks and then using these regions to form a motion compensated prediction. This prediction when used as an additional reference frame for the current frame, shows encouraging savings in bit rate over standalone HEVC reference coder.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Discovering Anomalies Based on Saliency Detection and Segmentation in Surveil...ijtsrd
This paper proposes extracting salient objects from motion fields. Salient object detection is an important technique for many content-based applications, but it becomes a challenging work when handling the clustered saliency maps, which cannot completely highlight salient object regions and cannot suppress background regions. We present algorithms for recognizing activity in monocular video sequences, based on discriminative gradient Random Field. Surveillance videos capture the behavioral activities of the objects accessing the surveillance system. Some behavior is frequent sequence of events and some deviate from the known frequent sequences of events. These events are termed as anomalies and may be susceptible to criminal activities. In the past, work was based on discovering the known abnormal events. Here, the unknown abnormal activities are to be detected and alerted such that early actions are taken. K. Shankar | Dr. S. Srinivasan | Dr. T. S. Sivakumaran | K. Madhavi Priya"Discovering Anomalies Based on Saliency Detection and Segmentation in Surveillance System" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-1 , December 2017, URL: http://www.ijtsrd.com/papers/ijtsrd5871.pdf http://www.ijtsrd.com/engineering/computer-engineering/5871/discovering-anomalies-based-on-saliency-detection-and-segmentation-in-surveillance-system/k-shankar
Ray-casting implementations require that the connectivity
between the cells of the dataset to be explicitly computed
and kept in memory. This constitutes a huge obstacle
for obtaining real-time rendering for very large models.
In this paper, we address this problem by introducing
a new implementation of the ray-casting algorithm for irregular
datasets. Our implementation optimizes the memory
usage of past implementations by exploring ray coherence.
The idea is to keep in main memory the information of
the faces traversed by the ray cast through every pixel under
the projection of a visible face. Our results show that
exploring pixel coherence reduces considerably the memory
usage, while keeping the performance of our algorithm
competitive with the fastest previous ones.
Automatic Foreground object detection using Visual and Motion SaliencyIJERD Editor
This paper presents a saliency-based video object extraction (VOE) framework. The proposed framework aims to automatically extract foreground objects of interest without any user interaction or the use of any training data (i.e., not limited to any particular type of object). To separate foreground and background regions within and across video frames, the proposed method utilizes visual and motion saliency information extracted from the input video. A conditional random field is applied to effectively combine the saliency induced features, which allows us to deal with unknown pose and scale variations of the foreground object (and its articulated parts). Based on the ability to preserve both spatial continuity and temporal consistency in the proposed VOE framework, experiments on a variety of videos verify that our method is able to produce quantitatively and qualitatively satisfactory VOE results.
Video Key-Frame Extraction using Unsupervised Clustering and Mutual ComparisonCSCJournals
Key-frame extraction is one of the important steps in semantic concept based video indexing and retrieval and accuracy of video concept detection highly depends on the effectiveness of keyframe extraction method. Therefore, extracting key-frames efficiently and effectively from video shots is considered to be a very challenging research problem in video retrieval systems. One of many approaches to extract key-frames from a shot is to make use of unsupervised clustering. Depending on the salient content of the shot and results of clustering, key-frames can be extracted. But usually, because of the visual complexity and/or the content of the video shot, we tend to get near duplicate or repetitive key-frames having the same semantic content in the output and hence accuracy of key-frame extraction decreases. In an attempt to improve accuracy, we proposed a novel key-frame extraction method based on unsupervised clustering and mutual comparison where we assigned 70% weightage to color component (HSV histogram) and 30% to texture (GLCM), while computing a combined frame similarity index used for clustering. We suggested a mutual comparison of the key-frames extracted from the output of the clustering where each key-frame is compared with every other to remove near duplicate keyframes. The proposed algorithm is both computationally simple and able to detect non-redundant and unique key-frames for the shot and as a result improving concept detection rate. The efficiency and effectiveness are validated by open database videos.
Key frame extraction for video summarization using motion activity descriptorseSAT Journals
Abstract Summarization of a video involves providing a gist of the entire video without affecting the semantics of the video. This has been implemented by the use of motion activity descriptors which generate relative motion between consecutive frames. Correctly capturing the motion in a video leads to the identification of the key frames in the video. This motion in the video can be obtained by using block matching techniques which is an important part of this process. It is implemented using two techniques, Diamond Search and Three Step Search, which have been studied and compared. The comparison process is tried across various videos differing in category, content, and objects. It is found that there is a trade-off between summarization factor and precision during the summarization process. Keywords: Video Summarization, Motion Descriptors, Block Matching
Key frame extraction for video summarization using motion activity descriptorseSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
IRJET-Feature Extraction from Video Data for Indexing and Retrieval IRJET Journal
Amanpreet Kaur ,Rimanpal Kaur "Feature Extraction from Video Data for Indexing and Retrieval ", International Research Journal of Engineering and Technology (IRJET), Vol2,issue-01 March 2015. e-ISSN:2395-0056, p-ISSN:2395-0072. www.irjet.net
Abstract
In recent years, the multimedia storage grows and the cost for storing multimedia data is cheaper. So there is huge number of videos available in the video repositories. With the development of multimedia data types and available bandwidth there is huge demand of video retrieval systems, as users shift from text based retrieval systems to content based retrieval systems. Selection of extracted features play an important role in content based video retrieval regardless of video attributes being under consideration. These features are intended for selecting, indexing and ranking according to their potential interest to the user. Good features selection also allows the time and space costs of the retrieval process to be reduced. This survey reviews the interesting features that can be extracted from video data for indexing and retrieval along with similarity measurement methods.
Recognition and tracking moving objects using moving camera in complex scenesIJCSEA Journal
In this paper, we propose a method for effectively tracking moving objects in videos captured using a
moving camera in complex scenes. The video sequences may contain highly dynamic backgrounds and
illumination changes. Four main steps are involved in the proposed method. First, the video is stabilized
using affine transformation. Second, intelligent selection of frames is performed in order to extract only
those frames that have a considerable change in content. This step reduces complexity and computational
time. Third, the moving object is tracked using Kalman filter and Gaussian mixture model. Finally object
recognition using Bag of features is performed in order to recognize the moving objects.
Discovering Anomalies Based on Saliency Detection and Segmentation in Surveil...ijtsrd
This paper proposes extracting salient objects from motion fields. Salient object detection is an important technique for many content-based applications, but it becomes a challenging work when handling the clustered saliency maps, which cannot completely highlight salient object regions and cannot suppress background regions. We present algorithms for recognizing activity in monocular video sequences, based on discriminative gradient Random Field. Surveillance videos capture the behavioral activities of the objects accessing the surveillance system. Some behavior is frequent sequence of events and some deviate from the known frequent sequences of events. These events are termed as anomalies and may be susceptible to criminal activities. In the past, work was based on discovering the known abnormal events. Here, the unknown abnormal activities are to be detected and alerted such that early actions are taken. K. Shankar | Dr. S. Srinivasan | Dr. T. S. Sivakumaran | K. Madhavi Priya"Discovering Anomalies Based on Saliency Detection and Segmentation in Surveillance System" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-1 , December 2017, URL: http://www.ijtsrd.com/papers/ijtsrd5871.pdf http://www.ijtsrd.com/engineering/computer-engineering/5871/discovering-anomalies-based-on-saliency-detection-and-segmentation-in-surveillance-system/k-shankar
Ray-casting implementations require that the connectivity
between the cells of the dataset to be explicitly computed
and kept in memory. This constitutes a huge obstacle
for obtaining real-time rendering for very large models.
In this paper, we address this problem by introducing
a new implementation of the ray-casting algorithm for irregular
datasets. Our implementation optimizes the memory
usage of past implementations by exploring ray coherence.
The idea is to keep in main memory the information of
the faces traversed by the ray cast through every pixel under
the projection of a visible face. Our results show that
exploring pixel coherence reduces considerably the memory
usage, while keeping the performance of our algorithm
competitive with the fastest previous ones.
Automatic Foreground object detection using Visual and Motion SaliencyIJERD Editor
This paper presents a saliency-based video object extraction (VOE) framework. The proposed framework aims to automatically extract foreground objects of interest without any user interaction or the use of any training data (i.e., not limited to any particular type of object). To separate foreground and background regions within and across video frames, the proposed method utilizes visual and motion saliency information extracted from the input video. A conditional random field is applied to effectively combine the saliency induced features, which allows us to deal with unknown pose and scale variations of the foreground object (and its articulated parts). Based on the ability to preserve both spatial continuity and temporal consistency in the proposed VOE framework, experiments on a variety of videos verify that our method is able to produce quantitatively and qualitatively satisfactory VOE results.
Video Key-Frame Extraction using Unsupervised Clustering and Mutual ComparisonCSCJournals
Key-frame extraction is one of the important steps in semantic concept based video indexing and retrieval and accuracy of video concept detection highly depends on the effectiveness of keyframe extraction method. Therefore, extracting key-frames efficiently and effectively from video shots is considered to be a very challenging research problem in video retrieval systems. One of many approaches to extract key-frames from a shot is to make use of unsupervised clustering. Depending on the salient content of the shot and results of clustering, key-frames can be extracted. But usually, because of the visual complexity and/or the content of the video shot, we tend to get near duplicate or repetitive key-frames having the same semantic content in the output and hence accuracy of key-frame extraction decreases. In an attempt to improve accuracy, we proposed a novel key-frame extraction method based on unsupervised clustering and mutual comparison where we assigned 70% weightage to color component (HSV histogram) and 30% to texture (GLCM), while computing a combined frame similarity index used for clustering. We suggested a mutual comparison of the key-frames extracted from the output of the clustering where each key-frame is compared with every other to remove near duplicate keyframes. The proposed algorithm is both computationally simple and able to detect non-redundant and unique key-frames for the shot and as a result improving concept detection rate. The efficiency and effectiveness are validated by open database videos.
Key frame extraction for video summarization using motion activity descriptorseSAT Journals
Abstract Summarization of a video involves providing a gist of the entire video without affecting the semantics of the video. This has been implemented by the use of motion activity descriptors which generate relative motion between consecutive frames. Correctly capturing the motion in a video leads to the identification of the key frames in the video. This motion in the video can be obtained by using block matching techniques which is an important part of this process. It is implemented using two techniques, Diamond Search and Three Step Search, which have been studied and compared. The comparison process is tried across various videos differing in category, content, and objects. It is found that there is a trade-off between summarization factor and precision during the summarization process. Keywords: Video Summarization, Motion Descriptors, Block Matching
Key frame extraction for video summarization using motion activity descriptorseSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
IRJET-Feature Extraction from Video Data for Indexing and Retrieval IRJET Journal
Amanpreet Kaur ,Rimanpal Kaur "Feature Extraction from Video Data for Indexing and Retrieval ", International Research Journal of Engineering and Technology (IRJET), Vol2,issue-01 March 2015. e-ISSN:2395-0056, p-ISSN:2395-0072. www.irjet.net
Abstract
In recent years, the multimedia storage grows and the cost for storing multimedia data is cheaper. So there is huge number of videos available in the video repositories. With the development of multimedia data types and available bandwidth there is huge demand of video retrieval systems, as users shift from text based retrieval systems to content based retrieval systems. Selection of extracted features play an important role in content based video retrieval regardless of video attributes being under consideration. These features are intended for selecting, indexing and ranking according to their potential interest to the user. Good features selection also allows the time and space costs of the retrieval process to be reduced. This survey reviews the interesting features that can be extracted from video data for indexing and retrieval along with similarity measurement methods.
Recognition and tracking moving objects using moving camera in complex scenesIJCSEA Journal
In this paper, we propose a method for effectively tracking moving objects in videos captured using a
moving camera in complex scenes. The video sequences may contain highly dynamic backgrounds and
illumination changes. Four main steps are involved in the proposed method. First, the video is stabilized
using affine transformation. Second, intelligent selection of frames is performed in order to extract only
those frames that have a considerable change in content. This step reduces complexity and computational
time. Third, the moving object is tracked using Kalman filter and Gaussian mixture model. Finally object
recognition using Bag of features is performed in order to recognize the moving objects.
VIDEO SUMMARIZATION: CORRELATION FOR SUMMARIZATION AND SUBTRACTION FOR RARE E...Journal For Research
The ever increasing number of surveillance camera networks being deployed all over the world has not only resulted in a high interest in the development of algorithms to automatically analyze the video footage, but has also opened new questions as how to efficiently manage the vast amount of information generated. The user may not have sufficient time to watch the entire video or the whole of video content may not be of interest to the user. In such cases, the user may just want to view the summary of the video instead of watching the whole video. In this paper, we present a video summarization technique developed in order to efficiently access the points of interest in the video footage. The technique aims to eliminate the sequences which contain no activity of significance. The system being developed actually captures each frame from the video, then it processes the frame; if the frame is of its interest, it retains the frames otherwise it discards the frame; hence the resultant video is very short. The proposed method is extended to obtain rare event detection for security systems. These rare event detections refer to suspicious scenarios. The system will consider a particular frame of interest from a video footage taken at given time and search for actions from video footages across the particular area of interest specified by the user. The user is then notified about the objects and actions occurred in the area of interest. This helps in detecting suspicious behavior that would have otherwise been deemed unsuspicious and gone unnoticed in the context of a narrow timeframe.
5 ijaems sept-2015-9-video feature extraction based on modified lle using ada...INFOGAIN PUBLICATION
Locally linear embedding (LLE) is an unsupervised learning algorithm which computes the low dimensional, neighborhood preserving embeddings of high dimensional data. LLE attempts to discover non-linear structure in high dimensional data by exploiting the local symmetries of linear reconstructions. In this paper, video feature extraction is done using modified LLE alongwith adaptive nearest neighbor approach to find the nearest neighbor and the connected components. The proposed feature extraction method is applied to a video. The video feature description gives a new tool for analysis of video.
Automated Traffic sign board classification system is one of the key technologies of Intelligent
Transportation Systems (ITS). Traffic Surveillance System is being more and important with improving
urban scale and increasing number of vehicles. This Paper presents an intelligent sign board
classification method based on blob analysis in traffic surveillance. Processing is done by three main
steps: moving object segmentation, blob analysis, and classifying. A Sign board is modelled as a
rectangular patch and classified via blob analysis. By processing the blob of sign boards, the meaningful
features are extracted. Tracking moving targets is achieved by comparing the extracted features with
training data. After classifying the sign boards the system will intimate to user in the form of alarms,
sound waves. The experimental results show that the proposed system can provide real-time and useful
information for traffic surveillance.
Multimodal video abstraction into a static document using deep learning IJECEIAES
Abstraction is a strategy that gives the essential points of a document in a short period of time. The video abstraction approach proposed in this research is based on multi-modal video data, which comprises both audio and visual data. Segmenting the input video into scenes and obtaining a textual and visual summary for each scene are the major video abstraction procedures to summarize the video events into a static document. To recognize the shot and scene boundary from a video sequence, a hybrid features method was employed, which improves detection shot performance by selecting strong and flexible features. The most informative keyframes from each scene are then incorporated into the visual summary. A hybrid deep learning model was used for abstractive text summarization. The BBC archive provided the testing videos, which comprised BBC Learning English and BBC News. In addition, a news summary dataset was used to train a deep model. The performance of the proposed approaches was assessed using metrics like Rouge for textual summary, which achieved a 40.49% accuracy rate. While precision, recall, and F-score used for visual summary have achieved (94.9%) accuracy, which performed better than the other methods, according to the findings of the experiments.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Key Frame Extraction in Video Stream using Two Stage Method with Colour and S...ijtsrd
Key Frame Extraction is the summarization of videos for different applications like video object recognition and classification, video retrieval and archival and surveillance is an active research area in computer vision. In this paper describe a new criterion for well presentative key frames and correspondingly, create a key frame selection algorithm based Two stage Method. A two stage method is used to extract accurate key frames to cover the content for the whole video sequence. Firstly, an alternative sequence is got based on color characteristic difference between adjacent frames from original sequence. Secondly, by analyzing structural characteristic difference between adjacent frames from the alternative sequence, the final key frame sequence is obtained. And then, an optimization step is added based on the number of final key frames in order to ensure the effectiveness of key frame extraction. Khaing Thazin Min | Wit Yee Swe | Yi Yi Aung | Khin Chan Myae Zin "Key Frame Extraction in Video Stream using Two-Stage Method with Colour and Structure" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-5 , August 2019, URL: https://www.ijtsrd.com/papers/ijtsrd27971.pdfPaper URL: https://www.ijtsrd.com/computer-science/data-processing/27971/key-frame-extraction-in-video-stream-using-two-stage-method-with-colour-and-structure/khaing-thazin-min
Similar to 24 7912 9261-1-ed a meaningful (edit a) (20)
In our homes or offices, security has been a vital issue. Control of home security system remotely always offers huge advantages like the arming or disarming of the alarms, video monitoring, and energy management control apart from safeguarding the home free up intruders. Considering the oldest simple methods of security that is the mechanical lock system that has a key as the authentication element, then an upgrade to a universal type, and now unique codes for the lock. The recent advancement in the communication system has brought the tremendous application of communication gadgets into our various areas of life. This work is a real-time smart doorbell notification system for home Security as opposes of the traditional security methods, it is composed of the doorbell interfaced with GSM Module, a GSM module would be triggered to send an SMS to the house owner by pressing the doorbell, the owner will respond to the guest by pressing a button to open the door, otherwise, a message would be displayed to the guest for appropriate action. Then, the keypad is provided for an authorized person for the provision of password for door unlocking, if multiple wrong password attempts were made to unlock, a message of burglary attempt would be sent to the house owner for prompt action. The main benefit of this system is the uniqueness of the incorporation of the password and messaging systems which denies access to any unauthorized personality and owner's awareness method.
Augmented reality, the new age technology, has widespread applications in every field imaginable. This technology has proven to be an inflection point in numerous verticals, improving lives and improving performance. In this paper, we explore the various possible applications of Augmented Reality (AR) in the field of Medicine. The objective of using AR in medicine or generally in any field is the fact that, AR helps in motivating the user, making sessions interactive and assist in faster learning. In this paper, we discuss about the applicability of AR in the field of medical diagnosis. Augmented reality technology reinforces remote collaboration, allowing doctors to diagnose patients from a different locality. Additionally, we believe that a much more pronounced effect can be achieved by bringing together the cutting edge technology of AR and the lifesaving field of Medical sciences. AR is a mechanism that could be applied in the learning process too. Similarly, virtual reality could be used in the field where more of practical experience is needed such as driving, sports, neonatal care training.
Image fusion is a sub field of image processing in which more than one images are fused to create an image where all the objects are in focus. The process of image fusion is performed for multi-sensor and multi-focus images of the same scene. Multi-sensor images of the same scene are captured by different sensors whereas multi-focus images are captured by the same sensor. In multi-focus images, the objects in the scene which are closer to the camera are in focus and the farther objects get blurred. Contrary to it, when the farther objects are focused then closer objects get blurred in the image. To achieve an image where all the objects are in focus, the process of images fusion is performed either in spatial domain or in transformed domain. In recent times, the applications of image processing have grown immensely. Usually due to limited depth of field of optical lenses especially with greater focal length, it becomes impossible to obtain an image where all the objects are in focus. Thus, it plays an important role to perform other tasks of image processing such as image segmentation, edge detection, stereo matching and image enhancement. Hence, a novel feature-level multi-focus image fusion technique has been proposed which fuses multi-focus images. Thus, the results of extensive experimentation performed to highlight the efficiency and utility of the proposed technique is presented. The proposed work further explores comparison between fuzzy based image fusion and neuro fuzzy fusion technique along with quality evaluation indices.
Graphs have become the dominant life-form of many tasks as they advance a
structure to represent many tasks and the corresponding relations. A powerful
role of networks/graphs is to bridge local feats that exist in vertices as they
blossom into patterns that help explain how nodal relations and their edges
impacts a complex effect that ripple via a graph. User cluster are formed as a
result of interactions between entities. Many users can hardly categorize their
contact into groups today such as “family”, “friends”, “colleagues” etc. Thus,
the need to analyze such user social graph via implicit clusters, enables the
dynamism in contact management. Study seeks to implement this dynamism
via a comparative study of deep neural network and friend suggest algorithm.
We analyze a user’s implicit social graph and seek to automatically create
custom contact groups using metrics that classify such contacts based on a
user’s affinity to contacts. Experimental results demonstrate the importance
of both the implicit group relationships and the interaction-based affinity in
suggesting friends.
This paper projects Gryllidae Optimization Algorithm (GOA) has been applied to solve optimal reactive power problem. Proposed GOA approach is based on the chirping characteristics of Gryllidae. In common, male Gryllidae chirp, on the other hand some female Gryllidae also do as well. Male Gryllidae draw the females by this sound which they produce. Moreover, they caution the other Gryllidae against dangers with this sound. The hearing organs of the Gryllidae are housed in an expansion of their forelegs. Through this, they bias to the produced fluttering sounds. Proposed Gryllidae Optimization Algorithm (GOA) has been tested in standard IEEE 14, 30 bus test systems and simulation results show that the projected algorithms reduced the real power loss considerably.
In the wake of the sudden replacement of wood and kerosene by gas cookers for several purposes in Nigeria, gas leakage has caused several damages in our homes, Laboratories among others. installation of a gas leakage detection device was globally inspired to eliminate accidents related to gas leakage. We present an alternative approach to developing a device that can automatically detect and control gas leakages and also monitor temperature. The system detects the leakage of the LPG (Liquefied Petroleum Gas) using a gas sensor, then triggred the control system response which employs ventilator system, Mobile phone alert and alarm when the LPG concentration in the air exceeds a certain level. The performance of two gas sensors (MQ5 and MQ6) were tested for a guided decision. Also, when the temperature of the environment poses a danger, LED (indicator), buzzer and LCD (16x2) display was used to indicate temperature and gas leakage status in degree Celsius and PPM respectively. Attension was given to the response time of the control system, which was ascertained that this system significantly increases the chances and efficiency of eliminating gas leakage related accident.
Feature selection problem is one of the main important problems in the text and data mining domain. This paper presents a comparative study of feature selection methods for Arabic text classification. Five of the feature selection methods were selected: ICHI square, CHI square, Information Gain, Mutual Information and Wrapper. It was tested with five classification algorithms: Bayes Net, Naive Bayes, Random Forest, Decision Tree and Artificial Neural Networks. In addition, Data Collection was used in Arabic consisting of 9055 documents, which were compared by four criteria: Precision, Recall, F-measure and Time to build model. The results showed that the improved ICHI feature selection got almost all the best results in comparison with other methods.
In this paper Gentoo Penguin Algorithm (GPA) is proposed to solve optimal reactive power problem. Gentoo Penguins preliminary population possesses heat radiation and magnetizes each other by absorption coefficient. Gentoo Penguins will move towards further penguins which possesses low cost (elevated heat concentration) of absorption. Cost is defined by the heat concentration, distance. Gentoo Penguins penguin attraction value is calculated by the amount of heat prevailed between two Gentoo penguins. Gentoo Penguins heat radiation is measured as linear. Less heat is received in longer distance, in little distance, huge heat is received. Gentoo Penguin Algorithm has been tested in standard IEEE 57 bus test system and simulation results show the projected algorithm reduced the real power loss considerably.
08 20272 academic insight on applicationIAESIJEECS
This research has thrown up many questions in need of further investigation.There was an expressive quantitative-qualitative research, which a common investigation form was used in.The dialogue item was also applied to discover if the contributors asserted the media-based attitude supplements their learning of academic English writing classes or not.Data recounted academic” insights toward using Skype as a sustaining implement for lessons releasing based on chosen variables: their occupation, year of education, and knowledge with Skype discovered that there were no important statistical differences in the use of Skype units because of medical academics major knowledge. There are statistically important differences in using Skype units. The findings also, disclosed that there are statistically significant differences in using Skype units due to the practice with Skype variable, in favors of academics with no Skype use practice. Skype instrument as an instructive media is a positive medium to be employed to supply academic medical writing data and assist education. Academics who do not have enough time to contribute in classes believe comfortable using the Skype-based attitude in scientific writing. They who took part in the course claimed that their approval of this media is due to learning academic innovative medical writing.
Cloud computing has sweeping impact on the human productivity. Today it’s used for Computing, Storage, Predictions and Intelligent Decision Making, among others. Intelligent Decision-Making using Machine Learning has pushed for the Cloud Services to be even more fast, robust and accurate. Security remains one of the major concerns which affect the cloud computing growth however there exist various research challenges in cloud computing adoption such as lack of well managed service level agreement (SLA), frequent disconnections, resource scarcity, interoperability, privacy, and reliability. Tremendous amount of work still needs to be done to explore the security challenges arising due to widespread usage of cloud deployment using Containers. We also discuss Impact of Cloud Computing and Cloud Standards. Hence in this research paper, a detailed survey of cloud computing, concepts, architectural principles, key services, and implementation, design and deployment challenges of cloud computing are discussed in detail and important future research directions in the era of Machine Learning and Data Science have been identified.
Notary is an official authorized to make an authentic deed regarding all deeds, agreements and stipulations required by a general rule. Activities carried out at the notary office such as recording client data and file data still use traditional systems that tend to be manual. The problem that occurs is the inefficiency in data processing and providing information to clients. Clients have difficulty getting information related to the progress of documents that are being taken care of at the notary's office. The client must take the time to arrive to the notary's office repeatedly to check the progress of the work of the document file. The purpose of this study is to facilitate clients in obtaining information about the progress of the work in progress, and make it easier for employees to process incoming documents by implementing an administrative system. This system was developed with the waterfall system development method and uses the Multi-Channel Access Technology integrated in the website to simplify the process of delivering information and requesting information from clients and to clients with Telegram and SMS Gateway. Clients will come to the office only when there is a notification from the system via Telegram or SMS notifying that the client must come directly to the notary's office, thus leading to an efficient time and avoiding excessive transportation costs. The overall functional system can function properly based on the results of alpha testing. The results of beta testing conducted by distributing the system feasibility test questionnaire to end users, get a percentage of 96% of users agree the system is feasible to be implemented.
In this work Tundra wolf algorithm (TWA) is proposed to solve the optimal reactive power problem. In the projected Tundra wolf algorithm (TWA) in order to avoid the searching agents from trapping into the local optimal the converging towards global optimal is divided based on two different conditions. In the proposed Tundra wolf algorithm (TWA) omega tundra wolf has been taken as searching agent as an alternative of indebted to pursue the first three most excellent candidates. Escalating the searching agents’ numbers will perk up the exploration capability of the Tundra wolf wolves in an extensive range. Proposed Tundra wolf algorithm (TWA) has been tested in standard IEEE 14, 30 bus test systems and simulation results show the proposed algorithm reduced the real power loss effectively.
In this work Predestination of Particles Wavering Search (PPS) algorithm has been applied to solve optimal reactive power problem. PPS algorithm has been modeled based on the motion of the particles in the exploration space. Normally the movement of the particle is based on gradient and swarming motion. Particles are permitted to progress in steady velocity in gradient-based progress, but when the outcome is poor when compared to previous upshot, immediately particle rapidity will be upturned with semi of the magnitude and it will help to reach local optimal solution and it is expressed as wavering movement. In standard IEEE 14, 30, 57,118,300 bus systems Proposed Predestination of Particles Wavering Search (PPS) algorithm is evaluated and simulation results show the PPS reduced the power loss efficiently.
In this paper, Mine Blast Algorithm (MBA) has been intermingled with Harmony Search (HS) algorithm for solving optimal reactive power dispatch problem. MBA is based on explosion of landmines and HS is based on Creativeness progression of musicians-both are hybridized to solve the problem. In MBA Initial distance of shrapnel pieces are reduced gradually to allow the mine bombs search the probable global minimum location in order to amplify the global explore capability. Harmony search (HS) imitates the music creativity process where the musicians supervise their instruments’ pitch by searching for a best state of harmony. Hybridization of Mine Blast Algorithm with Harmony Search algorithm (MH) improves the search effectively in the solution space. Mine blast algorithm improves the exploration and harmony search algorithm augments the exploitation. At first the proposed algorithm starts with exploration & gradually it moves to the phase of exploitation. Proposed Hybridized Mine Blast Algorithm with Harmony Search algorithm (MH) has been tested on standard IEEE 14, 300 bus test systems. Real power loss has been reduced considerably by the proposed algorithm. Then Hybridized Mine Blast Algorithm with Harmony Search algorithm (MH) tested in IEEE 30, bus system (with considering voltage stability index)- real power loss minimization, voltage deviation minimization, and voltage stability index enhancement has been attained.
Artificial Neural Networks have proved their efficiency in a large number of research domains. In this paper, we have applied Artificial Neural Networks on Arabic text to prove correct language modeling, text generation, and missing text prediction. In one hand, we have adapted Recurrent Neural Networks architectures to model Arabic language in order to generate correct Arabic sequences. In the other hand, Convolutional Neural Networks have been parameterized, basing on some specific features of Arabic, to predict missing text in Arabic documents. We have demonstrated the power of our adapted models in generating and predicting correct Arabic text comparing to the standard model. The model had been trained and tested on known free Arabic datasets. Results have been promising with sufficient accuracy.
In the present-day communications speech signals get contaminated due to
various sorts of noises that degrade the speech quality and adversely impacts
speech recognition performance. To overcome these issues, a novel approach
for speech enhancement using Modified Wiener filtering is developed and
power spectrum computation is applied for degraded signal to obtain the
noise characteristics from a noisy spectrum. In next phase, MMSE technique
is applied where Gaussian distribution of each signal i.e. original and noisy
signal is analyzed. The Gaussian distribution provides spectrum estimation
and spectral coefficient parameters which can be used for probabilistic model
formulation. Moreover, a-priori-SNR computation is also incorporated for
coefficient updation and noise presence estimation which operates similar to
the conventional VAD. However, conventional VAD scheme is based on the
hard threshold which is not capable to derive satisfactory performance and a
soft-decision based threshold is developed for improving the performance of
speech enhancement. An extensive simulation study is carried out using
MATLAB simulation tool on NOIZEUS speech database and a comparative
study is presented where proposed approach is proved better in comparison
with existing technique.
Previous research work has highlighted that neuro-signals of Alzheimer’s disease patients are least complex and have low synchronization as compared to that of healthy and normal subjects. The changes in EEG signals of Alzheimer’s subjects start at early stage but are not clinically observed and detected. To detect these abnormalities, three synchrony measures and wavelet-based features have been computed and studied on experimental database. After computing these synchrony measures and wavelet features, it is observed that Phase Synchrony and Coherence based features are able to distinguish between Alzheimer’s disease patients and healthy subjects. Support Vector Machine classifier is used for classification giving 94% accuracy on experimental database used. Combining, these synchrony features and other such relevant features can yield a reliable system for diagnosing the Alzheimer’s disease.
Attenuation correction designed for PET/MR hybrid imaging frameworks along with portion making arrangements used for MR-based radiation treatment remain testing because of lacking high-energy photon weakening data. We present a new method so as to uses the learned nonlinear neighborhood descriptors also highlight coordinating toward foresee pseudo-CT pictures starting T1w along with T2w MRI information. The nonlinear neighborhood descriptors are acquired through anticipating the direct descriptors interested in the nonlinear high-dimensional space utilizing an unequivocal constituent guide also low-position guess through regulated complex regularization. The nearby neighbors of every near descriptor inside the data MR pictures are looked during an obliged spatial extent of the MR pictures among the training dataset. By that point, the pseudo-CT patches are evaluated through k-closest neighbor relapse. The planned procedure designed for pseudo-CT forecast is quantitatively broke downward on top of a dataset comprising of coordinated mind MRI along with CT pictures on or after 13 subjects.
The cognitive radio prototype performance is to alleviate the scarcity of spectral resources for wireless communication through intelligent sensing and quick resource allocation techniques. Secondary users (SU’s) actively obtain the spectrum access opportunity by supporting primary users (PU’s) in cognitive radio networks (CRNs). In present generation, spectrum access is endowed through cooperative communication-based link-level frame-based cooperative (LLC) principle. In this SUs independently act as conveyors for PUs to achieve spectrum access opportunities. Unfortunately, this LLC approach cannot fully exploit spectrum access opportunities to enhance the throughput of CRNs and fails to motivate PUs to join the spectrum sharing processes. Therefore, to overcome this con, network level cooperative (NLC) principle was used, where SUs are integrated mutually to collaborate with PUs session by session, instead of frame based cooperation for spectrum access opportunities. NLC approach has justified the challenges facing in LLC approach. In this paper we make a survey of some models that have been proposed to tackle the problem of LLC. We show the relevant aspects of each model, in order to characterize the parameters that we should take in account to achieve a spectrum access opportunity.
In this paper, the author provides insights and lessons that can be learned from colleagues at American universities about their online education experiences. The literature review and previous studies of online educations gains are explored and summarized in this research. Emerging trends in online education are discussed in detail, and strategies to implement these trends are explained. The author provides several tools and strategies that enable universities to ensure the quality of online education. At the end of this research paper, the researcher provides examples from Arab universities who have successfully implemented online education and expanded their impact on the society. This research provides a strategy and a model that can be used by universities in the Middle East as a roadmap to implement online education in their regions.
About
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
• Remote control: Parallel or serial interface.
• Compatible with MAFI CCR system.
• Compatible with IDM8000 CCR.
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
• Easy in configuration using DIP switches.
Technical Specifications
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
Key Features
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
• Remote control: Parallel or serial interface
• Compatible with MAFI CCR system
• Copatiable with IDM8000 CCR
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
Application
• Remote control: Parallel or serial interface.
• Compatible with MAFI CCR system.
• Compatible with IDM8000 CCR.
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
• Easy in configuration using DIP switches.
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
2. IJEECS ISSN: 2502-4752
A meaningful Compact Key Frames Extraction in Complex… (Manar A. Mizher)
819
several methods for motion estimation such as feature matching, pel-recursive, deterministic
model, stochastic model, and optical flow. In addition, there are several techniques which have
been developed to approach the computation of optical flow such as gradient, correlation,
spatio-temporal energy, and phase [5].
The Lucas–Kanade method is widely used in differential methods for optical flow
estimation and computer vision by calculating the motion vector between two frames which are
taken at specific time. The advantages of the Lucas-Kanade method are: fast calculation,
suitability for real world tasks implementations, the accurate time derivatives on both real and
synthetic image sequences, producing accurate depth maps, and good noise tolerance.
The disadvantage of this technique is the rate of errors on boundaries of moving object.
Authors in [6] proposed a real time motion detection algorithm based on the integration
of accumulative optical flow and twice background filtering technique. Lucas-Kanade method
was employed to compute frame-to-frame optical flow to extract a 2D motion field.
The accumulative optical flow method was used to cope with variations in a changing
environment and to detect movement pixels in the video. The twice background filtering method
was used to extract moving object from the background information. The advantages of the
algorithm are: avoiding the need to learn the background model from a large number of frames,
and it can handle frame variations without prior knowledge of the object shapes and sizes.
The algorithm was reported to be able to detect tiny objects and even slow moving objects
accurately. Many authors focused on motion estimation and analysis to extract key frames to
protect important actions of original video and to provide a compact video representation.
However, most of the optical flow techniques are poor due to affected by motion discontinuities
and noise [7].
Generally, the technique for key frame extraction should provide a compact video
representation, but it should not be a complex and time consuming process. Key frame
extraction techniques can be categorized into one of these six groups: sequential comparison
between frames, global comparison between frames, reference frame, clustering,
curve simplification, and based on objects or events [1, 2, 8].
A variety of different key frame extraction techniques developed based on frame
difference, frame blocks differential, motion estimation and clustering were improved in recently
to extract key frames to segment video into shots or scenes. The simplest was the selection of
key frames by calculating the color histogram difference between two consecutive frames,
then computing the threshold based on the mean and the standard deviation of absolute
difference. After that, comparing the difference with the threshold if it is larger then select the
current frame as a keyframe. Steps were repeated till end of the video to extract all
keyframes [9]. In another typical method [10], key frames were extracted by comparing the
consecutive frame differences with the threshold value. The algorithm read each frame in a
video and converted them into grey level, and then it calculates the differences between two
consecutive frames. The algorithm then calculates the mean, standard deviation, and the
threshold which is equal to standard deviation multiply by a constant number. If the difference is
larger than the threshold the current frame will be saved as key frame. A key frame extraction
algorithm based on frame blocks differential accumulation with two thresholds was proposed
in [11]. In their algorithm, the first frame in a video is considered as a first reference frame. The
remaining video frames are then partitioned into equal sized image blocks. The created image
blocks are used to detect any local motion in the video. The color mean differences are
computed in RGB color space of the corresponding blocks in the reference frame and the
current frame. The algorithm counts the blocks changing in the current frame in relation to the
block changing in the reference frame. If the count number is greater than the global threshold,
this means the current frame has more changes than the reference frame. Then, the algorithm
uses the current frame as key frame instead of the reference frame and similar steps will be
repeated until the last frame. This algorithm was reported to show high efficiency to identify
movements and extract key frames with strong robustness in different types of video.
The majority of key frame extraction algorithms was developed to segment videos into
shots or scenes. There are very few researchers focused on extracting key frames within the
video shots with the camera moving, shaky camera, or dynamic background. However,
it is considered important in some fields which need a compact presentation of video scenes
such as in video forgery detection system based on fingerprint [12], video watermarking [13],
video copyright protection [14], and video summarization [15, 16]. Therefore, this paper
3. ISSN: 2502-4752
IJEECS Vol. 7, No. 3, September 2017 : 818 – 829
820
generated meaningful compact key frames using accumulative optical flow with self-adaptive
threshold inspired by the TRIZ inventive principles. The extracted key frames can be used to
represent the video as a whole and summarizes the important objects and the salient events of
the video. In Section 2, key frames extraction features based on TRIZ tools are analyzed,
and key frames extraction algorithm within the video shot is presented; Section 3 argues the
experiments and the analysis of the result. Finally, Section 4 concludes the work and suggests
future work.
2. Materials and methods
The extraction of key frames is the first step in video for detecting and segmenting
moving objects, detecting interesting regions, removing unwanted regions or objects,
or reconstructing and repairing damaged areas. Efficient algorithms for key frame extraction for
video sequences are highly desirable in the area of multimedia indexing and data retrieval due
to the challenges, including: moving objects with dynamic texture background, moving camera,
or shaky camera. The key frames extraction within a scene is an easier task than extracting
them inside a shot. The scene has a transition between two sequential shots and different views
from shot to shot, while the shot is a sequence of successive frames captured without
interruption by a same camera. Derive techniques to perform key frame extraction effectively
inside shots is a challenging task because video frames are attributed to many visual features
such as motion and color.
2.1. Analyzing Key Frames Extraction Features Based on TRIZ Tools
TRIZ is a Russian acronym for the Theory of Inventive Problem Solving (TIPS); it was
developed by Genrich Altshuller in 1946. TRIZ was applied in many disciplines and showed
some promising results [17-21]. TRIZ is known to provide a systematic approach to solve
innovative problems. It has many tools which include contradiction matrix, technical systems,
levels of innovation, ideality, and many more [22]. In this initial investigation, we applied one of
the popular TRIZ tools, namely, contradiction matrix, to study the effect of using static and
motion features and techniques on extracting key frame accuracy. The outcome of the TRIZ
systematic problem solving approach and the contradiction matrix will be a collection of TRIZ
recommended principles. Based on the ideas inspired by the TRIZ recommended principles, we
then derived a key frame extraction algorithm to perform inside a shot as it is an important issue
for some systems such as detecting video forgeries.
The TRIZ process involved for this key frame extraction problem includes: problem
identification, cause and effect chain analysis, contradiction matrix, ideation using TRIZ
principles.
2.1.1. Problem identification
At the start, some questions were raised as follows to help us understand the key frame
extraction problem:
A. Why it is difficult to choose any visual feature to extract the key frames?
Because every visual feature has its own advantages and disadvantages, and limited
by the dataset environment and camera conditions.
B. Why it will be a problem when choosing a wrong visual feature to extract the key frames?
Because it will affect the results in the video processing, and it will decrease the
accuracy of the algorithm.
Good features will be those that are able to detect the most meaningful keys and extract
compact count number of keys. The benefit if the key frame extraction problem is solved: it will
increase the algorithm accuracy of the result.
Further analysis of the advantages and disadvantages of the existing algorithms to
extract the key frames based on different visual features was conducted [23]. Several key
frames techniques are presented in Table 1. The extracted key frames method should compact
the important action in video with efficiency, feasibility, and robustness. It should avoid
computational complexity, and reduce as much redundancy as possible. The standard key
frame extraction methods can be classified in four categories which are: video shot method,
content analysis method, cluster method, and motion analysis method [24]. The advantages
and disadvantages of the key frame extraction algorithms are highlighted in Table 1.
4. IJEECS ISSN: 2502-4752
A meaningful Compact Key Frames Extraction in Complex… (Manar A. Mizher)
821
A common issue in these standard key frame extraction algorithms is that they were
used to segment videos into shots or scenes without focusing on extracting key frames within
shots. This issue is essential in some fields and it needs a compact summarization for video
shots such as in video forgery detection system based on fingerprint. Falsifying motion or faked
objects may occur within a single shot of the video and it can be conducted within the video
shots with a moving camera, shaky camera, or dynamic background.
The key frames techniques modelling were used to illustrate the key frames extraction
problem as an engineering system by defining the interaction between features and techniques.
The static features and motion are the main features in the key frames techniques problem.
The majority of key frame extraction techniques use these features with threshold to detect and
extract key frames; these are shown in Figure 1. In the key frame techniques modelling,
the main system features (mainly the color and motion) that play a role in causing accuracy
improvement are surrounded by a rounded rectangles of Figure 1.
Table 1. The analysis of the key frame extraction techniques within video
Classification Advantages Disadvantages
Content-based Method Very simple, select key frames according
to the change of content
Unsteady key frame extraction with motion
objects/background, affected by noise, not always the
most representative meaning
Cluster-based Method Accurate result Difficult to implement, more computational cost, difficult
to find general cluster parameters, needs pre-
processing steps
Video shot-based Method Temporal and spatial information, easy,
simple, low computation complexity,
quick extraction, representative meaning
Does not consider the complexity of content, fixed value
of key frames, does not efficiently describe the motion
content
Motion-based Method Easy to implement, detect motion, select
the appropriate number of key frames
Most suitable for static camera and background, high
computation, low robustness (local motion and does not
consider the content cumulative dynamic changes)
Selected fixed key frames Easy to implement, fast Missed sufficient information
Figure 1. Techniques modelling for key frame extraction problem
2.1.2. Cause and Effect Chain Analysis
The cause and effect chain analysis will look into the possible root causes that relate to
key frames detection features and extraction techniques trying to solve the accuracy problem.
Based on this problem statement, the cause and effect chain analysis can be summarized in the
following questions:
(a) Why does key frames extraction problem occur?
The causes that contribute to this problem are the imprecise in determining the main
features (static features, motion) threshold causes low accuracy in the final result. And the
difficulty in determining the threshold happened because it is very sensitive and heavily
dependent on the dataset used.
(b) Why is it difficult to determine the threshold?
5. ISSN: 2502-4752
IJEECS Vol. 7, No. 3, September 2017 : 818 – 829
822
It is attributed to many possible causes. The static feature detection techniques are
affected by noise or changing conditions during the video such as, shadow, light on/off. Motion
detection technique is not suitable for moving background/camera and shaky camera. The
cause and effect chain analysis can be illustrated as in Figure 2.
Figure 2. Cause and effect chain analysis of key frames extraction problem
2.1.3. Contradiction Atrix
The development of the solutions was based on TRIZ engineering contradictions that
were derived from possible root causes. The details of these engineering contradictions
including the improving and worsening features are related to each root cause.
These contradictions are summarized in Table 2 where the recommended inventive principles to
solve the respective root cause were determined as well.
Table 2. Summary of TRIZ tools process to solve the key frames extraction problem
Problem
Root
causelevel
1
Root
causelevel
2
Contradiction
Improving
feature
Worsening
feature
Inventive
Principles
Solutions
Keyframesextraction
Difficulttodetermine
Affectedby
noise
Static features with
threshold save time, but
affected by noise
Savetime
(25)
Noise(37)
35-
Parameter
changes
- Need to change static features
threshold parameters.
Affectedby
changingthe
conditions
Static features with
threshold easy to
implement, but affected by
changing the conditions
Easyto
implement(33)
Dominatedby
video
conditions(35)
1-Segmentati
on
16-Partial
- Focusing on detecting static
features on spatial domain.
- Need to remove small regions
to reduce the complexity.
Sensitivetodataset
Dominatedbyvideoenvironmentandcamera
conditions
Motion with threshold
accurate result, but
dominated by video
conditions
Accurate
result(28)
Dominatedby
video
conditions
(35)
35-Parameter
changes
2-
Segregation
- Need to change motion
threshold parameters.
- Focusing on detecting motion
on temporal domain.
Motion with threshold easy
to implement, but
dominated by video
conditions
Easyto
implement(33)
Dominatedby
videoconditions
(35)
1-Segmentation
16-Partial
- Focusing on detecting motion
on spatial domain.
- Need to remove small motion
to reduce the complexity.
Motion with threshold save
time, but dominated by
video conditions
Savetime
(25)
Dominated
byvideo
conditions
(35)
35-
Parameter
changes
- Need to change motion
threshold parameters.
6. IJEECS ISSN: 2502-4752
A meaningful Compact Key Frames Extraction in Complex… (Manar A. Mizher)
823
2.1.3. Ideation using TRIZ Principles
The threshold can be classified into global, adaptive, or global and adaptive
combined [2]. In addition, the motion estimation could be affected by different parameters such
as the video environment conditions (static or dynamic objects with static or dynamic
background and texture), video capturing conditions (static, shaky or moving camera), noise,
technique used to estimate the moving objects’ velocities, detection of spatial and temporal
information. Based on the solutions inspired by TRIZ principles in Table 2, we have derived an
algorithm, named as, Accumulative Optical Flow with Self-adaptive Threshold (AOF_ST) to
detect and extract key frames on a dataset with different video environment cases and
camera conditions.
The implemented TRIZ inspired solutions include:
1 Using a full self-adaptive threshold with variables: variable values could be calculated
depending on motion feature to have a full dynamic system to overcome the difficulty and
the sensitivity (35- parameter changes).
2 Focusing on reading motion on spatial domain using optical flow (1-segmentation).
3 For a temporal domain, using a fixed number of cumulative frames (2-segregation) will lead
to reducing the effects of videos environment and camera conditions and to provide more
accurate results.
Removing small motion (noise) to make it easier to implement and to reduce complexity
(16-partial).
2.2. Accumulative Optical Flow with Self-Adaptive Threshold (AOF_ST)
AOF_ST is an algorithm that has two phases: pre-processing phase and key frames
extraction phase. The flow chart of the proposed AOF_ST algorithm is shown in Figure 3.
The details of each phase are discussed:
Figure 3. Flow chart of AOF_ST algorithm
2.2.1. Preprocessing Phase
In this phase, a median filter was used to remove noise before resampling the video.
The video frames were reconstructed using a specific frame rate resampling and resolution
resampling size (320 W × 320 H), and a fixed rate (30 frames/second). In addition, we defined
video quality equals to 100 where the quality is a number between 0 and 100. Higher quality
numbers result higher video quality and larger file size. All videos were converted to .avi video
format to avoid any unexpected problems through next phase implementation.
2.2.2. Key Frames Extraction Phase
This phase extracts the most important key frames which contain critical motion of
objects or texture by estimating objects’ velocities based on accumulative optical flow with fixed
7. ISSN: 2502-4752
IJEECS Vol. 7, No. 3, September 2017 : 818 – 829
824
key frames between three to ten keys. For each video shot, and to have a limited number of key
frames to make it easier to use them in different systems such as object detection, a video
search with retrieval and generating a fingerprint is conducted. This will increase the level of
security in the process of video forgery detection.
After the preprocessing phase, AOF_ST reads both consecutive frames fi and fi-1 at time
i from the resampled video shots. It converts frames to single precision and then using the
optical flow to estimate objects’ velocities from each frame based on Lucas-Kanade method.
This process will generate coordinate points and draw lines to indicate flow. Equations 1 and 2
are used to calculate accumulative optical flow threshold depending on absolute frames
difference summation.
( ) (1)
∑ ∑ ( ) (2)
Where, FRdiff is the absolute difference frame between the two consecutive frames, fi is
the current frame, fi-1 is the previous frame. AccKey is the accumulative optical flow threshold, c
and r are the FRdiff number of columns and rows respectively.
When AccKey is greater than zero, there is a noticed motion between fi and fi-1 and
based on AccKey, accumulate 20 frames for each key frame. We chose 20 as an average for
accumulative key frames to be moderate and suitable for detecting moving objects with static or
dynamic background, and this would be suitable with a static camera, moving camera or shaky
camera videos.
The selection of key frames using AOF_ST depends on three cases. In the first case, if
the key frames count is between three and ten, the key frames are saved with their masks.
In the second case, if the key frames count is less than three, it means that the shot has no
important motions to detect. Thus, choose the first, middle, and last frames as key frames set.
In the third case, if the extracted key frames count is more than ten, it means the shot has highly
dynamic texture and/or more motion objects need to be detected. In the third case, all detected
key frames are flagged as temporally key frames, the self-adaptive threshold is used to filter
them and delete key frames with less motion difference. Eq.3 calculates the absolute
differences Dydiff between two consecutive temporally key frames. In Eq.4 based on Dydiff, the
self-adaptive DThreshold calculates, and in Eq.5, when the DThreshold is greater than the initial
Dvalue, the temporally key frame is saved as a filtered key frame. If it is less, then the Dvalue is
increased by a constant number con until filtered all temporally key frames. The initial Dvalue
and its increment were chosen by the user, and the final values determined based on
implementing several comparisons. Initial Dvalue equals 10000 while the increment value con
equals 5000.
( ) (3)
∑ ∑ ( ) (4)
(5)
Where, Dydiff is the absolute difference frame between the two consecutive temporally
key frames, Mx is the current temporal key frame, My is the next temporally key frame.
DThreshold is the self-adaptive threshold which compared with Dvalue to filtered key frames; c
and r are the number of Dydiff columns and rows respectively. Dvalue is the dynamic variable
increased by a con number, it improves the threshold because it adapts itself to the sequence
statistics, which increase the performance of the extraction method [25].
3. Experiments and Result Analysis
The aim of the experiments is to see whether choosing visual features have a possible
influence on the accuracy of extraction key frames or not. The experiments will focus on the
color and the motion because they are the main visual features widely used to detect and
extract key frames. The specific hypothesis “motion feature with self-adaptive threshold will
increase the accuracy of extracting key frames more than color feature”. The independent
8. IJEECS ISSN: 2502-4752
A meaningful Compact Key Frames Extraction in Complex… (Manar A. Mizher)
825
variables are the algorithms that being used to extract key frames, they are absolute color
histogram difference [9], the grey level consecutive frame differences [10], RGB color space
frame blocks differential accumulation [11], and motion on accumulative optical flow with self-
adaptive threshold (AOF_ST).
The dependent variable is indicator of extraction key frames accuracy which based on
the count number of key frames that were extracted. The possible confound was the motion
based on optical flow with self-adaptive threshold could increase the key extraction accuracy
more than the-state-of-the-art algorithms which based on color with local and/or global
thresholds. Thus, to measure the accuracy, we implemented different algorithms based on
these techniques.
Our experiments were performed on a laptop, Toshiba Satellite C850-B098 with CPU-
Intel (R) core (TM) i3-2312M CPU @ 2.10 GHz, memory RAM-2 GB, system type 32-bit. Our
system was programmed with Matlab2013a using Windows 7. Key frames were extracted using
our AOF_ST and it was compared with the-state-of-the-art algorithms, the absolute color
histogram difference [9], the grey level consecutive frame differences with threshold [10] and
RGB color space frame blocks differential accumulation [11] respectively. Five videos from KTH
as described in [9] were used as training set. All selected videos were represented as individual
action in a single shot. Therefore, the shot detection techniques were not performed.
RGB color space frame blocks differential accumulation [11] has two thresholds as
shown in Equations 6 and 7, the mValue is the mean differences of all blocks in each frame, m
and n are the number of columns and rows respectively in the frame. Variable a range is [0,1]
and variable b range is [-10,10]. We implemented RGB color space frame blocks differential
accumulation [11] with different chosen values for threshold variables a and b to determine the
most suitable thresholds values to have an accurate results and it had the best CR when a = 1
and b = - 6.
(6)
( ) (7)
The compactness measure of shots contents due to the extracted key frames was
computed using the compression ratio (CR) to evaluate the performance of key frame extraction
algorithms. The higher value of CR of an algorithm indicates that the algorithm is good [9].
The CR was calculated using the Equation 8 from [9].
(8)
The CR results of the absolute color histogram difference were described in [9], while
the CR results of our proposed algorithm, the grey level consecutive frame differences with
threshold [10], and RGB color space frame blocks differential accumulation [11] were obtained
by experiments. As shown in Table 3, our proposed algorithm had the best CR of extracting key
frames for the five videos.
Table 3. CR for AOF_ST and the-state-of-the-art algorithms
Video
Frames
count
Compression rate of the extracted key frames
Our alg. Color his. diff. Con. frame diff.
Frame blocks
diff. (1,-6)
Person1_runing_d1 335 47.85 5.68 4.14 1.58
Person1_ runing_d2 365 33.18 8.90 2.92 1.79
Person1_ runing_d3 350 43.75 6.48 3.72 1.57
Person2_ runing_d1 314 31.4 6.83 2.51 1.35
Person2_ runing_d2 1492 124.34 7.54 1.79 1.75
To study key frame extraction under different complex environmental cases, a validation
set with different 24 video shots were experimented. These were: moving objects with static
camera, moving objects with a moving camera, dynamic texture with a static camera and not
9. ISSN: 2502-4752
IJEECS Vol. 7, No. 3, September 2017 : 818 – 829
826
moving objects, dynamic texture with a static camera and moving objects, and dynamic texture
with a shaky camera. The validation set videos being used are described in Table 4.
Table 4. Description of the validation set videos
Original
video
Description
Faked
video
Forgery Description
v1 Static camera with moving object v2 Duplicate moving object then merge them
v3 Small shaky camera with moving
objects within dynamic texture
v4 Remove static object within dynamic texture
v5 Small shaky camera with moving
objects
v6 Objects motion interpolation
v7 Moving camera with moving objects v8 Object motion interpolation
v9 Zoom out camera with dynamic texture v10 Extended dynamic texture
v11 Small shaky camera with dynamic
texture
v12 Extended dynamic texture
v13 Static camera with dynamic texture v14 Added moving objects within dynamic texture
v15 Extended dynamic texture
v16 Static camera with dynamic texture v17 Extended dynamic texture
v18 Static camera with dynamic texture v19 Removed static object within dynamic texture
v20 Static camera with moving objects within
dynamic texture
v21 Added moving objects within dynamic texture
v22 Added moving object within dynamic texture
v23 Small shaky camera with dynamic
texture
v24 Extended dynamic texture and duplicated
frames
Table 5 shows the statistical analysis of the validation set videos. The shot’s size varied
between 4.93 MB to 32.7 MB, and with a different duration length between 3 seconds to 17
seconds before pre-processing and after pre-processing between 3.07 seconds to 14.87
seconds with size (320 W × 320 H).
Table 5. Statistics of the validation set videos
Featur Frames count Duration (Sec.)
Mean 241.17 8.0389
Std. Deviation 104.792 3.49300
Minimum 92 3.07
Maximum 446 14.87
To realize if there is any effect of choosing different visual features and techniques on
the accuracy of key frame extraction, Table 6 shows the mean and standard deviation of
implementing different algorithms based on different features on different videos conditions.
From the statistical description in Table 6, we can conclude that AOF_ST had a compact count
number of key frames compared to the-state-of-the-art algorithms (Mean 7.63 and Std. Dev.
2.28). The-state-of-the-art algorithms had “Mean” greater than 25 and Std. Dev. larger than 17.
Table 6. Descriptive statistics about the total count number of extracted key frames
AOF_ST Color his. diff.
Con. frame
diff.
Frame blocks diff.
(0.05,-6) (0.50,-6) (1,-6)
Mean 7.63 25.46 111.08 193.58 193.58 180.88
Std.
Deviation
2.281 17.983 125.301 106.223 106.223 113.084
Minimum 3 4 4 46 46 6
Maximum 10 60 443 444 444 441
We used the nonparametric methods (Binomial test) from the SPSS ver.20 statistical
package to understand whether accuracy of generating a compact number of key frames
differed based on features and techniques used. The statistical test conducted for our algorithm
and the-state-of-the-art algorithms is necessary to confirm their accuracy. If gained results are
near to the supposed value that we need to achieve, i.e. compact number of key frames which
is equal or less than 10 key frames. We investigated whether the proportion of key frames
10. IJEECS ISSN: 2502-4752
A meaningful Compact Key Frames Extraction in Complex… (Manar A. Mizher)
827
extracted differs from 0.90 (our null hypothesis states that this proportion is 0.90 of the entire
population). To define the dichotomous variable we used cut point equaling 10 to check if the
algorithms generate a compact count number of key frames equal or less 10. Since our
algorithm has 24 videos achieved the condition <=10 out of 24 observations, the observed
proportion is (24 / 24 = 1.0). The p value denoted by Exact Sig.(1-tailed) of our algorithm is 0.08.
If the proportion of key frames extraction is exactly 1.0 in the entire population, then there's only
a 8 % chance of finding more than ten key frames in any extracted video. We often reject the
null hypothesis if this chance is smaller than 5% (p < .05). While on the-state-of-the-art
algorithms the maximum observed proportion does not increase than 0.2 and their observed
proportions are smaller than the test proportion. Therefore, we accepted our algorithm results
and ignore the-state-of-the-art algorithms result. As a result, the Binomial test indicated that in
our algorithm the proportion of extracting key frames equal or less than ten of 1.0 was higher
than the expected 0.90, p = 0.08 (1- tailed) as shown in Table 7.
Table 7. The binomial test result of extracting key frames counts number
Algorithm Category N
Observed
Prop.
Test Prop.
Exact Sig.
(1-tailed)
AOF_ST <= 10 24 1.0 .90 .080
Color his. diff. <= 10 5 .2 .90 .000a
Con. frame diff. <= 10 2 .1 .90 .000a
Frame blocks diff. (0.05,-6) <= 10 0 0 .90 .000a
Frame blocks diff. (0.50,-6) <= 10 1 0 .90 .000a
Frame blocks diff. (1,-6) <= 10 2 .1 .90 .000a
Alternative hypothesis states that the proportion of cases in the first group < 0.90
4. Conclusion
After analyzing the results using the Binomial test, we concluded that the accuracy of
extracting key frames can be affected by visual features (color, motion). In general, motion
features give more information about movements on video when compared to color features.
As a result, motion features extracts a compact count number of key frames.
Our proposed algorithm, AOF_ST, which was developed based on motion feature,
provided good and meaningful compact key frames extraction. This proposed algorithm is useful
for video fingerprint generation, detecting video forgery system, video retrieval and searching
system. The-state-of-the-art algorithms in the experiments, namely, the absolute color
histogram difference [9], the grey level consecutive frame differences [10], RGB color space
frame blocks differential accumulation [11], were not able to extract a compact count number of
key frames in all condition cases which can affect the usability of their algorithms in different
systems. Our proposed algorithm has a controlled number of key frames between three and ten.
Spatio-temporal changes provide significant key frames with accurate representation
consequent to detect meaningful details about the salient events inside the video [26].
Our proposed algorithm, AOF_ST, focuses on estimating motion on spatial domain using optical
flow, and for a temporal domain uses a fixed number of cumulative frames. Thus, our proposed
algorithm has meaningful key frames, reduces the effects of videos environment and camera
conditions and provides more accurate results. In comparison, the-state-of-the-art algorithms
tested in our experiments were unable to describe the shots in an efficient manner due to the
redundancy or very similar key frames. In this regard, we created a summation of key frames
difference to show the redundancy of detecting moving objects within the key frames to give a
better understanding of the meaningful key frames.
As shown in Figure 4 (video 1), the absolute color histogram difference [9] and the grey
level consecutive frame differences [10] missed some salient events, while RGB color space
frame blocks differential accumulation [11] extracted a set of different number of key frames
based on the threshold, and missed more salient events than the absolute color histogram
difference [9] and the grey level consecutive frame differences [10].
The accuracy of the key frame in Figure 4 (video 2) shows that our proposed algorithm
AOF_ST is better than the-state-of-the-art algorithms in the experiments. The-state-of-the-art
algorithms in the experiments had extracted redundant or very similar key frames.
On the opposite, our proposed algorithm AOF_ST detected the important salient events in the
two videos with compact number of key frames.
11. ISSN: 2502-4752
IJEECS Vol. 7, No. 3, September 2017 : 818 – 829
828
Figure 4. Meaningful key frames extraction between AOF_ST and the-state-of-the-art algorithms
In addition, it is hard to determine the threshold value for the RGB color space frame
blocks differential accumulation [11] and several experiments need to be attempted to find the
appropriate threshold value. For all video shots, our proposed algorithm in this paper was able
to extract key frames automatically. The threshold in our algorithm was self-adaptive and that
makes our algorithm suitable for full automatic application in future.
5. Conclusion and Future Work
We analyzed the influence of using several techniques based on different visual
features (color, motion) on key frame extraction accuracy using TRIZ. Also, we devised an
algorithm to increase the accuracy by generating meaningful compact key frames using
accumulative optical flow with self-adaptive threshold inspired by the TRIZ inventive principles.
In our experiment, the extracted key frames were shown to be able to represent the whole video
and summarizes the important objects and salient events of the video. In addition, our proposed
algorithm was found to be able to extract key frames in video with slow or fast moving objects
and regions without prior knowledge of their shapes and sizes. On top of that, our proposed
algorithm achieved better compression rate in KTH data set in comparison with the-state-of-the-
art algorithms. In future work, we intend to utilise our proposed algorithm as a basis of the
development of advanced video processing systems to perform video summarization and
retrieval, and to increase the level of detection in the process of video forgery systems based on
fingerprint.
References
[1] A Nasreen. Keyframe Extraction from Videos - A survey. International Journal of Computer Science
and Communication Networks. 2013; 3(3): 194-8.
[2] W Hu, N Xie, L Li, X Zeng, S Maybank. A survey on Visual Content-Based Video Indexing and
Retrieva. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).
2011; 41(6): 797-819.
[3] MA Mizher, MC Ang. Hybrid Video Moving Objects Detection System. The Second Visual Informatics
International Seminar 2014(VIIS’14), in conjunction with the 7th MYREN Conference; 25-27 Nov.;
Kuala Lumpur, Malaysia. 2014: 46-52.
[4] C Sujatha, U Mudenagudi. A study on Keyframe Extraction Methods for Video Summary.
International Conference on Computational Intelligence and Communication Networks (CICN); 2011
7th Oct.: IEEE: 73-7.
[5] D Patel, S Upadhyay. Optical Flow Measurement Using Lucas Kanade Method. International Journal
of Computer Applications. 2013; 61(10).
[6] N Lu, J Wang, L Yang, QH Wu. Motion Detection Based On Accumulative Optical Flow and Double
Background Filtering. World Congress on Engineering (WCE); 2007 2-4 Jul.; London, U.K.: Citeseer:
602-7.
12. IJEECS ISSN: 2502-4752
A meaningful Compact Key Frames Extraction in Complex… (Manar A. Mizher)
829
[7] JC Sosa, R Rodríguez, VHG Ortega, R Hernández. Real-time Optical-flow Computation for Motion
Estimation under Varying Illumination Conditions. International Journal of Reconfigurable and
Embeded System (IJRES). 2012; 1(1): 25-36.
[8] I Chugh, R Gupta, R Kumar, P Sahay. Techniques for key frame extraction: Shot segmentation and
feature trajectory computation. 6th International Conference - Cloud System and Big Data
Engineering (Confluence); 2016 14-15 Jan: IEEE: 463-6.
[9] SCV, NK Narayanan. Key-frame Extraction by Analysis of Histograms of Video Frames Using
Statistical Methods. Procedia Computer Science. 2015; 70: 36-40.
[10] SD Thepade, A Tonge. An optimized Key Frame Extraction for Detection of near Duplicates in
Content Based Video Retrieval. International Conference onCommunications and Signal Processing
(ICCSP); 2014: IEEE: 1087-91.
[11] C Cao, Z Chen, G Xie, S Lei. Key Frame Extraction Based on Frame Blocks Differential
Accumulation. 24th Chinese Control and Decision Conference (CCDC); 2012: IEEE: 3621-5.
[12] MA Mizher, MC Ang, AA Mazhar, MA Mizher. A review of video falsifying techniques and video
forgery detection techniques. Int J Electronic security and digital forensics. 2017; 9(3): 191-209.
[13] R Ahuja, SS Bedi. Video Watermarking Scheme Based on Candidates I-frames for Copyright
Protection. Indonesian Journal of Electrical Engineering and Computer Science. 2017; 5(2): 391-400.
[14] Y Shi, H Yang, M Gong, X Liu, Y Xia. A Fast and Robust Key Frame Extraction Method for Video
Copyright Protection. Journal of Electrical and Computer Engineering. 2017; 2017, Article ID
1231794: 1-7.
[15] R Hamza, K Muhammad, Z Lv, F Titouna. Secure video summarization framework for personalized
wireless capsule endoscopy. Pervasive and Mobile Computing, publisher: Elsevier BV 2017.
[16] K Muhammad, M Sajjad, MY Lee, SW. Baik, Efficient visual attention driven framework for key
frames extraction from hysteroscopy videos. Biomedical Signal Processing and Control. 2017; 33:
161-8.
[17] MC Ang, DT Pham, AJ Soroka, KW Ng. PCB assembly optimisation using the Bees Algorithm
enhanced with TRIZ operators. 36th Annual Conference of the IEEE Industrial Electronics Society
(IECON-2010); 2010 7-10 Nov, 2010; Phoenix, Arizona, USA.
[18] SA Ahmad, DT Pham, KW Ng, MC Ang. TRIZ-inspired Asymmetrical Search Neighborhood in the
Bees Algorithm. The Asian Modelling Symposium (AMS2012): the 6th Asia International Conference
on Mathematical Modelling and Computer Simulation; 2012 28 May 2012 & 31 May 2012; Kuala
Lumpur, Malaysia & Bali, Indonesia: IEEE: 29-33.
[19] A Aghamohammadi, MC Ang, AS Prabuwono, M Mogharrebi, KW Ng. Enhancing an Automated
Inspection System on Printed Circuit Boards Using Affine-SIFT and TRIZ techniques. MCAIT 2013
conference; 2013; Shah Alam, Selangor, Malaysia: Springer Communications in Computer and
Information Science series (SCOPUS indexed).
[20] MC Ang, KW Ng, DT Pham, A Soroka. Simulations of PCB Assembly Optimisation Based on the
Bees Algorithm with TRIZ-inspired Operators. 3rd International Visual Informatics Conference (IVIC
2013); 2013 13-15 November 2013; Equatorial Hotel Bangi, Selangor, Malaysia: Springer
International Publishing, Switzerland, LNCS 8237, pp. 335-346. (SCOPUS indexed).
[21] MC Ang, KW Ng, SA Ahmad, ANA Wahab. Using TRIZ to generate ideas to solve the problem of the
shortage of ICT workers. Applied Mechanics and Materials 2014; 564: 733-9.
[22] PS Bajwa, D Mahto, Concepts. Tools and Techniques of Problem Solving through TRIZ: A review.
International Journal of Innovative Research in Science, Engineering and Technology (IJIRSET).
2013; 2(7): 3061-73.
[23] K Sahu, S Verma. Key Frame Extraction from Video Sequence: A Survey. International Research
Journal of Engineering and Technology (IRJET). 2017; 4(5): 1346-50.
[24] H Liu, L Pan, W Meng. Key frame Extraction From Online Video Based on Improved Frame
Difference Optimization. International Conference on Communication Technology (ICCT); 2012 9-11
Nov.; Chengdu, China: IEEE 940-4.
[25] J Majumdar, DKM, A Vijayendra. Design and Implementation of Video Shot Detection on Field
Programmable Gate Arrays. International Journal of Robotics and Automation (IJRA). 2013; 2(1): 17-
25.
[26] J Luo, C Papin, K Costello. Towards extracting semantically meaningful key frames from personal
video clips: from humans to computers. IEEE Transactions on Circuits and Systems for Video
Technology. 2009; 19(2): 289-301.