This document provides a literature review of augmented reality in medical applications. It discusses early concepts from the 1930s and summarizes seven categories of medical augmented reality technologies developed since the 1980s, including head-mounted displays, augmented optical devices like microscopes, and image overlay techniques. The review establishes relationships between subsets of work in the field and discusses remaining challenges for medical augmented reality research.
A NOVEL BACKGROUND SUBTRACTION ALGORITHM FOR PERSON TRACKING BASED ON K-NN csandit
Object tracking can be defined as the process of detecting an object of interest from a video scene and keeping track of its motion, orientation, occlusion etc. in order to extract useful
information. It is indeed a challenging problem and it’s an important task. Many researchers are getting attracted in the field of computer vision, specifically the field of object tracking in video surveillance. The main purpose of this paper is to give to the reader information of the present state of the art object tracking, together with presenting steps involved in Background Subtraction and their techniques. In related literature we found three main methods of object tracking: the first method is the optical flow; the second is related to the background subtraction, which is divided into two types presented in this paper, and the last one is temporal
differencing. We present a novel approach to background subtraction that compare a current frame with the background model that we have set before, so we can classified each pixel of the image as a foreground or a background element, then comes the tracking step to present our object of interest, which is a person, by his centroid. The tracking step is divided into two different methods, the surface method and the K-NN method, both are explained in the paper.Our proposed method is implemented and evaluated using CAVIAR database.
Satellite and Land Cover Image Classification using Deep Learningijtsrd
Satellite imagery is very significant for many applications including disaster response, law enforcement and environmental monitoring. These applications require the manual identification of objects and facilities in the imagery. Because the geographic area to be covered are great and the analysts available to conduct the searches are few, automation is required. The traditional object detection and classification algorithms are too inaccurate, takes a lot of time and unreliable to solve the problem. Deep learning is a family of machine learning algorithms that can be used for the automation of such tasks. It has achieved success in image classification by using convolutional neural networks. The problem of object and facility classification in satellite imagery is considered. The system is developed by using various facilities like Tensor Flow, XAMPP, FLASK and other various deep learning libraries. Roshni Rajendran | Liji Samuel "Satellite and Land Cover Image Classification using Deep Learning" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-5 , August 2020, URL: https://www.ijtsrd.com/papers/ijtsrd32912.pdf Paper Url :https://www.ijtsrd.com/computer-science/other/32912/satellite-and-land-cover-image-classification-using-deep-learning/roshni-rajendran
Using Mask R CNN to Isolate PV Panels from Background Object in Imagesijtsrd
Identifying foreground objects in an image is one of the most common operations used in image processing. In this work, Mask R CNN algorithm is used to identify solar photovoltaic PV panels in aerial images and create a mask that can be used to remove the background from the images. This allows processing the PV panels separately. Using ML to solve this problem can generate more accurate results in comparison to more traditional image processing techniques like using edge detection or Gaussian filtering especially in images where the view might not be easily separable from the objects of interest. The trained model was found to be successful in detecting the PV panels and selecting the pixels that belong to them while ignoring the background pixels. This kind of work can be useful in collecting information about PV installation present in aerial or satellite imagery, or in analyzing the health and integrity of PV modules in large scale installations e.g., in a solar power plant. The results show that this method is effective with a high potential for improved results if the model is trained using larger and more diverse datasets. Muhammet Sait | Atilla Erguzen | Erdal Erdal "Using Mask R-CNN to Isolate PV Panels from Background Object in Images" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-1 , December 2020, URL: https://www.ijtsrd.com/papers/ijtsrd38173.pdf Paper URL : https://www.ijtsrd.com/engineering/computer-engineering/38173/using-mask-rcnn-to-isolate-pv-panels-from-background-object-in-images/muhammet-sait
Satellite Image Classification with Deep Learning Surveyijtsrd
Satellite imagery is important for many applications including disaster response, law enforcement and environmental monitoring etc. These applications require the manual identification of objects in the imagery. Because the geographic area to be covered is very large and the analysts available to conduct the searches are few, thus an automation is required. Yet traditional object detection and classification algorithms are too inaccurate and unreliable to solve the problem. Deep learning is a part of broader family of machine learning methods that have shown promise for the automation of such tasks. It has achieved success in image understanding by means that of convolutional neural networks. The problem of object and facility recognition in satellite imagery is considered. The system consists of an ensemble of convolutional neural networks and additional neural networks that integrate satellite metadata with image features. Roshni Rajendran | Liji Samuel ""Satellite Image Classification with Deep Learning: Survey"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-2 , February 2020,
URL: https://www.ijtsrd.com/papers/ijtsrd30031.pdf
Paper Url : https://www.ijtsrd.com/engineering/computer-engineering/30031/satellite-image-classification-with-deep-learning-survey/roshni-rajendran
A NOVEL METHOD FOR PERSON TRACKING BASED K-NN : COMPARISON WITH SIFT AND MEAN...sipij
Object tracking can be defined as the process of detecting an object of interest from a video scene and
keeping track of its motion, orientation, occlusion etc. in order to extract useful information. It is indeed a
challenging problem and it’s an important task. Many researchers are getting attracted in the field of
computer vision, specifically the field of object tracking in video surveillance. The main purpose of this
paper is to give to the reader information of the present state of the art object tracking, together with
presenting steps involved in Background Subtraction and their techniques. In related literature we found
three main methods of object tracking: the first method is the optical flow; the second is related to the
background subtraction, which is divided into two types presented in this paper, then the temporal
differencing and the SIFT method and the last one is the mean shift method. We present a novel approach
to background subtraction that compare a current frame with the background model that we have set
before, so we can classified each pixel of the image as a foreground or a background element, then comes
the tracking step to present our object of interest, which is a person, by his centroid. The tracking step is
divided into two different methods, the surface method and the K-NN method, both are explained in the
paper. Our proposed method is implemented and evaluated using CAVIAR database.
Image restoration techniques covered such as denoising, deblurring and super-resolution for 3D images and models.
From classical computer vision techniques to contemporary deep learning based processing for both ordered and unordered point clouds, depth maps and meshes.
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to record depth maps and the drop points are recorded by a square infrared induction module. Firstly, convolutional neural networks are made use of to put the data obtained from depth maps in and get the prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to trainthe networks of different structure, and a network structure that could provide high enough accuracy for drop point prediction is established. The network model and parameters are modified to improve the accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a test group. The prediction results of test group reflect that the prediction algorithm effectively improves the accuracy of human motion perception.
A NOVEL BACKGROUND SUBTRACTION ALGORITHM FOR PERSON TRACKING BASED ON K-NN csandit
Object tracking can be defined as the process of detecting an object of interest from a video scene and keeping track of its motion, orientation, occlusion etc. in order to extract useful
information. It is indeed a challenging problem and it’s an important task. Many researchers are getting attracted in the field of computer vision, specifically the field of object tracking in video surveillance. The main purpose of this paper is to give to the reader information of the present state of the art object tracking, together with presenting steps involved in Background Subtraction and their techniques. In related literature we found three main methods of object tracking: the first method is the optical flow; the second is related to the background subtraction, which is divided into two types presented in this paper, and the last one is temporal
differencing. We present a novel approach to background subtraction that compare a current frame with the background model that we have set before, so we can classified each pixel of the image as a foreground or a background element, then comes the tracking step to present our object of interest, which is a person, by his centroid. The tracking step is divided into two different methods, the surface method and the K-NN method, both are explained in the paper.Our proposed method is implemented and evaluated using CAVIAR database.
Satellite and Land Cover Image Classification using Deep Learningijtsrd
Satellite imagery is very significant for many applications including disaster response, law enforcement and environmental monitoring. These applications require the manual identification of objects and facilities in the imagery. Because the geographic area to be covered are great and the analysts available to conduct the searches are few, automation is required. The traditional object detection and classification algorithms are too inaccurate, takes a lot of time and unreliable to solve the problem. Deep learning is a family of machine learning algorithms that can be used for the automation of such tasks. It has achieved success in image classification by using convolutional neural networks. The problem of object and facility classification in satellite imagery is considered. The system is developed by using various facilities like Tensor Flow, XAMPP, FLASK and other various deep learning libraries. Roshni Rajendran | Liji Samuel "Satellite and Land Cover Image Classification using Deep Learning" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-5 , August 2020, URL: https://www.ijtsrd.com/papers/ijtsrd32912.pdf Paper Url :https://www.ijtsrd.com/computer-science/other/32912/satellite-and-land-cover-image-classification-using-deep-learning/roshni-rajendran
Using Mask R CNN to Isolate PV Panels from Background Object in Imagesijtsrd
Identifying foreground objects in an image is one of the most common operations used in image processing. In this work, Mask R CNN algorithm is used to identify solar photovoltaic PV panels in aerial images and create a mask that can be used to remove the background from the images. This allows processing the PV panels separately. Using ML to solve this problem can generate more accurate results in comparison to more traditional image processing techniques like using edge detection or Gaussian filtering especially in images where the view might not be easily separable from the objects of interest. The trained model was found to be successful in detecting the PV panels and selecting the pixels that belong to them while ignoring the background pixels. This kind of work can be useful in collecting information about PV installation present in aerial or satellite imagery, or in analyzing the health and integrity of PV modules in large scale installations e.g., in a solar power plant. The results show that this method is effective with a high potential for improved results if the model is trained using larger and more diverse datasets. Muhammet Sait | Atilla Erguzen | Erdal Erdal "Using Mask R-CNN to Isolate PV Panels from Background Object in Images" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-1 , December 2020, URL: https://www.ijtsrd.com/papers/ijtsrd38173.pdf Paper URL : https://www.ijtsrd.com/engineering/computer-engineering/38173/using-mask-rcnn-to-isolate-pv-panels-from-background-object-in-images/muhammet-sait
Satellite Image Classification with Deep Learning Surveyijtsrd
Satellite imagery is important for many applications including disaster response, law enforcement and environmental monitoring etc. These applications require the manual identification of objects in the imagery. Because the geographic area to be covered is very large and the analysts available to conduct the searches are few, thus an automation is required. Yet traditional object detection and classification algorithms are too inaccurate and unreliable to solve the problem. Deep learning is a part of broader family of machine learning methods that have shown promise for the automation of such tasks. It has achieved success in image understanding by means that of convolutional neural networks. The problem of object and facility recognition in satellite imagery is considered. The system consists of an ensemble of convolutional neural networks and additional neural networks that integrate satellite metadata with image features. Roshni Rajendran | Liji Samuel ""Satellite Image Classification with Deep Learning: Survey"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-2 , February 2020,
URL: https://www.ijtsrd.com/papers/ijtsrd30031.pdf
Paper Url : https://www.ijtsrd.com/engineering/computer-engineering/30031/satellite-image-classification-with-deep-learning-survey/roshni-rajendran
A NOVEL METHOD FOR PERSON TRACKING BASED K-NN : COMPARISON WITH SIFT AND MEAN...sipij
Object tracking can be defined as the process of detecting an object of interest from a video scene and
keeping track of its motion, orientation, occlusion etc. in order to extract useful information. It is indeed a
challenging problem and it’s an important task. Many researchers are getting attracted in the field of
computer vision, specifically the field of object tracking in video surveillance. The main purpose of this
paper is to give to the reader information of the present state of the art object tracking, together with
presenting steps involved in Background Subtraction and their techniques. In related literature we found
three main methods of object tracking: the first method is the optical flow; the second is related to the
background subtraction, which is divided into two types presented in this paper, then the temporal
differencing and the SIFT method and the last one is the mean shift method. We present a novel approach
to background subtraction that compare a current frame with the background model that we have set
before, so we can classified each pixel of the image as a foreground or a background element, then comes
the tracking step to present our object of interest, which is a person, by his centroid. The tracking step is
divided into two different methods, the surface method and the K-NN method, both are explained in the
paper. Our proposed method is implemented and evaluated using CAVIAR database.
Image restoration techniques covered such as denoising, deblurring and super-resolution for 3D images and models.
From classical computer vision techniques to contemporary deep learning based processing for both ordered and unordered point clouds, depth maps and meshes.
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to record depth maps and the drop points are recorded by a square infrared induction module. Firstly, convolutional neural networks are made use of to put the data obtained from depth maps in and get the prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to trainthe networks of different structure, and a network structure that could provide high enough accuracy for drop point prediction is established. The network model and parameters are modified to improve the accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a test group. The prediction results of test group reflect that the prediction algorithm effectively improves the accuracy of human motion perception.
Maximizing Strength of Digital Watermarks Using Fuzzy Logicsipij
In this paper, we propose a novel digital watermarking scheme in DCT domain based fuzzy inference system and the human visual system to adapt the embedding strength of different blocks. Firstly, the original image is divided into some 8×8 blocks, and then fuzzy inference system according to different textural features and luminance of each block decide adaptively different embedding strengths. The watermark detection adopts correlation technology. Experimental results show that the proposed scheme has good imperceptibility and high robustness to common image processing operators.
Proposed Multi-object Tracking Algorithm Using Sobel Edge Detection operatorQUESTJOURNAL
ABSTRACT:Tracking of moving objects that is called video tracking is used for measuring motion parameters and obtaining a visual record of the moving objects, it is an important area of application in image processing. In general there are two different approaches to obtain object tracking: the first is Recognition-based Tracking, and the second is the Motion-based Tracking. Video tracking system raises a wide possibility in today’s society. This system is used in various applications such as military, security, monitoring, robotic, and nowadays in dayto-day applications. However the video tracking systems still have many open problems and various research activities in a video tracking system are explores. This paper presents an algorithm for video tracking of any moving targets with the uses of contour based detection technique that depends on the sobel operator. The proposed system is suitable for indoor and outdoor applications. Our approach has the advantage of extending the applicability of tracking system and also, as presented here improves the performance of the tracker making feasible high frame rate video tracking. The goal of the tracking system is to analyze the video frames and estimate the position of a part of the input video frame (usually a moving object), our approach can detect, tracked any object more than one object and calculate the position of the moving objects. Therefore, the aim of this paper is to construct a motion tracking system for moving objects. Where, at the end of this paper, the detail outcome and result are discussed using experiments results of the proposed technique
IEEE CASE 2016 On Avoiding Moving Objects for Indoor Autonomous QuadrotorsPeter SHIN
Abstract - A mini quadrotor can be used in many applica- tions, such as indoor airborne surveillance, payload delivery, and warehouse monitoring. In these applications, vision-based autonomous navigation is one of the most interesting research topics because precise navigation can be implemented based on vision analysis. However, pixel-based vision analysis approaches require a high-powered computer, which is inappropriate to be attached to a small indoor quadrotor. This paper proposes a method called the Motion-vector-based Moving Objects Detec- tion. This method detects and avoids obstacles using stereo motion vectors instead of individual pixels, thereby substan- tially reducing the data processing requirement. Although this method can also be used in the avoidance of stationary obstacles by taking into account the ego-motion of the quadrotor, this paper primarily focuses on providing our empirical verification on the real-time avoidance of moving objects.
iNEDI - Accuracy Improvements and Artifacts Removal in Edge Based Image Inter...Tecnick.com LTD
In this paper we analyse the problem of general purpose image upscaling that preserves edge features and
natural appearance and we present the results of subjective and objective evaluation of images interpolated
using different algorithms. In particular, we consider the well-known NEDI (New Edge Directed Interpolation,
Li and Orchard, 2001) method, showing that by modifying it in order to reduce numerical instability and
making the region used to estimate the low resolution covariance adaptive, it is possible to obtain relevant
improvements in the interpolation quality. The implementation of the new algorithm (iNEDI, improved New
Edge Directed Interpolation), even if computationally heavy (as the Li and Orchard’s method), obtained, in
both subjective and objective tests, quality scores that are notably higher than those obtained with NEDI and
other methods presented in the literature.
Multimodal RGB-D+RF-based sensing for human movement analysisPetteriTeikariPhD
Combining RGB-D based computer vision with commodity Wifi for pose estimation and human movement analysis for action recognition.
Think of applications especially in healthcare settings, where existing Wifi Access Point already exist and adding USB Wifi dongles to Raspberry Pi (or dedicated chips) is a very easy way to create "operational awareness" of all your patients.
Alternative download link:
https://www.dropbox.com/s/awkqqfhibesjcb9/multimodal_remote_MovementSensing.pdf?dl=0
Practical steps for non-machine learners on how to prepare your medical image dataset for deep learning modelling.
Here we use a fundus image dataset as an example that might have controls (healthy eyes) and glaucomatous fundus images with three different severities. In glaucoma, the optic disc is of a special interest so we want to annotate that from the images using a bounding box to help the deep learning training.
Deep Learning for Biomedical Unstructured Time SeriesPetteriTeikariPhD
1D Convolutional neural networks (CNNs) for time series analysis, and inspiration from beyond biomedical field. Short intro for various different steps involved in Time Series Analysis including outlier detection, imputation, denoising, segmentation, classification and forecasting.
Available also from:
https://www.dropbox.com/s/cql2jhrt5mdyxne/timeSeries_deepLearning.pdf?dl=0
Human motion is fundamental to understanding behaviour. In spite of advancement on single image 3 Dimensional pose and estimation of shapes, current video-based state of the art methods unsuccessful to produce precise and motion of natural sequences due to inefficiency of ground-truth 3 Dimensional motion data for training. Recognition of Human action for programmed video surveillance applications is an interesting but forbidding task especially if the videos are captured in an unpleasant lighting environment. It is a Spatial-temporal feature-based correlation filter, for concurrent observation and identification of numerous human actions in a little-light environment. Estimated the presentation of a proposed filter with immense experimentation on night-time action datasets. Tentative results demonstrate the potency of the merging schemes for vigorous action recognition in a significantly low light environment.
This paper represents a survey of various methods of video surveillance system which improves the security. The aim of this paper is to review of various moving object detection technics. This paper focuses on detection of moving objects in video surveillance system. Moving body detection is first important task for any video surveillance system. Detection of moving object is a challenging task. Tracking is required in higher level applications that require the location and shape of object in every frame. In this survey,paper described about optical flow method, Background subtraction, frame differencing to detect moving object. It also described tracking method based on Morphology technique.
Keywords -- Frame separation, Pre-processing, Object detection using frame difference, Optical flow,
Temporal Differencing and background subtraction. Object tracking
Moving Object Detection for Video SurveillanceIJMER
Video surveillance has long been in use to monitor security sensitive areas such as banks, department stores, highways, crowded public places and borders. The advance in computing power, availability of large-capacity storage devices and high speed network infrastructure paved the way for cheaper, multi sensor video surveillance systems. Traditionally, the video outputs are processed online by human operators and are usually saved to tapes for later use only after a forensic event. The increase in the number of cameras in ordinary surveillance systems overloaded both the human operators and the storage devices with high volumes of data and made it infeasible to ensure proper monitoring of sensitive areas for long times. In order to filter out redundant information generated by an array of cameras, and increase the response time to forensic events, assisting the human operators with identification of important events in video by the use of “smart” video surveillance systems has become a critical requirement. The making of video surveillance systems “smart” requires fast, reliable and robust algorithms for moving object detection, classification, tracking and activity analysis.
Possible future avenues for ophthalmic imaging combining advanced techniques and deep learning. "Bubbling under the surface, and inspiration from ‘bioimaging’ in general"
Purkinje imaging for crystalline lens density measurementPetteriTeikariPhD
Brief introduction for the non-invasive, inexpensive and fast Purkinje image -based method for measuring the spectral transmittance of the human crystalline lens density in vivo.
Alternative download link:
https://www.dropbox.com/s/588y7epy13n34xo/purkinje_imaging.pdf?dl=0
VIDEO SEGMENTATION FOR MOVING OBJECT DETECTION USING LOCAL CHANGE & ENTROPY B...csandit
Motion detection and object segmentation are an important research area of image-video
processing and computer vision. The technique and mathematical modeling used to detect and
segment region of interest (ROI) objects comprise the algorithmic modules of various high-level
techniques in video analysis, object extraction, classification, and recognition. The detection of
moving object is significant in many tasks, such as video surveillance & moving object tracking.
The design of a video surveillance system is directed on involuntary identification of events of
interest, especially on tracking and on classification of moving objects. An entropy based realtime
adaptive non-parametric window thresholding algorithm for change detection is
anticipated in this research. Based on the approximation of the value of scatter of sections of
change in a difference image, a threshold of every image block is calculated discriminatively
using entropy structure, and then the global threshold is attained by averaging all thresholds for
image blocks of the frame. The block threshold is calculated contrarily for regions of change
and background. Investigational results show the proposed thresholding algorithm
accomplishes well for change detection with high efficiency.
Using physics-based OCT Monte Carlo simulation and wave optics models for synthesising new OCT volumes for ophthalmic deep learning.
Alternative download link:
https://www.dropbox.com/s/ax15qy47yi76eex/OCT_MonteCarlo.pdf?dl=0
Time-resolved biomedical sensing through scattering mediumPetteriTeikariPhD
Time-resolved biomedical sensing through scattering medium | Case study with pupillometry through closed eyelids for neurological monitoring
Download link: https://www.dropbox.com/s/x0f5q6cz5ax33s4/timeResolvedSensing.pdf?dl=0
Cloud Architecture Patterns for Mere Mortals - Bill Wilder - Vermont Code Cam...Bill Wilder
How do you design applications for the cloud so that they will be scalable and reliable? In this talk, we will explain several architectural patterns which are popular for cloud computing: we will look at the need for the patterns generally, then look concretely at how you might realize them using capabilities of the Windows Azure Platform. CQRS, NoSQL, Sharding, and a few smaller patterns will be considered.
Presented by Bill Wilder at Vermont Code Camp III on Saturday September 10, 2011. http://blog.codingoutloud.com/2011/09/12/vermont-code-camp-iii/
Maximizing Strength of Digital Watermarks Using Fuzzy Logicsipij
In this paper, we propose a novel digital watermarking scheme in DCT domain based fuzzy inference system and the human visual system to adapt the embedding strength of different blocks. Firstly, the original image is divided into some 8×8 blocks, and then fuzzy inference system according to different textural features and luminance of each block decide adaptively different embedding strengths. The watermark detection adopts correlation technology. Experimental results show that the proposed scheme has good imperceptibility and high robustness to common image processing operators.
Proposed Multi-object Tracking Algorithm Using Sobel Edge Detection operatorQUESTJOURNAL
ABSTRACT:Tracking of moving objects that is called video tracking is used for measuring motion parameters and obtaining a visual record of the moving objects, it is an important area of application in image processing. In general there are two different approaches to obtain object tracking: the first is Recognition-based Tracking, and the second is the Motion-based Tracking. Video tracking system raises a wide possibility in today’s society. This system is used in various applications such as military, security, monitoring, robotic, and nowadays in dayto-day applications. However the video tracking systems still have many open problems and various research activities in a video tracking system are explores. This paper presents an algorithm for video tracking of any moving targets with the uses of contour based detection technique that depends on the sobel operator. The proposed system is suitable for indoor and outdoor applications. Our approach has the advantage of extending the applicability of tracking system and also, as presented here improves the performance of the tracker making feasible high frame rate video tracking. The goal of the tracking system is to analyze the video frames and estimate the position of a part of the input video frame (usually a moving object), our approach can detect, tracked any object more than one object and calculate the position of the moving objects. Therefore, the aim of this paper is to construct a motion tracking system for moving objects. Where, at the end of this paper, the detail outcome and result are discussed using experiments results of the proposed technique
IEEE CASE 2016 On Avoiding Moving Objects for Indoor Autonomous QuadrotorsPeter SHIN
Abstract - A mini quadrotor can be used in many applica- tions, such as indoor airborne surveillance, payload delivery, and warehouse monitoring. In these applications, vision-based autonomous navigation is one of the most interesting research topics because precise navigation can be implemented based on vision analysis. However, pixel-based vision analysis approaches require a high-powered computer, which is inappropriate to be attached to a small indoor quadrotor. This paper proposes a method called the Motion-vector-based Moving Objects Detec- tion. This method detects and avoids obstacles using stereo motion vectors instead of individual pixels, thereby substan- tially reducing the data processing requirement. Although this method can also be used in the avoidance of stationary obstacles by taking into account the ego-motion of the quadrotor, this paper primarily focuses on providing our empirical verification on the real-time avoidance of moving objects.
iNEDI - Accuracy Improvements and Artifacts Removal in Edge Based Image Inter...Tecnick.com LTD
In this paper we analyse the problem of general purpose image upscaling that preserves edge features and
natural appearance and we present the results of subjective and objective evaluation of images interpolated
using different algorithms. In particular, we consider the well-known NEDI (New Edge Directed Interpolation,
Li and Orchard, 2001) method, showing that by modifying it in order to reduce numerical instability and
making the region used to estimate the low resolution covariance adaptive, it is possible to obtain relevant
improvements in the interpolation quality. The implementation of the new algorithm (iNEDI, improved New
Edge Directed Interpolation), even if computationally heavy (as the Li and Orchard’s method), obtained, in
both subjective and objective tests, quality scores that are notably higher than those obtained with NEDI and
other methods presented in the literature.
Multimodal RGB-D+RF-based sensing for human movement analysisPetteriTeikariPhD
Combining RGB-D based computer vision with commodity Wifi for pose estimation and human movement analysis for action recognition.
Think of applications especially in healthcare settings, where existing Wifi Access Point already exist and adding USB Wifi dongles to Raspberry Pi (or dedicated chips) is a very easy way to create "operational awareness" of all your patients.
Alternative download link:
https://www.dropbox.com/s/awkqqfhibesjcb9/multimodal_remote_MovementSensing.pdf?dl=0
Practical steps for non-machine learners on how to prepare your medical image dataset for deep learning modelling.
Here we use a fundus image dataset as an example that might have controls (healthy eyes) and glaucomatous fundus images with three different severities. In glaucoma, the optic disc is of a special interest so we want to annotate that from the images using a bounding box to help the deep learning training.
Deep Learning for Biomedical Unstructured Time SeriesPetteriTeikariPhD
1D Convolutional neural networks (CNNs) for time series analysis, and inspiration from beyond biomedical field. Short intro for various different steps involved in Time Series Analysis including outlier detection, imputation, denoising, segmentation, classification and forecasting.
Available also from:
https://www.dropbox.com/s/cql2jhrt5mdyxne/timeSeries_deepLearning.pdf?dl=0
Human motion is fundamental to understanding behaviour. In spite of advancement on single image 3 Dimensional pose and estimation of shapes, current video-based state of the art methods unsuccessful to produce precise and motion of natural sequences due to inefficiency of ground-truth 3 Dimensional motion data for training. Recognition of Human action for programmed video surveillance applications is an interesting but forbidding task especially if the videos are captured in an unpleasant lighting environment. It is a Spatial-temporal feature-based correlation filter, for concurrent observation and identification of numerous human actions in a little-light environment. Estimated the presentation of a proposed filter with immense experimentation on night-time action datasets. Tentative results demonstrate the potency of the merging schemes for vigorous action recognition in a significantly low light environment.
This paper represents a survey of various methods of video surveillance system which improves the security. The aim of this paper is to review of various moving object detection technics. This paper focuses on detection of moving objects in video surveillance system. Moving body detection is first important task for any video surveillance system. Detection of moving object is a challenging task. Tracking is required in higher level applications that require the location and shape of object in every frame. In this survey,paper described about optical flow method, Background subtraction, frame differencing to detect moving object. It also described tracking method based on Morphology technique.
Keywords -- Frame separation, Pre-processing, Object detection using frame difference, Optical flow,
Temporal Differencing and background subtraction. Object tracking
Moving Object Detection for Video SurveillanceIJMER
Video surveillance has long been in use to monitor security sensitive areas such as banks, department stores, highways, crowded public places and borders. The advance in computing power, availability of large-capacity storage devices and high speed network infrastructure paved the way for cheaper, multi sensor video surveillance systems. Traditionally, the video outputs are processed online by human operators and are usually saved to tapes for later use only after a forensic event. The increase in the number of cameras in ordinary surveillance systems overloaded both the human operators and the storage devices with high volumes of data and made it infeasible to ensure proper monitoring of sensitive areas for long times. In order to filter out redundant information generated by an array of cameras, and increase the response time to forensic events, assisting the human operators with identification of important events in video by the use of “smart” video surveillance systems has become a critical requirement. The making of video surveillance systems “smart” requires fast, reliable and robust algorithms for moving object detection, classification, tracking and activity analysis.
Possible future avenues for ophthalmic imaging combining advanced techniques and deep learning. "Bubbling under the surface, and inspiration from ‘bioimaging’ in general"
Purkinje imaging for crystalline lens density measurementPetteriTeikariPhD
Brief introduction for the non-invasive, inexpensive and fast Purkinje image -based method for measuring the spectral transmittance of the human crystalline lens density in vivo.
Alternative download link:
https://www.dropbox.com/s/588y7epy13n34xo/purkinje_imaging.pdf?dl=0
VIDEO SEGMENTATION FOR MOVING OBJECT DETECTION USING LOCAL CHANGE & ENTROPY B...csandit
Motion detection and object segmentation are an important research area of image-video
processing and computer vision. The technique and mathematical modeling used to detect and
segment region of interest (ROI) objects comprise the algorithmic modules of various high-level
techniques in video analysis, object extraction, classification, and recognition. The detection of
moving object is significant in many tasks, such as video surveillance & moving object tracking.
The design of a video surveillance system is directed on involuntary identification of events of
interest, especially on tracking and on classification of moving objects. An entropy based realtime
adaptive non-parametric window thresholding algorithm for change detection is
anticipated in this research. Based on the approximation of the value of scatter of sections of
change in a difference image, a threshold of every image block is calculated discriminatively
using entropy structure, and then the global threshold is attained by averaging all thresholds for
image blocks of the frame. The block threshold is calculated contrarily for regions of change
and background. Investigational results show the proposed thresholding algorithm
accomplishes well for change detection with high efficiency.
Using physics-based OCT Monte Carlo simulation and wave optics models for synthesising new OCT volumes for ophthalmic deep learning.
Alternative download link:
https://www.dropbox.com/s/ax15qy47yi76eex/OCT_MonteCarlo.pdf?dl=0
Time-resolved biomedical sensing through scattering mediumPetteriTeikariPhD
Time-resolved biomedical sensing through scattering medium | Case study with pupillometry through closed eyelids for neurological monitoring
Download link: https://www.dropbox.com/s/x0f5q6cz5ax33s4/timeResolvedSensing.pdf?dl=0
Cloud Architecture Patterns for Mere Mortals - Bill Wilder - Vermont Code Cam...Bill Wilder
How do you design applications for the cloud so that they will be scalable and reliable? In this talk, we will explain several architectural patterns which are popular for cloud computing: we will look at the need for the patterns generally, then look concretely at how you might realize them using capabilities of the Windows Azure Platform. CQRS, NoSQL, Sharding, and a few smaller patterns will be considered.
Presented by Bill Wilder at Vermont Code Camp III on Saturday September 10, 2011. http://blog.codingoutloud.com/2011/09/12/vermont-code-camp-iii/
Optical Coherence Tomography: Technology and applications for neuroimagingManish Kumar
Optical coherence tomography (OCT) is an emerging imaging technology with applications in biology, medicine, and materials investigations. Attractive features include high cellular-level resolution, real-time acquisition rates, and spectroscopic feature extraction in a compact noninvasive instrument. OCT can perform ‘‘optical biopsies’’ of tissue, producing images approaching the resolution of histology without having to resect and histologically process tissue specimens for characterization and diagnosis.
Computer Vision Based 3D Reconstruction : A ReviewIJECEIAES
3D reconstruction are used in many fields starts from the object reconstruction such as site, and cultural artifacts in both ground and under the sea levels. The scientist are beneficial for these task in order to learn and keep the environment into 3D data due to the extinction. In this paper explained vision setup that is commonly used such as single camera, stereo camera, Kinect / Structured Light/ Time of Flight camera and fusion approach. The prior works also explained how the 3D reconstruction perform in many fields and using various algorithms.
Virtual reality: is the term that applies to computer simulated environment that can stimulate the physical presence in place of real world or imaginary world. All components of VR application and interrelations between them are thoroughly examined: input devices, output devices and software.This Paperwill show how to use Virtual Reality to increase the business productivity, application in health sector, entertainment industry, military sector and other sectors. The paper explain different types of virtual reality, architecture of the haptic devices methodologies of the devices, algorithm and conclusion of the research.
Case Study of Augmented Reality Applications in Medical Fieldijtsrd
Computerized applications are used at greater extent that helps the training in medical field. Augmented reality applications possess an interactive virtual layer on top of reality. The use of augmented reality applications acts as a boon to medical education because they combine digital parts with the practical learning environment. The aim of this research is to investigate the scope of augmented reality applications in medical professionals training [1] This technology is different from virtual reality, in which the user is immersed in a virtual world generated by the computer. The AR system brings the computer into the user world by augmenting the real environment with virtual objects. [2] In Augmented Reality, physical and artificial objects are mixed together in a hybrid space where the user can move without constraints. This paper aims to provide information of current technologies and benefits of augmented reality and to describe the benefits and open issues. [3] Vaishnavi D. Deolekar | Pratibha M. Deshmukh"Case Study of Augmented Reality Applications in Medical Field" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-4 , June 2018, URL: http://www.ijtsrd.com/papers/ijtsrd15714.pdf http://www.ijtsrd.com/medicine/other/15714/case-study-of-augmented-reality-applications-in-medical-field/vaishnavi-d-deolekar
Augmented Reality in Volumetric Medical Imaging Using Stereoscopic 3D Display ijcga
This paper is written about augmented reality in medicine. Medical imaging equipment (CT, PET, MRI) are produced 3D volumetric data, so using the stereoscopic 3D display, observer feels depth perception. The major factors about depth-Convergence, Accommodation, Relative size are tested. Convergence and Accommodation have affected depth perception but relative size is negligible.
A UGMENT R EALITY IN V OLUMETRIC M EDICAL I MAGING U SING S TEREOSCOPIC...ijcga
This paper is written about augment reality in medi
cine. Medical imaging equipment (CT, PET, MRI) are
produced 3D volumetric data, so using the stereosco
pic 3D display, observer feels depth perception. Th
e
major factors about depth-Convergence, Accommodatio
n, Relative size are tested. Convergence and
Accommodation have affected depth perception but re
lative size is negligibl
Motion Prediction Using Depth Information of Human Arm Based on Alexnetgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification
and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by
experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to
record depth maps and the drop points are recorded by a square infrared induction module. Firstly,
convolutional neural networks are made use of to put the data obtained from depth maps in and get the
prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to train
the networks of different structure, and a network structure that could provide high enough accuracy for
drop point prediction is established. The network model and parameters are modified to improve the
accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a
test group. The prediction results of test group reflect that the prediction algorithm effectively improves the
accuracy of human motion perception.
To support complex tasks in neurosurgery master-slave robotic systems with compact robotic forceps have been developed. Before these systems are inserted in the surgical room, it is necessary to ensure their robust and safe operation. Virtual Reality (VR) simulation may complement this development process by providing a software platform in where the robot control algorithms can be tested without damaging the expensive prototype. In recent years the increase in the computing power along with newer algorithms optimizations enables real time simulation with both visual and physical realism, allowing acquisition of kinematic data for surgical skills analysis, and synthetic data generation for training machine learning algorithms; ultimately interactive simulators provide a safe environment in where neurosurgeons can repeatedly practice without impacting the patient’s health. Despite state-of-the-art simulators, there is little work in simulation of robot-assisted microsurgery. In this talk we present the development of VR simulations for robotic neurosurgery focusing in the micro suturing task with haptic feedback, detailing the methods and current applications. This work was funded by ImPACT Program of Council for Science, Technology and Innovation, Cabinet Office, Government of Japan.
Cybernetics is an interdisciplinary study of regulatory systems, their structures, constraints and possibilities. Cybernetics was defined by Norbert Wiener in 1948 as “the scientific study of control and communication in living organism and the machine”. Cybernetics study includes but not limited to artificial learning, adaptation, cognition, convergence, Social Control, efficacy, efficiency, connectivity and communication [1]. It is known from science fiction; technically modified organism with exceptional skills called as cyborgs -it was originated from the term “cybernetic organism”. As a matter of fact, cyborgs that incorporates technical systems with living organism are already reality. For instance, smart machines that spontaneously operate to changing dynamic conditions, computer supported designs and fabrication based on magnetic tomography datasets or surface modifications for enhanced tissue integration that allowed major development in cybernetics technology [2,3].
A Simulation Method of Soft Tissue Cutting In Virtual Environment with HapticsIJERA Editor
Currently, virtual simulation has an increasing role in the medical field. Now virtual surgery simulation has been largely explored in medical field. Virtual surgery is a good complement to traditional Surgical Training. Modeling effects of soft tissue during cutting is quite complex, hence the concept of virtuality is used to develop realistic surgical instruments for providing exact force feedback to the surgeon during surgical operation and simulation of soft tissue processes. Scalpel is a basic instrument required for soft tissue simulation. Hence we will design a virtual organ to cut by using Scalpel in Haptic Environment.
Medical Image Fusion Using Discrete Wavelet TransformIJERA Editor
Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multimodal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. The domain where image fusion is readily used nowadays is in medical diagnostics to fuse medical images such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging) and MRA. This paper aims to present a new algorithm to improve the quality of multimodality medical image fusion using Discrete Wavelet Transform (DWT) approach. Discrete Wavelet transform has been implemented using different fusion techniques including pixel averaging, maximum minimum and minimum maximum methods for medical image fusion. Performance of fusion is calculated on the basis of PSNR, MSE and the total processing time and the results demonstrate the effectiveness of fusion scheme based on wavelet transform.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
9. SIELHORST et al.: ADVANCED MEDICAL DISPLAYS: A LITERATURE REVIEW OF AUGMENTED REALITY 459
Fig. 15. Simplified relationship between technology and potential benefits. Grey color indicates in situ visualization.
III. POTENTIAL BENEFITS OF AR VISUALIZATION
The crucial question regarding new visualization paradigms
is “What can it do for us that established technology cannot?”.
AR provides an intuitive human computer interface. Since in-
tuition is difficult to measure for an evaluation we subdivide
the differences between AR and ordinary display technology
in this section into four phenomena: Image fusion, 3D interac-
tion, 3D visualization, and hand-eye coordination. Fig. 15 de-
picts a simplified relationship between these phenomena and
AR technology.
A. Extra Value From Image Fusion
Fusing registered images into the same display offers the best
of two modalities in the same view.
An extra value provided by this approach may be a better
understanding of the image by visualizing anatomical context
that has not been obvious before. This is the case for endoscopic
camera and ultrasound images, where each image corresponds
only to a small area. (See paragraph II.E.3)
Another example for additional value is displaying two phys-
ical properties in the same image that can only be seen in either
of the modalities. An example is the overlay of beta probe ac-
tivity. Wendler et al. [86] support doctors by augmenting pre-
viously measured activity emitted by radioactive tracers on the
live video of a laparoscopic camera. By this means, physicians
can directly relate the functional tissue information to the real
view showing the anatomy and instrument position.
A further advantage concerns the surgical workflow. Cur-
rently, each imaging device introduces another display into the
operating room [see Fig. 16(b)] thus the staff spends valuable
time on finding a useful arrangement of the displays. A single Fig. 16. Current minimally invasive spine surgery setup at “‘Chirurgische
Klinik’”, hospital of Ludwigs-Maximilian Universität München. (a) Action
display integrating all data could solve this issue. Each imaging takes place on a very different position than the endoscope display. (b) Each
device also introduces its own interaction hardware and graph- imaging device introduces another display.
ical user interface. A unified system could replace the inefficient
multitude of interaction systems.
the viewport on virtual objects. Changing the eye position rela-
tive to an object is a natural approach for 3D inspection.
B. Implicit 3D Interaction
3D user interfaces reveal their power only in tasks that cannot
Interaction with 3D data is a cumbersome task with 2D dis- be easily reduced to two dimensions, because 2D user interfaces
plays and 2D interfaces (cf. Bowman [87]). Currently, there is no benefit from simplification by dimension reduction and the fact
best practice for three-dimensional user interfaces as opposed that they are widespread. Recent work by Traub et al. [88] sug-
to 2D interfaces using the WIMP paradigm (windows, icons, gests that navigated implant screw placement is a task that can
menus, and pointing). benefit from 3D user interaction, as surgeons were able to per-
AR technology facilitates implicit viewpoint generation by form drilling experiments faster with in situ visualization com-
matching the viewport of the eye/endoscope on real objects to pared to a navigation system with a classic display. Although
Authorized licensed use limited to: Amal Jyothi College of Engineering. Downloaded on August 10,2010 at 09:29:29 UTC from IEEE Xplore. Restrictions apply.
10. 460 JOURNAL OF DISPLAY TECHNOLOGY, VOL. 4, NO. 4, DECEMBER 2008
the performance speed is probably not a valid metric for a med- can be prepared for the augmented reality system. Optical (in-
ical drilling task, the experiments indicate that the surgeons had frared) tracking systems are already in use in modern operating
a faster mental access to the spatial situation. rooms for intraoperative navigation. In orthopedics, trauma
surgery, and neurosurgery, which only require a rigid body reg-
C. 3D Visualization istration, available navigation systems proved to be sufficiently
Many augmented reality systems allow for stereoscopic data accurate. King et al. [29] proved in clinical studies to have
representation. Stereo disparity and motion parallax due to overall errors in the submillimeter range for their microscope
viewpoint changes (see Section III-B) can give a strong spatial based augmented reality system for neurosurgery. For the pose
impression of structures. determination of the real view infrared tracking is currently the
In digital subtraction angiography, stereoscopic displays can best choice. Only augmented flexible endoscopes have to use
help doctors to analyze the complex vessel structures [89]. Cal- different ways of tracking [see Section II-E2)].
vano et al. [90] report on positive effects of the stereoscopic As the last piece in the alignment chain of real and virtual
view provided by a stereo endoscope for in-utero surgery. The there is the patient registration. The transformation between
enhanced spatial perception may be also useful in other fields. image data and patient data in the tracking coordinate system
has to be computed. Two possibilities may apply:
D. Improved Hand-Eye Coordination 1) Rigid Patient Registration: Registration algorithms are
A differing position and orientation between image acquisi- well discussed in the community. Their integration into the sur-
tion and visualization may interfere with the hand-eye coordina- gical workflow requires mostly a trade off between simplicity,
tion of the operating person. This is a typical situation in mini- accuracy, and invasiveness.
mally invasive surgery [see Fig. 16(a)]. Hanna et al. [91] showed Registration of patient data with the AR system can be
that the position of an endoscope display has a significant im- performed with fiducials that are fixed on the skin or implanted
pact on the performance of a surgeon during a knotting task. [94]. These fiducials must be touched with a tracked pointer
Their experiments suggest the best positions of the display to for the registration process. Alternatively, the fiducials can be
be in front of the operator at the level of his or her hands. segmented in the images of a tracked endoscope rather than
Using in situ visualization, there is no offset between working touching them with a tracked pointer for usability reasons.
space and visualization. No mental transformation is necessary Whereas Stefansic et al. propose the direct linear transform
to convert the viewed objects to the hand coordinates. (DLT) to map the 3D locations of fiducials into their corre-
sponding 2D endoscope images [95], Feuerstein et al. suggest a
triangulation of automatically segmented fiducials from several
IV. CURRENT ISSUES
views [96], [97]. Baumhauer et al. study different methods for
We have presented different systems in this paper with their endoscope pose estimation based on navigation aids stuck onto
history and implications. In this section we present current lim- the prostate and propose to augment 3D transrectal ultrasound
iting factors for most of the presented types and approaches to data on the camera images [98]. Using this method, no external
solve them. tracking system is needed.
Especially for maxillofacial surgery, fiducials can be inte-
A. Registration, Tracking, and Calibration grated in a reproducibly fixed geometry [29]. For spine surgery,
The process of registration is the process of relating two or Thoranaghatte et al. try to attach an optical fiducial to the ver-
more data sets to each other in order to match their content. For tebrae and use the endoscope to track it in situ [99].
augmented reality the registration of real and virtual objects is Point-based registration is known to be a reliable solution in
a central piece of the technology. Maintz and Viergever [92] principle. However, the accuracy of a fiducial-based registration
give a general review about medical image registration and its varies on the number of fiducials and quality of measurement of
subclassification. each fiducial, but also on the spatial arrangement of the fidu-
In the AR community the term tracking refers to the pose esti- cials [100].
mation of objects in real time. The registration can be computed Another approach is to track the imaging device and register
using tracking data after an initial calibration step that provides the data to it. This procedure has the advantage that no fiducials
the registration for a certain pose. This is only true if the ob- have to be added to the patient while preserving high accuracy.
ject moves but does not change. Calibration of a system can be Grzeszczuk et al. [101] and Murphy [102] use a fluoroscope to
performed by computing the registration using known data sets, acquire intraoperative X-ray images and register them to dig-
e.g., measurements of a calibration object. Tuceryan et al. [93] itally reconstructed radiographs (DRR) created from preopera-
describe different calibration procedures that are necessary for tive CT. This 2D–3D image registration procedure, which could
video augmentation of tracked objects. These include image dis- also be used in principle for an AR system, has the advantage
tortion determination, camera calibration, and object-to-fiducial that no fiducials have to be added to the patient while keeping
calibration. high accuracy. By also tracking the C-arm, its subsequent mo-
Tracking technology is one of the bottlenecks for augmented tions can be updated in the registered CT data set.
reality in general [10]. As an exception, this is quite different Feuerstein et al. [97], [103] augment 3D images of an intra-
for medical augmented reality. In medical AR, the working operative flat panel C-arm into a laparoscope. This approach is
volume and hence the augmented space is indoors, predefined, sometimes also called registration-free [104], because doctors
and small. Therefore, the environment, i.e., the operating room, need not perform a registration procedure. As a drawback such
Authorized licensed use limited to: Amal Jyothi College of Engineering. Downloaded on August 10,2010 at 09:29:29 UTC from IEEE Xplore. Restrictions apply.
11. SIELHORST et al.: ADVANCED MEDICAL DISPLAYS: A LITERATURE REVIEW OF AUGMENTED REALITY 461
an intrinsic registration is only valid as long as the patient does on the size of the object the latency should be ideally as little as 50
not move after imaging. ms. Little additions in latencies may cause big decreases of per-
Grimson et al. [42] follow a completely different approach by formance. According to their experiment and theory a latency of
matching surface data of a laser range scanner to CT data of the 175 ms can result in 1200 ms slower grasping than for immediate
head. For sinus surgery, Burschka et al. propose to reconstruct feedback. This depends on the difficulty of the task. Their exper-
3D structures using a non-tracked monocular endoscopic camera iments showed also a significantly larger percentage of error in
and register them to a preoperative CT data set [105]. For spine their performance with 180 ms latency than with a system that had
surgery, Wengert et al. describe a system that uses a tracked en- only 80 ms latency. The experiments of our group [117] confirm
doscope to achieve the photogrammetric reconstruction of the these findings for a medical AR system. Therefore an optimal
surgical scene and its registration to preoperative data [106]. system should feature data synchronization and short latency. We
2) Deformable Tissue: The approaches mentioned above proposed recently an easy and accurate way of measuring the la-
model the registration of a rigid transformation. This is useful tency in a video see-through system [118].
for the visualization before an intervention and for a visualiza-
tion of not deformed objects. The implicit assumption of a rigid C. Error Estimation
structure is correct for bones and tissue exposed to the same Tracking in medical AR is mostly fiducial-based because it
forces during registration and imaging, but not for soft tissue can guarantee a predictable quality of tracking, which is neces-
deformed by, e.g., respiration or heart beat. sary for the approval of a navigation system.
A well known example breaking this assumption is the brain For an estimation of the overall error calibration, registration
shift in open brain surgery. Maurer et al. [107] show clearly that and tracking errors have to be computed, propagated, and ac-
the deformation of the brain after opening the skull may result cumulated. Nicolau and colleagues [44] propose a registration
in misalignment of several millimeters. with error prediction for endoscopic augmentation. Fitzpatrick
There are three possible directions to handle deformable et al. [100] compute tracking based errors based on the spatial
anatomy. distribution of marker sets. Hoff et al. [18] predict the error for
1) Use very recent imaging data for a visualization that in- an HMD based navigation system.
cludes the deformation. Several groups use ultrasound im- An online error estimation is a desirable feature, since physi-
ages that are directly overlayed onto the endoscopic view cians have to rely on the visualized data. In current clinical prac-
[108]–[110]. tice, navigation systems stop their visualization in case a factor
2) Use very recent data to update a deformable model of the that decreases the accuracy is known to the system. Instead of
preoperative data. For instance Azar et al. [111] model and stopping the whole system it would be useful to estimate the re-
predict mechanical deformations of the breast. maining accuracy and visualize it, so that a surgeon can decide
3) Make sure that the same forces are exposed to the tissue. in critical moments, whether to carefully use the data or not.
Active breathing control is an example for compensating MacIntyre et al. [119] suggest in a non-medical setup to pre-
deformations due to respiration [112], [113]. dict the error empirically by an adaptive estimator. Our group
Baumhauer et al. [114] give a recent review on the perspec- [120] suggests a way of dynamically estimating the accuracy
tives and limitations in soft tissue surgery, particularly focusing for optical tracking modeling the physical situation. By inte-
on navigation and AR in endoscopy. grating the visibility of markers for each camera into the model,
B. Time Synchronization a multiple camera setup for reducing the line of sight problem
is possible, which ensures a desired level of accuracy. Nafis et
Time synchronization of tracking data and video images is an
al. [121] investigate the dynamic accuracy of electromagnetic
important issue for an augmented endoscope system. In the un-
(EM) tracking. The magnetic field in the tracking volume can be
synchronized case, data from different points of time would be
influenced by metallic objects, which can change the measure-
visualized. Holloway et al. [34] investigated the source of errors
ments significantly. Also the distance of the probe to the field
for augmented reality systems. The errors of time mismatch can
generator and its speed have a strong influence on the accuracy.
raise to be the highest error sources when the camera is moving.
Finally it is not enough to estimate the error, but the whole
To overcome this problem, Jacobs et al. [14] suggest methods to
system has to be validated (cf. Jannin et al. [122]). Standard-
visualize data from multiple input streams with different laten-
ized validation procedures have not been used to validate the
cies from only the same point of time. Sauer et al. [17] describe an
described systems in order to make the results comparable. The
augmented reality system that synchronizes tracking and video
validation of the overall accuracy of an AR system must include
data by hardware triggering. Their software waits for the slowest
the perception of the visualization. In the next section we dis-
component before the visualization is updated. For endoscopic
cuss the effect of misperception in spite of mathematically cor-
surgery, Vogt [115] also uses hardware triggering to synchronize
rect positions in visualizations.
tracking and video data by connecting the S-Video signal (PAL,
50 Hz) of the endoscope system to the synchronization card of
D. Visualization and Depth Perception
the tracking system, which can also be run at 50 Hz.
If virtual and real images do not show a relative lag it means The issue of wrong depth perception has been discussed as
that the images are consistent and there is no error due to a time early as 1992 when Bajura and colleagues [12] described their
shift. However there is still the visual offset to haptic senses. Ware system. When merging real and virtual images the relative posi-
et al. [116] investigated the effect of latency in a virtual environ- tion in depth may not be perceived correctly although all posi-
ment with a grasping experiment. They conclude that depending tions are computed correctly. When creating their first setup also
Authorized licensed use limited to: Amal Jyothi College of Engineering. Downloaded on August 10,2010 at 09:29:29 UTC from IEEE Xplore. Restrictions apply.
12. 462 JOURNAL OF DISPLAY TECHNOLOGY, VOL. 4, NO. 4, DECEMBER 2008
Edwards et al. [28] realized that “Experimentation with intra-op- suitable for complex spatial geometry, and the virtual window
erative graphics will be a major part of the continuation of the locally covers the real view. Lerotic et al. [128] suggest to super-
project”. Drascic and Milgram [123] provide an overview of per- impose contours of the real view on the virtual image for better
ceptual issues in augmented reality system. While many prob- depth perception.
lems of early systems have already been addressed, the issue of 2) Adaption: A wrong visual depth perception can be cor-
a correct depth visualization remains unsolved. Depth cues are rected by learning if another sense can disambiguate the spatial
physical facts that the human visual system can use in order to re- constellation. The sense of proprioception provides exact infor-
fine the spatial model of the environment. These include visual mation about the position of the human body. Biocca and Rolland
stimuli such as shading but also muscular stimuli such as accom- [129] set up an experiment where the point of view of each subject
modation and convergence. Psychologists distinguish between a was repositioned with a video see-through HMD. The adaption
number of different depth cues. Cutting and Vishton review and time for hand-eye coordination is relatively short and the adap-
summarize psychologists’ research on nine of the most relevant tion is successful. Unfortunately, another adaption process is
depth cues [124] revealing the relevance of different depth cues started when the subject is exposed to the normal view again.
in comparison to each other. They identify interposition as the 3) Motion Sickness: In the worst case conflicting visual cues
most important depth cue even though it is only an ordinary qual- can cause reduced concentration, headache, nausea etc. These
ifier. This means that it can only reveal the order but not a rel- effects have been discussed in the virtual reality and psychology
ative or absolute distance. Stereo disparity and motion parallax community [130].
are the next strongest depth cues in the personal space of up to Modern theories state that the sickness is not caused by the
two meters distance in the named order. The visual system calcu- conflict of cues, but the absence of better information to keep
lates the spatial information together with the depth cues of rela- the body upright [131]. Therefore engineers should concentrate
tive size/density, accommodation, conversion, and areal perspec- on providing more information to the sense of balance (e.g., by
tive. Especially the latest one is hardly taken into account for the making the user sit, unobstructed peripheral view) rather than
space under 2 meters unless the subject is in fog or under water. reducing conflicting visual cues in order to avoid motion sick-
It is the very nature of AR to provide a view that does not rep- ness. However, motion sickness does not seem to play a big role
resent the present physical conditions while the visual system in AR: In a video see-through HMD based experiment with 20
expects natural behavior of its environment for correct depth surgeons we [117] found no indication of the above symptoms
perception. What happens if conflicting depth cues are present? even after an average performance time of 16 minutes. In the
The visual system weights the estimates according to its impor- experiment the subjects were asked to perform a pointing task
tance and personal experience [124]. while standing. The overall lag was reported to be 100 ms for
Conflicting cues could result into misperception, adaption, fast rendering visualizations. The remote field of view was not
and motion sickness. covered. For less immersive AR systems than this one based on
1) Misperception: If there are conflicting depth cues it means an HMD and for systems with similar properties motion sick-
that at least one depth cue is wrong. Since the visual system ness is therefore expected to be unlikely.
is weighting the depth cues together the overall estimate will
generally not be correct even though the computer generates E. Visualization and Data Representation
geometrically correct images. 3D voxel data cannot be displayed directly with an opaque
Especially optical augmentation provides different parame- value for each voxel as for 2D bitmaps. There are three major
ters for real and virtual images resulting in possibly incompat- ways of 3D data representation.
ible depth cues. The effect is described as ghost-like visualiza- 1) Slice Rendering: Slice rendering is the simplest way of
tion resembling to its unreal and confusing spatial relationship rendering. Only a slice of the whole volume is taken for visual-
to the real world. The visual system is quite sensitive to relative ization. Radiologists commonly examine CT or MRI data rep-
differences. resented by three orthogonal slices intersecting a certain point.
Current AR systems handle depth cues well that are based The main advantage of this visualization technique is the preva-
on geometry, as for instance relative size, motion parallax, and lence of this method in medicine and its simplicity. Another ad-
stereo disparity. Incorrect visualization of interposition between vantage that should not be underestimated is the fact that the
real and virtual objects has already been identified to be a serious visualization defines a plane. Since one degree of freedom is
issue by Bajura et al. [12]. It has been discussed in more detail fixed, distances in this plane can be perceived easily. Traub et
by Johnson [125] for augmentation in operating microscopes, al. [88] showed that slice representations as used in first gen-
Furmanski et al. [126] and Livingston et al. [127] for an op- eration navigation systems have superior capabilities in repre-
tical see-through, and by our group [117] for video see-through senting the precise position of a target point. They also found
HMDs. The type of AR display makes a difference since rel- that for finding a target point it can be more efficient to take a
ative brightness plays a role in depth perception and optical different representation of data.
see-through technology can only overlay brighter virtual images When two or three points of interest and their spatial relation-
on the real background. Opaque superimposition of virtual ob- ship are supposed to be displayed an oblique slice can be useful.
jects that are inside a real one is not recommended. Alternatives Without a tracked instrument however it is cumbersome to po-
can be a transparent overlay, wireframes, and a virtual window. sition such a plane.
Each possibility imposes a trade off: Transparent overlay re- The major drawback of slice rendering is that this visualiza-
duces the contrast of the virtual image, the wireframe is not tion does not show any data off the plane. This is not a constraint
Authorized licensed use limited to: Amal Jyothi College of Engineering. Downloaded on August 10,2010 at 09:29:29 UTC from IEEE Xplore. Restrictions apply.
13. SIELHORST et al.: ADVANCED MEDICAL DISPLAYS: A LITERATURE REVIEW OF AUGMENTED REALITY 463
for measuring visualizations in plane à la How far can I go with 3D displays in common. Bowman [87] gives a comprehensive
a drill? but optimizing questions like In which direction would introduction into 3D user interfaces and detailed information
a drill be furthest from critical tissue? why 3D interaction is difficult. The book gives general advice
2) Surface Rendering: Surface rendering shows transitions for creating new user interfaces. Reitinger et al. [136] suggest a
between structures. 3D user interface for liver planning. Even though the suggested
Often these transitions are segmented and converted to planning is done in pure virtual space the ideas apply to AR as
polygons. The desired tissue is segmented either manually, well. They use tracked instruments and a tracked glass plane
semi-automatically, or automatically depending on the image for defining points and planes in a tangible way. They combine
source and the desired tissue. The surface polygons of a tangible 3D interaction and classic 2D user interfaces in an ef-
segmented volume can be calculated by the marching cubes fective way.
algorithm [132]. Graphic cards offer hardware support for this Navab et al. [137] suggest a new paradigm for interaction
vertex based 3D data. This includes light effects based on the with 3D data. A virtual mirror is augmented into the scene. The
normals of the surfaces with only little extra computation time. physician has the possibility to explore the data from any side
Recently ray casting techniques became fast enough on using the mirror without giving up the registered view. Since the
graphic cards equipped with a programmable graphics pro- interaction uses a metaphor that has a very similar real counter-
cessing unit (GPU) [133]. As the surfaces need not be trans- part, no extra learning is expected for a user.
formed to polygons the images are smoother. They do not Apart from 2D/3D issues, standard 2D computer interfaces
suffer from holes due to discontinuities in the image and the such as mice are not suited for the OR because of sterility and er-
sampling is optimal for a specific viewing direction. Integration gonomic reasons. Fortunately, medical systems are highly spe-
of physical phenomena like refraction, reflexion, and shadows cialized on the therapy. Since a specialized application has a
are possible with this rendering technique. limited number of meaningful visualization modes the user in-
As a welcome side effect of surface rendering distances and terface can be highly specialized as well. Context aware systems
cutting points can be calculated when visualizing surfaces. can further reduce the degree of interaction. Automatic work-
The segmentation step is a major obstacle for this kind of flow recovery as suggested by Ahmadi et al. [138] could detect
visualization. Segmentation of image data is still considered a phases of the surgery and with this information the computer
hard problem with brisk research going on. Available solutions system could offer suitable information for each phase.
offer automatic segmentation only for limited number of struc-
tures. Manual and semiautomatic solutions can be time-con- V. PERSPECTIVE
suming or at least time-consuming to learn. The benefit from
such visualization using an interactive segmentation has to jus- After two decades of research on medical AR the basic con-
tify the extra work load on the team. cepts seem to be well understood and the enabling technologies
3) Volume Rendering: Direct volume rendering [134] cre- are now enough advanced to meet the basic requirements for a
ates the visualization by following rays from a certain viewpoint number of medical applications. We are encouraged by our clin-
through 3D data. Depending on the source of data and the in- ical partners to believe that medical AR systems and solutions
tended visualization different functions are available for gener- could be accepted by physicians, if they are integrated seam-
ating a pixel from the ray. The most prominent function is the lessly into the clinical workflow and if they provide a signifi-
weighted sum of voxels. A transfer function assigns a color and cant benefit at least for one particular phase of this workflow. A
transparency to each voxel intensity. It may be further refined perfect medical AR user interface would be integrated in such a
with the image gradient. A special kind of volume rendering is way that the user would not feel its existence, while taking full
the digitally reconstructed radiograph (DRR) that provides pro- advantage of additional in situ information it provides.
jections of a CT data set that are similar to X-ray images. Generally, augmented optics and augmented endoscopes
The advantage of direct volume rendering is a visualization do not dramatically change the OR environment, apart from
that has the capability of emphasizing certain tissues without an adding a tracking system imposing free line of sight constraints,
explicit segmentation thanks to a certain transfer function. Clear and change the current workflow minimally and smoothly. The
transitions between structures are not necessary. Also cloudy major issue they are facing is appropriate depth perception
structures and their density can be visualized. within a mixed environment, which is subject of active re-
The major disadvantage used to be too slow rendering in par- search within the AR community [128], [137], [139], [140].
ticular for AR, but hardware supported rendering algorithms on Augmented medical imaging devices provide aligned views by
current graphic cards can provide sufficient frame rates on real construction and do not need additional tracking systems. If
3D data. Currently, 3D texture based [135] and GPU acceler- they do not restrict the working space of the physicians these
ated raycast [133] renderers are the state of the art in terms systems have the advantage of a smooth integration into the
of speed and image quality, where the latter offer better image medical workflow. In particular, the CAMC system is currently
quality. Also the ray casting technique needs clear structures in getting deployed within three German hospitals and will be
the image for acceptable results, which can be realized with con- soon tested on 40 patients in each of these medical centers.
trast agents or segmentation. The AR window and video-see-through HMD systems still
need hardware and software improvement in order to satisfy the
F. User Interaction in Medical AR Environments requirements of operating physicians. In both cases, the commu-
Classic 2D computer interaction paradigms such as windows, nity also needs new concepts and paradigms allowing the physi-
mouse pointer, menus, and keyboards do not translate well for cians to take full advantage of the augmented virtual data, to
Authorized licensed use limited to: Amal Jyothi College of Engineering. Downloaded on August 10,2010 at 09:29:29 UTC from IEEE Xplore. Restrictions apply.
14. 464 JOURNAL OF DISPLAY TECHNOLOGY, VOL. 4, NO. 4, DECEMBER 2008
easily and intuitively interact with it, and to experience this dy- [23] J. P. Rolland and H. Fuchs, “Optical versus video see-through
namic mixed environment as one unique and correctly perceived head-mounted displays in medical visualization,” Presence, vol. 9, pp.
287–309, 2000.
3D world. We share the expectations of business analysts [141] [24] O. Cakmakci and J. Rolland, “Head-worn displays: A review,” Int. J.
that the hype level of augmented reality will reach its maximum Display Int. Technol., vol. 2, pp. 199–216, Sept. 2006.
in a few years and that medical AR will be one of its first killer [25] P. J. Kelly, G. Alker, and S. Goerss, “Computer-assisted stereotactic
laser microsurgery for the treatment of intracranial neoplasms,” Neu-
applications, saving lives of many future patients. rosurgery, vol. 10, pp. 324–331, 1982.
[26] J. Hatch and J. S. D. W. Roberts, “Reference-display system for the
integration of CT scanning and the operating microscope,” in Proc.
REFERENCES 11th Annu. Northeast Bioeng. Conf., 1985.
[27] E. M. Friets, J. W. Strohbehn, J. F. Hatch, and D. W. Roberts, “A frame-
[1] H. Steinhaus, “Sur la localisation au moyen des rayons x,” Comptes less stereotaxic operating microscope for neurosurgery,” IEEE Trans.
Rendus de L’Acad. des Sci., vol. 206, pp. 1473–1475, 1938. Biomed. Eng., vol. 36, no. 6, Jun. 1989.
[2] I. Sutherland, “A head-mounted three dimensional display,” in Proc. [28] P. J. Edwards, D. D. Hill, D. D. Hawkes, and D. A. Colchester,
Fall Joint Computer Conf., 1968, pp. 757–764. “Neurosurgical guidance using the stereo microscope,” in Proc. First
[3] D. Roberts, J. Strohbehn, J. Hatch, W. Murray, and H. Kettenberger, Int. Conf. Computer Vision, Virtual Reality and Robotics in Medicine
“A frameless stereotaxic integration of computerized tomographic (CVRMed’95), 1995.
imaging and the operating microscope,” J. Neurosurg., vol. 65, no. 4, [29] A. P. King, P. J. Edwards, C. R. Maurer, Jr., D. A. de Cunha, D. J.
pp. 545–549, 1986. Hawkes, D. L. G. Hill, R. P. Gaston, M. R. Fenlon, A. J. Strong, C. L.
[4] T. Caudell and D. Mizell, “Augmented reality: An application of Chandler, A. Richards, and M. J. Gleeson, “Design and evaluation of
heads-up display technology to manual manufacturing technology a system for microscope-assisted guided interventions,” IEEE Trans.
augmented reality,” in Proc. Hawaii Int. Conf. on Syst. Sci., 1992. Med. Imag., vol. 19, no. 11, pp. 1082–1093, Nov. 2000.
[5] W. Mackay, G. Velay, K. Carter, C. Ma, and D. Pagani, “Augmenting [30] J. Berger and D. Shin, “Computer-vision-enabled augmented re-
reality: Adding computational dimensions to paper,” Commun. ACM, ality fundus biomicroscopy,” Ophthalmology, vol. 106, no. 10, pp.
vol. 36, no. 7, pp. 96–97, 1993. 1935–1941, 1999.
[6] S. Feiner, B. Macintyre, and D. Seligmann, “Knowledge-based aug- [31] W. Birkfellner, M. Figl, K. Huber, F. Watzinger, F. Wanschitz, J.
mented reality,” Commun. ACM, vol. 36, no. 7, pp. 53–62, 1993. Hummel, R. Hanel, W. Greimel, P. Homolka, R. Ewers, and H.
[7] P. Wellner, W. Mackay, and R. Gold, “Computer augmented environ- Bergmann, “A head-mounted operating binocular for augmented
ments: Back to the real world,” Commun. ACM, vol. 36, no. 7, pp. reality visualization in medicine—Design and initial evaluation,”
24–26, 1993. IEEE Trans. Med. Imag., vol. 21, no. 8, pp. 991–997, Aug. 2002.
[8] P. Milgram and F. Kishino, “A taxonomy of mixed reality visual dis- [32] W. Birkfellner, K. Huber, F. Watzinger, M. Figl, F. Wanschitz, R. Hanel,
plays,” IEICE Trans. Inf. Syst., pp. 1321–1329, 1994. D. Rafolt, R. Ewers, and H. Bergmann, “Development of the variosco-
[9] R. T. Azuma, “A survey of augmented reality,” Presence: Teleopera- pear—A see-through hmd for computer-aided surgery,” in Proc. IEEE
tors and Virtual Environments, vol. 6, no. 4, pp. 355–385, 1997. and ACM Int. Symp. on Augmented Reality, 2000, pp. 54–59.
[10] R. T. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. [33] M. Figl, C. Ede, J. Hummel, F. Wanschitz, R. Ewers, H. Bergmann, and
MacIntyre, “Recent advances in augmented reality,” IEEE Comput. W. Birkfellner, “A fully automated calibration method for an optical
Graphics Appl., vol. 21, pp. 34–47, 2001. see-through head-mounted operating microscope with variable zoom
[11] F. P. Brooks, “The computer scientist as toolsmith ii,” Commun. ACM, and focus,” IEEE Trans. Med. Imag., vol. 24, no. 11, pp. 1492–1499,
vol. 39, no. 3, pp. 61–68, 1996. Nov. 2005.
[12] M. Bajura, H. Fuchs, and R. Ohbuchi, “Merging virtual objects with [34] R. Holloway, “Registration error analysis for augmented reality,” Pres-
the real world: Seeing ultrasound imagery within the patient,” in Proc. ence: Teleoperators and Virtual Env., vol. 6, no. 4, pp. 413–432, 1997.
19th Annu. Conf. on Computer Graphics and Interactive Techniques, [35] Y. Masutani, M. Iwahara, O. Samuta, Y. Nishi, N. Suzuki, M. Suzuki,
1992, pp. 203–210. T. Dohi, H. Iseki, and K. Takakura, “Development of integral pho-
[13] M. Bajura and U. Neumann, “Dynamic registration correction in video- tography-based enhanced reality visualization system for surgical sup-
based augmented reality systems,” IEEE Comput. Graph. Appl., vol. port,” Proc. lSCAS, vol. 95, pp. 16–17, 1995.
15, no. 5, pp. 52–60, 1995. [36] H. Liao, S. Nakajima, M. Iwahara, E. Kobayashi, I. Sakuma, N. Ya-
[14] M. C. Jacobs, M. A. Livingston, and A. State, “Managing latencyin hagi, and T. Dohi, “Intra-operative real-time 3-D information display
complex augmented reality systems,” in Proc. ACM 1997 Symp. on system based on integral videography,” in Proc. Int. Conf. Med. Image
Interactive 3D Graphics, 1997, pp. 49–54. Computing and Computer Assisted Intervention (MICCAI), 2001, pp.
[15] A. State, M. A. Livingston, W. F. Garrett, G. Hirota, M. C. Whitton, 392–400.
E. D. Pisano, and H. Fuchs, “Technologies for augmented reality [37] H. Liao, N. Hata, S. Nakajima, M. Iwahara, I. Sakuma, and T. Dohi,
systems: Realizing ultrasound-guided needle biopsies,” in SIG- “Surgical navigation by autostereoscopic image overlay of integral
GRAPH’96: Proc. 23rd Annu. Conf. on Computer Graphics and videography,” IEEE Trans. Int. Technol. Biomed., vol. 8, no. 2, pp.
Interactive Techniques, New York, NY, USA, 1996, pp. 439–446. 114–121, 2004.
[16] M. Rosenthal, A. State, J. Lee, G. Hirota, J. Ackerman, E. D. P. K. [38] M. Blackwell, C. Nikou, A. M. D. Gioia, and T. Kanade, I. W. M.
Keller, M. Jiroutek, K. Muller, and H. Fuchs, “Augmented reality Wells, A. C. F. Colchester, and S. L. Delp, Eds., “An image overlay
guidance for needle biopsies: An initial randomized, controlled trial in system for medical data visualization,” in Proc. First Int. Conf. of Med-
phantoms,” Medical Image Anal., vol. 6, no. 3, pp. 313–320, 2002. ical Image Computing and Computer-Assisted Intervention (MICCAI)
[17] F. Sauer, F. Wenzel, S. Vogt, Y. Tao, Y. Genc, and A. Bani-Hashemi, Lecture Notes in Comput. Sci., Cambridge, MA, Oct. 1998, vol. 1496,
“Augmented workspace: Designing an AR testbed,” in Proc. IEEE and pp. 232–240.
ACM Int. Symp. on Augmented Reality, 2000, pp. 47–53. [39] G. Goebbels, K. Troche, M. Braun, A. Ivanovic, A. Grab, K. von
[18] W. A. Hoff and T. L. Vincent, “Analysis of head pose accuracy in Lübtow, R. Sader, F. Zeilhofer, K. Albrecht, and K. Praxmarer,
augmented reality,” IEEE Trans. Visualiz. Computer Graphics, vol. 6, “Arsystricorder—development of an augmented reality system for
2000. intraoperative navigation in maxillofacial surgery,” in Proc. IEEE and
[19] D. L. Wright, J. P. Rolland, and A. R. Kancherla, “Using virtual reality ACM Int. Symp. on Mixed and Augmented Reality (ISMAR), 2003.
to teach radiographic positioning,” Radiologic Int. Technol., vol. 66, [40] S. Wesarg, E. Firle, B. Schwald, H. Seibert, P. Zogal, and S. Roed-
no. 4, pp. 233–238, 1995. diger, R. L. Galloway, Jr., Ed., “Accuracy of needle implantation in
[20] Y. Argotti, L. Davis, V. Outters, and J. Rolland, “Dynamic superimpo- brachytherapy using a medical AR system: A phantom study,” Proc.
sition of synthetic objects on rigid and simple-deformable real objects,” 2004 SPIE Medical Imaging: Visualization, Image-Guided Procedures,
Computers & Graphics, vol. 26, no. 6, pp. 919–930, 2002. and Display, vol. 5367, pp. 341–352, 2004.
[21] T. Sielhorst, T. Blum, and N. Navab, “Synchronizing 3d movements [41] W. Lorensen, H. Cline, C. Nafis, R. Kikinis, D. Altobelli, L. Gleason,
for quantitative comparison and simultaneous visualization of actions,” G. Co, and N. Schenectady, “Enhancing reality in the operating room,”
in Proc. IEEE and ACM Int. Symp. on Mixed and Augmented Reality in IEEE Conf. on Visualization, 1993, pp. 410–415.
(ISMAR), 2005. [42] W. E. L. Grimson, T. Lozano-Perez, W. M. Wells III, G. J. Ettinger, S. J.
[22] G. Luo and E. Peli, “Use of an augmented-vision device for visual White, and R. Kikinis, “An automatic registration method for frameless
search by patients with tunnel vision,” Investigative Ophthalmology & stereotaxy, image guided surgery, and enhanced reality visualization,”
Visual Science, vol. 47, no. 9, pp. 4152–4159, 2006. IEEE Trans. Med. Imag., vol. 15, no. 2, pp. 129–140, Apr. 1996.
Authorized licensed use limited to: Amal Jyothi College of Engineering. Downloaded on August 10,2010 at 09:29:29 UTC from IEEE Xplore. Restrictions apply.