PRINCIPLES OF COMPUTER GRAPHICS – UNITN 0809
Project – Live Video in World Wind
                                          ...
Summary
Project description .................................................................................................
Project description

The goal of the project is to show videos from different sources like webcams or local video on the N...
texture can be updated periodically. The proposed architecture of the new developed Layer is shown in Figure
3.




      ...
Proposed architecture


Class diagram
The architecture of the developed application is shown into the class diagram in Fig...
The VideoLayer class extends AbstractLayer and is responsible for updating DynamicSurfaceImage classes
getting images from...
Visibility of images
As described in previous paragraphs VideoLayer manages the updating process of DynamicSurfaceImages.
...
Figure 7: An intersection detection




Quality of images
Captured images can have variable dimensions and sometimes can b...
In Figure 9 are shown some statistics about how much time requires the rendering process at varying the
image dimension. W...
Managing Images
The application is an Applet that has a menu panel on the left side as we can see in Figure 11.




      ...
Figure 12: Insert mode

The meta information that user has to insert are the country, the place name, the altitude of the ...
Figure 13: Image Menu

Clicking on the setting button will be shown a second menu that allow user to modify meta-informati...
some parameters like direction, zoom etc. There are webcams that can be accessed directly bi IP. These cams
are called IP-...
Data representation

Videos and Webcams information are stored on local memory as xml files and have a structure like in F...
Sequence diagrams
In Figure 4 is shown the class diagram of the entire project. In the next figures are shown the interact...
Figure 18: Sequence Diagram of the Search operation

User fills the form for the searching function and then press the Sea...
Figure 19: Drag and Drop of image points




The user activates the menu in one of the two ways (in this figure using the ...
Conclusions
WWJ don’t provide features for reloading images dynamically and the texture loading process is expensive in
te...
Figure 21: A bridge in Newcastle, UK




Figure 22: Stuttgart City airport, Germany




                   19
Upcoming SlideShare
Loading in …5
×

Live Video in World Wind

788 views
717 views

Published on

The goal of the project is to show videos from different sources like webcams or local video on the Nasa World Wind.

Published in: Education, Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
788
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Live Video in World Wind

  1. 1. PRINCIPLES OF COMPUTER GRAPHICS – UNITN 0809 Project – Live Video in World Wind Jovan Stevovic Raffaele De Amicis, Giuseppe Conti
  2. 2. Summary Project description ...................................................................................................................................................3 Proposed architecture ..............................................................................................................................................5 Class diagram ........................................................................................................................................................5 Visibility of images ................................................................................................................................................7 Quality of images ..................................................................................................................................................8 Managing Images............................................................................................................................................... 10 Supported Webcams ......................................................................................................................................... 12 Data representation .............................................................................................................................................. 14 Sequence diagrams................................................................................................................................................ 15 Conclusions ............................................................................................................................................................ 18 2
  3. 3. Project description The goal of the project is to show videos from different sources like webcams or local video on the Nasa World Wind. The actual Java SDK version 1.4 provides a lot of features for drawing objects, images, icons, annotations etc. on the World Window but there are not features for displaying videos or images that can changes their aspect during the time. The main task that was elaborated within the project was trying to find the best way in terms of performance and quality, to shows images those changes continuously. The architecture of the current implementation of World Wind Java SDK is shown in Figure 1. Figure 1: The WWJ architecture The SDK provides a simple way to add own implementations and compositions of objects on the Globe using Layers. A Layer (Figure 2.) contains Renderable objects and provides the possibility to personalize the way in which they will be displayed. A Renderable object provides some functions that permit to draw and move objects on the Globe. These objects can be stored locally, into the cache or can be on remote servers. Figure 2: Layer architecture Within the project was developed mainly a new Layer that contains Videos that has an image source and displays them using a new Renderable objects. The new Renderable object has the main characteristic that the 3
  4. 4. texture can be updated periodically. The proposed architecture of the new developed Layer is shown in Figure 3. Figure 3: Developed architecture The Layer is responsible also for schedule the update of Renderables in a way that permits the normal usage of World Wind. This means that the fluid and fast navigation cannot be compromised. Adding video sources to the “system” introduces some problems in terms of efficiency. For the data streams managing in Java was used the JMF 2.1.1e library. The classic and simplest way to implement a reproduction of some source of information is using a Player. Player in JMF is thread based data stream managers that provides a lot of functions for acquisition, processing and delivery of media. That media is transformed in Images, scaled to the preferred quality, rotated and delivered to the Renderable object. The Renderable object reloads the texture and then draws it on the Globe. All these tasks are scheduled into the Layer level using threads in order to avoid blocking of the navigation and considering the trade-off between the quality and the speed of updating of images and the usability of the application. In the next chapters will be illustrated in details the entire architecture of the application. 4
  5. 5. Proposed architecture Class diagram The architecture of the developed application is shown into the class diagram in Figure 4. The diagram shows all implemented classes. Some of them are utility classes for data management and some of them extends or implements existent WWJ classes and are the core of the application. The WWJ’s classes that are used for developing are blue painted. The 2 also blue painted Packages are used for data streams and player management. Figure 4 : Class Diagram The main class is the ApplicationMain that extends a JApplet and starts the World Wind adding the VideoLayer to it and adding also a menu for managing webcam types and searching webcams. 5
  6. 6. The VideoLayer class extends AbstractLayer and is responsible for updating DynamicSurfaceImage classes getting images from VideoObjects. The update task is done at variable intervals depending on how much time it spends on getting and updating images. More time is spent for updating images larger are intervals. This is done for usability reasons. The DynamicSurfaceImage extends Renderable and is very similar to the SurfaceImage class with methods for updating the source of the image. VideoObject is the class that represents a video instance. It has a VideoSource, a VideoInfo and a VideoMenu. The VideoSource class is an interface that specifies essential functions that one specialized video source must implement. In this project are implemented two types of video sources. A player for local video tapes and a Webcam parser that is able to capture images from some types of remote webcams. VideoInfo contains all information about the webcam and the place where it’s situated. VideoMenu implements the menu of an Image and uses two types of icons. UserFacingIcon for buttons and GlobeAnnotation for the annotation that shows info about the place. It uses an IconRenderer for rendering the icons. PointsManager has the main function to manage points that represents image corners. As we can see later the user are able to drag and drop image corners in order to create a perspective view of the image. This class move points, recalculate the sector dimensions of that image and supply some other utility functions. SearchUtil is used to perform search activities on webcams. ImagesUtils provides all functions needed for managing images before rendering them. XMLUtils provides all functions needed for loading information from configuration files and from webcams folder. 6
  7. 7. Visibility of images As described in previous paragraphs VideoLayer manages the updating process of DynamicSurfaceImages. Some considerations can be done about the visibility of images and their updating process in order to improve performances of the application. If we are viewing an image the others that are not in the View shouldn’t be updated and then can be paused. An example is shown in Figure 5. Figure 5: Videos not in View are paused. Another consideration is that if the image is so sloped then the user probably is not interested in that image and in any case doesn’t see the image very well, so the updating can be paused. Example in Figure 6. Figure 6: A sloped view of an image If there are objects between the view eye and the image then the updating of the image has to be stopped. This is useful in particular in mountain landscape. Example in figure 7. 7
  8. 8. Figure 7: An intersection detection Quality of images Captured images can have variable dimensions and sometimes can be very big having dimensions greater than 800x600. During the navigation, the user can be close or far from the image like in Figure 8. If the user is so close then the image has to be showed in the best possible quality, but if the user is far from the image then the dimension can be reduced in order to improve the efficiency of reloading texture. Figure 8: Far and near view of the same image 8
  9. 9. In Figure 9 are shown some statistics about how much time requires the rendering process at varying the image dimension. We can see that the required time for rendering images decrease drastically on decreasing image dimension. Figure 9: Time to render images of different quality All these checking functions are done at every updating cycle and permits to have a bigger number of simultaneously played videos. As described before the update cycles are delayed more if the updating and rendering requires more time. The update and render time are proportional to the number of played videos. In Figure 10 are shown some statistics. Figure 10: Time to reload a number of images Required time for reloading is stored in the Statistics class and at every updating cycle the VideoLayer ask to Statistics class an estimated time to “wait” for the next upload. This improves the usability of the application. 9
  10. 10. Managing Images The application is an Applet that has a menu panel on the left side as we can see in Figure 11. Figure 11: The application In the box 1 are listed all types of webcams in witch can be categorized. The user can remove or add types or clicking on the name can list all webcams of this type. Box 2 has two buttons that permits to add a webcam or a country to the list of countries. Clicking on the add webcam button, the boxes 2, 3, 4 will be replaced by the insert form as we can see in next figure. The box 3 permits to search webcam on the Globe. The user can select the text to search, to search only in active webcams or also into them that are disabled by users in the box 1. The user can also select the country in which to search and to include the URL into the possible result. In Box 4 are shown search results. The user can go to the webcam, to modify it or also to delete it. If the user click to Add Webcam then will be shown a form that contains some essential information like the URL and some other meta-information. The user has to pick two angles of the rectangle that represents the dimension and position of the image. The form and rotation can be changed successively. 10
  11. 11. Figure 12: Insert mode The meta information that user has to insert are the country, the place name, the altitude of the place, the type of the webcam and a little description. When the webcam is playing the user can open the menu of the image double-clicking on it. The menu contains a button that permits the deletion of the webcam, a button that pause the execution, a settings button and an information button that shows an annotation with meta-information. On the right side there are 2 buttons that permits to increase or decrease the quality of the image. Some webcams deliver images with very high quality but some other don’t. The user can try to increase then the quality reaching the maximum. 11
  12. 12. Figure 13: Image Menu Clicking on the setting button will be shown a second menu that allow user to modify meta-information and also the form of the image. The possibility to manage the form of the image permits to adapt them to the terrain conformation. There are two types of buttons. The angles one permits to drag only a corner. The middles one permit to move both angles at the corners simultaneously. Activating the drag and drop menu is activated also the BasicDragger that permits to move the entire image. Using the menu on the MenuPanel the user can also rotate the image by some degree. This is done rotating buttons circularly by the centre using trigonometric functions. Figure 14: Image modification menu Supported Webcams Nowadays there are a lot of webcams that are published on internet and are accessible in various ways. The simplest way is by URL. Webcams captures images and saves them on some accessible directory. Then the user access on that directory by URL and display the image with a browser. If the user wants to see the “next frame” has to reload the browser. Some other webcams are showed by Java Applets and sometimes users can manage 12
  13. 13. some parameters like direction, zoom etc. There are webcams that can be accessed directly bi IP. These cams are called IP-Cams and can be accessed from any client that support streams of media. In this project are used Jipcam libraries that provide access to “Axis” (http://www.axis.com/products/video/camera/index.htm) webcams. These webcams are the most popular and has the advantage that provides a stream of data that can be captured easily. The supported models are: • Axis205 • Axis206M • Axis207 accessing mode: “/axis-cgi/mjpg/video.cgi” or “/axis-cgi/jpg/image.cgi” • Axis207M • Axis207MW • Axis2100 accessing mode: “/cgi-bin/image.cgi” or “/mjpg/mjpg/video.mjpg” • Axis2120 • Axis2130 Google provides a way to search pages with some substring into the URL, so if we search for example • inurl: /axis-cgi/mjpg/video.cgi • intitle:Live view - / - AXIS • inurl:axis-cgi/jpg • inurl:view/indexFrame.shtml • inurl:view/view.shtml • inurl:view/index.shtml • allintitle: Network Camera NetworkCamera • intitle:liveapplet • intitle:"WJ-NT104 Main" • intitle:"snc-rz30 home" the result will be a list of available cameras that the user can add to the project. There are also a lot of websites that classifies and provides lists of webcams by manufacturer and what these webcams shows. The list of the most popular cites: • http://www.camhacker.com/ • http://www.123cam.com/category-webcam-Axis.html • http://www.ttrix.com/ipvisionpubcameras.html • http://www.snow-cams.com/ 13
  14. 14. Data representation Videos and Webcams information are stored on local memory as xml files and have a structure like in Figure 15. The main element is a Video. It has some elements that describe video characteristics and a sub-element that contains information about the place that the video shows. Figure 15: The XML representation of a video All XML files that contain video information are divided into folders by country and then by continent in a way that simplifies the insertion and modification process. Figure 16: The webcam classification 14
  15. 15. Sequence diagrams In Figure 4 is shown the class diagram of the entire project. In the next figures are shown the interactions that all classes have during the starting of the application and the normal usage. In Figure 17 is shown the sequence diagram of the starting process of the application. Figure 17: Sequence Diagram of starting the application The main applet creates the VideoLayer and start it. The VIdeoLayer initialize renderables invoking the load function from XMLUtil class that load data from xml files described before. When the VideoLayer obtains the renderables then starts the Thread that updates them continuously. At each cycle the layer asks for the current frame captured by VideoSources that are inside VideoObjects. Once obtained the elaborated frame as BufferedImage then substitute it to the DynamicSurfaceImage source of texture and render it. At each cycle VideoLayer measure the time for getting and updating images and add it to the Statistics class. After that the VideoLayer get the new estimated time to sleep in order to have a fluid navigation through the World Wind. A lot of features are situated into the MenuPanel on the left side of the Canvas. User can view webcams of a desiderated type, search them, add, cancel, modify etc. These functionalities are provided mainly by XML and Search utilities that implements algorithms to access and modify the xml files using JDom. In the next Sequence Diagram is shown the search process. 15
  16. 16. Figure 18: Sequence Diagram of the Search operation User fills the form for the searching function and then press the Search button. If the user choose to search only from active webcams then the search function ask for the list of active webcams to the VideoLayer. Once returned the function pass all parameters of the search form to the Search utility that read xml files and return results to the MenuPanel. Then the results are viewed on the results panel. Another important functionality that is provided by menu buttons is the Drag and Drop function of image angles that permits to modify the form of the image as described in previous chapter. There are two ways to access the Drag and Drop menu; from image menu of from MenuPanel on the left side of the Canvas. 16
  17. 17. Figure 19: Drag and Drop of image points The user activates the menu in one of the two ways (in this figure using the MenuPanel). The Points are showed by VideoMenu class that has an IconRender. User picks points (angle or middle points) and moves them. The new positions of picked icons are passed to the PointsManager that recalculate the new positions of other points, the new sector dimension and the new image proportions in that sector of the image. These proportions as described before are passed to the Video Source and at each update cycle the image will be elaborated using these points. The update of image is not immediately showed because the update is done by the VideoLayer using Statistic functions. 17
  18. 18. Conclusions WWJ don’t provide features for reloading images dynamically and the texture loading process is expensive in terms of CPU time. Invoking a reloading function at each incoming frame will compromise the usage of the application. The update function must be invoked a number of times smaller than the frame rate of videos in order to have a good usability. The process of getting and elaborating images is also expensive. Every Player has an internal Thread that wait for events from the source. If the source has a high frame rate and we want to elaborate each frame then this will probably consume all CPU time that we have. So we have to analyze very well the trade-off between the quality of images, the frequency of reloading and the usability of the application. The way in which the trade-off is implemented into the application consists in two steps. One is given by Statistics evaluation of consumed time to reload and elaborate images in order to obtain a good time to wait at each reloading cycle. The second one is given by asynchronous invocation of functions that returns Images from sources. For doing that is used the Threaded Queue of tasks that WWJ implements. In this way the image is got asynchronously and the reloading is maybe delayed but the navigation is fluid. The totality of CPU time is spent by the acquisition of images and then by the showing process on the Globe. During the work was considered the possibility to use the GPU computing power (CUDA for NVIDIA graphics cards) but there is not sufficient amount of arithmetic operations for justify the use of this technique that can compromise the compatibility of the application with other operating systems and machines. Webcams has an angle of view variable. Some cameras show a street angle and some other shows an entire mountain. The quality of WW image is not very high so if we want to put a webcam that shows a street we don’t have a way to show it on real street and real position. In the next few examples are shown webcams on Google Earth images quality. In these examples we can see how can be useful and nice the combination of high quality of images and webcam geolocalization. Figure 20:Seebruke Selling, Germany 18
  19. 19. Figure 21: A bridge in Newcastle, UK Figure 22: Stuttgart City airport, Germany 19

×