Twet

Twet Alexandru Burada, Gheorghe Prelipcean Faculty of Computer Science, Al. I. Cuza University Iasi, Romania alexandru.burada@info.uaic.ro, gheorghe.prelipcean.info.uaic.ro Abstract. Twet is a mashup that uses WordNet to search terms the user filled in and using the results from the WordNet thesaurus, search popular social sites like Twitter, Flickr or Youtube for content related to those terms. Also, using Bing Maps, it searches for the location of the photos. Keywords: mashup, wordnet, synset, twitter, flickr, geo-location 1 Introduction A mashup is an application that combines multiple external services to create a content aggregation useful to the user. Twet is a mashup because it uses services offered by popular social sites such as Twitter, Flickr or Youtube, together with Bing Maps and WordNet. This application allows users to search on popular social sites not only by keywords, but by trying to find similar terms to the ones they requested. This way, the results of the search are not limited by the actual words they used, but are enhanced by using the synsets provided by WordNet. In this context the application gives the possibility they could create an overview of how the idea or information is perceived globally, can locate images or ideas of people who were interested in this subject. A usage scenarios is when a user has vague knowledge about a certain idea, not necessarily something very concrete. The best way he can get introduced to the term is a collection of images, texts and videos related to this concept, and this is where Twet comes into play. This ensures the results are, in most cases, closer to what the user intended to search. The risk is that by searching on a greater number of terms, the results are broader and thus less exact, but this can be controlled by the number of terms used in the search. 2 Architecture Twet was designed using the .NET Platform, as a Silverlight 4 application. Silverlight 4 is currently in beta status and it can only be used together with Visual Studio 2010,

2 Alexandru Burada, Gheorghe Prelipcean also in beta. Because of the size of the WordNet database, we decided to add a server behind the application, that will be called for requesting synsets. All other calls will be made by the Silverlight client, using Microsoft's new web platform at its full potential. The server is stateless and it is exposed by a single, general purpose request handler. The Request Handler is something similar to Java Servlets, so it is a basic communication method. We have managed to use this simple method at its full power, by designing a general method of calling server methods, albeit we only used it for the single purpose of searching the WordNet thesaurus. The fallowing diagram illustrates the design of the client - server call mechanism. Fig. 1. Class flow diagram for client - server communication The key to making a generic call mechanism is to use reflection and invoke dynamic methods on the server. The client has to know the name of method, the name of the class, assembly and the parameters need for invoking the server method. This information is contained in the AsyncContext class, that will be serialized to the Request Handler. AsyncCallContext c = new AsyncCallContext() { AssemblyName = "Twet.Web", ClassName = "Twet.Web.WordNet.TwetWordNet", MethodName = "FindSynonyms", Request = request, ResultCallBack = target,

Twet 3 DestinationThread = destinationDispatcher, RequestData = word }; Serialization is the responsibility of the Serializer class. For now, it can only serialize primitive types and collections, but it can be enhanced to support serializing any kind of objects (section 3). Once the request gets to the server, a general purpose RequestHandler invokes the requested method via reflection. string assemblyName = reader.ReadString(); string className = reader.ReadString(); string methodName = reader.ReadString(); object o = Serializer.DeserializeObject(reader); Type classType = Type.GetType(className + "," + assemblyName); object cls = Activator.CreateInstance(classType); response = classType.InvokeMember(methodName, BindingFlags.InvokeMethod | BindingFlags.Public | BindingFlags.Instance, null, cls, new object[] {o}); BinaryWriter w = new BinaryWriter(context.Response.OutputStream); Serializer.SerializeObject(w, response); w.Close(); Since all external communication in Silverlight is asynchronous, we have a problem with executing the response on the same thread as the thread of the controls. This problem is solved by passing the Dispatcher object when calling a server method, and also passing a AsyncDataReceiver, a handler to be executed when the response is received. The fallowing code snippet show how we use the Dispatcher passed by the caller to invoke the response handler. private static void CallReceiver(Dispatcher disp, AsyncDataReceiver receiver, ReceivedData data) { if (receiver == null) return; if (disp != null) { disp.BeginInvoke(() => receiver(data)); } else { receiver(data); }

4 Alexandru Burada, Gheorghe Prelipcean } 2.1 Silverlight Application The entry point to the web interface is the represented by the MainPage.xaml. Fig. 2. Twet welcome screen The main canvas where the text box and search button are present overlaps the map control provided by Bing Maps Silverlight Control [2]. Bing Maps SDK requires a key to be provided. The words the user has filled in are sent to the server, to be searched on the WordNet synsets. The twitter messages returned by the server are displayed using a 3D-like animation, that we are going to discuss next. Because the total number of results is big and adding too many messages to the interface will make it very hard to read (see Fig. 3.), we only added at most 3 messages from each synonym in the synset. For the 3D text animation we have used the an implementation done by Terence Tsang [3] distributed under GNU GPL. This implementation has been modified to support displaying a control made of an image representing the avatar of the user who posted that message, his username on twitter and the actual message. Also, we needed a way to stop the animation when moving away from the page.

Twet 5 The animation consists of a 3D space where the twitter message comes from somewhere far away from the viewpoint until it passes the camera viewpoint, something similar to space movement. Using Silverlight's ScaleTransform, this can be easily achieved. Another feature of the 3D space is that, when you click on one of the messages, it will move the camera in the direction of that message, faster than it would normally take for that text to come right in front of the viewpoint. As the text moves closer, it's opacity is reduced to become more easily readable. All this time, the map is shown behind the canvas of the 3D space and the readability of the text is not affected by the transparency of the canvas. Fig. 3. 3D Text Space view We decided not to geo-locate twitter messages because we noticed that the percent of messages that contain the geo data is very small, and the user could be confused as to why some messages have been located on the map and why the others have not. Also, twitter does not offer a way to make a query that will return messages that satisfy a search and contain geo information. 2.2 Flickr search Searching photos containing the words returned by the WordNet thesaurus was made using the Flickr API. All request were initiated by the Silverlight application, without the need of calling the back-end server. Unlike Twitter, Flickr has a parameter that limits the results to only those containing information for their geo-location. Using the RESTfull API offered by Flickr, Twet makes a request for all photos containing the specified tags and having

6 Alexandru Burada, Gheorghe Prelipcean geo-data. The returned XML needs to be parsed and interpreted. After finding the URI of the photos, we can instantiate the Image objects that will be the placeholder for displaying the photos. An interesting security problem was raised by this feature of Twet. The problem was making cross-domain calls. Silverlight applications are restricted to making such requested unless the target domain has a crossdomain.xml or clientaccesspolicy.xml. Since photos are saved on different flickr servers, all need to have this file at the root of their domain for Silverlight to allow such requests. Loading photos from a crossdomain site is, this way, possible, but with other security limitations. One such limitation is that we cannot access the actual pixel of the photo when loaded into a WritableBitmap object. This object is needed to make a scale a rotate transformation, while the actual Image object that is loading the image can be recycled if we make a copy of the WritableBitmap; since we cannot access the pixels and cannot copy the image, we cannot free the memory of the Image class, so our application will require a little more memory. Scaling the image can only be done once the image was loaded (we do not have information about the photos size before this), so we attached an event handler for the loaded event of each photo. In this event handler we compute the display width and height of the photo, in the fallowing way: BitmapImage bi = sender as BitmapImage; if (cx >= cy) { // Set width to 300 and compute height cy *= 300.0 / cx; cx = 300.0; } else { // Set height to 300 and compute height cx *= 300.0 / cy; cy = 300.0; } WriteableBitmap wb1 = new WriteableBitmap((int)cx, (int)cy); ScaleTransform transform = new ScaleTransform(); transform.ScaleX = cx / bi.PixelWidth; transform.ScaleY = cy / bi.PixelHeight; wb1.Render(image, transform); wb1.Invalidate(); Geo-locating an image is postponed until the user makes the request of locating the image. Since the request and responses are very small, this operation is very fast and the user does not even notice the request. Using the API offered by Bing Maps Silverlight SDK, we can zoom on the map and add a pushpin on the specified coordinates. The map is integrated in the

Twet 7 Silverlight Application. In fact, this is the direction Bing Maps is taking, to using Silverlight for displaying maps. The advantages of this are a better user experience since images are no longer loaded by the browser, with the usual lag. Also, by using Silverlight, some animations can be used for zooming to an area or displaying the Street View. For a live demo of the new Bing Maps, go to [5] Fig. 4. The photo canvas Images can be dragged across the canvas and clicking on it will bring in front of the others, so it can be viewed better. When the mouse is over the image and it is not being dragged, a small icon is displayed in the top left corner. If the users clicks on this icon, he will see the location on the map where that photo was taken. double x = e.GetPosition(null).X; double y = e.GetPosition(null).Y; double dx = x - _lastx; double dy = y - _lasty; //save the new position for future delta calculation _lastx = x; _lasty = y;

8 Alexandru Burada, Gheorghe Prelipcean Fig. 5. Pin added in the location where the photo was localized. The zoom level is set accroding to the the accurecy reported by Flickr. The photo canvas is minimez to the left of the screen, and a button is displayed allowing the photo canvas to return to normal size Dragging images is a simple operation because the parent of all rendered photos is a Silverlight Canvas element. The Canvas has a collection of children, but it does not keep the position for any one of them; instead, each child keeps his own position relative to the canvas. When dragging an image, we move it by setting the Canvas.Left and Canvas.Top properties. They are called attached properties because they are not regular dependency properties of the object. They are attached to that object and interpreted only by the object's parent. 2.3 YouTube search Integrating the YouTube video presented a challenge. The first option was to get a hold on a H.264 stream from YouTube and, since Silverlight supports this format, we could display it our application. The problem, also described in [4], is that it can be hard to get a hold on the source of the h.264 encoding stream. One solution was using a Greasemonkey script on a video that was uploaded in HD. Since this was very hard to implement in a programmatic manner, we looked for another solution. Silverlight allows html code to be displayed on top or behind the Silverlight application if we use it in the windowless mode. This is a initialization parameter of the Silverlight plugin and it allows html and silverlight to be on top of each other. The main problem is that this has significant impact on the performance of the application.

Twet 9 Our 3D text space animation can be the main feature of Twet affected by this option, but it is the only solution of embedding the Youtube video. One of the new features introduced in Silverlight 3 was out-of-browser experience. This meant installing the application on the client machine, where he can run it as a usual desktop application. In this case, there is no page to host the application so we cannot include the html object of the YouTube video. Solving this was done by using a new feature introduced in Silverlight 4, which is the WebBrowser element. This feature is only available for out-of-browser Silverlight applications and it allows embedding html code in the application. One advantage is that it does not have any performance impact, since this is a regular UI Element. Another feature introduced in Silverlight 4 was native support for mouse wheel event. This way, the scroll bar that appears on the ListBox holding the list of YouTube videos matching a search can respond correctly to the wheel event. In Silverlight 3 this could have been implemented by the developer, while in previous versions it was unsupported. Fig. 6. YouTube video embedded in the out-of-browser Twet application. On the left of the screen, the ListBox containing the list of videos matching the query, each having a thumbnail displayed.

10 Alexandru Burada, Gheorghe Prelipcean 2.4 WordNet .Net Accessing the WordNet database was done using the WordNet.NET [1] project. This is an open-source project indented to provide access to the WordNet thesaurus from a .NET project. WordNet.NET needs the path to the installation of WordNet. This is set using a static variable and is used for all future calls to the library. It exposes method for finding the part of speech for a single word and for retrieving a list of synonyms for that word. This way, the complexity of the WordNet format is hidden and the developers can concentrate on the actual application. 3 Enhancements Currently, the Serializer only supports primitive types and collections. But there is a way to enhance it to support serialization of any object. For this we need to define an interface, ISerializable (note that this interface was removed in the Silverlight runtime, so we can reuse this name) with two methods, Serialize(BinaryWriter) and Deserialize(BinaryWriter). Any object that wants to be serialized has to implement these two methods so that the Serializer class can call them when an object of type ISerializable is passed. The class that will be serialized needs to be on both client and server side. References [1] WordNet.NET, http://opensource.ebswift.com/WordNet.Net/ [2] Bing Maps Silverlight Control SDK, http://www.microsoft.com/downloads/details.aspx? &FamilyID=beb29d27-6f0c-494f-b028-1e0e3187e830 [3] Terence Tsang, http://wwdraw.comw.shine [4] YouTube video in Silverlight 3, http://www.85turns.com/2009/06/28/youtube-video-in- silverlight-3/ [5] Silverlight Bing Maps, www.bing.com/maps/explore

Twet

by Computer Science on Jan 29, 2010

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Twet Document Transcript