Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The advent of network technology has allowed us to move from a world of complete computer isolation ...
  • to a world of connectivity with devices of all kinds are able to communicate with each other. Furthermore, as network capacity and ubiquity have increased, we have come to witness a phenomenon that has been denominated...
  • the dis-integration of the computer where our computing environment is no longer confined to a single box on our desktop, but instead it's spread out all around us, and networks replace what once were internal communication paths in the computer. two of the most popular examples of this phenomenon are <click> network storage and <click> clusters and grid computing a third example of the dis-integration of the computer is...
  • ...remote display. remote display is a client/server architecture that decouples a computer from the devices used to interact with it, such as the monitor, keyboard, and mouse, and <click> uses the network to provide a communication channel between these devices and the computer. the mode of operation is simple: graphical output from the computer's desktop that would normally be sent to the local video hardware, is instead <click> redirected over the network to a client to be displayed. similarly, in response to the user interacting with the desktop, input events are generated and <click> forwarded from the client back to the server.
  • given the number of benefits associated with it, it's not surprising the popularity that remote display systems enjoy today.
  • quick: for example, it enables ubiquitous access to our desktop. all we need is a network connection and a simple client device. complete: for example, it enables ubiquitous access to our desktop, and all we need is a network connection and a client. and since the required client functionality is so basic, it can be provided almost everywhere, using anything from a simple viewer application to specialized access devices
  • quick: you can collaborate remotely by sharing access to the same computer sosp: as the display output is redirected over the network, it can also be replicated and forwarded to multiple clients simultaneously. in this manner, groups of globally distributed users can collaborate by sharing access to a single computer and the applications and instruments on it.
  • in a similar manner, remote display creates a new paradigm for providing live computer help and technical support. sharing the desktop provides the perfect environment for either showing somebody else how to perform a computer task, or help them troubleshoot a problem.
  • quick: and finally, they're the key technology behind thin clients, where all application logic and data are moved to a secure location and access is done through very simple, stateless devices. sosp: finally, remote display is the core technology enabling thin clients. A thin client system moves all application processing and data to centralized servers and secure server rooms, and using a remote display protocol provides access to these servers from simple and stateless devices. the major benefit of the thin-client model is the reduced management costs associated with it. this reduction comes from the fact that the edges of the network, where management is more costly, are composed of simple devices that require little or no maintenance.
  • remote display has been around for some time now and as a result a number of remote display systems exist today. among the most popular are ... however, the remote display world as it stands today is suffering from a fundamental performance problem <click>
  • Most existing remote display systems have focused on supporting office applications on local area network environments or reducing data transfer for low bandwidth links. as a result, they are unable to provide a high fidelity visual and interactive experience for display intensive applications which form the basis of today's desktop computing and they are unable to efficiently support access on higher latency wide area network enviroments
  • In this context this dissertation introduces THINC, a virtual and remote display architecture for desktop computing designed to address the challenges facing current remote display technology
  • in particular, this dissertation introduces the following contributions. - first, we introduce a high performance remote display system for LAN and WAN environments. our system differs from existing technologies in that we focus on the architecture of the system and not just on the remote display protocol, as a mechanism to improve performance.. - second, we introduce the first remote display system to natively support multimedia content in a way that is both transparent to applications and format independent - third we present a system to deliver superior remote display support for mobile devices, both in terms of performance and usability. our system also provides a competitive alternative to native mobile applications
  • fourth, we introduce mobidesk, a desktop utlity computing infrastructure that enables service providers to host desktop sessions in fully virtualized environments. hosted sessions can be remotely accessed using THINC's remote display, and they can be migrated across computers to provide high-availability - fifth, we present A2M, a system to protect MobiDesk type infrastructures from DDoS attacks by combining an indirection based network and THINC's remote display architecture. A2M is able to provide protection with low overhead by exploiting the asymmetric characteristics of remote display, where the uplink... - finally, we move beyond remote display and show how THINC's architecture can be used to provide continuous, low overhead recording of a desktop. we also introduce a novel way to leverage desktop a11y services to allow users to search recording based on captured text content.
  • our first architectural component is the point at which graphical output from the desktop is intercepted so it can be redirected over the network. Pictured here is the display pipeline of a typical desktop system. At the top are desktop applications, such as web browser/email. Below it is the window system, which is in charge of managing display resources and maintaining the state of the desktop. As application requests are generated, the window system translates them into commands suitable for the display hardware. These commands are passed on to the device driver, a piece of code that knows how to drive a particular hardware device. finally at the bottom is the framebuffer, which contains the contents of the desktop which are displayed on the screen. given this display pipeline there are a number of places we can perform interception.
  • the first option is to intercept high-level requests between the applications and the window system , an approach used by X and X-based proxies such as NX. In this model, everything below the window system is executed on the client. Intercepting at this level has the advantage of giving the rd system complete knowledge and control over the system. unfortunately it all has two serious drawbacks. first, since the window system is at the client, ... second, since an application's logic and its user interface are typically tightly coupled, running the ui on the client and the logic in the client may result in a need for continuous synchronization over the network. In high-latency wide are network environments this kind of synchronization results in substantial performance degradation.
  • at the other end of the
  • ... this model has a number of benefits
  • first, using the standard device driver interface allows THINC to provide remote display in a manner that is transparent to the applications and the window system.
  • second, since the device driver sits below the window system it can leverage existing window system technology instead of having to reimplement, and it can readily take advantage of improvements in commodity window systems
  • third, it enables us to have a simple, low-level protocol that closely mimics the kind of low-level requests delivered to a traditional device driver.
  • finally, the simple low level protocol means that clients can be simple and stateless. in most cases, all they are required to do is read commands from the network and pass them down to their local hardware.
  • how do we translate from application commands to the display protocol?
  • how and when do we send display updates?
  • in order to maximize the responsiveness of the system, the server transmits updates to the client as soon as possible, by pushing them on the network. this approach guarantees minimum delay plus it doesn't suffer as much as network latency increases to properly adapt to varying network conditions and slow responding clients, coalesce + discard old updates
  • srsf analogous to srpt srpt optimal for minimizing mean response time as updates are generated, sort them in queues. the queues are flushed in increasing size additional real time queue allows us to prioritize updates that provide feedback to user (e.g. cursor change)‏
  • to demonstrate the effectiveness of THINC's rd arch, we implemented a prototype as a device driver for the X window system, and conducted an evaluation of its web browsing performance, first in a direct comparison with a number of state of the art remote display systems, and second by measuring its performance on real WAN environments. we measured web browsing performance using Mozilla web browser, and a mechanical device that generates mouse clicks a regular intervals. each click causes a new web page to be loaded. our primary measure of performance is page download latency, defined to be the time elapsed between the mouse click and the page being completely rendered on the client.
  • the results for our direct comparison results are shown in this graph. the x axis .. the y axis. - as can be seen from the graph, THINC outperforms all systems, and can provide up to 4.8 times better performance. two additional points i want to show: - comparing to VNC and X on the LAN provides validation for our interception approach, and shows that THINC can outperform both X's high level interception, and VNCs low-level pixel based approach. - comparing to SunRay demonstrates the effectiveness of our translation mechanisms, in particular offscreen. as mentioned before, sunray and thinc's protocol is quite similar, however sunray incurs higher overhead because it lacks support for offscreen rendering which is heavily used by - finally,
  • - subsecond latency except in korea (really far away, 7k miles)‏ - provides support for our delivery mechanisms, work in high latency environments.
  • - mobile devices pretty much ubiquitous. cellphones, pdas. would be nice if they could serve as extensions of our desktop. - however they provide a very limited environment and limited applications. constrained by low power, need to conserve battery, limited storage. - but they do have network, and graphics cards
  • TCP: - we have no provisions for lost/corrupted data - we need for non-blocking operation
  • This dissertation introduced THINC, a remote and virtual display architecture for desktop computing. We presented the design and evaluation of a high performance remote display system that is able to outperform existing solutions by focusing our efforts, not only on encoding and protocol primitives, but also on the architecture of the system We also presented the first system able to provide full support ... Finally, we showed a solution which can provide superior remote display support for mobile devices, both in terms of performance and display quality, and which is also able to provide a better experience that native applications.
  • we want to: - provide users with a recording of all output that is generated on their desktop, no matter if they saved it to disk, bookmarked it, etc. idea is no need for that. - the recording should behave like a movie. can pause, rewind, fast forward, random access to any point in time - since it will be a large amount of data need to provide a mechanism to search the information
  • desktop most important: only 32 KB/s on average
  • perfect except when connecting from korea (really far away, 7k miles)‏
  • - almost everybody performs well (subsecond) better than everybody in both LAN and WAN (obvious). - compared to SunRay shows the effectiveness of our translation mechanisms, in particular offscreen. we have also measured just for THINC and found lack of offscreen caused a 70% increase in latency. - compared to VNC shows the
  • This graph shows the results of displaying a video clip on the most popular remote display systems, and measuring the playback quality as perceived by the client. As you can see, none of the systems is able to provide performance that is even close to that of today's desktop computer.
  • in particular, this dissertation: - introduces a remote display architecture for high performance remote display on LAN and WAN environments.
  • The advent of network technology has allowed us to move from a world of complete computer isolation ...
  • Translation decoupled from delivery
  • so what does THINC's architecture look like?
  • defense.ppt

    1. 1. THINC: A Virtual and Remote Display Architecture for Desktop Computing Ricardo A. Baratto Network Computing Laboratory Columbia University
    2. 2. isolation...
    3. 3. Source: Internet Mapping Project ( ... connectivity
    4. 4. dis-integration of the computer network storage clusters and grid computing
    5. 5. remote display display updates input
    6. 6. benefits
    7. 7. ubiquitous access
    8. 8. remote collaboration
    9. 9. online help
    10. 10. thin clients stateless client application processing and data secure server room
    11. 11. existing systems
    12. 12. what's wrong? <ul><li>problem: </li></ul><ul><li>focused on office applications on LAN and reducing data transfer on low bandwidth links </li></ul><ul><li>therefore: </li></ul><ul><li>poor support for display intensive interactive applications </li></ul><ul><li>poor support for higher latency WAN environments </li></ul>
    13. 13. THINC <ul><ul><li>virtual and remote display architecture </li></ul></ul><ul><ul><li>for desktop computing </li></ul></ul>
    14. 14. contributions <ul><li>high performance remote display system [SOSP 05] </li></ul><ul><li>first to natively support multimedia applications </li></ul><ul><li>superior remote display support for mobile devices [WWW 06, SCC 06] </li></ul>
    15. 15. contributions II <ul><li>MobiDesk: desktop utility computing infrastructure [MobiCom 04, best student paper award] </li></ul><ul><li>A 2 M: protect MobiDesk from DDoS attacks </li></ul><ul><li>beyond remote display: desktop recording [SOSP 07] </li></ul>
    16. 16. THINC remote display <ul><li>simple display protocol </li></ul><ul><ul><li>COPY, Solid Fill, Tile Fill, BITMAP, RAW </li></ul></ul><ul><li>focus on architecture of the system to improve performance </li></ul><ul><ul><li>interception </li></ul></ul><ul><ul><li>translation </li></ul></ul><ul><ul><li>delivery </li></ul></ul>
    17. 17. interception: traditional display pipeline applications window system device driver framebuffer
    18. 18. interception <ul><li>stateful client hurts mobility </li></ul><ul><li>app – window system synchronization </li></ul>applications window system device driver framebuffer high-level requests
    19. 19. interception <ul><li>lose semantics: expensive encoding </li></ul>applications window system framebuffer device driver raw pixels high-level requests
    20. 20. virtual display architecture
    21. 21. benefits
    22. 22. benefits
    23. 23. benefits
    24. 24. benefits
    25. 25. translation <ul><ul><li>use and preserve semantic information for efficient translation </li></ul></ul>
    26. 26. <ul><li>use semantic information when doing translation </li></ul>translation
    27. 27. use request semantics to generate update req: fill window W, color C application window system req: fill [x,y,w,h] color C THINC update: solid fill [x,y,w,h] color C
    28. 28. <ul><li>use semantic information when doing translation </li></ul><ul><li>preserve semantic information throughout the system </li></ul>translation
    29. 29. why preserve semantics: offscreen rendering draw offscreen regions copy display
    30. 30. offscreen rendering (cont)‏ offscreen region command log <ul><ul><li>merge, clip, and discard commands as needed </li></ul></ul>
    31. 31. delivery <ul><ul><li>maximize interactive response of the system </li></ul></ul>
    32. 32. delivery <ul><li>transmit updates as soon as possible </li></ul><ul><li>merge, clip, and discard updates as needed </li></ul><ul><li>... not all display content created equal </li></ul>
    33. 33. shortest remaining size first scheduler client buffer C 1 C 2 C 3 ... C n real time . . . queue 1 queue n cmd size
    34. 34. experimental results <ul><li>X/Linux prototype </li></ul><ul><li>web browsing performance </li></ul><ul><ul><li>comparison to existing systems </li></ul></ul><ul><ul><li>performance on wide area networks </li></ul></ul>
    35. 35. web browsing performance ... up to 4.8 times better performance
    36. 36. WAN web browsing performance
    37. 37. multimedia
    38. 38. existing systems
    39. 39. THINC multimedia <ul><li>native support for video playback </li></ul><ul><li>bidirectional audio </li></ul><ul><li>synchronized audio/video playback </li></ul><ul><li>format independent and transparent to applications </li></ul>
    40. 40. video playback video player virtual device driver hw video acceleration interfaces YUV leverage client hardware scaling for free
    41. 41. audio applications OS virtual audio driver audio daemon audio data audio data
    42. 42. experimental results
    43. 43. a/v playback quality ... perfect playback and up to two orders of magnitude better
    44. 44. mobile devices <ul><li>becoming ubiquitous </li></ul><ul><li>have network connectivity </li></ul><ul><li>but limited environment and applications </li></ul>
    45. 45. pTHINC <ul><li>provide superior remote display support for mobile devices </li></ul><ul><li>competitive alternative to limited mobile applications </li></ul>
    46. 46. ? server-resized updates
    47. 47. user interface <ul><li>zoom, scroll, rotate </li></ul><ul><li>full screen by leveraging external controls </li></ul>
    48. 48. experimental results
    49. 49. PDA web browsing performance ... up to 17x faster than other systems, and up to 8x faster than native
    50. 50. native vs pTHINC
    51. 51. btw
    52. 52. limitations <ul><li>need better compression </li></ul><ul><ul><li>NX/SunRay/VNC, adaptive </li></ul></ul><ul><li>multimedia </li></ul><ul><ul><li>too much bandwidth? </li></ul></ul><ul><ul><li>limited on mobile devices </li></ul></ul><ul><ul><li>flash video </li></ul></ul><ul><li>no support for new generation of high-end desktops and 3D applications </li></ul>
    53. 53. impact <ul><li>Technology licensed for commercial use </li></ul><ul><li>VESA standard in progress </li></ul>
    54. 54. conclusions <ul><li>high performance remote display system able to outperform existing solutions </li></ul><ul><li>first to provide full support for multimedia content, transparently to applications and independent of format </li></ul><ul><li>superior remote display support for mobile devices </li></ul>
    55. 55. backup
    56. 56. Page-by-page comparison <ul><li>THINC faster on all pages except image-only </li></ul><ul><li>Our approach works better than both low and high-level on non-image content </li></ul>
    57. 57. Microbenchmarks <ul><li>It's not each individual component, but the combination </li></ul><ul><li>Not clear microbenchmark results can be extrapolated to overall performance </li></ul><ul><li>But do have some results: </li></ul><ul><ul><li>No offscreen: ~70% (all RAW)‏ </li></ul></ul><ul><ul><li>Only interception and redirection: ~225% </li></ul></ul>
    58. 58. future work <ul><li>remote display standard (Net2Display)‏ </li></ul><ul><li>3D </li></ul><ul><li>indexing and searching improvements </li></ul>
    59. 59. list of publications <ul><li>MobiDesk: Mobile Virtual Desktop Computing [MobiCom 04] </li></ul><ul><li>THINC: A Virtual Display Architecture for Thin-Client Computing [SOSP 05] </li></ul><ul><li>Remotely Keyed Cryptographics: Secure Remote Display Access Using (Mostly) Untrusted Hardware [ICICS 05] </li></ul><ul><li>pTHINC: A Thin-Client Architecture for Mobile Wireless Web [WWW 06] </li></ul><ul><li>An Application Streaming Service for Mobile Handheld Devices [SCC 06] </li></ul><ul><li>Net2Display: A Proposed VESA Standard for Remoting Displays and I/O Devices over Networks [ADEAC 06] </li></ul><ul><li>DejaView: A Personal Virtual Computer Recorder [SOSP 07] </li></ul>
    60. 60. beyond remote display: desktop recording <ul><li>provide recording of all desktop output </li></ul><ul><li>allow recording to be searched </li></ul>
    61. 61. desktop recording text capture daemon index visual record client/ viewer applications virtual device driver visual output accessibility services text output
    62. 62. recording runtime overhead
    63. 63. storage growth 32KB/s
    64. 64. wan a/v playback quality
    65. 65. mouth-to-ear latency overhead
    66. 66. PDA video playback quality
    67. 67. web browsing data transfer
    68. 68. A/V data transfer
    69. 69. PDA web browsing data transfer
    70. 70. PDA video data transfer
    71. 71. browse and search latency
    72. 72. playback speedup
    73. 73. web browsing performance ... up to 4.8 times better performance
    74. 74. existing performance problem
    75. 75. implementation <ul><li>X/Linux and windows display server </li></ul><ul><li>Linux audio driver (alsa) + daemon </li></ul><ul><li>X/Linux windows, PDA, and Java clients </li></ul><ul><li>X/Linux desktop recording </li></ul><ul><ul><li>capture using ATK (Gnome)‏ </li></ul></ul><ul><ul><li>index using Postgres + Tsearch2 </li></ul></ul>
    76. 76. offscreen drawing draw offscreen regions copy display
    77. 77. command queues offscreen region command queue
    78. 78. copy onscreen client queue 1 2 3 3 2 1
    79. 79. how? applications client hardware caps video
    80. 80. how we deliver updates display updates client buffer C 1 C 2 C 3 ... C n
    81. 81. desktop virtualization <ul><li>decouple desktop from underlying video hardware </li></ul><ul><li>encapsulate window and graphical state </li></ul><ul><li>can move around: </li></ul><ul><ul><li>carry in pocket </li></ul></ul><ul><ul><li>around hosting servers </li></ul></ul>
    82. 82. MobiDesk
    83. 83. A 2 M: Access Assured Mobile Desktop Computing
    84. 84. old slides
    85. 85. contributions <ul><li>high performance remote display system [SOSP 05] </li></ul><ul><ul><li>focus on system architecture to improve performance </li></ul></ul><ul><ul><li>outperforms existing systems </li></ul></ul><ul><li>first to natively support multimedia applications </li></ul><ul><ul><li>transparent and format-independent </li></ul></ul><ul><li>wireless mobile device support [WWW 06, SCC 06] </li></ul><ul><ul><li>server-side display scaling for improved display quality and performance </li></ul></ul><ul><ul><li>user interface tailored for constrained environment </li></ul></ul>
    86. 86. contributions II <ul><li>MobiDesk: desktop utility computing infrastructure [MobiCom 04] </li></ul><ul><ul><li>fully virtualized environment for hosting desktops </li></ul></ul><ul><ul><li>hosted sessions can be migrated for high availability </li></ul></ul><ul><li>A 2 M: protect MobiDesk from DDoS attacks </li></ul><ul><ul><li>indirection-based network + remote display </li></ul></ul><ul><ul><li>exploit asymmetric traffic to minimize overhead </li></ul></ul><ul><li>beyond remote display: desktop recording [SOSP 07] </li></ul><ul><ul><li>low overhead recording, and efficient access </li></ul></ul><ul><ul><li>novel use of accessibility services for indexing and searching </li></ul></ul>
    87. 87. display protocol <ul><li>Inspired on Sun Ray protocol </li></ul><ul><li>2D Primitives </li></ul><ul><li>copy </li></ul><ul><li>solid and tile fill </li></ul><ul><li>bitmap fill </li></ul><ul><li>raw </li></ul>
    88. 88. demo
    89. 89. video <ul><li>leverage existing hardware acceleration interfaces </li></ul><ul><li>YUV (luminance-chrominance) color space </li></ul><ul><ul><li>format independence </li></ul></ul><ul><ul><li>client hardware acceleration (scaling for free)‏ </li></ul></ul>
    90. 90. YUV <ul><li>Standard hardware interface </li></ul><ul><li>Format independence </li></ul><ul><li>Hardware acceleration (fullscreen for free!)‏ </li></ul>
    91. 91. isolation...
    92. 92. ... and a PC                                              
    93. 93. <ul><ul><li>system architecture </li></ul></ul><ul><ul><li>as important </li></ul></ul><ul><ul><li>as protocol and encoding </li></ul></ul>
    94. 94. goals <ul><li>minimize latency </li></ul><ul><li>simple and portable </li></ul><ul><li>transparent operation </li></ul>
    95. 95. application requests display updates THINC translate commands deliver
    96. 96. basic static translation Draw API standard device driver commands THINC commands
    97. 97. <ul><li>high performance remote display </li></ul><ul><li>LAN and WAN environments </li></ul><ul><li>transparent operation in exisiting desktop systems </li></ul><ul><li>full screen, full motion audio/video playback </li></ul>THINC
    98. 98. server-resized updates
    99. 99. system architecture
    100. 100. two key problems how do we translate from application commands to the display protocol? how and when do we send display updates?
    101. 101. but... <ul><li>performance problems </li></ul><ul><ul><li>issues supporting display intensive interactive applications </li></ul></ul><ul><ul><li>issues coping with higher latency network environments </li></ul></ul><ul><li>lack of “features” </li></ul><ul><ul><li>limited or no support for multimedia content </li></ul></ul><ul><ul><li>subpar support for mobile devices </li></ul></ul>
    102. 102. system architecture is key <ul><li>interception </li></ul><ul><li>translation </li></ul><ul><li>delivery </li></ul>