Cloudlet-Based Cyber-Foraging in Resource-Constrained Environments

  • 824 views
Uploaded on

First responders and others operating in crisis environments increasingly make use of handheld devices to help with tasks such as face recognition, language translation, decision-making and mission …

First responders and others operating in crisis environments increasingly make use of handheld devices to help with tasks such as face recognition, language translation, decision-making and mission planning. These resource-constrained environments are characterized by dynamic context, limited computing resources, high levels of stress, and poor network connectivity. Cyber-foraging in general is the leverage of external resource-rich surrogates to augment the capabilities of resource-limited devices. In Cloudlet-Based Cyber-Foraging, resource-intensive computation and data is offloaded to cloudlets – discoverable, generic servers located in single-hop proximity of mobile devices. I will present the basics of cloudlet-based cyber-foraging in addition to future work in this area to address system-wide quality attributes that are critical in resource-constrained environment, beyond energy, performance and fidelity of results.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
824
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
34
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. © 2013 Carnegie Mellon University Cloudlet-Based Cyber- Foraging in Resource- Constrained Environments VU University Amsterdam September 26, 2013 Grace A. Lewis (glewis@sei.cmu.edu)
  • 2. 2 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Motivation First responders and others operating in crisis and hostile environments increasingly are making use of handheld devices to help with tasks such as speech and image recognition, natural language processing, decision-making and mission planning Challenges for mobile devices in resource- constrained edge environments •  Mobile devices offer less computational power than conventional desktop or server computers •  Computation-intensive tasks take a heavy toll on battery power •  Networks are often unreliable and bandwidth is limited and inconsistent
  • 3. 3 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Characteristics of Resource-Constrained Environments Wimpy edge nodes •  Limited resources (CPU, battery and memory) on mobile nodes Limited or no end-to-end network connectivity •  Implicit assumption of WAN connectivity is not always valid High cognitive load •  Application latency and fidelity become important in situations of high stress Bounded elasticity •  Upper bound on number of consumers known in advance Dynamic environment •  Static deployment topologies cannot be assumed •  Survivability is essential
  • 4. 4 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Mobile Cloud Computing Mobile Cloud Computing refers to the combination of mobile devices and cloud computing in which cloud resources perform computing- intensive tasks and store massive amounts of data Work in this field is currently divided into three areas 1.  Extending access to cloud services to mobile devices 2.  Enabling mobile devices to work collaboratively as cloud resource providers 3.  Augmenting execution of mobile applications using cloud resources Cyber-foraging is part of the last area
  • 5. 5 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cyber-Foraging Cyber-foraging* is the leverage of external resource-rich surrogates to augment the capabilities of resource-limited devices •  Increase computational capability •  Conserve battery on a mobile device •  Access remote or shared data •  Provide better application fidelity Two main forms of cyber-foraging •  Code/Computation Offload – Offload of expensive computation in order to extend battery life and increase computational capability •  Data Staging – Improve data transfers between mobile computers and the cloud by temporarily staging data in transit * Satyanarayanan, Mahadev (2001). "Pervasive Computing: Vision and Challenges". IEEE Personal Communications (IEEE)
  • 6. 6 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Challenges for Cyber-Foraging in Resource- Constrained Environments Most cyber-foraging solutions rely on •  conventional Internet for connectivity to the cloud •  strategies that tightly couple mobile clients with servers at deployment time These solutions are not appropriate for resource-constrained environments because they do not address the challenge of unreliable networks and dynamic environments
  • 7. 7 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cyber-Foraging Questions Where to offload? •  Proximate (single-hop) or remote (multi-hop) servers? When to offload? •  With optimization in mind, when does it make sense to offload? What to offload? •  What is the granularity of the computation (or data) that is offloaded? •  What is the payload?
  • 8. 8 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cyber-Foraging: Where to Offload? Mobile Enterprise Cloud (Data Center) Mobile Surrogate Enterprise Cloud (Data Center) Synchronous network I/O, Multi-hop network Synchronous Network I/O Asynchronous Network I/O tcloud tsurrogate Single-hop network tcloud Assuming that tsurrogate is less than tcloud, nearby surrogates are a better option from an energy consumption and latency perspective
  • 9. 9 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cyber-Foraging: When to Offload? Never offload It Depends: Size of this area depends on optimization algorithm Always Offload Offloading is beneficial when large amounts of computation (C) are needed with relatively small amounts of communication (D)* Goal of Optimization •  Maximize a utility function •  Typically energy, execution time, fidelity or results Computation (C) Communication(D) * Based on Karthik Kumar and Yung-Hsiang Lu. Cloud computing for mobile users. Computer, 99, 2010.
  • 10. 10 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cyber-Foraging: What to Offload? Another aspect of what to offload is the actual payload •  Computation is transferred from the mobile device to the surrogate at runtime, or •  Code already exists on the surrogate and is remotely invoked, in which case the payload is simply the input parameters Physical Machine OS Infrastructure VM Image (VM Manager, e.g., KVM, OpenStack) Process/Application/Service (Application Server) Method/Thread (Language-Specific Virtual Machine, e.g., JVM, CLR) Surrogate Mobile Code (Runtime Environment) One aspect is granularity
  • 11. 11 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cloudlet-Based Cyber-Foraging Discoverable, generic, stateless servers located in single-hop proximity of mobile devices Can operate in disconnected mode — communication with the central core is only needed for provisioning Virtual-machine based to promote flexibility and mobility Applications statically partitioned into client and server •  Client runs on mobile device •  Server runs on cloudlet Enterprise Cloud Cloudlet Cloudlet Cloudlet Single-­‐Hop  Network Multi-­‐  or  Single-­‐Hop  Network Cloudlet
  • 12. 12 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Reference Architecture for Cloudlet- Based Cyber- Foraging Cloudlet  HostMobile  Client Legend System   Boundary Cloudlet-­‐ Ready   Client  App   1 Custom  Runtime   Component Cloudlet   Client File  Read/ WriteCall File VM Manager Guest  VM  1 Server  1 Cloudlet   Server Discovery Service Cloudlet  Metadata Broadcast 3rd Party Runtime Component Server  Offload  Code  +  Metadata Cloudlet-­‐ Ready   Client  App   2 Cloudlet-­‐ Ready   Client  App   n ... ... Guest  VM  2 Server  2 Guest  VM  n Server  n ... Cloudlet   Metadata Client   App  1   Metadata Client   App  n   Metadata Client   App  2   Metadata Server  1   Offload   Code Server  2   Offload   Code Server  n   Offload   Code
  • 13. 13 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Original Proposal for Cloudlet Provisioning: VM Synthesis Application overlay is transmitted to the cloudlet at runtime Cloudlet applies application overlay to the Base VM and produces a Complete VM Start Base VM Base VM Disk Image Install Application Complete Memory Snapshot Suspend VM Save Disk and Memory Snapshots Complete Disk Snapshot Obtain Base VM from Central Core Start Application Base Memory Snapshot Calculate Diff Between Complete Snapshots and Base Snapshots VM Disk Image Overlay Memory Snapshot Overlay Application Overlay Base Disk Snapshot
  • 14. 14 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University VM Synthesis Process 1.  Mobile device discovers cloudlet 2.  Mobile device sends application overlay (always offload strategy) 3.  Cloudlet applies overlay to base VM 4.  Cloudlet starts the constructed VM and informs client that the application server is ready for execution Cloudlet HostMobile Client Legend System Boundary Cloudlet- Ready Client App 1 Custom Runtime Component Cloudlet Client Application Overlay n File Read/ WriteCall File VM Manager Guest VM 1 Server 1 Cloudlet Server Discovery Service Cloudlet Server IP Address/Port Broadcast Base VM Image 3rd Party Runtime Component Application Overlay 2 Application Overlay 1 Application Overlay Cloudlet- Ready Client App 2 Cloudlet- Ready Client App n ... ... Guest VM 2 Server 2 Guest VM n Server n ...
  • 15. 15 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Problems with VM Synthesis in Resource- Constrained Environments VM overlays are very large •  Large payloads incur large energy costs •  Limited bandwidth can make application ready time unreasonable Payload NULL OBJECT SPEECH FACE Linux (MB) Windows (MB) Linux (MB) Linux (MB) Windows (MB) Windows (MB) Compressed Disk Overlay 0.1 0.4 42.8 104.8 113.7 33.5 Compressed Disk + Memory Overlays 21.2 4.2 144.5 226.8 425.7 141.5 Question: Is it possible to maintain the flexibility and mobility of a VM- based strategy but reduce payload size?
  • 16. 16 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cloudlet Provisioning using Application Virtualization Emulates OS functionality for an application Runtime component intercepts all system calls from an application and redirects these to resources inside the virtualized application Application itself is unmodified and unaware that it is interacting with virtual OS services Sample Tools •  CDE (short for Code, Data and Environment) – Application virtualizer for Linux – Virtualizes applications by monitoring their execution •  Cameyo – Application virtualizer for Windows – Virtualized applications by monitoring the installation process
  • 17. 17 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Application Virtualization Process Mobile  Device Application  1   Package Cloudlet  Host VM  Hypervisor Guest  VM  – OS  1 Cloudlet  Server Application  1  Server Application  i Server Guest  VM  – OS  n Discovery Application  1 Client Application  1   Metadata HTTP  Server Cloudlet  Server Application  j  Server Application  m  Server Discovery HTTP  Server Cloudlet  ClientDiscovery Application  i Client Application  j Client Application  m Client Application  i Package Application  i   Metadata Application  j   Package Application  j   Metadata Application  m   Package Application  m   Metadata Application  Package Application  Metadata Application  Package Application  Metadata IP  Address  :  Port Cloudlet  Properties Legend System  Boundary Custom  Runtime Component 3rd Party  Runtime Component File Blocking  Call Non-­‐Blocking  Call File  Read HTTP Multicast 1.  Mobile device discovers cloudlet 2.  Mobile device sends application package and metadata 3.  Cloudlet deploys application package inside matching guest VM 4.  Cloudlet informs client that the application server is ready for execution
  • 18. 18 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University VM Synthesis vs. Application Virtualization Cyber- Foraging Strategy Transmitted Package NULL OBJECT SPEECH FACE Linux (MB) Windows (MB) Linux (MB) Linux (MB) Windows (MB) Windows (MB) Application Virtualization Compressed Virtualized Application 1.1 0.9 28.5 67.7 65.4 13.1 VM Synthesis Compressed Disk Overlay 0.1 0.4 42.8 104.8 113.7 33.5 Compressed Disk + Memory Overlays 21.2 4.2 144.5 226.8 425.7 141.5 Experiments show that energy consumption is proportional to file size Even though application virtualization creates payloads of smaller size, it is still large for resource-constrained environments Exploring two alternative options •  Cloudlet Push •  On-Demand VM Provisioning
  • 19. 19 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cloudlet Push Opposite approach to traditional cyber-foraging and cloudlet provisioning •  Pushes client application from the cloudlet to the mobile device
  • 20. 20 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Cloudlet Push (PROTOTYPE) Legend System   Boundary Client  App   1 Custom  Runtime   Component Cloudlet   Client File  Read/ WriteCall File VM Manager Guest  VM  1 Server  1 Cloudlet   Server Discovery Service Cloudlet  Metadata Broadcast 3rd Party Runtime Component Client  App Client  App   2 Client  App   n ... Guest  VM  2 Guest  VM  n Server  n ... Cloudlet   Metadata Client   Metadata  Client   App Server  2 Client  Metadata Cloudlet  HostCloudlet  HostMobile  ClientMobile  Client 1.  Mobile device discovers cloudlet (and available applications) 2.  Mobile device sends client metadata and selected app 3.  Cloudlet sends client app 4.  Mobile device installs and starts client app
  • 21. 21 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University On-Demand Provisioning Leverages commercial cloud provisioning tools such as Puppet or Vagrant to “compose” VMs on demand to match the requirements of the mobile application client •  What is transferred from the mobile device to the cloudlet is a provisioning script Composition could be •  Instantiating a pre-configured VM from a VM image repository that contains the full server component •  Instantiating a pre-configured base VM and adding components •  Putting together a VM from a set of basic components
  • 22. 22 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University On-Demand VM Provisioning (WORK IN PROGRESS) 1.  Mobile device discovers cloudlet 2.  Mobile device sends provisioning script to cloudlet 3.  Cloudlet runs VM provisioning script and starts VM 4.  Cloudlet informs client that the application server is ready for execution Cloudlet  HostMobile  Client Legend System   Boundary Cloudlet-­‐ Ready   Client  App   1 Custom  Runtime   Component Cloudlet   Client File  Read/ WriteCall File VM Manager Guest  VM  1 Server  1 Cloudlet   Server Discovery Service Cloudlet  Metadata Broadcast 3rd  Party   Runtime   Component Provisioning  Script Cloudlet-­‐ Ready   Client  App   2 Cloudlet-­‐ Ready   Client  App   n ... ... Guest  VM  2 Server  2 Guest  VM  n Server  n ... Cloudlet   Metadata Server  1   Provisioning   Script Server  n   Provisioning   Script Server  2   Provisioning   Script VM  Image   Repository Database
  • 23. 23 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Comparison of Cloudlet Provisioning Strategies VM Synthesis Application Virtualization Cloudlet Push On-Demand VM Provisioning Cloudlet Contents (plus Cloudlet Server) Exact Base VM(s) VM(s) compatible with client app(s) •  Paired VM (server code) •  Client app(s) •  VM provisioning software •  Server components Mobile Device Contents •  Cloudlet client •  Application overlay •  Client app •  Cloudlet client •  Virtualized server code •  Client app •  Cloudlet client •  Cloudlet client •  VM provisioning script(s) •  Client app Advantages Guarantees replica of pre-configured surrogate Portability across OS distribution boundaries Supports most client nodes at runtime Small data transfer Constraints Exact Base VM — limits distribution and patches All dependencies must be captured at packaging time Cloudlet has a client app version matching mobile client OS version Cloudlet has all required VM components Distribution to Mobile Devices Deployment Deployment Runtime Deployment Runtime Transfer Cost Depends on overlay size Medium Low Very low
  • 24. 24 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Current Work: Architecture Strategies for Cyber-Foraging in Resource-Constrained Environments Research Question What are the strategies/patterns/tactics — expressed as architecture decisions and promoted quality attributes — that would provide architecture alternatives for cyber-foraging that adapt to unreliable networks and dynamic environments while conserving battery power
  • 25. 25 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Related Work 1 Work in cyber-foraging dates back to 2001 when the term was coined •  Most proposed solutions for cyber-foraging come from electrical and computer engineering with very little emphasis on software engineering or software architecture •  Most solutions favor one quality attribute (typically battery consumption or latency) with very little consideration of architecture tradeoffs •  Many solutions assume that code can execute on both the mobile device and the surrogate, which is highly unlikely for the complex algorithms used by edge applications •  Many solution tightly couple mobile clients with surrogates at deployment time
  • 26. 26 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Related Work 2 Representative Cyber- Foraging Solutions Quality Attributes Basic Tradeoff Space Extended Tradeoff Space Energy Execution Time Fidelityof Results Network Capabilities Survivability Resiliency Security Flexibility Spectra [Flinn 2002] X X X Chroma [Balan 2003] X X X Surrogates [Goyal 2004] X X Slingshot [Su 2005] X X X X Cloudlets [Satyanarayanan 2009] X X MAUI [Cuervo 2010] X X X X Scavenger [Kristensen 2010] X X CloneCloud [Chun 2011] X X X Odessa [Ra 2011] X X
  • 27. 27 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Goal Identify architecture strategies that address a much broader range of quality attributes that are critical in resource-constrained environments, e.g. •  Survivability — fitness/readiness, awareness, anticipation, speed (responsiveness), agility (e.g., flexibility/ maneuver), and evolvability* •  Resiliency — robustness (e.g., redundancy), diversity, active defense, rapid reconstitution* * Source: Air Force Cyber Vision 2025. July 2012.
  • 28. 28 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Contact Information Grace A. Lewis Advanced Mobile Systems (AMS) Initiative Software Engineering Institute 4500 Fifth Avenue Pittsburgh, PA 15213-2612 USA Phone: +1 412-268-5851 Email: glewis@sei.cmu.edu WWW: http://www.sei.cmu.edu/staff/glewis/
  • 29. 29 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Selected Cyber-Foraging References [Balan 2003] Rajesh K. Balan, Mahadev Satyanarayanan, So Young Park, and Tadashi Okoshi. Tactics-based remote execution for mobile computing. In Proceedings of the 1st International Conference on Mobile Systems, Applications and Services, pages 273–286, San Francisco, CA, May 2003. [Chun 2011] Byung-Gon Chun, Sunghwan Ihm, Petros Maniatis, Mayur Naik, and Ashwin Patti. CloneCloud: Elastic execution between mobile device and cloud. In Proceedings of the European Conference on Computer Systems, Salzburg, Austria, April 2011 [Cuervo 2010] Eduardo Cuervo, Aruna Balasubramanian, Dae ki Cho, Alec Wolman, Stefan Saroiu, Ranveer Chandra, and Paramvir Bahl. MAUI: Making smartphones last longer with code offload. In Proceedings of the 8th International Conference on Mobile Systems, Applications and Services, pages 49–62, San Francisco, CA, June 2010. [Flinn 2002] Jason Flinn, SoYoung Park, and Mahadev Satyanarayanan. Balancing Performance, Energy, and Quality in Pervasive Computing. In Proceedings of the 22nd International Conference on. Distributed Computing Systems, Vienna, Austria, July 2002. [Flinn 2012] Jason Flinn. Cyber Foraging: Bridging Mobile and Cloud Computing. Synthesis Lectures on Mobile and Pervasive Computing. Vol. 7, No. 2, pp. 1-103. 2012. [Goyal 2004] Sachin Goyal and John Carter. A lightweight secure cyber foraging infrastructure for resource-constrained devices. In Proceedings of the 6th IEEE Workshop on Mobile Computing Systems and Applications, Lake Windermere, England, December 2004. [Kristensen 2010] Mads Daro Kristensen. Scavenger: Transparent deployment of efficient cyber foraging applications. In IEEE International Conference on Pervasive Computing and Communications (PerCom), pages 217–226, San Diego, CA, March 2010. [Ra 2011] Moo-Ryong Ra, Anmol Sheth, Lily Mummert, Padmanabhan Pillai, DavidWetherall, and Ramesh Govidan. Odessa: Enabling interactive perception applications on mobile devices. In Proceedings of the 9th International Conference on Mobile Systems, Applications and Services, Washington, DC, June 2011. [Satyanarayanan 2009] Mahadev Satyanarayanan, Paramvir Bahl, Ramón Cáceres, and Nigel Davies. The case for VM- based cloudlets in mobile computing. IEEE Pervasive Computing, 8(4):14–23,October–December 2009. [Su 2005] Ya-Yunn Su and Jason Flinn. Slingshot: Deploying stateful services in wireless hotspots. In Proceedings of the 3rd International Conference on Mobile Systems, Applications and Services, pages 79–92, Seattle, WA, June 2005.
  • 30. 30 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Our Publications Refereed •  Lewis, Grace; Simanta, Soumya; Novakouski, Marc; Cahill, Gene; Boleng, Jeff; Morris, Edwin; and Root, James. Architecture Patterns for Mobile Systems in Resource- Constrained Environments. 32nd Annual Military Communications Conference (MilCom 2013). Novermber, 2013. •  Simanta, Soumya; Lewis, Grace; Morris, Ed; Ha, Kiryong; and Satyanarayanan, Mahadev. A Reference Architecture for Mobile Code Offload in Hostile Environments. Proceedings of the 4th International Conference on Mobile Computing, Applications and Services (MobiCASE 2012). October 2012. •  Simanta, Soumya; Lewis, Grace; Morris, Ed; Ha, Kiryong; and Satyanarayanan, Mahadev. A Reference Architecture for Mobile Code Offload in Hostile Environments. Proceedings of the Joint 10th Working IEEE/IFIP Conference on Software Architecture & 6th European Conference on Software Architecture (WICSA/ECSA 2012). August 2012. Other •  S. Simanta, G. Lewis, E. Morris, K. Ha, and M. Satyanarayanan, "Cloud Computing at the Tactical Edge," Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, Technical Note CMU/SEI-2012-TN-015, 2012. http:// www.sei.cmu.edu/library/abstracts/reports/12tn015.cfm
  • 31. 31 Cloudlet-Based Cyber-Foraging in Resource- Constrained Environments © 2013 Carnegie Mellon University Copyright 2013 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. DM-0000299