Collaborative eResearch in a Social Cloud


Published on

Social networks provide a useful basis for enabling collaboration among groups of individuals. This is applicable not only to social communities but also to the scientific community. Already scientists are leveraging social networking concepts in projects to form groups, share information and communicate with their peers. For scientific projects which require large computing resources, one useful aspect of collaboration is the sharing of computing resources among project members. A social network provides an ideal platform to share these resources. This paper introduces a framework for Social Cloud computing with a view towards collaboration and resource sharing within a scientific community. The architecture of a Social Cloud, where individuals or institutions contribute the capacity of their computing resources by means of Virtual Machines leased through the social network, is outlined. Members of the Social Cloud can contribute, request, and use Virtual Machines from other members, as well as form Virtual Organizations among groups of members.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Good to have the general public participate in science and research Boinc until recently had more compute available than the top supercomputer Tianchi-III. Only recently surpassed by the step change at he top of the SC table. The resources available through facebook are massive.
  • Tools that are designed for NON EXPERT users!!!
  • Social Networks model relationships – can also model collaborations. Authentication e.g. Facebook Connect, rather than Open ID Automated Service Provisioning ENvironment (ASPEN) [6] exposes applications hosted by Cloud providers to user communities in Facebook. The focus of ASPEN is exposing applications and sharing data within an enterprise through an intuitive and integrated environment, however this could also be applied to a scientific domain does not proviion the infrastructure via FB. OpenSocial & OpenId, used by most social networking sites, and Facebook ’ s bespoke application framework
  • One role for a social cloud is in the early stages of research when the costs of dedicated research infrastructure would be prohibitive. It is light weight, The value of social networking has been observed in multi- ple scientific domains as a means of facilitating collaboration. Increasingly social networks are being used to coordinate re- search communities, two such examples are MyExperiment [3] for biologists and nanoHub [4] for the nanoscience community. MyExperiment provides a virtual research environment where collaborators can share and execute scientific work- flows. nanoHub allows users to share data and transparently execute applications on distributed resource providers such as TeraGrid. While similar to a Social Cloud, MyExperiment and nanoHub each have specific sharing focuses and build their own proprietary social network stack.
  • User-specific groups, defined by relationship types, are shown in the context of a Social network. In this example group A is composed of only co-worker members, whereas group B is formed by family members and group C includes only friends. Clearly the level of trust and mechanisms for social correction (identifying incentives and disincentives for users to participate) differ between groups. This figure also highlights that Social Clouds are not mutually exclusive, that is, users may be simultaneously members of multiple Social Clouds. Whereas a VO is often associated with a particular application or activity, and is often disbanded once this activity completes, a group is longer lasting and may be used in the context of multiple applications or activities.
  • Lower/remove barriers to usability. To a larger, non expert base of users. A minimal move away from comfort zone, make the facilities fit the users rtaher than the users fit the facilities. There are many hurdles struck by putting users pushed into/on raw infrastructure, certificates particularly problematic. Old unfamilar techniques such as command lines etc. intimidating for non expert users. But the point of the social cloud is uniform over resources due to being within Social Network.
  • The Social Storage cloud (SaaS the first implementation), The Social Cloud for public eScience, and The Social Cloud for Collaborative Scientific Computation (IaaS or PaaS).
  • FB renders the interface, provides the social connectivity data – computation is done outside FB on our own servers.
  • Take it or leave it fixed price Used MDS for resource discovery.
  • Attributes of a Second Price Sealed Bid auction Encourages truth telling Lowers communication overhead We weren ’ t really happy with the economic models underpinning resource trading between friends. But this was a first attempt at implementing a social cloud, and in this it was successful, we built an application in FB that enabled FB users to trade storage with their friends. We didn ’ t implement obvious facilities such as replication as that would have been a trivial extension had we gone beyond the protoctype.
  • Talk about why facebook. Maturity. Challenges, API lacks stability. Baby steps! Obviously the end goal would be SaaS – members can provide IaaS and SaaS freely within the VRE.
  • OCCI = Open Cloud Computing Interface, draft standard. Used for IOC and between user resources and application server.
  • ** most of our recent work has been on the scheduler, now 2 layer meta scheduler that schedules from multiple virtual clusters. Open Cloud Computing Interface (OCCI) adopted the OCCI draft specification in the SoCC for communication between user resources and the application server, as well as for communication between user resources.
  • Also an Android app.
  • With spot pricing, both for consumption and provision of Compute Shares. Credits here = shares. Experiment naturally resource limited – potentially an artifical situation. Each VO consumes credits. Members earn credits through providing resources that can then be used in turn for computation. Global price. After preparation, the AuverGrid trace had the following characteristics: • Number of users: 401 • Number of VOs: 8 (VO1: 158 users, VO2: 107 users, VO3: 16 users, VO4: 47 users, VO5: 46 users, VO6: 11 users, VO7: 10 users, VO8: 8 users) • Number of computes: 339314
  • Asym, researchers vs public. Not always clear what public gets out of it, hence incentives. Sym, peers, collaborations, participants on equal footing. Group/VRE ower can kick out misbehaving members.
  • Collaborative eResearch in a Social Cloud

    1. 1. The Social Cloud forCollaborative Scientific ComputationAshfag Thaufeeg, Kris Bubendorfer and Kyle Chard eScience 2011
    2. 2. Background  Increasingly research requires access to computation and storage of scientific data.  This is often beyond the resources that individual scientists or individual organisations can provide.  Access to national fixed infrastructure is often limited to selected larger projects and well established researchers.  Research conducted on commercial clouds is potentially expensive.  Large usability barriers for non expert users, i.e. command line etc.  Coordination within existing projects and establishing new collaborations is difficult, tools and systems vary across members and organisations. 22eScience 2011
    3. 3. Researchers & Social Networks  Multi-institute collaboration is desirable, but  communication and coordination is hard in practice, and  face to face meetings tend to occur only sporadically, i.e., at conferences and workshops.  Social networks can potentially provide a natural basis for collaboration, they:  decrease the effort needed to initiate a new collaborative effort, by using tools and infrastructure already familiar to and understood by most people,  Can facilitate discovery of other scientists working on similar projects, through relationships, feeds and invitations. 33eScience 2011
    4. 4. Social Networks Are based around pre-existing relationships,  i.e. your friends and colleagues, friends of friends… Have a pre-existent fabric of trust inherently interwoven into the network  How many of your friends do you not trust?  Friends can be grouped based on interest and level of trust. Many applications now use social networks as a platform for:  Authentication e.g. Facebook Connect  Application Portals e.g. ASPEN and PolarGrid projects Established application APIs 4
    5. 5. Researchers & Social Networks  But, the social network itself is not a complete collaboration solution.  collaboratorsalso often have resources they wish to share dynamically for the duration of a project, and  resources may include documents, media, data, services, compute, storage, and so on.  Currently this is a difficult process that  Requires access to unfamiliar tools and systems outside the social network,  manual (reciprocal) user account creation etc.  sharing involves access, rights, accounting and auditing – that increases the overhead of project management. 55eScience 2011
    6. 6. The Social Cloud  The Social Cloud is a new way of thinking about providing cloud like resources for eResearch. A Social Cloud is a resource and service sharing framework utilizing relationships established between members of a social network.  Social networks (such as Facebook, Linkedin etc):  provide considerable infrastructure, including authorisation and APIs for external apps.  contain many tools for managing relationships, forming groups and defining security policies.  Include basic incentive mechanisms to moderate behaviour.  This is ‘free’ user infrastructure we can take advantage of. 66eScience 2011
    7. 7. The Social Cloud  What is different about the social cloud?  The social network comes first: It is not a cloud or collaboration environment extended with a social network, it is a social network extended with cloud functionality.  The people and their networks form the basic infrastructure, the resources they access and shared are formed around their unique social graph.  In some ways this turns the provision of research infrastructure on its head. Rather than fitting the researcher to the infrastructure, we fit the infrastructure to the researcher.  Users fulfill all roles, from provision of resources, management of collaborations through to use of services and computation - all of which are performed with familiar Social Networking tools.  The researchers choose how to delegate resources between the 77eScience 2011 groups within which they collaborate.
    8. 8. Groups in a Social Cloud  We exploit the analogy between social networking groups and dynamic virtual organizations.  Groups like virtual organizations have an intent, membership and policies that define sharing (in social networks this relates to photos, media, etc).  We extend the concept of sharing to include resources such as, compute, storage and other services.  This forms a Virtual Research Environment (VRE) within the SoCC.  We explicitly bind a VRE/VO to a Facebook group within the SoCC. 88eScience 2011
    9. 9. Social Cloud and Groups Co-workers Family B Friends A C 99eScience 2011
    10. 10. Advantages  We believe that a cloud architecture integrated with a social network can benefit the scientific community in the following ways:  Lowers usability barriers, a minimal move away from a non CS expert’s comfort zone.  It allows researchers to share resources for the duration of collaboration.  Such collaborations are light-weight and dynamic, resources can be delegated, removed and accessed using the SN group structure.  Authorization and access control take place transparently for the owner and users.  no visible certificate management,  single sign on and  seamless integration with SN functionality. 10 10eScience 2011
    11. 11. Advantages  A Social Cloud could also be considered a:  resource fabric overlay over a Social Network, or,  generalized and extended form of crowd sourcing.  It is not intended to replace specialist HPC infrastructure for big science.  The social cloud is intended to be smaller, more general, adhoc and dynamic.  Created and destroyed on need, ideal for early work, initial studies, smaller projects etc. 11 11eScience 2011
    12. 12. The Social Cloud Project  Collaboration between researchers at:  Victoria, Cardiff, Chicago and KiT  There are a number of implementations exploring different aspects of the Social Cloud domain:  The Social Storage cloud (the first implementation),  The Social Cloud for public eScience, and  The Social Cloud for Collaborative Scientific Computation.  In addition projects have been looking at:  Business models for the social cloud,  Incentives, economies, gamification to underpin sharing.  The Social Cloud project began in 2009. 12 12eScience 2011
    13. 13. Social Storage Cloud SaaS: Simple Storage Service  Implemented as a Facebook application.  first experiments with a socially oriented market. Agreement 13
    14. 14. Posted Price  Enables interactions based upon active trading and or collaborative decisions  Intuitively facilitates reciprocal resource exchange  Current “norm” in industry solutions Social Cloud MDS User UR Capaci Price ID L ty User1 100MB 5 Storag User2 500MB 10 e Storag Storag User3 5GB 7 e e 14
    15. 15. Dynamic Auctions Auction:  Enables dynamic participant pairing  Sealed bid second price reverse auction Could be extended to any other auction mechanism 15
    16. 16. SoCC Architecture  A IaaS platform constructed and accessed via a facebook application.  Schedules VMs across a VRE (group/VO)  Built on top of Nimbus  Handles the VM lifecycle.  Facebook interface to:  Group/VRE management.  Create, join, leave a VRE  Delegate resources to a VRE  Monitor your VMs  Monitor your delegated resources.  Facebook interface, used for groups, tweets, feeds, and other project personnel coordination. 16 16eScience 2011
    17. 17. SoCC Architecture 17 17eScience 2011
    18. 18. SoCC Architecture  Application Server (logical):  The application server hosts a web application built using the Facebook public API that renders directly inside the Facebook UI, thereby giving the impression of seamless integration to users.  The application server is responsible for collecting registered user data from Facebook through the graph API and providing an interface to the SoCC.  Within the logical Application Server there is also an Image Store, Scheduler and Context Broker. 18 18eResearch 2011
    19. 19. SoCC Architecture  Image store: contains the base set of OS images for the VMs. These images are pre-packaged to provide quick instantiation for users who do not want to prepare their own images  Scheduler: schedules submitted VMs to one of the available clusters** depending on resources owed and available.  Context Broker: contextualizes the VMs, including the setting up ssh keys and user accounts so that members belonging to the same VO (social network group) can connect to the VM.  OCCI 19 19eResearch 2011
    20. 20. VM control 20 20eScience 2011
    21. 21. VM startup times 1000 created->running 900 received->created 800 Total Time (seconds) 700 600 500 400 300 200 100 0 ylinux-op on-1 ubuntu-op on-1 ylinux-op on-2 ubuntu-op on-2 21 21eScience 2011
    22. 22. Sharing Different contexts require different models, we are exploring a range of allocation tecniques in the social cloud context:  Storage– credits  Compute – shares or slices  Volunteer – gamification, incentives (talk tomorrow) 22
    23. 23. Simulation Results 23 23eResearch 2011
    24. 24. Summary  A new cloud paradigm, the social cloud.  The Social Cloud for Collaborative Computation  Uses existing Social Network infrastructure.  Scientists contribute to groups, both with resources and postings (social media) forming VREs.  Open flexible, extensible. Global.  Access to resources based on implicit trust. 24 24eScience 2011