The document provides an overview of the Open Cloud Consortium (OCC), emphasizing its member-driven structure, focus on developing standards, benchmarks, and reference implementations in cloud computing. It discusses various types of clouds and use cases, particularly in managing large data applications, alongside ongoing working groups aimed at improving interoperability and performance measurement. The OCC also sponsors events and welcomes new members to participate in its initiatives.

1. www.opencloudconsortium.orgAn Overview of the Open Cloud ConsortiumRobert GrossmanOpen Cloud ConsortiumOMG Cloud Computing Interoperability Workshop July 13, 2009

2. This talk represents my personal opinions and does not necessarily represent a consensus position of the Open Cloud Consortium or positions of any of its members.2

3. Part 1. Introduction3

4. Two Types of CloudsOn-demand resources & services over a network at the scale of a data centerOn-demand computing instances (IaaS)IaaS: Amazon EC2, S3, etc.; Eucalyptussupports many Web 2.0 applications/usersOn-demand cloud services for large data cloud applications (Large Data PaaS)GFS/MapReduce/Bigtable, Hadoop, Sector, …Manage and compute with large data (say 100+ TB)4

5. Delivering On-Demand Computing Instances5AppAppAppAppAppTable-based Data (BigTable) ServicesRelational Data Services (Pig, etc.)AppApp…Cloud Compute Services (MapReduce, Variants & Generalizations)AppAppCloud Storage ServicesFor both architectures, you are interested in filling a data center.AppAppApp

6. Activities Currently Focused Around Five Use CasesMoving an existing cloud application from Cloud 1 to Cloud 2 without changing the application.Providing surge capacity for an application on Cloud 1 using any of the Clouds 2, 3, … (without changing the application).Migrate / portSurge / burstCloud 2Cloud 1

7. Large Data Cloud Use CasesMoving a large data cloud application from one large data cloud storage service to another.Moving a large data cloud application from one large data cloud compute service to another.App 1App 2Large Data Cloud Compute Services Large Data Cloud Storage Services

8. Inter-Cloud Use CaseInter-cloud communication between two HIPAA compliant clouds.HIPPA Cloud 2HIPAA Cloud 1

9. Part 2. The Open Cloud Consortium (OCC)www.opencloudconsortium.org9

10. www.opencloudconsortium.orgOCC is a member driven organization that:Supports the development of standards.Supports the development of benchmarks.Supports reference implementations for cloud computing, preferably open source. Sponsors workshops and other events related to cloud computing.10

11. OCC MembersCurrent MembersAerospace, Cisco, MIT Lincoln Labs, Northwestern University, Open Data Group, Sector Project, University of Chicago, University of Illinois at Chicago and Yahoo.Current Contributing Members & PartnersCalIT2, Johns Hopkins University, National Lambda Rail

12. OCC StructureOCC is a consortium operated by a not-for-profit.OCC is member-driven. OCC is oriented to companies and organizations, not individuals.Governance structure is available from web site.12

13. OCC Working GroupsMembers set up working groups.Large Data Clouds Working GroupOpen Cloud Testbed Working Group.Standard Cloud Performance Measurement (SCPM)- Working collaboratively with CCIFInformation Sharing and Security Working Group.

14. Part 3. Large Data Clouds Benchmarks and Interoperability Working Group14**OCC is not affiliated with the Hadoop Project.

15. Cloud BenchmarksTo help inform purchase decisions.To help inform the design of cloud applications.To help inform cloud architecture decisions.

16. Benchmarks for Large Data CloudsUntil recently, the only benchmark used was Terasort (sorting 10 billion 100 byte records)Replaced by Gray Sort and Minute SortGray Sort tries to maximize TB / min sorted on 100 TB or more of data.Hadoop holds the current Gray Sort and Minute Sort records.Problem: sort is just one of the types of work load for analytic applications

17. MalStoneMalGen – generates synthetic data with realistic distributions.MalStone A & B – “stylized” computations that can be used as benchmarks for architectures, software and systems for large data clouds.Open source and available at code.google.com/p/malgen17

18. MalStone B18entitiessitesdk-2dk-1dktime

19. MalStone B Benchmark19

20. ThriftStoreCloud App 1Cloud App 2App n…ThriftStoreHadoop DFSKFSSector DFSEach cloud application can access any of the large data cloud storage services without changing any of its APIs.Based upon the open source Thrift project.

21. Part 4. Standard Cloud Performance Measurement (SCPM) Working Group21

22. Establish Benchmarks Appropriate for Four Use CasesMoving an application between two clouds.Obtaining burst instances from multiple cloud service providers for a private/public hybrid application.Moving a large data cloud application to another large data cloud storage service.Moving a large data cloud application to another large data cloud compute service.

23. Hierarchy of BenchmarksBenchmarks for basic operationsKernel BenchmarksBasic functions and routinesStripped down versions of programsFull applicationsGoal is to develop small suite of these benchmarks for four use cases.

24. Please Get Involved…The Standard Cloud Performance Measurement (SCPM) Working Group is just starting, please get involved.

25. Part 5. OCC Sponsored Events25

26. OCC/CCIF Jointly Sponsored EventsInternational Summit on Governmental Cloud ComputingOttawa's Westin Hotel on October 6, 2009Cloud Computing and Its Applications 2009Chicago on October 20-21, 2009

27. We Welcome New MembersCompanies and organizations are welcome to join the Open Cloud Consortium (OCC)www.opencloudconsortium.org/membership.htmlJoin one of our working groupsLarge Data Clouds Working GroupStandard Cloud Performance Measurement (SCPM) Working GroupInformation Sharing & Security Working Group

28. For More Informationinfo@opencloudconsortium.orgwww.opencloudconsortium.org