This document summarizes a presentation about evaluating the performance of the OpenStack Swift object storage system using the COSBench benchmark. Key points include:
- COSBench is an open source benchmark developed by Intel to measure the performance of cloud object storage services like OpenStack Swift and Amazon S3.
- A case study was conducted using COSBench to evaluate Swift performance on different hardware configurations and identify bottlenecks.
- The results showed that for small object operations the proxy CPU was the bottleneck, while for large object operations disk and network bandwidth were limiting factors.
- Going forward the presenter plans to support additional storage services with COSBench and make it more openly available to help optimize
The document describes COSBench, a benchmark tool for evaluating the performance of cloud object storage services. It provides an overview of COSBench's key components, including its configurable workload definition file, controller for managing tests, and drivers for generating load. The document also shares sample results from using COSBench to measure the throughput and response times of OpenStack Swift in different configurations. It found that the proxy node's CPU was the bottleneck for larger workloads on one setup. The goal is to open source COSBench to help storage providers optimize performance.
The document discusses COSBench, a benchmark tool developed by Intel to evaluate the performance of cloud object storage services. It describes key components of COSBench including its configurable workload definition, drivers to generate load, and a web console to view results. The document also uses COSBench to analyze the performance of OpenStack Swift, finding that insufficient processing power can throttle overall performance.
The document discusses several case studies of companies using Hadoop and MapReduce for large-scale data processing problems. Common themes across the case studies include using Cascading and AWS Elastic MapReduce to develop multi-stage ETL pipelines to extract, transform and analyze large amounts of daily log and user behavior data. The case studies demonstrate how Hadoop can be used to gain business insights through modeling, machine learning, and predictive analytics on petabytes of user data.
Building Scale Free Applications with Hadoop and Cascadingcwensel
Many more applications are suitable to be built on Apache Hadoop than many developers realize.
In this presentation, we hope to give attendees enough information on how Hadoop works, how MapReduce can be leveraged to perform common and well understood data processing operations, and how the Cascading open-source project helps developers rapidly build sophisticated Hadoop applications that can be simply tested locally and executed remotely.
Presented at OSBridge June 2009.
[AzurePT] Desenvolvimento para o Windows Azure: Diferença para o developerVitor Tomaz
The document discusses cloud computing models including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). It also provides diagrams illustrating the architecture and components of the Windows Azure cloud platform, including roles, virtual machines, storage, networking, load balancing, and fabric controllers.
Windows Sql Azure Cloud Computing PlatformEduardo Castro
The document provides an overview of a presentation on Windows Azure and SQL Azure. It discusses the key components of the Windows Azure platform including Windows Azure, SQL Azure, and AppFabric. It also summarizes some of the core capabilities like flexible application hosting, storage services, SQL Azure as a database service, and connecting applications. The document outlines the global availability and billing models for the Windows Azure platform.
ApacheCon EU 2009 Tales from the front line - ActiveMQ ServiceMix and CXFAdrian Trenaman
This document discusses how ActiveMQ, ServiceMix, Camel, and CXF are used to solve real-world integration problems. It provides three examples: a healthcare system that enables access to patient data via PDAs, a retail pharmacy distribution and warehouse management solution, and integrating backend systems for a telecommunications company. The document highlights how these open source tools provide flexible, reliable, and scalable integration capabilities for enterprise applications.
This document discusses cloud computing models including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). It provides an overview of Windows Azure and its core services including compute, storage, SQL Azure, AppFabric Access Control, and Service Bus. Finally, it outlines the resources included with different Microsoft subscriptions levels for developing applications on Windows Azure.
The document describes COSBench, a benchmark tool for evaluating the performance of cloud object storage services. It provides an overview of COSBench's key components, including its configurable workload definition file, controller for managing tests, and drivers for generating load. The document also shares sample results from using COSBench to measure the throughput and response times of OpenStack Swift in different configurations. It found that the proxy node's CPU was the bottleneck for larger workloads on one setup. The goal is to open source COSBench to help storage providers optimize performance.
The document discusses COSBench, a benchmark tool developed by Intel to evaluate the performance of cloud object storage services. It describes key components of COSBench including its configurable workload definition, drivers to generate load, and a web console to view results. The document also uses COSBench to analyze the performance of OpenStack Swift, finding that insufficient processing power can throttle overall performance.
The document discusses several case studies of companies using Hadoop and MapReduce for large-scale data processing problems. Common themes across the case studies include using Cascading and AWS Elastic MapReduce to develop multi-stage ETL pipelines to extract, transform and analyze large amounts of daily log and user behavior data. The case studies demonstrate how Hadoop can be used to gain business insights through modeling, machine learning, and predictive analytics on petabytes of user data.
Building Scale Free Applications with Hadoop and Cascadingcwensel
Many more applications are suitable to be built on Apache Hadoop than many developers realize.
In this presentation, we hope to give attendees enough information on how Hadoop works, how MapReduce can be leveraged to perform common and well understood data processing operations, and how the Cascading open-source project helps developers rapidly build sophisticated Hadoop applications that can be simply tested locally and executed remotely.
Presented at OSBridge June 2009.
[AzurePT] Desenvolvimento para o Windows Azure: Diferença para o developerVitor Tomaz
The document discusses cloud computing models including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). It also provides diagrams illustrating the architecture and components of the Windows Azure cloud platform, including roles, virtual machines, storage, networking, load balancing, and fabric controllers.
Windows Sql Azure Cloud Computing PlatformEduardo Castro
The document provides an overview of a presentation on Windows Azure and SQL Azure. It discusses the key components of the Windows Azure platform including Windows Azure, SQL Azure, and AppFabric. It also summarizes some of the core capabilities like flexible application hosting, storage services, SQL Azure as a database service, and connecting applications. The document outlines the global availability and billing models for the Windows Azure platform.
ApacheCon EU 2009 Tales from the front line - ActiveMQ ServiceMix and CXFAdrian Trenaman
This document discusses how ActiveMQ, ServiceMix, Camel, and CXF are used to solve real-world integration problems. It provides three examples: a healthcare system that enables access to patient data via PDAs, a retail pharmacy distribution and warehouse management solution, and integrating backend systems for a telecommunications company. The document highlights how these open source tools provide flexible, reliable, and scalable integration capabilities for enterprise applications.
This document discusses cloud computing models including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). It provides an overview of Windows Azure and its core services including compute, storage, SQL Azure, AppFabric Access Control, and Service Bus. Finally, it outlines the resources included with different Microsoft subscriptions levels for developing applications on Windows Azure.
The document provides instructions for students on note taking using the Cornell note taking system. It details the materials and assignments needed for an upcoming science lab, including completing a Mission Possible lab packet and Cornell notes. Students are given directions on organizing their notes, including setting up the paper, taking notes in the Cornell format, and reflecting on and reviewing the notes. Key strategies for effective note taking like highlighting, abbreviations, and symbols are also outlined.
Edgar Allan Poe was a 19th century American author known for works of the Gothic genre. The Gothic style features bleak or remote settings, macabre or violent incidents, and characters experiencing psychological or physical torment. Poe aimed for a "single effect" in his writing, focusing on creating an intense mood or sensation. The document provides background on Poe and the Gothic style as an introduction for students to analyze his short story "The Fall of the House of Usher".
ISN is a non-profit organization that provides financial services and matches businesses with investors worldwide. They offer assistance throughout the financing process, including developing business plans and locating loans, equity, venture capital, and other funding. ISN maintains confidentiality and aims to streamline the funding process for their clients.
This document provides an introduction and overview of an international business project presented by a group of students to their professor. It includes the names and student IDs of the group members, as well as brief sections on globalization, global companies and global managers. Specific companies discussed include Toyota, McDonald's, PepsiCo, and managers such as Jack Welch and Indian businessman Lakshmi Mittal.
This document provides an introduction to data structures and algorithms. It defines an algorithm as a set of steps to solve a problem and describes data structures as organized ways to store and access data. Common data structures include arrays, stacks, queues, linked lists, trees and graphs. Abstract data types define the fundamental operations on data objects in a data structure independently of their implementation. Linear data structures like arrays represent lists in one dimension while non-linear structures represent two-dimensional relationships. Stacks follow the last-in, first-out principle with push and pop operations, while queues are first-in, first-out. Selection of the appropriate data structure depends on the application.
- The analyst remains neutral on JDS Uniphase Corp. (JDSU) and Finisar Corp. (FNSR) as optical component data points remain mixed, with no clear evidence that the demand pause has ended.
- Recent data from Infonetics showed optical systems demand rebounded in Q2, driven by spending increases from Huawei and ZTE, though the timing of when component vendors will see this benefit is unclear.
- FN Services posted revenue above estimates for its June quarter and cited "drop-in" orders in the last two weeks, indicating leaner inventory levels, but its guidance was slightly below consensus estimates.
- Overall, the analyst waits for more positive signals from channel
iLiving App is a new form of learning. With iLiving App (The Inspired Living Application) you have professional and personal development direct on your mobile device. Wherever you are you can improve your knowledge with iLiving App. This is the future of learning
La Unión Europea ha propuesto un nuevo paquete de sanciones contra Rusia que incluye un embargo al petróleo. El embargo se aplicaría gradualmente durante seis meses para el crudo y ocho meses para los productos refinados. Además, se desconectaría al mayor banco ruso, Sberbank, de SWIFT y se prohibirían las transmisiones de tres canales estatales rusos.
This very short document does not contain enough content to create a meaningful 3 sentence summary. The document consists of only 3 letters - THE END - without any other words or context provided.
Street art began as art created in public spaces and has evolved over time. It originated from cave paintings and graffiti tags, and became prominent in New York City in the late 1800s. Street art allows self-expression and comes in many forms from political messages to cartoon-like doodles. It has faced criticism for being illegal vandalism, though some see it as a form of art. Notable street artists include Jon Burgerman, known for vibrant cartoon-like doodles; Keith Haring, who used lines to create contained works; and JR, who pastes large black and white portraits in public spaces.
Salespeople lose orders when they are outsold by competitors. To win competitive opportunities, salespeople need to develop a Winning Sales Plan (WSP) that involves assessing the business context, setting a sales objective, determining the opportunity score based on value and political alignment, and outlining a competitive strategy. The WSP framework provides a process for understanding customer needs and outlining how a company can best meet those needs to defeat competitors.
In dit document wordt een weergave gegeven van de gemiddelde kosten van een aanbesteding naar procedure.
Ook is een onderbouwing gegeven voor de besparing die de dienstverlening van de Aanbestedingswijzer geeft voor onderhandse aanbestedingen.
The document lists produce prices and discusses the priceless benefits of growing your own produce while establishing connections to the environment, yourself, and community. It then provides details on an urban gardening organization such as its founding, size, and capacity for new gardens each year. The summary concludes with the organization needing funds to continue growing sustainably by maintaining donor relationships and enthusiasm for urban gardening while building a consistent brand image.
This document introduces COSBench, a benchmark tool developed by Intel to measure the performance of cloud object storage services. It describes key components of COSBench including the workload configuration, performance metrics collected, and a web console for managing tests. The document also provides a case study using COSBench to evaluate the performance of OpenStack Swift, an open source cloud storage system, by describing the entities and architecture of OpenStack Swift and the test configuration used.
The document discusses Netflix's cloud architecture on Amazon Web Services (AWS). It aims to be faster, scalable, available and allow developers to work more productively. Some key points are moving from a central SQL database to distributed NoSQL stores, replacing sticky in-memory sessions with a shared cache, and optimizing for latency tolerance over chatty protocols. The architecture also focuses on layered service interfaces over tangled code and instrumenting services rather than code.
The document provides instructions for students on note taking using the Cornell note taking system. It details the materials and assignments needed for an upcoming science lab, including completing a Mission Possible lab packet and Cornell notes. Students are given directions on organizing their notes, including setting up the paper, taking notes in the Cornell format, and reflecting on and reviewing the notes. Key strategies for effective note taking like highlighting, abbreviations, and symbols are also outlined.
Edgar Allan Poe was a 19th century American author known for works of the Gothic genre. The Gothic style features bleak or remote settings, macabre or violent incidents, and characters experiencing psychological or physical torment. Poe aimed for a "single effect" in his writing, focusing on creating an intense mood or sensation. The document provides background on Poe and the Gothic style as an introduction for students to analyze his short story "The Fall of the House of Usher".
ISN is a non-profit organization that provides financial services and matches businesses with investors worldwide. They offer assistance throughout the financing process, including developing business plans and locating loans, equity, venture capital, and other funding. ISN maintains confidentiality and aims to streamline the funding process for their clients.
This document provides an introduction and overview of an international business project presented by a group of students to their professor. It includes the names and student IDs of the group members, as well as brief sections on globalization, global companies and global managers. Specific companies discussed include Toyota, McDonald's, PepsiCo, and managers such as Jack Welch and Indian businessman Lakshmi Mittal.
This document provides an introduction to data structures and algorithms. It defines an algorithm as a set of steps to solve a problem and describes data structures as organized ways to store and access data. Common data structures include arrays, stacks, queues, linked lists, trees and graphs. Abstract data types define the fundamental operations on data objects in a data structure independently of their implementation. Linear data structures like arrays represent lists in one dimension while non-linear structures represent two-dimensional relationships. Stacks follow the last-in, first-out principle with push and pop operations, while queues are first-in, first-out. Selection of the appropriate data structure depends on the application.
- The analyst remains neutral on JDS Uniphase Corp. (JDSU) and Finisar Corp. (FNSR) as optical component data points remain mixed, with no clear evidence that the demand pause has ended.
- Recent data from Infonetics showed optical systems demand rebounded in Q2, driven by spending increases from Huawei and ZTE, though the timing of when component vendors will see this benefit is unclear.
- FN Services posted revenue above estimates for its June quarter and cited "drop-in" orders in the last two weeks, indicating leaner inventory levels, but its guidance was slightly below consensus estimates.
- Overall, the analyst waits for more positive signals from channel
iLiving App is a new form of learning. With iLiving App (The Inspired Living Application) you have professional and personal development direct on your mobile device. Wherever you are you can improve your knowledge with iLiving App. This is the future of learning
La Unión Europea ha propuesto un nuevo paquete de sanciones contra Rusia que incluye un embargo al petróleo. El embargo se aplicaría gradualmente durante seis meses para el crudo y ocho meses para los productos refinados. Además, se desconectaría al mayor banco ruso, Sberbank, de SWIFT y se prohibirían las transmisiones de tres canales estatales rusos.
This very short document does not contain enough content to create a meaningful 3 sentence summary. The document consists of only 3 letters - THE END - without any other words or context provided.
Street art began as art created in public spaces and has evolved over time. It originated from cave paintings and graffiti tags, and became prominent in New York City in the late 1800s. Street art allows self-expression and comes in many forms from political messages to cartoon-like doodles. It has faced criticism for being illegal vandalism, though some see it as a form of art. Notable street artists include Jon Burgerman, known for vibrant cartoon-like doodles; Keith Haring, who used lines to create contained works; and JR, who pastes large black and white portraits in public spaces.
Salespeople lose orders when they are outsold by competitors. To win competitive opportunities, salespeople need to develop a Winning Sales Plan (WSP) that involves assessing the business context, setting a sales objective, determining the opportunity score based on value and political alignment, and outlining a competitive strategy. The WSP framework provides a process for understanding customer needs and outlining how a company can best meet those needs to defeat competitors.
In dit document wordt een weergave gegeven van de gemiddelde kosten van een aanbesteding naar procedure.
Ook is een onderbouwing gegeven voor de besparing die de dienstverlening van de Aanbestedingswijzer geeft voor onderhandse aanbestedingen.
The document lists produce prices and discusses the priceless benefits of growing your own produce while establishing connections to the environment, yourself, and community. It then provides details on an urban gardening organization such as its founding, size, and capacity for new gardens each year. The summary concludes with the organization needing funds to continue growing sustainably by maintaining donor relationships and enthusiasm for urban gardening while building a consistent brand image.
This document introduces COSBench, a benchmark tool developed by Intel to measure the performance of cloud object storage services. It describes key components of COSBench including the workload configuration, performance metrics collected, and a web console for managing tests. The document also provides a case study using COSBench to evaluate the performance of OpenStack Swift, an open source cloud storage system, by describing the entities and architecture of OpenStack Swift and the test configuration used.
The document discusses Netflix's cloud architecture on Amazon Web Services (AWS). It aims to be faster, scalable, available and allow developers to work more productively. Some key points are moving from a central SQL database to distributed NoSQL stores, replacing sticky in-memory sessions with a shared cache, and optimizing for latency tolerance over chatty protocols. The architecture also focuses on layered service interfaces over tangled code and instrumenting services rather than code.
Amazon Redshift is a fast, powerful, fully managed data warehouse service that allows for petabyte-scale data warehousing at very low costs. It uses columnar storage and data compression techniques to dramatically reduce I/O and allow for very fast query performance. Redshift automatically provisions clusters on optimized hardware and scales easily from terabytes to petabytes of data with no downtime. It integrates with popular BI tools and simplifies tasks like provisioning, administration, backup/restore and scaling the data warehouse.
SQL Server 2008 Fast Track Data Warehouse 2.0
This was a presentation to the Silicon Valley SQL Server User Group in February 2010.
Speaker: Phil Hummel of WinWire Technologies
Presentation developed by Bruce Campbell
Western Region Data Warehouse Specialist, Microsoft
For more information about the SQL Server User Group, contact Mark Ginnebaugh, President of DesignMind, at mark@designmind.com
source: http://www.sfbayacm.org/?p=1394
The specifics of a cloud’s computing architecture may have an impact on application design. This is particularly important in Infrastructure as a Service (IaaS) cloud environments.
This presentation analyzes aspects of the Amazon EC2 IaaS cloud environment that differ from a traditional datacenter and introduces general best practices for ensuring data privacy, storage persistence, and reliable DBMS backup. Best practices for application robustness and scalability on demand are reviewed and are especially significant in leveraging the full potential of an IaaS cloud. The need for a cloud application management and configuration system is briefly reviewed and two alternate approaches to cloud application management are described (RightScale and Kaavo).
The document discusses a content repository, which is a generic API for content storage that provides CRUD functionality as well as versioning, transactions, and search capabilities. It describes how a content repository enforces simplicity, encourages standardization, and improves scalability. Examples of content repository implementations are provided, including Apache Jackrabbit and eXo Platform. Key features of content repositories are explored such as the content model, repository structure with workspaces and nodes/properties, and node type definitions.
The document discusses building open source clouds using Apache CloudStack, an open source cloud computing platform. It provides an overview of CloudStack's architecture and features, including its flexible IaaS capabilities. The presentation aims to introduce developers to CloudStack and various tools for building and automating open source clouds.
OpenStack is an open source cloud operating system that controls pools of compute, storage, and networking resources throughout a datacenter, managed through APIs, dashboards, and command-line tools. It includes several core projects like Nova for compute resources, Swift for object storage, and Glance for images. OpenStack is developed as open source by an international community of over 1,600 individuals from over 100 companies and is used in production private and public clouds by companies in industries like financial services, e-commerce, and more.
WSO2 Carbon and WSO2 Stratos Summer Release Roundup WSO2
- The webinar covered the upcoming releases of WSO2 Carbon and Stratos in summer 2012, including new products, features, and capabilities.
- Carbon 4.0 includes improvements to deployment synchronization, performance, and multi-tenancy, as well as new products like API Manager and Storage Server.
- Stratos 2.0 features a new cartridge model for multiple languages/frameworks, support for additional IaaS providers, and an enhanced management console.
- Both releases focus on improved scalability, manageability, and a modular approach to building and deploying middleware components.
Windows Azure + PHP, Java, MySQL, LOLCODE?
This session explores the interop possiblities Windows Azure has to offer starting from the architecture that leverages The Freedom of Choice and ending up with relevant examples and usage scenarios.
The document provides an overview of tuning the Oracle E-Business Suite environment. It discusses tuning the applications tier, concurrent manager, client tier and network, database tier, and applications. Specific tips are provided for each area, such as upgrading technology stacks, minimizing network traffic, using specialized managers, enabling SQL tracing and profiling, and isolating the database and applications tiers on a private network.
The document discusses HP Networking's Intelligent Management Center (IMC), highlighting its ability to provide single pane management of HP networking devices, integrate IMC modules for data center orchestration, and provide efficient architectures, complete visibility, and lower total cost of ownership. The IMC aims to simplify network management by addressing common challenges around having too many management tools, network instability from changes, lack of network visibility and control, and controlling access.
RMG206 Introduction to Amazon Elastic Beanstalk - AWS re: Invent 2012Amazon Web Services
AWS Elastic Beanstalk is a service that allows developers to quickly deploy and manage applications in the AWS cloud without worrying about the underlying infrastructure. It provisions the required resources such as load balancers, EC2 instances, Auto Scaling, and storage and deploys the application. It also handles automatic scaling and high availability of the application. Developers can focus on coding applications while Elastic Beanstalk handles the deployment and management of the infrastructure.
The document discusses the "Madison" SQL Server parallel data warehouse architecture from Microsoft. It covers concepts of symmetric multiprocessing (SMP) vs massively parallel processing (MPP) architectures. It then presents Microsoft's "FastTrack" reference architectures which provide validated hardware configurations for data warehousing. These balanced configurations are designed to optimize performance and scalability by avoiding bottlenecks. They also reduce costs and risks compared to custom-built systems.
This document discusses building a disaster recovery solution using OpenStack. It outlines the goals of providing a configurable warm standby solution with a known recovery point objective (RPO) and reduced recovery time objective (RTO) to minimize business impact. The document describes challenges in replicating an application across clouds while preserving the running environment. It provides an overview of the disaster recovery process using OpenStack, including taking snapshots, creating volumes, and mounting new instances. Optimizations discussed include incremental backups and parallel transfers to improve large data transfer speeds across cloud datacenters for disaster recovery.
The document discusses VNG Corporation's use of OpenStack for its infrastructure platform. VNG is a Vietnamese game publisher and developer that also operates online media, social network, and e-commerce platforms. Its IRD department is researching and building an infrastructure on OpenStack to enhance business operations and meet new technology needs. The document provides an overview of OpenStack architecture, networking, deployment models using Ceph storage, and the monitoring component Ceilometer. It aims to introduce OpenStack and how VNG is leveraging it to power its cloud infrastructure.
The document discusses VNG Corporation's use of OpenStack for its infrastructure platform. VNG is a Vietnamese game publisher and developer that also operates online media, social network, and e-commerce platforms. Its IRD department is researching and building an infrastructure on OpenStack to enhance business operations and meet new technology needs. The document provides an overview of OpenStack architecture, networking, deployment models using Ceph storage, and the monitoring component Ceilometer. It aims to introduce OpenStack and how VNG is leveraging it to power its cloud infrastructure.
Apache CloudStack is an open source cloud computing platform that provides infrastructure as a service. It was originally developed by Citrix and is now an Apache incubator project. It is hypervisor-agnostic and supports platforms like Xen, KVM, and VMware. It provides services for compute, storage, networking and a web-based user interface.
Flink Forward SF 2017: Malo Deniélou - No shard left behind: Dynamic work re...Flink Forward
The Apache Beam programming model is designed to support several advanced data processing features such as autoscaling and dynamic work rebalancing. In this talk, we will first explain how dynamic work rebalancing not only provides a general and robust solution to the problem of stragglers in traditional data processing pipelines, but also how it allows autoscaling to be truly effective. We will then present how dynamic work rebalancing works as implemented in the Google Cloud Dataflow runner and which path other Apache Beam runners link Apache Flink can follow to benefit from it.
This session walks through approaches for large-scale media processing scenarios. We cover hybrid and cloud-based transcoding, file transfer, media preparation, and media management. We expect attendees to come away with an understanding of best practices for architecting and deploying hybrid and cloud-based systems for media processing.
This document discusses containers and related technologies:
1. Containers provide isolated, portable environments for running applications and their dependencies. Docker is a popular container platform that packages applications into containers using Linux kernel features like namespaces and cgroups.
2. The Open Container Initiative (OCI) aims to develop standards around container formats and runtime. Technologies like Docker, rkt, and AppC implement the OCI specifications.
3. Container orchestration systems like Kubernetes and Mesos manage the deployment and lifecycles of containers at scale across clusters of hosts.
The lessons I learned is that Open source quickly becomes the natural choice wherever commoditization is happening in the software stack. Thus we expect business-to-business open source, which is already a significant trend in recent history, to become an increasingly common form of open source collaboration. Companies who understand the ground rules of business-to-business open source will be better positioned to identify and take advantage of open source opportunities in the competitive spaces that they share with other companies.
So I will share why open strategy is import for the enterprise. And how to do contributions for the open source projects n today’s topic.
This document discusses Docker and OpenStack integration. It begins with introductions to OpenStack and Docker, explaining that OpenStack is an open source cloud operating system and Docker is a container-based virtualization framework. It then discusses how Docker can be used with OpenStack, with Nova supporting Docker as a hypervisor starting in Havana. It concludes with mentioning a demo of Docker + OpenStack integration and inviting questions.
TryStack.cn is a non-profit OpenStack testbed and community project in China that aims to promote OpenStack adoption. It operates the largest OpenStack testbed in China with hardware from various vendors. TryStack.cn provides reference architectures, best practices, and contributes code back to the community. It also organizes OpenStack meetups and training to help grow the OpenStack ecosystem in China.
This document provides an overview of how to set up OpenStack using DevStack in 2 hours. It discusses installing VirtualBox and Vagrant, setting up a Vagrantfile to define and configure VMs, installing DevStack which deploys OpenStack services onto the VMs, and using localrc to customize the OpenStack deployment. The document then demonstrates how to launch and interact with the OpenStack environment to deploy and manage resources.
Trystakc.cn was announced in OpenStack Summit San Diego 2012(www.slideshare.net/openstack/trystack-introfinalpdf
).It was a Non-profit OpenStack community projects.
By Stackers, for stackers.Experience the latest OpenStack features.
Welcoming contributions and feedback, Join the fun !
This document summarizes the architecture of Quantum, the network service for OpenStack. It discusses the key components of Quantum including the Quantum server, plugins, agents, and databases. It describes the network models in Quantum including tenant networks, provider networks, and floating IPs. It also outlines the communication between Quantum components using AMQP messaging.
OpenStorage refers to open source storage software that allows for disaggregated hardware components from different vendors. NexentaStor is a leading OpenStorage solution that runs on standard hardware and provides file and block access using protocols like NFS, CIFS, and iSCSI. It offers storage efficiency features like deduplication, compression, and thin provisioning. NexentaStor can also be used with OpenStack Nova to provision volumes and attach them to virtual machines. Nexenta has contributed code to OpenStack Swift to leverage NexentaStor's self-healing capabilities for object storage. OpenStorage is growing with adoption in cloud computing and more integration with projects like OpenStack.
Swift is an object storage software used at SinaAppEngine for storage. It uses consistent hashing to distribute objects uniformly across multiple servers for high reliability and availability. Data is replicated across multiple servers and zones to protect against hardware failures. Consistency is ensured through a quorum-based protocol where writes require acknowledgment from a majority of replicas and reads require a majority to respond. Problems include inefficient replication that impacts performance and availability, and improvements involve optimizing replication to run during idle times and adding nodes for scaling out.
Libvirt is an open source library that provides a standardized interface to manage virtualization platforms. It supports various hypervisors like QEMU/KVM, Xen, VMWare and containers. It provides APIs to manage domains, networks, storage and other virtualization components in a platform-independent way. Libvirt has over 25 maintainers and receives 200-300 patches per release to support new features and improve existing functionality.
3. OpenStack Key Components
Nova
Compute
Flexible
Workloads
Virtualized
Storage Infrastructure
Swift Open Quantu
Platforms
Network m
Cinder Common
Fabrics
4. 工欲善其事,必先利其器
Business DB Performance
Performance Requirement
Random small
(OLTP, OLAP) Storage
TPC-C/E/H
IOMeter
(Requests per second)
YCSB, Large analytics
HiBench (e.g NoSQL, Hadoop)
Sequential Large
Capacity
Storage
COSBench Cloud Object storage
(e.g. photos/videos)
Gigabytes Terabytes Petabytes Exabytes
Capacity Requirement
fferent Usage model requires different benchmark to
5. COSBench Introduction
• COSBench is an Intel developed Benchmark to
measure Cloud Object Storage Service
performance
– For S3, OpenStack Swift like Object Storage
– Not for File system (NFS e.g) and Block Device system
(EBS e.g.) COSBench
• Benefits:
– Compare public Cloud Object Storage services (User)
– Evaluate different Hardware/Software Stacks (Provider)
– Identify bottleneck and make optimization (Provider)
The IOMeter for Cloud Storage Services.
7. Web Console
Driver list
Workload
List
History list
Intuitive UI to get overview.
8. Workload Configuration
Flexible load control
object size distribution
Read/Write Operations
Workflow for complex stages
Flexible configuration parameters.
9. Performance Metrics
• Throughput (Operations/s): the operations completed in one second
• Response Time (in ms): the duration between operation initiation and
completion.
• Bandwidth (KB/s): the total data in KiB transferred in one second
• Success Ratio (%): the ratio of successful operations
10. Performance Reporting
Id Op RT TH BW Succ% Timeline
Throughput
W1-s1- 500 160
write 24,8390.16 172 100% 400
140
1 300
120
100
summa timelin 80
200 60
ry e 40
100
20
0 0
histogra loadlin Time (in 5s)
m e
Response Time Histogram Performance Loadline (100% Read)
80,000 120% 6,000 400
Response Time (ms)
Throughput (Op/s) Throughput (Op/s) 362
70,000 100% 5,000 350
Avg Response Time
60,000 300
80% (ms)
50,000 4,000 250
read
40,000 60% 3,000 200
CDF (%)
30,000 40% 150
20,000 2,000
20% 91 100
10,000 1,000
45 50
0 0% 15 15 19 26
0 0
0~10 60~70 120~130 180~190 240~250
16 32 64 128 256 512 2048
Workers
11. Extensible API
• Easily extend for new
storage system: AuthAPI Auth
• Support
– OpenStack Swift Context
PUT
– Amplistor* GET
– Adding More StorageAPI DELETE
Extensible API is able to support more storages.
13. 128KB-Read
# 95%- Throughpu
Setup-SAS 128KB-Read Worker ResTime t
ms op/s
Throughput Avg-ResTime 95%-ResTime
5 20.00 369.49
response time (ms)
6.0k 1000 10 20.00 711.24
throughput (op/s)
5.02k5.00k4.95k4.84k
4.69k 20 20.00 1383.30
4.5k 800
3.66k 40 30.00 2517.94
600
3.0k 2.52k 80 46.67 3662.71
400 160 56.67 4693.97
1.38k
1.5k 0.71k 200 320 106.67 5019.85
0.37k
640 230.00 4998.13
0.0k 0
5 20 80 320 1280 1280 470.00 4947.15
2560 923.33 4840.19
Total Number of Worker
bottleneck
The was identified to be the proxy’s
CPU
-- The CPU utilization at that node was ~100%!
For more complete information about setup-SATA was 5576 results in higher
-- The peak throughput for Better CPU op/s (640
performance and benchmark results, visit
workers) throughput
14. 128KB-Write
• SLA: 200ms + 128KB/1MBps =
325ms # 95%- Throughpu
Setup-SAS 128KB-Write Worker ResTime t
ms op/s
Throughput Avg-ResTime 95%-ResTime
5 40.00 219.73
1.87k1.89k 4000
response time (ms)
2.0k 1.77k1.77k 10 40.00 391.14
throughput (op/s)
1.59k
3200 20 50.00 668.19
1.5k 1.33k
40 70.00 1022.07
1.02k 2400
1.0k 80 100.00 1333.34
0.67k 1600 160 143.33 1594.12
0.5k 0.22k0.39k 800 320 370.00 1769.55
640 1223.33 1773.12
0.0k 0
5 20 80 320 1280 1280 1690.00 1871.58
2560 3160.00 1886.81
Total Number of Worker
The Disks at storage nodes had significant impact on overall
throughput
-- The peak throughput for setup-SATA was 1208 op/s (320
Workers)
For more complete information about
-- still gaps even after we had put account/container DB files
performance and benchmark results, visit
15. 10MB-Read
• SLA: 200ms + 10MB/1MBps =
1200ms # 95%- Throughpu
Setup-SAS10MB -Read Worker ResTime t
ms op/s
Throughput Avg-ResTime 95%-ResTime
5 270.00 34.69
response time (ms)
80 70 71 73 60,000 10 320.00 51.87
throughput (op/s)
67 69 70
65
60 50,000 20 480.00 59.91
60 52
40,000 40 900.00 65.48
40 35 30,000 80 1636.67 67.37
20,000 160 3093.33 68.69
20 320 5950.00 69.58
10,000
640 11906.67 70.18
0 0
5 20 80 320 1280 1280 24090.00 71.41
2560 52090.00 72.90
Total Number of Worker
bottleneck was identified to be the clients’
The
NIC BandWidth
Double client receive bandwidth can double the throughput
For more complete information about
performance and benchmark results, visit
16. 10MB-Write
• SLA: 200ms + 10MB/1MBps =
1200ms # 95%- Throughpu
Setup-SAS 10MB-Write Worker ResTime t
ms op/s
Throughput Avg-ResTime 95%-ResTime
5 536.67 13.12
24
response time (ms)
25 23 23 250,000
22 10 936.67 17.50
throughput (op/s)
21 21 22
20
20 17 200,000 20 1596.67 20.28
15 13 150,000 40 2786.67 21.30
80 5133.33 21.38
10 100,000 160 9800.00 22.21
5 50,000 320 18623.33 23.19
640 41576.67 23.23
0 0
5 20 80 320 1280 1280 102090.00 21.55
2560 200306.67 23.71
Total Number of Worker
bottleneck
The might be the storage nodes’
NICs
-- in setup-SATA, the peak throughput was 23.23 op/s (10
clients)
For more complete information about
performance and benchmark results, visit
-- in both setups, the write performance was 1/3 of the read
17. Next Step
• Support more storage services like Amazon S3,
Google Cloud Storage, and Microsoft Azure…
• Make COSBench available (so far under NDA, will
open source finally) for industry and community to
build efficient cloud storage service.
• Continue to use COSBench to analyze and optimize
OpenStack* Swift performance on Intel Platform and
share our findings back to community.