My talk on MetaCDN for the Cloudslam 2009 virtual conference.
Many 'Cloud Storage' providers have launched in the last two years, providing internet accessible data storage and delivery in several continents that is backed by rigorous Service Level Agreements (SLAs), guaranteeing specific performance and uptime targets. The facilities offered by these providers is leveraged by developers via provider-specific Web Service APIs. For content creators, these providers have emerged as a genuine alternative to dedicated Content Delivery Networks (CDNs) for global file storage and delivery, as they are significantly cheaper, have comparable performance and no ongoing contract obligations. As a result, the idea of utilising Storage Clouds as a 'poor mans' CDN is very enticing. However, many of these 'Cloud Storage' providers are merely basic storage services, and do not offer the capabilities of a fully-featured CDN such as intelligent replication, failover, load redirection and load balancing. Furthermore, they can be difficult to use for non-developers, as each service is best utilised via unique web services or programmer APIs. In this presentation, we describe the design, architecture, implementation and user-experience of MetaCDN, a system that integrates these 'Cloud Storage' providers into an unified CDN service that provides high performance, low cost, geographically distributed content storage and delivery for content creators. MetaCDN harnesses the power of 'Cloud Storage' for novices and seasoned users alike, offering an easy to use web portal and a sophisticated Web Service API.
Deployment Pipeline for Magento Enterprise in the Cloud. The Talk covers the Amazon Cloud Infrastructure; Scaling and Autoscaling in the Cloud, the Deployment Pipeline used to do continuous deployments...
발표 영상 다시보기: https://kr-resources.awscloud.com/data-databases-and-analytics/%EC%A7%80%EA%B8%88-%EB%8B%B9%EC%9E%A5-dynamo-db-%ED%99%9C%EC%9A%A9%ED%95%98%EA%B8%B0-%EA%B0%95%EB%AF%BC%EC%84%9D-aws-database-modernization-day-%EC%98%A8%EB%9D%BC%EC%9D%B8-2
DynamoDB는 대량의 트래픽에 대해 빠른 응답시간을 보장하는 AWS의 NOSQL Database 서비스 입니다. 본 세션에서는 DynamoDB를 생성하고 테이블 디자인 후 데이터 입력, 삭제, 업데이트 및 성능에 관련된 설정에 대해서 진행합니다. 이 세션후 참석자들은 DyanmoDB에 대한 이해하며 직접 구성 및 사용할 수 있습니다.
A round up of all the features & improvements released since our last update, we'll walk through the timeline to bring you up to speed on the continuous innovation at AWS.
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...Adrian Cockcroft
The document discusses Netflix replacing its Oracle database with Apache Cassandra on AWS to support its transition to becoming a global cloud-based service. Key points include migrating data from Oracle to Cassandra for improved scalability and availability across regions; using AWS services like S3, EC2 and SimpleDB during the transition; and addressing challenges around backups, disaster recovery and analytics with the new architecture.
Netflix uses cloud computing to address challenges in scaling its infrastructure to support unpredictable growth. It has transitioned its website to be nearly 100% cloud-based using Amazon Web Services (AWS) to gain the scale, availability and agility needed. AWS provides tools and features like auto-scaling that allow Netflix to easily expand capacity as its subscriber base grows by over 50% per year. By leveraging AWS' mature cloud platform, Netflix can focus on its core video business rather than managing data centers.
- CloudStack is an open source cloud computing platform that was donated to the Apache Software Foundation in 2012. It provides infrastructure as a service and supports various hypervisors and physical hardware.
- CloudStack has a scalable architecture designed to support thousands of hosts and VMs across multiple availability zones. It provides rich networking and storage capabilities.
- CloudStack can support both traditional server virtualization workloads as well as "Amazon-style" workloads with software defined networks and object storage.
- The CloudStack community is growing rapidly and encourages participation through mailing lists, IRC, forums and meetup groups.
SDEC2011 Big engineer vs small entreprenuerKorea Sdec
This document discusses the differences between a big engineer at an established company and a small entrepreneur starting their own company. It describes the challenges a small startup faces in mobile service development including unpredictable traffic, lack of resources, and difficulty building their own cultures. The entrepreneur considers using clouds but has concerns about latency, reliability and costs. They decide to build their own hybrid system to meet goals of low latency, high performance, reliability and scalability at a low price.
Deployment Pipeline for Magento Enterprise in the Cloud. The Talk covers the Amazon Cloud Infrastructure; Scaling and Autoscaling in the Cloud, the Deployment Pipeline used to do continuous deployments...
발표 영상 다시보기: https://kr-resources.awscloud.com/data-databases-and-analytics/%EC%A7%80%EA%B8%88-%EB%8B%B9%EC%9E%A5-dynamo-db-%ED%99%9C%EC%9A%A9%ED%95%98%EA%B8%B0-%EA%B0%95%EB%AF%BC%EC%84%9D-aws-database-modernization-day-%EC%98%A8%EB%9D%BC%EC%9D%B8-2
DynamoDB는 대량의 트래픽에 대해 빠른 응답시간을 보장하는 AWS의 NOSQL Database 서비스 입니다. 본 세션에서는 DynamoDB를 생성하고 테이블 디자인 후 데이터 입력, 삭제, 업데이트 및 성능에 관련된 설정에 대해서 진행합니다. 이 세션후 참석자들은 DyanmoDB에 대한 이해하며 직접 구성 및 사용할 수 있습니다.
A round up of all the features & improvements released since our last update, we'll walk through the timeline to bring you up to speed on the continuous innovation at AWS.
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...Adrian Cockcroft
The document discusses Netflix replacing its Oracle database with Apache Cassandra on AWS to support its transition to becoming a global cloud-based service. Key points include migrating data from Oracle to Cassandra for improved scalability and availability across regions; using AWS services like S3, EC2 and SimpleDB during the transition; and addressing challenges around backups, disaster recovery and analytics with the new architecture.
Netflix uses cloud computing to address challenges in scaling its infrastructure to support unpredictable growth. It has transitioned its website to be nearly 100% cloud-based using Amazon Web Services (AWS) to gain the scale, availability and agility needed. AWS provides tools and features like auto-scaling that allow Netflix to easily expand capacity as its subscriber base grows by over 50% per year. By leveraging AWS' mature cloud platform, Netflix can focus on its core video business rather than managing data centers.
- CloudStack is an open source cloud computing platform that was donated to the Apache Software Foundation in 2012. It provides infrastructure as a service and supports various hypervisors and physical hardware.
- CloudStack has a scalable architecture designed to support thousands of hosts and VMs across multiple availability zones. It provides rich networking and storage capabilities.
- CloudStack can support both traditional server virtualization workloads as well as "Amazon-style" workloads with software defined networks and object storage.
- The CloudStack community is growing rapidly and encourages participation through mailing lists, IRC, forums and meetup groups.
SDEC2011 Big engineer vs small entreprenuerKorea Sdec
This document discusses the differences between a big engineer at an established company and a small entrepreneur starting their own company. It describes the challenges a small startup faces in mobile service development including unpredictable traffic, lack of resources, and difficulty building their own cultures. The entrepreneur considers using clouds but has concerns about latency, reliability and costs. They decide to build their own hybrid system to meet goals of low latency, high performance, reliability and scalability at a low price.
Amazon offers multiple cloud storage options including S3, EBS, and local instance storage. S3 provides scalable object storage, EBS provides block-level storage volumes for EC2 instances, and local storage provides temporary disk space. Each has different performance, durability, cost and management characteristics suited for various use cases. CloudFront provides a global content delivery network to distribute cached content from S3 across its worldwide edge locations.
AWS Update | London - Performance Update and Provisioned IOPSAmazon Web Services
The document discusses updates to Amazon Web Services in London regarding improved storage performance options. It introduces provisioned IOPS for EBS volumes, which allows specifying input/output operations per second. It also covers EBS-optimized EC2 instances for dedicated throughput to EBS storage, and using provisioned IOPS with RDS. A new high I/O EC2 instance type is presented as well for workloads requiring high random I/O performance. Pricing and customer examples are provided for these different performance-optimized storage and compute options on AWS.
Amazon CloudFront - Delivering Dynamic Content From The EdgeAmazon Web Services
This document discusses Amazon CloudFront, a content delivery network (CDN) that improves performance and reliability for delivering dynamic and static web content. It works by caching content at edge locations close to users to reduce latency. Key benefits include faster load times, scalability, availability, and cost effectiveness. CloudFront delivers content through a global network of edge locations and integrates with other AWS services like S3 and EC2. It supports dynamic, private, and streaming content along with programmatic invalidation of cached objects.
Deep Dive into Container Scheduling with Amazon ECS - CON404 - re:Invent 2017Amazon Web Services
The document discusses container scheduling with Amazon ECS. It provides an overview of ECS's core scheduling components including scheduling engines, placement engines, and extensions. It also discusses how ECS supports massive scale, placement strategies, task scheduling patterns, and examples of using placement constraints and priorities.
MED101 Introduction to Amazon CloudFront - AWS re: Invent 2012Amazon Web Services
End users expect to be able to view media content anytime, anywhere, and on any device. Amazon CloudFront is a web service for content delivery used to distribute content to end users around the globe with low latency, high data transfer speeds, and no commitments. In this session, learn what a content delivery network (CDN) such as Amazon CloudFront is and how it works, the benefits it provides, common challenges and needs, performance, pricing, and examples of how customers are using CloudFront.
The Cloud provides a transparent cost model, with options from on-demand provisioning, to reserved capacity to low cost, spot pricing. This session outlines how to use these pricing models in concert to optimise the running cost of your application on the AWS Cloud. We'll outline new utilisation based reserved instances and bidding strategies when running on spot instances, to cost prediction.
Games + Amazon = Love - Presentation quo vadis 2011Thomas Lobinger
This document discusses using Amazon Web Services for game development. It begins with an introduction to AWS and its core services including Elastic Compute Cloud (EC2) for compute, Simple Storage Service (S3) for storage, and others. It then provides examples of how games can benefit from AWS's scalability, flexibility, and cost reduction. The document concludes by introducing Scalarium, a tool that manages EC2 clusters and automates deployment, scaling, and maintenance of applications on AWS.
"Cloudian’s new solution, Cloudian 2.4, maximizes data protection, availability and security, which are critical for enterprises looking to deploy their own private cloud or move their data to the public cloud. For multi-datacenter deployments, Cloudian now provides Dynamic Consistency Levels, offering unprecedented data protection and ensuring continuous, uninterrupted operations in the case of network or site failures. "
See more detail at Cloudian Press Release.
http://www.cloudian.com/news/press-releases/press-release-25.html
Cloud Computing for Developers and Architects - QCon 2008 TutorialStuart Charlton
The document provides an overview of a tutorial on cloud computing for developers and architects. It discusses defining cloud computing, qualities of clouds, examples of Amazon Web Services including Simple Storage Service (S3) and Elastic Compute Cloud (EC2), and how to provision and operate cloud environments. The agenda covers cloud industry trends, a cloud reference model, managing cloud systems, cloud architectures, and questions.
MBL303 Scalable Mobile and Web Apps - AWS re: Invent 2012Amazon Web Services
AWS offers an array of products and services to handle the unprecedented volumes of traffic, enormous user numbers and vast amounts of data being experienced by a successful mobile app that takes off. Learn how with new found agility and amazingly low time to market, these must-know best practices and techniques in the rapidly evolving and highly demanding mobile landscape can ensure success. Featuring Intuit's txtWeb architecture as a case study.
This document provides information about Microsoft's SQL Data Services (SDS), a relational database service running in the cloud. The summary discusses the key points:
- SDS will provide a highly scalable and available relational data store in the cloud, accessible using familiar SQL Server tools and APIs.
- Initially, SDS will support core SQL Server capabilities but future versions may include additional data platform capabilities.
- SDS uses a symmetrical programming model designed to provide a consistent experience whether using the database on-premises or in the cloud.
- Microsoft is currently working towards commercial availability of SDS integrated with the Windows Azure platform in 2009.
OSCON Data 2011 -- NoSQL @ Netflix, Part 2Sid Anand
The document discusses translating concepts from relational databases to key-value stores. It covers normalizing data to avoid issues like data inconsistencies and loss. While key-value stores don't support relations, transactions, or SQL, the relationships can be composed in the application layer for smaller datasets. Picking the right data for key-value stores involves accessing data primarily by key lookups.
AmebaPico is a social networking game launched in 2010. It uses AWS services like S3, CloudFront, EC2, and MongoDB. The game had 60 million monthly active users at its peak. It was developed using Flash for the front-end and ran on AWS infrastructure with a MongoDB database. Scaling issues arose as traffic grew, which required optimizing the database and EC2 instance configurations.
The presentation discussed moving applications to the cloud for scalability, flexibility and pay-as-you-go pricing, noting key differences between RSAWEBCloud and AWS; challenges for developers include optimizing applications for production environments and handling scaling which requires separating concerns like data types and using caching, load balancing, and autoscaling tools.
Developing polyglot applications on Cloud Foundry (#oredev 2012)Chris Richardson
Developing web applications used to be simple. Your single war-file web application served up HTML to a desktop browser and used a relational database. Today however, web applications are much more complex: the front-end uses HTML5 and NodeJS, the middle tier is decomposed into multiple services, and the back-end uses a mix of SQL and NoSQL databases. Developing these kind of applications can be challenging since there are so many moving parts that need to be correctly installed and configured. Deployment is even more difficult.
In this talk, you will learn why we need to build applications with this architectural style and how Cloud Foundry, which is modern, multi-lingual, multi-service, extensible open-source PaaS, can help. We will talk about how to develop modern applications that run on Cloud Foundry and cover what’s new and different about the cloud environment. You will learn how your application can consume the various services that are provided by Cloud Foundry. We will discuss the various ways of using Cloud Foundry including the Micro Cloud that runs on a laptop as well as the hosted CloudFoundry.com.
Deep Dive on Amazon EC2 Instances - AWS Summit Cape Town 2017Amazon Web Services
Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key platform features, and the concept of instance generations. We dive into the current generation design choices of the different instance families, including the General Purpose, Compute Optimized, Storage Optimized, Memory Optimized, and Accelerated Computing (GPU and FPGA) instance families. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.
AWS Speaker: Ian Massingham, Sr Mgr, Technical Evangelist - Amazon Web Services
Customer Speaker: Andrew Mori, Konga, Technical Director
Comparison of Word2vec and Doc2Vec model driven Sentiment Analysis using SVM,...SofiaDutta1
The document compares various models for sentiment analysis including Word2Vec, Doc2Vec, logistic regression, SVM, XGBoost, convolutional neural networks, and bidirectional LSTMs. It finds that Keras bidirectional LSTMs and Keras CNNs combined with bidirectional LSTMs performed best, especially when using pre-trained Word2Vec embeddings. Word2Vec generally outperformed Doc2Vec, and combinations of Doc2Vec models like DBOW+DMM performed better than single Doc2Vec models. Bidirectional LSTMs were most effective due to their ability to consider current and previous inputs for sequential text data.
The GE Business Screen matrix is subjective in how it depicts new businesses that are just starting to grow. Hofer and Schendel developed the Product/Market Evolution Portfolio Matrix to better show these young companies. The new matrix plots businesses based on their competitive position and stage of product or market evolution. An analysis of the matrix suggests strategic alternatives for different business units - units A and B show potential, C needs intervention, D maintains position, E and F generate cash, and G should be divested. Most company portfolios resemble one of three ideal types: growth, profit, or balanced.
This document discusses knowledge architecture analysis, which is a key component of Open Text Corporation's project methodology. It involves characterizing a problem or opportunity in terms of knowledge management, verifying expected gains from using technology, and comparing the target application's logic to Livelink functionality. The analysis develops a first cut project approach by refining the problem, pinpointing the underlying knowledge architecture, and assessing the existing infrastructure and integration requirements. The goal is to establish the best foundation for evaluating technology features by integrating best practices from various disciplines like business process reengineering.
Amazon offers multiple cloud storage options including S3, EBS, and local instance storage. S3 provides scalable object storage, EBS provides block-level storage volumes for EC2 instances, and local storage provides temporary disk space. Each has different performance, durability, cost and management characteristics suited for various use cases. CloudFront provides a global content delivery network to distribute cached content from S3 across its worldwide edge locations.
AWS Update | London - Performance Update and Provisioned IOPSAmazon Web Services
The document discusses updates to Amazon Web Services in London regarding improved storage performance options. It introduces provisioned IOPS for EBS volumes, which allows specifying input/output operations per second. It also covers EBS-optimized EC2 instances for dedicated throughput to EBS storage, and using provisioned IOPS with RDS. A new high I/O EC2 instance type is presented as well for workloads requiring high random I/O performance. Pricing and customer examples are provided for these different performance-optimized storage and compute options on AWS.
Amazon CloudFront - Delivering Dynamic Content From The EdgeAmazon Web Services
This document discusses Amazon CloudFront, a content delivery network (CDN) that improves performance and reliability for delivering dynamic and static web content. It works by caching content at edge locations close to users to reduce latency. Key benefits include faster load times, scalability, availability, and cost effectiveness. CloudFront delivers content through a global network of edge locations and integrates with other AWS services like S3 and EC2. It supports dynamic, private, and streaming content along with programmatic invalidation of cached objects.
Deep Dive into Container Scheduling with Amazon ECS - CON404 - re:Invent 2017Amazon Web Services
The document discusses container scheduling with Amazon ECS. It provides an overview of ECS's core scheduling components including scheduling engines, placement engines, and extensions. It also discusses how ECS supports massive scale, placement strategies, task scheduling patterns, and examples of using placement constraints and priorities.
MED101 Introduction to Amazon CloudFront - AWS re: Invent 2012Amazon Web Services
End users expect to be able to view media content anytime, anywhere, and on any device. Amazon CloudFront is a web service for content delivery used to distribute content to end users around the globe with low latency, high data transfer speeds, and no commitments. In this session, learn what a content delivery network (CDN) such as Amazon CloudFront is and how it works, the benefits it provides, common challenges and needs, performance, pricing, and examples of how customers are using CloudFront.
The Cloud provides a transparent cost model, with options from on-demand provisioning, to reserved capacity to low cost, spot pricing. This session outlines how to use these pricing models in concert to optimise the running cost of your application on the AWS Cloud. We'll outline new utilisation based reserved instances and bidding strategies when running on spot instances, to cost prediction.
Games + Amazon = Love - Presentation quo vadis 2011Thomas Lobinger
This document discusses using Amazon Web Services for game development. It begins with an introduction to AWS and its core services including Elastic Compute Cloud (EC2) for compute, Simple Storage Service (S3) for storage, and others. It then provides examples of how games can benefit from AWS's scalability, flexibility, and cost reduction. The document concludes by introducing Scalarium, a tool that manages EC2 clusters and automates deployment, scaling, and maintenance of applications on AWS.
"Cloudian’s new solution, Cloudian 2.4, maximizes data protection, availability and security, which are critical for enterprises looking to deploy their own private cloud or move their data to the public cloud. For multi-datacenter deployments, Cloudian now provides Dynamic Consistency Levels, offering unprecedented data protection and ensuring continuous, uninterrupted operations in the case of network or site failures. "
See more detail at Cloudian Press Release.
http://www.cloudian.com/news/press-releases/press-release-25.html
Cloud Computing for Developers and Architects - QCon 2008 TutorialStuart Charlton
The document provides an overview of a tutorial on cloud computing for developers and architects. It discusses defining cloud computing, qualities of clouds, examples of Amazon Web Services including Simple Storage Service (S3) and Elastic Compute Cloud (EC2), and how to provision and operate cloud environments. The agenda covers cloud industry trends, a cloud reference model, managing cloud systems, cloud architectures, and questions.
MBL303 Scalable Mobile and Web Apps - AWS re: Invent 2012Amazon Web Services
AWS offers an array of products and services to handle the unprecedented volumes of traffic, enormous user numbers and vast amounts of data being experienced by a successful mobile app that takes off. Learn how with new found agility and amazingly low time to market, these must-know best practices and techniques in the rapidly evolving and highly demanding mobile landscape can ensure success. Featuring Intuit's txtWeb architecture as a case study.
This document provides information about Microsoft's SQL Data Services (SDS), a relational database service running in the cloud. The summary discusses the key points:
- SDS will provide a highly scalable and available relational data store in the cloud, accessible using familiar SQL Server tools and APIs.
- Initially, SDS will support core SQL Server capabilities but future versions may include additional data platform capabilities.
- SDS uses a symmetrical programming model designed to provide a consistent experience whether using the database on-premises or in the cloud.
- Microsoft is currently working towards commercial availability of SDS integrated with the Windows Azure platform in 2009.
OSCON Data 2011 -- NoSQL @ Netflix, Part 2Sid Anand
The document discusses translating concepts from relational databases to key-value stores. It covers normalizing data to avoid issues like data inconsistencies and loss. While key-value stores don't support relations, transactions, or SQL, the relationships can be composed in the application layer for smaller datasets. Picking the right data for key-value stores involves accessing data primarily by key lookups.
AmebaPico is a social networking game launched in 2010. It uses AWS services like S3, CloudFront, EC2, and MongoDB. The game had 60 million monthly active users at its peak. It was developed using Flash for the front-end and ran on AWS infrastructure with a MongoDB database. Scaling issues arose as traffic grew, which required optimizing the database and EC2 instance configurations.
The presentation discussed moving applications to the cloud for scalability, flexibility and pay-as-you-go pricing, noting key differences between RSAWEBCloud and AWS; challenges for developers include optimizing applications for production environments and handling scaling which requires separating concerns like data types and using caching, load balancing, and autoscaling tools.
Developing polyglot applications on Cloud Foundry (#oredev 2012)Chris Richardson
Developing web applications used to be simple. Your single war-file web application served up HTML to a desktop browser and used a relational database. Today however, web applications are much more complex: the front-end uses HTML5 and NodeJS, the middle tier is decomposed into multiple services, and the back-end uses a mix of SQL and NoSQL databases. Developing these kind of applications can be challenging since there are so many moving parts that need to be correctly installed and configured. Deployment is even more difficult.
In this talk, you will learn why we need to build applications with this architectural style and how Cloud Foundry, which is modern, multi-lingual, multi-service, extensible open-source PaaS, can help. We will talk about how to develop modern applications that run on Cloud Foundry and cover what’s new and different about the cloud environment. You will learn how your application can consume the various services that are provided by Cloud Foundry. We will discuss the various ways of using Cloud Foundry including the Micro Cloud that runs on a laptop as well as the hosted CloudFoundry.com.
Deep Dive on Amazon EC2 Instances - AWS Summit Cape Town 2017Amazon Web Services
Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key platform features, and the concept of instance generations. We dive into the current generation design choices of the different instance families, including the General Purpose, Compute Optimized, Storage Optimized, Memory Optimized, and Accelerated Computing (GPU and FPGA) instance families. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.
AWS Speaker: Ian Massingham, Sr Mgr, Technical Evangelist - Amazon Web Services
Customer Speaker: Andrew Mori, Konga, Technical Director
Comparison of Word2vec and Doc2Vec model driven Sentiment Analysis using SVM,...SofiaDutta1
The document compares various models for sentiment analysis including Word2Vec, Doc2Vec, logistic regression, SVM, XGBoost, convolutional neural networks, and bidirectional LSTMs. It finds that Keras bidirectional LSTMs and Keras CNNs combined with bidirectional LSTMs performed best, especially when using pre-trained Word2Vec embeddings. Word2Vec generally outperformed Doc2Vec, and combinations of Doc2Vec models like DBOW+DMM performed better than single Doc2Vec models. Bidirectional LSTMs were most effective due to their ability to consider current and previous inputs for sequential text data.
The GE Business Screen matrix is subjective in how it depicts new businesses that are just starting to grow. Hofer and Schendel developed the Product/Market Evolution Portfolio Matrix to better show these young companies. The new matrix plots businesses based on their competitive position and stage of product or market evolution. An analysis of the matrix suggests strategic alternatives for different business units - units A and B show potential, C needs intervention, D maintains position, E and F generate cash, and G should be divested. Most company portfolios resemble one of three ideal types: growth, profit, or balanced.
This document discusses knowledge architecture analysis, which is a key component of Open Text Corporation's project methodology. It involves characterizing a problem or opportunity in terms of knowledge management, verifying expected gains from using technology, and comparing the target application's logic to Livelink functionality. The analysis develops a first cut project approach by refining the problem, pinpointing the underlying knowledge architecture, and assessing the existing infrastructure and integration requirements. The goal is to establish the best foundation for evaluating technology features by integrating best practices from various disciplines like business process reengineering.
Presenter: Saguenay Baruss, AvePoint Client Services Senior Solutions Architect
Event: SharePoint Summit Vancouver 2014
Date: Tuesday, October 28, 2014
Description: This session is intended for organizations that have made a significant investment in SharePoint and are required to provide enterprise records management. It considers the capabilities of SharePoint as a records management system, and shares strategies for extending SharePoint-based collaboration to also include the management of corporate records, both with and without the integration of existing Enterprise Content Management (ECM) systems.
More from AvePoint: http://www.avepoint.com/community
The fundamentals and best practices of securing your Hadoop cluster are top of mind today. In this session, we will examine and explain the components, tools, and frameworks used in Hadoop for authentication, authorization, audit, and encryption of data and processes. See how the latest innovations can let you securely connect more data to more users within your organization.
This document discusses Sentry, an open source authorization module for Hive and Impala. It provides fine-grained, role-based authorization to define user and application access to data. Sentry aims to enable secure authorization that existing options like advisory permissions and HDFS impersonation lack. The document outlines Sentry's architecture and capabilities, and how it integrates with the Hadoop ecosystem through the Hive metastore. A demo is offered to showcase Sentry's authorization policies and features.
• Provides theoretical background on records management
• Starts from the business challenges, then uses SharePoint to address them
• Relates SharePoint records management to standards like ISO 15489
• Build a working end-to-end solution
• Shows how enhance RM functionality by using out-of-the-box features
Architecture talk aimed at a well informed developer audience (i.e. QConSF Real Use Cases for NoSQL track), focused mainly on availability. Skips the Netflix cloud migration stuff that is in other talks.
This document discusses different architectural approaches that can be used when deploying workloads on AWS like startups. It summarizes virtual machine-based n-tier architectures, container-based architectures using ECS, and serverless architectures using Lambda. It also discusses how these architectures impact cost, performance, reliability and other factors. The document recommends letting development teams choose the right tools for their needs and adopting a microservices approach to scale complexity over time.
As interest in cloud solutions and their use with enterprise applications has increased, MavenWire has taken a lead in implementing and benchmarking several instances of OTM using Amazon Web Services (AWS) and Elastic Cloud Compute (EC2). This presentation outlines how the instances were set up and configured; potential benefits of OTM in the cloud; cost and performance comparisons between the cloud and "traditional" server configurations; areas of concern and issues to be aware of when implementing OTM in the cloud. In addition, we will also outline what we believe the future direction of cloud OTM will be, as well as where we believe it is best suited to customer needs.
AWS re:Invent 2016: Born in the Cloud; Built Like a Startup (ARC205)Amazon Web Services
This presentation provides a comparison of three modern architecture patterns that startups are building their business around. It includes a realistic analysis of cost, team management, and security implications of each approach. It covers Elastic Beanstalk, Amazon ECS, Docker, Amazon API Gateway, AWS Lambda, Amazon DynamoDB, and Amazon CloudFront, as well as Docker.
The document discusses Netflix's use of open source technologies in its cloud architecture. It summarizes how Netflix leverages open source software to build cloud native applications that are highly scalable and available on AWS. Key aspects include building stateless microservices, using Cassandra for data storage in a quorum across multiple availability zones, and tools like Edda for configuration management and monitoring. The document advocates for open sourcing Netflix's best practices to help drive innovation.
Best Practices Scaling Web Application Up to Your First 10 Million UsersAmazon Web Services
If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Cloud computing gives you a number of advantages, such as the ability to scale your web application on demand. Join us in this webinar to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
This document provides an overview and introduction to Amazon Web Services (AWS) by Jeff Barr, an evangelist for AWS. The summary includes:
1) Barr introduces AWS and discusses its goals of showing what others are doing with cloud computing, alerting the audience to possibilities, and starting conversations about cloud computing.
2) AWS provides scalable computing resources like servers, storage, databases, and more via web services that can be accessed on-demand using a pay-as-you-go model. This solves problems around managing infrastructure and reduces costs.
3) Barr highlights some key AWS services including Elastic Compute Cloud (EC2) for virtual servers, Simple Storage Service (S3) for online storage
by James Broberg - Presentation given at the 2nd International Workshop on Web APIs and Mashups (at ICSOC2008) on December 1st, 2008 in Sydney, Australia. http://www.icsoc-mashups.org/
by Itzik Paz, Solutions Architect & Rich Cowper, Solutions Architect Manager, AWS
This presentation compares three modern architecture patterns that startups are building their businesses around. It includes a realistic analysis of cost, team management, and security implications of each approach. It covers AWS Elastic Beanstalk, Amazon ECS, Docker, Amazon API Gateway, AWS Lambda, Amazon DynamoDB, and Amazon CloudFront.
The document discusses real-time web analytics company LiveStats' transition from conventional hosting to Amazon Web Services (AWS) cloud hosting. It provides reasons for choosing AWS like flexibility, scalability, and pay-as-you-use pricing. It also discusses challenges of moving to the cloud but advantages like full control and lower barriers to entry. The document outlines LiveStats' architecture on AWS including load balancing, auto-scaling, and decoupling services, and how they monitor systems and implement best practices like scaling only when needed.
Join this workshop to understand the core concepts of “Cloud Computing” and how businesses around the world are running the infrastructure that supports their websites to lower costs, improve time-to-market, and enable rapid scalability matching resource to demands of users. Whether you are an enterprise looking for IT innovation, agility and resiliency or small and medium business who wants to accelerate growth without a big upfront investment in cash or time for technology, the AWS Cloud provides a complete set of services at zero upfront costs which are available with a few clicks and within minutes.
- Amazon Redshift is a fast, powerful, and simple petabyte-scale data warehouse service that runs in the cloud and is delivered as a managed service with no hardware to provision or manage.
- It offers very fast performance at a very low price, with no upfront costs and a pay-as-you-go model, and supports popular BI tools through JDBC/ODBC connectivity.
- Using columnar storage and massively parallel query execution, Redshift can analyze vast amounts of structured data quickly and cost effectively for both large enterprises and small companies on a budget.
Magento Imagine 2013: Fabrizio Branca - Learning To Fly: How Angry Birds Reac...AOE
1) The document discusses how the Angry Birds store was able to reach high performance levels through optimizations like using a CDN, caching, database tuning, splitting databases, and offloading processing to improve page load times and handle high traffic volumes.
2) Key optimizations included using Varnish caching, separating databases, running background processes, and profiling to reduce bottlenecks.
3) Maintaining high code quality, continuous testing and integration, and automating processes were also important lessons to support the store's growth.
This document discusses how to integrate Eclipse with Microsoft cloud services. It provides an overview of different compute and storage options on Microsoft Azure like Web Roles, Worker Roles, and VM Roles. It also discusses tools for Java developers to use Azure services like the Windows Azure Toolkit for Java, Windows Azure Starter Kit for Java, and JDBC drivers for SQL Server and SQL Azure. Links are provided for interoperability bridges between Azure and other platforms as well as resources for more information.
JClouds at San Francisco Java User GroupMarakana Inc.
This document discusses key-value storage concepts like global namespaces, metadata, and various cloud storage vendors. It compares features of different vendors like file size limits, consistency models, authorization policies, and API complexity. It also covers content delivery networks, coding HTTP requests, dealing with errors and concurrency, and introduces jclouds as an open source library that aims to abstract away differences between cloud providers and handle common tasks like provisioning.
Andy Parsons gave a workshop on understanding and scaling the cloud for startups. He began by defining cloud computing and discussing its impact on startups. He then covered the essential aspects of cloud computing like self-service provisioning, pooled resources, and metered billing. Parsons also discussed the different service and deployment models, components of AWS, and how the cloud infrastructure works using virtualization. He ended by providing recommendations for how development teams should architect applications for the cloud and factors to consider when choosing a cloud provider.
This presentation compares three modern architecture patterns that startups are building their businesses around. It includes a realistic analysis of cost, team management, and security implications of each approach. It covers AWS Elastic Beanstalk, Amazon ECS, Amazon API Gateway, AWS Lambda, Amazon DynamoDB, and Amazon CloudFront. Attendees will also hear from venture capital investor Third Rock Ventures (TRV) who has launched 40+ biotech startups over the last 10 years. TRV will outline how it launches cloud native startups that turn bleeding edge science into new treatments across the spectrum of disease, with highlights drawn Relay Therapeutics and Tango Therapeutics.
Similar to MetaCDN: Enabling High Performance, Low Cost Content Storage and Delivery via the Cloud (20)
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Infrastructure Challenges in Scaling RAG with Custom AI modelsZilliz
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
MetaCDN: Enabling High Performance, Low Cost Content Storage and Delivery via the Cloud
1. MetaCDN: Enabling High
Performance, Low Cost
Content Storage and Delivery
via the Cloud
Dr. James Broberg (brobergj@csse.unimelb.edu.au)
http://www.csse.unimelb.edu.au/~brobergj
http://www.metacdn.org
2. Content Delivery
Networks (CDNs)
• What is a CDN?
• Content Delivery Networks (CDNs) such as
Akamai, Mirror Image and Limelight place web
server clusters in numerous geographical
locations to improve the responsiveness and
locality of the content it hosts for end-users.
3. Existing CDN providers
• Akamai is the clear leader in coverage and
market share (approx. 80%), however...
• Price is prohibitive for SME, NGO, Gov...
• Anecdotally 2−15 times more expensive
than Cloud Storage, and require 1−2 year
commitments and min. data use (10TB+)
• Academic CDNs include Coral, Codeen,
Globule, however...
• No SLA / QoS provided, only ‘best effort’
4. Major CDN pricing
• Most major CDN providers do not publish
prices
• “Average” prices from the 4-5 major CDNs
in the market:
• 50TB: $0.40 - $0.50 per GB delivered
• 100TB: $0.25 - $0.45 per GB delivered
• 250TB: $0.10 - $0.30 per GB delivered
• Information taken from Dan Rayburn @
www.cdnpricing.com, StreamingMedia.com
7. Feature Comparison
Nirvanix Amazon Mosso
Amazon S3
Global SDN CloudFront CloudFiles
99.9 99-99.9 99-99.9 99.9
SLA
256GB 5GB 5GB 5GB
Max File Size
Yes Yes Yes Yes
US PoP
EU PoP Yes Yes Yes Yes
Asia PoP Yes No Yes Yes
AUS PoP No No No Yes
Yes Yes Yes Yes
Per file ACL
Yes No Yes Yes
Automatic replication
of files
Yes Yes Yes Yes
Developer API’s /
Web Services
10. Introducing MetaCDN
• What if we could create a low-cost, high performance overlay
CDN using these storage clouds?
• MetaCDN harnesses the diversity of multiple storage cloud
providers, offering a choice of deployment location, features
and pricing.
• MetaCDN provides a uniform method / API for harnessing
multiple storage clouds.
• MetaCDN provides a single namespace to cover all supported
providers, making it simple to integrate into origin sites, and
handles load balancing for end-users.
• The MetaCDN system offers content creators (who are not
necessarily programmers!) a trivial way to harness the power
of multiple cloud storage providers via a web portal
11. How MetaCDN works
Amazon S3 & Mosso Cloud Microsoft Azure
Nirvanix SDN Coral CDN
CloudFront Files CDN Storage Service
JetS3t toolkit Nirvanix SDK Cloud Files SDK CoralConnector AzureConnector
Java SDK Java SDK Java SDK Java stub Java stub
Open Source Nirvanix, Inc Mosso, Inc MetaCDN.org MetaCDN.org
AmazonS3Connector NirvanixConnector CloudFilesConnector
Java stub Java stub Java stub Shared/Private
MetaCDN.org MetaCDN.org MetaCDN.org Host
MetaCDN
WebDAVConnector
Java stub
MetaCDN MetaCDN QoS MetaCDN MetaCDN
MetaCDN.org
Manager Monitor Allocator Database
SCPConnector
Java stub
Web Portal Load Redirector Web Service MetaCDN.org
Java (JSF/EJB) based portal Random redirection SOAP Web Service FTPConnector
Support HTTP POST Geographical redirection RESTful Web Service Java stub
New/view/modify deployment Least cost redirection Programmatic access MetaCDN.org
13. MetaCDN Allocator
• The MetaCDN Allocator allows users to
deploy files either directly or from a public
origin URL, with the following options:
• Maximise coverage and performance
• Deploy content in specific locations
• Cost optimised deployment
• Quality of Service (QoS) optimised
14. MetaCDN Manager
• The MetaCDN Manager ensures that:
• All current deployments are meeting QoS
targets (where applicable)
• Replicas are removed when no longer
required (minimising cost)
• A users’ budget has not been exceeded, by
tracking usage (i.e. storage/download)
15. MetaCDN QoS Monitor
• The MetaCDN QoS Monitor:
• Tracks the performance of participating
providers at all times
• Monitors and records throughput, response
time and uptime from a variety of locations
• Ensures that upstream providers are
meeting their Service Level Agreements
16. MetaCDN Database
1 0:M 1 1
MetaCDN CDN CDN
has for
User Credentials Provider
1
1 1
hosted
has has
by
M
1 M
M 1
1 0:M MetaCDN Coverage
deployed hosted
Content
Replica locations
as at
1
M
QoS Monitor measures
17. MetaCDN Web Portal
• Developed using Java Enterprise and Java
Server Faces (JSF) technologies
• JPA/MySQL back-end to store persistent data
• Web portal acts as the entry point to the
system and application-level load balancer
• Most suited for small or ad-hoc deployments,
and especially useful for less technically
inclined content creators.
18. Don't have a MetaCDN account? Sign in
Register Username: master
Password: ••••••
Login
All trademarks mentioned herein are the exclusive property of their respective owners.
This project is supported by the:
19. Register new MetaCDN account:
Username: Full Name:
joebloggs Joe Bloggs
Password: Email:
•••••• joe@blogs.com
Preferred Providers: Nirvanix SDN Amazon S3
Mosso Cloud Files Microsoft Azure Storage Service
Shared/Private Host
Microsoft Azure Storage Service Shared/Private Host
Nirvanix SDN Amazon S3 Mosso Cloud Files
Enter your Amazon S3 Credentials:
******************************
AWS Access Key:
******************************
AWS Secret Key:
Enable CloudFront:
Register Cancel
All trademarks mentioned herein are the exclusive property of their respective owners.
This project is supported by the:
20. Welcome to the MetaCDN Portal!
Deploy content View existing content Deployment Map
View existing replicas MetaCDN Analytics
Logout
Edit Account
All trademarks mentioned herein are the exclusive property of their respective owners.
This project is supported by the:
21. Sideload Content:
Choose URL: http://pitchfork.com/media/frontend/images/header_logo.gif
North America Europe
Deployment region(s):
Asia Australasia
Host Until: 30/05/2009
Deploy Cancel
All trademarks mentioned herein are the exclusive property of their respective owners.
This project is supported by the:
25. MetaCDN Web Service
• Makes all functionality available via Web
Services (SOAP & REST/HTTP)
• Web interface is useful for novices and for
ad-hoc deployments, but doesn’t scale
• Larger customers have 1,000’s - 10,000 -
100,000s of files that need deployment
• Let them automate their deployment and
management via Web Services!
• Perfect for Mashup developers!!!
26. MetaCDN Load
Redirector
• We have created a unified namespace to
simply deployment, routing, management
• Currently, file deployment results in multiple
URLs, each mapping to a replica
• http://metacdn-eu-user.s3.amazonaws.com/myfile.mp4
• http://metacdn-us-user.s3.amazonaws.com/myfile.mp4
• http://node3.nirvanix.com/MetaCDN/user/myfile.mp4
• Single namespace is created for fine control
• http://www.metacdn.org/FileMapper?itemid=2
27. MetaCDN Load
Redirector (cont.)
• Actual load redirection logic depends on deployment
option used by MetaCDN user:
• Maximise coverage and performance? - Find closest
physical replica
• Deploy content in specific locations? - Find closest
physical replica
• Cost optimised deployment? - Find cheapest
replica, minimises cost to maximise lifetime
• Quality of Service (QoS) optimised? - Find best
(historically) performing replica
28. MetaCDN Load
Redirector (cont.)
MetaCDN end-user DNS Server MetaCDN gateway
Resolve www.metacdn.org
Return IP of closest MetaCDN gateway,
www-na.metacdn.org
GET http://metacdn.org/MetaCDN/FileMapper?itemid=1
processRequest ()
geoRedirect ()
HTTP 302 Redirect to
http://metacdn-us-username.s3.amazonaws.com/filename.pdf
Amazon S3 USA
Resolve metacdn-us-username.s3.amazonaws.com
Return IP of metacdn-us-username.s3.amazonaws.com
GET http://metacdn-us-username.s3.amazonaws.com/filename.pdf
Return replica
29. Redirector Overhead
from USA
1.8
S3 USA
MetaCDN
1.6
1.4
1.2
1
Seconds
0.8
0.6
0.4
0.2
0
0 5 10 15 20 25
Hour
30. Redirector Overhead
from Australia
0.9
Nirvanix SDN #3
MetaCDN
0.85
0.8
0.75
Seconds
0.7
0.65
0.6
0.55
0 5 10 15 20 25
Hour
31. Evaluating MetaCDN
performance
• Ran tests over 24 hour period (mid-week)
• Downloading test replicas (1KB, 10MB) 30
times per hour, take average and conf. inter.
• 10MB - Throughput, 1KB - Response Time
• Ran test from 6 locations: Melbourne (AUS),
Paris (FRA), Vienna (AUT), San Diego & New
Jersey (USA), Seoul (KOR)
• Replicas located across US, EU, ASIA, AUS
32. Summary of Results -
Throughput (KB/s)
S3 S3 SDN SDN SDN SDN
Coral
US EU #1 #2 #3 #4
Melbourne
264.3 389.1 30 366.8 408.4 405.5 173.7
Australia
703.1 2116 483.8 2948 416.8 1042 530.2
Paris France
Vienna
490.7 1347 288.4 2271 211 538.7 453.4
Austria
Seoul
312.8 376.1 466.5 411.8 2456 588.2 152
South Korea
San Diego
1234 323.5 5946 380.1 506.1 820.4 338.5
USA
New Jersey
2381 1949 860.8 967.1 572.8 4230 636.4
USA
33. Summary of results -
Response Time (Sec)
S3 S3 SDN SDN SDN SDN
Coral
US EU #1 #2 #3 #4
Melbourne
1.378 1.458 0.663 0.703 1.195 0.816 5.452
Australia
0.533 0.2 0.538 0.099 1.078 0.316 3.11
Paris France
Vienna
0.723 0.442 0.585 0.099 1.088 0.406 3.171
Austria
Seoul
1.135 1.21 0.856 0.896 1 0.848 3.318
South Korea
San Diego
0.232 0.455 0.23 0.361 0.775 0.319 4.655
USA
New Jersey
0.532 0.491 0.621 0.475 1.263 0.516 1.916
USA
34. Summary of results
(cont)
• Clients benefited greatly from local replicas
• Results are consistent in terms of response time
and throughput with previous studies of
dedicated CDNs
• Back-end providers have sufficient performance
and reliability to be used to host replicas in the
MetaCDN system
• Adding “CDN” Cloud Providers (Amazon
CloudFront, Mosso Cloud Files) will likely
significantly improve results.
35. MetaCDN features in
planning / development
• Support as many providers as possible
• Windows Azure Storage Service support will
be finished very shortly.
• Non-“Cloud” storage (i.e. FTP, WebDav) will
be available that is seamlessly integrated.
• Integrated Flash video and MP3 audio streaming
with embeddable players for customers to place
on their origin sites.
36. MetaCDN features in
planning / development
• Autonomic deployment management
(expansion/contraction) based on demand
• Autonomic deployment management
(expansion/contraction) based on QoS
• Security / ACL framework that spans all
cloud storage providers
• “One time” MetaCDN URLs
37. Collaborations
• Always looking for people to collaborate on
the project
• Work to be done on:
• Load balancing / redirection algorithms
• Intelligent Caching / replication algorithms
• Security / Access Control of content
• Improving MetaCDN Web Services
• Please contact me if you are interested...
38. Acknowledgements
• Australian Research Council (ARC) for
funding the project.
• Cory & Barry (Nirvanix) and Eric (Rackspace/
Mosso) for development support.
39. Publications
• J. Broberg and Z. Tari. MetaCDN: Harnessing Storage Clouds for High
Performance Content Delivery. In Proceedings of The Sixth International
Conference on Service-Oriented Computing [Demonstration Paper]
(ICSOC 2008), LNCS 5364, pp. 730–731, 2008.
• J. Broberg, R. Buyya and Z. Tari. Creating a ‘Cloud Storage’ Mashup for High
Performance, Low Cost Content Delivery, Second International Workshop
on Web APIs and Services Mashups (Mashups’08), In Proceedings of The
Sixth International Conference on Service-Oriented Computing
Workshops, LNCS 5472, pp. 178–183, 2009.
• J. Broberg, R. Buyya, and Z. Tari, MetaCDN: Harnessing ‘Storage Clouds’ for
high performance content delivery, Journal of Network and Computer
Applications (JNCA), To appear, 2009
• R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic. Cloud computing
and emerging IT platforms: Vision, hype, and reality for delivering computing
as the 5th utility, Future Generation Computer Systems. To appear, 2009.
40. Thanks
Latest information will be available at:
www.metacdn.org
International Workshop on Cloud Computing
(Cloud 2009):
http://www.gridbus.org/cloud2009
Editor's Notes
MetaCDN is a system that leverages several existing ‘storage clouds’, creating an integrated overlay network that provides a low cost, high performance content delivery network for content creators.
*Content Delivery Networks (CDNs) such as Akamai and Mirror
Image place web server clusters in numerous geographical locations
to improve the responsiveness and locality of the content it hosts for
end-users.
*However, their services are priced out of reach for all but the largest enterprise customers.
Major CDN providers are notoriously cagey about revealing their prices. Most will only reveal their prices if you are serious customer and are willing to commit to a contract and minimum data usage (as detailed in the previous slide). As such Dan Rayburn @ StreamingMedia.com (a blog run for streaming media and CDN professionals) undertakes an informal sampling of pricing (taken from CDN customers) every few months.
Numerous ‘storage cloud’ providers (or ‘Storage as a Service’) have emerged that can
provide data storage and delivery in several continents, offering S.L.A. backed performance and uptime promises for their services.
Each storage provider outlines their own cost structure for data transferred in and out of their service, as well as charging for persistent storage. In each of these cases, the costs are in the order of cents per gigabyte. Pricing scales downward based on higher usage for all providers. There is no minimum data usage requirement and no contracts - you only pay for what you store and transfer.
The providers themselves have very similar core functionality, but there are some key differences, for example, the largest allowable file size, the coverage footprint or specific features.
It is easy to see why storage clouds provide a compelling alternative to traditional CDNs for content producers that transfer significant amounts of data to their customers.
Amazon CloudFront offers a CDN-like service that is significantly cheaper than tradional CDNs. Amazon charges different rates depending on where the data is delivered from to reflect the cost of data transfer and operations in different locations.
1. MetaCDN is more likely to meet the needs of content creators than a single provider could.
2. There is no ‘unified’ or familiar interface for all storage clouds. Consider Amazon S3, Nirvanix SDN, Mosso Cloud Files and Microsoft Azure Storage Service. These four cloud storage providers have four separate access APIs that a developer would need to learn to access these services.
3. If a content creator attempted to utilise these providers themselves, they would essentially need to perform the load balancing and redirection themselves at their origin sites (complex!)
The service is presented to end-users as a web portal for small or ad-hoc deployments or as Web Services (currently under development) for integration of customers with more complex and frequently changing content delivery needs. The web portal was developed using Java Enterprise and Java Server Faces (JSF) technologies, with a MySQL back-end to store user accounts, deployments, and the capabilities and pricing of service providers.
Introduce connectors, and major components.
The MetaCDN system integrates with each storage provider via connectors that provides an abstraction to hide the complexity arising from the differing ways each provider allows access to their systems. The connectors encapsulate basic operations like creation, deletion and renaming of files and folders. If an operation is not supported on a particular service, then the connector for that service should throw a FeatureNotSupportedException.
1. MetaCDN deploys as many replicas as possible to all available locations.
2. A user nominates regions and MetaCDN matches the requested regions
with providers that service those areas.
3. Where MetaCDN deploys as many replicas in the locations requested by the user as their
storage and transfer budget will allow, keeping them active until that budget is exhausted.
4. MetaCDN deploys to providers that match specific QoS targets that a user specifies, such as average throughput or response time from a particular location, which is tracked by persistent probing from the MetaCDN QoS monitor.
The MetaCDN database tracks all pertinent information such as users of the system, credentials for various providers, details about the providers capabilities, pricing and footprint and details of replicas deployed.
Using the web portal, users can sign up for an account on the MetaCDN system, and
enter credentials for any cloud storage or other provider they have an account with. Once this simple step has been performed, they can utilise the MetaCDN system to intelligently deploy content onto storage providers according to their performance requirements and budget limitations.
A MetaCDN user is required to register the credentials of providers they have accounts with. Once this step is done they do not need to worry about how to interact with each of the providers. Eventually we would like MetaCDN users to not require accounts with specific providers - rather MetaCDN would provide consolidated billing of users for storage and transfer of content.
The MetaCDN \"Control Panel\" gives easy access to the core features of the service. You can deploy content, view existing deployments (via a high level content view or a detailed replica view), and view a deployment map overlayed onto Google Maps or Google Earth.
Here we can see an example of geographical-based deployment. A user nominates regions and MetaCDN matches the requested regions with providers that service those areas. The user also specifies the desired lifetime of the deployment, after which the replicas will be removed.
We can view details of our past deployments. We store and track information such as the origin id (i.e. the original source of the content), a unique GUID, the MetaCDN URL that represents the deployment, the number of times this content has been downloaded, the last time this content was downloaded and how many replicas were generated from this deployment.
Here we can see the specific replicas that have been generated from our various deployments. For each replica, we can see which provider and location was utilised, the public URL of the replica, the number of times the replica has been downloaded, the last access time of a specific replica and options to modify, delete, or view the replica if we wish to fine tune our deployment.
We can get a birds eye view of where our replicas are stored, and how many are stored in each location. MetaCDN generates a KML file for each user that is used to overlay on Google Maps (shown here) or we can view our deployments in Google Earth. We expect to overlay more useful information in these views in the near future, such as the cost expenditure at each location and the location of client (i.e. file consumer) hotspots.
A web service interface is under development that will make all the functionality of the web portal available in a programmatic fashion. Obviously it's not feasible to deploy thousands of files manually via the web portal so we need to prove the facility for advanced customers to scale out easily and rapidly.
*With multiple sources (and multiple URL’s) the complexity of load
balancing is imposed on the origin / content provider
*With single namespace we can have coarse and fine-grained control
via DNS redirection and layer4/7 load balancing
http://www.metacdn.org:8080/MetaCDN/FileMapper?itemid=1
http://www.metacdn.org:8080/MetaCDN/FileMapper?itemid=1&policy=RAN
http://www.metacdn.org:8080/MetaCDN/FileMapper?itemid=1&policy=GEO
During the development phase only, only 1 copy of portal/redirector is running in Melbourne, Australia. The plan is to deploy portals in several locations across US, Asia and Europe. We will see from next slide why this is necessary.
Let's assume that a consumer in the USA was accessing a replica directly (i.e. it magically knew the best replica to select), or via MetaCDN. Here we can see that there is around 0.4 seconds of overhead, which is predominantly the round-trip time to access the gateway in Australia.
When a consumer has a gateway that is close to it (in this case, a consumer in Australia is utilising a local gateway) the overhead is significantly smaller, in the order of 0.05 seconds per request. It is obvious that local gateways are needed in key areas to maximise performance.
In the second half of 2008 we evaluated the two major cloud storage providers at the time, Amazon S3 and Nirvanix SDN. We ran the test over 24 hours from a variety of client and replica locations to see whether the providers demonstrated sufficient performance (i.e. throughput and response times) to act as a \"poor man's\" CDN.
In 5 out of 6 client locations there were at least 2 replicas each that delivered throughput that is consistent with what we would expect from a traditional CDN service.
<Kilobytes per second>
In 4 our of 6 client locations there was 1-3 replicas that delivered response time that is consistent with what we would expect from a traditional CDN service. It is worth noting here that these times represent the end to end latency and connection time (i.e. a HTTP connection is made), they are not simply ping measurements.
FTP/WebDav support will be useful in locations where cloud providers do not (and are unlikely to) service.
There is a lot of demand from customers to move away from Youtube and Vimeo flash video hosts and host their own streaming content directly on their origin site. This way they control the look and feel and ad monetization of their content.