These are the slides from my presentation at CLOUDCOMP 2009 on AppScale, an open source platform for running Google App Engine apps on. See our project home page at http://appscale.cs.ucsb.edu or our code page at http://code.google.com/p/appscale
These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much material on these topics.
These slides are from my Ph.D. defense at the University of California, Santa Barbara, discussing how we contribute research tools to forward how science is performed with cloud systems.
Designing the Call of Cthulhu app with Google App EngineChris Bunch
These are slides from a talk I gave at UCSB to the Senior Capstone class on 02/10/10 on how I developed the Call of Cthulhu application using Google App Engine.
These are the slides from my talk about the AppScale project at the SBonRails meetup. It covers AppScale as well as Google App Engine and the research projects have come out of it, including Neptune, a Ruby DSL focused on computation-heavy workloads.
These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much material on these topics.
These slides are from my Ph.D. defense at the University of California, Santa Barbara, discussing how we contribute research tools to forward how science is performed with cloud systems.
Designing the Call of Cthulhu app with Google App EngineChris Bunch
These are slides from a talk I gave at UCSB to the Senior Capstone class on 02/10/10 on how I developed the Call of Cthulhu application using Google App Engine.
These are the slides from my talk about the AppScale project at the SBonRails meetup. It covers AppScale as well as Google App Engine and the research projects have come out of it, including Neptune, a Ruby DSL focused on computation-heavy workloads.
StreamSQL Feature Store (Apache Pulsar Summit)Simba Khadder
Input features are the building blocks for machine learning models. You cannot have a great model without great features. By building on top of Apache Pulsar's infinite retention of events, we built infrastructure to serve features in production and to generate training datasets. It allowed our machine learning teams to change, test, and deploy personalization features at an extraordinary rate to 10s of millions of end-users.
This talk will discuss:
- What event-sourcing is and why it's so powerful for machine learning infrastructure.
- How we built the StreamSQL feature store on top of Pulsar, Flink, and Cassandra.
- How a feature store accelerates ML development.
Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2Amazon Web Services
Amazon EC2 Container Service (Amazon ECS) is a new AWS service that makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. Amazon ECS lets you define, schedule, and stop sets of containers. You have access to the state of your resources, making it easy to confirm that tasks are running or view the utilization of EC2 instances in your cluster. This session will describe the benefits of containers, introduce ECS, and demonstrate how to use ECS for your applications.
An introductory presentation on Chatbots with Serverless (AWS Lambda). It covers AWS Lez, its terminologies and AWS Lambda in detail. It also showcases on how to connect your Lex bot to Facebook Messenger.
Kafka for Microservices – You absolutely need Avro Schemas! | Gerardo Gutierr...HostedbyConfluent
Whether you are deploying a new application in Microservices or transitioning from a monolithic database application to a cloud-ready architecture, you will inevitably face the decision of either creating a service mesh of API’s – or – using an event bus for better durability, reliability and extensibility of your application. If you choose to go the event bus route, Kafka is an excellent choice for several reasons. One key technology not to overlook is Avro Schemas. They provide a definition for your event payload, just like an API, to ensure all of the event consumers can reliably consume the events. They also handle schema evolution as requirements change and much, much more.
In this talk we will discuss all the nuances and considerations around using Avro Schemas for your JSON event payloads. From developer tools, to DevOps approaches, versioning, governance and some “gotchas” we found when working with Avro Schemas and the Confluent Schema Registry.
(DEV302) Hosting ASP.Net 5 Apps in AWS with Docker & AWS CodeDeployAmazon Web Services
The .NET Platform is undergoing a revolution with a new modularized .NET Framework and CoreCLR, a new cross platform runtime. ASP.NET 5 gives .NET developers the ability to develop and run their applications outside of Windows. In this session we will explore how to develop and deploy ASP.NET 5 applications on Windows with AWS CodeDeploy and Linux with Docker. For Docker we will explore using Docker with both Elastic Beanstalk and EC2 Container Service.
(APP309) Running and Monitoring Docker Containers at Scale | AWS re:Invent 2014Amazon Web Services
If you have tried Docker but are unsure about how to run it at scale, you will benefit from this session. Like virtualization before, containerization (à; la Docker) is increasing the elastic nature of cloud infrastructure by an order of magnitude. But maybe you still have questions: How many containers can you run on a given Amazon EC2 instance type? Which metric should you look at to measure contention? How do you manage fleets of containers at scale?
Datadog is a monitoring service for IT, operations, and development teams who write and run applications at scale. In this session, the cofounder of Datadog presents the challenges and benefits of running containers at scale and how to use quantitative performance patterns to monitor your infrastructure at this magnitude and complexity. Sponsored by Datadog.
Google Cloud Platform, Compute Engine, and App EngineCsaba Toth
Introduction to Google Cloud Platform's compute section, Google Compute Engine, Google App Engine. Place these technologies into the cloud service stack, and later show how Google blurs the boundaries of IaaS and PaaS.
StreamSQL Feature Store (Apache Pulsar Summit)Simba Khadder
Input features are the building blocks for machine learning models. You cannot have a great model without great features. By building on top of Apache Pulsar's infinite retention of events, we built infrastructure to serve features in production and to generate training datasets. It allowed our machine learning teams to change, test, and deploy personalization features at an extraordinary rate to 10s of millions of end-users.
This talk will discuss:
- What event-sourcing is and why it's so powerful for machine learning infrastructure.
- How we built the StreamSQL feature store on top of Pulsar, Flink, and Cassandra.
- How a feature store accelerates ML development.
Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2Amazon Web Services
Amazon EC2 Container Service (Amazon ECS) is a new AWS service that makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. Amazon ECS lets you define, schedule, and stop sets of containers. You have access to the state of your resources, making it easy to confirm that tasks are running or view the utilization of EC2 instances in your cluster. This session will describe the benefits of containers, introduce ECS, and demonstrate how to use ECS for your applications.
An introductory presentation on Chatbots with Serverless (AWS Lambda). It covers AWS Lez, its terminologies and AWS Lambda in detail. It also showcases on how to connect your Lex bot to Facebook Messenger.
Kafka for Microservices – You absolutely need Avro Schemas! | Gerardo Gutierr...HostedbyConfluent
Whether you are deploying a new application in Microservices or transitioning from a monolithic database application to a cloud-ready architecture, you will inevitably face the decision of either creating a service mesh of API’s – or – using an event bus for better durability, reliability and extensibility of your application. If you choose to go the event bus route, Kafka is an excellent choice for several reasons. One key technology not to overlook is Avro Schemas. They provide a definition for your event payload, just like an API, to ensure all of the event consumers can reliably consume the events. They also handle schema evolution as requirements change and much, much more.
In this talk we will discuss all the nuances and considerations around using Avro Schemas for your JSON event payloads. From developer tools, to DevOps approaches, versioning, governance and some “gotchas” we found when working with Avro Schemas and the Confluent Schema Registry.
(DEV302) Hosting ASP.Net 5 Apps in AWS with Docker & AWS CodeDeployAmazon Web Services
The .NET Platform is undergoing a revolution with a new modularized .NET Framework and CoreCLR, a new cross platform runtime. ASP.NET 5 gives .NET developers the ability to develop and run their applications outside of Windows. In this session we will explore how to develop and deploy ASP.NET 5 applications on Windows with AWS CodeDeploy and Linux with Docker. For Docker we will explore using Docker with both Elastic Beanstalk and EC2 Container Service.
(APP309) Running and Monitoring Docker Containers at Scale | AWS re:Invent 2014Amazon Web Services
If you have tried Docker but are unsure about how to run it at scale, you will benefit from this session. Like virtualization before, containerization (à; la Docker) is increasing the elastic nature of cloud infrastructure by an order of magnitude. But maybe you still have questions: How many containers can you run on a given Amazon EC2 instance type? Which metric should you look at to measure contention? How do you manage fleets of containers at scale?
Datadog is a monitoring service for IT, operations, and development teams who write and run applications at scale. In this session, the cofounder of Datadog presents the challenges and benefits of running containers at scale and how to use quantitative performance patterns to monitor your infrastructure at this magnitude and complexity. Sponsored by Datadog.
Google Cloud Platform, Compute Engine, and App EngineCsaba Toth
Introduction to Google Cloud Platform's compute section, Google Compute Engine, Google App Engine. Place these technologies into the cloud service stack, and later show how Google blurs the boundaries of IaaS and PaaS.
A Public Cloud Based SOA Workflow for Machine Learning Based Recommendation A...Ram G Athreya
Over the past decade the field of Cloud Computing has been the focus of intensive research. In this paper we propose a framework that will simulate the architectural setup of a cloud environment and examine how it can leverage Apriori and Sequential Pattern based recommendation algorithms through R. Furthermore, we present a multi layered application encompassing its backend architecture, user interface built using the responsive web design technique and its development workflow. The proposed system was also exhaustively load tested using Apache JMeter to ensure its reliability at scale and the experimental results are presented.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any Web application.
Cost is often the conversation starter when customers think about moving to the cloud. AWS helps lower costs for customers through its “pay only for what you use” pricing model, frequent price drops, and pricing model choice to support variable & stable workloads. In this session, you will learn about the financial considerations of owning and operating a traditional data center or managed hosting provider versus utilizing AWS. We will detail our TCO methodology and showcase cost comparisons for some common customer use-cases. We’ll also cover a few AWS cost optimization areas, including Spot and Reserved Instances, EC2 Auto Scaling, and consolidated billing.
Presenter:
Amit Sharma, Solution Architect, Amazon Internet Services
Krishnenjit Roy, Director IT Operations, Freshdesk
Cask Webinar
Date: 08/10/2016
Link to video recording: https://www.youtube.com/watch?v=XUkANr9iag0
In this webinar, Nitin Motgi, CTO of Cask, walks through the new capabilities of CDAP 3.5 and explains how your organization can benefit.
Some of the highlights include:
- Enterprise-grade security - Authentication, authorization, secure keystore for storing configurations. Plus integration with Apache Sentry and Apache Ranger.
- Preview mode - Ability to preview and debug data pipelines before deploying them.
- Joins in Cask Hydrator - Capabilities to join multiple data sources in data pipelines
- Real-time pipelines with Spark Streaming - Drag & drop real-time pipelines using Spark Streaming.
- Data usage analytics - Ability to report application usage of data sets.
- And much more!
Getting Started with AWS Lambda and the Serverless Cloud - AWS Summit Cape T...Amazon Web Services
Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more. In this session, you’ll learn how to get started with serverless computing with AWS Lambda, which lets you run code without provisioning or managing servers. We’ll introduce you to the basics of building with Lambda and how you can benefit from features such as continuous scaling, built-in high availability, integrations with AWS and third-party apps, and subsecond metering pricing. We’ll also introduce you to the broader portfolio of AWS services that help you build serverless applications with Lambda, including Amazon API Gateway, Amazon DynamoDB, AWS Step Functions, and more.
AWS Speaker : Danilo Poccia, Technical Evangelist - Amazon Web Services
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
3. Terminology
Software-as-a-Service (SaaS)
e.g., SalesForce, Gmail
Provides remote application access
Platform-as-a-Service (PaaS)
e.g., Google App Engine
Provides scalable runtime stack
Infrastructure-as-a-Service (IaaS)
e.g., Amazon Web Services
Provides full system images
4. • Open-source, Platform-as-a-Service for research
and engineering of cloud computing components,
applications, and services
• Automated deployment of applications to high-
performance databases
• Fine grain control over application environment
• Google App Engine apps hosting on your cluster
– Real applications
– Familiar API (that is extensible for lock-in avoidance)
– Your data and code on your resources
5. From Google App Engine (GAE)
to AppScale
• GAE Application Programming Interface
– Datastore (get/put)
– Memcache
– URL Fetching
– Mail
– Images
– Authentication
• Write Python/Java GAE app
– Use SDK locally to test and generate indexes
• APIs implemented as non-scalable, simple versions
6. From Google App Engine (GAE)
to AppScale
• GAE Application Programming Interface
– Datastore (get/put) BigTable
– Memcache Memcached
– URL Fetching
– Mail GMail
– Images
– Authentication Google Accounts
• Write Python/Java GAE app
– Use SDK locally to test and generate indexes
• APIs implemented as non-scalable, simple versions
– Upload to Google resources
• Highly scalable API implementation
7. Sandboxed Runtime
• Restricted subset of library calls
• No reading/writing from/to file system
• Data persistence only via get/put interface
• Computation bounded: 30 secs per request
• Access web services over via HTTP / HTTPS
only (ports 80 and 443)
8. Recent GAE Additions
• Python and JVM SDKs
– JRuby, Clojure, etc. available through Java
• Task Queue, Cron, XMPP APIs
• New SLAs for paying customers
– $0.10 per CPU core hour
– $0.10 per GB bandwidth in
– $0.12 per GB bandwidth out
– $0.15 per GB data stored per month
9. Protocol Buffers
• Google App Engine’s internal data format
– And AppScale’s
• Similar to C-style structs:
message Person {
required int32 id = 1;
optional string name = 2;
}
10. From Google App Engine (GAE)
to AppScale
• AppScale extends the GAE SDK
– Replaces the simple, non-scalable API implementation
with pluggable, distributed, scalable components
• Using open-source solutions as available/possible
• Communication over SSL
• Available as source and as system image
– Each instance can implement any component
• Self configuring as part of AppScale cloud deployment
– Deploys over
• Virtual machine monitors (Xen, KVM)
• Infrastructure (IaaS) cloud layers
11. IaaS Cloud Systems
• Amazon Web Services (AWS)
– Elastic Compute Cloud (EC2), Persistent Storage (S3, EBS)
– For-fee, as negotiated in SLA (CPU, network, storage)
– Vast resources available
• Users access small (opaque) subset, can scale-out
• Eucalyptus
– Open source implementation of the AWS APIs
– Inspiration for AppScale – familiar, widely-used API
implementation for execution on your cluster
• Limited only by the hardware you have available
12. Differences in AppScale
Deployment Options
• Xen / KVM:
– Static deployment
• Can use as many nodes as are manually configured
• Eucalyptus / EC2
– Dynamic deployment
• Can use as many nodes as the system can support (or pay for
for EC2 deployment)
– As part of ongoing/future work: support for dynamic scaling
• Front-end (user-facing) & back-end (data managment & computation)
• SLA renegotiation
13. AppScale System Layout
• AppLoadBalancer (ALB)
• AppServer (AS)
• Database Master/Slave/Peer (DB M/S/P)
GAE App AppScale
DB M/P
Developer tools
ALB
(AppScale
Admin) App
DB S/P
Controller
GAE App
GAE App
GAE App AS
Users
Users
Users HTTPS
14. AppController (AC)
• SOAP Server written in Ruby
– Runs on all nodes
• Middleware layer
• Controls and sets up a node for use
– Sets up configuration files (data replication)
– Sets up firewall for security
• Master AC “heartbeats” all other nodes
– Collects performance info as well
15. AppLoadBalancer (ALB)
• Ruby on Rails application
• Handles authentication and routing of users
to AppServers
• Three copies are deployed via Mongrel
– Load balanced via nginx
16. Database Management
• Five databases currently available:
– HBase, Hypertable: Master / Slave
– Cassandra, Voldemort: Peer / Peer
– Clustered MySQL: Relational
• Two main components
– Protocol Buffer Server: Data access / storage
– User / App Server: Authentication
17. AppServer (AS)
• Modified Google App Engine SDK
• App requests internally are Protocol Buffers
– Forwards requests to PB Server
• Minimal request set:
– Put(id)
– Get(id)
– Query: Equivalent to get_all_in_table
– Delete(id)
– Count: Total number of items in database
– GetSchema
18. AppScale Tools
• Ruby scripts that initiate AppScale
deployment
– Initializes the first AppController for use
– Uploads AppEngine app
• Conceptually similar to Amazon AWS EC2
tools
– describe-instances
– upload-app: Introduce additional apps
– terminate-instances
19. Fault Tolerance
• System can survive the following failures:
– AppServer failure
– Database Slave failure
– Database Peer failure
– AppLoadBalancer failure *
– AppController failure *
20. Testing Methodology
• Load testing done via the Grinder
• Test specifics:
– Initially 3 users
– 3 users added every 5 seconds
– Done until 160 seconds have passed
• Each user navigates the page, performs
some scripted action
• Measured total transactions performed and
average response time
21. AppScale Evaluation Cluster
• Three Grinder nodes, four AppScale nodes
– One master, three slaves
– Virtualized via Xen
– Database: HBase (3x replication) 64 MB HDFS blocks
• PBServer via Thrift; stores entire protocol buffers
• Hardware
– Quad-core 2.66 GHz machines
– 8 GB of RAM
– Connected via Gigabit Ethernet
22. Applications Tested
• Tasks - a to-do list
– Read and write intensive (44 transactions per user)
• Cccwiki – allows users to edit web pages
– Read intensive, updates only (74 transactions per
user)
• Guestbook – allows users to post messages
– Retrieves ten most recent posts only (9 transactions
per user)
• Shell – provides an interactive Python shell
– Compute intensive (14 transactions per user)
26. Room for Improvement
• Current bottlenecks:
– Queries perform filtering server-side
– Filtering is done outside of the DB
– AppEngine, PB Server are single-threaded
– Entry point to some DBs is single-threaded
• Future work will address these problems
– Will also compare performance across DBs
– e.g., BigTable-like DBs vs. P2P DBs
27. Related Work
• AppDrop
– Proof-of-concept Rails app
• TyphoonAE
– Relatively new (alpha release)
– Runs MongoDB only
• Microsoft Azure
– Uses .NET as the platform
– Has a similar pricing model to AppEngine
28. AppScale Recap
• Distributed, multi-component system
– Deployed as a single system image (self
configuring)
• Static deployment over Xen/KVM
• Dynamic deployment over Eucalyptus/EC2
• Databases supported:
– HBase, Hypertable, MySQL, Cassandra,
Voldemort
• Fault-tolerant
29. AppScale Recap
• Open cloud research platform
– International user community
• Goals
– Easy to use and extend
– Automatic deployment of PaaS cloud and
GAE apps on resources other than Google’s
– Support real applications and users
• Experimentation and testing in real environments
• Current performance results are a baseline
30. Performance Improvements
• AppEngine now multi-process, load balanced
• PB Server now multi-threaded
• Storing data like Google for HBase and
Hypertable
– Three tables: Reference, Sort Ascending, Sort
Descending
31. Future Work
• Expand out of the web services domain
– Investigating opportunities in streaming
– Integrated MapReduce support for high-
performance computing (HPC)
– Co-locate AppEngines and use shared
memory
• Additional databases:
– MongoDB, Scalaris, CouchDB
32. Thanks!
• To the AppScale team!
– Co-lead Navraj Chohan
– Advisor Prof. Chandra Krintz
• To the open-source community
• To Google, NSF, and IBM for financial support
• To you all for coming out today
• Check us out on the web:
– http://appscale.cs.ucsb.edu