Cloud computing and software engineering

Cloud Computing
and
Software Engineering
Ravindra Dastikop
http://dastikop.blogspot.in
View video of this presentation :
http://youtube.com/user/dastikop

Purpose
●Cloud represents at the same time a new emerging
opportunity and new challenge
●This presentation aims to identify the challenges
inherent to cloud applications and suggests some
approaches towards the same.

Agenda for Presentation
●What is a Cloud Application? (project)
●Cloud resources for cloud application
developers
●Cloud application examples
○General examples
○Cloud applications (that our students have
built)
●A sample cloud Application demo
●Q & A session
●Summary

Evolution of Computer
Applications

Desktop Applications
Full control and ownership of machine, data,
software

Web Application
No ownership, data stored elsewhere, application comes
from service provider

Rich, remote and responsive (now)
Picture credit: http://wiki.sproutcore.com

Goal of Cloud Applications
●Cloud application represents the state of
art in application development
●They try to combine best of both worlds -
○richness of desktop with
○remoteness of web applications

Cloud Application :
Meaning
● Cloud computing applications are the applications available
as SaaS over the internet
● which facilitates the user to run those applications without
installing them on his own computer.
● This saves the cost of infrastructure and hardware
maintenance.
● In general cloud computing customers do not own the
infrastructure but use it from a third party provider as a
service.

Cloud Application :
Where do they come from?
●Cloud computing applications are loaded at the
server of service provider and the service provider
give access to these applications through an
interface using internet.
●Every customer has its own unique account from
which they log on to the cloud of the provider and
access the application they have paid for.
●Service provider also provides space at his own
server for the customer’s data.

CloudApp :
A Simple Example
CloudApp allows you to share as well as save files
easily on the web. It works well for links, images,
video and music. Check out how the entire thing
works-
●Select a file
●Drag the file to the menu bar
CloudAPP

Cloud Apps:
The difference
Statelessness and server failures are givens in the
cloud
The big difference between a traditional application
and cloud application is that cloud application itself is
able to provision the resources that the application
needs
Using API’s applications for cloud can be designed so
that they request more resources from the cloud
provider

Cloud Apps: The difference
Cloud Applications have to be stateless
If cloud application has states, it becomes a challenge
In cloud if something dies, you kill it and reincarnate
There is no concept of a local disk, no registry
All these is encapsulated by being a stateless
application

Cloud applications need to be designed for
redundancy and should acknowledge that commodity
machines are being used in the cloud
It is a guarantee that machines are going to fail, hence
cloud application need to designed for redundancy

Cloud Apps parts can be scattered in many places
● Presentation layer might be on the Facebook
storage could be on Amazon’s S3
● application logic could run somewhere else entirely
In traditional applications, entire application used to
built on own servers
Building cloud application requires solid engineering
and design

Database are not same in the cloud
Cloud databases are non-relational
schema-free

Example:
Google App Engine uses Google’s Big Table data store
for persistent storage. Big Table is not a SQL
database, and the reason for that is because some of
the functionalities supported by SQL databases - for
example- joins- make it very difficult to split a
database across multiple machines

In cloud database using Big Table - denormalization
is encouraged from the design phase.
This enables developers to store data in multiple
places at the same time. As a consequence
applications run very efficient queries

What developers have found that in very high-traffic
situations, relational databases are extremely
difficult to manage and that ends up being a huge
money and resource sink for developers

Get used to rapid change in the cloud
● Things change much more rapidly in the cloud.
● Cloud providers offer new releases several time a year
and each upgrade might have something a developer
wants to take advantage of
● A developer need to stay abreast of those developments,
keep eye on lot of the different blogs and also particpate
in webinars

Developer need to be aware of different design
patterns such as eventual consistency- in which a
change to an application might not register for a few
million seconds. The consequence of that is that you
can not utilize a database to keep track of the next
value “
This needs a different programming approach when
they are utilizing the cloud because of such things

Developers can let go most of plumbing concerns
in the Cloud
The loosely coupled nature of web makes it an easier
development platform.
Developers can focus on innovation and business logic
instead of worrying about plumbing and infrastructure such
as the operating system and hardware.
Cloud service providers offer security, workflow,
administration and load-balancing as seperate modules

One enterprise wanted to build college
admission applications on Microsoft’s .NET
platform but found that it was many times
cheaper to develop on Force-.com- this is
because of its use of pre-built functionality.

Developers need to keep in mind the
difference between cloud platforms and
licensing models different cloud models have
different pricing models

Developers should develop applications in such a way
that they could be moved off that platform
For example, Google enables such mobility by
supporting the popular Python language and the
Django web framework and supports open source
uploading and downloading tools for moving data in
and out of GAE

Software engineering
Software engineering is the application of systematic,
disciplined, quantifiable approach to the development,
operation, and maintenance of software
Cloud computing offers new possibilities for
multilateral software

ASPECT Traditional
Software
multilateral
development (Cloud
Software
Composition
coherent set of
software
modules
interoperable third
party components

ASPECT Traditional Software multilateral
development ( Cloud
source
code
full source available no source code for
third party companies

development ( Cloud
execution
model
single computer often distributed
between multiple
computeres

development ( Cloud
Ownership
and control
single team or
enterprise
distributed between
multiple enterr

Cloud Apps: Challenges
ASPECT cloud computing Challenge
Source
code
No source code for
third party
compoenents
system comprehension
Execution
model
often distributed
between multiple
computers
state inspection and
debugging
Ownership
and control
distributed separation of
ownership

Cloud Apps: key principles
● The Map-Reduce paradigm for independent
computation
● schema-free databases and their use
● service-oriented computing
● multi-tenancy
● security and compliance
● design for resilience
● loosely-coupled

Cloud Apps: promises
● Reduced development time through use of
high-level service
● maximum utilization of resources: pay for
what u use
● reduced operation costs
● increased development productivity
●

Cloud Computing
Resources
What is available for developers

Cloud Application Building Resources
Open source
●Java
●C# DotNet
●AJAX- real time collaboration
●GAE- Google Application Engine
●PHP It provides web developers with a full suite of tools for
building dynamic websites
●Python: integrate your systems more effectively
●API - Royal route resources Google APIs,
●Facebook
Commercial
●AWS - Amazon Web services
●Microsoft Azure

Developing a Cloud
Application
Demonstration

Cloud applications using Open sources
Cloud Teaching System
using
Google Application Engine(GAE)
(Our Final Year Project)
A Placed and Happy Student

My Maps Locator
developed using Google Map API
Challenge is customizing it for Mobile
A simple Cloud application

Examples- Business
Housing Maps Route Finder

Facebook Game
Deployed on AWS.
Fametown
Best Example
Twitter

Cloud Application Using Open Source
Social site aggregator
using
Facebook
Twitter
Linkedlin
( A Final Year Project )

CloudCourse:
An Open source cloud application
CloudCourse is a course scheduling system. It is built on
GAE
CloudCourse

Cloud Applications using Open Source
AlumniMaps
Procures Facebook users data and Plots on Google
Uses Flex, Google Maps API, Facebook API
Final Year Project
A Happy Student

Training and Placement cell ( PES,
Mandya
Training and Placement cell
using Google App Engine and Google APP Store

Virtualization Project ( NIE, Mysore)

Cloud Computing for SMES- BVB
Students
Cloud Computing for SMEs
First version of this project is done by a group of Hyderabad
engineering students.
BVB gr oup is build it further on

FaceBook Applications
Developing a Facebook Application: First Step
Developing FB applications: Tools

Blog Post Translator
Application Objective:
To translate Blog post in English to Different languages
Resource used
● Google App Engine- A Paas Solution
● Google Translate API
● Language: Python

AWS Applications
Building Your First Cloud Application on AWS

To be Presented at
Recent Advances in Web Technologies,
St.Joseph Engineering College, Mangalore,
27-29 Jan, 2011
Cloud Computing:
Engineering Approach
click here to see updated version
Ravindra Dastikop
email : Ravindra.dastikop@gmail.com
http://dastikop.blogspot.com

Purpose
In this this presentation , we
●explain why cloud applications business
appeal
●list challenges encountered during
development of cloud scale applications
●provide an introduction to newly emerging
discipline – Cloud Engineering
●Share our experience of Cloud Engineering

Agenda for Presentation
●Challenges- Developing a cloud
application?
●Cloud Engineering – Discipline for
develop cloud applications
●Q & A
●Summary

Emergence of Cloud Computing Age
●Cloud has become
a preferred destination for enterprises to
host applications
●Designing and building applications for the
Cloud requires specialized skills
●It demands a new mind set and also
architecrual thinking (MS)

Understanding Cloud Phenomenon
In order to understand Cloud Engineering, it is
necessary to know about Cloud Computing
and Cloud application. Please refer to the
following presentations before u proceed
further
●Cloud computing- Foundation of Cloud computing
●Cloud application- Fruits of Cloud Computing
●

The Challenge
Developing Cloud scale Applications

Cloud Application: An Example
The Cloud Evolution
CloudApp
allows something as fun and simple as sharing
of images, links, music, videos and files by simple choosing
a file and drag it to the menu bar to be easy, faster and reliable
by providing short link automatically copied to your clipboard
that you can use to share your upload with co-workers and
friends.

Why cloud-deployable
applications?
●Cloud computing infrastructure is an
interesting proposition because of its
●Pay-per-use model
●Ability to leverage the benefits of platform
as a services.
●Result: Cloud application is
○A major requirement (business)
○A challenge for developers

How a FREE Cloud Application can
become a business
Cloudapp Pro is available with
more features and is now charged

Challenges in Cloud Computing
●Number 1. Business Continuity and Service Availability
●Number 2. Data Lock-In
●Number 3. Data Confidentiality/Auditability
●Number 4. Data Transfer Bottlenecks
●Number 5. Performance Unpredictability
●Number 6: Scalable Storage
●Number 7: Bugs in Large-Scale Distributed Systems
●Number 8: Scaling Quickly
●Number 9: Reputation Fate Sharing
●Number 10: Software Licensing
Source:

Cloud engineering- Business
Reason
●Cloud services typically deliver
commodity-like capabilities, often with
consumer-grade service-level
agreements, and organizations will be
dealing with the inherent challenges
in this business model

Requirements of Cloud
application
●High reliability
●Dynamic scalability
○Millions of users
○Volumes of data
●Low latency
●Across all layers
○Database
○Messaging
○Web

Cloud
Engineering
Building Cloud applications

Cloud Engineering
●Cloud Engineering is the process of
designing the systems necessary to
leverage the power and economics
of cloud resources to solve business
problems.

Cloud Engineering
An Analogy
Comparing CE with Transportation
engineering

Transportation engineering ( Golden
Qudrangle

Transportation engineering
Build Roads

Transportation engineering
Build Vehicles ( Applications)

Transportation Engineering
Develop Standard Practices (Rules)

Cloud engineering Cycle
Source:

TRADITIONAL APPROACH
1
2
3 4

Engineering Perspective
Research and Practical
challenges

A cloud application: Ticketing
System
●Describe the systems

Cloud Engineering : Simple definition
Cloud engineering is the process of
designing systems to leverage cloud
architecture

Challenge No 1: High Reliability
Services without any disruption
●Vertical scaling
○Adding more CPUs and Disks
○Example: A Dating web site scaled up their application
to handle over a billion requests per month by moving
to 512 GB RAM, 32 CPU Machine
○Cost 100K USD ( costly high-end configuration
●Horizontal scaling
○Second option is to use run application on commodity
hardware, scaling horizontally by adding more box, as
the need to scale up arose
●Moving application to cloud

Applications scale up
●Scaling up must happen in all tiers –
○web,
○messaging and
○database

Scaling Up database-1
●Databases are not built for scale
●The primary factor for scaling up a database is
disk I/O performance
●Vertical Scaling can be achieved by adding high-
end disks with greater speeds and replication
●Affordable RAID 6 or RAID 10 disk can be used
to improve disk performance
●RAID disks Upper limit on disk transfer speed is
200MBps to 1GBps- which limits scaling

Scaling up of database –Option 2
●Adding more database instances with master-
slave replication strategy where master handles
writes, and replicates data to multiple slaves
●MySQL supports master-slave scale-out
configuration where data gets replicated
transparently to the slaves.
●When application spends most of the time in
reads, the application scales as the reads can
be served from any slaves

Example- Master-slave
●Master-slave is deployed in Dekoh.com.
●During user registration at Dekoh.com where all
new users sign-up requests, which involve write-
to database, are routed to the master, and the
login requests to slaves
●Since user login occurs frequently compared to
user registration, the above configuration scales
up well,
●Master does not scale when there are more
writes and when it also adds a slave lag as the
data gets replicated

Sharding and De-normalization
●Portioning data across master would distribute
writes to different instances and both the reads
and writes scale well
●To achieve reliability replication should be used
along with lesser number of slaves to overcome
the slave lag
●Example: Flickr.com
○Moved from pure replication to the Sharding
mechanism to be able to scale
●Shading schemes – need changes in application
architecture

The consequence of Sharding
●Portioning data into shards add more complexity
in terms of maintaining the integrity of data,
application architecture and Joins. ( ACID)
●Any change in the portioning scheme would
require reorganizing the entire data which is
expensive
●Joins are not possible as data is broken up into
different shards. (Join demands data to be
available on same memory space)
●Solution is to introduce some kind of de-
normalization – (end of a cherished practice)

De-normalization
●De-normalization is basically bringing ,
often redundant data together, to improve
the query performance. (fundamental go
for a toss)

De-normalization – an example
● One application has a message infrastructure where messages are
sent to recipients (users).
● Messages are stored in “Message” Table and the “recipient” table
contains message to recipient mapping.
● To obtain recent 10 messages sent to a particular recipient, all
messages-ids for recipient are obtained from the “recipient” table
and a join is performed on the “message” table filtering the recent 10
timestamps.
● If there are lot of messages, sent to a particular recipient, a join is
performed on the messages table with lot of rows from the
“recipients” table and then the time stamp filter is applied.
● Instead, if the timestamp of the message is duplicated in the
recipient table, it is easy to filter out the first 10 messages ids and
then perform a join.
● With this approach the query takes lesser time to execute and
minimize disk I/O

Cloud Engineering by Google
●Google has come out with its own datastore
implementation
●Google’s Bigtable is a distributed ‘scheme-less” key-
value store, which was developed for the web search
engine.
●It is applied to Orkut, Docs, Google Maps, Earth and
others.
●Bigtable runs on the top of Google File System and
provides the needed scalability at its core, supporting
high availability at the file system level
●Google App Engine offers Bigtable as the primary
datastore for application developers.

Cloud Engineering by Amazon
●Amazon’s SimpleDB is a distributed key-value store
which supports a SQL-like syntax for retrieving data and
exposes REST API for all operations.
●SimpleDB is available as a paid service and is very
effective when working with huge amount of data
●Limitations
●Restrictions imposed on size of results, comparisons,
predicates used in the query
●No supports for transactions, aggregate functions, data
types and full-text search
●For above reasons, adoption of service is less.

Cloud Engineering by Facebook
●Facebook uses Apache Hive
●It is a data warehouse that runs on the top of the Hadoop
distributed system
●Facebook uses Hive to analyze historical data of users
and content using brute force mechanism
●Hive is not a datastore but is only used for analytics on
the large amounts of data.
●The other datastore include
○ HBase
○ Hypertable
○ Cassandra
○ CoudDB
○ Voldermort

Contact Information
Ravindra Dastikop
Assistant Professor, CSE
SDM College of Engineering & Technology
Dharwad 580 002
email: ravindra.dastikop@gmail.com
web site:

Cloud computing and software engineering

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Cloud computing and software engineering

Similar to Cloud computing and software engineering (20)

More from Ravindra Dastikop

More from Ravindra Dastikop (20)

Recently uploaded

Recently uploaded (20)

Cloud computing and software engineering