Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Cloud Computing
and
Software Engineering
Ravindra Dastikop
http://dastikop.blogspot.in
View video of this presentation :
h...
Purpose 
●Cloud represents at the same time a new emerging
opportunity and new challenge
●This presentation aims to identi...
Agenda for Presentation
●What is a Cloud Application? (project)
●Cloud resources for cloud application
developers
●Cloud a...
Evolution of Computer
Applications
Desktop Applications
Full control and ownership of machine, data,
software
Web Application
No ownership, data stored elsewhere, application comes
from service provider
Rich, remote and responsive (now)
Picture credit: http://wiki.sproutcore.com
Goal of Cloud Applications
●Cloud application represents the state of
art in application development
●They try to combine ...
Cloud Application :
Meaning
● Cloud computing applications are the applications available
as SaaS over the internet 
● whi...
Cloud Application :
Where do they come from?
●Cloud computing applications are loaded at the
server of service provider an...
CloudApp :
A Simple Example
CloudApp allows you to share as well as save files
easily on the web. It works well for links,...
Any Questions?
Cloud Apps:
The difference
Statelessness and server failures are givens in the
cloud
The big difference between a traditio...
Cloud Apps: The difference
Cloud Applications have to be stateless
If cloud application has states, it becomes a challenge...
Cloud Apps: The difference
Cloud applications need to be designed for
redundancy and should acknowledge that commodity
mac...
Cloud Apps: The difference
Cloud Apps parts can be scattered in many places
● Presentation layer might be on the Facebook
...
Cloud Apps: The difference
Database are not same in the cloud
Cloud databases are non-relational
schema-free
Cloud Apps: The difference
Database are not same in the cloud
Example:
Google App Engine uses Google’s Big Table data stor...
Cloud Apps: The difference
Database are not same in the cloud
In cloud database using Big Table - denormalization
is encou...
Cloud Apps: The difference
Database are not same in the cloud
What developers have found that in very high-traffic
situati...
Cloud Apps: The difference
Get used to rapid change in the cloud
● Things change much more rapidly in the cloud.
● Cloud p...
Cloud Apps: The difference
Developer need to be aware of different design
patterns such as eventual consistency- in which ...
Cloud Apps: The difference
Developers can let go most of plumbing concerns
in the Cloud
The loosely coupled nature of web ...
Cloud Apps: The difference
One enterprise wanted to build college
admission applications on Microsoft’s .NET
platform but ...
Any Questions?
Cloud Apps: The difference
Developers need to keep in mind the
difference between cloud platforms and
licensing models dif...
Cloud Apps: The difference
Developers should develop applications in such a way
that they could be moved off that platform...
Software engineering
Software engineering is the application of systematic,
disciplined, quantifiable approach to the deve...
Cloud Apps: The difference
ASPECT Traditional
Software
multilateral
development (Cloud
Software
Composition
coherent set o...
Cloud Apps: The difference
ASPECT Traditional Software multilateral
development ( Cloud
source
code
full source available ...
Cloud Apps: The difference
ASPECT Traditional Software multilateral
development ( Cloud
execution
model
single computer of...
Cloud Apps: The difference
ASPECT Traditional Software multilateral
development ( Cloud
Ownership
and control
single team ...
Cloud Apps: Challenges
ASPECT cloud computing Challenge
Source
code
No source code for
third party
compoenents
system comp...
Any Questions?
Cloud Apps: key principles
● The Map-Reduce paradigm for independent
computation
● schema-free databases and their use
● s...
Cloud Apps: promises
● Reduced development time through use of
high-level service
● maximum utilization of resources: pay ...
Cloud Computing
Resources
What is available for developers
Cloud Application Building Resources
Open source
●Java
●C# DotNet
●AJAX- real time collaboration
●GAE- Google Application ...
Any Questions?
Developing a Cloud
Application
Demonstration
Cloud applications using Open sources
Cloud Teaching System
using
Google Application Engine(GAE)
(Our Final Year Project)
...
My Maps Locator
developed using Google Map API
Challenge is customizing it for Mobile
A simple Cloud application
Examples- Business
Housing Maps Route Finder
Facebook Game
Deployed on AWS.
Fametown
Best Example
Twitter
Cloud Application Using Open Source
Social site aggregator
using
Facebook
Twitter
Linkedlin
( A Final Year Project )
CloudCourse:
An Open source cloud application
CloudCourse is a course scheduling system. It is built on
GAE
CloudCourse
Cloud Applications using Open Source
AlumniMaps
Procures Facebook users data and Plots on Google
Uses Flex, Google Maps AP...
Training and Placement cell ( PES,
Mandya
Training and Placement cell
using Google App Engine and Google APP Store
Virtualization Project ( NIE, Mysore)
Cloud Computing for SMES- BVB
Students 
Cloud Computing for SMEs
First version of this project is done by a group of Hyder...
FaceBook Applications
Developing a Facebook Application: First Step
Developing FB applications: Tools
Blog Post Translator
Application Objective:
To translate Blog post in English to Different languages
Resource used
● Googl...
AWS Applications
Building Your First Cloud Application on AWS
MS in Software Engineering
To be Presented at
Recent Advances in Web Technologies,
St.Joseph Engineering College, Mangalore,
27-29 Jan, 2011
Cloud Co...
Purpose
In this this presentation , we
●explain why cloud applications business
appeal
●list challenges encountered during...
http://dastikop.blogspot.com
Agenda for Presentation
●Challenges- Developing a cloud
application?
●Cloud Engineering – Dis...
http://dastikop.blogspot.com
Emergence of Cloud Computing Age
●Cloud has become
a preferred destination for enterprises to...
Understanding Cloud Phenomenon
In order to understand Cloud Engineering, it is
necessary to know about Cloud Computing
and...
The Challenge
Developing Cloud scale Applications
Cloud Application: An Example
The Cloud Evolution
CloudApp
allows something as fun and simple as sharing
of images, links,...
http://dastikop.blogspot.com
Why cloud-deployable
applications?
●Cloud computing infrastructure is an
interesting proposit...
How a FREE Cloud Application can
become a business
Cloudapp Pro is available with
more features and is now charged
http://dastikop.blogspot.com
Challenges in Cloud Computing
●Number 1. Business Continuity and Service Availability
●Number...
Cloud engineering- Business
Reason
●Cloud services typically deliver
commodity-like capabilities, often with
consumer-grad...
http://dastikop.blogspot.com
Requirements of Cloud
application
●High reliability
●Dynamic scalability
○Millions of users
○...
http://dastikop.blogspot.com
Cloud
Engineering
Building Cloud applications
Cloud Engineering
●Cloud Engineering is the process of
designing the systems necessary to
leverage the power and economics...
Cloud Engineering
An Analogy
Comparing CE with Transportation
engineering
Transportation engineering ( Golden
Qudrangle
Transportation engineering
Build Roads
Transportation engineering
Build Vehicles ( Applications)
Transportation Engineering
Develop Standard Practices (Rules)
Cloud engineering Cycle
Source:
http://dastikop.blogspot.com
TRADITIONAL APPROACH
1
2
3 4
http://dastikop.blogspot.com
http://dastikop.blogspot.com
Engineering Perspective
Research and Practical
challenges
http://dastikop.blogspot.com
A cloud application: Ticketing
System
●Describe the systems
Cloud Engineering : Simple definition
Cloud engineering is the process of
designing systems to leverage cloud
architecture
http://dastikop.blogspot.com
Challenge No 1: High Reliability
Services without any disruption
●Vertical scaling
○Adding mo...
http://dastikop.blogspot.com
Applications scale up
●Scaling up must happen in all tiers –
○web,
○messaging and
○database
http://dastikop.blogspot.com
Scaling Up database-1
●Databases are not built for scale
●The primary factor for scaling up a...
http://dastikop.blogspot.com
Scaling up of database –Option 2
●Adding more database instances with master-
slave replicati...
http://dastikop.blogspot.com
Example- Master-slave
●Master-slave is deployed in Dekoh.com.
●During user registration at De...
http://dastikop.blogspot.com
Sharding and De-normalization
●Portioning data across master would distribute
writes to diffe...
http://dastikop.blogspot.com
The consequence of Sharding
●Portioning data into shards add more complexity
in terms of main...
http://dastikop.blogspot.com
De-normalization
●De-normalization is basically bringing ,
often redundant data together, to ...
http://dastikop.blogspot.com
De-normalization – an example
● One application has a message infrastructure where messages a...
http://dastikop.blogspot.com
Cloud Engineering by Google
●Google has come out with its own datastore
implementation
●Googl...
http://dastikop.blogspot.com
Cloud Engineering by Amazon
●Amazon’s SimpleDB is a distributed key-value store
which support...
http://dastikop.blogspot.com
Cloud Engineering by Facebook
●Facebook uses Apache Hive
●It is a data warehouse that runs on...
Contact Information
Ravindra Dastikop
Assistant Professor, CSE
SDM College of Engineering & Technology
Dharwad 580 002
ema...
Upcoming SlideShare
Loading in …5
×

Cloud computing and software engineering

2,344 views

Published on

Cloud computing is new platform for developers. It provides new opportunities and poses new challenges. This presentation considers software engineering in the context of cloud application/service development

Published in: Education
  • Be the first to comment

Cloud computing and software engineering

  1. 1. Cloud Computing and Software Engineering Ravindra Dastikop http://dastikop.blogspot.in View video of this presentation : http://youtube.com/user/dastikop
  2. 2. Purpose  ●Cloud represents at the same time a new emerging opportunity and new challenge ●This presentation aims to identify the challenges inherent to cloud applications and suggests some approaches towards the same.
  3. 3. Agenda for Presentation ●What is a Cloud Application? (project) ●Cloud resources for cloud application developers ●Cloud application examples ○General examples ○Cloud applications (that our students have built) ●A sample cloud Application demo ●Q & A session ●Summary 
  4. 4. Evolution of Computer Applications
  5. 5. Desktop Applications Full control and ownership of machine, data, software
  6. 6. Web Application No ownership, data stored elsewhere, application comes from service provider
  7. 7. Rich, remote and responsive (now) Picture credit: http://wiki.sproutcore.com
  8. 8. Goal of Cloud Applications ●Cloud application represents the state of art in application development ●They try to combine best of both worlds - ○richness of desktop with ○remoteness of web applications   
  9. 9. Cloud Application : Meaning ● Cloud computing applications are the applications available as SaaS over the internet  ● which facilitates the user to run those applications without installing them on his own computer.  ● This saves the cost of infrastructure and hardware maintenance.  ● In general cloud computing customers do not own the infrastructure but use it from a third party provider as a service.
  10. 10. Cloud Application : Where do they come from? ●Cloud computing applications are loaded at the server of service provider and the service provider give access to these applications through an interface using internet. ●Every customer has its own unique account from which they log on to the cloud of the provider and access the application they have paid for.  ●Service provider also provides space at his own server for the customer’s data.
  11. 11. CloudApp : A Simple Example CloudApp allows you to share as well as save files easily on the web. It works well for links, images, video and music. Check out how the entire thing works- ●Select a file ●Drag the file to the menu bar  CloudAPP
  12. 12. Any Questions?
  13. 13. Cloud Apps: The difference Statelessness and server failures are givens in the cloud The big difference between a traditional application and cloud application is that cloud application itself is able to provision the resources that the application needs Using API’s applications for cloud can be designed so that they request more resources from the cloud provider
  14. 14. Cloud Apps: The difference Cloud Applications have to be stateless If cloud application has states, it becomes a challenge In cloud if something dies, you kill it and reincarnate There is no concept of a local disk, no registry All these is encapsulated by being a stateless application
  15. 15. Cloud Apps: The difference Cloud applications need to be designed for redundancy and should acknowledge that commodity machines are being used in the cloud It is a guarantee that machines are going to fail, hence cloud application need to designed for redundancy
  16. 16. Cloud Apps: The difference Cloud Apps parts can be scattered in many places ● Presentation layer might be on the Facebook storage could be on Amazon’s S3 ● application logic could run somewhere else entirely In traditional applications, entire application used to built on own servers Building cloud application requires solid engineering and design
  17. 17. Cloud Apps: The difference Database are not same in the cloud Cloud databases are non-relational schema-free
  18. 18. Cloud Apps: The difference Database are not same in the cloud Example: Google App Engine uses Google’s Big Table data store for persistent storage. Big Table is not a SQL database, and the reason for that is because some of the functionalities supported by SQL databases - for example- joins- make it very difficult to split a database across multiple machines
  19. 19. Cloud Apps: The difference Database are not same in the cloud In cloud database using Big Table - denormalization is encouraged from the design phase. This enables developers to store data in multiple places at the same time. As a consequence applications run very efficient queries
  20. 20. Cloud Apps: The difference Database are not same in the cloud What developers have found that in very high-traffic situations, relational databases are extremely difficult to manage and that ends up being a huge money and resource sink for developers
  21. 21. Cloud Apps: The difference Get used to rapid change in the cloud ● Things change much more rapidly in the cloud. ● Cloud providers offer new releases several time a year and each upgrade might have something a developer wants to take advantage of ● A developer need to stay abreast of those developments, keep eye on lot of the different blogs and also particpate in webinars
  22. 22. Cloud Apps: The difference Developer need to be aware of different design patterns such as eventual consistency- in which a change to an application might not register for a few million seconds. The consequence of that is that you can not utilize a database to keep track of the next value “ This needs a different programming approach when they are utilizing the cloud because of such things
  23. 23. Cloud Apps: The difference Developers can let go most of plumbing concerns in the Cloud The loosely coupled nature of web makes it an easier development platform. Developers can focus on innovation and business logic instead of worrying about plumbing and infrastructure such as the operating system and hardware. Cloud service providers offer security, workflow, administration and load-balancing as seperate modules
  24. 24. Cloud Apps: The difference One enterprise wanted to build college admission applications on Microsoft’s .NET platform but found that it was many times cheaper to develop on Force-.com- this is because of its use of pre-built functionality.
  25. 25. Any Questions?
  26. 26. Cloud Apps: The difference Developers need to keep in mind the difference between cloud platforms and licensing models different cloud models have different pricing models
  27. 27. Cloud Apps: The difference Developers should develop applications in such a way that they could be moved off that platform For example, Google enables such mobility by supporting the popular Python language and the Django web framework and supports open source uploading and downloading tools for moving data in and out of GAE
  28. 28. Software engineering Software engineering is the application of systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software Cloud computing offers new possibilities for multilateral software
  29. 29. Cloud Apps: The difference ASPECT Traditional Software multilateral development (Cloud Software Composition coherent set of software modules interoperable third party components
  30. 30. Cloud Apps: The difference ASPECT Traditional Software multilateral development ( Cloud source code full source available no source code for third party companies
  31. 31. Cloud Apps: The difference ASPECT Traditional Software multilateral development ( Cloud execution model single computer often distributed between multiple computeres
  32. 32. Cloud Apps: The difference ASPECT Traditional Software multilateral development ( Cloud Ownership and control single team or enterprise distributed between multiple enterr
  33. 33. Cloud Apps: Challenges ASPECT cloud computing Challenge Source code No source code for third party compoenents system comprehension Execution model often distributed between multiple computers state inspection and debugging Ownership and control distributed separation of ownership
  34. 34. Any Questions?
  35. 35. Cloud Apps: key principles ● The Map-Reduce paradigm for independent computation ● schema-free databases and their use ● service-oriented computing ● multi-tenancy ● security and compliance ● design for resilience ● loosely-coupled
  36. 36. Cloud Apps: promises ● Reduced development time through use of high-level service ● maximum utilization of resources: pay for what u use ● reduced operation costs ● increased development productivity ●
  37. 37. Cloud Computing Resources What is available for developers
  38. 38. Cloud Application Building Resources Open source ●Java ●C# DotNet ●AJAX- real time collaboration ●GAE- Google Application Engine ●PHP It provides web developers with a full suite of tools for building dynamic websites ●Python: integrate your systems more effectively ●API - Royal route resources Google APIs, ●Facebook Commercial ●AWS - Amazon Web services ●Microsoft Azure
  39. 39. Any Questions?
  40. 40. Developing a Cloud Application Demonstration
  41. 41. Cloud applications using Open sources Cloud Teaching System using Google Application Engine(GAE) (Our Final Year Project) A Placed and Happy Student
  42. 42. My Maps Locator developed using Google Map API Challenge is customizing it for Mobile A simple Cloud application
  43. 43. Examples- Business Housing Maps Route Finder
  44. 44. Facebook Game Deployed on AWS. Fametown Best Example Twitter
  45. 45. Cloud Application Using Open Source Social site aggregator using Facebook Twitter Linkedlin ( A Final Year Project )
  46. 46. CloudCourse: An Open source cloud application CloudCourse is a course scheduling system. It is built on GAE CloudCourse
  47. 47. Cloud Applications using Open Source AlumniMaps Procures Facebook users data and Plots on Google Uses Flex, Google Maps API, Facebook API Final Year Project A Happy Student
  48. 48. Training and Placement cell ( PES, Mandya Training and Placement cell using Google App Engine and Google APP Store
  49. 49. Virtualization Project ( NIE, Mysore)
  50. 50. Cloud Computing for SMES- BVB Students  Cloud Computing for SMEs First version of this project is done by a group of Hyderabad engineering students. BVB gr oup is build it further on
  51. 51. FaceBook Applications Developing a Facebook Application: First Step Developing FB applications: Tools
  52. 52. Blog Post Translator Application Objective: To translate Blog post in English to Different languages Resource used ● Google App Engine- A Paas Solution ● Google Translate API ● Language: Python
  53. 53. AWS Applications Building Your First Cloud Application on AWS
  54. 54. MS in Software Engineering
  55. 55. To be Presented at Recent Advances in Web Technologies, St.Joseph Engineering College, Mangalore, 27-29 Jan, 2011 Cloud Computing: Engineering Approach click here to see updated version Ravindra Dastikop email : Ravindra.dastikop@gmail.com http://dastikop.blogspot.com
  56. 56. Purpose In this this presentation , we ●explain why cloud applications business appeal ●list challenges encountered during development of cloud scale applications ●provide an introduction to newly emerging discipline – Cloud Engineering ●Share our experience of Cloud Engineering
  57. 57. http://dastikop.blogspot.com Agenda for Presentation ●Challenges- Developing a cloud application? ●Cloud Engineering – Discipline for develop cloud applications ●Q & A ●Summary
  58. 58. http://dastikop.blogspot.com Emergence of Cloud Computing Age ●Cloud has become a preferred destination for enterprises to host applications ●Designing and building applications for the Cloud requires specialized skills ●It demands a new mind set and also architecrual thinking (MS)
  59. 59. Understanding Cloud Phenomenon In order to understand Cloud Engineering, it is necessary to know about Cloud Computing and Cloud application. Please refer to the following presentations before u proceed further ●Cloud computing- Foundation of Cloud computing ●Cloud application- Fruits of Cloud Computing ●
  60. 60. The Challenge Developing Cloud scale Applications
  61. 61. Cloud Application: An Example The Cloud Evolution CloudApp allows something as fun and simple as sharing of images, links, music, videos and files by simple choosing a file and drag it to the menu bar to be easy, faster and reliable by providing short link automatically copied to your clipboard that you can use to share your upload with co-workers and friends.
  62. 62. http://dastikop.blogspot.com Why cloud-deployable applications? ●Cloud computing infrastructure is an interesting proposition because of its ●Pay-per-use model ●Ability to leverage the benefits of platform as a services. ●Result: Cloud application is ○A major requirement (business) ○A challenge for developers
  63. 63. How a FREE Cloud Application can become a business Cloudapp Pro is available with more features and is now charged
  64. 64. http://dastikop.blogspot.com Challenges in Cloud Computing ●Number 1. Business Continuity and Service Availability ●Number 2. Data Lock-In ●Number 3. Data Confidentiality/Auditability ●Number 4. Data Transfer Bottlenecks ●Number 5. Performance Unpredictability ●Number 6: Scalable Storage ●Number 7: Bugs in Large-Scale Distributed Systems ●Number 8: Scaling Quickly ●Number 9: Reputation Fate Sharing ●Number 10: Software Licensing Source:
  65. 65. Cloud engineering- Business Reason ●Cloud services typically deliver commodity-like capabilities, often with consumer-grade service-level agreements, and organizations will be dealing with the inherent challenges in this business model
  66. 66. http://dastikop.blogspot.com Requirements of Cloud application ●High reliability ●Dynamic scalability ○Millions of users ○Volumes of data ●Low latency ●Across all layers ○Database ○Messaging ○Web
  67. 67. http://dastikop.blogspot.com Cloud Engineering Building Cloud applications
  68. 68. Cloud Engineering ●Cloud Engineering is the process of designing the systems necessary to leverage the power and economics of cloud resources to solve business problems.
  69. 69. Cloud Engineering An Analogy Comparing CE with Transportation engineering
  70. 70. Transportation engineering ( Golden Qudrangle
  71. 71. Transportation engineering Build Roads
  72. 72. Transportation engineering Build Vehicles ( Applications)
  73. 73. Transportation Engineering Develop Standard Practices (Rules)
  74. 74. Cloud engineering Cycle Source:
  75. 75. http://dastikop.blogspot.com TRADITIONAL APPROACH 1 2 3 4
  76. 76. http://dastikop.blogspot.com
  77. 77. http://dastikop.blogspot.com Engineering Perspective Research and Practical challenges
  78. 78. http://dastikop.blogspot.com A cloud application: Ticketing System ●Describe the systems
  79. 79. Cloud Engineering : Simple definition Cloud engineering is the process of designing systems to leverage cloud architecture
  80. 80. http://dastikop.blogspot.com Challenge No 1: High Reliability Services without any disruption ●Vertical scaling ○Adding more CPUs and Disks ○Example: A Dating web site scaled up their application to handle over a billion requests per month by moving to 512 GB RAM, 32 CPU Machine ○Cost 100K USD ( costly high-end configuration ●Horizontal scaling ○Second option is to use run application on commodity hardware, scaling horizontally by adding more box, as the need to scale up arose ●Moving application to cloud
  81. 81. http://dastikop.blogspot.com Applications scale up ●Scaling up must happen in all tiers – ○web, ○messaging and ○database
  82. 82. http://dastikop.blogspot.com Scaling Up database-1 ●Databases are not built for scale ●The primary factor for scaling up a database is disk I/O performance ●Vertical Scaling can be achieved by adding high- end disks with greater speeds and replication ●Affordable RAID 6 or RAID 10 disk can be used to improve disk performance ●RAID disks Upper limit on disk transfer speed is 200MBps to 1GBps- which limits scaling
  83. 83. http://dastikop.blogspot.com Scaling up of database –Option 2 ●Adding more database instances with master- slave replication strategy where master handles writes, and replicates data to multiple slaves ●MySQL supports master-slave scale-out configuration where data gets replicated transparently to the slaves. ●When application spends most of the time in reads, the application scales as the reads can be served from any slaves
  84. 84. http://dastikop.blogspot.com Example- Master-slave ●Master-slave is deployed in Dekoh.com. ●During user registration at Dekoh.com where all new users sign-up requests, which involve write- to database, are routed to the master, and the login requests to slaves ●Since user login occurs frequently compared to user registration, the above configuration scales up well, ●Master does not scale when there are more writes and when it also adds a slave lag as the data gets replicated
  85. 85. http://dastikop.blogspot.com Sharding and De-normalization ●Portioning data across master would distribute writes to different instances and both the reads and writes scale well ●To achieve reliability replication should be used along with lesser number of slaves to overcome the slave lag ●Example: Flickr.com ○Moved from pure replication to the Sharding mechanism to be able to scale ●Shading schemes – need changes in application architecture
  86. 86. http://dastikop.blogspot.com The consequence of Sharding ●Portioning data into shards add more complexity in terms of maintaining the integrity of data, application architecture and Joins. ( ACID) ●Any change in the portioning scheme would require reorganizing the entire data which is expensive ●Joins are not possible as data is broken up into different shards. (Join demands data to be available on same memory space) ●Solution is to introduce some kind of de- normalization – (end of a cherished practice)
  87. 87. http://dastikop.blogspot.com De-normalization ●De-normalization is basically bringing , often redundant data together, to improve the query performance. (fundamental go for a toss)
  88. 88. http://dastikop.blogspot.com De-normalization – an example ● One application has a message infrastructure where messages are sent to recipients (users). ● Messages are stored in “Message” Table and the “recipient” table contains message to recipient mapping. ● To obtain recent 10 messages sent to a particular recipient, all messages-ids for recipient are obtained from the “recipient” table and a join is performed on the “message” table filtering the recent 10 timestamps. ● If there are lot of messages, sent to a particular recipient, a join is performed on the messages table with lot of rows from the “recipients” table and then the time stamp filter is applied. ● Instead, if the timestamp of the message is duplicated in the recipient table, it is easy to filter out the first 10 messages ids and then perform a join. ● With this approach the query takes lesser time to execute and minimize disk I/O
  89. 89. http://dastikop.blogspot.com Cloud Engineering by Google ●Google has come out with its own datastore implementation ●Google’s Bigtable is a distributed ‘scheme-less” key- value store, which was developed for the web search engine. ●It is applied to Orkut, Docs, Google Maps, Earth and others. ●Bigtable runs on the top of Google File System and provides the needed scalability at its core, supporting high availability at the file system level ●Google App Engine offers Bigtable as the primary datastore for application developers.
  90. 90. http://dastikop.blogspot.com Cloud Engineering by Amazon ●Amazon’s SimpleDB is a distributed key-value store which supports a SQL-like syntax for retrieving data and exposes REST API for all operations. ●SimpleDB is available as a paid service and is very effective when working with huge amount of data ●Limitations ●Restrictions imposed on size of results, comparisons, predicates used in the query ●No supports for transactions, aggregate functions, data types and full-text search ●For above reasons, adoption of service is less.
  91. 91. http://dastikop.blogspot.com Cloud Engineering by Facebook ●Facebook uses Apache Hive ●It is a data warehouse that runs on the top of the Hadoop distributed system ●Facebook uses Hive to analyze historical data of users and content using brute force mechanism ●Hive is not a datastore but is only used for analytics on the large amounts of data. ●The other datastore include ○ HBase ○ Hypertable ○ Cassandra ○ CoudDB ○ Voldermort
  92. 92. Contact Information Ravindra Dastikop Assistant Professor, CSE SDM College of Engineering & Technology Dharwad 580 002 email: ravindra.dastikop@gmail.com web site: http://dastikop.blogspot.com

×