File Repository on GAE


Published on

Distributed system class project presentation material.

Published in: Education, Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

File Repository on GAE

  1. 1. File Repository<br />Cloud Computing Topic<br />LinaQu<br />439-Distributed System by Prof. Thiruvathukal<br />2011.05.01<br />
  2. 2. Outline<br />Introduction<br />Cloud Computing<br />Google App Engine™ platform<br />File Repository<br />Summary<br />References<br />
  3. 3. 1. Introduction<br />Using the cloud computing platform, Google App Engine™ platform in this case, to develop and deploy a simple application ---- File repository. <br />Through this project, we can learn briefly:<br /><ul><li>Cloud Computing and its service models;
  4. 4. Google App Engine(GAE) as develop platform;
  5. 5. Big Table as a distributed storage system;
  6. 6. An example project actually runs on GAE.</li></li></ul><li>2. Cloud Computing<br />2.1 What is Cloud Computing?<br />A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet[1].<br />Cloud computing refers to the provision of computational resources on demand via a computer network, such as applications, databases, file services, email, etc.  <br />(----Wikipedia)<br />
  7. 7. 2. Cloud Computing<br />2.2 Key characters of Cloud Computing<br /><ul><li> On-demand self-service
  8. 8. Broad network access
  9. 9. Resource pooling
  10. 10. Rapid elasticity
  11. 11. Measured Service</li></li></ul><li>2. Cloud Computing<br />2.3 Service models of Cloud Computing<br />Software as a Service (SaaS)<br />Platform as a Service (PaaS)<br />Infrastructure as a Service (IaaS)<br />Google Pack,, NetSuite.<br />Google App Engine, Windows® Azure,<br />Amazon EC2 (Elastic Cloud Computing) Service, S3 (Simple Storage Service), IBM® Cloud.<br />
  12. 12. 2. Cloud Computing<br />2.3 Service models of Cloud Computing<br />
  13. 13. 3. Google App Engine™ platform<br />3.1 What is Google App Engine?<br />It lets you run your web applications on Google’s infrastructure.<br />
  14. 14. 3. Google App Engine™ platform<br />3.2 Why Google App Engine?<br /><ul><li> Easy to build
  15. 15. Easy to get Started[17]
  16. 16. Simple app configuration
  17. 17. Easy to maintain
  18. 18. Cost efficient hosting
  19. 19. Risk free trial period
  20. 20. The reliability, performance and security of Google's infrastructure
  21. 21. Easy to scale
  22. 22. Automatic scalability
  23. 23. APIs</li></li></ul><li>3. Google App Engine™ platform<br />3.3 Google App Engine Architecture<br />req/resp<br />stateless APIs<br />R/O FS<br />Python<br />VM<br />process<br />urlfech<br />stdlib<br />mail<br />app<br />images<br />datastore<br />stateful<br />APIs<br />memcache<br />Source: [15] <br />
  24. 24. 3. Google App Engine™ platform<br />3.3 Google App Engine Architecture<br />The App Engine request handling architecture[13].<br />
  25. 25. 3. Google App Engine™ platform<br />3.4 Key Concepts in Google App Engine<br /><ul><li>Sandbox: </li></ul>-Security; It isolates your application in its own secure, reliable environment ; the hardware, operating system and physical location of the web server.<br />
  26. 26. 3. Google App Engine™ platform<br />3.4 Key Concepts in Google App Engine<br /><ul><li>Bigtable: A Distributed Storage System for Structured Data[8].</li></ul>-It uses the distributed Google File System(GFS[25]) to store log and data files.<br />-It uses Google SSTable file format to store data. (persistent, ordered immutable map from keys to values)<br />-It relies on a highly-available and persistent distributed lock service called Chubby.<br />-It is designed to scale to a very large size.<br />-It does not support a full relational data model.<br />
  27. 27. 3. Google App Engine™ platform<br />3.4 Key Concepts in Google App Engine<br /><ul><li>Datastore: A distributed data storage service grows with your data.</li></ul>-Abstraction on top of Bigtable; It provides a scalable, transactional storage system for creating, storing, and querying data objects.<br />-Two data storage options[18, 19]: The Master/Slave Datastore and the High Replication Datastore.<br />- APIs For Java: Java Data Objects(JDO) 2.3 and Java Persistence API (JPA) 1.0<br />
  28. 28. 3. Google App Engine™ platform<br />3.4 Key Concepts in Google App Engine<br /><ul><li>HierarchicalDatastore</li></li></ul><li>3. Google App Engine™ platform<br />3.4 Key Concepts in Google App Engine<br /><ul><li> Indexes
  29. 29. index.yaml;  the development server automatically determines the index configuration for you[22].
  30. 30. Properties are automatically indexed by type+value[15, 21].
  31. 31. Queries</li></ul>-JDOQL:  is similar to SQL, but is more appropriate for object-oriented databases like the App Engine datastore.<br />-GQL[24]: is a SQL-like language for retrieving entities or keys from the App Engine scalable datastore.<br />
  32. 32. 3. Google App Engine™ platform<br />3.5 Develop Web app with Google App Engine<br />GAE Application Development Cycle[10]<br />
  33. 33. 3. Google App Engine™ platform<br />3.6 Managing Your App<br />
  34. 34. 3. Google App Engine™ platform<br />3.6 Managing Your App<br /><ul><li> Quotas</li></ul>-5M pageviews/month<br />-Channel API Calls: 657K calls<br />-Stored Data: 1GB<br />-Datastore API Calls: 141M calss<br />-Number of Indexes: 200<br />-Requests(includes HTTPS): 43.2M requests<br />-CPU Time: 6.5 CPU-hours/day<br />-Task Queue API Calls: 100K<br />-UrlFetch API Calls: 657K calls<br />-XMPP API Calls: 46M calls<br />
  35. 35. 3. Google App Engine™ platform<br />3.7 Support<br /><ul><li> Java[7]
  36. 36. Java 6
  37. 37. Common Java web technology standards
  38. 38. Plugin for the Eclipse IDE
  39. 39. Other languages: JRuby, JavaScript(Thino), Scala.
  40. 40. Spring, Struts 1&2, Apache Commons FileUpload, etc.
  41. 41. Python[27]</li></ul>-Complete Python standard library<br />-Django<br />-CGI or WSGI<br />
  42. 42. 4. File Repository<br />4.1 Function<br /><ul><li> Sign In with Google Accounts
  43. 43. File Upload
  44. 44. File Download
  45. 45. File List
  46. 46. Log Out</li></li></ul><li>4. File Repository<br />4.2 Code<br /><ul><li> Configure</li></ul><servlet><br /><servlet-name>upload</servlet-name><br /><servlet-class>loyola.lqu.server.servlet.FileUploadServlet</servlet-class><br /></servlet><br /><servlet-mapping><br /><servlet-name>upload</servlet-name><br /><url-pattern>/upload</url-pattern><br /></servlet-mapping><br />
  47. 47. 4. File Repository<br />4.2 Code<br /><ul><li> Sign In</li></ul><td><br /><%<br />UserServiceuserService = UserServiceFactory.getUserService();<br />if (!userService.isUserLoggedIn()) {<br />%><br />Please <a<br />href="<%=userService.createLoginURL("/jsp/home.jsp")%>">sign<br />in</a> to upload your file and manage the repository.<br /><% } else {<br />response.sendRedirect("/jsp/home.jsp");<br />}<br /> %></td><br />
  48. 48. 4. File Repository<br />4.2 Code<br /><ul><li> Deploy</li></ul><application>lqu-filerepo</application><br /><version>1</version><br />
  49. 49. 4. File Repository<br />4.3 Technology<br /><ul><li> Apache Commons FileUpload[28]
  50. 50. Apache HttpClient
  51. 51. JSP(CSS, JavaScript)
  52. 52. The Blobstore Java API[29]
  53. 53. The Users Java API[30]</li></li></ul><li>5. Summary<br /><ul><li> Google App Engine
  54. 54. Cloud Computing</li></li></ul><li>6. Reference<br />Cloud Computing and Grid Computing 360-Degree Compared, Ian Foster, Yong Zhao, IoanRaicu, Shiyong Lu.<br />Cloud computing service models, Part 2: Platform as a Service : Learn the benefits of PaaS.<br />Cloud and industry, Part 1: PaaS best practices and patterns : Learn about the requirements and functions of three models to deliver industry solutions, IaaS, PaaS, SaaS, and how you can use best practices and patterns with the PaaS framework in particular to deploy and manage cloud computing solutions. <br />Connecting to the cloud, Part 1: Leverage the cloud in applications: Explore cloud computing and the offerings from the major cloud platform vendors: Amazon, Google, Microsoft®, and<br />Google App Engine Home Page: Learn more about Google App Engine™ platform.<br />Install Google Plugin for Eclipse: Get started with the plug-ins.<br />
  55. 55. 6. Reference<br />Will it play in App Engine for Java?: Lists the level of compatibility of various Java technologies and App Engine (Java).<br />What's BigTable?: Read the Google research publication to find out.<br />Cloud computing fundamentals: Aid you in assimilating the reality of the revolution, so you can use it for your own profit and well being. <br />Stanford University Wiki – Google App Engine Project.<br />Slatkin, Brett. Building Scalable Web Applications with Google App Engine. Google IO 2008.<br />Ashcraft, Ken. Best Practices - Building a Production Quality Application on Google App Engine. Google IO 2008.<br />Dan Sanderson, Programming Google App Engine. O’Reilly, GoogleTM Press.<br />Getting Started: Java. Helpful sample from Google App Engine site.<br />
  56. 56. 6. Reference<br />Presentation by Guido van Rossum. Google App Engine: Run your web applications on Google's infrastructure. Stanford EE Computer Systems Colloquium. 5 Nov 2008.<br />Why App Engine. Brief explain why developers choose Google App Engine.<br />Google App Engine Samples. A list of samples and its source code.<br />Choosing a Datastore(Java). Comparing the Data Storage Options, choose the right one.<br />Datastore Overview(Python).<br />Barrett, Ryan. Under the Covers of the Google App Engine Datastore. Google IO 2008.<br />Introduction to Indexes. <br />Java Index Configuration Using YAML.<br />
  57. 57. 6. Reference<br />Queries in JDO. <br />GQL Reference.<br />The Google File System.  Get a in-depth understanding of GFS. <br />Google App Engine Articles. Articles related to Google App Engine in many aspects by category.<br />Google App Engine Python SDK Release Notes.<br /> Apache Commons FileUpload.<br />The Blobstore Java API.<br />The Users Java API.<br />Google File Service on Google App Engine for Java. Main Reference for code.<br />
  58. 58. End<br />Cloud Computing Topic<br />LinaQu<br />439-Distributed System by Prof. Thiruvathukal<br />2011.05.01<br />