File Repository on GAE

  • 1,626 views
Uploaded on

Distributed system class project presentation material. …

Distributed system class project presentation material.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,626
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
15
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. File Repository
    Cloud Computing Topic
    LinaQu
    439-Distributed System by Prof. Thiruvathukal
    2011.05.01
  • 2. Outline
    Introduction
    Cloud Computing
    Google App Engine™ platform
    File Repository
    Summary
    References
  • 3. 1. Introduction
    Using the cloud computing platform, Google App Engine™ platform in this case, to develop and deploy a simple application ---- File repository.
    Through this project, we can learn briefly:
    • Cloud Computing and its service models;
    • 4. Google App Engine(GAE) as develop platform;
    • 5. Big Table as a distributed storage system;
    • 6. An example project actually runs on GAE.
  • 2. Cloud Computing
    2.1 What is Cloud Computing?
    A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet[1].
    Cloud computing refers to the provision of computational resources on demand via a computer network, such as applications, databases, file services, email, etc.  
    (----Wikipedia)
  • 7. 2. Cloud Computing
    2.2 Key characters of Cloud Computing
    • On-demand self-service
    • 8. Broad network access
    • 9. Resource pooling
    • 10. Rapid elasticity
    • 11. Measured Service
  • 2. Cloud Computing
    2.3 Service models of Cloud Computing
    Software as a Service (SaaS)
    Platform as a Service (PaaS)
    Infrastructure as a Service (IaaS)
    Google Pack, Salesforce.com, NetSuite.
    Google App Engine, Windows® Azure, Force.com.
    Amazon EC2 (Elastic Cloud Computing) Service, S3 (Simple Storage Service), IBM® Cloud.
  • 12. 2. Cloud Computing
    2.3 Service models of Cloud Computing
  • 13. 3. Google App Engine™ platform
    3.1 What is Google App Engine?
    It lets you run your web applications on Google’s infrastructure.
  • 14. 3. Google App Engine™ platform
    3.2 Why Google App Engine?
    • Easy to build
    • 15. Easy to get Started[17]
    • 16. Simple app configuration
    • 17. Easy to maintain
    • 18. Cost efficient hosting
    • 19. Risk free trial period
    • 20. The reliability, performance and security of Google's infrastructure
    • 21. Easy to scale
    • 22. Automatic scalability
    • 23. APIs
  • 3. Google App Engine™ platform
    3.3 Google App Engine Architecture
    req/resp
    stateless APIs
    R/O FS
    Python
    VM
    process
    urlfech
    stdlib
    mail
    app
    images
    datastore
    stateful
    APIs
    memcache
    Source: [15]
  • 24. 3. Google App Engine™ platform
    3.3 Google App Engine Architecture
    The App Engine request handling architecture[13].
  • 25. 3. Google App Engine™ platform
    3.4 Key Concepts in Google App Engine
    • Sandbox:
    -Security; It isolates your application in its own secure, reliable environment ; the hardware, operating system and physical location of the web server.
  • 26. 3. Google App Engine™ platform
    3.4 Key Concepts in Google App Engine
    • Bigtable: A Distributed Storage System for Structured Data[8].
    -It uses the distributed Google File System(GFS[25]) to store log and data files.
    -It uses Google SSTable file format to store data. (persistent, ordered immutable map from keys to values)
    -It relies on a highly-available and persistent distributed lock service called Chubby.
    -It is designed to scale to a very large size.
    -It does not support a full relational data model.
  • 27. 3. Google App Engine™ platform
    3.4 Key Concepts in Google App Engine
    • Datastore: A distributed data storage service grows with your data.
    -Abstraction on top of Bigtable; It provides a scalable, transactional storage system for creating, storing, and querying data objects.
    -Two data storage options[18, 19]: The Master/Slave Datastore and the High Replication Datastore.
    - APIs For Java: Java Data Objects(JDO) 2.3 and Java Persistence API (JPA) 1.0
  • 28. 3. Google App Engine™ platform
    3.4 Key Concepts in Google App Engine
    • HierarchicalDatastore
  • 3. Google App Engine™ platform
    3.4 Key Concepts in Google App Engine
    • Indexes
    • 29. index.yaml;  the development server automatically determines the index configuration for you[22].
    • 30. Properties are automatically indexed by type+value[15, 21].
    • 31. Queries
    -JDOQL:  is similar to SQL, but is more appropriate for object-oriented databases like the App Engine datastore.
    -GQL[24]: is a SQL-like language for retrieving entities or keys from the App Engine scalable datastore.
  • 32. 3. Google App Engine™ platform
    3.5 Develop Web app with Google App Engine
    GAE Application Development Cycle[10]
  • 33. 3. Google App Engine™ platform
    3.6 Managing Your App
  • 34. 3. Google App Engine™ platform
    3.6 Managing Your App
    • Quotas
    -5M pageviews/month
    -Channel API Calls: 657K calls
    -Stored Data: 1GB
    -Datastore API Calls: 141M calss
    -Number of Indexes: 200
    -Requests(includes HTTPS): 43.2M requests
    -CPU Time: 6.5 CPU-hours/day
    -Task Queue API Calls: 100K
    -UrlFetch API Calls: 657K calls
    -XMPP API Calls: 46M calls
  • 35. 3. Google App Engine™ platform
    3.7 Support
    • Java[7]
    • 36. Java 6
    • 37. Common Java web technology standards
    • 38. Plugin for the Eclipse IDE
    • 39. Other languages: JRuby, JavaScript(Thino), Scala.
    • 40. Spring, Struts 1&2, Apache Commons FileUpload, etc.
    • 41. Python[27]
    -Complete Python standard library
    -Django
    -CGI or WSGI
  • 42. 4. File Repository
    4.1 Function
  • 4. File Repository
    4.2 Code
    • Configure
    <servlet>
    <servlet-name>upload</servlet-name>
    <servlet-class>loyola.lqu.server.servlet.FileUploadServlet</servlet-class>
    </servlet>
    <servlet-mapping>
    <servlet-name>upload</servlet-name>
    <url-pattern>/upload</url-pattern>
    </servlet-mapping>
  • 47. 4. File Repository
    4.2 Code
    • Sign In
    <td>
    <%
    UserServiceuserService = UserServiceFactory.getUserService();
    if (!userService.isUserLoggedIn()) {
    %>
    Please <a
    href="<%=userService.createLoginURL("/jsp/home.jsp")%>">sign
    in</a> to upload your file and manage the repository.
    <% } else {
    response.sendRedirect("/jsp/home.jsp");
    }
    %></td>
  • 48. 4. File Repository
    4.2 Code
    • Deploy
    <application>lqu-filerepo</application>
    <version>1</version>
  • 49. 4. File Repository
    4.3 Technology
    • Apache Commons FileUpload[28]
    • 50. Apache HttpClient
    • 51. JSP(CSS, JavaScript)
    • 52. The Blobstore Java API[29]
    • 53. The Users Java API[30]
  • 5. Summary
    • Google App Engine
    • 54. Cloud Computing
  • 6. Reference
    Cloud Computing and Grid Computing 360-Degree Compared, Ian Foster, Yong Zhao, IoanRaicu, Shiyong Lu.
    Cloud computing service models, Part 2: Platform as a Service : Learn the benefits of PaaS.
    Cloud and industry, Part 1: PaaS best practices and patterns : Learn about the requirements and functions of three models to deliver industry solutions, IaaS, PaaS, SaaS, and how you can use best practices and patterns with the PaaS framework in particular to deploy and manage cloud computing solutions. 
    Connecting to the cloud, Part 1: Leverage the cloud in applications: Explore cloud computing and the offerings from the major cloud platform vendors: Amazon, Google, Microsoft®, and SalesForce.com.
    Google App Engine Home Page: Learn more about Google App Engine™ platform.
    Install Google Plugin for Eclipse: Get started with the plug-ins.
  • 55. 6. Reference
    Will it play in App Engine for Java?: Lists the level of compatibility of various Java technologies and App Engine (Java).
    What's BigTable?: Read the Google research publication to find out.
    Cloud computing fundamentals: Aid you in assimilating the reality of the revolution, so you can use it for your own profit and well being.
    Stanford University Wiki – Google App Engine Project.
    Slatkin, Brett. Building Scalable Web Applications with Google App Engine. Google IO 2008.
    Ashcraft, Ken. Best Practices - Building a Production Quality Application on Google App Engine. Google IO 2008.
    Dan Sanderson, Programming Google App Engine. O’Reilly, GoogleTM Press.
    Getting Started: Java. Helpful sample from Google App Engine site.
  • 56. 6. Reference
    Presentation by Guido van Rossum. Google App Engine: Run your web applications on Google's infrastructure. Stanford EE Computer Systems Colloquium. 5 Nov 2008.
    Why App Engine. Brief explain why developers choose Google App Engine.
    Google App Engine Samples. A list of samples and its source code.
    Choosing a Datastore(Java). Comparing the Data Storage Options, choose the right one.
    Datastore Overview(Python).
    Barrett, Ryan. Under the Covers of the Google App Engine Datastore. Google IO 2008.
    Introduction to Indexes.
    Java Index Configuration Using YAML.
  • 57. 6. Reference
    Queries in JDO.
    GQL Reference.
    The Google File System.  Get a in-depth understanding of GFS.
    Google App Engine Articles. Articles related to Google App Engine in many aspects by category.
    Google App Engine Python SDK Release Notes.
    Apache Commons FileUpload.
    The Blobstore Java API.
    The Users Java API.
    Google File Service on Google App Engine for Java. Main Reference for code.
  • 58. End
    Cloud Computing Topic
    LinaQu
    439-Distributed System by Prof. Thiruvathukal
    2011.05.01