SlideShare a Scribd company logo
1 of 24
Download to read offline
I’ve (probably) been using

Google App Engine
     for a week longer than you have

Simon Willison - http://simonwillison.net/
BarCamp London 4
31st May 2008
Except you have to re-write
     your whole application
If you totally rethink the
 way you use a database
What it can do
• Serve static files
• Serve dynamic requests
• Store data
• Call web services (sort of)
• Authenticate against Google’s user database
• Send e-mail, process images, use memcache
The dev environment
                    is really, really nice


•   Download the (open source) SDK

    •   a full simulation of the App Engine environment

•   dev_appserver.py myapp     for a local webserver

•   appcfg.py update myapp     to deploy to the cloud
Options
• You have to use Python
• You can choose how you use it:
 • CGI-style scripts
 • WSGI applications
 • Google’s webapp framework
 • Django (0.96 provided, or install your own)
Hello World
# helloworld.py
print quot;Content-Type: text/htmlquot;
print
print quot;Hello, world!quot;

# app.yaml
application: simonwillison-helloworld
version: 1
runtime: python
api_version: 1

handlers:
- url: /.*
  script: helloworld.py
With webapp and WSGI
import wsgiref.handlers
from google.appengine.ext import webapp

class MainPage(webapp.RequestHandler):
  def get(self):
    self.response.headers['Content-Type'] = 'text/html'
    self.response.out.write('Hello, webapp World!')

def main():
  application = webapp.WSGIApplication(
    [('/', MainPage)], debug=True)
  wsgiref.handlers.CGIHandler().run(application)

if __name__ == quot;__main__quot;:
  main()
With Django
from django.conf.urls.defaults import *
from django.http import HttpResponse

def hello(request):
    return HttpResponse(quot;Hello, World!quot;)

urlpatterns = patterns('',
    ('^$', hello),
)

(And django_dispatch.py for boilerplate)
•   Don't use CGI: it requires reloading for every hit

•   Why use Django over webapp?

    •   Django has easy cookies and custom 500 errors

    •   Django is less verbose

    •   Django middleware is really handy

•   You can use other WSGI frameworks if you like
Static files
# in app.yaml

handlers:
- url: /css
  static_dir: css
- url: /img
  static_dir: img
- url: /favicon.ico
  static_files: img/favicon.ico
  upload: img/favicon.ico
  mime_type: image/x-icon
The Datastore API
“Bigtable is a distributed storage system for
 managing structured data that is designed
  to scale to a very large size: petabytes of
    data across thousands of commodity
servers. Many projects at Google store data
 in Bigtable, including web indexing, Google
         Earth, and Google Finance.”
The App Engine datastore
• Apparently based on BigTable
• Absolutely not a relational database
 • No joins (they do have “reference fields”)
 • No aggregate queries - not even count()!
 • Hierarchy affects sharding and transactions
• All queries must run against an existing index
Models and entities
• Data is stored as entities
 • Entities have properties - key/value pairs
 • An entity has a unique key
• Entities live in a hierarchy, and siblings exist in
  the same entity group - these are actually really
  important for transactions and performance
• A model is kind of like a class; it lets you define
  a type of entity
AppEngine Models
   from google.appengine.ext import db

   class Account(db.Model):
     slug = db.StringProperty(required=True)
     owner = db.UserProperty()
     onlyme = db.BooleanProperty()
     referrers = db.StringListProperty()

(There is a ReferenceProperty, but I haven’t used it yet)
Inserting data
account = Account(
    key_name = slug,
    slug = slug,
    referrers = ['...', '...'],
    onlyme = False,
    owner = users.get_current_user()
)
db.put(account) # Or account.put()

Browser.get_or_insert(key_name,
    parent = account,
    slug = browser_slug
)
Running queries

Browser.all().ancestor(account)

Account.gql(quot;WHERE slug = :1quot;, slug))

Story.all().filter(
  'title =', 'Foo'
).order('-date')
BUT...
•   All queries must run against an existing index

•   Filtering or sorting on a property requires that
    the property exists

•   Inequality filters are allowed on one property only

•   Properties in inequality filters must be sorted
    before other sort orders

•   ... and various other rules

•   Thankfully the dev server creates most indexes
    for you automatically based on usage
How indexes are used
1. The datastore identifies the index that
  corresponds with the query’s kind, filter
  properties, filter operators, and sort orders.
2. The datastore starts scanning the index at the
  first entity that meets all of the filter conditions
  using the query’s filter values.
3. The datastore continues to scan the index,
  returning each entity, until it finds the next entity
  that does not meet the filter conditions, or until
  it reaches the end of the index.
Further limitations
• If you create a new index and push it live,
  you have to wait for it to rebuilt
  • This can take hours, and apparently can go
    wrong
• You can’t safely grab more than about 500
  records at once - App Engine times out
• You can’t delete in bulk
Other random notes
•   You have to use the URL Fetch API to do
    HTTP requests (e.g. for web services) - and it
    times out agressively at about 5 seconds
•   The Google accounts Users API is ridiculously
    easy to use, but...
    •   no permanent unique identifier; if the user
        changes their e-mail address you’re screwed
•   The new image and memcache APIs are neat
Final thoughts
• It’s really nice not to have to worry about hosting
• But... the lack of aggregate queries and ad-hoc
  queries really hurts
• Perfect for small projects you don’t want to
  worry about and big things which you’re sure will
  have to scale
• Pricing is comparable to S3 - i.e. stupidly cheap
Pricing

• $0.10 - $0.12 per CPU core-hour
• $0.15 - $0.18 per GB-month of storage
• $0.11 - $0.13 per GB outgoing bandwidth
• $0.09 - $0.11 per GB incoming bandwidth
Thank you!

More Related Content

What's hot

Platform as a service google app engine
Platform as a service   google app enginePlatform as a service   google app engine
Platform as a service google app engineDeepu S Nath
 
Google App Engine: An Introduction
Google App Engine: An IntroductionGoogle App Engine: An Introduction
Google App Engine: An IntroductionAbu Ashraf Masnun
 
Developing Java Web Applications In Google App Engine
Developing Java Web Applications In Google App EngineDeveloping Java Web Applications In Google App Engine
Developing Java Web Applications In Google App EngineTahir Akram
 
Gentle App Engine Intro
Gentle App Engine IntroGentle App Engine Intro
Gentle App Engine Introrobinb123
 
What is Google App Engine?
What is Google App Engine?What is Google App Engine?
What is Google App Engine?weschwee
 
Google Application Engine
Google Application EngineGoogle Application Engine
Google Application Engineguestd77e8ae
 
Introduction to Google App Engine
Introduction to Google App EngineIntroduction to Google App Engine
Introduction to Google App Enginerajdeep
 
Google app engine - Overview
Google app engine - OverviewGoogle app engine - Overview
Google app engine - OverviewNathan Quach
 
Google app engine
Google app engineGoogle app engine
Google app engineSuraj Mehta
 
Google app engine
Google app engineGoogle app engine
Google app engineRenjith318
 
Introduction to Google App Engine
Introduction to Google App EngineIntroduction to Google App Engine
Introduction to Google App EngineColin Su
 
Google App Engine Introduction
Google App Engine IntroductionGoogle App Engine Introduction
Google App Engine IntroductionSimon Su
 
Google App Engine - Overview #3
Google App Engine - Overview #3Google App Engine - Overview #3
Google App Engine - Overview #3Kay Kim
 
Google App Engine (Introduction)
Google App Engine (Introduction)Google App Engine (Introduction)
Google App Engine (Introduction)Praveen Hanchinal
 
Powering your Apps via Google Cloud Platform
Powering your Apps via Google Cloud PlatformPowering your Apps via Google Cloud Platform
Powering your Apps via Google Cloud PlatformRomin Irani
 
Google Cloud Platform Updates
Google Cloud Platform UpdatesGoogle Cloud Platform Updates
Google Cloud Platform UpdatesRomin Irani
 

What's hot (20)

Google App Engine
Google App EngineGoogle App Engine
Google App Engine
 
Google App Engine
Google App EngineGoogle App Engine
Google App Engine
 
Platform as a service google app engine
Platform as a service   google app enginePlatform as a service   google app engine
Platform as a service google app engine
 
Google App Engine: An Introduction
Google App Engine: An IntroductionGoogle App Engine: An Introduction
Google App Engine: An Introduction
 
Developing Java Web Applications In Google App Engine
Developing Java Web Applications In Google App EngineDeveloping Java Web Applications In Google App Engine
Developing Java Web Applications In Google App Engine
 
Gentle App Engine Intro
Gentle App Engine IntroGentle App Engine Intro
Gentle App Engine Intro
 
What is Google App Engine?
What is Google App Engine?What is Google App Engine?
What is Google App Engine?
 
Google Application Engine
Google Application EngineGoogle Application Engine
Google Application Engine
 
Introduction to Google App Engine
Introduction to Google App EngineIntroduction to Google App Engine
Introduction to Google App Engine
 
Google app engine - Overview
Google app engine - OverviewGoogle app engine - Overview
Google app engine - Overview
 
Google App engine
Google App engineGoogle App engine
Google App engine
 
Google app engine
Google app engineGoogle app engine
Google app engine
 
App Engine
App EngineApp Engine
App Engine
 
Google app engine
Google app engineGoogle app engine
Google app engine
 
Introduction to Google App Engine
Introduction to Google App EngineIntroduction to Google App Engine
Introduction to Google App Engine
 
Google App Engine Introduction
Google App Engine IntroductionGoogle App Engine Introduction
Google App Engine Introduction
 
Google App Engine - Overview #3
Google App Engine - Overview #3Google App Engine - Overview #3
Google App Engine - Overview #3
 
Google App Engine (Introduction)
Google App Engine (Introduction)Google App Engine (Introduction)
Google App Engine (Introduction)
 
Powering your Apps via Google Cloud Platform
Powering your Apps via Google Cloud PlatformPowering your Apps via Google Cloud Platform
Powering your Apps via Google Cloud Platform
 
Google Cloud Platform Updates
Google Cloud Platform UpdatesGoogle Cloud Platform Updates
Google Cloud Platform Updates
 

Viewers also liked (16)

Big table
Big tableBig table
Big table
 
ROCKING YOUR SEAT AT THE BIG TABLE - ROB BAILEY
ROCKING YOUR SEAT AT THE BIG TABLE - ROB BAILEYROCKING YOUR SEAT AT THE BIG TABLE - ROB BAILEY
ROCKING YOUR SEAT AT THE BIG TABLE - ROB BAILEY
 
Google's BigTable
Google's BigTableGoogle's BigTable
Google's BigTable
 
Bigtable
BigtableBigtable
Bigtable
 
google Bigtable
google Bigtablegoogle Bigtable
google Bigtable
 
Google Bigtable Paper Presentation
Google Bigtable Paper PresentationGoogle Bigtable Paper Presentation
Google Bigtable Paper Presentation
 
Big table
Big tableBig table
Big table
 
GOOGLE BIGTABLE
GOOGLE BIGTABLEGOOGLE BIGTABLE
GOOGLE BIGTABLE
 
Bigtable
BigtableBigtable
Bigtable
 
Big table
Big tableBig table
Big table
 
Bigtable
BigtableBigtable
Bigtable
 
Bigtable: A Distributed Storage System for Structured Data
Bigtable: A Distributed Storage System for Structured DataBigtable: A Distributed Storage System for Structured Data
Bigtable: A Distributed Storage System for Structured Data
 
Google Big Table
Google Big TableGoogle Big Table
Google Big Table
 
Google Bigtable paper presentation
Google Bigtable paper presentationGoogle Bigtable paper presentation
Google Bigtable paper presentation
 
Google BigTable
Google BigTableGoogle BigTable
Google BigTable
 
The Google Bigtable
The Google BigtableThe Google Bigtable
The Google Bigtable
 

Similar to I've (probably) been using Google App Engine for a week longer than you have

App Engine On Air: Munich
App Engine On Air: MunichApp Engine On Air: Munich
App Engine On Air: Munichdion
 
Castles in the Cloud: Developing with Google App Engine
Castles in the Cloud: Developing with Google App EngineCastles in the Cloud: Developing with Google App Engine
Castles in the Cloud: Developing with Google App Enginecatherinewall
 
Intro To Django
Intro To DjangoIntro To Django
Intro To DjangoUdi Bauman
 
Google App Engine for Java
Google App Engine for JavaGoogle App Engine for Java
Google App Engine for JavaLars Vogel
 
Google App Engine for Java
Google App Engine for JavaGoogle App Engine for Java
Google App Engine for JavaLars Vogel
 
iPhone Development For Experienced Web Developers
iPhone Development For Experienced Web DevelopersiPhone Development For Experienced Web Developers
iPhone Development For Experienced Web Developerslisab517
 
An Introduction to Web Components
An Introduction to Web ComponentsAn Introduction to Web Components
An Introduction to Web ComponentsRed Pill Now
 
Introduction to App Engine Development
Introduction to App Engine DevelopmentIntroduction to App Engine Development
Introduction to App Engine DevelopmentRon Reiter
 
GDD Japan 2009 - Designing OpenSocial Apps For Speed and Scale
GDD Japan 2009 - Designing OpenSocial Apps For Speed and ScaleGDD Japan 2009 - Designing OpenSocial Apps For Speed and Scale
GDD Japan 2009 - Designing OpenSocial Apps For Speed and ScalePatrick Chanezon
 
"Progressive Web Apps" by Riza Fahmi (Hacktiv8)
"Progressive Web Apps" by Riza Fahmi	(Hacktiv8)"Progressive Web Apps" by Riza Fahmi	(Hacktiv8)
"Progressive Web Apps" by Riza Fahmi (Hacktiv8)Tech in Asia ID
 
Progressive Web Apps. What, why and how
Progressive Web Apps. What, why and howProgressive Web Apps. What, why and how
Progressive Web Apps. What, why and howRiza Fahmi
 
Cannibalising The Google App Engine
Cannibalising The  Google  App  EngineCannibalising The  Google  App  Engine
Cannibalising The Google App Enginecatherinewall
 
Drupal performance and scalability
Drupal performance and scalabilityDrupal performance and scalability
Drupal performance and scalabilityTwinbit
 
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
AD113  Speed Up Your Applications w/ Nginx and PageSpeedAD113  Speed Up Your Applications w/ Nginx and PageSpeed
AD113 Speed Up Your Applications w/ Nginx and PageSpeededm00se
 
Google Cloud Developer Challenge - GDG Belgaum
Google Cloud Developer Challenge - GDG BelgaumGoogle Cloud Developer Challenge - GDG Belgaum
Google Cloud Developer Challenge - GDG Belgaumsandeephegde
 
How and why we evolved a legacy Java web application to Scala... and we are s...
How and why we evolved a legacy Java web application to Scala... and we are s...How and why we evolved a legacy Java web application to Scala... and we are s...
How and why we evolved a legacy Java web application to Scala... and we are s...Katia Aresti
 

Similar to I've (probably) been using Google App Engine for a week longer than you have (20)

App Engine On Air: Munich
App Engine On Air: MunichApp Engine On Air: Munich
App Engine On Air: Munich
 
Castles in the Cloud: Developing with Google App Engine
Castles in the Cloud: Developing with Google App EngineCastles in the Cloud: Developing with Google App Engine
Castles in the Cloud: Developing with Google App Engine
 
Intro To Django
Intro To DjangoIntro To Django
Intro To Django
 
Google App Engine for Java
Google App Engine for JavaGoogle App Engine for Java
Google App Engine for Java
 
Google App Engine for Java
Google App Engine for JavaGoogle App Engine for Java
Google App Engine for Java
 
Angularjs
AngularjsAngularjs
Angularjs
 
Gae
GaeGae
Gae
 
iPhone Development For Experienced Web Developers
iPhone Development For Experienced Web DevelopersiPhone Development For Experienced Web Developers
iPhone Development For Experienced Web Developers
 
An Introduction to Web Components
An Introduction to Web ComponentsAn Introduction to Web Components
An Introduction to Web Components
 
Introduction to App Engine Development
Introduction to App Engine DevelopmentIntroduction to App Engine Development
Introduction to App Engine Development
 
Google App Engine tutorial
Google App Engine tutorialGoogle App Engine tutorial
Google App Engine tutorial
 
GDD Japan 2009 - Designing OpenSocial Apps For Speed and Scale
GDD Japan 2009 - Designing OpenSocial Apps For Speed and ScaleGDD Japan 2009 - Designing OpenSocial Apps For Speed and Scale
GDD Japan 2009 - Designing OpenSocial Apps For Speed and Scale
 
"Progressive Web Apps" by Riza Fahmi (Hacktiv8)
"Progressive Web Apps" by Riza Fahmi	(Hacktiv8)"Progressive Web Apps" by Riza Fahmi	(Hacktiv8)
"Progressive Web Apps" by Riza Fahmi (Hacktiv8)
 
Progressive Web Apps. What, why and how
Progressive Web Apps. What, why and howProgressive Web Apps. What, why and how
Progressive Web Apps. What, why and how
 
Cannibalising The Google App Engine
Cannibalising The  Google  App  EngineCannibalising The  Google  App  Engine
Cannibalising The Google App Engine
 
Drupal performance and scalability
Drupal performance and scalabilityDrupal performance and scalability
Drupal performance and scalability
 
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
AD113  Speed Up Your Applications w/ Nginx and PageSpeedAD113  Speed Up Your Applications w/ Nginx and PageSpeed
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
 
Presentation Tier optimizations
Presentation Tier optimizationsPresentation Tier optimizations
Presentation Tier optimizations
 
Google Cloud Developer Challenge - GDG Belgaum
Google Cloud Developer Challenge - GDG BelgaumGoogle Cloud Developer Challenge - GDG Belgaum
Google Cloud Developer Challenge - GDG Belgaum
 
How and why we evolved a legacy Java web application to Scala... and we are s...
How and why we evolved a legacy Java web application to Scala... and we are s...How and why we evolved a legacy Java web application to Scala... and we are s...
How and why we evolved a legacy Java web application to Scala... and we are s...
 

More from Simon Willison

Cheap tricks for startups
Cheap tricks for startupsCheap tricks for startups
Cheap tricks for startupsSimon Willison
 
The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)Simon Willison
 
How we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graphHow we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graphSimon Willison
 
Web Services for Fun and Profit
Web Services for Fun and ProfitWeb Services for Fun and Profit
Web Services for Fun and ProfitSimon Willison
 
Tricks & challenges developing a large Django application
Tricks & challenges developing a large Django applicationTricks & challenges developing a large Django application
Tricks & challenges developing a large Django applicationSimon Willison
 
Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & FabricAdvanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & FabricSimon Willison
 
How Lanyrd uses Twitter
How Lanyrd uses TwitterHow Lanyrd uses Twitter
How Lanyrd uses TwitterSimon Willison
 
Building Things Fast - and getting approval
Building Things Fast - and getting approvalBuilding Things Fast - and getting approval
Building Things Fast - and getting approvalSimon Willison
 
Rediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The LibrariesRediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The LibrariesSimon Willison
 
Building crowdsourcing applications
Building crowdsourcing applicationsBuilding crowdsourcing applications
Building crowdsourcing applicationsSimon Willison
 
Evented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunniesEvented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunniesSimon Willison
 
Cowboy development with Django
Cowboy development with DjangoCowboy development with Django
Cowboy development with DjangoSimon Willison
 
Crowdsourcing with Django
Crowdsourcing with DjangoCrowdsourcing with Django
Crowdsourcing with DjangoSimon Willison
 
Class-based views with Django
Class-based views with DjangoClass-based views with Django
Class-based views with DjangoSimon Willison
 
Web App Security Horror Stories
Web App Security Horror StoriesWeb App Security Horror Stories
Web App Security Horror StoriesSimon Willison
 
Web Security Horror Stories
Web Security Horror StoriesWeb Security Horror Stories
Web Security Horror StoriesSimon Willison
 

More from Simon Willison (20)

How Lanyrd does Geo
How Lanyrd does GeoHow Lanyrd does Geo
How Lanyrd does Geo
 
Cheap tricks for startups
Cheap tricks for startupsCheap tricks for startups
Cheap tricks for startups
 
The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)
 
Building Lanyrd
Building LanyrdBuilding Lanyrd
Building Lanyrd
 
How we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graphHow we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graph
 
Web Services for Fun and Profit
Web Services for Fun and ProfitWeb Services for Fun and Profit
Web Services for Fun and Profit
 
Tricks & challenges developing a large Django application
Tricks & challenges developing a large Django applicationTricks & challenges developing a large Django application
Tricks & challenges developing a large Django application
 
Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & FabricAdvanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
 
How Lanyrd uses Twitter
How Lanyrd uses TwitterHow Lanyrd uses Twitter
How Lanyrd uses Twitter
 
ScaleFail
ScaleFailScaleFail
ScaleFail
 
Building Things Fast - and getting approval
Building Things Fast - and getting approvalBuilding Things Fast - and getting approval
Building Things Fast - and getting approval
 
Rediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The LibrariesRediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The Libraries
 
Building crowdsourcing applications
Building crowdsourcing applicationsBuilding crowdsourcing applications
Building crowdsourcing applications
 
Evented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunniesEvented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunnies
 
Cowboy development with Django
Cowboy development with DjangoCowboy development with Django
Cowboy development with Django
 
Crowdsourcing with Django
Crowdsourcing with DjangoCrowdsourcing with Django
Crowdsourcing with Django
 
Django Heresies
Django HeresiesDjango Heresies
Django Heresies
 
Class-based views with Django
Class-based views with DjangoClass-based views with Django
Class-based views with Django
 
Web App Security Horror Stories
Web App Security Horror StoriesWeb App Security Horror Stories
Web App Security Horror Stories
 
Web Security Horror Stories
Web Security Horror StoriesWeb Security Horror Stories
Web Security Horror Stories
 

Recently uploaded

Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfQ4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfTejal81
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc
 
Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updateadam112203
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptxHansamali Gamage
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 

Recently uploaded (20)

Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfQ4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
 
Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 update
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 

I've (probably) been using Google App Engine for a week longer than you have

  • 1. I’ve (probably) been using Google App Engine for a week longer than you have Simon Willison - http://simonwillison.net/ BarCamp London 4 31st May 2008
  • 2. Except you have to re-write your whole application If you totally rethink the way you use a database
  • 3. What it can do • Serve static files • Serve dynamic requests • Store data • Call web services (sort of) • Authenticate against Google’s user database • Send e-mail, process images, use memcache
  • 4. The dev environment is really, really nice • Download the (open source) SDK • a full simulation of the App Engine environment • dev_appserver.py myapp for a local webserver • appcfg.py update myapp to deploy to the cloud
  • 5. Options • You have to use Python • You can choose how you use it: • CGI-style scripts • WSGI applications • Google’s webapp framework • Django (0.96 provided, or install your own)
  • 6. Hello World # helloworld.py print quot;Content-Type: text/htmlquot; print print quot;Hello, world!quot; # app.yaml application: simonwillison-helloworld version: 1 runtime: python api_version: 1 handlers: - url: /.* script: helloworld.py
  • 7. With webapp and WSGI import wsgiref.handlers from google.appengine.ext import webapp class MainPage(webapp.RequestHandler):   def get(self):     self.response.headers['Content-Type'] = 'text/html'     self.response.out.write('Hello, webapp World!') def main():   application = webapp.WSGIApplication( [('/', MainPage)], debug=True)   wsgiref.handlers.CGIHandler().run(application) if __name__ == quot;__main__quot;:   main()
  • 8. With Django from django.conf.urls.defaults import * from django.http import HttpResponse def hello(request): return HttpResponse(quot;Hello, World!quot;) urlpatterns = patterns('', ('^$', hello), ) (And django_dispatch.py for boilerplate)
  • 9. Don't use CGI: it requires reloading for every hit • Why use Django over webapp? • Django has easy cookies and custom 500 errors • Django is less verbose • Django middleware is really handy • You can use other WSGI frameworks if you like
  • 10. Static files # in app.yaml handlers: - url: /css static_dir: css - url: /img static_dir: img - url: /favicon.ico static_files: img/favicon.ico upload: img/favicon.ico mime_type: image/x-icon
  • 12. “Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance.”
  • 13. The App Engine datastore • Apparently based on BigTable • Absolutely not a relational database • No joins (they do have “reference fields”) • No aggregate queries - not even count()! • Hierarchy affects sharding and transactions • All queries must run against an existing index
  • 14. Models and entities • Data is stored as entities • Entities have properties - key/value pairs • An entity has a unique key • Entities live in a hierarchy, and siblings exist in the same entity group - these are actually really important for transactions and performance • A model is kind of like a class; it lets you define a type of entity
  • 15. AppEngine Models from google.appengine.ext import db class Account(db.Model): slug = db.StringProperty(required=True) owner = db.UserProperty() onlyme = db.BooleanProperty() referrers = db.StringListProperty() (There is a ReferenceProperty, but I haven’t used it yet)
  • 16. Inserting data account = Account( key_name = slug, slug = slug, referrers = ['...', '...'], onlyme = False, owner = users.get_current_user() ) db.put(account) # Or account.put() Browser.get_or_insert(key_name, parent = account, slug = browser_slug )
  • 17. Running queries Browser.all().ancestor(account) Account.gql(quot;WHERE slug = :1quot;, slug)) Story.all().filter( 'title =', 'Foo' ).order('-date')
  • 18. BUT... • All queries must run against an existing index • Filtering or sorting on a property requires that the property exists • Inequality filters are allowed on one property only • Properties in inequality filters must be sorted before other sort orders • ... and various other rules • Thankfully the dev server creates most indexes for you automatically based on usage
  • 19. How indexes are used 1. The datastore identifies the index that corresponds with the query’s kind, filter properties, filter operators, and sort orders. 2. The datastore starts scanning the index at the first entity that meets all of the filter conditions using the query’s filter values. 3. The datastore continues to scan the index, returning each entity, until it finds the next entity that does not meet the filter conditions, or until it reaches the end of the index.
  • 20. Further limitations • If you create a new index and push it live, you have to wait for it to rebuilt • This can take hours, and apparently can go wrong • You can’t safely grab more than about 500 records at once - App Engine times out • You can’t delete in bulk
  • 21. Other random notes • You have to use the URL Fetch API to do HTTP requests (e.g. for web services) - and it times out agressively at about 5 seconds • The Google accounts Users API is ridiculously easy to use, but... • no permanent unique identifier; if the user changes their e-mail address you’re screwed • The new image and memcache APIs are neat
  • 22. Final thoughts • It’s really nice not to have to worry about hosting • But... the lack of aggregate queries and ad-hoc queries really hurts • Perfect for small projects you don’t want to worry about and big things which you’re sure will have to scale • Pricing is comparable to S3 - i.e. stupidly cheap
  • 23. Pricing • $0.10 - $0.12 per CPU core-hour • $0.15 - $0.18 per GB-month of storage • $0.11 - $0.13 per GB outgoing bandwidth • $0.09 - $0.11 per GB incoming bandwidth