Frontend London, one day edition, namely EpicFEL. 30 minutes about Performance and Optimization with Google AppEngine and AngularJS, with 2 case studies developed for Google Creative Lab London and Red Bull Music Academy / Google+ London

1. Performance + Optimization
Thomas Alisi, Solution Architect
@grudelsud @stinkdigital
EpicFEL Oct2014, Sadler’s Wells, London
photo courtesy of https://www.flickr.com/photos/8064990@N08/

2. Stinkdigital is an interactive production company,
working with clients and advertising agencies
worldwide.
!
Our services include creative concepting, design
and high-end execution. We create everything from
live-action films and websites, through to mobile
apps and installations.
See our 2013 showreel

3. Project Google Title
+ The Barbican
BDreavnadrt
Body A new platform creative text coders goes here.
from Google and the Barbican that aims to reward and inspire
everywhere.
VIEW SITE

4. Revolutions in Sound
Red Bull
We teamed up with Google+ & Red Bull Music Academy to create a living
archive of UK club culture.
VIEW SITE
ABOUT THE TECHNOLOGY

5. Does this number look familiar to you? 86,400

6. Does this number look familiar to you? 86,400
= 60’’ x 60’ x 24h [number of seconds in 1 day]

7. What about this one? 31,536,000

8. What about this one? 31,536,000
= 86,400’’ x 365d [number of seconds in 1 year]

9. OK, now try and guess the last one 252,000,000

10. OK, now try and guess the last one 252,000,000
= 31.5M’’ x 8y [number of seconds in 8 years]

11. but also… 252,000,000
= 36M x 7’’ [7 seconds saved for each of the 36M visits
we had during the first 6 months on DevArt]

12. How did we do it?
1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

13. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

14. - Grunt, Gulp, Browserify, Webpack… AAARGH!
! - we like vanilla JS and have been fan of Gulp
(at least over the past 10 minutes…)
! - we (I) tend to use a super simple Gulp-Browserify-
AngularJS boilerplate
https://github.com/grudelsud/angularjs-gulp-browserify-boilerplate
! - but there are other good examples too
e.g. https://github.com/unit9/coffee-bone
!
!
divide et impera

15. - GAE is not opinionated (which is good)
! - don’t use Django on GAE unless you really want to
(e.g. use legacy modules or deploy something really
quick)
! - Flask is OK and does not have preferences for a specific
ORM (which is great)
! - webapp2 is really super simple (a bit too simple…)

16. bespoke REST micro-framework
class Jsonifiable(ndb.Model):
def from_dict(cls, dict):
pass
!
def to_dict(self, async=False):
pass
!
def resolve_future_blobs(cls, async_blobs):
pass
!!
class JsonRestHandler(webapp2.RequestHandler):
JSON_MIMETYPE = "application/json"
!
def write(self, data):
self.response.out.write(data)
!
extend ndb.Model
extend webapp2.RequestHandler
basic permission check (skipped here)
def send_success(self, obj=None, cache_expiry=None):
asynchronous conversion
self.response.headers["Content-Type"] = self.JSON_MIMETYPE
!
self.write(json.dumps(obj, cls=JsonifiableEncoder))
bespoke encoder

17. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

18. this always seems a good way to start

20. abstraction
class ClubNight(BaseModel):
name = ndb.StringProperty()
content = ContentProperty(ndb.TextProperty)
blob_key_logo = ContentProperty(ndb.BlobKeyProperty)
genre = ndb.KeyProperty(kind=’Genre’)
website = ContentProperty(ndb.StringProperty)
address = ContentProperty(ndb.TextProperty)
location = ContentProperty(ndb.GeoPtProperty)
!
class Connection(ndb.Model):
from_key = ndb.KeyProperty()
to_key = ndb.KeyProperty()

21. reality
class ClubNight(BaseModel):
name = ndb.StringProperty()
content = ContentProperty(ndb.TextProperty)
blob_key_logo = ContentProperty(ndb.BlobKeyProperty)
genre = ndb.KeyProperty(kind=’Genre’)
website = ContentProperty(ndb.StringProperty)
address = ContentProperty(ndb.TextProperty)
location = ContentProperty(ndb.GeoPtProperty)
!
class Connection(ndb.Model):
from_key = ndb.KeyProperty()
to_key = ndb.KeyProperty()
to_name = ndb.StringProperty()
to_kind = ndb.StringProperty()
to_slug = ndb.StringProperty()
to_genre_colour = ndb.StringProperty()
to_image = ndb.BlobKeyProperty()
to_popularity = ndb.IntegerProperty()
is_published = ndb.BooleanProperty()
is_public = ndb.BooleanProperty(default=False)
is_featured = ndb.BooleanProperty(default=False)

22. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

24. - Frontend caching: most of the API requests are fired when
initialising the view
! - Only request data that will be used/visible
! - Remove the tap delay on mobile
! - Request images in the size they will be actually displayed
! - Rendering optimizations, be aware of DOM structure and
calls to RenderObject https://speakerdeck.com/
jaffathecake/rendering-without-lumps
! - Create a custom font with all icons used (glyphs) https://
icomoon.io/

25. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

26. The get_serving_url() method allows you to generate a stable,
dedicated URL for serving web-suitable image thumbnails.
!
You simply store a single copy of your original image in Blobstore,
and then request a high-performance per-image URL.
[https://cloud.google.com/appengine/docs/python/images/]
!
ex.
!
// Resize the image to 32 pixels (aspect-ratio preserved)
http://your_app_id.appspot.com/randomStringImageId=s32
!
// Crop the image to 32 pixels
http://your_app_id.appspot.com/randomStringImageId=s32-c

27. =s500
=s100

28. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

29. GAE docs are rubbish! :) i.e. read it, then forget it:
https://cloud.google.com/appengine/articles/load_test
!
3rd party services are OK, but run your own if you can
!
create a meaningful simulation of users’ behaviour
!
hit it as hard as you can, but don’t forget your wallet!

30. locust
class ApiNav(TaskSet):
@task(1)
def api_global(self):
self.client.get('/api/global?locale=%s' % langs[random.randint(0, len(langs)-1)], **kwargs)
!
@task(1)
def api_user(self):
self.client.get('/api/user', **kwargs)
!
@task(4)
def api_gallery(self):
self.client.get('/api/gallery?i=0&l=15', **kwargs)
!
@task(8)
def api_search(self):
self.client.get('/api/gallery?i=0&l=15&q=%s' % terms[random.randint(0, len(terms)-1)], **kwargs)
!
@task(6)
def api_feeling_lucky(self):
self.client.get('/api/page/feeling_lucky', **kwargs)
!
@task(2)
def api_big_gallery(self):
self.client.get('/api/gallery?i=0&l=30', **kwargs)
!
@task(2)
def api_featured(self):
self.client.get('/api/gallery?i=0&l=15&t=featured', **kwargs)
!
class MyLocust(HttpLocust):
host = 'https://sd-goog-devart.appspot.com'
task_set = ApiNav
min_wait = 5000
max_wait = 15000
init data
gallery
random search
random project page
categorized views

31. - approximately 5000 concurrent user hitting the backend
API with a "casual navigation" simulation from different
location (London, New York, AWS data centre in Ireland)
! - 85 running instances (class F2) at peak
! - no errors reported other than random https sockets
timeout
! - average response times - < 2s for gallery content navigation - < 1s for singe project page navitation - < 3s for static contend (loaded just once)

32. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

38. 1. Google App Engine (GAE) and AngularJS
2. data structures
3. views optimisations
4. image management
5. load testing
6. GAE benchmarking tool and task inspector
7. GAE asynchronous API

39. get_serving_url_async
@classmethod
def fix_dict(cls, dict, async=False):
!
async_blobs = []
!
def _fix_dict(k, v):
!
# blob
if isinstance(v, blobstore.BlobKey):
try:
# create futures and put them apart, we'll resolve these later
output = {'key': str(v), 'url': '', 'rpc': images.get_serving_url_async(v, secure_url=True)}
async_blobs.append(output)
return output
except BaseException as e:
# logging.warn('error while fetching serving url for [%s] maybe using corrupted image?' % (v,))
return None
!
for k, v in dict.iteritems():
dict[k] = _fix_dict(k, v)
!
if async is True:
return dict, async_blobs
else:
cls.resolve_future_blobs(async_blobs)
return dict
!
@classmethod
def resolve_future_blobs(cls, async_blobs):
# resolve futures
for blob in async_blobs:
try:
blob['url'] = blob['rpc'].get_result()
except BaseException:
blob['url'] = ''
del(blob['rpc'])
1. get async url
2. resolve future blobs
3. get real url

40. Thanks!
uh, just two notes:
!
1. we are hiring! send us your CV - careers@stinkdigital.com
2. we organise a Meetup with UNIT9 and B-Reel, get in touch!
tomalisi@stinkdigital.com