This is a new version of a talk I presented at a Varnish Users Group meeting in Paris in 2012. We've added a few useful tools and improved our Puppet module since then.
Presented at the Devops Norway meetup in Oslo on 17th of September 2014.
A bit of history, frustration-driven development, and why and how we started looking into Puppet at Opera Software. What we're doing, successes, pain points and what we're going to do with Puppet and Config Management next.
This is a new version of a talk I presented at a Varnish Users Group meeting in Paris in 2012. We've added a few useful tools and improved our Puppet module since then.
Presented at the Devops Norway meetup in Oslo on 17th of September 2014.
A bit of history, frustration-driven development, and why and how we started looking into Puppet at Opera Software. What we're doing, successes, pain points and what we're going to do with Puppet and Config Management next.
IT Infrastructure Through The Public Network Challenges And SolutionsMartin Jackson
Identifying the challenges that companies face when they wish to adopt Infrastructure as a Service like those from Amazon and Rackspace and possible solutions to those problems. This presentation seeks to provide insight and possible solutions, covering the areas of security, availability, cloud standards, interoperability, vendor lock in and performance management.
Puppet is a configuration management tool which allows easy deployment and configuration ranging from 1 to 1 thousand servers (and even more). Even though its common knowledge for devops, puppet is still a strange piece of software for developers. How does it work and what can it do for you as a developer?
Bulletproof Networks provides managed hosting services to some of the largest companies in Australia. Bulletproof implements strong isolation of customer environments, and this can present unique challenges when re-using Puppet code across our customer base. Additionally, the environments range in size from small to very large, and our tools + processes need to be able to handle both uses cases equally well.
In this talk Lindsay + Mick will cover how Bulletproof's approach to these problems has evolved over the last 4 years, and some of the tools Bulletproof has developed and built upon to provide an awesome service to our customers.
->Introduction
->>What is Ansible?
->>Ansible history
->Basic concepts
->>Inventory
->>Playbook
->>Role
->>Module
->>Plugin
->Diving into Ansible roles
->>Getting started
->>Create a role
->>Roles under the hood
->>How to use roles?
Challenges when building high profile editorial sitesYann Malet
This talk will be a walk through the challenges encountered when building a high profile editorial sites. My goal is to present some of the common pitfalls we have encountered at Lincoln Loop and to explain how we solved:
* Legacy migration always take longer
* devops
* Multiple environment
* Easy deployment
* Responsive design impacts the backend
* Journey of an image
* Picturefill.js
* Danger of reusing published django applications
* Caching strategy
* Html fragment
* Varnish
Audience Decision maker that are going to rebuild their magazine Developer bidding for this kind of projects for the first time
Practical Performance: Understand and improve the performance of your applica...Chris Bailey
This session discusses how you can maximize the performance of your application deployment with tools that are native to your server platform as well as cross-platform Java analysis and monitoring tools. The session begins with systematic steps you can take to locate a performance problem in a complex system and moves on to analysis you can do to understand the root cause of the problem. The picture is completed by consideration of the tools and techniques available to monitor application performance in normal operation so that you can catch performance issues before they build up into serious problems.
Presented at JavaOne 2012
Video available from Parleys.com:
https://www.parleys.com/talk/the-hidden-world-your-java-application-what-its-really-doing
kubernetes install and practice
* Environment (bare metal installation, not using cloud service)
- VM 1 : Mater node, 30GB, 2 vCPU, 4GB Mem
- VM 2 : Worker node, 30GB, 2 vCPU, 4GB Mem
* Practice
- deploying pod, make a deployment and service
- expose service using ingress(nginx-ingress)
Slides with notes from Ruby Conf 2014 on using simple techniques to create sl...Justin Gordon
Slides with notes from Ruby Conf 2014 on using simple techniques to create slimer, clearer models, controllers, and views in Ruby on Rails. Looks better at http://www.slideshare.net/justingordon/rails-conf-2014concernsdecoratorspresentersserviceobjectshelpershelpmedecideapril222014
blog: http://www.railsonmaui.com
Code samples: https://github.com/justin808/fat-code-refactoring-techniques
React on Rails is the leading solution for integrating Ruby on Rails with React, using 1st class JavaScript tooling such as Webpack and Babel. It support React, Redux, React-Router, all with server rendering.
IT Infrastructure Through The Public Network Challenges And SolutionsMartin Jackson
Identifying the challenges that companies face when they wish to adopt Infrastructure as a Service like those from Amazon and Rackspace and possible solutions to those problems. This presentation seeks to provide insight and possible solutions, covering the areas of security, availability, cloud standards, interoperability, vendor lock in and performance management.
Puppet is a configuration management tool which allows easy deployment and configuration ranging from 1 to 1 thousand servers (and even more). Even though its common knowledge for devops, puppet is still a strange piece of software for developers. How does it work and what can it do for you as a developer?
Bulletproof Networks provides managed hosting services to some of the largest companies in Australia. Bulletproof implements strong isolation of customer environments, and this can present unique challenges when re-using Puppet code across our customer base. Additionally, the environments range in size from small to very large, and our tools + processes need to be able to handle both uses cases equally well.
In this talk Lindsay + Mick will cover how Bulletproof's approach to these problems has evolved over the last 4 years, and some of the tools Bulletproof has developed and built upon to provide an awesome service to our customers.
->Introduction
->>What is Ansible?
->>Ansible history
->Basic concepts
->>Inventory
->>Playbook
->>Role
->>Module
->>Plugin
->Diving into Ansible roles
->>Getting started
->>Create a role
->>Roles under the hood
->>How to use roles?
Challenges when building high profile editorial sitesYann Malet
This talk will be a walk through the challenges encountered when building a high profile editorial sites. My goal is to present some of the common pitfalls we have encountered at Lincoln Loop and to explain how we solved:
* Legacy migration always take longer
* devops
* Multiple environment
* Easy deployment
* Responsive design impacts the backend
* Journey of an image
* Picturefill.js
* Danger of reusing published django applications
* Caching strategy
* Html fragment
* Varnish
Audience Decision maker that are going to rebuild their magazine Developer bidding for this kind of projects for the first time
Practical Performance: Understand and improve the performance of your applica...Chris Bailey
This session discusses how you can maximize the performance of your application deployment with tools that are native to your server platform as well as cross-platform Java analysis and monitoring tools. The session begins with systematic steps you can take to locate a performance problem in a complex system and moves on to analysis you can do to understand the root cause of the problem. The picture is completed by consideration of the tools and techniques available to monitor application performance in normal operation so that you can catch performance issues before they build up into serious problems.
Presented at JavaOne 2012
Video available from Parleys.com:
https://www.parleys.com/talk/the-hidden-world-your-java-application-what-its-really-doing
kubernetes install and practice
* Environment (bare metal installation, not using cloud service)
- VM 1 : Mater node, 30GB, 2 vCPU, 4GB Mem
- VM 2 : Worker node, 30GB, 2 vCPU, 4GB Mem
* Practice
- deploying pod, make a deployment and service
- expose service using ingress(nginx-ingress)
Slides with notes from Ruby Conf 2014 on using simple techniques to create sl...Justin Gordon
Slides with notes from Ruby Conf 2014 on using simple techniques to create slimer, clearer models, controllers, and views in Ruby on Rails. Looks better at http://www.slideshare.net/justingordon/rails-conf-2014concernsdecoratorspresentersserviceobjectshelpershelpmedecideapril222014
blog: http://www.railsonmaui.com
Code samples: https://github.com/justin808/fat-code-refactoring-techniques
React on Rails is the leading solution for integrating Ruby on Rails with React, using 1st class JavaScript tooling such as Webpack and Babel. It support React, Redux, React-Router, all with server rendering.
Rails Conf 2014 Concerns, Decorators, Presenters, Service-objects, Helpers, H...Justin Gordon
Slides Ruby Conf 2014 on using simple techniques to create slimer, clearer models, controllers, and views in Ruby on Rails.
blog: http://www.railsonmaui.com
Code samples: https://github.com/justin808/fat-code-refactoring-techniques
xUnit and TDD: Why and How in Enterprise Software, August 2012Justin Gordon
“A comprehensive suite of JUnit tests is one of the most import aspects of a software project because it reduces bugs, facilitates adding new developers, and enables refactoring and performance tuning with confidence. Test-driven development (TDD) is the best way to build a suite of tests. And the Dependent Object Framework is the best way to test against database objects.” This presentation covers the benefits of TDD along with practical advice on how to implement TDD in complex projects.
Organizations often need to quickly analyze large amounts of data, such as logs generated from a wide variety of sources and formats. However, traditional approaches require a lot of time and effort designing complex data transformation and loading processes; and configuring data warehouses. Using AWS, you can start querying your datasets within minutes. In this session you will learn how you can deploy a managed Presto environment in minutes to interactively query log data using standard ANSI SQL. Presto is a popular open source SQL engine for running interactive analytic queries against data sources of all sizes. We will talk about common use cases and best practices for running Presto on Amazon EMR.
Introduction to node.js by Ran Mizrahi @ Reversim SummitRan Mizrahi
Node.js is a platform built on Chrome V8 javascript runtime engine for building fast and scalable, non-blocking, real-time and network applications. In this session I'll introduce you to node.js and developing large code bases using it. We'll cover the following aspects:
* What is node.js?
* Apache vs. Nginx performance (One thread per connection vs. event loop) and what it has to do with node.js.
* Why node was written in Javascript?
* Main tools and frameworks (Express, socket.io, mongoose etc.)
* TDD/BDD with node.js using mocha and Expect.js
Lean microservices through ahead of time compilation (Tobias Piper, Loveholid...London Microservices
Recorded at the London Microservices Meetup: https://www.meetup.com/London-Microservices/
- Date: 05/08/2020
- Event page: https://www.meetup.com/London-Microservices/events/272223163/
Follow us on Twitter! https://twitter.com/LondonMicrosvc
---
Services interpreted at runtime often suffer from slow startup times when components are intitialized at runtime. Ahead of time compilation allows condensing down an application by stripping down unused dependencies and pre-initializing components for a short time to first request served. This is explored by the example of GraalVM and native images.
Key takeaways:
- Be aware what your service contains
- Approaches to improve startup time of a service
- Reduction of resource requirements for a service
Tobias works at Loveholidays.com as a senior software engineer extending it's microservice architecture and turning it more event-driven.
Big Data & Cloud | Cloud Storage Simplified | Adrian ColeJAX London
2011-11-01 | 04:20 PM - 05:10 PM
Key/value stores are the most common storage offerings in the cloud today. We'll introduce storage concepts and differences between cloud storage providers including Amazon S3, Rackspace Cloud Files & Microsoft Azure Blob Service. We'll demonstrate how jclouds BlobStore helps Java and Clojure developers avoid lock-in and increase testability without restricting access to provider-specific features
We use Gearman for managing queue system. This covers why we should use a queue in many situations on web-based interface as well as server-side application.
Alfresco Environment Validation and "Day Zero" ConfigurationAlfresco Software
This session will commence with the environmental checks that should be performed prior to the installation of Alfresco, and then describe the "day zero" configuration changes that should be made to ensure that the installed Alfresco instance is optimally configured.
From continuous integration servers to blogging systems, we've all seen and used pluggable applications. Writing our own though can be an elusive task. That need not be the case, though, as the Java EE spec contains all you need to do just that. In this session, we'll see how we can leverage the power of CDI to write, for example, easily extensible JSF applications. When the session is over, you'll have all you need to write the next killer app, and, thanks to Java EE, you'll be surprised to see how little work it really is.
Mobile & Desktop Cache 2.0: How To Create A Scriptable CacheBlaze Software Inc.
In this webinar, we’ll describe how you can build your own Scriptable Cache based on HTML5 localStorage, making Mobile cache work and giving Desktop Cache a boost. We’ll discuss the value of a Scriptable Cache, show the key elements you’ll need to create, and mention some of the pitfalls you need to beware of.
Similar to Improving Your Heroku App Performance with Asset CDN and Unicorn (20)
Mobile & Desktop Cache 2.0: How To Create A Scriptable Cache
Improving Your Heroku App Performance with Asset CDN and Unicorn
1. Optimizing rails applications’
performance
with Asset CDN and Unicorn
Simon Bagreev, @status_200
sbagreev@gmail.com
Friday, January 11, 13
2. Question
Does anyone know what
f5dd
is?
Friday, January 11, 13
3. What This Preso is NOT
• not a coding demo or tutorial (strangely)
• not a best practices showcase
Friday, January 11, 13
4. What This Preso IS
• tips and tricks on tuning rails application
• personal experience
Friday, January 11, 13
5. Disclaimer
There are many other ways to improve app’s performance:
•database performance (indexes, N+1, slow queries)
•caching
•background processing
•changing interpreter, GC
•conditional asset loading, etc
•removing cruft!
Try them first!
Friday, January 11, 13
6. This Presentation - Two Parts
• CDN Asset Host using aws*
• unicorn web server*
* Both approaches were tested on heroku, but must work with other hosting solutions
Friday, January 11, 13
12. After Deployment
should see entries like this in your log
cache: [GET /assets/application-c0747cab950350f59304a3815f980622.css] miss, store
cache: [GET /assets/application-032691d4988a7003c42a10995819a0ce.js] miss, store
cache: [GET /assets/s_code.js] miss, store
cache: [GET /assets/application-c0747cab950350f59304a3815f980622.css] fresh
cache: [GET /assets/application-032691d4988a7003c42a10995819a0ce.js] fresh
cache: [GET /assets/s_code.js] fresh
Friday, January 11, 13
13. Using Rack::Cache Effect
down from 4.91 / 201 / 2.29 before the change -- not bad for 8 lines of code!
Friday, January 11, 13
14. Step 2 - S3 bucket for assets
heroku instance has more time to serve application code because all assets are served from aws s3
Friday, January 11, 13
15. S3 Bucket for Assets
# Gemfile
gem "asset_sync" # will push compiled assets into CDN
# Command line
heroku config:add FOG_PROVIDER=AWS
AWS_ACCESS_KEY_ID=xxx
AWS_SECRET_ACCESS_KEY=yyy
heroku config:add FOG_DIRECTORY=yourappname-assets
# config/environments/production.rb
config.action_controller.asset_host =
"//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"
# make sure to use AssetTagHelper methods (like image_tag)
# to ensure assets are properly referenced
Friday, January 11, 13
16. Now, on git push heroku
assets are automatically synced to s3 anytime ON rake assets:precompile
Friday, January 11, 13
17. S3 Bucket effect
down from 4.45 / 179 / 2.43 before the change -- even better!
Friday, January 11, 13
18. Step 3 - AWS CloudFront
Friday, January 11, 13
19. CloudFront Effect
down from 3.85 / 179 / 2.19 before the change -- Awesome!
Friday, January 11, 13
20. Loading Single File
$ time curl http://careersingear.mobi/assets/application-
bdb77a926724ccc3c20b923ab168d89d.js
real 0m0.896s
user 0m0.008s
sys 0m0.016s
----------------
$ time curl http://d3kd72psxbec02.cloudfront.net/assets/
application-bdb77a926724ccc3c20b923ab168d89d.js
real 0m0.293s
user 0m0.006s
sys 0m0.010s
getting a single application.js file from cloud front is 3x faster
Friday, January 11, 13
23. Meet Unicorn
• HTTp server for Ruby
• Starts one master process
• forks worker processes
• workers handle requests
• master returns
• one port, several
concurrent requests
Friday, January 11, 13
24. Server Setup
Unicorn setup
classic setup
nginx -> unix domain socket -> unicorn
nginx -> smart balancer -> pool of mongrels
workers (os handles load balancing)
Friday, January 11, 13
26. Unicorn for Rails App
# config/unicorn.rb
worker_processes 3
timeout 30
preload_app true
before_fork do |server, worker|
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
Rails.logger.info('Disconnected from ActiveRecord')
end
if defined?(Resque)
Resque.redis.quit
Rails.logger.info('Disconnected from Redis')
end
end
after_fork do |server, worker|
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
Rails.logger.info('Connected to ActiveRecord')
end
if defined?(Resque)
Resque.redis = ENV["REDISTOGO_URL"]
Rails.logger.info('Connected to Redis')
end
end
Friday, January 11, 13
27. After Implementing Unicorn
down from 3.64 / 46.7 / 1.17 before the change -- good, but also...
Friday, January 11, 13
28. Better Concurrency Handling
require 'typhoeus'
require "benchmark"
URL = "http://careersingear.mobi"
HYDRA = Typhoeus::Hydra.new(max_concurrency: 20)
1000.times do
request = Typhoeus::Request.new(URL, method: :get, timeout: 10000)
request.on_complete do |response|
puts response.code
end
HYDRA.queue(request)
end
Benchmark.bm(7) do |x|
x.report("first:") { HYDRA.run }
end
# using thin
# user system total real
# 1.030000 0.380000 1.410000 ( 16.713791)
# using unicorn
# user system total real
# 1.050000 0.390000 1.440000 ( 7.843766)
Friday, January 11, 13
29. And ...
my app can process six concurrent requests on two heroku dynos
Friday, January 11, 13
31. To Conclude
• implemented asset cdn
• configured unicorn
• brought down average end user load time from almost
5 sec to 3.5 sec
• app can serve more requests faster and for less $$$
Friday, January 11, 13
33. Credits
defunkt, unicorn! https://github.com/blog/517-unicorn
heroku dev center, using rack::cache with memcached in rails 3.1+ https://devcenter.heroku.com/articles/
rack-cache-memcached-rails31
Rice, david, using a cdn asset host with rails 3.1 https://devcenter.heroku.com/articles/cdn-asset-
host-rails31
Sikkes, Michael, Complete Guide to serving your Rails assets over S3 with asset_sync http://blog.firmhouse.com/
complete-guide-to-serving-your-rails-assets-over-s3-with-asset_sync
van roijen, michael, more concurrency on a single heroku dyno with the new celadon cedar stack http://
michaelvanrooijen.com/articles/2011/06/01-more-concurrency-on-a-single-heroku-dyno-
with-the-new-celadon-cedar-stack/
Friday, January 11, 13
34. Q&A
This presentation can be found on github github.com/semmin/asset-cdn-and-unicorn-preso
twitter: @status_200
Email: sbagreev@gmail.com
Questions?
Friday, January 11, 13