While it’s easy to get started developing applications using Ruby on Rails, the hard part is (as with most technologies) how to put it into production? There are a lot of options like Mongrel, nginx, Apache and JRuby, all of them difficult to evaulate without deeper knowledge of Rails and your application’s scaling needs. We will look at the available options to get a clearer picture how each setup is in different situations, and how you can utilize Capistrano for an easy deployment process in those situations. Simple as well as more complex setups will be discussed. We’ll try to include your problems as well as your experiences to discuss specific issues with deployment and scalability.
Plack basics for Perl websites - YAPC::EU 2011leo lapworth
Run a website with Perl? - you should learn how to use Plack. Most Perl web frameworks support it and it makes your life a lot easier and a lot more fun
JavaOne 2015 : How I Rediscovered My Coding Mojo by Building an IoT/Robotics ...Mark West
Is your project dragging you down? Are you stuck with the same old technologies? Are you bored with coding? If you answer “yes” to any of these questions, you may have lost your coding mojo—just like this session’s speaker did a few years back. Come hear how he learned new technologies and rediscovered his coding mojo by building an IoT/robotics prototype: a voice-controlled robot. Along the way, you’ll hear about HTML5 speech recognition, controlling hardware with Node.js and Johnny-Five, using WebSocket and MQTT for communication between components, and finally how you can easily combine the Raspberry Pi and Arduino platforms to gain ultimate power over your own projects.
Even basic AEM deployment involves some network communication. All services need to be aware of each other to make the entire AEM stack usable for both content editors and end users.
The truth is, basic AEM deployments are not that common these days. In many cases it's much more complex - there's plenty of services around you (search engines, caching servers, data feeds, etc) and you need to talk to them in this way or another. Even though that's not the case in your project, most probably you have more than one environment to deal with (unless you're Facebook, as they run just production). All in all it makes perfect sense to run service discovery tool in your AEM infrastructure, as in a long term it gets really painful to manage all these communication channels by hand.
During my talk I'd present how Cognifide combined Consul and Chef to:
- make sure AEM always talk to correct endpoint, no matter how many instances of given service we run
- no longer worry about hardcoded IP addresses in AEM configs or Chef cookbooks
- automatically pick up new services as they go online
- enable even faster, zero-downtime deployments
- orchestrate the entire AEM infrastructure
An interesting fact is that we were able to achieve all of these without a single change in our AEM app!
It's crucial to make sure everyone's local environment is identical if you're a part of geographically distributed team and your project is composed of many services. It's also quite unlikely that all cogs in your app will spin self-sufficiently. Search engines, various APIs, authentication endpoints and HTTP accelerators - all of these services are standard these days. Your app is a part of a much wider picture.
This complexity forces you not only to broaden your mind during design phase, but it also affects the way your software is deployed. All the moving parts should be available on every single environment - including the one that works on your machine. Having in mind aforementioned complexity double click on a ZIP file is a poor man's solution. Duct taping things will end badly. And if anything can go wrong, it will, so better be prepared.
Our goal was clear - portable, versionable and predictable environment everyone can run. Throughout our journey with Vagrant, Packer and Chef we learned a lot of things:
- how box over-optimization can lead to poor user experience
- why it's important to make first "vagrant up" as quick as possible
- not everything should be dockerized
- Vagrant box is a 'virtual' release artifact and needs the same care your app receives
- successful Packer build is often half of the story
- why Consul deployment in a Vagrant box is not as trivial as it may sound
- how to optimize long Chef runs and provide more granular approach to your dev team
- it's surprisingly easy to get from "I don't want to run any VMs on my laptop!" to "Hope you have Vagrant for that"
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesikmfrancis
OSGi Community Event 2014
Abstract:
Experience level: Beginner
Apache Karaf is a small OSGi based runtime which provides a lightweight container onto which various components and applications can be deployed.
This session gives an overview over Apache Karaf, how to manage and configure Apache Karaf and how to use the comprehensive command shell that Karaf provides. By the example of a simple pet clinic application we will demonstrate how to build and deploy a web application on Apache Karaf. This talk will introduce the different deployment mechanisms available in Karaf and how to deploy bundles from the file system, command shell or via a maven repository. You will learn how to manage external dependencies via features, use the Karaf maven plugin to create features and build and configure persistence bundles for Apache Karaf. We will further give an overview, how to use Karaf in a cloud environment and how to use the Apache Cellar project to build up an Apache Karaf cluster.
Speaker Bios:
Torsten Frank
Torsten Frank is an entrepreneur and healthcare IT expert with international experience and a strong background in product and business development. He is CEO and founder of medisite, a company that delivers highly specialized healthcare IT solutions to their customers for more than 10 years. medisite is a winner of the Federal Ministry for Economic Affairs and Energy (BMWi) “Trusted Cloud” technology competition, geared toward secure cloud computing for SMEs and the public sector. Torsten Frank holds a medical degree from the Hannover Medical School, where he also has worked for several years as a physician at the Department of Cardiothoracic Surgery after completing his medical studies in Hannover, Germany and Chicago, USA.
Alexander Grzesik
Alexander is the head of development of medisite Systemhaus GmbH and responsible for the development of the the PaaS+ cloud platform the clinical information system m.life and software architect for the TRESOR Project.
He has 15 years of work experience in medical Software development as team leader and software architect.
Expert for Software Architecture, OSGi, Java and Java EE.
Alexander has been a speaker at several conferences including EclipseCon Europe and the OpenShift Community Day.
A talk given to JCConf 2015 on 2015/12/05.
在程式設計領域,“immutable objects” 是相當重要的設計模式。同樣的,在虛擬化及雲端時代,“immutable infrastructure” 也成為新一代的顯學。在資源及流程的充分配合下,這將會大大簡化系統的複雜度,穩定性也會大大提升。
本演講將會從觀念出發,並佐以部份實作建議,讓大家有足夠資訊來評估此架構的好處。
Video: https://youtu.be/9j008nd6-A4
You have amazing content and you want to get it to your users as fast as possible. In today’s industry, milliseconds matter and slow websites will never keep up. You can use a CDN but they are expensive, make you dependent on a third party to deliver your content, and can be notoriously inflexible. Enter Varnish, a powerful, open-source caching reverse proxy that lives in your network and lets you take control of how your content is managed and delivered. We’ll discuss how to install and configure Varnish in front of a typical web application, how to handle sessions and security, and how you can customize Varnish to your unique needs. This session will teach you how Varnish can help you give your users a better experience while saving your company and clients money at the same time.
Introduction to deployment with Ruby on Rails presented at JAX09 in Mainz by Jonathan Weiss.
Learn about the deployment architectures and setups (web and app tier) and using Capistrano/Webistrano.
Plack basics for Perl websites - YAPC::EU 2011leo lapworth
Run a website with Perl? - you should learn how to use Plack. Most Perl web frameworks support it and it makes your life a lot easier and a lot more fun
JavaOne 2015 : How I Rediscovered My Coding Mojo by Building an IoT/Robotics ...Mark West
Is your project dragging you down? Are you stuck with the same old technologies? Are you bored with coding? If you answer “yes” to any of these questions, you may have lost your coding mojo—just like this session’s speaker did a few years back. Come hear how he learned new technologies and rediscovered his coding mojo by building an IoT/robotics prototype: a voice-controlled robot. Along the way, you’ll hear about HTML5 speech recognition, controlling hardware with Node.js and Johnny-Five, using WebSocket and MQTT for communication between components, and finally how you can easily combine the Raspberry Pi and Arduino platforms to gain ultimate power over your own projects.
Even basic AEM deployment involves some network communication. All services need to be aware of each other to make the entire AEM stack usable for both content editors and end users.
The truth is, basic AEM deployments are not that common these days. In many cases it's much more complex - there's plenty of services around you (search engines, caching servers, data feeds, etc) and you need to talk to them in this way or another. Even though that's not the case in your project, most probably you have more than one environment to deal with (unless you're Facebook, as they run just production). All in all it makes perfect sense to run service discovery tool in your AEM infrastructure, as in a long term it gets really painful to manage all these communication channels by hand.
During my talk I'd present how Cognifide combined Consul and Chef to:
- make sure AEM always talk to correct endpoint, no matter how many instances of given service we run
- no longer worry about hardcoded IP addresses in AEM configs or Chef cookbooks
- automatically pick up new services as they go online
- enable even faster, zero-downtime deployments
- orchestrate the entire AEM infrastructure
An interesting fact is that we were able to achieve all of these without a single change in our AEM app!
It's crucial to make sure everyone's local environment is identical if you're a part of geographically distributed team and your project is composed of many services. It's also quite unlikely that all cogs in your app will spin self-sufficiently. Search engines, various APIs, authentication endpoints and HTTP accelerators - all of these services are standard these days. Your app is a part of a much wider picture.
This complexity forces you not only to broaden your mind during design phase, but it also affects the way your software is deployed. All the moving parts should be available on every single environment - including the one that works on your machine. Having in mind aforementioned complexity double click on a ZIP file is a poor man's solution. Duct taping things will end badly. And if anything can go wrong, it will, so better be prepared.
Our goal was clear - portable, versionable and predictable environment everyone can run. Throughout our journey with Vagrant, Packer and Chef we learned a lot of things:
- how box over-optimization can lead to poor user experience
- why it's important to make first "vagrant up" as quick as possible
- not everything should be dockerized
- Vagrant box is a 'virtual' release artifact and needs the same care your app receives
- successful Packer build is often half of the story
- why Consul deployment in a Vagrant box is not as trivial as it may sound
- how to optimize long Chef runs and provide more granular approach to your dev team
- it's surprisingly easy to get from "I don't want to run any VMs on my laptop!" to "Hope you have Vagrant for that"
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesikmfrancis
OSGi Community Event 2014
Abstract:
Experience level: Beginner
Apache Karaf is a small OSGi based runtime which provides a lightweight container onto which various components and applications can be deployed.
This session gives an overview over Apache Karaf, how to manage and configure Apache Karaf and how to use the comprehensive command shell that Karaf provides. By the example of a simple pet clinic application we will demonstrate how to build and deploy a web application on Apache Karaf. This talk will introduce the different deployment mechanisms available in Karaf and how to deploy bundles from the file system, command shell or via a maven repository. You will learn how to manage external dependencies via features, use the Karaf maven plugin to create features and build and configure persistence bundles for Apache Karaf. We will further give an overview, how to use Karaf in a cloud environment and how to use the Apache Cellar project to build up an Apache Karaf cluster.
Speaker Bios:
Torsten Frank
Torsten Frank is an entrepreneur and healthcare IT expert with international experience and a strong background in product and business development. He is CEO and founder of medisite, a company that delivers highly specialized healthcare IT solutions to their customers for more than 10 years. medisite is a winner of the Federal Ministry for Economic Affairs and Energy (BMWi) “Trusted Cloud” technology competition, geared toward secure cloud computing for SMEs and the public sector. Torsten Frank holds a medical degree from the Hannover Medical School, where he also has worked for several years as a physician at the Department of Cardiothoracic Surgery after completing his medical studies in Hannover, Germany and Chicago, USA.
Alexander Grzesik
Alexander is the head of development of medisite Systemhaus GmbH and responsible for the development of the the PaaS+ cloud platform the clinical information system m.life and software architect for the TRESOR Project.
He has 15 years of work experience in medical Software development as team leader and software architect.
Expert for Software Architecture, OSGi, Java and Java EE.
Alexander has been a speaker at several conferences including EclipseCon Europe and the OpenShift Community Day.
A talk given to JCConf 2015 on 2015/12/05.
在程式設計領域,“immutable objects” 是相當重要的設計模式。同樣的,在虛擬化及雲端時代,“immutable infrastructure” 也成為新一代的顯學。在資源及流程的充分配合下,這將會大大簡化系統的複雜度,穩定性也會大大提升。
本演講將會從觀念出發,並佐以部份實作建議,讓大家有足夠資訊來評估此架構的好處。
Video: https://youtu.be/9j008nd6-A4
You have amazing content and you want to get it to your users as fast as possible. In today’s industry, milliseconds matter and slow websites will never keep up. You can use a CDN but they are expensive, make you dependent on a third party to deliver your content, and can be notoriously inflexible. Enter Varnish, a powerful, open-source caching reverse proxy that lives in your network and lets you take control of how your content is managed and delivered. We’ll discuss how to install and configure Varnish in front of a typical web application, how to handle sessions and security, and how you can customize Varnish to your unique needs. This session will teach you how Varnish can help you give your users a better experience while saving your company and clients money at the same time.
Introduction to deployment with Ruby on Rails presented at JAX09 in Mainz by Jonathan Weiss.
Learn about the deployment architectures and setups (web and app tier) and using Capistrano/Webistrano.
Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in RailsJonathan Weiss
Scaling a web application is a very hard problem, especially for small project and teams who do not have sufficient manpower, money, and time to solve this problem. Luckily Amazon already had to solve this problem in their datacenters and offers their services to other developers. This talk will introduce the two most important Amazon Web Services, the Elastic Compute Cloud (EC2) and the Simple Storage Service (S3), and will present different ways to leverage them in your own Rails application.
Presented at RailsConfEurope Berlin 2007
Dynamic Languages & Web Frameworks in GlassFishIndicThreads
“Dynamic languages such as JRuby, Groovy, and Jython are increasingly playing an important role in the web these days. The associated frameworks such as Rails, Grails, and Django are gaining importance because of the agility provided by them.
The GlassFish project provides an easy-to-use and robust development and deployment platform for hosting these web applications. It also enables the various languages to leverage the investment in your existing Java Platform, Enterprise Edition (Java EE platform) infrastructure. This session gives an overview of various Dynamic Languages and associated Web frameworks that can be used on the GlassFish project.
It starts with a brief introduction to JRuby and details on how the GlassFish project provides a robust development and deployment platform for Rails, Merb, Sinatra and other similar applications without pain. As a basis for further discussion, this presentation shows the complete lifycycle for JRuby-on-Rails applications on GlassFish v2 and v3. It discusses the various development options provided by GlassFish v3, demonstrates how popular Rails applications can be easily deployed on GlassFish without any modification, and shows how v3 Gem can be used as an effective alternative to WEBrick and Mongrel. It also demonstrates debugging of Rails applications using NetBeans IDE. For enterprise users, it shows how JMX and other mechanisms can be used to monitor Rails applications.
It also talks in detail about the Groovy/Grails and Python/Django development and deployment models in context of GlassFish v3. By following the simple deployment steps the presentation shows, developers will be able to deploy their existing web applications on the GlassFish project.The session also describes the known limitations and workarounds for each of them.
The talk will show a working sample created in different frameworks and deployed on GlassFish v3. The demo will show how different features of the underlying GlassFish runtime are easily accessible to the frameworks running on top of it.”
In order to understand how to scale Node.js you need to know how the internals work together and what type of problems are best suited for it. With the right combination of tools you can easily have a scalable and reliable Node.js cluster.
A 1 hour presentation of how GlassFish v3 Prelude provides support for Scripting / Dynamic Languages. Ruby/JRuby/Rails and Groovy/Grails are specifically described.
AWS OpsWorks & Chef at the Hamburg Chef User Group 2014Jonathan Weiss
An introduction to AWS OpsWorks and how it uses Chef. Differences between OpsWorks and Chef server.
Presented by Jonathan Weiss on January 14th 2014 at the Hamburg Chef User Group.
An introduction into Backbone.js – a lightweight MVC framework. Backbone supplies structure to JavaScript-heavy applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing application over a RESTful JSON interface.
Build your own clouds with Chef and MCollectiveJonathan Weiss
One important part of the DevOps movement is infrastructure automation, especially if you are running your application on top of services like Amazon EC2.
Everybody's dream is to be able to bootstrap and deploy hundreds or even thousands of machines with a few simple commands. This talk will tell you how you can do this using Open Source tools like Chef and mcollective. Chef manages your servers configuration using a nice Ruby DSL while mcollective orchestrates and commands all your nodes.
Platforms like Amazon EC2 promise scalable and redundant systems for a couple of pennies. As soon as you start to build complex systems or migrate existing apps there are many knobs to set. This talk will explain how you can create and deploy reliable and redundant applications to EC2 and will point out all the little things you need to know, like how to automatically provision new servers with tools like Chef.
Presented by Jonathan Weiss at PHP UK Conference 2011 in London.
Overview of how to manage deployments and clusters in the Amazon cloud. Introduction into Chef. Presented by Jonathan Weiss at RailsCamp DE in Cologne.
Rails in the Cloud - Experiences from running on EC2Jonathan Weiss
Overview of architectures in EC2 and services like EBS, ELB, RDS, and ElasticIPs. How to get your app on EC2. Configuration and deployment with Chef. Presented by Jonathan Weiss at RailsWayCon 2010 in Berlin
Jonathan Weiss gives an overview of the NoSQL databases. Why would you consider one, what are the tradeoffs? Given at BarCampRuhr3 2010 in Essen, Germany (Slides are English).
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
2. Who are we?
Jonathan Weiss
• Consultant for Peritor GmbH in Berlin
• Specialized in Rails, Scaling, Deployment, and Code Review
• Webistrano - Rails deployment tool
• FreeBSD Rubygems and Ruby on Rails maintainer
http://www.peritor.com
http://blog.innerewut.de
2
3. Who are we?
Mathias Meyer
• Consultant for Peritor GmbH in Berlin
• Specialized in Rails, Performance/Database Tuning,
Deployment, and Refactoring
• Maintainer of Macistrano –
Webistrano-Client for Mac OS X (and soon the iPhone),
run_later and acts_as_solr
http://www.paperplanes.de
3
9. Typical Rails Setup
• A load-balancer distributes the incoming requests
• Some load-balancers will deliver static requests themselves
• Several Rails instances handle all requests
• Number of concurrent requests equals number of Rails instances
9
14. FastCGI
• Protocol to communicate with long-running CGI applications
• Usage of either mod_fcgi with Apache 1.3 or mod_fcgi with Lighttpd
• Proxy local and remote FastCGI instances
• Oldest way of deploying Rails
• Deprecated and unstable
• Hard to debug (FastCGI protocol)
14
19. Mongrel
• Developed by Zed Shaw as an alternative to FastCGI
• Complete HTTP-Server that can load arbitrary Ruby-servlets
• Built-in Rails support
19
25. Simple Mongrel and Apache 2.2
Redirect all traffic to the Mongel cluster
25
26. A more complex example
• Redirect dynamic requests
• Serve static content
• Support cached pages
• Support maintenance page
• Enable client-side caching of
images, stylesheets, and JavaScript
• Compress output if supported
26
27. Mongrel
• Very robust
• Strict HTTP parser
• Easy to debug (HTTP!)
• Used to be defacto deployment setup with Apache 2.2 and mod_proxy_balancer
• Can be a bit difficult to setup (mongrel_cluster, ports, Apache)
• Not so easy on mass/virtual hosting
27
29. mod_rails a.k.a Phusion Passenger
• Module for Apache 2.2 (and Nginx)
• Allows Apache to control Rails instances
• Apache starts and stops application instances depending on the application load
• Able to run any Rack-compatible Ruby application (Merb, Sinatra & Co.)
• Only manages Rails on one host - no remote instances
• Combine with HTTP-Proxy / balancing solution
29
36. Phusion Passenger
• Ready for production
• Makes setup easier – on the single machine level
• Multiple servers still require load balancer
• Suitable for mass-hosting
• upcoming standard way of deploying Rails
36
38. • Ruby Runtime on the Java Virtual Machine
• Implemented in Java and Ruby
• Compiles Ruby into Java-bytecode
• Integrates with Java code and libraries
• Java’s promises of native threads and JIT
• Allows for Ruby/Rails applications to be packaged as WAR files
• WAR files deployable on any J2EE-container:
Glassfish, JBoss, Tomcat, Jetty, …
38
44. Glassfish for development
Shrunk-down version of Glassfish v3 - packaged as a gem
Can run any Rack-based application
Installation:
• $ jruby -S gem install glassfish
• $ glassfish
• There is no Step 3
44
47. Proxy Requirements
• Hide cluster backend from the user
• Load-balancer backend instances
• Recognize down hosts
• Fair scheduler
• (Deliver static content)
47
48. Apache 2.2
• Apache 2.2 introduced mod_proxy_balancer
• mod_proxy_balancer can speak to multiple backends and balance requests
• Apache can acts as a pure proxy or can also serve static files
48
49. Apache 2.2
• Apache 2.2 introduced mod_proxy_balancer
• mod_proxy_balancer can speak to multiple backends and balance requests
• Apache can acts as a pure proxy or can also serve static files
49
50. Apache 2.2
• Apache 2.2 introduced mod_proxy_balancer
• mod_proxy_balancer can speak to multiple backends and balance requests
• Apache can acts as a pure proxy or can also serve static files
50
51. Apache 2.2
Pro
• Stable, robust, and mature
• Many people know how to work with Apache
• Integrates well with other modules (SVN, DAV, Auth, …)
Con
• Apache can be complicated to configure
• The stock Apache is quite resource-hungry compared to pure proxy solutions
51
52. Nginx – From Russia with love
• Nginx - popular Russian webserver with good proxy support
• Can load-balance multiple backends and deliver static content
• Quite popular with Mongrel as the Rails backend
• Recent Passenger support
52
54. Nginx
Pro
• Stable, robust, and fast
• Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files
• Simpler configuration file
• Can directly talk to memcached - SSI
Con
• More documentation would be nice
• No equivalent for many Apache modules
54
55. Lighttpd
• Lightweight and fast webserver
• Balancing proxy support
• Good FastCGI support
• Used to be popular – until Mongrel came around
55
57. Lighttpd
Pro
• Fast and lightweight
• Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files
• Simpler configuration file
Con
• Unstable for some people
• Slow development cycle
• More documentation would be nice
• Configuration file can be too simple (virtual host aliasing)
• No equivalent for many Apache modules
• (No Passenger support)
57
58. HA-Proxy
• HAProxy – reliable, high performance TCP/HTTP load balancer
• Proxying and content inspection
• No content serving, just a proxy
• Mature proxy module (fair scheduler)
• ACL support
See also similar Pound and Pen
58
78. Ruby Enterprise Edition
• Copy-On-Write patches to Ruby 1.8
• Saves memory when spawning several Rails instances
• Used by Phusion Passenger if available
78
79. Thin, Ebb, Evented Mongrel & Co.
• Alternatives to Mongrel
• Claim to be faster, lighter, and what have you
• Rendering “Hello World” is usually not your bottleneck
Stick with stable and robust Mongrel, or better yet, with Passenger
79
92. Basic Ingredients - Variables
• Configure basic project information
• Override Capistrano’s default assumptions
• Once set, variables are available globally
• Defined using the set method
92
93. Basic Ingredients - Roles
• Define types of servers
• Default roles
• :www
• :app
• :db
• All can point to the same server
• But all three must be defined
• At least one database server needs to be primary
93
94. Basic Ingredients - Roles
Define custom roles as you please
Can be reused when defining tasks
94
95. Basic Ingredients - Tasks
• Define an atomic set of actions for Capistrano
• Can be called from the command line
• Or other tasks
95
96. Basic Ingredients - Tasks
To find all the tasks available in your project, use
96
97. Basic Ingredients - Namespaces
Group tasks together logically
Namespaces and tasks are separated with “:”
97
100. Get Your Capistrano On
config/deploy.rb, application specific configuration
100
101. Capistrano’s Defaults
• Your SCM is Subversion
• Deployment directory is /u/apps/#{application_name}
• User for SCM and SSH is the currently logged-in user
• Commands are run with sudo
101
107. The Deployment Lifecycle
The initial deployment
1. Checks the revision from the local machine
2. Checks out the code on the remote machines
3. Sets a link called current pointing to the lates release
4. Runs the migrations
5. Fires up application servers
107
109. The Deployment Lifecycle
Subsequent deployments
1. Checks the revision from the local machine
2. Checks out the code on the remote machines
3. Updates current link
4. Restarts application servers
109
129. The Rest
• Gem dependencies
• Support for deploying through gateway servers
• Parallel execution on multiple servers
• Server setup with deprec gem
129
134. Vlad the Deployer
• Simple, sometimes too simple
• No callback mechanism, instead the Rake syntax must be used
• Very non-verbose output, using --trace recommended
134
135. Chef
• System Integration Framework
• Written in Ruby – inspired by Puppet
• Manage multiple nodes
• Configuration
• Packages
• Custom scripts and commands
135
141. Chef and chef-deploy
• In heavy development: sometimes unstable and hard to debug
• Be prepared to get your hands dirty
• Better suited for configuration than deployment
• Community recipes at
http://github.com/opscode/cookbooks
141
144. Webistrano
• Web-UI to Capistrano
• Manages projects and their stages
• Alerting and Accounting
• Scriptable and extendable
• BSD License
http://labs.peritor.com/webistrano
144
145. Macistrano
• Mac-GUI to Webistrano
• Fire and monitor deployments from your desktop
http://github.com/mattmatt/macistrano
145
150. Tasks
• Make memcached restart with every deploy
• (Usually not a good idea, but why not?)
150
151. Tasks
• Make Apache recognize maintenance.html and display it if exists
151
152. Maintenance Page
• Create symbolic link to /etc/apache2/mods-available/rewrite.load in /etc/apache2/
mods-enabled
• Update virtual host configuration with rewrite rules
• Run cap deploy:web:disable
• Hook it into the deployment process
152
153. Tasks
• Deploy a Rack-based application
• Install Sinatra locally and on the server
• git://github.com/mattmatt/railswaycon.git
• Create deployment configuration
• Create Apache configuration
153
158. Monit
• Process-level monitoring
• Checks PID-files, ports, and permissions
• Reacts by executing a script and/or alerting
• Has a web GUI for administering nodes
158
164. Munin
• Host-level monitoring
• Master periodically asks nodes for local data
• Checks system resources and records historical data
• Allows to recognize trends and make predictions
• Alerting support
164
168. Exception Monitoring
• Monitor errors raised by your application
• Usually notifying you by email
• Good old exception_notification Plugin
• Exceptions as a Service
168
176. Full Text Search Engine
Separate Service
• Creates full text index
• Application queries search daemon
• Index update through application or
database
Possible Engines
• Ferret
• Sphinx
• Solr
• Lucene
• …
176
177. Search Slave
Database replication slave
• Has complete dataset
• Migrates slow search queries from master
• Can use different database table engine
177
178. Database Index
PostgreSQL Tsearch2
• Core since 8.3
• Allows to create full text index on multiple columns
or arbitrary SQL expressions
MySQL MyISAM FULLTEXT index
• Only works with MySQL <= 5.0 and MyISAM tables
• Full text index on multiple columns
178
179. What to use?
Different characteristics
• Real-time updates and stale data
• Lost updates
• Performance
• Document content and format
• Complexity
179
181. Problem
Long running tasks
• Resizing uploaded images
• Mailing
• Computing an expensive operation
• Accessing slow back-ends
When running inside request-response-cycle
• Blocks user
• Blocks Rails instance
• Hard to monitor and debug
181
187. Cloud Infrastructure
Servers come and go
• You do not know your servers before deploying
• Restarting is the same as introducing a new machine
You can’t hardcode IPs
database.yml
187
188. Solution #1
Query and manually adjust
• Servers do not change that often
• New nodes probably need manual intervention
• Use AWS ElasticIPs to ease the pain
Set servers dynamically AWS Elastic IP
188
189. Solution #2
Use a central directory service
• A central place to manage your running instances
• Instances query the directory and react
189
190. Solution #2
Use a central directory service
• A central place to manage your running instances
• Instances query the directory and react
190