The document provides tips for improving the performance of Rails applications that have common ActiveRecord performance problems related to has_many relationships. It discusses ways to avoid unnecessary SQL queries through eager loading with :include and limiting columns with :select. Writing custom SQL queries is presented as the fastest approach but also notes the tradeoffs around maintaining custom SQL. The document emphasizes testing applications with large realistic data sets and using indexes, counters, and frameworks like Merb to further boost performance.
Learn about the MariaDB 10 features that exist for developers: microseconds, virtual columns, PCRE regular expressions, DELETE ... RETURNING, geospatial extensions (GIS), dynamic columns. Use cases, and a hint of storage engines
Rego University: Hidden Automation & Gel Scripting, CA PPM (CA Clarity PPM)Rego Consulting
GEL scripting is one of the most powerful but underutilized capabilities in CA PPM (CA Clarity PPM). In this session, you will learn how to create GEL scripts that perform SQL updates, send formatted emails, XOG (import/ export) data in and out of objects, and perform integrations, such as simple GEL that FTPs, loads to a table, and XOGs non labor with error handling.
You can find the presentation file here: http://regouniversity.com/presentations-14/
Technical Track Training. For more CA PPM training, visit http://regouniversity.com or http://regoconsulting.com and find free Clarity educational community solutions at http://www.regoxchange.com/
Matteo Collina | Take your HTTP server to Ludicrous Speed | Codmeotion Madrid...Codemotion
In my journey through nodeland, I always wonder about the cost of my abstractions. Express, Hapi, Restify, or just plain Node.js core? require(‘http’) can reach 30k requests/sec, Express 22k, and Hapi 21k. I started a journey to write an HTTP framework with extremely low overhead, and Fastify was born. With its ability to reach an astonishing 37k requests/sec, Fastify can halve your cloud server bill. How can Fastify be so.. fast? We will discover all the not-so-secret techniques that were used to optimize it. In Fastify we reach a point where even allocating a callback is too slow: Ludicrous
Learn about the MariaDB 10 features that exist for developers: microseconds, virtual columns, PCRE regular expressions, DELETE ... RETURNING, geospatial extensions (GIS), dynamic columns. Use cases, and a hint of storage engines
Rego University: Hidden Automation & Gel Scripting, CA PPM (CA Clarity PPM)Rego Consulting
GEL scripting is one of the most powerful but underutilized capabilities in CA PPM (CA Clarity PPM). In this session, you will learn how to create GEL scripts that perform SQL updates, send formatted emails, XOG (import/ export) data in and out of objects, and perform integrations, such as simple GEL that FTPs, loads to a table, and XOGs non labor with error handling.
You can find the presentation file here: http://regouniversity.com/presentations-14/
Technical Track Training. For more CA PPM training, visit http://regouniversity.com or http://regoconsulting.com and find free Clarity educational community solutions at http://www.regoxchange.com/
Matteo Collina | Take your HTTP server to Ludicrous Speed | Codmeotion Madrid...Codemotion
In my journey through nodeland, I always wonder about the cost of my abstractions. Express, Hapi, Restify, or just plain Node.js core? require(‘http’) can reach 30k requests/sec, Express 22k, and Hapi 21k. I started a journey to write an HTTP framework with extremely low overhead, and Fastify was born. With its ability to reach an astonishing 37k requests/sec, Fastify can halve your cloud server bill. How can Fastify be so.. fast? We will discover all the not-so-secret techniques that were used to optimize it. In Fastify we reach a point where even allocating a callback is too slow: Ludicrous
A story about developing an application for an online store, persisting all the data as JSON.
Gives an overview of JSON functionality in Oracle Database 19c.
Yes, you read it correctly, we are jumping from 5.7 to 8.0 (that sounds familiar, doesn't it?). The new version doesn't only change the number but also changes how you write SQL. Recursive queries will allow you to generate series and work with hierarchical data. New JSON functions and performance improvements were also added to 8.0 to help you work on non-relational data. Expect to see what is new and improved in this talk to power up your application even more.
PostgreSQL Portland Performance Practice Project - Database Test 2 HowtoMark Wong
Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation is a set of examples to go along with the live presentation given on March 12, 2009.
The new JSON fields are some of the most talking about new features in MySQL 5.7. But they are by no means the only awesome things this version has to offer. MySQL 5.7 is a year old, so this talk won't be an introduction to this version. We will be digging into 5.7 to see how to make the most of the tools available in it. Want to tackle important practical problem solving for your data, make your query performance analysis more efficient or look at how virtual columns can help you index data? This talk is for you!
A story about developing an application for an online store, persisting all the data as JSON.
Gives an overview of JSON functionality in Oracle Database 19c.
Yes, you read it correctly, we are jumping from 5.7 to 8.0 (that sounds familiar, doesn't it?). The new version doesn't only change the number but also changes how you write SQL. Recursive queries will allow you to generate series and work with hierarchical data. New JSON functions and performance improvements were also added to 8.0 to help you work on non-relational data. Expect to see what is new and improved in this talk to power up your application even more.
PostgreSQL Portland Performance Practice Project - Database Test 2 HowtoMark Wong
Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation is a set of examples to go along with the live presentation given on March 12, 2009.
The new JSON fields are some of the most talking about new features in MySQL 5.7. But they are by no means the only awesome things this version has to offer. MySQL 5.7 is a year old, so this talk won't be an introduction to this version. We will be digging into 5.7 to see how to make the most of the tools available in it. Want to tackle important practical problem solving for your data, make your query performance analysis more efficient or look at how virtual columns can help you index data? This talk is for you!
Transforming Learning: A Local and Global PerspectiveJulie Lindsay
NECC2008 The purpose of this panel is to discuss educational technology globally for 21st Century learning. A panel of international educators will be assembled to discuss issues and challenges around the integration of ICT around the world.
How are different countries using technology in their educational environments- as learning about technology or as a means to communicate, collaborate, transform education- Julie (IBO and Qatar)
This presentation was prepared for a Webcast where John Yerhot, Engine Yard US Support Lead, and Chris Kelly, Technical Evangelist at New Relic discussed how you can scale and improve the performance of your Ruby web apps. They shared detailed guidance on issues like:
Caching strategies
Slow database queries
Background processing
Profiling Ruby applications
Picking the right Ruby web server
Sharding data
Attendees will learn how to:
Gain visibility on site performance
Improve scalability and uptime
Find and fix key bottlenecks
See the on-demand replay:
http://pages.engineyard.com/6TipsforImprovingRubyApplicationPerformance.html
Another year goes by, and most likely, another data access framework has been invented. It will claim to be the fastest, smartest way to talk to the database, and just like all those that came before it, it will not be. Because the best database access tool has been there for more than 30 years now, and that is PL/SQL. Although we all sometimes fall prey to the mindset of “Oh look, a shiny new tool, we should start using it," the performance and simplicity of PL/SQL remain unmatched. This session looks at the failings of other data access languages, why even a cursory knowledge of PL/SQL will make you a better developer, and how to get the most out of PL/SQL when it comes to database performance.
Drupal database Mssql to MySQL migrationAnton Ivanov
How to migrate Drupal 7 or 8 full database from Microsoft SQL Server to MySQL Server.
(Drupal Camp Kyiv 2017, June 10)
http://camp17.drupal.ua/en/schedule
Empowering the AWS DynamoDB™ application developer with AlternatorScyllaDB
Getting started with AWS DynamoDB™ is famously easy, but as an application grows and evolves it often starts to struggle with DynamoDB’s limitations. We introduce Scylla’s Alternator, which provides the same API as DynamoDB but aims to empower the application developer. In this presentation we will survey some of Alternator’s developer-centered features: Alternator lets you test and eventually deploy your application anywhere, on any public cloud or private cluster. It efficiently supports multiple tables so it does not require difficult single-table design. Finally, Alternator provides the developer with strong observability tools. The insights provided by these tools can detect bottlenecks, improve performance and even lower its cost.
PHP classes in mumbai, Introduction to PHP/MYSQL..
best PHP/MYSQL classes in mumbai with job assistance.
our features are:
expert guidance by IT industry professionals
lowest fees of 5000
practical exposure to handle projects
well equiped lab
after course resume writing guidance
For more Visit: http://vibranttechnologies.co.in/php-classes-in-mumbai.html or http://phptraining.vibranttechnologies.co.in
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
2. Discussion tonight
• Intended for new Rails Developers
• People that think Rails is slow
• Focus on simple steps to improve
common :has_many performance problems
• Short - 15mins
• All links/references up on
http://work.rowanhick.com tomorrow
2
3. About me
• New Zealander (not Australian)
• Product Development Mgr for a startup in Toronto
• Full time with Rails for 2 years
• Previously PHP/MySQL for 4 years
• 6 years Prior QA/BA/PM for Enterprise CAD/
CAM software dev company
3
4. Disclaimer
• For sake of brevity and understanding, the SQL
shown here is cut down to “psuedo sql”
• This is not an exhaustive in-depth analysis, just
meant as a heads up
• Times were done using ApacheBench through
mongrel in production mode
• ab -n 1000 http://127.0.0.1/orders/test_xxxx
4
5. ActiveRecord lets you get in
trouble far to quick.
• Super easy syntax comes at a cost.
@orders = Order.find(:all)
@orders.each do |order|
puts order.customer.name
puts order.customer.country.name
end
✴Congratulations, you just overloaded your DB
with (total number of Orders x 2) unnecessary
SQL calls
5
6. What happened there?
• One query to get the orders
@orders = Order.find(:all)
“SELECT * FROM orders”
• For every item in the orders collection
customer.name:
“SELECT * FROM customers WHERE id = x”
customer.country.name:
“SELECT * FROM customers WHERE id = y”
6
7. Systemic Problem in
Web development
I’ve seen:
- 15 Second page reloads
- 10000 queries per page
“<insert name here> language performs
really poorly, we’re going to get it
redeveloped in <insert new language
here>”
7
8. Atypical root cause
• Failure to build application with *real* data
• ie “It worked fine on my machine” but the
developer never loaded up 100’000 records
to see what would happen
• Using Rake tasks to build realistic data sets
• Test, test, test
• tail -f log/development.log
8
9. Faker to the rescue
• in lib/xchain.rake
namespace :xchain do
desc quot;Load fake customersquot;
task :load_customers => :environment do
require 'Faker'
Customer.find(:all, :conditions => quot;email LIKE('%XCHAIN_
%')quot;).each { |c| c.destroy }
1..300.times do
c = Customer.new
c.status_id = rand(3) + 1
c.country_id = rand(243) + 1
c.name = Faker::Company.name
c.alternate_name = Faker::Company.name
c.phone = Faker::PhoneNumber.phone_number
c.email = quot;XCHAIN_quot;+Faker::Internet.email
c.save
end
end
$ rake xchain:load_customers
9
10. Eager loading
• By using :include in .finds you create sql joins
• Pull all required records in one query
find(:all, :include => [ :customer, :order_lines ])
✓ order.customer, order.order_lines
find(:all, :include => [ { :customer
=> :country }, :order_lines ])
✓ order.customer order.customer.country
order.order_lines
10
11. Improvement
• Let’s start optimising ...
@orders = Order.find(:all, :include => {:customers => :country} )
• Resulting SQL ...
“SELECT orders.*, countries.* FROM orders LEFT JOIN
customers ON ( customers.id = orders.customers_id )
LEFT JOIN countries ON ( countries.id =
customers.country_id)
✓ 7.70 req/s 1.4x faster
11
12. Select only what you
need
• Using the :select parameter in the find
options, you can limit the columns you are
requesting back from the database
• No point grabbing all columns, if you only
want :id and :name
Orders.find(:all, :select => ‘orders.id,
orders.name’)
12
13. The last slide was very
important
• Not using selects is *okay* provided you
have very small columns, and never any
binary, or large text data
• You can suddenly saturate your DB
connection.
• Imagine our Orders table had an Invoice
column on it storing a pdf of the invoice...
13
14. Oops
• Can’t show a benchmark
• :select and :include don’t work together !,
reverts back to selecting all columns
• Core team for a long time have not
included patches to make it work
• One little sentence in ActiveRecord rdoc
“Because eager loading generates the SELECT
statement too, the :select option is ignored.”
14
15. ‘mrj’ to the rescue
• http://dev.rubyonrails.org/attachment/ticket/
7147/init.5.rb
• Monkey patch to fix select/include problem
• Produces much more efficient SQL
15
17. r8672 change
• http://blog.codefront.net/2008/01/30/living-on-the-
edge-of-rails-5-better-eager-loading-and-more/
• The following uses new improved association load
(12 req/s)
@orders = Order.find(:all, :include => [{:customer
=> :country}, :order_status] )
• The following does not
@orders = Order.find(:all, :include => [{:customer
=> :country}, :order_status], :order =>
‘order_statuses.sort_order’ )
17
18. r8672 output...
• Here’s the SQL
Order Load (0.000837) SELECT * FROM `orders` WHERE (order_status_id <
100) LIMIT 10
Customer Load (0.000439) SELECT * FROM `customers` WHERE
(customers.id IN (2106,2018,1920,2025,2394,2075,2334,2159,1983,2017))
Country Load (0.000324) SELECT * FROM `countries` WHERE (countries.id
IN (33,17,56,150,194,90,91,113,80,54))
OrderStatus Load (0.000291) SELECT * FROM `order_statuses` WHERE
(order_statuses.id IN (10))
18
19. But I want more
• Okay, this still isn’t blazing fast. I’m building
the next killr web2.0 app
• Forgetabout associations, just load it via
SQL, depending on application, makes a
huge difference
• Concentrate on commonly used pages
19
20. Catch 22
• Hard coding SQL is the fastest solution
• No construction of SQL, no generation of
ActiveRecord associated classes
• If your DB changes, you have to update
SQL
‣ Keep SQL with models where possible
20
21. It ain’t pretty.. but it’s
fast
• Find by SQL
class order
def self.find_current_orders
find_by_sql(quot;SELECT orders.id, orders.created_at, customers.name
as customer_name, countries.name as country_name, order_statuses.name
as status_name FROM orders LEFT OUTER JOIN `customers` ON
`customers`.id = `orders`.customer_id LEFT OUTER JOIN `countries` ON
`countries`.id = `customers`.country_id LEFT OUTER JOIN
`order_statuses` ON `order_statuses`.id = `orders`.order_status_id
WHERE order_status_id < 100 ORDER BY order_statuses.sort_order
ASC,order_statuses.id ASC, orders.id DESCquot;)
end
end
• 28.90 req/s ( 5.49x faster )
21