Posted by Lance Ivy
Wed, 28 Nov 2007 00:42:00 GMT
I’ve been on a benchmarking blogging spree lately. Creating BenchmarkForRails (B4R) with its non-invasive benchmarking technique has made it very easy to play around with timing of key Rails methods and develop a better picture of the true costs. The results have been amazing.
First of all, apologies to those of you still running Rails 1.2.x. B4R’s non-invasive technique only works well in Rails 2.0 due to some very nice refactoring, especially in Dispatcher and ActionController::Filters.
Blind Spots
Take a look at these benchmarks for the exact same request, as seen by Rails and B4R. This was done in development mode on an unoptimized page, so don’t worry too much about the times, ok? It’s the differences that are important.
Rails
Completed in 0.85246 (1 reqs/sec)
Rendering: 0.48872 (57%)
DB: 0.02561 (3%)
BenchmarkForRails
- [0.9532] GET /plans ----------------------------
0.8528 processing action
0.5015 rendering
0.2533 activerecord find
0.0461 development mode
0.0073 before filters
0.0046 session management
0.0000 after filters
------------------------------ BenchmarkForRails -
The first line is from Rails’ built-in benchmarks. The following lines are from B4R. Please note that the benchmarks in B4R are not meant to add up, which is no different than Rails’ benchmarks.
Completion Time
Rails claims that the completion time was 0.85 seconds, and B4R claims that the completion time was 0.95. That’s a difference of 0.1 seconds. Big deal? It can be! But where’s the extra time coming from? Well, notice that B4R has an entry for “processing action”, and notice that this time (0.85) matches Rails’ declared completion time.
The problem is that there’s more machinery involved in handling a request then just processing the controller action. Here’s a quick list of some potentially significant things Rails is missing in its reported completion time:
- routing recognition
- before filters
- after filters
- session management
But these are all very real costs that you can (and should) strive to optimize. They’re costs you should know about. And they’re costs that definitely affect your requests-per-second.
DB Access
Rails’ idea of query costs is how much time was spent waiting for your database to answer, and so it comes back with an incredibly low 0.025 seconds. With numbers like that, why would anyone optimize queries! Why would anyone bother with eager-loading?
Pure database access time only matters for big queries. But if you make just ten small queries, the true costs are somewhere else: ActiveRecord’s query construction.
Now go back to the sample benchmarks above. Notice the “activerecord find” entry? That’s 0.25 seconds. No, it didn’t misplace the decimal – the real cost of executing queries on the page was 10x what Rails reported. Why? Because the database is faster than ActiveRecord, obviously.
So yeah, there’s some real motivation for eager loading, yeah?
Tags benchmarkforrails, benchmarking | 5 comments
Posted by Lance Ivy
Tue, 27 Nov 2007 00:30:00 GMT
My current client, Charlie of PearBudget fame, is actually expressing interest in learning more about Rails backend code. How cool is that? I started writing an introductory primer for him about how to approach backend optimization, then realized others may enjoy reading it as well.
The techniques and tools mentioned here have been discussed in detail all over the web, so rather than provide detailed pointers this article paints a broad picture about how all the approaches fit together.
Guided vs Intuitive Optimization
Early in the development process, efficient optimization is driven by intuition. The developer isn’t studying benchmarks, he’s just trying to clean up code whenever it’s worthwhile and doesn’t interfere with development. The moment that optimization itself becomes the goal, it should be driven by reliable and reproducible numbers, otherwise you risk spending a day optimizing something that has no discernable effect on the application’s performance.
Finding the Problem
Reliable and reproducible numbers come in three flavors described below. The numbers should always be collected in production mode, though. In development Rails reloads certain files like your models and controllers, which is awesome, but the time spent reloading will throw off your benchmarks. It’s easy to run in production mode on your own machine by first opening your config/database.yml and copying the development configuration settings to the production configuration settings (don’t check this into Subversion!), then running `script/server -e production`.
Average response times
There’s not much to learn from any single response time, due to the wide variation. To determine a reliable time for any given page you need to use average the response times over a number of requests. Generic tools for this include Apache AB, Httperf, and RailsBench. Note that these tools can themselves slam your processor, so they’re best when not run from the machine processing the requests.
Traffic patterns
A page may on average respond slowly, but unless it’s being run with any significant frequency that doesn’t matter. A page that’s slow once per month per user just isn’t worth optimizing until the common pages are ticking along smoothly. Traffic patterns can be determined either through log analysis or through web analytics (Google, Overture, etc.). A common log analysis tool for Rails logs is Rails Analyzer, although it can be difficult to use because it requires your log to be in a certain format (Hodel3000CompliantLogger helps here).
Per-request times
Once you’ve identified a slow page, it’s time to determine why it’s slow. You can time the duration of any given request and even break that request down into component times. This is what the benchmarks in the Rails log attempt to do, though I believe I’ve improved on them with the BenchmarkForRails plugin.
Benchmarks on this level are mostly good for determining ratios. Two identical requests will never have the same exact times, but they will have generally consistent ratios between the components. For example, two requests for the home page may take 0.2 and 0.3 seconds (a significant difference), but the important take-home lesson from these times might be that rendering is 50% of each request.
Once you’ve identified a component that appears slow, you can start digging into the real cause. This is where in-depth tools like RubyProf are useful. Profilers are too slow to leave on for normal operation, and they have a learning curve for deciphering the output, but they can really open your eyes to unknown speed hits in the codebase. Note that these profilers will not help you profile database query performance.
Solving the Problem
Once you’ve determined what exactly is slow, it’s time to determine what can be done about it. Sometimes simply restructuring a code block can be all that’s needed. But in web applications it usually just comes down to database usage.
Adding Database Indexes
If you find a slow query, the first thing to check is that it’s well-written. Assuming that’s the case, you should immediately ask yourself whether you can index a database column (or set of columns) to aid the database in searching through the massive amounts of data your application has picked up. A simple database index can easily cut a query’s time by 90% or more.
Caching
Are you making the same query or running a calculation multiple times per page? Cache it. But don’t break encapsulation – when possible, cache it without the application really even knowing. For example, make sure the methods on your fat models embrace the “@var ||= calculation” pattern.
Query Merging
Efficient queries can become very inefficient in bulk. This is when it’s time to consider merging the queries, whether through eager-loading or by querying a set of records and sorting them into groups afterwards (instead of querying them in groups). The reason is that there’s some overhead in every database query, ranging from the costs of constructing the query to the costs of communicating with the database to the database’s own overhead costs.
Note that it’s entirely possible to take this too far. When in doubt, check the benchmarks.
Denormalization
In the end, your queries may just be performing as well as they can given your pristine database schema. But if you find some massive queries that join 75% of the tables in your database, you might consider denormalizing. Rails makes denormalization manageable with well-placed callbacks on your ActiveRecord models, and careful denormalization can create shortcuts between key data objects and really cut down the query complexity.
Better Hardware
When all else fails, upgrade your hosting. Add more memory. Upgrade to a cluster. These are perfectly valid ways to make performance problem go away until you have more time to focus on them.
Optimal Optimization
The final thing to realize is that your application will never be performing as fast as it could. Accept it! There will always be something you can optimize, but it won’t always be worth it. Careful application of the above techniques (or cascade of techniques) can help you keep your optimization passes efficient and focused.
Tags benchmarking, optimization | no comments
Posted by Lance Ivy
Fri, 23 Nov 2007 22:00:00 GMT
There’s a new Rails benchmarking plugin in town – BenchmarkForRails. Why? Because with Rails’ benchmarks I never knew how long the request was really taking, including filters and sessions and all that jazz. And because building queries can take longer than executing them, which makes Rails’ DB benchmark somewhat, um, misleading.
This plugin introduces benchmarking for:
- the entire process lifecycle
- session initialization
- before filters (as a group)
- after filters (as a group)
- time spent in ActiveRecord::Base.find
- time spent in a respond_to block
Furthermore, it makes adding more benchmarks incredibly easy. Just follow these simple steps:
- Find the method you want to benchmark. Note the class/module, the method name, and whether it’s an instance or class method. Let’s suppose you want to benchmark the link_to helper in ActionView::Helpers::UrlHelper.
- In your config somewhere (e.g. initializers/custom_benchmarks.rb), tell BenchmarkForRails to watch that method, like so:
BenchmarkForRails.watch(
"link_to calls",
ActionView::Helpers::UrlHelper,
:link_to
)
And some sample output, from the log file:
- [0.0489] GET SiteController#index ----
0.0184 session startup
0.0013 activerecord find
0.0001 before filters
0.0000 after filters
0.0000 respond_to block
-------------------- BenchmarkForRails -
Errata:
- This has only been tested with Rails 2.0 RC1. I really really doubt it works in Rails 1.2.
- Subversion URL is http://benchmarkforrails.googlecode.com/svn/trunk
Posted in Ruby on Rails, Open Source, Releases | Tags benchmarkforrails, benchmarking, rubyonrails | 9 comments
Posted by Lance Ivy
Wed, 14 Nov 2007 03:19:00 GMT
“You can’t compare Rails and PHP, because one is a framework and the other is a language.”
Ok, sure. Let’s assume that what makes Rails a framework is that it tells you how to structure a project. Yes, Rails is a framework (it has structure) and PHP is a language (egads, it has no structure!).
We can still compare them.
For starters, because you can do web development in PHP as much as you can in Rails. It damages the wrists and the psyche, yes, but you can do it. PHP supports sessions, cookies, parameters, and HTML generation. It gleefully mixes all that support in with the core language, for better and worse.
But more to the point, you can still compare them because within the context of web development it’s not useful to compare Ruby vs PHP but it is useful to compare Rails vs PHP. How? By comparing one vs many in a single stroke. When someone compares Rails and PHP, I think what they’re really doing is comparing Rails against anything that could come out of PHP.
Rubyfied, the comparison expands like this:
PHP::Frameworks.any? {|framework| framework >= Ruby::Frameworks::Rails}
=> false
Is that really fair? Isn’t there a lot of detail lost making that kind of broad sweeping comparison. Yes! But is anything relevant lost? That’s a much harder argument. Go ahead and try, I’ll listen, but when it comes right down to it I’m prepared to short-circuit any such argument with one claim:
Anything built on PHP will have to fight that taint. It’s just not worth the effort anymore.
Posted in Ruby on Rails | Tags php, rubyonrails | 2 comments
Posted by Lance Ivy
Mon, 12 Nov 2007 00:30:00 GMT
I added support in RecordSelect trunk for HTML record description and used it to spice up the demo a bit. Then I figured hey, why not put it online where people can actually see it in action?
Tada! Online Demo.
Posted in Ruby on Rails, Open Source | Tags recordselect | no comments
Posted by Lance Ivy
Sat, 10 Nov 2007 03:02:00 GMT
It’s time to tag this puppy – RecordSelect is officially 1.0 today. It’s now more stable and cross-browser than ever before, thanks to reports, patches, ideas, and a little pushing from the inimitable Ed Moss (among others).
See the full list of changes
for more details. Curious how people are using it? Come ask real users at our little mailing list.
Posted in Ruby on Rails, Open Source, Releases, JavaScript | Tags recordselect
Subscribe