I’m needing to do more analysis of my production logs, and at every turn I see “requires Hodel 3000 Complaint blah blah”. But that’s not what I have—I have standard logs! And thanks to buffered logging, they are quite parseable as they are.
http://github.com/cainlevy/timber
This fresh plugin provides simple objects to parse a production log into requests and extract whatever data you need. By default it extracts controller, method, and request time. But if you have special data in your log files that you want to analyze, it’s quite easy to add.
For example, I want to use Oink to analyze which requests most frequently bloat my worker thread sizes. Here’s a snip that adds Oink parsing to the Request class:
module Timber
module Plugins
module Oink
# create simple :pid attribute
attr_reader :pid
# create memory attribute, but return as integer
def memory
@_memory ||= @memory ? @memory.to_i : nil
end
end
end
# add to Request
Request.class_eval {include Plugins::Oink}
# add the regexp => vars pattern for the Oink log format
Request::PATTERNS[/Memory usage: (\d+) \| PID: (\d+)/] = [:memory, :pid]
endWith that, I can rewrite the Oink log parser to offload the log parsing to Timber and simplify it down to its own reporting logic:
module Oink
class MemoryUsageReporter < Base
def print(output)
output.puts "---- MEMORY THRESHOLD ----"
output.puts "THRESHOLD: #{@threshold/1024} MB\n"
output.puts "\n-- REQUESTS --\n" if @format == :verbose
@inputs.each do |input|
logfile = Timber::Log.new(input)
while request = logfile.shift
if pid = @pids[request.pid] and request.memory
delta = request.memory - pid[:last_memory_reading]
if delta > @threshold
@bad_actions[request.action] ||= 0
@bad_actions[request.action] += 1
@bad_requests.push(OinkedMemoryRequest.new(request.action, request.time, request.lines, delta))
if @format == :verbose
request.lines.each { |b| output.puts b }
output.puts "---------------------------------------------------------------------"
end
end
else
@pids[request.pid] = {:last_memory_reading => request.memory}
end
end
end
print_summary(output)
end
end
endI’ve been doing rolling deploys with a Mongrel cluster for over half a year now, and it works. Most of the time. Unless a Mongrel takes too long to shut down or something. Eh.
But then we all saw GitHub’s switch to Unicorn. I thought it sounded very cool, but maybe too much work to bother switching.
Oops.
Have a couple minutes? Good. Here you go …
Getting Started
Hit up the gem repository and gem install unicorn. It needs to compile, but that’s no big deal.
Now, shut down your script/server. Just go ahead and close that. Ready? Okay, now run unicorn_rails -p 3000.
Done. Good stuff. Was it good for you too?
Is It Different in Production?
Well sure. Let’s play around a little bit. It’s super simple to run Unicorn with a bunch of workers and send it commands just like you would in production.
First, create a basic config/unicorn.rb.
worker_processes 2
preload_app true
timeout 30
listen 3000
after_fork do |server, worker|
ActiveRecord::Base.establish_connection
endWant to know what all of that does? Check the docs, or the official example, or the Github sample. Go ahead, I’ll be here when you get back.
Now start your Unicorn server, except this time in the background and with this config file:
$ unicorn_rails -c config/unicorn.rb -D
$ pgrep -lf unicorn_rails
12113 unicorn_rails master -c config/unicorn.rb -D
12118 unicorn_rails worker[0] -c config/unicorn.rb -D
12119 unicorn_rails worker[1] -c config/unicorn.rb -D
$ cat tmp/pids/unicorn.pid
12113Next, we’re going to send it some commands. You do this by sending signals. You’ve used the kill command? Yeah, well, it doesn’t always kill things. Watch this:
$ kill -s TTOU 12113
$ pgrep -lf unicorn_rails
12113 unicorn_rails master -c config/unicorn.rb -D
12118 unicorn_rails worker[0] -c config/unicorn.rb -DWe just dynamically removed a worker from the pool. Neat, huh? Let’s add it back, plus one:
$ kill -s TTIN 12113
$ kill -s TTIN 12113
$ pgrep -lf unicorn_rails
12113 unicorn_rails master -c config/unicorn.rb -D
12118 unicorn_rails worker[0] -c config/unicorn.rb -D
12136 unicorn_rails worker[1] -c config/unicorn.rb -D
12137 unicorn_rails worker[2] -c config/unicorn.rb -DWhere’s My Seamless Deploy!
Nice. But I know what you’re really here for. You want to know what it’s like to hot-swap Unicorn for seamless deploys? First, add this to your config/unicorn.rb:
before_fork do |server, worker|
old_pid = RAILS_ROOT + '/tmp/pids/unicorn.pid.oldbin'
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# someone else did our job for us
end
end
endAnd for the win. Notice that I pgrep`d fast enough and caught it in transition.
$ kill -s USR2 12113
$ pgrep -lf unicorn_rails
12113 unicorn_rails master (old) -c config/unicorn.rb -D
12118 unicorn_rails worker[0] -c config/unicorn.rb -D
12136 unicorn_rails worker[1] -c config/unicorn.rb -D
12137 unicorn_rails worker[2] -c config/unicorn.rb -D
12239 /usr/bin/ruby1.8 /usr/bin/unicorn_rails -c config/unicorn-dev.rb -D
$ pgrep -lf unicorn_rails
12239 unicorn_rails master -c config/unicorn-dev.rb -D
12245 unicorn_rails worker[0] -c config/unicorn-dev.rb -D
12246 unicorn_rails worker[1] -c config/unicorn-dev.rb -DDone. Good stuff. Was it good for you, too?
Wow, is it complicated to format mailing address for international delivery!
Let’s do something about it.
I’ve started a library called Snail based on quality resources like Frank’s Compulsive Guide to Postal Addresses and the USPS International Mailing Manual. But it’s far from complete. So far it’s set up to format addresses from the United States to many other countries. That fills most of my needs at the moment. But what I’d love to see is:
- Support for other origin countries
- Support for more destination countries
- Validation tools based on each destination country. Just knowing which fields are required in which countries would be a great start!
- Form helpers to generate some of the best-practice address forms.
So! If this scratches an itch, then please consider helping out wherever you can.