WEBlog.Ruby

Are you still adding printf/puts calls and restarting your app to figure what went wrong? Sometimes, the problem is hard to reproduce, or you only discover it in production. You’ve got a process that exhibits the bug, but you didn’t run it under ruby-debug, so there’s no choice but kill it and reproduce after adding some code to inspect your program, right?

Sure not. Jamis Buck blogged about how to use GDB to inspect a live Ruby process, and showed how to get a stack-trace using Ruby’s C API and some GDB scripting:

(gdb) set $ary = (int)backtrace(-1)
(gdb) set $count = *($ary+8)
(gdb) set $index = 0
(gdb) while $index < $count
>  x/1s *((int)rb_ary_entry($ary, $index)+12)
>  set $index = $index + 1
>end

But it gets much easier than that. How about this:

(gdb) eval "caller"

or

(gdb) eval "local_variables"

Once you’ve opened that door, you get a full-powered Ruby interpreter inside GDB. Ruby’s introspection capabilities do the rest. Local variables, instance variables, classes, methods, Ruby threads, object counts… evil eval can bring us pretty far. You can find the scripts to turn GDB into a sort of IRB that can attach to running processes below.

  A synthetic example

For the sake of illustration, let’s take this trivial program:

def foo
  a = 1
  bar + a
end
def bar
  a = 2
  b = 2
  baz + 1
end
def baz
  c = 232
  c + gets.to_i
end
puts "foo returned #{foo}"

$ gdb ruby 
GNU gdb 6.1-debian
...
(gdb) attach 13658
Attaching to program: /home/batsman/usr/bin/ruby, process 13525
...
0xa7ec526e in read () from /lib/tls/libc.so.6
(gdb) 

I don’t want to pollute gdb with lots of commands, so I’ve put the Ruby magic in a separate script that can be loaded with

(gdb) session-ruby

(I also have a session-asm that imports a number of commands useful for low-level inspection.)

At this point, I could just

 (gdb) eval "caller"

but that would #p() the caller’s result in the stdout from the process being examined. So I first redirect $stdout with another command:

 (gdb) redirect_stdout
 $1 = 2
 (gdb) eval "caller"

This way, the result from eval is put in /tmp/ruby-debug.PID, and I get:

 $ tail /tmp/ruby-debug.13658 
 ["foo.rb:16:in `baz'", "foo.rb:11:in `bar'", "foo.rb:5:in `foo'",
 "foo.rb:19"]

  Basic introspection

The stacktrace tells me where I am, but there’s more much information available:

 (gdb) rb_object_counts

   220  String
   183  Class
    16  Module
    12  Array
     5  Float
     3  Bignum
     3  IO
     3  Object
     2  File
     1  Proc
     1  SystemStackError
     1  Binding
     1  fatal
     1  Thread
     1  ThreadGroup
     1  NoMemoryError
     1  Hash

(gdb) local_variables

["c"]

The last result might look surprising. How come there’s a (Ruby) local variable, if we’re running a method implemented in C (rb_f_gets)? It seems ruby doesn’t get out of its way to overwrite the (Ruby) frame info when it calls a C method, so the data you get corresponds to the enclosing Ruby method.

  Running until the end of the current method (aka “finish”)

I want to read Ruby’s sources, which order is best?

I’ve been answering to that question a few times a year, sometimes on ruby-talk, and as of late responding to private emails. The last time I took the extra effort to draft a self-study guide to ruby’s internals. Here’s a reformatted version.

I just wrote a small script that uses introspection to tell you the methods defined in a file plus their argument names and default values. It’s a quick hack, but it works surprisingly well.

UPDATE: works with more .rb files now (interface also improved somewhat).

Here are some examples:

   $ method_args csv 
   CSV::Cell#initialize (data = "", is_null = false)
   CSV::Cell#data ()
   CSV.open (path, mode, fs = nil, rs = nil)
   CSV.foreach (path, rs = nil)
   CSV.read (path, length = nil, offset = nil)
   [...]

method_args.rb can detect arguments with default values and “splat args”:

   $ method_args benchmark
   Benchmark#benchmark (caption = "", label_width = nil, fmtstr = nil, *labels)
   Benchmark#bm (label_width = 0, *labels)
   Benchmark#bmbm (width = 0)
   Benchmark#measure (label = "")
   [...]

The nice thing is that method_args.rb actually requires your library code and will take into account its dependencies too. So it’ll tell you all the methods you’d pull into your runtime if you required some file.

At the time being, all this script does is writing the methods it found on stdout, but it could for instance send them to another application through a socket to provide fairly accurate intellisense information, or generate rich tags, or…

Ruby 1.8.5 has been out for a couple weeks and broke Binding.of_caller and Rail’s breakpointer. I’d promised I’d provide a workaround, and I finally wrapped it up.

I’ve just released call_stack: backtrace data and 1.8.5-safe breakpoint/Binding.of_caller, which provides an alternative Binding.of_caller implementation and some additional functionality to obtain backtrace information (the “call stack”). You can override a pre-existent binding_of_caller.rb by loading breakpoint185.rb:

 ruby -rbreakpoint185 myapp.rb

Here’s how you’d use it with Rails’ breakpointer:

1. put the breakpoint calls in the code you’re inspecting, as usual
2. start the “breakpointer” as usual with script/breakpointer
3. run the application with ruby -rbreakpoint185 script/server

  RubyGems caveats

If you’ve installed call_stack with RubyGems, -rbreakpoint185 will not work (RubyGems cannot handle -rlib options passed to ruby yet), and you’ll have to do something like:

 ruby -e "require 'breakpoint185'; load 'script/server'"

(assuming you’ve set RUBYOPT=-rubygems, otherwise prepend it to the above line or add require ‘rubygems’)

matz just released Ruby 1.8.5.

It took a bit longer than originally planned but we can hope it will have been worth it 🙂

I have written a summary of the changes in 1.8.5 similar to the one I maintain for Ruby 1.9; the page is editable and corrections/additions are welcome.

You might have heard of Florian Groß’ Binding.of_caller before. More probably, you might have used his breakpoint library (yes, the one included in Rails, which can be used with script/breakpointer). If you read ruby-core a few months ago, you’ll know that it only worked thanks to a bug which is being fixed in Ruby 1.8.5. Which means that breakpoint will be breaking in under a week time, since matz is releasing 1.8.5 next Friday or Saturday at the latest*1.

Fortunately, I’d kept an unreleased rcov branch with some 500 lines of C to implement a better Kernel#caller method, which I had discarded at first because there was a much easier way to proceed when bindings were not needed.

So, I have the functionality that will keep breakpoint alive after the arrival of 1.8.5, but I need some help to refine the interface before it can be released; better get the naming right the first time, and this is something several brains tend to do better than one.

This is what it looks like currently:

require 'binding_n'
def a; _a = 1; b end
def b; _b = 2; c end
def c; _c = 3; d end
def d; _d = 4; send(:e) end
def e
  levels = binding_n(10)
  levels.each do |klass, id, file, line, binding, lang|
    puts "=" * 80
    p [klass, id, file, line, lang]
    next unless binding
    p eval("local_variables", binding).map{|x| [x, eval(x, binding)]}
  end
end
def x
  Kernel.install_binding_n_hook
  a
  Kernel.remove_binding_n_hook
end
x

New stuff added to the ruby-wmii community page (yes, this is a wiki, *hint*) and yet another nail in WIMP’s (Windows, Icons, Menus and Pointing) coffin…

Nathan Howell emailed me that his ruby-wmii plugins are ready for more exposure. I’ve been using his ssh magic since he showed it to me about one month ago, and Nathan is to be thanked for the generalized program lists his plugins rely on (included in ruby-wmii 0.3.1: bookmark manager, generalized menus, view history…), and several aspects of the plugin system.

This is the functionality you’ll find in his darcs repository:

• menu-based music controller supporting:
  • totem
  • banshee
  • rhythmbox
  • beep-media-player
  • mpc/mpd
  • squeezebox
• support for .desktop files from Gnome, KDE, ROX, XFCE in the program menu
• listing of connectable ssh hosts in the program menu
• pushing/pulling selected client from/to current view

  Install

I ran into a tricky bug today. I was working on an extension that uses an event_hook and needed quite some time to figure this out:

require 'binding_n'
def e
  l = eval("local_variables", binding)
  puts "The array: #{l.inspect}"
  l.map do |str|
    puts "PROCESSING #{str.inspect}"
    [str, eval(str, binding)]
  end
end
Kernel.install_binding_n_hook
e
Kernel.remove_binding_n_hook

The block was evaluated twice, even though the array had but one element:

The array: ["l"]
PROCESSING "l"
PROCESSING nil
t.rb:7:in `eval': can't convert nil into String (TypeError)
        from t.rb:7:in `e'
        from t.rb:5:in `e'
        from t.rb:12

I finally tracked it down to this function, which was being executed on RUBY_EVENT_(C)_RETURN events:

static inline code_site_t *
callsite_stack_pop(callsite_stack_t *stack)
{
    if (stack->ptr == stack->start) {
        return stack->ptr;
    }
    rb_hash_delete(additional_gc_roots, (stack->ptr-1)->binding); 
    return --stack->ptr;
}

Can you spot the culprit?

  Methods taking a block and implemented in C, at risk

If the key is not found, rb_hash_delete will yield it to the current block, if present. But which block is that? It’s harder to see in C…

That would be the block passed to the current method, i.e. the one you rb_define_method()’d. Or, in this particular case, the method we were returning from (remember this was being executed in an event_hook).

This means that you have to be careful whenever you use a function from Ruby’s C API that admits a block and you do not want one passed. The fix looks like this:

I just released ruby-wmii 0.3.1. If you didn’t know it, it’s a Ruby script plus associated plugins to control the wmii window manager.

The most important (and heaviest) change in 0.3.1 is the inclusion of the bookmark manager I wrote about earlier. It can import and sync your bookmarks against del.icio.us, is keypress by keypress the most efficient bookmark manager out there, and allows complex queries. It supports regular expressions on both titles and URLs, progressive refining, temporal expressions… For instance, you can open all the articles with the redhanded tag, including ‘wmii’ in either the title or the URL, bookmarked/visited in the last 2 months with

 :redhanded wmii ~d <2m
 !o

If you already had ruby-wmii installed, you can enable the bookmark manager by adding something like this to your ~/.wmii-3/wmiirc-config.rb:

from "standard" do
  use_binding "bookmark"      # , "MODKEY-Shift-b" # uncomment and change to
  use_binding "bookmark-open" # , "MODKEY-b"       # override the default keys
end
plugin_config["standard:bookmark"]["del.icio.us-user"] = 'myusername'
plugin_config["standard:bookmark"]["del.icio.us-password"] = 'mypass'

I’ve also added recently an experimental binding to move across the view history, using MODKEY-plus (forward) and MODKEY-minus (backwards) by default. I’ve been using it for a few days and it seems to complement MODKEY-r (move to previous view, cycles over the last 2 views) nicely. Here’s the incantation:

from "standard" do
    use_binding "history-move-forward"
    use_binding "history-move-back"
end

ruby-wmii 0.3.1 also includes a few additional features and a number of bugfixes.

  Community

I’ve created a couple editable nodes on eigenclass.org you can use to find and share configurations, plugins, tips, etc:

• ruby-wmii configuration and plugins ("community"): feel free to show your configuration, point to your cool plugins or comment about ruby-wmii
• ruby-wmii FAQ

  Download

The latest version of ruby-wmii is available here.

  Changes since 0.3.0 (2006-07-04)

You might have realized that eigenclass.org has been changing subtly as of late. In addition to the cosmetic modifications, I’ve upgraded to hiki 0.8.6 and rewritten my hiki hacks as self-contained plugins.

The last one I’ve been working on allows you to enable modifications to a set of nodes without opening up all of the wiki: creation of new nodes is disabled, and some operations are restricted. This is what the admin sees when editing a page:

hiki already ships with a plugin that allows you to restrict modification so that only authenticated users can edit the wiki (and also to freeze some pages), but my code allows you to open specific pages to everybody, while keeping the rest of the wiki frozen*1.

I’ve created a few “open” nodes, enabling modifications to them:

• rcov FAQ for questions about rcov
• ruby-wmii configuration and plugins ("community") where you can contribute your snippets/plugins for ruby-wmii: Ruby configuration/scripting for the wmii window manager
• ruby-wmii FAQ for questions regarding ruby-wmii

Larger parts of eigenclass.org will become open if spam doesn’t get too annoying. I have several ideas to fight it, were it needed.

I’ll release my hiki plugins in a while (which I sort of promised quite a long time ago), but here’s bliki.rb, so you can see how easily hiki can be extended: