Introducing FastRI: faster RI docs for Ruby, across machines - one Ring to find them (via DRb + Rinda)

Never felt that ri is way too slow? It was never very fast, but it's become worse as of late, since it also searches the RI documentation in your RubyGems packages...

eban recently proposed a shell script to cache the results:

 #! /bin/sh
 
 cachedir=$HOME/.fri
 mkdir -p $cachedir
 args="$@"
 cachename=$cachedir/"$(echo -- $args | md5sum | cut -d' ' -f1)"
 if [ ! -e $cachename ]; then
   ri -T "$@" > $cachename
 fi
 ${PAGER-less} $cachename

It's handy for frequently consulted methods, but it doesn't make new lookups any faster.

I've written FastRI, a DRb-enabled, fast alternative to ri. It is *much* faster, and also allows you to access RI over DRb.

 $ time ri -T Sexp
 [...]
 real	0m0.901s
 user	0m0.752s
 sys	0m0.112s

vs.

 $ time fri Sexp
 [...]
 real	0m0.068s
 user	0m0.052s
 sys	0m0.000s

FastRI consists of a server (fastri-server) that provides the RI lookup service, and the client (fri) which will locate the server automatically and obtain the requested documentation via DRb.

Since fastri-server is running in the background, it will not have to search all your documentation directories on each request. This makes it much faster*1 than plain old ri.

FastRI uses a Rinda Ring to allow the server to be discovered automatically without needing to indicate the DRb URI manually. It can work also across machines, so you could run the server in one machine and use fri in several hosts across your network.

Getting it

FastRI has been released.

You can also access the darcs repository at http://eigenclass.org/repos/fastri/head .

Issue w/ some classes - Michael Irwin (2006-11-11 (Sat) 09:16:03)

If I do `fri Array` or `fri String`, I get the following error:

(druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `concat': can't convert nil into Array (TypeError)
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `merge_in'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/site_ruby/1.8/fastri/ri_index.rb:328:in `get_class'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/site_ruby/1.8/fastri/ri_index.rb:324:in `get_class'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/site_ruby/1.8/fastri/ri_service.rb:207:in `info'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/site_ruby/1.8/fastri/ri_service.rb:379:in `capture_stdout'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/site_ruby/1.8/fastri/ri_service.rb:206:in `info'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1512:in `perform'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `run'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1424:in `run'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1344:in `initialize'
       from (druby://127.0.0.1:37238) /usr/local/lib/ruby/1.8/drb/drb.rb:1624:in `start_service'
       from (druby://127.0.0.1:37238) /usr/local/bin/fastri-server:133
       from /usr/local/bin/fri:109

These are the only two that give this error that I've run into so far.

mfp 2006-11-11 (Sat) 10:49:46

Which version of ruby are you using? I think it's a bug in rdoc/ri solved in 1.8.5.

In my ri_descriptions.rb:

 97       else
 98         unless old.comment.nil? or old.comment.empty? then
 99           @comment << SM::Flow::RULE.new
100           @comment.concat old.comment 
101         end
102       end

Note that the call to concat happens in line 100, and is guarded by

 unless old.comment.nil?

I could rescue the exception in FastRI's code, but the real solution is getting the new ri_descriptions.rb (e.g. by installing ruby 1.8.5) or applying this patch.

Michael Irwin 2006-11-11 (Sat) 12:53:54

I'm using v1.8.4.

mfp 2006-11-11 (Sat) 14:47:34

That's it then; you can either apply that patch or reopen the class by adding this to fastri-server:

module ::RI
  class ModuleDescription
    remove_method :merge_in
    # merge in another class desscription into this one
    def merge_in(old)
      merge(@class_methods, old.class_methods)
      merge(@instance_methods, old.instance_methods)
      merge(@attributes, old.attributes)
      merge(@constants, old.constants)
      merge(@includes, old.includes)
      if @comment.nil? || @comment.empty?
        @comment = old.comment
      else
        unless old.comment.nil? or old.comment.empty? then
          @comment << SM::Flow::RULE.new
          @comment.concat old.comment
        end
      end
    end 
  end
end 

I'm probably adding that workaround to the next version.

RingServer - Eric Hodel (2006-11-01 (Wed) 15:47:56)

Why not use RingyDingy to wrap up all the RingServer work?

mfp 2006-11-01 (Wed) 18:05:23

hmmm all the RingServer work takes exactly 5 lines of code :) so I'd hesitate to add a hard dependency.

I think I'll probably change the code to use RingyDingy if it's available, falling back to the plain RingServer otherwise.

No Title - Anonymous (2006-11-01 (Wed) 08:48:04)

Hi Mauricio,

I just pulled down the code from darcs and running fastri-server I get:

~/tmp/fastri $ fastri-server                                 
/usr/lib/ruby/1.8/rinda/ring.rb:212:in `lookup_ring_any': RingNotFound (RuntimeError)
       from /usr/lib/ruby/1.8/rinda/ring.rb:109:in `finger'
       from /usr/lib/ruby/1.8/rinda/ring.rb:118:in `primary'
       from /usr/lib/ruby/1.8/rinda/ring.rb:241:in `provide'
       from /usr/bin/fastri-server:308

Any ideas what might be going wrong?

mfp 2006-11-01 (Wed) 09:27:04

It seems the Ring was bound to some interface which couldn't be reached by the RingProvider to publicize the FastRI service. Are you using IP6?

You can try to bind the ring to a different interface with

 fastri-server -s YOUR_IP

By default, it'll bind to 127.0.0.1.

Saimon 2006-11-01 (Wed) 09:50:06

Hi Mauricio,

No just plain IP4..

I get the same exception, running:

sudo fastri-server -s 192.168.1.5
sudo fastri-server -a 192.168.1.0/25 -s 192.168.1.5

(Linux 2.6.17)

Looking at the code it appears the RingServer is binding via a UDP socket. Shouldn't it be binding via a tcpip socket?

mfp 2006-11-01 (Wed) 12:14:55

AFAIK that's how it should work (the RingServer listens on a broadcast UDP addr, the RingFinger sends a datagram with the DRb URI where it is listening for a reply, and the RingServer contacts the client to notify it of its existence).

Do the sample DRb programs in Ruby's sources work for you (sample/ring_*)? Here's the simple RingServer ring_place.rb and an echo client/server ring_echo.rb. (ring_place.rb forks & runs as a daemon by default, don't forget to kill it if you want to run another ring server).

Saimon 2006-11-01 (Wed) 12:31:00

Yes those work:

~/tmp/fastri $ ruby ring_place.rb 
~/tmp/fastri $ ruby ring_echo.rb  
#<RingEcho:0xb7ab2c60 @name="druby://iris.olympos:49364">
druby://iris.olympos:49364: Hello, World

mfp 2006-11-01 (Wed) 12:49:45

Got an idea; what about this

 $ fastri-server -s 0.0.0.0

(some -a might be needed too).

If that doesn't work, I'll read rinda's code carefully to spot the difference (ring_echo.rb should be doing basically the same thing as RingProvider).

Saimon 2006-11-01 (Wed) 13:02:30

Nope, still doesn't work.

mfp 2006-11-01 (Wed) 18:00:24

:-| Could you ping me at mfp@acm.org? (include 'fastri' in the subject to bypass the spam filters) I'd like to give you some more code to test, if you don't mind.

mfp 2006-11-05 (Sun) 05:11:49

For the record (in case it helps somebody), we got it to work with either

 fastri-server    (own Ring server,  binding to 127.0.0.1)

or

external Ring server, for instance ring_place.rb from Ruby's sample/ dir + 
fastri-server -a iris.olympos -s iris.olympos

Thanks - James (2006-10-31 (Tue) 17:40:18)

Thanks! I hacked around with assorted local-server versions of in-memory ri, but kept getting distracted. This looks good.

No Title - Kent (2006-10-31 (Tue) 13:31:19)

Can you provide a gem version? PLEASE!

mfp 2006-10-31 (Tue) 16:30:31

There will be a RubyGems package soon.

I just wrote a drop-in replacement for ri-emacs.rb (which inspired FastRI); I'm now writing the vim counterpart. The actual release is not far in time.

gem docs - addsw (14-11-2006 (Mar) 02:45:27)

Hi,

I'm unable to view gem docs (both with ri and with fri).

My gem folders only contains rdoc/ subfolders.

mfp 2006-11-14 (Tue) 04:24:38

ri support was added to RubyGems in 0.9.0. Only gems installed with RubyGems >= 0.9.0 will have the required ri/ subdirectories. You can either:

• install RubyGems 0.9.0 and reinstall the gems whose docs you want to obtain, or
• generate the ri documentation manually by running

 rcov --ri -o appropriate_dir lib/

in the gem directory, where appropriate_dir is a dir "parallel to" the rdoc dir for that gem (e.g. $prefix/lib/ruby/gems/1.8/doc/somegem-0.1.1/ri)