The All Your FOAF toy indexing tool has been rewritten in Ruby. The original was a standalone short Perl script that showed how a Web of RDF documents can be explored using pretty simple code. The Ruby re-write uses the RubyRDF library, and has a nicer architecture.
See ayf.rb for the main code, myScutter.rb for an example crawler that uses it, and who.rb for a 25 line demo script that shows a custom crawler-based app (sample output).
This shows one of the attractions of Ruby as a scripting language. You can have one page hacks (like my original Perl script, ayf.pl) and migrate them smoothly into more reasonable chunks of OO-ish code. The result of rewriting this in Ruby (and using a proper RDF library) is that scripts such as who.rb are easier to create and extend, since all you have to do is write a snippet of code that gets called when each RDF document has been parsed.
This is of course still pretty basic. A better RDF Web indexer would need better facilities for controlling link-following behaviour, caching etc., and needs to be hooked up to persistent RDF storage systems. But hopefully it'll show the fundamental simplicity behind some of this stuff...
Posted by danbri at December 12, 2002 01:54 PM