RDFWeb and Friend of a Friend (FOAF)

FOAFbot and Gnome Dashboard

Edd Dumbill writes with news of his latest work on FOAFbot, the chat-based FOAF aggregator.

FOAFbot is now based on the 'twisted' framework, opening up possibilities for various new interfaces to the data beyond the current IRC one. This is interesting as it makes it easier for people to adapt FOAFbot, eg. for instance messenger or HTTP interfaces, or to hook it up to alternate front-ends such as foafnaut. Edd's weblog provides more details on all this, including links to source code and a teaser screenshot of FOAFbot data showing up in Gnome Dashboard. Nice work! :)

FOAF Checker utility for PGP-signed FOAF profiles

Ken MacLeod has announced a new tool, 'FOAF Check', which reads a FOAF file, checks its PGP signature and extracts portions of the RDF, eg. for inclusion in weblog comments (via. Movable Type cookies etc.).

This approach provides a way to avoid re-entering the same data over and again across multiple sites - you just point them at your FOAF document and let them read what you've already said about yourself. Ken's writeup points to previous work on FOAF/PGP-based decentralised profiles.

This is really cool, basically. Looking at the detail it does suggest we need to do a little more work on the FOAF vocabulary for clarifying which of several people mentioned in any given FOAF file is the topic of that file. There are currently several ways we could indicate this; one needs to be picked and documented more clearly in the FOAF spec. It also isn't immediately clear (to me!) how we get from knowing that a FOAF file validates against some PGP key to knowing that the PGP identity is the one that our application wants it to be. But it's a step in the right direction...

Also the combination of foafbot and tools like foaf-check provide good reason for people to actually start using PGP (or GNU Privacy Guard (GPG), the Free GNU version), That can only be a good thing.

One other topic this throws up: is it reasonable to expect people to remember and quote their FOAF URLs? Or should we ask for their homepage, and look for a LINK REL in their homepage markup pointing to their FOAF data?

Spring v1.3.1: a MacOS X desktop tool with FOAF support

UserCreations have just announced the release of Spring v1.3.1 for MacOS X.

Spring provides a canvas-style desktop interface, replacing the ageing 'documents and folders' metaphor with a UI based on several content types (people, places, products, books, music, ...), with appropriate kinds of action associated with each type of thing. So you might invite a person to an event, draw a map of a location, etc.

Version 1.3 added 'drag and drop' FOAF support:

Friend of a Friend (FOAF) format is an increasingly common web format for people. If you see a FOAF button on a web page, drag it in to Spring

This support is improved in v1.3.1: a "Refesh From Web" feature allowing FOAF-derived person entries in Spring to pick up the changes from remote FOAF files.

So needless to say, I've been experimenting with Spring on my Mac laptop, and it looks pretty interesting and a hint at what desktops might evolve into. I really like the way it associates actions with entity-types in a way that takes advantage of remote Web services. That, coupled with the "refresh from Web" approach to using remote RDF/XML descriptions (FOAF etc.) is quite a compelling combination.

So in practice, I found the FOAF support worked with some but not all FOAF files from the public FOAFBulletinBoard. I'm not sure what the issue is, perhaps something to do with the mime-types these documents are served up as, or perhaps Spring has certain expectations for the syntactic form of a FOAF document. If others are trying out the FOAF support, perhaps leave comments here or at the UserCreations article, perhaps we can figure out how to help.

3 degrees

From the oops-I-forgot-to-post-this drafts folder: Microsoft have a new thing called three degrees that may be of interest to FOAF developers and others working on standards-based collaborative tools. Does anyone know whether it's a closed box, or whether there is some prospect of interop with other systems?

Weblog discovery and filtering with RDF - CPAN demo

This is a quick writeup of an application of RDF query tools to the problems of weblog discovery and filtering. It shows off RDF's ability to merge XML-encoded data from multiple sources, answering queries which couldn't be answered by considering each source separately.

Imagine you are looking for Weblog URLs (and perhaps their associated RSS feeds) of people who have written Perl software that is in CPAN. At our disposal are two data sources: a data dump listing the SHA1 hashes of CPAN author email addresses. For each author, we have an @cpan address, plus one of their normal addresses (cpan2foaf.pl). A second RDF/XML source of FOAF data is a document listing other information about several people who happen to write for CPAN. That file is a stand-in for their real FOAF files, which would typically be collected from each of their sites individually. For simplicity, we show that data as a single document.

The first datasource mentions several hundred people who contribute to CPAN, and enough information to identify them (by hash of email address). The second datasource contains some of the same identifiers (their hashed home email address) plus, more interestingly, information about their names, homepages, weblogs and suchlike. How can we get answers to questions like "find me weblog addresses for CPAN contributors?".

So... we load the RDF up into an (SQL-backed) RDF database, make sure to merge the data (more on which another time), and it is available for query.

Here's the SquishSQL query we send to the RDF software:

SELECT ?name, ?homepage, ?weblog, ?x, ?feed, 
WHERE  (foaf::name ?x ?name)
               (dc::contributor ?x http://www.cpan.org/)
               (foaf::homepage ?x ?homepage)
               (foaf::weblog ?x ?weblog)
               (foaf::rssfeed ?weblog ?feed)
USING 	dc for http://purl.org/dc/elements/1.1/
               foaf for http://xmlns.com/foaf/0.1/

This RDF query language is called Squish as it is loosly SQL-ish in the way it is used. We send the database a question asking for a selection of fields, and we get back a table of results. Here are name/homepage/weblog 'hits' corresponding to the rows returned by the query system:

Earle Martin (http://downlode.org/ weblog: http://downlode.org/blog.pl)
Jo Walsh (http://www.zooleika.org.uk weblog: http://www.zooleika.org.uk/blog.html)
Norm Walsh (http://nwalsh.com/ weblog: http://norman.walsh.name/)

Here is a simple Ruby lookup script which extracts the tabular result set from the database. The query extracts information (in this case weblog address etc) about people who contribute to CPAN and are known to the database.

Bugs and disclaimers:

I used the dc:contributor property incorrectly in this demo. It should relate a document to one of its contributors. I used it backwards, and slightly incorrectly, by relating a person to a homepage of a project they contribute to. I should use foaf:project instead.

The demo links are also a bit inconsistent, in that I added in a clause to the query which also asks for the RSS feed of a weblog, but didn't commit the new resultset output to the website yet. I'll try to get that fixed.

This brief writeup assumes perhaps a bit too much background knowledge for those new to RDF to follow in detail, also things like SHA1 'hashing' could do with elaboration. Still I hope the basic approach is clear: RDF allows us to merge disparate datasources and get back answers that depend on their combination. And that we can use that for filtering and discovery of weblogs, given a description of some weblogs and their authors.

More Japanese FOAF

Some interesting work with FOAF and XSLT documented in a Japanese weblog. The links to foaf.xml, foafstyle.xsl and foaf.html show the generation of an HTML Web page from RDF/XML source using an XSLT transformation. The article also references another foaf.xsl, another Weblog post about FOAF and a Japanese Wiki page about FOAF. So the relatively new foaf:weblog property seem to be catching on, presumably since it was covered in Masahide Kanzaki's Introduction to FOAF.

Apologies for the shallow writeup, I wish I could read these pages! I couldn't even manage copy a foaf:name my browser into this weblog entry... Feel free to add more details using the 'comments' feature if you can read Japanese.

FOAF 4 IndyMedia

IndyMedia is a project well worth supporting. The mainstream media industry is great and all, but after a while, it gets boring being lied to. IndyMedia is a brave effort towards a more inclusive approach to media access, although currently it lacks much by way of trust and 'provenance' infrastructure. Slashdot, the geek news site, has more filtering and trust machinery than most current IndyMedia sites. IndyMedia risks being drowned out by noise and spam...

So I've been experimenting with a proposal for FOAF augmented RSS feeds in the Mir content management system which now powers the UK IMC. It raises issues about privacy, identity, and (most immediately) technical concerns about the proper use of 'dc:creator' in RSS 1.0 feeds.This is a modest step towards some machinery for filtering IndyMedia news through 'web of trust' machinery. Not censorship, just a way of keeping track of who said what, and what other things those sayers have said...

FOAF MT plugin(s)

I'm experimenting with Eric Sigler's
FOAF plugin software for use with Six Apart's Movable Type. In passing, I've upgraded the rdfweb.org MT installation and installed Ben Trott's OpenPGP and XML::FOAF Perl modules from CPAN. This in turn uses
Ginger Alliance's RDF::Core module (which in turn uses XML::Parser, Expat etc...).

So after installing this stack of interesting software on rdfweb.org, I'm back reading the manual for Movable Type and musing on how the FOAF MT plugin relates to MT's commenting system and ideas for using FOAF for login information, distributed identity description and suchlike. And also, to be honest, geting used to the fact that there's a Perl module for FOAF now. Interesting times...

So anyway, I'll be fiddling around with the comments system on this site, Advance apologies if I break anything. At this point, all I've done is install the software, I'll be experimenting with it in my spare time over the next week or few.

Nearby in the Web: efforts to use MT plugins and FOAF at Burning Man

FOAFBot lives!

FOAFBot, Edd Dumbill's RDF-based IRC bot, is back!. Not content with writing readable introductions to FOAF (also in Japanese, Chinese), Edd created one of the most interesting tools that consume FOAF. FOAFBot hangs out in an IRC channel and answers questions about people based on data it found by indexing the Web of interlinked FOAF documents.

FOAFBot is particularly interesting because it keeps track of 'who said what' in the RDF data it collects, even using PGP signatures of the data files where available. This also explains why FOAFBot was offline for a while. Edd's second article on FOAF explained the way in which the original FOAFBot kept track of this 'provenance' or 'attribution' metadata. FOAFBot is written in Python, on top of Dave Beckett's Redland RDF toolkit.

Since FOAFBot was created, Redland and its parser, Raptor, has improved in many ways; in particular Dave added native support for provenance-tracking, via a 'contexts' mechanism. This allows each RDF statement in a database to be tagged with its origins, and allowed Edd to rewrite FOAFBot in a faster and sleeker form. Nice work all round :)

Furthermore, since FOAFBot's data-structures now use Redland's built-in way of representing contexts, FOAFBot databases can now be accessed from all languages that Redland has bindings to (Python, Java, Perl, Ruby, ...). This means that a developer can install Redland and FOAFBot, create an index of FOAF data by running FOAFBot as a harvester ('scutter'), and then access that database from, say, Perl to create alternate views of the data, eg in HTML or SVG.

You can read more on Edd's FOAFBot page or stop by the #foaf IRC channel to see it in action (or talk to other FOAF folk if the bot is offline).

Web passwords in FOAF?

Here's a quick scribbled description of an application of FOAF I've been thinking about lately. I am forever forgetting the passwords for various Web sites; usually those of my friends (photos etc), rather than huge dot-com sites. I'd like to make it easy for such sites to offer password-protected access to their content, without my having to remember loads of different passwords.

So I'm wondering how feasible it would be to do something like the following: in my FOAF file, or another RDF/XML document linked from it, list a bunch of 'account descriptions' giving my username and (in some cases) crypted passwords. Some of these could be generic accounts, eg 'generic friends photo login'. FOAF-aware sites could use this if they want to give me access but defer password management to external systems. I'd update/edit/delete account descriptions, including passwords, and sites would read my FOAF file regularly to keep up to date.

The bare bones of this could be implemented with relatively little scripting, but it raises some issues that need careful thought. First up, it isn't super-wise to have a single username/password used on loads of sites, especially if it is sent in cleartext HTTP, or if you're not careful about which sites you send it to. Secondly,we'd need to use something like PGP's identity-assurance mechanisms, otherwise my friends might accidentally use evil-danbri-impersonator.rdf to check passwords, and allow the wrong users to see their content. So I might PGP-sign a chunk of RDF that says 'this is danbri@rdfweb.org's low-security photo-website generic account'.

There's more to think and write and test on all this, but I just wanted to scribble the basic idea. I suspect the next step is a prototype...

Learning the ropes

So I'm getting to grips with the Movable Type system. I've seen some nice templates on other sites that display blockquotes nicely. This is a test entry to see how blockquote looks.

Some stuff here.

Wonder how to quote markup...? eg. for XML/RDF examples...

All Your FOAF

The All Your FOAF toy indexing tool has been rewritten in Ruby. The original was a standalone short Perl script that showed how a Web of RDF documents can be explored using pretty simple code. The Ruby re-write uses the RubyRDF library, and has a nicer architecture.

See ayf.rb for the main code, myScutter.rb for an example crawler that uses it, and who.rb for a 25 line demo script that shows a custom crawler-based app (sample output).

This shows one of the attractions of Ruby as a scripting language. You can have one page hacks (like my original Perl script, ayf.pl) and migrate them smoothly into more reasonable chunks of OO-ish code. The result of rewriting this in Ruby (and using a proper RDF library) is that scripts such as who.rb are easier to create and extend, since all you have to do is write a snippet of code that gets called when each RDF document has been parsed.

This is of course still pretty basic. A better RDF Web indexer would need better facilities for controlling link-following behaviour, caching etc., and needs to be hooked up to persistent RDF storage systems. But hopefully it'll show the fundamental simplicity behind some of this stuff...