RDFWeb and Friend of a Friend (FOAF)

W3C OWL language now a Candidate Recommendation

W3C's OWL language has just advanced to 'Candidate Recommendation' stage. OWL is used to publish and share sets of terms called ontologies; FOAF is one such ontology, as it uses OWL to define its terms in a machine-processable way.

The FOAF project relies on a number of W3C standards, including XML, RDF and the Web Ontology language, OWL. As OWL stabilises, we should benefit from an increasing number of tools that exploit the ability to reason about OWL-based vocabularies such as FOAF.

FOAF makes use of OWL in particular for describing when a certain kind of property (eg. 'homepage', 'mbox', 'dnaChecksum', 'mbox_sha1sum', 'jabberID' etc.) is uniquely identifying. This mechanism underpins the working of many FOAF aggregators and toolkits. FOAF benefits from W3C's work on OWL, since it allows us to adopt standard conventions shared across the Semantic Web community, rather than invent conventions that would only be understood by FOAF tools. Consequently FOAF implementations can make more use of generic RDF-based software, allowing FOAF developers to concentrate on FOAF-related functionality rather than on basic data-merging and storage techniques. This is particularly important to us as FOAF is designed to be freely mixed with other Semantic Web data (eg. geographic, calendar, taxonomy, bibliographic etc.).

See the W3C news item, press release and FAQ for more information on OWL and its relationship to other W3C technologies.

Updated FOAF specification

We've just shipped a major update to the FOAF Specification. The spec now provides a lot more detail on the terms ("classes" and "properties") which make up the FOAF vocabulary. There's plenty more work to do, of course, but this is an healthy step in the creeping professionalisation of our documentation. Uniquely identifying properties in FOAF are indicated using W3C's OWL language, making it possible for generic Semantic Web tools to easily merge together scattered, decentralised collections of FOAF descriptions.

The spec includes detailed and cross-referenced documentation for each term, as well as information about its status ('unstable','testing','stable'). The FOAF specification is presented to humans in XHTML form, but is also available in RDF form for machines, drawing upon W3C's RDF Schema and Web Ontology (OWL) languages.

FOAF 'vs' Introvertster

Things are heating up in the battle for the hearts, minds and contact lists of introverted Web geeks. While an Introvertster craze sweeps through workplaces and homes, FOAF's under-used foaf:myersBriggs support seems to be slowly catching on, revealing a disturbing trend. According to members of the Myers Briggs personality classification cult, individuals branded as IN** (ie INTJ, INFJ,INTP,INFP) should number no more than about 4 or 5% of the population. Yet a recent sampling by RDFWeb's FOAF scutter shows an alarming outburst of introversion amongst FOAF early adopters...

(I'm mostly serious...;-)

While I generally find MBTI a bit contentious, including foaf:myersBriggs has been an interesting exercise and even from the rather small dataset we have currently, there appear to be a distinct leaning towards the IN** side of things.

Dusting off my sociologists hat, there are various hypotheses we could investigate. Maybe FOAF's current audience is mainly geeky early adopters, people who show up as IN** when they go through the annoying-but-probing online questionnaires which try to approximate a full (tm)-compatible MBTI analysis. Maybe the character summaries for INTP are more flattering and ego-stroking than for other 'types' (or at least, more flattering to the kinds of people who show up as INTP in the surveys). Or at least, more likely to evoke a 'heheh, yeah that is sorta like me' reaction. Maybe INTP's like FOAF as a non-extraverted way of describing what they're like, and perhaps this extends to attaching a 'beware I'm an INTP / introverted freak, don't expect me to talk to you' health warning to their machine-readable homepage. Maybe INTP-ish people are the only ones obsessive and nit-picky enough to deal with RDF and XML syntax, while other types have malformed FOAF files which don't get past the RDF parser? Maybe the survey sample is too statistically insignificant to tell us anything.

Enough armchair sociologising about how 4% turned into 50%+, where's the data?

The results mentioned above come from an incomplete (but 24hrs+) run of my FOAF harvester ('scutter') from a couple days ago, ie. are based on a bunch of source-attributed RDF statements stored in a PostgreSQL database. I'll give the code example first, then the results, summarising all the foaf:myersBriggs properties that the database knows about. It uses my rough-n-ready RubyRdf library, and might some day be nicely packaged for others to use. In the meantime, here's the rough idea...

The mbti.rb script:

#!/usr/bin/env ruby
#
# Quick script to query RDF store for Myers Briggs data
$LOAD_PATH.unshift '../lib/'
$LOAD_PATH.unshift '../../lib/'

require 'basicrdf'
require 'squish'
require 'dbi'

query = SquishQuery.new.parseFromText <<EOQ;
    SELECT ?x, ?n, ?mb,
    WHERE
        (foaf::name ?x ?n)
        (foaf::myersBriggs ?x ?mb)
    USING
      foaf for http://xmlns.com/foaf/0.1/
EOQ

seen=Hash.new(false)
stats=Hash.new(0)
log=''
DBI.connect('DBI:Pg:scutter1','danbri','') do |connection|
  connection.select_all(query.toSQLQuery) do |row|
    if !seen[row['x']]
      log += "foaf:name='#{row['n']}' foaf:myersBriggs='#{row['mb']}'\n"
      seen[row['x']]=true
      stats[row['mb']]=stats[row['mb']]+1
    end
  end
end
puts "Statistics:"
stats.each_key do |type| puts "mbti type: #{type} count:#{stats[type]}\n" end
puts "Details:\n"+log

And here's the results of running it just now:

./mbti.rb
Stats:
mbti type: .... count:1
mbti type: ENFP count:1
mbti type: INTP count:11
mbti type: ENTJ count:1
mbti type: ISFJ count:1
mbti type: INFP count:2
mbti type: INTJ count:3
mbti type: ISTP count:1
mbti type: ISFP/ISFJ count:1
mbti type: ENTP count:1
mbti type: ISFP count:1
Details:
foaf:name='Michael Hanscom' foaf:myersBriggs='ISFP'
foaf:name='Bill Bradford' foaf:myersBriggs='ISFJ'
foaf:name='Jay Fienberg' foaf:myersBriggs='INTP'
foaf:name='Hiroki Ono' foaf:myersBriggs='INTP'
foaf:name='Eric Wahlforss' foaf:myersBriggs='ENTP'
foaf:name='Margaret Hart' foaf:myersBriggs='....'
foaf:name='Adam Gessaman' foaf:myersBriggs='INTP'
foaf:name='Simon' foaf:myersBriggs='INTP'
foaf:name='Jack Letourneau' foaf:myersBriggs='INTJ'
foaf:name='Earle Martin' foaf:myersBriggs='INFP'
foaf:name='Atsushi Suzuki' foaf:myersBriggs='INTP'
foaf:name='Evan Williams' foaf:myersBriggs='INTP'
foaf:name='Amy van der Hiel' foaf:myersBriggs='INTJ'
foaf:name='Dave Seidel' foaf:myersBriggs='ISFP/ISFJ'
foaf:name='\343\201\221\343\202\223\343\201\237\343\202\215' foaf:myersBriggs='INTP'
foaf:name='Nicole Sullivan' foaf:myersBriggs='INFP'
foaf:name='Bounty Hunters Guild' foaf:myersBriggs='INTP'
foaf:name='Chun-shek Chan' foaf:myersBriggs='ENFP'
foaf:name='Amy Alford' foaf:myersBriggs='INTJ'
foaf:name='Mark Pilgrim' foaf:myersBriggs='ISTP'
foaf:name='Jim Ley' foaf:myersBriggs='INTP'
foaf:name='Manuel Gonz\303\241lez Mart\303\255nez' foaf:myersBriggs='INTP'
foaf:name='Justin Knol' foaf:myersBriggs='INTP'
foaf:name='Masahide Kanzaki' foaf:myersBriggs='ENTJ'

Not much more to add really, thought this might be a fun mini-tutorial on RDF query. But I would be interested in pointers to other statistical analysis of MBTI data. In particular I reckon it would be worthwhile to feed the raw answers to the online quizes to statisticians, to see if the 16 types proposed by MBTI-advocates actually reflect clusterings within the data...

Spiders, scutters, harvesters, indexers, aggregators...

...whatever we call them, FOAF and RDF/XML developers who write Web robots that traverse the Web of inter-linked FOAF files should stick to the rules. Julian Bond has written a very handy overview of the things that need to be taken into account. These include general HTTP and Web indexing stuff like robots.txt, Etags, last-modified headers, HTTP response codes etc. In addition, there are many lessons from the RSS experience which folk writing scutters (FOAF harvesters) should be aware of.

Related reading, see the (not yet a) ScutterSpec in the FOAF wiki, as well as the Scutter entry.

And this is all without (yet) going into the various responsibilities associated with the fact that these systems aren't just aggregating random Web page content or weblog entries, but data about people...

Why FOAF has foaf:knows rather than foaf:friend...

In brief, foaf:knows was designed to be relatively harmless. It's use (and non-use) should be inoffensive. We dropped foaf:friend and foaf:knowsWell a year or two ago as they proved undeployably awkward, both when used and more interestingly, when not used.