A few days ago, I wrote about adding Triplify to your web application. Specifically, I wrote about adding it to WordPress, but the same information can be applied to most web publishing platforms. Earlier this month, TALIS announced their Connected Commons platform and yesterday they announced a commercial version of their platform for the structured storage of Linked Data. Storage is all very well, but more importantly they have an API for developers, so that the data can be queried and creatively re-used or mashed up.
So this got me thinking about JISCPress, our recent JISC Rapid Innovation Programme bid, which proposes a WordPress Multi-User based platform for publishing JISC funding calls and the reports of funded projects. This is based on my experience of running WriteToReply with Tony Hirst.
Although a service for comment and discussion around documents, one of the things that interests me most about WriteToReply and, consequently the JISCPress proposal, is the cumulative storage of data on the platform and how that data might be used. No surprise really as my background is in archiving and collections management. As with the University of Lincoln blogs, WriteToReply and the proposed JISCPress platform, aggregate published content into a site-wide ‘tags’ site that allows anyone to search and browse through all content that has been published to the public. In the case of the university blogs, that’s a large percentage of blogs, but for WriteToReply and JISCPress, it would be pretty much every document hosted on the platform.
You can see from the WriteToReply tags site that over time, a rich store of public documents could be created for querying and re-use. The site design is a bit clunky right now but under the hood you’ll notice that you can search across the text of every document, browse by document type and by tag. The tags are created by publishing the content to OpenCalais, which returns a whole bunch of semantic keywords for each document section. You’ll also notice that an RSS feed is available for any search query, any category and any tag or combination of tags.
Last night, I was thinking about the WriteToReply site architecture (note that when I mention WriteToReply, it almost certainly applies to JISCPress, too – same technology, similar principles, different content). Currently, we categorise each document by document type so you’ll see ‘Consultations‘, ‘Action Plans‘ ‘Discussion Papers‘, etc.. We author all documents under the WriteToReply username, too and tag each document section both manually and via OpenCalais. However, there’s more that we could do, with little effort, to mark up the documents and I’ve started sketching it out.
You’ll see from the diagram that I’m thinking we should introduce location and subject categories. There will be formal classification schemes we could use. For example, I found a Local Government Classification Scheme, which provides some high level subjects that are the type of thing I’m thinking about. I’m not suggesting we start ‘cataloguing’ the documents, but simply borrow, at the top level, from recognised classification schemes that are used elsewhere. I’m also thinking that we should start creating a new author for each document and in the case of WriteToReply, the author would be the agency who issued the consultation, report, or whatever.
So following these changes, we would capture the following data (in bold), for example:
The Home Office created Protecting the public in a changing communications environment on April 27th which is a consultation document for England, Wales and Scotland, categorised under Information and communication technology with 18 sections.
Section one is tagged Governor, Home Department, Office of Public Sector Information, Secretary of State, Surrey.
Section two is tagged communications data, communications industry, emergency services, Home Secretary, Jacqui Smith MP, Rt Hon Jacqui Smith MP.
Section three is tagged Broadband, BT, communications, communications changes, communications data, communications data capability, communications data limits, communications environment, communications event, communications industry, communications networks, communications providers, communications service providers, communications services, emergency services, Her Majesty’s Revenue and Customs, Home Office, intelligence agencies, internet browsing, Internet Protocol, Internet Service, IP, mobile telephone system, physical networks, public telecommunications service, registered owner, Serious Organised Crime Agency, social networking, specified communications data, The communications industry, United Kingdom.
Section four is tagged …(you get the picture)
Section five, paragraph six, has the comment “fully compatible with the ECHR” is, of course, an assertion made by the government, about its own legislation. Has that assertion ever been tested in a court? authored by Owen Blacker on April 28th 11:32pm.
Selected text from Section five, paragraph eight, has the comment Over my dead body! authored by Mr Angry on April 28th 9:32pm
Note that every author, document, section, paragraph, text selection, category, tag, comment and comment author has a URI, Atom, RSS and RDF end point (actually, text selection and comment author feeds are forthcoming features).
Now, with this basic architecture mapped out, we might wonder what Triplify could add to this. I’ve already shown in my earlier post that, with little effort, it re-publishes data from a relational database as N-Triples semantic data, so everything you see above, could be published as RDF data (and JSON, too).
So, in my simple view of the world, we have a data source that requires very little effort to generate content for and manage (JISCPress/WriteToReply/WordPress), a method of automatically publishing the data for the semantic web (Triplify) and, with TALIS, an API for data storage, data access, query, and augmentation. As always, my mantra is ‘I am not a developer’, but from where I’m standing, this high-level ‘workflow’ seems reasonable.
The benefits for the JISC community would primarily be felt by using the JISCPress website, in a similar way (albeit with better, more informed design) to the WriteToReply ‘tags’ site. We could search across the full text of funding calls, browse the reports by author, categories and tags and grab news feeds from favourite authors, searches, tags or categories. This is all in addition to the comment, feedback and discussion features we’ve proposed, too. Further benefits would be had from ‘re-publishing’ the site content as semantic data to a platform such as TALIS. Not only could there be further Rapid Innovation projects which worked on this data, but it would be available for any member of the public to query and re-use, too. No longer would our final project reports, often the distillation of our research, sit idle as PDF files on institutional websites and in institutional repositories. If the documentation we produce it worth anything, then it’s worth re-publishing openly as semantic data.
Finally, in order to benefit from the (free) use of TALIS Connected Commons, the data being published needs to be licensed under a public domain or Creative Commons ‘zero’ licence. I suspect Crown Copyright is not compatible with either of these licenses, although why the hell public consultation documents couldn’t be licensed this way, I don’t know. Do you? For JISCPress, this would be a choice JISC could make. The alternative is to use the commercial TALIS platform or something similar.
As usual, tell me what you think… Thanks.
Hi Joss,
This is a great idea and I think the workflow fits together very nicely. Building a semantic index over documents published elsewhere on the web, and then enriching that index with additional metadata from services like OpenCalais is both a great use of the technologies and of the Platform itself.
I’d be really interested to see something like this built around the Platform. Let me know how I can help. If you’re interested in developer access, then just drop me a line and I can set that up.
With regards to the copyright issue. While I’m not sure that Crown Copyright is compatible with CC0/PDDL it is very permissive (more so than many people realise). My understanding is that there is ongoing work to look at how material licensed under Crown Copyright can be more easily remixed with data from elsewhere.
Of course this issue is only really relevant if you plan to store the actual documents in the Platform — if you’re just publishing the comments and related metadata, then you don’t have any issue, other than getting approval of those people writing the comments. I also don’t think you have any issue in publishing the results of running OpenCalais over these documents (but IANAL, so its worth checking!), as this is separate data, you’re not re-publishing the document itself.
If there’s anything I can do to move this idea forward then let me know,
Cheers,
L.
Both our JISCPress proposal and WriteToReply publish the full text of the document, so copyright is an issue, but with enough will, shouldn’t stand in the way. One thing I didn’t really address in my post was the link from Triplify to Talis. I need to ask on the Triplify mailing list how publishing data automatically from one to the other might happen. Perhaps you have a suggestion?
It would be good to bring a few people together from government, Talis, OpenCalais, Triplify, etc. to discuss how we could achieve this, even something that demonstrates how it would work and the value to be gained from it. I’ll see what I can do and get back to you. Thanks for your encouraging comments.
Actually, thinking about it Leigh, you’re right. There’s no need to re-publish the actual document content to Talis. Triplify could just select the metadata and publish that, which would get around the Copyright issue. For WriteToReply, the comments are made under a CC-BY license, but again, the comments themselves need not be re-published, just the metadata. Hmmmm…
Looking back at this post, I can see I completely misunderstood the licensing issues. Triplify produces a flat RDF/XML file of triples but does not contain any of the source content (i.e. blog posts, etc) in the file. As Leigh has pointed out, the RDF/XML output from Triplify can be hosted on Talis Connect Commons if it is licensed under an appropriate license. This does not affect the licensing of the source content, which can be licensed under a difference license (or not at all).