Extending econ-ws Web Services with JSON-LD and Other RDF Output Formats

From the beginning, our econ-ws (terminology) web services for economics produce tabular output, very much like the results of a SQL query. Not a surprise - they are based on SPARQL, and use the well-defined table-shaped SPARQL 1.1 query results formats in JSON and XML, which can be easily transformed to HTML. But there are services, whose results not really fit this pattern, because they are inherently tree-shaped. This is true especially for the /combined1 and the /mappings service. For the former, see our prior blog post; an example of the latter may be given here: The mappings of the descriptor International trade policy are (in html) shown as:

concept prefLabel relation targetPrefLabel targetConcept target
<http://zbw.eu/stw/descriptor/10616-4> "International trade policy" @en <http://www.w3.org/2004/02/skos/core#exactMatch> "International trade policies" @en <http://aims.fao.org/aos/agrovoc/c_31908> <http://zbw.eu/stw/mapping/agrovoc/target>
<http://zbw.eu/stw/descriptor/10616-4> "International trade policy" @en <http://www.w3.org/2004/02/skos/core#closeMatch> "Commercial policy" @en <http://dbpedia.org/resource/Commercial_policy> <http://zbw.eu/stw/mapping/dbpedia/target>

That´s far from perfect - the "concept" and "prefLabel" entries of the source concept(s) of the mappings are identical over multiple rows.

Often, a consuming application will have to re-build the original tree structure, which can be visualized as:

Graph for International trade policy

In order to support such results more appropriately, we have added RDF formats (rdf-xml, n-triples, turtle and json-ld) to the output formats of the /combined1 and /mappings services. Under the hood, these outputs are generated by SPARQL CONSTRUCT (instead of SELECT) queries. In Turtle, the above result looks like this:

<http://zbw.eu/stw/descriptor/10616-4>
  skos:closeMatch  <http://dbpedia.org/resource/Commercial_policy> ;
  skos:exactMatch  <http://aims.fao.org/aos/agrovoc/c_31908> ;
  skos:prefLabel   "Internationale Handelspolitik"@de , "International trade policy"@en .

<http://dbpedia.org/resource/Commercial_policy>
  dcterms:isPartOf  <http://zbw.eu/stw/mapping/dbpedia/target> ;
  skos:prefLabel    "Commercial policy"@en .

<http://aims.fao.org/aos/agrovoc/c_31908>
  dcterms:isPartOf  <http://zbw.eu/stw/mapping/agrovoc/target> ;
  skos:prefLabel    "WELTHANDELSPOLITIK"@de , "International trade policies"@en .

The turtle syntax reflects fact that the Agrovoc and the DBpedia concept are both connected to the STW concept. Artificially built names, such as prefLabel and targetPrefLabel, which are required in the result table above to distinguish columns, are avoided. And by the way, it allows us to output prefLabels in English and German, without having to duplicate every row in the above table.

A great advantage of the RDF output formats is that the the results can be readily canned into other RDF-enabled web services. To demonstrate this, we provide two example pages for the /mappings and /combined1 services, where you can enter an arbitrary query or concept. The URI for the invocation of the respective econ-ws service is build from your input by a Javascript function, and passed to the VisualRDF service, which asks for the actual RDF and transforms it to a visual graph (as the one shown above).

JSON for Linking Data

For Semantic Web-savvy geeks, Turtle is very familiar and intuitive. However, web programmers without such a background would clearly prefer structured JSON, which can be parsed easily in Javascript (and almost any other language). In order to represent Linked Data in JSON, a W3C community group has developed JSON-LD and an accompanying API. Some days ago both reached W3C Candidate Recommendation state. On json-ld.org, lots of resources and an online playground for experiments are available.

Since currently our SPARQL server, Fuseki, does not deliver JSON-LD formatted results, we included Markus Lanthaler's JsonLD library to post-process turtle results delivered by Fuseki. This is working fine - for the above result we get:

{
  "@context": {
    "prefLabel": "http://www.w3.org/2004/02/skos/core#prefLabel",
    "exactMatch": "http://www.w3.org/2004/02/skos/core#exactMatch",
    "closeMatch": "http://www.w3.org/2004/02/skos/core#closeMatch",
    "isPartOf": "http://purl.org/dc/terms/isPartOf"
  },
  "@graph": [{
    "@id": "http://aims.fao.org/aos/agrovoc/c_31908",
    "isPartOf": {
      "@id": "http://zbw.eu/stw/mapping/agrovoc/target"
    },
    "prefLabel": [{
      "@language": "de",
      "@value": "WELTHANDELSPOLITIK"
    }, {
      "@language": "en",
      "@value": "International trade policies"
    }]
  }, {
    "@id": "http://dbpedia.org/resource/Commercial_policy",
    "isPartOf": {
      "@id": "http://zbw.eu/stw/mapping/dbpedia/target"
    },
    "prefLabel": {
      "@language": "en",
      "@value": "Commercial policy"
    }
  }, {
    "@id": "http://zbw.eu/stw/descriptor/10616-4",
    "closeMatch": {
      "@id": "http://dbpedia.org/resource/Commercial_policy"
    },
    "exactMatch": {
      "@id": "http://aims.fao.org/aos/agrovoc/c_31908"
    },
    "prefLabel": [{
      "@language": "de",
      "@value": "Internationale Handelspolitik"
    }, {
      "@language": "en",
      "@value": "International trade policy"
    }]
  }]
}

The result is straight JSON. The context part may or may not be used by an application. The last section of the graph part contains the overall structure for the source concept, whereas the details about the mapped DBpedia and Agrovoc concepts can be found in separate sections.

To Frame or Not To Frame

The graph showed above could transformed to an even more intuitive one by embedding the properties of the mapped concepts into the main tree derived from the source concept, resulting in one overall tree structure. This kind of shaping of the output is called "Framing" in JSON-LD. Since such a graph can take multiple forms, depending on application demands, it requires something in the kind of a template, named "Frame" here. We provide here a link to the JSON-LD Playground which will load our example and a simple frame document. (Please open the "Framed" tab, and feel free to experiment.)

The JSON-LD Framing spec is not on the W3C Standards Track, and has some open issues. We did not include a framed approach into the current version of econ-ws, but we do see it as quite promising: Framed JSON-LD results can directly, without further processing, be used as a, e.g., Javascript data structure. The hard work to assemble the RDF "triple soup" into something meaningful can be done once and for all be the providers of a service, and is taken from the shoulders of the application programmer. However, if this application programmer happens to be a semweb geek, she can lossless get the raw triples to combine them with triples from other sources and assemble them into something completely different.

An emerging pattern for unified Linked Data resource lookup and web service results?

In our Linked Data services, there is a not really satisfactory dichotomy between the econ-ws services, which mostly deliver query results of some kind, and the results you get when you look up resource URIs such as the one for International trade policy: The former delivered (up to now) easily understandable, table-shaped XML or JSON, while the latter deliver RDFa, RDF/XML or Turtle, which requires some effort for parsing, especially for web developers which are not deeply in Semantic Web technology. (BTW, this situation is reflected in the fact that we have marked the /narrower and /labels service as "deprecated" - as the data can be obtained by resource lookup -, but not yet have removed them.)

Perhaps, JSON-LD could bridge the gap, in that it can be used in both kinds of services, and in that it is able to express complex data structures in a widely familiar and easy-to-use way.