How it works

Overview

Every taxonomy in the Library can be traced back to the exact document it came from.

Each source edition is given a permanent URL in an archive, then extracted and manually checked, mapped to an open schema, and published as RDF. Once data is published, the website will be generated from it, so what you read here will always match what you can download and cite.

The flow

ON THE WEB ON GITHUB ohtl.org Ofcom WEF crosswalks archive official archiveopen repositoryWayback Machine first that applies, in order sources Ofcom WEF PDFURL PDFURL archived unaltered links to original source website generation scripts taxonomies published RDF build pipeline private extract, review, map, validate schema SKOS-based, pinned release Ofcom pagesWEF pages Ofcom RDFWEF RDF Ofcom parserWEF parser cites permanent URL downloads private copy generates builds deploys maps to THIRD PARTIES researcherscompliance teamssafety vendors reused by feed into schema
Two example sources, Ofcom and the World Economic Forum. Each edition is given a permanent URL in the first archive that applies: an official archive if it is already in one, otherwise an open repository if the licensing terms allow, otherwise the Wayback Machine. The build pipeline works from its own private copy, each published taxonomy cites the permanent URL of the edition it came from, and ohtl.org links to the original. Researchers, compliance teams and safety vendors reuse the published taxonomies and schema, and feed back into the schema.

Steps

  1. Source We start from a taxonomy published by a regulator, researcher or company, such as Ofcom's codes and guidance or the World Economic Forum's Typology of Online Harms.
  2. Archive Every edition we use gets a permanent URL, so it can always be found. If the edition already has a permanent URL in an official archive, such as legislation.gov.uk or EUR-Lex, we use that. Otherwise, if the licensing terms allow, we upload an unaltered copy to an open repository such as Zenodo, which gives it a DOI. Otherwise, we trigger a capture in the Wayback Machine and use that, while we ask the copyright holder for permission to upload it. We also record a checksum for every edition, so anyone can check a copy matches the original.
  3. Extract and map The build pipeline extracts categories, definitions and examples from its own copy of the source, kept privately for research and not shared. The extraction is manually checked before it is mapped to the open schema and validated.
  4. Publish The result is published as RDF, recording the source edition, its permanent URL and the schema version it was built from, so anyone who reuses the data can trace it to the exact edition.
  5. Present ohtl.org will be generated from the published RDF. Each taxonomy will link to its original source, its permanent URL and matching categories in other taxonomies.

Repositories

  • schemaThe open schema, built on the W3C SKOS standard
  • taxonomiesPublished taxonomies as RDF, one folder per source edition