A release should carry its own evidence

Sergio Leon
Sergio Leon
Author

There's an obvious objection to Erden: "I already keep my ontology on GitHub."

Keep it there. Erden doesn't change where you write an ontology. It changes what a release is.

What a release should mean

A claim in a knowledge graph is worth very little if you can't trace where it came from. Provenance is the reason to trust the graph at all, so it belongs in the format from the start. The same goes for a release as a whole.

My premise is that a release should carry its own evidence, checked the same way every time, so someone can trust an ontology they didn't write without redoing the work.

That takes three properties:

  • It can't change after it's published. If the bytes can move, the evidence means nothing.
  • Its dependencies point at exact releases. "Whatever's current" isn't something you can check.
  • Its results are tied to the exact files and tools that produced them. A green check is only useful if you know what was checked, and by what.

Four stages, and no partial credit

Publishing runs a package through four stages:

Reasoning. A fixed version of an OWL reasoner computes the release's conclusions. An independent proof checker then verifies each one and records a certificate, so nobody has to take the reasoner's word for it.

SHACL validation. Shapes, severities and targets, checked against the declared data and constraints.

Competency questions. SPARQL queries the ontology must be able to answer. The idea goes back to Grüninger and Fox in 1995. It's the closest thing ontologies have to a test suite written in terms of what the ontology is for.

Citation checks. The evidence an ontology cites has to be present.

A release exists only if every stage passes. There's no published-with-failures state to explain later.

The SHACL and competency-question checks also run on your own computer with erden check, so you can catch those problems before you publish.

Trust that goes up the chain

Evidence gets more useful as ontologies build on one another. Every release you depend on carries its own results, so the whole dependency closure can be read as a graph. You can see what was checked upstream and how, instead of taking it on faith.

That's the part I most want to see working across organizations, because trusting something you wrote yourself was never the hard case.

Where this stands

The pipeline runs on an internal deployment for now, which is a long way from something other people can rely on. It covers a documented part of OWL rather than all of it, and it hasn't yet met enough real-world ontologies for me to know what breaks first.

That's the next thing to find out.