The Open Measurement Layer

Why open education AI needs one more layer — and what it’s made of

Kumar Sumbhav Srivastava · Quantum Learning Machines · v1.0 · July 2026

Education AI is having its open moment. Open curricula (OpenSciEd, Illustrative Mathematics) tell tools what to teach. Open knowledge graphs give them shared structure. Open services and open-weights models give teachers real alternatives to black boxes. A serious open stack for teaching is assembling in public, and the biggest players are now building on it.

One layer is still missing, and it is the layer that decides whether the rest can be trusted: open measurement.

Today, the question “does this AI tool actually teach well?” is answered by the vendor selling it. Demos substitute for evidence. Benchmarks, where they exist, are private, contaminated, or self-graded. A district choosing between AI tutors has open curricula to point them at content — and marketing to tell them about quality. Open inputs, closed judgment.

The fix is the same move the field already made twice. Open curricula opened what is taught. Open platforms opened where practice happens. The third layer opens how quality is known. We call it the open measurement layer, and it has five components:

Shared ontologies of learning constructs. Public, research-cited vocabularies of what there is to measure — misconceptions, skills, prerequisite structure — so that tools, researchers, and benchmarks can talk about the same things. A misconception detector is only comparable to another if both are pointed at a common map.

Open evaluators. The models that judge AI output — misconception detectors, pedagogy scorers, restraint classifiers — released as open weights with documented training data, so anyone can run the judge, inspect the judge, and disagree with the judge.

Contamination-controlled benchmarks. Evaluation sets built for integrity from day one: held-out splits that never ship, evaluation tiers with detection signals redacted, canary strings, versioned releases. A benchmark a model may have trained on is a press release, not a measurement.

Limitations-first documentation. Model cards and dataset cards that lead with what the artifact cannot do, with every number traceable to a versioned result. The test of a card is whether a skeptic can check it.

Interoperability with the rest of the open stack. Measurement artifacts published in formats that plug into the open knowledge graphs, curricula, and practice platforms already in classrooms — so evaluation becomes plumbing, not product.

This layer is not hypothetical. Our pieces of it are live today: a 423-construct, research-cited misconception ontology (CC-BY-4.0, open API, with learning-graph and standards-alignment layers); two released evaluation classifiers with limitations-first cards — including honest, unflattering numbers, published because a measurement company that hides its own measurements is a contradiction; an openly licensed tutoring model documented the same way; and an evaluation-tier design that lets one public dataset serve honestly as both training data and benchmark. Interop with the open knowledge-graph ecosystem is in progress. Others hold other pieces. The point of naming the layer is that no one company should own it — it should be built the way open curricula were built: in public, by many hands, held to common standards.

The lineage matters.This is not a new idea; it is the OER movement’s third act. Open content came first. Open tools and practice followed. Open measurement completes the loop — because a stack that is open everywhere except at the point of judgment is open in inputs and closed where it counts.

An invitation. If you build teacher tools: publish your evaluations, not just your features. If you fund education AI: require the five components above before believing a quality claim. If you research learning: the ontologies and evaluators are yours to use, extend, and refute — telling us where they are wrong is a contribution. And if you are a district: there is a five-question checklist version of this document, written for procurement. Ask every vendor all five.

When the measurement layer is open, “which tool is good” stops being a marketing question. That is the ecosystem students and teachers deserve.

← Back to Resources Download PDF ↓