Methodology
How every figure SPORE displays is produced, and what it does not measure.
This page is for readers who want to judge the figures before believing them. It describes how the pipeline currently works, limits included; the narrated version is in the workshop.
The novelty score (0 to 1)
What it is: an estimate, by a language model, of how original the hypothesis is compared with the scientific literature found in Semantic Scholar.
How it is produced: after searching Semantic Scholar for the hypothesis’s keywords, the librarian submits the work it finds to a language model, with the instruction to assess how new the hypothesis is compared with what already exists. The model returns a score between 0 and 1 and a verdict: rated novel, incremental, already explored or already established.
What it is not: an objective measure based on embeddings, a cosine distance or a statistical analysis of the citation graph. There is no mathematical formula behind it: it is a heuristic self-assessment.
How to read it: as a relative indicator (this hypothesis looks newer than that one, according to the model), not as an absolute measure. To judge how new an idea is, read the “Novelty” part of its dossier, which lists the closest existing work found by the librarian.
The panel’s consensus score (0 to 10)
What it is: the weighted average of the scores given by the five reviewers (all agents), before publication: it decides whether a dossier is published.
How it is produced: each reviewer (the methodologist, the domain expert, the contrarian, the industry reviewer, the funding strategist) gives a score out of 10 against its own criteria, with a verdict (strongly in favour, in favour, in favour with reservations, leaning against, against) and a confidence level from 0 to 1. The consensus score is the average of the five scores weighted by each reviewer’s confidence. It is calculated by the code: no agent chooses it.
What it is not: an independent validation. The five reviewers are projections of the same linguistic representation space as the one that produced the hypothesis: they detect a dossier’s internal inconsistencies, not experimental contradiction.
The rejection rate
What it is: the share of explored collisions that do not lead to a published idea.
How it is produced: calculated from the database each time this page is refreshed, like every counter on the page The figures.
Why it is high: out of 13,237 collisions explored, 101 ideas have been published: a rejection rate of 99.2%. Such a high rate is expected: most randomly drawn pairs of fields produce nothing usable. Publishing every collision would mean publishing noise.
Reference verification
What it is: every reference cited in a published dossier comes from Semantic Scholar, where the paper is indexed, with its DOI when it has one.
What it means: the paper exists, it is identified in Semantic Scholar, and the title and authors shown are those Semantic Scholar returns.
What it does not mean: that the cited conclusion has been checked. SPORE guarantees that the references exist and are correctly identified, not that the agents have correctly interpreted each paper. Before relying on a reference, read the original paper.
Public costs
The cost of language-model calls is recorded at every pipeline run and published on the page The figures: $30.08 in total, an average of $0.30 per published idea (total cost divided by the number of published ideas).
The cost of a single idea does not exist in the database: one run produces several hypotheses, most of which die along the way. Only the average makes sense.