Unigram baseline
observed
Word frequencies only, no grammar. The floor any grammar model must beat.
11.10
bits / word
2199
perplexity
2.7%
top-1
5.4%
top-5
reference point
Loading precomputed artifact for held-out evaluation…parsing voynich-model.json (no inference runs in your browser)
Predictive accuracy
The success metric for this project is generalisation. Every fifth folio is withheld from fitting; models are then asked to predict its words. A grammar that captures something real about the system will be less surprised by pages it has never seen. Whether a reading “sounds like a herbal” plays no part in this score.
21,291 training tokens7,904 held-out tokens45 withheld folios of 225
held-out results
Model comparison
Lower perplexity is better. Top-1 / top-5 are next-word accuracy over a fixed sample of held-out positions against the full training vocabulary.
Word bigram
observed
Previous word only, interpolated with unigram. Raw local sequencing, no categories.
10.98
bits / word
2018
perplexity
2.9%
top-1
8.1%
top-5
8.2% lower perplexity than the no-grammar baseline
Inferred class grammar
inferred
P(class | previous class) × P(word | class) using the induced categories.
11.27
bits / word
2467
perplexity
2.7%
top-1
5.4%
top-5
-12.2% lower perplexity than the no-grammar baseline
Class + bigram hybrid
inferredbest
Grammar model interpolated with lexical bigrams. This is the model behind the translations.
10.93
bits / word
1956
perplexity
2.9%
top-1
6.3%
top-5
11.0% lower perplexity than the no-grammar baseline
interpretation
What these numbers do and do not license
- They license: the claim that the induced categories capture real sequencing regularities, because they reduce surprisal on folios that played no part in fitting them.
- They license: ranking rival grammar hypotheses. Any change to the class model can be re-scored here, and a change that does not improve held-out prediction is not an improvement.
- They do not license: any claim about meaning. A model can predict the next word perfectly while the text remains semantically opaque — or even while it is meaningless.
- They do not license: comparing these perplexities with figures published for natural-language corpora. Vocabulary size, tokenisation and line structure all differ.
protocol
Evaluation protocol
- Split: every fifth folio side, taken in manuscript order, is held out whole; a further fifth is used only to tune interpolation weights. Counts are taken from neither, so no test folio influences the models.
- Smoothing: interpolated absolute discounting for bigrams, backing off to a Kneser-Ney continuation distribution rather than raw frequency; a Dirichlet prior on class emissions; add-k on unigrams with mass reserved for unseen types.
- Out-of-vocabulary words: held-out words absent from the training text keep a non-zero probability from the reserved mass, so they are scored rather than skipped and cannot flatter any model.
- Line edges: line starts are modelled explicitly, as the manuscript treats lines as strong units.
- Determinism: clustering uses a fixed seed, so the same corpus always yields the same grammar and the same scores.
artifact provenance
Where these numbers come from
Every figure on this page is read from a single precomputed artifact. Nothing is fitted in the browser, so the same artifact always yields the same output.
artifact v2built 2026-09-06 20:23:10 UTC44.7s inference
- artifact
- src/data/voynich-model.json
- source corpus
- src/data/voynich-corpus.json
- corpus sha-256
- 92a4dc1c3409a3d5
- corpus size
- 322 KB
- build runtime
- bun 1.3.3
- generated at
- 2026-09-06T20:23:10.613Z
- clustering seed
- 20260803
- hybrid weights
- bigram 0.6 · class 0.05 · unigram 0.35
To reproduce: run bun scripts/precompute-analysis.ts against a corpus with the same hash and these settings; the artifact, and therefore every reading and score shown here, will be byte-identical.