Research reports, reproducible experiments and technical notes from our public repositories.
OPEN RESEARCH / ACROSS THE REPOSITORIES
More questions. More evidence.
Nine selected notes across six public repositories: small models, Polish tokenizers, evaluation and data preparation. Browse measured results, provisional pilots and technical methods with their source records.
RESEARCH ATLAS / FOLLOW THE CONNECTIONS
What builds on what?
From data and representation to measurement and decisions. Select a note to see the connection, its purpose and the source evidence.
Documented reuseConceptual connectionArrows show relationships, not a timeline.
Scroll the diagram sideways, or select a note below.
01 / REPRESENTATION
Polish BPE
A handwritten byte-level BPE is compared with Hugging Face variants at the same 6,756-token vocabulary. The question is how implementation and pretokenisation change compression.
The reported range is 833–844 tokens on a small held-out. It does not establish a downstream model-quality gain.
The BPE experiment shows that token counts depend on tokenisation. This motivates looking beyond per-token quantities when comparing models. The recorded panel uses bits per byte; this is a methodological connection, not evidence that Probierz used this tokenizer.
The audit compares the existing data pipeline with published corpus-cleaning approaches and identifies candidate quality, deduplication and filtering checks.
This is a gap analysis for future experiments, not a measured cleaning ablation.
Data-cleaning research includes evaluation-overlap checks. The Ladder pilot explicitly leaves training-overlap review pending. Both address the same validity problem; no shared implementation is established by these notes.
Probierz defines measurement dimensions, provenance and overlap checks, and a programme of falsifiable evaluation. It also provides scoring tools used by the recorded model panel.
The panel is evidence for particular metrics, not validation of the entire multidimensional measurement programme.
The BPE experiment shows that token counts depend on tokenisation. This motivates looking beyond per-token quantities when comparing models. The recorded panel uses bits per byte; this is a methodological connection, not evidence that Probierz used this tokenizer.
The Track result manifest identifies Probierz’s scorer_hf.py, bpb_scorer.py, run_forced_choice.py and d3 syntax eval items, including hashes. These tools and items turn part of the measurement programme into a recorded model panel.
Data-cleaning research includes evaluation-overlap checks. The Ladder pilot explicitly leaves training-overlap review pending. Both address the same validity problem; no shared implementation is established by these notes.
The cost note explicitly reuses the 2,000-pair pilot: 111.3 seconds for Koliber, 100.4 seconds for Pollock and an observed probability difference of 0.3169. It uses these measurements to anchor workload estimates; it cannot compute seed-calibrated DF.
The result archive stores per-document compression scores, paired-bootstrap artifacts and a syntax panel. Its manifest names and hashes Probierz scoring tools and syntax items.
BPB uses 300 documents; the reported syntax core contains 148 pairs. This panel and the Ladder pilot use different items and metrics.
The Track result manifest identifies Probierz’s scorer_hf.py, bpb_scorer.py, run_forced_choice.py and d3 syntax eval items, including hashes. These tools and items turn part of the measurement programme into a recorded model panel.
The historical panel reports scores and uncertainty. Ladder asks the additional question: can a metric distinguish the intended models reliably enough to justify its cost? This connects research questions; the cost note does not claim to reuse the Track panel.
The planning note reuses the Polish pilot’s observed runtimes and contrast, then estimates the workload of other benchmarks to plan the next paired evaluation.
English-battery times are proxies. Seed-calibrated discrimination is unavailable, so the note does not establish which benchmark should be adopted.
The cost note explicitly reuses the 2,000-pair pilot: 111.3 seconds for Koliber, 100.4 seconds for Pollock and an observed probability difference of 0.3169. It uses these measurements to anchor workload estimates; it cannot compute seed-calibrated DF.
The historical panel reports scores and uncertainty. Ladder asks the additional question: can a metric distinguish the intended models reliably enough to justify its cost? This connects research questions; the cost note does not claim to reuse the Track panel.
Two reuse links are documented in the inspected notes. Three dashed links are editorial interpretations of shared research questions. They do not imply a shared codebase, dataset or chronological dependency. Sources are pinned to the revisions used in this catalogue.
A simple ensemble reaches PPL 5.15 on the mixed waltz/reel task; the corrected stitch result is 5.18. Includes measurement corrections and representation-analysis limits.
A matched 6,756-token vocabulary study compares a handwritten BPE with Hugging Face variants. Reported token counts span 833–844; the note cautions against strong claims from this small sample.
Provenance, overlap audits, blind exchange and a proposed multidimensional evaluation design. The repository includes runnable probes and identifies incomplete validation.
Separates measured Polish-probe runtime from estimated English-battery cost and explains why seed-calibrated discrimination cannot be inferred from a single score.
These are selected research notes and artifacts, not nine peer-reviewed papers. Measurements retain their own datasets and protocols; scores from different studies are not combined.