Fabryka AI / Research agenda 2026
Three connected programmes — from learning with limited compute to measuring the systems we deploy. Current artifacts and planned studies are identified separately.
01 — FAB-SLM-001 / SMALL LANGUAGE MODELS
RESEARCH QUESTION
How do data composition, tokenization and teacher supervision affect model quality under a fixed parameter and training-compute budget?
STUDY DESIGN
Matched training runs on fixed evaluation splits; vary one factor at a time; record tokenizer, data revisions, tokens, seeds and compute.
SCOPE & COMPUTE
Sub-150M models first. 1B and 10B studies are roadmap directions, subject to compute and data readiness.
| R0 ctrl | R1 | R2 | R3 | R4 | |
|---|---|---|---|---|---|
| Tokenizer | PL-32k | ||||
| Data mixture | +PL | ||||
| Teacher | KD | ||||
| Seed | ×3 |
02 — FAB-SYS-001 / EFFICIENT INFERENCE
RESEARCH QUESTION
How do memory bandwidth, attention implementations, quantization and decoding strategies affect latency, throughput, memory use and output quality?
STUDY DESIGN
Same model and workload across documented hardware and software. Measure prefill, TTFT, decode speed and latency distributions under specified concurrency. Quality is evaluated separately from speed.
SCOPE
Quantization, speculative decoding incl. DFlash, routing, commodity GPUs. Every result states context length, batch size and load.
| FP16 | INT8 | INT4 | +spec | |
|---|---|---|---|---|
| RTX 3090 | planned cell | planned cell | planned cell | planned cell |
| RTX 4090 | planned cell | planned cell | planned cell | DFlash priority |
| RTX 5090 | planned cell | planned cell | planned cell | DFlash priority |
| L4 | planned cell | planned cell | planned cell | planned cell |
03 — FAB-DATA-001 / POLISH AI EVALUATION & DATA
RESEARCH QUESTION
How do dataset composition and evaluation design affect our estimates of model capability in Polish, including document and OCR tasks?
STUDY DESIGN
Document provenance, licensing, splits and scoring. Check contamination and reference quality. Use task-appropriate metrics and matched protocols; separate our measurements from externally reported results.
Bar widths indicate stage order, not measured volumes.
FABRYKA AI NOTES / REPORTS & PROPOSALS
Technical reports and proposals shared by SlayerLab, Fabryka AI and collaborators. Each entry retains its authors, version and result status from the source repository.
Research proposal · Author-reviewed
No results
Read illustrated article →, A data queue with quarantine: proposal for online data selection in GoLLeM-v6Read PDF ↗, A data queue with quarantine: proposal for online data selection in GoLLeM-v6Metadata & attribution ↗, A data queue with quarantine: proposal for online data selection in GoLLeM-v6
Technical report · Draft
Preliminary results
Read illustrated article →, GoLLeM-v5: Small Language Models for the Glint Tiny-ML Leaderboard. Measurement, Method and Negative ResultsRead PDF ↗, GoLLeM-v5: Small Language Models for the Glint Tiny-ML Leaderboard. Measurement, Method and Negative ResultsMetadata & attribution ↗, GoLLeM-v5: Small Language Models for the Glint Tiny-ML Leaderboard. Measurement, Method and Negative Results
“Author-reviewed” is the repository’s review status; it does not indicate journal peer review. Drafts and proposals are works in progress. See each entry’s metadata for licensing; an undecided license does not grant reuse rights.
Browse Fabryka AI Notes on GitHub ↗OPEN RESEARCH / ACROSS THE REPOSITORIES
Nine selected notes across six public repositories: small models, Polish tokenizers, evaluation and data preparation. Browse measured results, provisional pilots and technical methods with their source records.
RESEARCH ATLAS / FOLLOW THE CONNECTIONS
From data and representation to measurement and decisions. Select a note to see the connection, its purpose and the source evidence.
Documented reuseConceptual connectionArrows show relationships, not a timeline.
Scroll the diagram sideways, or select a note below.
01 / REPRESENTATION
A handwritten byte-level BPE is compared with Hugging Face variants at the same 6,756-token vocabulary. The question is how implementation and pretokenisation change compression.
The reported range is 833–844 tokens on a small held-out. It does not establish a downstream model-quality gain.
Different token counts → a common unit
The BPE experiment shows that token counts depend on tokenisation. This motivates looking beyond per-token quantities when comparing models. The recorded panel uses bits per byte; this is a methodological connection, not evidence that Probierz used this tokenizer.
02 / DATA QUALITY
The audit compares the existing data pipeline with published corpus-cleaning approaches and identifies candidate quality, deduplication and filtering checks.
This is a gap analysis for future experiments, not a measured cleaning ablation.
Audit overlap before interpreting scores
Data-cleaning research includes evaluation-overlap checks. The Ladder pilot explicitly leaves training-overlap review pending. Both address the same validity problem; no shared implementation is established by these notes.
Dataset cleaning: source ↗ · Polish agreement probe: source ↗
03 / MEASUREMENT
Probierz defines measurement dimensions, provenance and overlap checks, and a programme of falsifiable evaluation. It also provides scoring tools used by the recorded model panel.
The panel is evidence for particular metrics, not validation of the entire multidimensional measurement programme.
Different token counts → a common unit
The BPE experiment shows that token counts depend on tokenisation. This motivates looking beyond per-token quantities when comparing models. The recorded panel uses bits per byte; this is a methodological connection, not evidence that Probierz used this tokenizer.
Scoring tools + syntax items
The Track result manifest identifies Probierz’s scorer_hf.py, bpb_scorer.py, run_forced_choice.py and d3 syntax eval items, including hashes. These tools and items turn part of the measurement programme into a recorded model panel.
04 / PILOT
A matched-item pilot records continuous pair probabilities and runtime for Koliber and Pollock. Its measurements supply the cost-planning note.
Native review and overlap checks are pending. Corpus BPB and training-seed variance were not measured; language-training domains differ.
Audit overlap before interpreting scores
Data-cleaning research includes evaluation-overlap checks. The Ladder pilot explicitly leaves training-overlap review pending. Both address the same validity problem; no shared implementation is established by these notes.
Dataset cleaning: source ↗ · Polish agreement probe: source ↗
Measured runtime + observed contrast
The cost note explicitly reuses the 2,000-pair pilot: 111.3 seconds for Koliber, 100.4 seconds for Pollock and an observed probability difference of 0.3169. It uses these measurements to anchor workload estimates; it cannot compute seed-calibrated DF.
05 / RECORDED EVIDENCE
The result archive stores per-document compression scores, paired-bootstrap artifacts and a syntax panel. Its manifest names and hashes Probierz scoring tools and syntax items.
BPB uses 300 documents; the reported syntax core contains 148 pairs. This panel and the Ladder pilot use different items and metrics.
Scoring tools + syntax items
The Track result manifest identifies Probierz’s scorer_hf.py, bpb_scorer.py, run_forced_choice.py and d3 syntax eval items, including hashes. These tools and items turn part of the measurement programme into a recorded model panel.
From a score to a useful decision
The historical panel reports scores and uncertainty. Ladder asks the additional question: can a metric distinguish the intended models reliably enough to justify its cost? This connects research questions; the cost note does not claim to reuse the Track panel.
06 / NEXT DECISION
The planning note reuses the Polish pilot’s observed runtimes and contrast, then estimates the workload of other benchmarks to plan the next paired evaluation.
English-battery times are proxies. Seed-calibrated discrimination is unavailable, so the note does not establish which benchmark should be adopted.
Measured runtime + observed contrast
The cost note explicitly reuses the 2,000-pair pilot: 111.3 seconds for Koliber, 100.4 seconds for Pollock and an observed probability difference of 0.3169. It uses these measurements to anchor workload estimates; it cannot compute seed-calibrated DF.
From a score to a useful decision
The historical panel reports scores and uncertainty. Ladder asks the additional question: can a metric distinguish the intended models reliably enough to justify its cost? This connects research questions; the cost note does not claim to reuse the Track panel.
Two reuse links are documented in the inspected notes. Three dashed links are editorial interpretations of shared research questions. They do not imply a shared codebase, dataset or chronological dependency. Sources are pinned to the revisions used in this catalogue.
Illustrated research note · micro-models
Completed pilot · limited sample
Read illustrated article →, Small models, complementary errorsSource note ↗, Small models, complementary errors
Illustrated research note · tokenizer
Workshop research · measured tokenizer metrics
Read illustrated article →, A smaller vocabulary, fitted to PolishSource note ↗, A smaller vocabulary, fitted to Polish
Illustrated research note · ladder
Provisional diagnostic · incomplete tier
Read illustrated article →, A Polish probe, with its limits visibleSource note ↗, A Polish probe, with its limits visible
Experiment report · micro-models
Reported results · corpus-specific
Source note ↗, Music experts: ensemble, stitching and representation similarity
Workshop research · tokenizer
Small held-out · 371 words
Source note ↗, Polish BPE: implementation and pretokenisation
Benchmark methodology · Probierz
Experimental programme
Source note ↗, Probierz: evaluation dimensions and falsifiable measurement
Technical research note · slayer-dsbench
Gap analysis · not an ablation result
Source note ↗, Dataset cleaning: pipeline gaps and proposed experiments
Benchmark artifact · fabryka-track
Historical matched panel
Source note ↗, Polish compression and syntax: recorded model panel
Evaluation planning note · ladder
Measured probe + explicit cost proxies
These are selected research notes and artifacts, not nine peer-reviewed papers. Measurements retain their own datasets and protocols; scores from different studies are not combined.
PUBLICATION STANDARD
Each report connects question to limitations, names authors, dates, funding and compute, and links every artifact needed to inspect or reproduce it. Results appear here only once a corresponding artifact exists.
| ARTIFACT | PROGRAMME | TYPE | STATUS |
|---|---|---|---|
| Pollock Mini LM 125M | SLM-001 | Model | Public |
| Polish DynaWord | DATA-001 | Dataset | Public |
| CodeSOTA | DATA-001 | Benchmark | Public |
| Token-probability tools | SYS-001 | Tool | Live |
| Inference platform | SYS-001 | Service | Live |
| Data-mixture scaling study | SLM-001 | Technical report | Planned |
| Commodity-GPU inference comparison | SYS-001 | Technical report | Planned |