Slayer1B 1.08b (experimental)
Historical two-arm continued-pretraining experiment, trained on 7 November 2026. Weights are not published in this repository. Slayer1B 1.08b continues Slayer1B 1.07 in a reasoning-data arm and a replay-only control. It is distinct from the later Slayer1B 1.08 model; the two names do not represent the same checkpoint.
What was trained
Continued pretraining from the final Slayer1B 1.07 checkpoint (step 22327), as an ablation with two arms of equal length and identical settings:
- 1.08b (arm p00): step-by-step reasoning data (mathematics, code, logic) plus Polish replay;
- 1.08b control (arm p00c): Polish replay only.
| Start | Slayer1B 1.07, step 22327 |
| Steps | 22328 โ 23073 (746 steps per arm) |
| Training tokens per arm | 782 237 696 (746 steps ร 1 048 576) |
| Data per arm | 782 469 318 tokens |
| Global batch | 256 sequences ร 4096 tokens (1 048 576 tokens per step) |
| Learning rate | constant 6e-4 (no decay; the same as 1.07, so the arms differ only in data) |
| Hardware | 8 ร NVIDIA RTX 5090, four GPUs per arm, both arms in parallel |
| Date | 7 November 2026 |
Training data and processing (decontamination, personal-data masking, held-out overlap scan) are described in the data card SlayerLab/slayer-1.08b-pretrain.
Training curves: 1.08b ยท 1.08b control. The training loss of the two arms is measured on different data mixtures and is not comparable between them.
Model
- Parameters: ~1.07B
- Architecture: 20 layers, 2048 dim, 16 heads, RoPE, SwiGLU, QK-Norm, value residual
- Context: 4096 tokens
- Tokenizer: 32 768 (Slayer v6), the same as 1.00โ1.07
- Optimizer: Muon + AdamW
Evaluation record
The final update-23,073 checkpoints of both arms were evaluated against the 1.07 starting checkpoint. This was one run per arm with a constant learning rate. The versioned evaluation record preserves the numerical tables, methods, uncertainty estimates and scorer corrections.
Those historical reasoning measurements did not establish a gain over the replay-only control. The initial reasoning set did not discriminate between the models; the recalibrated set had low sensitivity. The English-language panel recorded lower scores than 1.07 for both arms, but the experiment did not isolate the cause. These are observations about this ablation, not a rule for model naming or publication.
We thank BenchTeam / Fabryka AI for its work on standard benchmark protocols and evaluation. This card does not report the later nine-task measurements for 1.08b: results for 1.08 or the 1.09 continuation must not be attributed to this different experiment.
Limitations
- Base model, not instruction-tuned; responses may be inconsistent.
- Experimental: constant learning rate, one run per arm and one seed; the historical reasoning comparison had low sensitivity.
- The experiments do not establish general reasoning ability, instruction-following reliability or professional suitability.
Authorship and license
Authorship: Arkadiusz Sลota โ Fabryka AI SlayerLab. License: apache-2.0 (as 1.00โ1.07); data licenses are in the data card.