bibr-front-tagger (small)

An mmBERT-small encoder with a CRF head that reads the first two pages of a scientific paper and labels every word as part of the title, subtitle, author names (given and family), author and affiliation marks, affiliations, e-mail, ORCID, abstract, keywords, journal data (journal, volume, issue, pages, ISSN, publisher, date, licence), DOI, or page furniture (running heads, banners, page numbers).

bibr uses it for front matter in its fast mode, where no LLM is called: the title, abstract, keywords and author list come from this tagger, and the affiliation links are resolved from its marks. In full mode a confident tagger answer may stand in for the LLM call, and the LLM keeps the fields the tagger is unsure of. It runs on the CPU through ONNX Runtime; bibr's core install needs no PyTorch.

This repository holds the small model (140M parameters, of which 42M outside the embeddings), stored as fp32 with int8 token embeddings: one model.onnx of 269 MB.

Model

• Architecture: jhu-clsp/mmBERT-small (ModernBERT layers, Gemma 2 tokenizer with 256,000 pieces, revision abc32620) + a gated residual that adds 32 per-word features + a linear layer to 43 BIO tags + a CRF whose parameters are stored in onnx/bibr_onnx.json and decoded in numpy (Viterbi for the tags, forward-backward for the confidence of each tag).
• Input (contract version 2): the text regions of the first pages in reading order, joined with a newline between regions, and for every word 32 floats: where its region sits on the page (relative x, y, width, height, centre offset, relative font size, bold, first on page, reading-order position), word position and shape (upper case, capitalised, digit, punctuation, superscript mark, CJK, length) and the layout label of its region (document title, paragraph title, text, abstract, header, footer, number, footnote, aside, reference, other). The regions and their layout labels come from bibr's layout detector and OCR or text layer; this model does not read page images. A subtoken takes the features of the first word it overlaps; a word no subtoken starts in takes the I- form of the tag of the subtoken that covers it.
• Output: one BIO tag and one posterior per word. bibr's assembler turns the tagged spans back into the printed title, the first printed abstract and keyword list, and the author list with their affiliations.
• Windows: max_length is 4,096 subtokens. A longer input runs in overlapping windows (stride 3/4 of the window); every token keeps the emissions of the window it sits deepest in and one CRF pass decodes the stitched sequence. bibr reads pages 1-2, and re-tags later windows (pages 2-3, then 3-4) when no abstract was found.
• Precision: fp32 with int8 token embeddings. The 256,000-row token-embedding table (70% of this model's weights) is stored as int8 with one fp32 scale per row and expanded right after the lookup (Gather, Cast, Mul; standard opset-17 nodes); every other weight and every computation is fp32. Checked against the fp32 graph on 260 papers outside the training data: this file tags 99.971% of words identically (245 of 260 papers word for word), and the front-matter composite (title match, author-name F1, abstract ROUGE-L) differs by -0.09 points (95% interval -0.30 to +0.04, paired bootstrap over papers). The file is 269 MB; the fp32 graph of the same model is 563 MB.

Training data

3,960 rows train the model, 233 select the checkpoint and 463 are held out for the test below. A row is one paper. The split is made by paper; Spanish, Portuguese, Russian, Ukrainian, Japanese and Indonesian papers are stratified into every split, and roughly two thirds of the rows are English. Every paper is an open-access paper whose own licence allows training a published model:

source	what it adds	where the labels come from
Europe PMC open-access JATS papers and PLOS	English biomedical and life-science papers	the publisher's XML, projected onto the words of the printed page
SciELO	Spanish and Portuguese papers	SciELO's JATS metadata, projected the same way
J-STAGE	Japanese papers	the landing-page metadata
a pool drawn through OpenAlex	psychology, social science, economics, business and other fields; journal articles and preprints	the teacher described below, kept only where it agrees with OpenAlex
Russian and Ukrainian papers drawn through OpenAlex	Cyrillic-script papers	machine labels from a larger (27B) model reading the paper

Licence policy, in plain words. A paper is used only when its own licence is CC BY or CC0 (any version) or, for the OpenAlex pool, CC BY-SA or public domain (about one in eleven pool papers is CC BY-SA, a share-alike licence). Papers licensed non-commercial, no-derivatives, text-and-data-mining-only, with an unspecified "open access" or with no licence statement are dropped. Papers used to develop and evaluate bibr, including its withheld test sets, are excluded by DOI, OpenAlex id and title. No page image or paper text is redistributed with this model.

Scanned papers. This recipe uses no synthetic scans.

How the labels were produced. For the JATS, SciELO and J-STAGE papers the labels are the publisher's own metadata, aligned word by word to the page. For the OpenAlex pool the labels are silver: bibr's own pipeline, with its 4B extraction model (scienceverse/bibr-extract-qwen35-4b) acting as the teacher, read the OCR of the paper, and a paper entered training only when the teacher agreed with its OpenAlex record: title token F1 of at least 0.9, the same number of authors with family-name F1 of at least 0.9 and, where OpenAlex has an abstract, abstract token F1 of at least 0.8. Where they disagree, the paper was left out rather than corrected. Machine labels inherit the teacher's conventions and mistakes, and so does this model.

Evaluation

Held-out test split of the training corpus (463 rows; the same sources as training, so it measures how well the model reproduces its labels, not how well it does on a new kind of paper; part of the labels are silver):

metric	value
word-level F1 (43 tags)	0.965
span F1 over all fields	0.911
span F1, title and authors	0.956
title exactly right	0.959
author list exactly right	0.836 (initial-tolerant 0.838)
papers whose assembled front matter raises no problem	0.801

At bibr's default confidence bar (0.9 lowest posterior on the title and names, mean on the abstract) the tagger answers for the title of 87% of the test papers (exact on 98.0% of those) and for the author list of 70% (right on 91.4% of those); the remaining papers go to the LLM in full mode.

Against bibr's 4B extraction model on papers it was not trained on (front-matter composite: the mean of title match, author-name F1 and abstract ROUGE-L, with a paired bootstrap over papers):

bibr 0.9.0 fast mode with this bundle, minus bibr 0.8.0's 4B extraction model, in points with the 95% interval. A paper with no export scores as empty. These are bibr's development sets: the tagger did not train on them, but bibr's front-matter rules were tuned while looking at them, so read the figures as development results, not a held-out test.

Born-digital papers, text-layer route (no OCR):

set	papers	fast minus 4B
1	200	−0.7 [−3.0, +1.5]
2	60	0.0 [−2.4, +2.4]
3	190	−1.3 [−3.0, +0.4]
4	86	+0.2 [−0.4, +1.1]
5	59	+1.9 [−4.0, +7.8]
all five	595	−0.4 [−1.6, +0.7]

On its own, author-name F1 over all five sets is −1.5 [−3.1, 0.0].

Scanned papers, OCR route, 80 papers: −15.1 [−22.9, −7.8], measured on a pre-release build of 0.9.0. Ten of the 80 gave no export and count as empty; on the 70 that gave one the difference is −4.7 [−9.5, +0.3]. This bundle learned from born-digital pages only. For scans, use full mode with an LLM where you can.

Speed on CPU (ONNX Runtime, 4 threads, front matter of one paper including the later windows when needed):

0.93 s per paper (median; mean 1.03 s, slowest 2.0 s) over 16 papers and 20,705 words, on 4 reserved cores of an AMD Ryzen 7 5700X3D with ONNX Runtime set to 4 threads and nothing else running on those cores. Loading the model takes 1.8 s once; peak memory was 1.6 GB. This is the tagger alone: a whole fast-mode paper on the same 4 cores, layout and reading included, took 31 to 33 s with bibr's default settings (60 papers, run twice), and 28 to 30 s with equation reading and the corpus tables switched off.

Limitations

• Scanned papers are weaker. The model learned from born-digital pages and, for the scan-aware recipe, from synthetic scans; on real scans the abstract and the byline are noticeably less reliable than on born-digital papers. bibr keeps the LLM for a scanned first page when one is configured.
• Abstract position. The tagger reads two pages at a time. An abstract that is printed on page 3 or 4 (title-page manuscripts, preprint cover sheets) is found by bibr re-tagging later windows, not by this model alone.
• Several printed versions. When a paper prints parallel abstracts or keyword lists (for example in two languages), the model tags all of them and bibr's assembler keeps the first printed one.
• Names. Silver labels follow the teacher's conventions for particles, suffixes and middle initials. OCR that glues a superscript mark onto a surname can cost the family name.
• Languages. Trained on English, Spanish, Portuguese, Russian, Ukrainian, Japanese and Indonesian papers (a few more with a handful of examples each). Other scripts and languages are untested, even though the encoder was pretrained on 1,800 languages.
• Not a general tagger. It needs bibr's region features and layout labels as input. Fed plain text it will not reproduce the numbers above.
• No page images. Layout and OCR errors upstream of the tagger (merged or split regions, missing text) show up as tagging errors.

Files

File	Purpose
onnx/model.onnx	input_ids [batch, length], attention_mask [batch, length], token_features [batch, length, 32] to emissions [batch, length, 43] (opset 17)
onnx/bibr_onnx.json	tag set, feature keys, contract version, CRF start / end / transition matrices, encoder revision, precision
onnx/tokenizer.json	the tokenizer of jhu-clsp/mmBERT-small, byte for byte (sha256 197d4cc5406ee12c...)
LICENSE	MIT, with the upstream notice

Using it in bibr

bibr resolves the bundle like its other ONNX models: ML_FRONT_TAGGER_MODEL_ID takes this repo id or a local directory that holds onnx/, ML_FRONT_TAGGER_REVISION pins the commit, and the downloaded bundle is cached in the Hugging Face cache, so a cached bundle also loads offline.

bibr 0.9.0 and later use this repository by default, at a pinned commit, so fast mode needs no setting:

bibr chew --fast paper.pdf

In full mode the tagger stays off unless ML_FRONT_TAGGER_MODE=primary is set.

License

MIT, inherited from jhu-clsp/mmBERT-small. The training papers keep their own licences and are not redistributed.

Citation

The encoder is mmBERT:

@misc{marone2025mmbertmodernmultilingualencoder,
  title={mmBERT: A Modern Multilingual Encoder with Annealed Language Learning},
  author={Marc Marone and Orion Weller and William Fleshman and Eugene Yang and Dawn Lawrie and Benjamin Van Durme},
  year={2025},
  eprint={2509.06888},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2509.06888}
}