Bocklitz-Lab/lit2vec-subfield-classifier-model

Lit2Vec Subfield Classifier (MLP)

A multi-label classifier that assigns chemistry abstracts and summaries to one or more of 18 subfields. It is a small Keras MLP that works on top of intfloat/e5-large-v2 embeddings and is used to produce the predicted_subfield field of the Lit2Vec workflow (paper, code). Training code: lit2vec-subfield-classifier. Dataset: lit2vec-subfield-classifier-dataset. Demo: Space.

• Input: a 1,024-dimensional, L2-normalised e5-large-v2 embedding of the text abstract: <text> or summary: <text>
• Output: 18 sigmoid scores, one per subfield; a label is assigned when its score is above 0.5
• Not accepted: raw text. You have to embed it first, as shown below.

Labels

ID	Subfield
0	Catalysis
1	Organic Chemistry
2	Polymer Chemistry
3	Inorganic Chemistry
4	Materials Science
5	Analytical Chemistry
6	Physical Chemistry
7	Biochemistry
8	Environmental Chemistry
9	Energy Chemistry
10	Medicinal Chemistry
11	Chemical Engineering
12	Supramolecular Chemistry
13	Radiochemistry & Nuclear Chemistry
14	Forensic & Legal Chemistry
15	Food Chemistry
16	Chemical Education
17	Others

The mapping is also in label_mapping.json. The training labels were generated by GPT-4o from a controlled vocabulary. They were not curated by experts (see Limitations).

Quick start

# pip install -r requirements.txt
import json
import numpy as np
from huggingface_hub import hf_hub_download
from tensorflow import keras
from sentence_transformers import SentenceTransformer

REPO_ID = "Bocklitz-Lab/lit2vec-subfield-classifier-model"
EMBED_MODEL = "intfloat/e5-large-v2"
TEXT_PREFIX = {"abstract": "abstract: ", "summary": "summary: "}  # the prefixes used for the training embeddings
THRESHOLD = 0.5

model = keras.models.load_model(hf_hub_download(REPO_ID, "mlp_model.h5"), compile=False)
with open(hf_hub_download(REPO_ID, "label_mapping.json"), encoding="utf-8") as f:
    index_to_label = {int(k): v for k, v in json.load(f)["index_to_label"].items()}
encoder = SentenceTransformer(EMBED_MODEL)

def predict(text: str, text_type: str = "abstract", threshold: float = THRESHOLD):
    x = encoder.encode([TEXT_PREFIX[text_type] + text], normalize_embeddings=True).astype("float32")
    probs = model.predict(x, verbose=0)[0]
    labels = [index_to_label[i] for i, p in enumerate(probs) if p > threshold]
    return labels, probs

labels, probs = predict("The adsorption capacity of Helix aspera shell for Pb2+, Zn2+ and Ni2+ has been studied ...")
print(labels)

The text prefix matters. The embeddings in the training dataset were computed with abstract: for abstracts and summary: for summaries. If you embed text without the prefix, or with e5's usual passage: , the vectors are close but not identical (cosine similarity of about 0.998), and scores can shift slightly.

For many texts, embed them in a batch and call model.predict(X) on the stacked array.

Training

Framework	TensorFlow / Keras 2.15
Architecture	Dense(256, ReLU) - BatchNorm - Dropout(0.3), twice, then Dense(18, sigmoid); 334,866 parameters
Loss	Binary cross-entropy weighted per class by (negatives / positives) of the training split
Optimiser	Adam, learning rate 1e-3, batch size 32, at most 50 epochs
Callbacks	EarlyStopping on validation loss (patience 5, best weights restored), ReduceLROnPlateau (factor 0.5, patience 3)
Run	stopped after 17 epochs; the weights of epoch 12 (lowest validation loss) are the ones in this repository
Model selection	5-fold cross-validation on train and validation, then one final training on the official splits
Seed	42

The full settings are in config.yaml, and training_history.json holds the curves of the final run.

Evaluation

Validation metrics at epoch 12: loss 1.064, binary accuracy 0.964, ROC-AUC 0.972, PR-AUC 0.854 (macro-averaged over the labels, as computed by Keras).

Test set (3,990 records, threshold 0.5):

Metric	Value
Micro F1	0.808
Macro F1	0.748
Weighted F1	0.800
Samples F1	0.804
Micro precision / recall	0.854 / 0.766
Exact-match accuracy	0.508
ROC-AUC (micro / macro)	0.984 / 0.978
PR-AUC (micro / macro)	0.901 / 0.843

We recomputed these numbers from the weights in this repository and the test split of the dataset, and they match the evaluation outputs in the training repository.

The test split overlaps with the training split

Every paper in the dataset appears twice, once with its abstract and once with its GPT-4o summary, and both records carry the same labels. The dataset was split randomly by record, not by paper. As a result, 3,573 of the 3,990 test records (90%) have the other text of the same paper in the training or validation split. The scores above are therefore somewhat optimistic. On test records whose paper does not occur in the training split the model scores:

Test records	n	Micro F1	Macro F1	Weighted F1
All	3,990	0.808	0.748	0.800
Paper not in the training split	838	0.804	0.707	0.794
Paper not in training or validation	417	0.790	0.680	0.775

The micro-averaged scores change little. Macro F1 drops more, partly because the rare classes have only 4 to 6 examples in the smaller subsets. For an estimate of how the model behaves on new papers, use the second row.

F1 per label (full test set)

Subfield	F1	Support
Catalysis	0.80	197
Organic Chemistry	0.70	245
Polymer Chemistry	0.72	120
Inorganic Chemistry	0.71	203
Materials Science	0.80	917
Analytical Chemistry	0.71	633
Physical Chemistry	0.63	240
Biochemistry	0.92	2,106
Environmental Chemistry	0.79	508
Energy Chemistry	0.79	166
Medicinal Chemistry	0.82	1,343
Chemical Engineering	0.53	413
Supramolecular Chemistry	0.68	34
Radiochemistry & Nuclear Chemistry	0.65	20
Forensic & Legal Chemistry	0.70	16
Food Chemistry	0.83	282
Chemical Education	0.85	20
Others	0.83	19

Support is the number of test records that carry the label. The model is strongest on frequent classes such as Biochemistry, Medicinal Chemistry and Materials Science, and weakest on Chemical Engineering and Physical Chemistry.

Intended use

Tagging chemistry abstracts and summaries with broad subfields for browsing, filtering and retrieval, for example the subfield labels in the Lit2Vec corpus.

Limitations

• Labels are model-generated. The training labels come from GPT-4o. Two domain experts reviewed a random sample of 45 annotated abstracts and found them generally reasonable (see the paper). The classifier learns the labelling habits and the errors of GPT-4o, so treat the output as navigation metadata and not as ground truth.
• Rare classes (Supramolecular, Radiochemistry, Forensic, Chemical Education, Others) have very few examples, and their scores are noisy.
• Overlapping splits. See the section above.
• Domain. It was trained on chemistry abstracts and summaries. It may not work on other fields or on long full text.
• Embedding space. It only works with the e5-large-v2 embedding described above.

Files

• mlp_model.h5: the Keras model
• label_mapping.json: label name and ID mapping
• training_history.json: training and validation curves of the final run
• config.yaml: training settings
• requirements.txt: packages for the quick start

Citation

Please cite the Lit2Vec paper:

@misc{amiri2026lit2vec,
  title         = {Lit2Vec: A Reproducible Workflow for Building a Legally Screened Chemistry Corpus from S2ORC for Downstream Retrieval and Text Mining},
  author        = {Amiri, Mahmoud and Jafari, Jamile Mohammad and Mostafapour, Sara and Bocklitz, Thomas},
  year          = {2026},
  eprint        = {2604.12498},
  archivePrefix = {arXiv},
  doi           = {10.48550/arXiv.2604.12498},
  url           = {https://arxiv.org/abs/2604.12498}
}

License

The model weights are released under Apache-2.0. The training dataset is released under CC BY 4.0. Its labels and summaries were generated with GPT-4o, so check OpenAI's terms of use if you use them to train other models.

Acknowledgements

The embeddings come from intfloat/e5-large-v2. The abstracts come from S2ORC (Allen Institute for AI and Semantic Scholar). The work was developed in the Bocklitz Lab at the Leibniz Institute of Photonic Technology (Leibniz IPHT) and Friedrich Schiller University Jena and was funded by the BMFTR within the programme Photonics Research Germany (13N15466, 13N15710, 13N15715). Questions: mahmoud.amiri@uni-jena.de.