MDFLD v3: cartoon face landmark detection (68 points)

Model weights for Towards Multi-Domain Face Landmark Detection with Synthetic Data from Diffusion Model (Yuanming Li, Gwantae Kim, Jeong-gi Kwak, Bon-hwa Ku, Hanseok Ko; ICASSP 2024).

Paper (arXiv:2401.13191) · Code · Project page

v3 (default in the mdfld package) is the v2 network with frozen DINOv3 ViT-B/16 features added to the input of its hourglass stack through a small adapter, trained with more cartoon data. With the package's three-pass crop:

Test set	v1 (paper)	v2	v3
CariFace test, 1,560 caricatures: NME (%)	6.70	5.03	4.64
CariFace test: faces with NME above 10%	–	3.6%	2.0%
Real faces, 313 held-out video frames (automatic labels): NME (%)	3.03	1.74	1.69
Anime illustrations, 980 held-out images: disagreement above 15% with an anime face detector	–	10.5%	6.1%

v3 is larger (419 MB vs 69 MB) and slower (71 vs 38 ms per face on an RTX 4090 D; 1.8 vs 0.4 s on 8 CPU threads). v2 stays available as the smaller, faster model. See the GitHub README for all regions, the protocol, the training data and the limitations.

Use

The easiest way is the mdfld package, which downloads the model, finds the face and adds eyelid curves (--model v2 for v2):

pip install "mdfld[sam] @ git+https://github.com/zenonLEE/MDFLD"
mdfld image face.png --out face.json --vis face_mdfld.png

Both TorchScript files can also be used directly with PyTorch, without any code from the repository:

import torch
net = torch.jit.load("mdfld_cartoon_v3.torchscript.pt", map_location="cuda")   # or mdfld_cartoon_v2.torchscript.pt
# x: (N, 3, 256, 256) float tensor of BGR face crops, values pixel / 255 * 2 - 1
with torch.no_grad(), torch.jit.fuser("none"):   # without fusers the first call on a GPU takes ~1 s instead of 20-30 s
    out = net(x)                    # (N, 68, 2), x and y in [-1, 1]
pts = ((out + 1) * 256 - 1) / 2     # pixels of the 256 x 256 crop

The crop is a square centred on the face box with side equal to the larger side of the box, resized to 256 × 256. For the best accuracy, run two more passes, each on a square of 1.1 × the larger side of the box of the previous points, as the package does (mdfld/landmarks.py). The 68 points follow the iBUG 300-W order (0-based).

Files

File	Use
mdfld_cartoon_v3.torchscript.pt	v3, TorchScript (fp32, 419 MB); contains DINOv3 ViT-B/16 weights
mdfld_cartoon_v2.torchscript.pt	v2, TorchScript (69 MB)
LICENSE_DINOv3.md	DINOv3 License, which also applies to the v3 file

The same weights in the original checkpoint format (mdfld_cartoon_v2.pkl, for the STAR-based code in the repository) are in the v2.0 release.

Terms of use

Research use only, without warranty. The models were trained on third-party data with their own terms (CariFace is for research only; several cartoon sets contain third-party artwork). The v3 file contains DINOv3 weights and is distributed under the DINOv3 License, including its trade-control restrictions; if you publish results obtained with v3, that license requires acknowledging DINOv3 (Siméoni et al., 2025). See the terms of use.

Citation

@inproceedings{li2024towards,
  title={Towards Multi-Domain Face Landmark Detection with Synthetic Data from Diffusion Model},
  author={Li, Yuanming and Kim, Gwantae and Kwak, Jeong-gi and Ku, Bon-hwa and Ko, Hanseok},
  booktitle={ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={6730--6734},
  year={2024},
  organization={IEEE}
}

Teaser images: public-domain works from Wikimedia Commons and frames of Sintel (© copyright Blender Foundation | durian.blender.org, CC BY 3.0).