software-mansion/react-native-executorch-llama-3.2

llama-3.2

This repository hosts the llama-3.2 models exported for the React Native ExecuTorch library as ExecuTorch .pte programs, ready to run on device.

Variants

Path	Backend	Precision
1b/mlx/llama_3_2_1b_mlx_int4.pte	mlx	4w
1b/xnnpack/llama_3_2_1b_xnnpack_bf16.pte	xnnpack	bf16
1b/xnnpack/llama_3_2_1b_xnnpack_spinquant.pte	xnnpack	spinquant
3b/mlx/llama_3_2_3b_mlx_int4.pte	mlx	4w
3b/xnnpack/llama_3_2_3b_xnnpack_bf16.pte	xnnpack	bf16
3b/xnnpack/llama_3_2_3b_xnnpack_spinquant.pte	xnnpack	spinquant

Repository structure

1b/mlx/config.json                             1.3 kB
1b/mlx/llama_3_2_1b_mlx_int4.pte               1.1 GB
1b/xnnpack/config.json                         2.3 kB
1b/xnnpack/llama_3_2_1b_xnnpack_bf16.pte       2.3 GB
1b/xnnpack/llama_3_2_1b_xnnpack_spinquant.pte  1.1 GB
3b/mlx/config.json                             1.3 kB
3b/mlx/llama_3_2_3b_mlx_int4.pte               2.3 GB
3b/xnnpack/config.json                         2.3 kB
3b/xnnpack/llama_3_2_3b_xnnpack_bf16.pte       6.0 GB
3b/xnnpack/llama_3_2_3b_xnnpack_spinquant.pte  2.4 GB
config.json                                    31 B
tokenizer.json                                 9.4 MB
tokenizer_config.json                          53.2 kB

Compatibility

These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.

To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.

To load these files in your own ExecuTorch runtime, read the compatibility note first.

Model details

Upstream model: Llama 3.2 collection.

Both the 1B and 3B sizes are published, each as bf16 and as SpinQuant, the quantization Meta trained for on-device use.

Model details

Upstream model: Llama 3.2 collection.

Both the 1B and 3B sizes are published in three forms: XNNPACK bf16, XNNPACK SpinQuant, the quantization Meta trained for on-device use, and MLX int4.

The MLX builds quantize from the bf16 weights rather than loading a pre-quantized checkpoint, so they are independent of SpinQuant rather than derived from it.