Idol on LiteRT: one photo to a 3D Gaussian splat on the phone

Idol (speridlabs/idol with the idol-turbo LoRA) as LiteRT graphs for CompiledModel, converted with litert-torch: on a Galaxy S26, the Android sample in android/ turns one photo into a 3D Gaussian splat of 262,144 Gaussians in 303 s (the first photo after start on a cool phone, GPU fp16 with fp32 accumulation, LiteRT 2.3.0, measured 2026-11-09).

speridlabs/idol turns one photo of an object into a 3D Gaussian splat of 262,144 Gaussians that you can turn around, including the sides the picture never showed. This repo holds the author's denoiser with the speridlabs/idol-turbo LoRA (5 steps, no guidance), its octree and Gaussian decoders, the FLUX.2 VAE encoder, and the author's BiRefNet background remover. The DINOv3 image encoder is in a separate, gated repo under its own licence: litert-community/DINOv3-ViT-H16plus-Idol-LiteRT.

Left: the photo by Komet Flicker (Pexels License). Right: the splat the sample app made from it on a Galaxy S26, rendered from the front, the side and the back.

Speed

On a Galaxy S26, the sample app turns one photo into a splat of 262,144 Gaussians in 303 s, starting from a cool phone (the first photo after start). The phone throttles its GPU during that photo: the GPU clock cap fell from 1,300 to 422 MHz within its 5 minutes (thermal status 1 → 3). Photos taken one after another on a phone that was already warm (thermal status 2) took 339–449 s. The graphs do not all fit in the phone's memory at once, so the app closes graphs and compiles them again when the run comes back to them: about 115 compiles per photo, 70 s of the first one. Background removal (BiRefNet) runs on the CPU: 11.1 s of that time on the Galaxy S26. The other networks run on the GPU.

Speed still being tuned (updates follow).

Galaxy S26 (SM-S942Q), the sample app, LiteRT 2.3.0; from a cool phone (GPU clock cap 1,300 MHz, thermal status 1 at the start)	first photo after start
photo → .ply written	303 s
background removal (BiRefNet, CPU)	11.1 s
DINOv3 encoder	6.3 s
VAE encoder	3.7 s
denoiser, 5 steps	228.4 s
octree decoder	16.2 s
Gaussian decoder	36.0 s
PLY writer	0.2 s
graphs compiled during the photo	115 (70 s)
peak resident memory (VmHWM)	4.71 GiB
peak GPU memory (kgsl)	5.60 GiB

Galaxy S26 (SM-S942Q), the sample app, LiteRT 2.3.0; thermal status 2 (moderate) from the first photo	first photo after start	photo 2	photo 3
photo → .ply written	339 s	416 s	414 s
background removal (BiRefNet, CPU)	8.5 s	17.7 s	15.9 s
DINOv3 encoder	8.8 s	31.0 s	26.7 s
VAE encoder	5.4 s	8.5 s	7.3 s
denoiser, 5 steps	262.9 s	306.9 s	311.9 s
octree decoder	16.0 s	15.1 s	15.3 s
Gaussian decoder	35.6 s	35.7 s	36.2 s
PLY writer	0.2 s	0.1 s	0.1 s
graphs compiled during the photo	115 (78 s)	114 (97 s)	116 (93 s)
peak resident memory (VmHWM)	4.91 GiB	4.16 GiB	4.09 GiB
peak GPU memory (kgsl)	5.60 GiB	5.33 GiB	5.45 GiB

How close it is to the author's model

Each check runs the author's PyTorch pipeline (float32, CPU) and these graphs on the same input with the same random numbers, against thresholds fixed before the first run. The inputs are two images from the author's gallery (nb-armchair, fig-axolotl) and one photo (a succulent, Pexels). Per part, each graph gets the author's own input for that part and its output is compared with the author's (cosine; the threshold is 0.99). The whole pipeline is compared by rendering both splats from four sides with the same renderer (PSNR; the threshold is 25 dB).

Per-part agreement on the Mac (LiteRT GPU, Metal, fp16): cosine vs the author's model

Part	Compared tensor	armchair	axolotl	succulent
DINOv3 encoder	feature1	0.99989	0.99994	0.99992
VAE encoder	latent tokens	0.99995	0.99993	0.99996
Denoiser (5 steps)	latent velocity, worst step	0.99931	0.99923	0.99931
Octree decoder (8 levels)	child probabilities, worst level	0.99946	0.99937	0.99943
Gaussian decoder	xyz	1.00000	1.00000	1.00000
	colour (f_dc)	0.99970	0.99963	0.99976
	scale	0.99833	0.99837	0.99652
	rotation	0.99891	0.99915	0.99901
	opacity	0.99987	0.99871	0.99988

Per-part agreement on a Galaxy S26 (the app; GPU fp16 with fp32 accumulation, BiRefNet on the CPU): cosine vs the author's model

Part	Compared tensor	armchair
DINOv3 encoder	feature1	1.00000
VAE encoder	latent tokens	0.99999
Denoiser (5 steps)	latent velocity, worst step	0.99999
Octree decoder (8 levels)	child probabilities, worst level	0.99994
Gaussian decoder	xyz	1.00000
	colour (f_dc)	0.99997
	scale	0.99990
	rotation	0.99986
	opacity	0.99993

Per-part agreement on a Pixel 11 Pro (the app; GPU fp16 with fp32 accumulation, BiRefNet on the CPU): cosine vs the author's model

Part	Compared tensor	armchair
DINOv3 encoder	feature1	0.99998
VAE encoder	latent tokens	0.99999
Denoiser (5 steps)	latent velocity, worst step	0.99989
Octree decoder (8 levels)	child probabilities, worst level	0.99970
Gaussian decoder	xyz	1.00000
	colour (f_dc)	0.99989
	scale	0.99964
	rotation	0.99976
	opacity	0.99998

Background removal

Input	Where	uint8 alpha pixels that differ from the author's	largest difference	crop square (author / here)
armchair	Mac, LiteRT CPU	2,544 of 1,048,576	1 / 255	same
armchair	Galaxy S26, the app	2,542 of 1,048,576	1 / 255	same
armchair	Pixel 11 Pro, the app	2,543 of 1,048,576	1 / 255	same
axolotl	Mac, LiteRT CPU	709 of 1,048,576	1 / 255	same
succulent	Mac, LiteRT CPU	6,115 of 1,048,576	1 / 255	same

Whole pipeline: 4-view render PSNR

Input	Where	PSNR, 4 views pooled (threshold: 25 dB)	per view: 0°, 90°, 180°, 270°
armchair	Mac, LiteRT GPU (Metal, fp16)	35.51 dB	34.86, 37.92, 34.15, 36.00
armchair	Galaxy S26, the app	46.59 dB	46.53, 45.59, 46.41, 48.25
axolotl	Mac, LiteRT GPU (Metal, fp16)	32.21 dB	33.85, 29.70, 34.58, 32.43
axolotl	Galaxy S26, the app	42.34 dB	40.72, 44.21, 40.59, 46.32
succulent	Mac, LiteRT GPU (Metal, fp16)	36.38 dB	35.41, 36.84, 36.14, 37.39
succulent	Galaxy S26, the app	41.43 dB	40.30, 41.09, 42.18, 42.51

Minimal usage

Python: the whole pipeline

hf download litert-community/Idol --local-dir Idol
cd Idol
pip install -r python/requirements.txt
python python/run_idol.py photo.jpg --out out --models .   # writes out/gaussian.ply

run_idol.py downloads the DINOv3 repo (accept its licence on the Hub and hf auth login first) and runs every graph with CompiledModel, the GPU for all but BiRefNet. The steps between graphs are in python/idol_runner.py and python/idol_glue.py. On an M4 Max Mac (Metal GPU) one photo took 64 s and up to 22 GB of memory.

Python: one graph with CompiledModel (background removal)

import sys

import numpy as np
from PIL import Image
from ai_edge_litert.compiled_model import CompiledModel
from ai_edge_litert.hardware_accelerator import HardwareAccelerator
from ai_edge_litert.options import Options

path, photo = sys.argv[1], sys.argv[2]   # bir.tflite from this repo, any photo
model = CompiledModel.from_file(path, options=Options(hardware_accelerators=HardwareAccelerator.CPU))
img = Image.open(photo).convert("RGB")
x = np.asarray(img.resize((1024, 1024), Image.BILINEAR), np.float32) / 255
x = ((x - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]).transpose(2, 0, 1)[None].astype(np.float32)
inputs, outputs = model.create_input_buffers(0), model.create_output_buffers(0)
inputs[0].write(x.reshape(-1))                       # [1, 3, 1024, 1024]
model.run_by_index(0, inputs, outputs)
alpha = outputs[0].read(1024 * 1024, np.float32).reshape(1024, 1024)   # 0..1, 1 = the object
mask = Image.fromarray((alpha * 255).astype(np.uint8)).resize(img.size, Image.BILINEAR)
img.putalpha(mask)
img.save("cutout.png")
print(alpha.min(), alpha.max(), (alpha > 0.5).mean())

Kotlin (Android)

val env = Environment.create()
val cpu = CompiledModel.Options(Accelerator.CPU)            // bir.tflite: BiRefNet, the one CPU graph
val bir = CompiledModel.create(File(dir, "bir.tflite").absolutePath, cpu, env)
val inputs = bir.createInputBuffers()
val outputs = bir.createOutputBuffers()
inputs[0].writeFloat(image)        // [1, 3, 1024, 1024]: RGB / 255, ImageNet mean / std
bir.run(inputs, outputs)
val alpha = outputs[0].readFloat() // [1024 * 1024] sigmoid map, 1 = the object
for (b in inputs + outputs) b.close()
bir.close()
val gpu = CompiledModel.Options(Accelerator.GPU).apply {   // every other graph
    gpuOptions = CompiledModel.GpuOptions(precision = CompiledModel.GpuOptions.Precision.FP16_WITH_FP32_ACCUM)
}

The Android sample app (photo or camera → progress per stage → a splat you can turn with a finger) is in android/. The graphs are too large for the APK; a script copies them into the app's files folder over adb:

hf download litert-community/Idol --local-dir ~/Downloads/Idol
hf download litert-community/DINOv3-ViT-H16plus-Idol-LiteRT --local-dir ~/Downloads/DINOv3-ViT-H16plus-Idol-LiteRT
cd ~/Downloads/Idol/android
chmod +x gradlew scripts/install_to_device.sh        # the Hub does not keep the executable bit
./gradlew :app:installDebug
IDOL_DIR=~/Downloads/Idol DINO_DIR=~/Downloads/DINOv3-ViT-H16plus-Idol-LiteRT ./scripts/install_to_device.sh

Background removal (BiRefNet on the CPU) needs about 2,670 MB of free memory for one call. Before compiling it, the app closes GPU graphs until that much is free on top of its 2,500 MB reserve; if other apps hold the rest, it stops with "Not enough memory for background removal (bir) … Close other apps and try again." before starting the call.

What runs where

Stage	Files	Runs on	Calls per photo
Background removal (BiRefNet)	bir.tflite	CPU	1
DINOv3 ViT-H+/16 image encoder	dino_0..3.tflite (DINOv3 repo)	GPU	4
FLUX.2 VAE encoder	vae_0..3.tflite, vae_attn.tflite, vae_4.tflite	GPU	6
Denoiser with the turbo LoRA	den_ctx.tflite, den_in.tflite, den_joint_00..23.tflite	GPU	5 steps × 25, plus den_ctx once per LoRA phase
Octree decoder	oct.tflite (5 signatures by row count)	GPU	8 levels
Gaussian decoder	gs_0..7.tflite	GPU	8
Octree sampling, Gaussian activation, PLY	host code	CPU	—

manifest.json lists every graph with its signatures, accelerator and precision. On Android the GPU graphs run with FP16_WITH_FP32_ACCUM precision.

Notes

• Background removal runs on the CPU. BiRefNet's 20 deformable convolutions sample at positions computed from the image. They are rewritten as a gather with bilinear weights and one matmul (same arithmetic; 5.5e-6 from the author's map in PyTorch), and the LiteRT GPU has no kernel for that gather. The gathers run in row tiles one after another: each tile's offsets wait for the previous tile's result. Without that wait the CPU runtime ran every tile's gather before using any of them; the first export ran out of memory on the Galaxy S26, and this one peaks at 3.96 GiB of resident memory (VmHWM) in one call on it.
• The VAE's GroupNorm scales each group by its largest |x − mean| to stay in fp16 range. The graph finds that maximum as a max pool, then a max over the 32 × 32 result (the same value). One REDUCE_MAX over the whole 1024 × 1024 map took 8 s per GPU call on the Galaxy S26; this form takes 0.83 s.
• Attention over 4,096 keys or more is summed in chunks. The product of the attention weights and the values is summed over key chunks of 1,024 after centring the values (exact, since each softmax row sums to 1). One fp16 sum over 8,192 keys lost 18 % in the Gaussian decoder's first block; the chunked form loses 0.6 %. Keep it if you re-export.
• Keep each GPU call short on the Galaxy S26. In the feasibility run the VAE encoder ran as one graph, about 9 s per GPU call; the screen compositor stalled and the process was killed (signal 9) on its fifth call. Every stage here is split into several graphs for that reason (on the Galaxy S26 every GPU call takes 0.04–1.83 s: 45 graphs, 49 signatures).

Licence and provenance

Part	Weights	Licence
Denoiser, octree decoder, Gaussian decoder	speridlabs/idol	Apache-2.0
Turbo LoRA (applied to the denoiser graphs)	speridlabs/idol-turbo	Apache-2.0
FLUX.2 VAE encoder	black-forest-labs/FLUX.2-klein-4B vae/	Apache-2.0
BiRefNet background remover	birefnet.safetensors in speridlabs/idol (BiRefNet)	MIT
DINOv3 ViT-H+/16 (separate repo)	facebook/dinov3-vith16plus-pretrain-lvd1689m, read from the bf16 copy Idol loads (Comfy-Org/Idol clip_vision/dino_v3_vit_h_bf16.safetensors)	DINOv3 License

The graphs here are Apache-2.0 except bir.tflite (MIT). LICENSES.md records where each licence was read.