Delete those GGUF
Stop encouraging inferior GGUFs. GGUF format is not meant for AI image and video models, leading to degraded loss. INT8 Convrot will run on a 6GB VRAM card with ComfyUI. It is the standard now.
You should delete the GGUF models.
Agreed, these broken gguf's that do not support all features are not helpful. Besides I appreciate VRAi for the release it makes no sense to release broken checkpoint files.
GGUF is just a container, my man. The way it is dequantized depends on the inference backend.
It can run as fast as int8_convrot quant if you want. But comfyanonymous doesn't want to support it.
GGUF is just a container, my man. The way it is dequantized depends on the inference backend.
It can run as fast as int8_convrot quant if you want. But comfyanonymous doesn't want to support it.
Unfortunately, it is not "just" a container. The way it can hold the quanting of keys is different and is not the same architecture like safetensors do. It is fundamentally different and that's why the gguf lacks the features.
I'm here for the GGUFs :D
GGUF is just a container, my man. The way it is dequantized depends on the inference backend.
It can run as fast as int8_convrot quant if you want. But comfyanonymous doesn't want to support it.
GGUF has no meaning to Blackwell users and yes, its quality is degraded as a lower performing quantization compared to an INT8 Convrot. The only people who complain to keep GGUF's are the Ampere users.