GGUFs models for local AI Inference quantized with MACaQ (https://github.com/huawei-csl/MACaQ)
HUAWEI Computing Systems Lab
company
Verified
AI & ML interests
None defined yet.
Recent Activity
Papers
MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
This collection contains SINQ GGUF models
-
AC-LoRA: (Almost) Training-Free Access Control-Aware Multi-Modal LLMs
Paper • 2505.11557 • Published • 7 -
SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights
Paper • 2509.22944 • Published • 81 -
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
Paper • 2606.03458 • Published • 61
GGUFs models for local AI Inference quantized with MACaQ (https://github.com/huawei-csl/MACaQ)
This collection contains SINQ GGUF models
This collection contains the models quantized with the SINQ quantization method.
-
AC-LoRA: (Almost) Training-Free Access Control-Aware Multi-Modal LLMs
Paper • 2505.11557 • Published • 7 -
SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights
Paper • 2509.22944 • Published • 81 -
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
Paper • 2606.03458 • Published • 61