Hugging Face
Search models, datasets, users...
Ctrl K
Models
Datasets
Spaces
Buckets
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Hardware
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
None
CompressedGemma
4
Follow
nakwMt's profile picture
emanubiz's profile picture
osasuio's profile picture
5 followers
·
6 following
AI & ML interests
None yet
Recent Activity
updated
a model
about 9 hours ago
CompressedGemma/SMQ-Quantize-0.1
reacted
to
Ternii
's
post
with 🔥
1 day ago
llama.cpp-tr r4 is out: our trit-quantised models now run on CUDA as well as Metal. New in r4: - A Linux arm64 build with CUDA for the DGX Spark, alongside the macOS build. On a DGX Spark, prompt processing at 2,048 tokens went from 1,387 to 1,924 tokens/s on Qwen3.6-35B-A3B (×1.39) and from 607 to 806 on Qwen3.8-Flash-Next (×1.33), compared with the build's plain CUDA kernels. - On the Mac, a Metal kernel for 2–8 tokens at a time (several users decoding together). With 2–8 users, total decode is 8–24 % faster; with one user it is level (no speed-up). Answers are identical to r3 on our 60-prompt check. (Corrected 22 Nov: this line first said the kernel speeds up speculative-decoding checks; with one user those are ~65-token passes that use a different kernel.) - x86 (AVX2) support in the source; build it yourself with `apply.sh`. No x86 binary yet. Accuracy: the CUDA tensor-core prompt path changes outputs slightly. We measured it against the path switched off (`GGML_CUDA_TQ_MM=0`) and found no significant difference: Qwen3.6-35B-A3B GSM8K 90.83 vs 90.60, HumanEval+ 91.46 vs 92.68, MBPP+ 91.53 vs 91.27; Qwen3.8-Flash-Next GSM8K 92.80 vs 93.25 (n = 1,319). Caveats: CUDA is tested on a DGX Spark (GB10) only. The one-line Mac installers stay on r3. The builds need our patched llama.cpp; LM Studio and Ollama can't load these models. Release: https://github.com/ternii-ai/tritq-vllm/releases/tag/llama.cpp-tr-b11721-r4 Models: https://huggingface.co/Ternii/Qwen3.6-35B-A3B-TR2.25-G-GGUF · https://huggingface.co/Ternii/Qwen3.8-Flash-Next-TR2.31-G-GGUF
published
a model
6 days ago
CompressedGemma/SMQ-Quantize-0.1
View all activity
Organizations
None yet
CompressedGemma
's datasets
None public yet