Bot PRO

GGUF used to have a use case in Comfy, but Comfy Kitchen is rapidly beating it. Some other non-Comfy apps do also use GGUF though, and it makes sense because GGUF is generally a really nice standard! A friend of mine uses sd.cpp (for their agent to generate images without a UI) and it's actually nice there, because that has good kernels for it!

The 4-bit floor for DiT weights is the claim I'd want a number on.

I honestly consider 4 bits to be the floor for most things. For image models, especially step-distilled ones, it's possible lower precisions work decently well. I'm a bit privileged in that I've never had to bother with it before. Where low precision really starts to sting is with video models. (Which is a shame because that's exactly where you'd want it to work well!)

Comfy Kitchen's AsymInt8 layouts use fp8 group scales and a per-tensor Lloyd-Max codebook to dequantize to grouped INT8, which is accelerated on everything going back to Turing, where NVFP4 is only fast on Blackwell.

The a8 half worries me more than the w4... Do Comfy's w4a8 kernels use one static activation scale for every step, or a dynamic one?

It's handled dynamically in the kernel, yes. It's not calibrated, but neither is NVFP4, and the point is that it's easy to work with and fast on the cards people actually own.

In my personal experience, NVFP4 makes worse videos than w4a8 and takes way longer to do it. int8_convrot was rapidly taken up by the ComfyUI crowd for good reason, it's just superior to fp8_scaled in every way unless you're on a truly ancient GPU.

There's almost definitely better quantization strategies out there (I've seen some uptake of GPTQ which is interesting!) but AsymInt8 is some excellent work for what ComfyUI in particular needs.

Using GGUF in Comfy, on the other hand, bypasses all of that kernel work, doesn't work with Aimdo (at least not when I last tried it), complicates workflows and is overall just a worse user experience. I'm not sure anything really stops anybody from shipping AsymInt8-type layouts in the GGUF container format, but nobody does, so it's moot, and it wouldn't run through Comfy Kitchen's kernel.

I could probably get some updated numbers, though. I just remember using GGUF for LTX-2.3, being aggravated by the fact that it turned my PC into sludge, and then getting the FP8 on a lark and being like "what the hell, it actually works properly now." Maybe the situation's improved with GGUF since. I just don't see the point.

Thank you for coming to my TED talk

Nothing I'm just remarking on the rather poor quality of the public posts feed :P
Don't take me too seriously I'm just saying words recreationally

Be nice if they disallowed glazing ridiculous slop that nobody cares about too, but at least we have that!

Good stuff. That actually sounds kind of interesting.

Just the least you could do is write the model card yourselves such that you catch these things. I only continued reading because I was like "wait I think I might see the vision" but nothing came of it. That's why I got so frustrated. The least we can do for each other is read the things we post. I'd be pretty disappointed if the model card sold my work short so hard.

I acknowledge I went a little far, it's just tiring to be in this space sometimes.

Looking forward to the detailed report.

Thanks GPT-4o

Oh my god I completely missed this.

YOU'RE ALREADY USING MERGEKIT????????

Did you even read the documentation for the tools you're using????

I actually shouted at my screen and then into my hands after I saw this. You're KILLING Me, guys!!

ok i'm back so like not a single word of this is meaningful in any way and is riddled with factual errors in the claims it does make

first off, evolutionary merging as a concept isn't new, mergekit can already do this

There is no way you merged two models of different architectures and got a positive result. If the "mother" were "text only", it would definitionally have to also be multimodal. Otherwise it's not Qwen3.5. And there is no architecture that is even remotely compatible with Qwen3.5's, the DeltaNet attention heads see to this fact. If what you're saying is true, then you're splicing a cat's brain into a dog's (or, to be somewhat reasonable, a cat's brain into an ocelot's.) This is all I needed but there's more actually.

201 Languages ❌ Potentially degraded ✅ Inherited from Father

You presume. Any amount of finetuning is going to result in specialization in the target area. 201 languages are going to necessarily be represented across the model in a way that layer-wise merging can't preserve without other techniques that are already implemented in a mathematical way (tl;dr Task Arithmetic Works.)

Benchmark Transparency ❌ No scores published ✅ Fully open

Your "mother" is a community finetune. This is vexatious language that degrades a hobbyist project as being "opaque." Not that a human wrote this model card and I don't even need to ask Pangram about that.

Model MRI Integration — CT-scans parent models layer by layer before merging, guiding evolution with structural insight
If conventional merging is "mixing recipes blindfolded," Darwin V5 is "precision surgery with X-ray guidance."

Extremely waffly use of medical terminology with no technical definition whatsoever in this context, neither extant nor provided.

Traditional model merging relies on humans setting hyperparameters like ratio and density by intuition. Set ratio=0.5, density=0.9, run once, and hope for the best. The result depends on luck, and applying the same ratio uniformly across billions of parameters ignores each layer's unique role.

Laughably wrong. The entire point of merging is that iteration is fast. "By intuition" is not a meaningful critique because intuition is the only way that humans do this. "and applying the same ratio uniformly across billions of parameters ignores each layer's unique role" presumes that A) that layers have a "unique role" (if it were that simple, mechanistic interpretability would be solved), and B) that we have to use the same ratios, essentially "model merging hasn't evolved since 2023" which I can literally prove by Just Look At It.

Darwin V4's Advance
Darwin V4 solved this with evolutionary algorithms — automatically searching hundreds of parameter combinations and selecting survivors by real benchmark scores.

You didn't invent that. See above.

You need more than this, my guy. You can't just expect us to take your word for this. Give some actual theory or get lost.

Discovering attn=0.168 and ffn=0.841 — this extreme asymmetry — is virtually impossible by human intuition.

Perhaps not those precise numbers, but people literally already do layerwise merge ratios, and this is literally already what my friends in Allura have found in their experiments with finetuning; changing attention vs. feed-forward layers provides drastically different results. We're a bunch of gooner dorks in our bedrooms, you've rediscovered this as a government-funded AI lab. What's going on here exactly?

No rigorous definition of "dead" is provided through this entire model card. From what I can tell it means "inactive to a higher degree"

MRI didn't apply uniform ratios. It split 40 layers into 3 blocks:

Thanks GPT-4o.

But again, these terms are meaningless. We don't know what "MRI" means, we have no way to verify that your process actually results in the numbers you're providing.

Dead Expert 50~65% is the fingerprint of Claude text-only distillation. The fine-tuning killed multimodal and multilingual experts that were no longer activated during text-only training.

Didn't you say at the top that the Claude distill is a text-only model?? Why would you expect layers with connections to the multimodal tower to be activated?????? Are we for real??????????

Father MRI: Healthy Generalist (Organ Donor)

Yet another metaphor with no technical definition extant or provided.

The Father (Qwen3.5-35B-A3B) shows healthy, uniform expert activation across all 40 layers — a well-balanced generalist with all experts alive. This is the "organ donor" that revives the Mother's dead 50–65% experts.

Of course. It's the base model. You would expect that

Why This Matters

Thanks GPT-4o.

I can't critique this section but I don't think I have to because the reason I can't critique it is because it's unfalsifiable on account of the blatant and egregious lack of any kind of technical direction in this model card. There is nothing to critique. This is Ancient Aliens tier. This is a wall made of saltine crackers.

So what do we have here?

A layer-wise merge of a Claude 4.6 Opus distillation onto the Qwen 3.5 base, improving degradation caused by what might have been an underdeveloped finetune methodology, that results in better performance, because model merging is a validated technique that works well. The layerwise ratios were discovered with an evolutionary process, a thing that already exists, but isn't often done because it's more expensive.

That alone is interesting enough to promote. It's good PR for evolutionary merging, which I think more people should be focusing on.

But what's stapled on top is a cheap facade of irrelevant jargon from medicine that communicates nothing of value to anything that might have changed about the process, along with false claims about merging that demonstrate that nobody involved with this project respects it as a method, with a model card shat out by a free-tier LLM that understands what it's saying perhaps less than the humans who could have conceivably produced the graphs.

I am insulted having spent my time reading this. There is so much more I could go into but I just keep repeating myself over and over and I only have so many hours in a day.

Come back with a paper with some actual math on it, and I'll change my tone. Until then, stay off of our HF feed, please. This crap makes us all look bad.

Get that government bag tho I guess.

ok this is a scam but i'm on the phone so i'll go over why in a minute

do you explain what you mean by these medical terms that you're using in an AI context anywhere, or what?