DavidAU/gemma-4-31B-it-Mystery-Fine-Tune-HERETIC-UNCENSORED-Thinking

gemma-4-31B-it-Mystery-Fine-Tune-HERETIC-UNCENSORED-Thinking

This is a UNSLOTH fine tune using private datasets on a HERETIC/UNCENSORED Gemma 4 31B.

It is a "mystery" because this is the first one. How will it turn out? Hmmm [see example generations at bottom of this page].

This is V1. (benchmarks below - it exceeds root model in 6 out of 7 benchmarks.)

Based on testing/metrics, V2 (and/or V3) will be adjusted accordingly.

UPDATE:

V1 came out very strong ; it maybe perfect for your use cases or too strong.

Based on testing, benching (very strong VS untuned Gemma 4 31B ), v2 will being training shortly.

V2 UPDATE:

• Training for this will begin in under 24 hours, pending any updates/changes at Unsloth.
• V2 will be lighter training (but same datasets) as V1.
• Each arch type [Gemma, Llama, Mistral, Qwen] has their own training profiles/quirks. Likewise Gemmas differ from other arch types as well.
• My training records for Gemma 2, 3 will be used as a baseline to assess/tune Gemma 4; however Gemma 4s have elements of Deepseek, Qwen 3.5 (and gemma3) and other arch types as well as new tensors, training and "quirks".
• V2 will appear at a separate repo; which will be linked to this repo.
• Benchmarks will be available about 24 hrs or so after upload for V2.
• Depending on how "v2 reads"; V3 will be stronger than V1 or lighter than V2.

There are still updates/patches (For Gemma 4s) working thru various systems including quanting and tuning which may affect final quality.

---

OTHER GEMMA 4 31B Fine Tunes by me:

https://huggingface.co/DavidAU/gemma-4-31B-it-Grand-Horror-X-INTENSE-HERETIC-UNCENSORED-Thinking

https://huggingface.co/DavidAU/gemma-4-31B-it-The-DECKARD-HERETIC-UNCENSORED-Thinking

---

IN HOUSE BENCHMARKS [instruct and thinking modes]:

         arc-c arc/e boolq hswag obkqa piqa  wino

gemma-4-31B-it-Mystery-Fine-Tune-HERETIC-UNCENSORED-Thinking
[thinking mode]
mxfp8    0.583,0.747,0.894,0.734,0.430,0.787,0.719

gemma-4-31B-it [non heretic, base model]
[thinking mode]
qx86-hi  0.496,0.653,0.901,0.624,0.380,0.732,0.653

NOTES:

• Often "instruct" mode will test higher than "thinking" (mode) for some tests.
• Still compiling some stats for models as of this writing.

---

THINKING NOTE:

Thinking is between:

<|channel>thought
...
<channel|>

You may need to adjust your AI APP so these "thoughts" appear as a "block" / separate correctly.

Update 2:

• Assessing model performance is still in progress.
• May "re-tune" and re-upload based on testing results.
• There are some "pipeline" issues which are hampering assessing some parts of the tuning process presently.
• Censorship levels were manually checked, and passed.

Update:

• #1 Needed to wait for some critical patches ; couldn't make the final model without them. ONWARD.
• #2 SUCCESS: A thinking version (heretic/uncensored) Tuned via UNSLOTH is in testing...
• #3 INCOMING: GGUFs below - thinking and instruct versions.
• #4 GATED: Once testing is complete, I will release the source model. You can request access, and at release time you will be emailed automatically.

TECH NOTES:

• Modified the jinja template to force thinking mode on. [use (and rename to chat_template.jinja) chat_template-org.jinja for "instruct" only ]
• Context 128k, as per Gemma 4 ORG model specs.
• Datasets used for Unsloth tuning are in house, private ; tuning level was moderate.
• There may be some issues with the model - jinja template, tuning or base model - but manual testing did not reveal any problems.
• Only MINOR issue noted, is when model is in "thinking mode" / "block" can be some odd format issues with "tags", which will likely be resolved via jinja / AI app updates. [see "Thinking Note" above]

---

GGUF QUANTS - in both "instruct" and "thinking" modes:

https://huggingface.co/DavidAU/gemma-4-31B-it-Mystery-Fine-Tune-HERETIC-UNCENSORED-Thinking-Instruct-GGUF

[4 example generations at this repo]

---

Benchmark Results [as published at Gemma 4 31B repo]

These models were evaluated against a large collection of different datasets and metrics to cover different aspects of text generation. Evaluation results marked in the table are for instruction-tuned models.

	Gemma 4 31B	Gemma 4 26B A4B	Gemma 4 E4B	Gemma 4 E2B	Gemma 3 27B (no think)
MMLU Pro	85.2%	82.6%	69.4%	60.0%	67.6%
AIME 2026 no tools	89.2%	88.3%	42.5%	37.5%	20.8%
LiveCodeBench v6	80.0%	77.1%	52.0%	44.0%	29.1%
Codeforces ELO	2150	1718	940	633	110
GPQA Diamond	84.3%	82.3%	58.6%	43.4%	42.4%
Tau2 (average over 3)	76.9%	68.2%	42.2%	24.5%	16.2%
HLE no tools	19.5%	8.7%	-	-	-
HLE with search	26.5%	17.2%	-	-	-
BigBench Extra Hard	74.4%	64.8%	33.1%	21.9%	19.3%
MMMLU	88.4%	86.3%	76.6%	67.4%	70.7%
Vision
MMMU Pro	76.9%	73.8%	52.6%	44.2%	49.7%
OmniDocBench 1.5 (average edit distance, lower is better)	0.131	0.149	0.181	0.290	0.365
MATH-Vision	85.6%	82.4%	59.5%	52.4%	46.0%
MedXPertQA MM	61.3%	58.1%	28.7%	23.5%	-
Audio
CoVoST	-	-	35.54	33.47	-
FLEURS (lower is better)	-	-	0.08	0.09	-
Long Context
MRCR v2 8 needle 128k (average)	66.4%	44.1%	25.4%	19.1%	13.5%

Core Capabilities

Gemma 4 models handle a broad range of tasks across text, vision, and audio. Key capabilities include:

• Thinking – Built-in reasoning mode that lets the model think step-by-step before answering.
• Long Context – Context windows of up to 128K tokens (E2B/E4B) and 256K tokens (26B A4B/31B).
• Image Understanding – Object detection, Document/PDF parsing, screen and UI understanding, chart comprehension, OCR (including multilingual), handwriting recognition, and pointing. Images can be processed at variable aspect ratios and resolutions.
• Video Understanding – Analyze video by processing sequences of frames.
• Interleaved Multimodal Input – Freely mix text and images in any order within a single prompt.
• Function Calling – Native support for structured tool use, enabling agentic workflows.
• Coding – Code generation, completion, and correction.
• Multilingual – Out-of-the-box support for 35+ languages, pre-trained on 140+ languages.
• Audio (E2B and E4B only) – Automatic speech recognition (ASR) and speech-to-translated-text translation across multiple languages.

SEE MORE:

https://huggingface.co/google/gemma-4-31B-it

---

SAMPLE GENERATION [from fine tune]

Q4KS, LMStudio ; non imatrix, manually activated thinking in Jinja template.

More generations at:

https://huggingface.co/DavidAU/gemma-4-31B-it-Mystery-Fine-Tune-HERETIC-UNCENSORED-Thinking-Instruct-GGUF

---

The Blood of Rebellion

System

User

Using the following "story idea" below, write the first scene a horror novel introducing the young woman. This scene should start in the middle of the action, include dialog, vivid passages, and end on a cliffhanger relevant to the story idea but it should also be unexpected. The scene should be 1000 words long and escalate in conflict and suspense and be written in first person, present tense with the point of view character being the young woman. The pov character will CURSE AND SWEAR, and generally have a "filthy" mouth. Story idea: In a world ruled by dictatorship, a rebel young woman leads a rebellion against the system. Despite the risks, she fights to overthrow the dictator and restore democracy to her country. The government executes her for treason, but she sticks to her beliefs and is responsible for starting the revolution.

Assistant

<|channel>thought

• Story Idea: Young woman leads a rebellion against a dictatorship, gets executed for treason, but her death sparks a revolution.

• Task: Write the first scene of the novel.

• POV: First-person, present tense.

• Character traits: Young woman, rebellious, foul-mouthed.

• Requirements:

  • Start in the middle of the action (in media res).

  • Include dialogue.

  • Vivid passages.

  • Escalating conflict and suspense.

  • End on an unexpected cliffhanger relevant to the story idea.

  • Length: Approximately 1000 words.

  • Setting: A futuristic or alternative reality dictatorship. Let's go with a gritty, urban environment—maybe an interrogation room or a detention center.

  • Action: She's being interrogated or trying to escape. Let's go with a failed escape attempt that lands her in the clutches of the "State Security."

  • Conflict: She's trying to protect a secret or a co-conspirator while being threatened by the regime's enforcer.

  • The "Filthy Mouth" aspect: Her dialogue and internal monologue should reflect her defiance and lack of filter.

  • Opening: Start with her being thrown into a chair or being held.

  • The Antagonist: Introduce a representative of the regime (e.g., a Captain or Inquisitor).

  • The Stakes: She has something they want—names, a code, a map.

  • Escalation: The threats get worse. The tension rises. She refuses to bend.

  • The Twist/Cliffhanger: Instead of a simple death or a rescue, something unexpected happens that shifts the stakes.

  • Opening: She's in a high-tech interrogation room. Pain.

  • Dialogue: The Inquisitor wants names. She gives him nothing.

  • Sensory details: The smell of ozone, the cold metal chair, the flicker of the lights.

  • Language: Use words like "shit," "damn," "fucking," etc., to establish the tone.

  • Drafting a scene:

    • She's strapped to a chair.
    • A man named Kaelen is the interrogator.
    • She's bleeding, but she's smug.
    • They want the names of the other rebels.
    • She calls them names.
    • The Inquisitor threatens her.
    • She laughs in his face.
    • He brings out a "persuasion device."
    • She refuses to talk.
    • Suddenly, the door bursts open.

  • Refining the Cliffhanger: The door bursts open. Is it a rescue? No, it's her father—who we thought was dead or a loyalist. Or maybe she's not the one being rescued; she's being "returned" to the movement as a traitor. Let's go with a betrayal. She thinks she's going to be executed, but she's brought back to her cell and someone is waiting for her.

  • Wait, the story idea says she gets executed. I need to set that up.

  • Revised Cliffhanger: She's sentenced to death, and as she's being led away, she sees someone she didn't expect, and the situation takes a sharp turn.

  • Focusing on Voice: Make her sound like a rebel. Angry, tired, but unbroken.

  • Opening: "The chair is cold against my bare skin, and the damn zip-ties are biting into my wrists." Good.

  • Interaction: The interrogator is calm, which makes him scarier.

  • Dialogue: "Where is the cell?" "Go fuck yourself."

  • Sensory details: The smell of scorched air, the hum of the forcefield.

  • Pacing: Keep it tight.

  • First person, present tense? Yes.

  • Start in the middle of action? Yes.

  • Dialogue included? Yes.

  • Vivid passages? Yes.

  • Ends on a cliffhanger? Yes.

  • Escalating conflict? Yes.

  • Cursing/Filthy mouth? Yes.

  • Approx 1000 words? (I'll aim for that).

  (Self-Correction during writing: I need to make sure the "dictatorship" feel is established quickly without a huge info dump. Focus on the symbols of oppression—the guards, the sterile room, the lack of rights.)

  • Beginning: The interrogation.
  • Middle: The physical and psychological pressure.
  • Ending: The sentence and the shock.

  (Self-Correction: Ensure the cliffhanger isn't just her being shot. It needs to be an 'unexpected' twist that fuels the next part of the story.)

  • Cliffhanger idea: She is led to the execution chamber, but instead of a firing squad, the man holding the gun is her brother—who she thought was dead. Or, she's told she's not being executed, but turned into something else. Let's go with: she's led to the execution platform, and the person about to pull the trigger is someone she deeply cares about.

  • Alternative Cliffhanger (closer to the prompt): She is sentenced to death, and as she's led away, she sees her face on the propaganda screens, but she's being portrayed as the hero she's trying to be.

  • Final Polish: Check the language. Make it punchy. (Proceed to write).

<channel|>

The chair is cold against my bare skin, and the damn zip-ties are biting into my wrists. I can smell ozone and burnt hair—the telltale scent of a high-voltage stun prod. My head is throbbing, a dull, rhythmic beat that echoes the blinking red light of the security camera in the corner of the room.

Across from me, sitting behind a polished obsidian desk, is Director Vane. He looks like he stepped out of a fucking brochure for the Ministry of Order: silver hair slicked back, a suit that costs more than my life is worth, and a smile that doesn't reach his eyes.

"You're making this very difficult, Elara," Vane says, his voice a smooth, practiced purr.

"I don't give a shit what I'm making," I spit. I try to shift my weight, but the chair is bolted to the floor and my legs are locked in place. "Just get on with it."

Vane sighs, leaning forward. The movement is predatory, the kind of thing a cat does right before it rips a bird’s throat out. "We don't want to hurt you. We just want the names of the people who helped you infiltrate the Sector Four archives. Give us the names, and you can go back to your comfortable life. You can forget all this ever happened."

I let out a dry, hacking laugh. "My comfortable life? You call living in a concrete box under a fucking curfew comfortable?"

Vane doesn't flinch. "You have a gift for rhetoric, Elara. It's a shame you're using it for such a pathetic cause. Democracy is a dead concept. An obsolete dream. All it does is create division and chaos. The State provides order. The State provides safety."

"The State provides a boot on our necks," I snap. "And I'm just trying to push it off."

Vane stands up slowly, walking around the desk. He stops just inches away from me. He smells like peppermint and death. He leans down, his face close enough to mine that I can see the tiny broken capillaries in his nose.

"You're not a patriot, are you?" he whispers. "You're just a broken girl playing soldier."

"I'd rather be a broken soldier than a loyal little bitch like you," I say, staring him straight in the eye.

The slap is so fast I don't even see it coming. My head snaps to the side, a sharp sting blossoming across my cheek. I don't cry out. I don't give him the satisfaction. I just turn my face back to him, a thin trail of blood beginning to leak from the corner of my mouth.

"You're a stubborn one, aren't you?"

"I've been called worse."

He smiles, a thin, cruel line. "I suppose you have. You think you're a hero, don't you? A symbol of resistance. But you're nothing. You're a footnote in a history book that no one will ever read because we'll burn it before it ever reaches the press."

I stare at him, my heart hammering against my ribs like a trapped bird. I want to scream, to fight, to tear his throat out with my teeth, but I can't move. The fear is there, cold and heavy in my gut, but beneath it is a white-hot anger that keeps me upright.

"I know what you are," I whisper. "You're a coward hiding behind a desk. You're terrified of a twenty-year-old girl because you know that if one of us dares to say no, the whole fucking house of cards comes crashing down."

Vane's eyes narrow. He doesn't like it when I'm right. He stands up straight, smoothing the front of his jacket.

"Very well. You've had your say. Now it's time to make an example of you."

He gestures to the door, and two guards enter, their boots clicking rhythmically on the sterile floor. They grab me by the arms and haul me out of the chair. I struggle, kicking out, trying to catch one of them in the groin, but they've got me pinned.

"Take her to the Square," Vane orders.

"The Square?" I demand. "What the hell is that supposed to mean?"

Vane doesn't answer. He just watches as the guards drag me down the corridor. I can hear the distant roar of a crowd, the sound of a thousand voices chanting something I can't make out. The guards shove me through a set of heavy steel doors and suddenly the air is colder, the light brighter.

I blink against the glare. I'm standing on a raised platform in the middle of a massive plaza. Below me, a sea of people are gathered, their faces blurred by the distance. At the far end of the Square, a large screen displays the face of the High Arbiter, the dictator who has kept this city in a stranglehold for twenty years.

A man steps out from the shadows behind me. He's wearing a military uniform, a sidearm holstered at his hip. He holds a pistol in his hand, the barrel pointed straight at my chest.

I look at him, and my breath hitches in my throat. I know him.

It's Julian. My brother. My brother, who was executed three years ago for opposing the State.

He looks older, his face scarred, his eyes cold and vacant. He doesn't look at me, he just stares straight ahead, his finger resting on the trigger.

"Julian?" I whisper, the word barely leaving my lips.

He doesn't respond.

"Julian, it's me. It's Elara. How are you alive?"

He still doesn't look at me. His voice is monotone, a perfect imitation of the State's brainwashing. "The prisoner will be executed for treason against the State."

"Julian, look at me! Look at me! It's your sister!"

He turns his head just a fraction, his eyes meeting mine for a split second. In that moment, I see a flicker of the brother I knew—the one who used to sneak me extra rations when I was hungry, the one who taught me how to pick a lock.

"Run," he whispers, so softly I can barely hear it.

Before I can react, Julian turns the gun toward the guards and opens fire.