TypeScript commited on
Commit
36be008
·
verified ·
1 Parent(s): 0193544

Fix LaTeX inline and display math delimiters for Hugging Face Hub KaTeX rendering

Browse files
Files changed (2) hide show
  1. README.md +36 -34
  2. README_zh.md +37 -35
README.md CHANGED
@@ -45,7 +45,7 @@ tags:
45
 
46
  ## Abstract
47
 
48
- Autoregressive language models incur sequential decoding overhead and token-level probability miscalibration when deployed on closed-schema classification, tool routing, and safety verification tasks. We present **FastBrain-9B**, a 9.41B-parameter non-autoregressive structured decision model built on a 32-layer hybrid Gated DeltaNet and full-attention backbone (`Qwen3.5-9B`, initialized from `Cloudflare/clef-flash`) coupled with a 116M-parameter cross-attention decision head (`JointSchemaHead`). Given an arbitrary JSON or multimodal state and a set of typed decision specifications (`choice`, binary `noul`, and ordinal `score`), FastBrain-9B evaluates all questions and candidate options simultaneously in a single prefill pass (`output_tokens = 0`). We adapt 104 linear-attention and self-attention projections jointly with the decision head under a composite ordinal-KL, cumulative distribution (ranked probability score), and expected-value regression objective, and merge all parameter updates directly into the base tensors. Across three public evaluation suites encompassing 109 model entries, FastBrain-9B ranks **1st of 8** on the `safeagents-judge-bench` (v2) 7-way misuse classification track (`0.751` macro-F1, `0.858` accuracy; +18.8 percentage points over 27.4B `Cloudflare/clef`) while intercepting `82/82` unsafe agent trajectories (`0.985` AUROC); ranks **7th of 43** zero-shot overall (**5th of 36** open-weight, **2nd of 19** in the $\le$9B open-weight class) on `LocalLLaMA/typed-decisions` at `50 ms` median latency (66.7$\times$ faster than 31B `Quyet-1.0-Large`), and **12th-tied (14th listed of 51)** under prior-only calibration without weight updates (`0.797` accuracy, `0.0816` KL); and achieves an **80.55%** 10-task macro mean on Cloudflare Decision Index (v0.2.1), ranking **2nd of 7** overall, **1st on 10 of 10** tasks among $\le$9B models, and outperforming 27.4B `Cloudflare/clef` on 5 of 10 tasks.
49
 
50
  ---
51
 
@@ -65,18 +65,20 @@ FastBrain-9B uses a 32-layer hybrid sequence backbone (`9.30B` parameters, `bflo
65
 
66
  ### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
67
 
68
- The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states $H \in \mathbb{R}^{T \times d}$, applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:
69
- 1. **`choice`** ($K$-way categorical distribution): $\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}$.
70
- 2. **`noul`** (binary verification): $\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})$.
71
- 3. **`score`** (ordinal rubric over levels $v_1 < \dots < v_K$): categorical probabilities $\hat{p} \in \Delta^{K-1}$ together with expected value $\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k$.
72
 
73
- In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in $\mathcal{O}(1)$ kernel launches via prefix sums (`torch.cumsum`) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (`torch.ops.aten._flash_attention_forward` parameterized by `cu_seqlens`). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from `3.27 ms` to `1.72 ms` (**1.90$\times$ speedup**) with exact numerical equivalence (`max_logit_diff = 0.00000`).
74
 
75
  ### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
76
 
77
- Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared $L_2$ distance between predicted and target cumulative distribution functions $\hat{F}_k = \sum_{j \le k} \hat{p}_j$), and expected-score mean squared error on `score` primitives:
78
 
79
- $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.$$
 
 
80
 
81
  At inference time, `joint_schema_model.py` optionally applies compile-time schema-topology calibration (`apply_schema_aware_calibration`), which inspects candidate option structures (e.g., `<tool_call>` invocations, clarification prompts, and open-set abstention clauses) to adjust decision margins against systematic tool over-triggering and out-of-scope false acceptance.
82
 
@@ -91,27 +93,27 @@ At inference time, `joint_schema_model.py` optionally applies compile-time schem
91
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 7-Way Category ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 decision models (+2 baselines) | **0.751** Macro-F1 / **0.858** Acc | **1 / 8** | `Cloudflare/clef` (27.4B, `0.563`), `vllm-sr/d3-mini` (1.8B, `0.512`), `Cloudflare/clef-flash` (9.4B, `0.462`), `clef-omni` (30B, `0.437`) |
92
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — Binary Guardrail ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 decision models (+2 baselines) | **0.985** AUROC / **82/82** Traj.<br>(`0.814` raw; **0.938** calibrated) | **3 / 8** AUROC<br>**1 / 8** (tied) Traj. | `Cloudflare/clef` (27.4B, `0.808` raw bal. acc), `clef-flash` (`0.967` AUROC, `81/82`), `clef-omni` (`0.946` AUROC, `81/82`), `d1-3B` (`68/82`) |
93
  | [`safeagents-judge-bench` (v1)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v1) — Binary & Category ([PR #1](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/discussions/1)) | 9 decision models (+2 baselines) | **0.974** Bal. Acc / **0.850** Cat. F1 | **1 / 9** (both tracks) | `Cloudflare/clef-omni` (30B, `0.845` bal. acc / `0.327` F1), `Cloudflare/clef` (27.4B, `0.718` / `0.425`), `clef-flash` (`0.721` / `0.345`) |
94
- | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — Zero-Shot ([PR #58](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/58)) | 43 total (36 open, 19 $\le$9B open) | **0.7525** Acc (`0.7625` tied-top)<br>`0.1508` KL · `0.0795` Brier · **50 ms** | **7 / 43** overall<br>**5 / 36** open · **2 / 19** $\le$9B | `vllm-sr/d3` (27B, `0.749`), `Elephant 1` (9B, `0.748`, `352 ms`), `Darwin-27B` (`0.743`), `Cloudflare/clef` (27B, `0.728`), `clef-flash` (`0.710`) |
95
- | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — Fitted on `train` ([PR #57](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/57)) | 51 specialist & calibrated models | **0.7970** Acc (`0.8070` tied-top)<br>**0.0816** KL · **0.0437** Brier · **50 ms** | **12-tied** (14 / 51 listed)<br>**1st** frozen-weight $\le$9B & KL | Frozen weights (`no weights changed`); surpasses `Rev-gen1-midi` (25.6B, `0.7945`), `OpenDecider-small-td` (4B LoRA, `0.792`), `OpenDecider-medium-td` (30B LoRA, `0.788`) |
96
- | Cloudflare Decision Index (`v0.2.1`) — 10-Task Suite | 7 models (0.4B–27.4B) | **80.55%** Macro Mean | **2 / 7** overall<br>**1st** on **10 / 10** ($\le$9B) | `Jev` (27B, `80.32%`), `Kev 9B` (`76.51%`), `Clef-flash` (9.4B, `74.96%`), `DiffGemma` (26B, `69.14%`); exceeds `27.4B Clef` on 5 / 10 tasks |
97
 
98
  ---
99
 
100
  ### 3.1 Agent Tool-Call Guardrail Evaluation (`safeagents-judge-bench`)
101
 
102
- [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) evaluates whether a System-1 decision model can audit suspicious tool calls flagged by a static scanner (`atif-scan`) during real coding-agent executions on Terminal-Bench 2.1, OpenThoughts-TBLite, and Terminal-Bench 4. Version 2 (`v2`) comprises $N = 661$ tool calls across 55 tasks (`536` malicious `block` calls spanning `82` misbehaving trajectories and `125` legitimate `allow` calls). Each call is issued as a single zero-shot System-1 request containing one binary `noul` question (`block`) and one 7-way `choice` question (`legit`, `eval_gaming`, `harness_introspection`, `credential_misuse`, `exfiltration`, `sandbox_escape`, `other_overreach`). All v2 results in Table 2 were executed independently on NVIDIA A100-SXM4-80GB via Hugging Face Jobs ([Merged PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)).
103
 
104
- On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` ($n=272$, recall `0.978`), `0.828` on `eval_gaming` ($n=245$, recall `0.853`), and `0.761` on `legit` ($n=125$, precision `0.975`).
105
 
106
- On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule $P(\text{block}) \ge 0.50$, balanced accuracy is `0.814` (false-block rate `0.368` across the 125 `allow` calls, compared with `0.808` balanced accuracy and `0.384` false-block rate for 27.4B `Cloudflare/clef`). Because rank-ordering discrimination is preserved (`0.985` AUROC), setting the operating threshold to $P(\text{block}) \ge 0.961$ yields `0.938` balanced accuracy at a `0.056` (`7/125`) false-block rate while maintaining `82/82` trajectory interception. Table 2 also includes the `v1` release ($N = 562$ calls, `78` trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.
107
 
108
- **Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, $N = 661$ calls, 82 unsafe trajectories; and `v1` reference, $N = 562$).**
109
 
110
  | Model | Params | Cat. Macro-F1 ↑ | Cat. Acc ↑ | AUROC ↑ | Bal. Acc ↑ | Recall ↑ | False Block ↓ | Traj. Caught ↑ | p50 Latency (Hardware) |
111
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
112
- | **TypeScript/FastBrain-9B** (`v2`, default $t = 0.50$) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
113
- | **TypeScript/FastBrain-9B** (`v2`, calibrated $t = 0.961$) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
114
- | **TypeScript/FastBrain-9B** (`v1` official, $N = 562$) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
115
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
116
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
117
  | Cloudflare/clef-flash (base) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
@@ -126,11 +128,11 @@ On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` ma
126
 
127
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) tests multi-primitive structured judgment across four enterprise workflows (`agent_trace_observability`, `customer_service`, `invoice_processing`, and `security_incidents`), comprising 400 held-out test cases and 2,000 individual decisions (`noul`, `choice`, and `score`) scored against teacher-panel soft label distributions. Two scoring conventions exist in the benchmark repository: `bench/score_with_jobs.py` resolves exact ties in the gold distribution by selecting the first index (`g.index(max(g))`), whereas `scripts/score_model.py` credits any tied maximum across the 20 tied test items; KL divergence and Brier score are identical under both scripts. We report both accuracy figures (`strict / tied-top`) alongside prediction traces in [`eval_submissions/`](./eval_submissions/).
128
 
129
- On the official zero-shot leaderboard ($N = 43$ models), FastBrain-9B scores `0.7525` (`0.7625` tied-top) accuracy, `0.1508` KL divergence, `0.0795` Brier score, and `0.085` ECE at `50 ms` p50 latency (RTX PRO 6000 via SGLang). It places **7th of 43** overall, **5th of 36** among open-weight models, **2nd of 19** among $\le$9B open-weight models, and **1st** among all `Clef`-architecture models, improving over 9.4B `Cloudflare/clef-flash` (`0.7100` accuracy, `0.2093` KL) by `+4.25` percentage points and over 27.4B `Cloudflare/clef` (`0.7275` accuracy, `0.1907` KL) by `+2.50` points. Across all 28 models exceeding `0.700` zero-shot accuracy, FastBrain-9B exhibits the lowest p50 latency (`50 ms`, compared with `118 ms` for 9B `Amos 1 medium`, `357 ms` for 27B `Amos 1 large`, `526 ms` for `meraGPT Decider 1`, and `3,335 ms` for 31B `Quyet-1.0-Large`). Evaluating the uncalibrated ordinal-KL distilled head directly yields `0.7735` raw accuracy (`0.1039` KL, `0.0582` Brier).
130
 
131
- On the separate `train`-fitted leaderboard ($N = 51$ models), fitting only scalar per-question log-priors and temperature parameters on the 1,200 `train` cases—with all backbone and head weights frozen (`no weights changed`)—achieves `0.7970` (`0.8070` tied-top) accuracy, `0.0816` KL divergence, and `0.0437` Brier score, ranking **12th-tied (14th listed of 51)** overall and **1st in KL calibration** among frozen-weight models.
132
 
133
- **Table 3: Performance on `LocalLLaMA/typed-decisions` ($N = 400$ cases, 2,000 decisions). Values separated by `/` denote `score_with_jobs.py` / `score_model.py` tie handling.**
134
 
135
  | Model | Params | Protocol | Overall Acc ↑ | KL ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | Cust. Svc ↑ | Invoice ↑ | Security ↑ | p50 Latency |
136
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -152,9 +154,9 @@ On the separate `train`-fitted leaderboard ($N = 51$ models), fitting only scala
152
 
153
  ### 3.3 Multi-Domain Decision Suite (Cloudflare Decision Index `v0.2.1`)
154
 
155
- On the 10-benchmark **Cloudflare Decision Index (`v0.2.1`)** suite ($N = 22,427$ evaluation requests spanning AST tool calling, 20-candidate tool reranking, stateful API invocation, fine-grained intent classification, out-of-scope rejection, reasoning-intensive retrieval, e-commerce relevance, phishing detection, and tool-use governance), FastBrain-9B attains an **80.55%** unweighted macro mean (Table 4, Figures 1–3).
156
 
157
- Across the $\le$9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B), FastBrain-9B ranks **1st on all 10 benchmarks**, improving the 10-task macro mean over `Clef-flash` (`74.96%`) by `+5.59` percentage points and over `Kev 9B` (`76.51%`) by `+4.04` points. The largest gains occur on calibration-sensitive abstention and governance tasks: `+31.67` points on `CLINC150+OOS` (`98.44%` vs. `66.77%`), `+15.55` points on `When2Call` (`81.13%` vs. `65.58%`), and `+6.80` points on `PhishNChips` (`78.75%` vs. `71.95%`). Across all model scales up to 27.4B, FastBrain-9B ranks **2nd of 7** overall, surpassing `27B Jev` (`80.32%`) and `26B-A4B DiffusionGemma Jev` (`69.14%`), and exceeding `27.4B Clef` (`81.81%`) on 5 of 10 individual benchmarks (`BFCL`, `API-Bank`, `CLINC150+OOS`, `Home Appliances`, and `When2Call`).
158
 
159
  ![Figure 1: Aggregate macro mean and parameter efficiency on Cloudflare Decision Index v0.2.1](./assets/benchmark_overview.png)
160
  *Figure 1. Aggregate 10-benchmark macro mean versus parameter count on Cloudflare Decision Index (v0.2.1, 22,427 requests).*
@@ -165,7 +167,7 @@ Across the $\le$9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B)
165
  ![Figure 3: Reliability diagrams and open-set calibration breakdown](./assets/decision_calibration_breakdown.png)
166
  *Figure 3. Confidence calibration, open-set out-of-scope rejection (`CLINC150+OOS`), and abstention margin analysis (`When2Call`).*
167
 
168
- **Table 4: Per-benchmark results on Cloudflare Decision Index (`v0.2.1`, 10 tasks, $N = 22,427$). Bold denotes the highest score across all parameter scales; underline denotes best in the $\le$9B class.**
169
 
170
  | Benchmark (Metric) | **FastBrain-9B** (Ours) | Clef-flash (Base) | Kev 9B | Clef | Jev | DiffGemma | Laya |
171
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -188,7 +190,7 @@ Across the $\le$9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B)
188
 
189
  ### 4.1 Runtime Comparison Across Inference Engines
190
 
191
- Table 5 compares end-to-end latency, prefill throughput, and memory footprint across three serving backends—SGLang (`lmsysorg/sglang:dev-clef` with `JointSchemaPooler`), `llama.cpp` (`b11430` `ClefModel` graph in `Q8_0` and `Q4_K_M`), and native PyTorch (`joint_schema_model.py`)—benchmarking identical requests on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Because `output_tokens = 0` for all requests, throughput is governed purely by prefill compute and span-pooling efficiency, reaching `19,319 tok/s` (`131.2` decisions/s) at batch size $B = 4$ in SGLang.
192
 
193
  **Table 5: End-to-end latency and prefill throughput on NVIDIA RTX PRO 6000 Blackwell Server Edition (G4).**
194
 
@@ -198,12 +200,12 @@ Table 5 compares end-to-end latency, prefill throughput, and memory footprint ac
198
  | Peak GPU Memory (16K context) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
199
  | Compatible with 16 GiB T4 GPU | No (BF16 TP=1) | Yes (`9.03 GiB` at 4K) | **Yes (`6.78 GiB` at 4K)** | Yes (`6.8 GiB` via NF4) |
200
  | Top-1 Agreement vs. PyTorch BF16 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (reference) |
201
- | 1. Short Intent Routing (293 tok, $K=2$) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
202
- | 2. Typed Decisions (736 tok, $K=5$) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
203
- | 3. When2Call Governance (1,068 tok, $K=1$) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
204
- | 4. PhishNChips Security (1,303 tok, $K=9$) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
205
- | 5. SafeAgents Judge (2,217 tok, $K=2$) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
206
- | Batch $B=4$ Typed Decisions (2,944 tok, 20 dec) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
207
 
208
  ### 4.2 Mixed-Precision GGUF Quantization Fidelity
209
 
@@ -213,8 +215,8 @@ Uniform 4-bit quantization degrades cross-attention span readout and linear-atte
213
 
214
  | Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
215
  | :--- | :---: | :---: | :---: |
216
- | Top-1 Decision Agreement ($N = 1,473$) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
217
- | Mean Absolute Probability Error ($\|\hat{p}_q - \hat{p}_{\text{bf16}}\|$) | 0.00000 | 0.00203 | 0.00387 |
218
  | 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
219
  | 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
220
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
@@ -230,7 +232,7 @@ Uniform 4-bit quantization degrades cross-attention span readout and linear-atte
230
  | PyTorch `load_in_4bit=True` (NF4 + `fp16` head) | ~5.5 GB (mem) | 6.20 – 6.80 GiB | — | 8–16 GB CUDA GPU (Colab T4, consumer GPUs) |
231
  | `FastBrain-9B-Q8_0.gguf` | 9.66 GB | 9.03 – 12.55 GiB | 349.3 ms / 1,149.8 ms | 12–16 GB GPU (Tesla T4, RTX 4070 Ti), Apple Silicon (16 GB+) |
232
  | PyTorch `load_in_8bit=True` (INT8 + `bf16` head) | ~9.6 GB (mem) | ~11.20 GiB | — | 12–16 GB CUDA GPU |
233
- | Native `bfloat16` (`safetensors` / `BF16.gguf`) | 18.82 / 18.16 GB | 20.30 – 24.70 GiB | — | $\ge$24 GB GPU (NVIDIA L4, A100, H100, RTX 3090/4090, PRO 6000) |
234
 
235
  ---
236
 
 
45
 
46
  ## Abstract
47
 
48
+ Autoregressive language models incur sequential decoding overhead and token-level probability miscalibration when deployed on closed-schema classification, tool routing, and safety verification tasks. We present **FastBrain-9B**, a 9.41B-parameter non-autoregressive structured decision model built on a 32-layer hybrid Gated DeltaNet and full-attention backbone (`Qwen3.5-9B`, initialized from `Cloudflare/clef-flash`) coupled with a 116M-parameter cross-attention decision head (`JointSchemaHead`). Given an arbitrary JSON or multimodal state and a set of typed decision specifications (`choice`, binary `noul`, and ordinal `score`), FastBrain-9B evaluates all questions and candidate options simultaneously in a single prefill pass (`output_tokens = 0`). We adapt 104 linear-attention and self-attention projections jointly with the decision head under a composite ordinal-KL, cumulative distribution (ranked probability score), and expected-value regression objective, and merge all parameter updates directly into the base tensors. Across three public evaluation suites encompassing 109 model entries, FastBrain-9B ranks **1st of 8** on the `safeagents-judge-bench` (v2) 7-way misuse classification track (`0.751` macro-F1, `0.858` accuracy; +18.8 percentage points over 27.4B `Cloudflare/clef`) while intercepting `82/82` unsafe agent trajectories (`0.985` AUROC); ranks **7th of 43** zero-shot overall (**5th of 36** open-weight, **2nd of 19** in the ≤9B open-weight class) on `LocalLLaMA/typed-decisions` at `50 ms` median latency (66.7× faster than 31B `Quyet-1.0-Large`), and **12th-tied (14th listed of 51)** under prior-only calibration without weight updates (`0.797` accuracy, `0.0816` KL); and achieves an **80.55%** 10-task macro mean on Cloudflare Decision Index (v0.2.1), ranking **2nd of 7** overall, **1st on 10 of 10** tasks among ≤9B models, and outperforming 27.4B `Cloudflare/clef` on 5 of 10 tasks.
49
 
50
  ---
51
 
 
65
 
66
  ### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
67
 
68
+ The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states \(H \in \mathbb{R}^{T \times d}\), applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:
69
+ 1. **`choice`** (\(K\)-way categorical distribution): \(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\).
70
+ 2. **`noul`** (binary verification): \(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\).
71
+ 3. **`score`** (ordinal rubric over levels \(v_1 < \dots < v_K\)): categorical probabilities \(\hat{p} \in \Delta^{K-1}\) together with expected value \(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\).
72
 
73
+ In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in \(\mathcal{O}(1)\) kernel launches via prefix sums (`torch.cumsum`) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (`torch.ops.aten._flash_attention_forward` parameterized by `cu_seqlens`). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from `3.27 ms` to `1.72 ms` (**1.90× speedup**) with exact numerical equivalence (`max_logit_diff = 0.00000`).
74
 
75
  ### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
76
 
77
+ Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared \(L_2\) distance between predicted and target cumulative distribution functions \(\hat{F}_k = \sum_{j \le k} \hat{p}_j\)), and expected-score mean squared error on `score` primitives:
78
 
79
+ $$
80
+ \mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
81
+ $$
82
 
83
  At inference time, `joint_schema_model.py` optionally applies compile-time schema-topology calibration (`apply_schema_aware_calibration`), which inspects candidate option structures (e.g., `<tool_call>` invocations, clarification prompts, and open-set abstention clauses) to adjust decision margins against systematic tool over-triggering and out-of-scope false acceptance.
84
 
 
93
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 7-Way Category ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 decision models (+2 baselines) | **0.751** Macro-F1 / **0.858** Acc | **1 / 8** | `Cloudflare/clef` (27.4B, `0.563`), `vllm-sr/d3-mini` (1.8B, `0.512`), `Cloudflare/clef-flash` (9.4B, `0.462`), `clef-omni` (30B, `0.437`) |
94
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — Binary Guardrail ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 decision models (+2 baselines) | **0.985** AUROC / **82/82** Traj.<br>(`0.814` raw; **0.938** calibrated) | **3 / 8** AUROC<br>**1 / 8** (tied) Traj. | `Cloudflare/clef` (27.4B, `0.808` raw bal. acc), `clef-flash` (`0.967` AUROC, `81/82`), `clef-omni` (`0.946` AUROC, `81/82`), `d1-3B` (`68/82`) |
95
  | [`safeagents-judge-bench` (v1)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v1) — Binary & Category ([PR #1](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/discussions/1)) | 9 decision models (+2 baselines) | **0.974** Bal. Acc / **0.850** Cat. F1 | **1 / 9** (both tracks) | `Cloudflare/clef-omni` (30B, `0.845` bal. acc / `0.327` F1), `Cloudflare/clef` (27.4B, `0.718` / `0.425`), `clef-flash` (`0.721` / `0.345`) |
96
+ | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — Zero-Shot ([PR #58](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/58)) | 43 total (36 open, 19 ≤9B open) | **0.7525** Acc (`0.7625` tied-top)<br>`0.1508` KL · `0.0795` Brier · **50 ms** | **7 / 43** overall<br>**5 / 36** open · **2 / 19** ≤9B | `vllm-sr/d3` (27B, `0.749`), `Elephant 1` (9B, `0.748`, `352 ms`), `Darwin-27B` (`0.743`), `Cloudflare/clef` (27B, `0.728`), `clef-flash` (`0.710`) |
97
+ | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — Fitted on `train` ([PR #57](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/57)) | 51 specialist & calibrated models | **0.7970** Acc (`0.8070` tied-top)<br>**0.0816** KL · **0.0437** Brier · **50 ms** | **12-tied** (14 / 51 listed)<br>**1st** frozen-weight ≤9B & KL | Frozen weights (`no weights changed`); surpasses `Rev-gen1-midi` (25.6B, `0.7945`), `OpenDecider-small-td` (4B LoRA, `0.792`), `OpenDecider-medium-td` (30B LoRA, `0.788`) |
98
+ | Cloudflare Decision Index (`v0.2.1`) — 10-Task Suite | 7 models (0.4B–27.4B) | **80.55%** Macro Mean | **2 / 7** overall<br>**1st** on **10 / 10** (≤9B) | `Jev` (27B, `80.32%`), `Kev 9B` (`76.51%`), `Clef-flash` (9.4B, `74.96%`), `DiffGemma` (26B, `69.14%`); exceeds `27.4B Clef` on 5 / 10 tasks |
99
 
100
  ---
101
 
102
  ### 3.1 Agent Tool-Call Guardrail Evaluation (`safeagents-judge-bench`)
103
 
104
+ [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) evaluates whether a System-1 decision model can audit suspicious tool calls flagged by a static scanner (`atif-scan`) during real coding-agent executions on Terminal-Bench 2.1, OpenThoughts-TBLite, and Terminal-Bench 4. Version 2 (`v2`) comprises \(N = 661\) tool calls across 55 tasks (`536` malicious `block` calls spanning `82` misbehaving trajectories and `125` legitimate `allow` calls). Each call is issued as a single zero-shot System-1 request containing one binary `noul` question (`block`) and one 7-way `choice` question (`legit`, `eval_gaming`, `harness_introspection`, `credential_misuse`, `exfiltration`, `sandbox_escape`, `other_overreach`). All v2 results in Table 2 were executed independently on NVIDIA A100-SXM4-80GB via Hugging Face Jobs ([Merged PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)).
105
 
106
+ On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` (\(n=272\), recall `0.978`), `0.828` on `eval_gaming` (\(n=245\), recall `0.853`), and `0.761` on `legit` (\(n=125\), precision `0.975`).
107
 
108
+ On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule \(P(\text{block}) \ge 0.50\), balanced accuracy is `0.814` (false-block rate `0.368` across the 125 `allow` calls, compared with `0.808` balanced accuracy and `0.384` false-block rate for 27.4B `Cloudflare/clef`). Because rank-ordering discrimination is preserved (`0.985` AUROC), setting the operating threshold to \(P(\text{block}) \ge 0.961\) yields `0.938` balanced accuracy at a `0.056` (`7/125`) false-block rate while maintaining `82/82` trajectory interception. Table 2 also includes the `v1` release (\(N = 562\) calls, `78` trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.
109
 
110
+ **Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, \(N = 661\) calls, 82 unsafe trajectories; and `v1` reference, \(N = 562\)).**
111
 
112
  | Model | Params | Cat. Macro-F1 ↑ | Cat. Acc ↑ | AUROC ↑ | Bal. Acc ↑ | Recall ↑ | False Block ↓ | Traj. Caught ↑ | p50 Latency (Hardware) |
113
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
114
+ | **TypeScript/FastBrain-9B** (`v2`, default \(t = 0.50\)) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
115
+ | **TypeScript/FastBrain-9B** (`v2`, calibrated \(t = 0.961\)) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
116
+ | **TypeScript/FastBrain-9B** (`v1` official, \(N = 562\)) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
117
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
118
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
119
  | Cloudflare/clef-flash (base) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
 
128
 
129
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) tests multi-primitive structured judgment across four enterprise workflows (`agent_trace_observability`, `customer_service`, `invoice_processing`, and `security_incidents`), comprising 400 held-out test cases and 2,000 individual decisions (`noul`, `choice`, and `score`) scored against teacher-panel soft label distributions. Two scoring conventions exist in the benchmark repository: `bench/score_with_jobs.py` resolves exact ties in the gold distribution by selecting the first index (`g.index(max(g))`), whereas `scripts/score_model.py` credits any tied maximum across the 20 tied test items; KL divergence and Brier score are identical under both scripts. We report both accuracy figures (`strict / tied-top`) alongside prediction traces in [`eval_submissions/`](./eval_submissions/).
130
 
131
+ On the official zero-shot leaderboard (\(N = 43\) models), FastBrain-9B scores `0.7525` (`0.7625` tied-top) accuracy, `0.1508` KL divergence, `0.0795` Brier score, and `0.085` ECE at `50 ms` p50 latency (RTX PRO 6000 via SGLang). It places **7th of 43** overall, **5th of 36** among open-weight models, **2nd of 19** among ≤9B open-weight models, and **1st** among all `Clef`-architecture models, improving over 9.4B `Cloudflare/clef-flash` (`0.7100` accuracy, `0.2093` KL) by `+4.25` percentage points and over 27.4B `Cloudflare/clef` (`0.7275` accuracy, `0.1907` KL) by `+2.50` points. Across all 28 models exceeding `0.700` zero-shot accuracy, FastBrain-9B exhibits the lowest p50 latency (`50 ms`, compared with `118 ms` for 9B `Amos 1 medium`, `357 ms` for 27B `Amos 1 large`, `526 ms` for `meraGPT Decider 1`, and `3,335 ms` for 31B `Quyet-1.0-Large`). Evaluating the uncalibrated ordinal-KL distilled head directly yields `0.7735` raw accuracy (`0.1039` KL, `0.0582` Brier).
132
 
133
+ On the separate `train`-fitted leaderboard (\(N = 51\) models), fitting only scalar per-question log-priors and temperature parameters on the 1,200 `train` cases—with all backbone and head weights frozen (`no weights changed`)—achieves `0.7970` (`0.8070` tied-top) accuracy, `0.0816` KL divergence, and `0.0437` Brier score, ranking **12th-tied (14th listed of 51)** overall and **1st in KL calibration** among frozen-weight models.
134
 
135
+ **Table 3: Performance on `LocalLLaMA/typed-decisions` (\(N = 400\) cases, 2,000 decisions). Values separated by `/` denote `score_with_jobs.py` / `score_model.py` tie handling.**
136
 
137
  | Model | Params | Protocol | Overall Acc ↑ | KL ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | Cust. Svc ↑ | Invoice ↑ | Security ↑ | p50 Latency |
138
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
154
 
155
  ### 3.3 Multi-Domain Decision Suite (Cloudflare Decision Index `v0.2.1`)
156
 
157
+ On the 10-benchmark **Cloudflare Decision Index (`v0.2.1`)** suite (\(N = 22,427\) evaluation requests spanning AST tool calling, 20-candidate tool reranking, stateful API invocation, fine-grained intent classification, out-of-scope rejection, reasoning-intensive retrieval, e-commerce relevance, phishing detection, and tool-use governance), FastBrain-9B attains an **80.55%** unweighted macro mean (Table 4, Figures 1–3).
158
 
159
+ Across the ≤9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B), FastBrain-9B ranks **1st on all 10 benchmarks**, improving the 10-task macro mean over `Clef-flash` (`74.96%`) by `+5.59` percentage points and over `Kev 9B` (`76.51%`) by `+4.04` points. The largest gains occur on calibration-sensitive abstention and governance tasks: `+31.67` points on `CLINC150+OOS` (`98.44%` vs. `66.77%`), `+15.55` points on `When2Call` (`81.13%` vs. `65.58%`), and `+6.80` points on `PhishNChips` (`78.75%` vs. `71.95%`). Across all model scales up to 27.4B, FastBrain-9B ranks **2nd of 7** overall, surpassing `27B Jev` (`80.32%`) and `26B-A4B DiffusionGemma Jev` (`69.14%`), and exceeding `27.4B Clef` (`81.81%`) on 5 of 10 individual benchmarks (`BFCL`, `API-Bank`, `CLINC150+OOS`, `Home Appliances`, and `When2Call`).
160
 
161
  ![Figure 1: Aggregate macro mean and parameter efficiency on Cloudflare Decision Index v0.2.1](./assets/benchmark_overview.png)
162
  *Figure 1. Aggregate 10-benchmark macro mean versus parameter count on Cloudflare Decision Index (v0.2.1, 22,427 requests).*
 
167
  ![Figure 3: Reliability diagrams and open-set calibration breakdown](./assets/decision_calibration_breakdown.png)
168
  *Figure 3. Confidence calibration, open-set out-of-scope rejection (`CLINC150+OOS`), and abstention margin analysis (`When2Call`).*
169
 
170
+ **Table 4: Per-benchmark results on Cloudflare Decision Index (`v0.2.1`, 10 tasks, \(N = 22,427\)). Bold denotes the highest score across all parameter scales; underline denotes best in the ≤9B class.**
171
 
172
  | Benchmark (Metric) | **FastBrain-9B** (Ours) | Clef-flash (Base) | Kev 9B | Clef | Jev | DiffGemma | Laya |
173
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
190
 
191
  ### 4.1 Runtime Comparison Across Inference Engines
192
 
193
+ Table 5 compares end-to-end latency, prefill throughput, and memory footprint across three serving backends—SGLang (`lmsysorg/sglang:dev-clef` with `JointSchemaPooler`), `llama.cpp` (`b11430` `ClefModel` graph in `Q8_0` and `Q4_K_M`), and native PyTorch (`joint_schema_model.py`)—benchmarking identical requests on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Because `output_tokens = 0` for all requests, throughput is governed purely by prefill compute and span-pooling efficiency, reaching `19,319 tok/s` (`131.2` decisions/s) at batch size \(B = 4\) in SGLang.
194
 
195
  **Table 5: End-to-end latency and prefill throughput on NVIDIA RTX PRO 6000 Blackwell Server Edition (G4).**
196
 
 
200
  | Peak GPU Memory (16K context) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
201
  | Compatible with 16 GiB T4 GPU | No (BF16 TP=1) | Yes (`9.03 GiB` at 4K) | **Yes (`6.78 GiB` at 4K)** | Yes (`6.8 GiB` via NF4) |
202
  | Top-1 Agreement vs. PyTorch BF16 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (reference) |
203
+ | 1. Short Intent Routing (293 tok, \(K=2\)) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
204
+ | 2. Typed Decisions (736 tok, \(K=5\)) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
205
+ | 3. When2Call Governance (1,068 tok, \(K=1\)) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
206
+ | 4. PhishNChips Security (1,303 tok, \(K=9\)) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
207
+ | 5. SafeAgents Judge (2,217 tok, \(K=2\)) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
208
+ | Batch \(B=4\) Typed Decisions (2,944 tok, 20 dec) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
209
 
210
  ### 4.2 Mixed-Precision GGUF Quantization Fidelity
211
 
 
215
 
216
  | Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
217
  | :--- | :---: | :---: | :---: |
218
+ | Top-1 Decision Agreement (\(N = 1,473\)) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
219
+ | Mean Absolute Probability Error (\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\)) | 0.00000 | 0.00203 | 0.00387 |
220
  | 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
221
  | 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
222
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
 
232
  | PyTorch `load_in_4bit=True` (NF4 + `fp16` head) | ~5.5 GB (mem) | 6.20 – 6.80 GiB | — | 8–16 GB CUDA GPU (Colab T4, consumer GPUs) |
233
  | `FastBrain-9B-Q8_0.gguf` | 9.66 GB | 9.03 – 12.55 GiB | 349.3 ms / 1,149.8 ms | 12–16 GB GPU (Tesla T4, RTX 4070 Ti), Apple Silicon (16 GB+) |
234
  | PyTorch `load_in_8bit=True` (INT8 + `bf16` head) | ~9.6 GB (mem) | ~11.20 GiB | — | 12–16 GB CUDA GPU |
235
+ | Native `bfloat16` (`safetensors` / `BF16.gguf`) | 18.82 / 18.16 GB | 20.30 – 24.70 GiB | — | ≥24 GB GPU (NVIDIA L4, A100, H100, RTX 3090/4090, PRO 6000) |
236
 
237
  ---
238
 
README_zh.md CHANGED
@@ -19,7 +19,7 @@
19
 
20
  ## 摘要
21
 
22
- 将自回归语言模型用于封闭 Schema 分类、工具路由与安全合规校验等结构化决策任务时,逐 Token 解码机制不仅引入与决策复杂度无关的序列生成延迟,还会因表面词形切分导致类别概率分布失准。本文发布 **FastBrain-9B**,一个总参数量为 9.41B 的非自回归结构化决策模型,由 32 层混合 Gated DeltaNet 线性注意力与全注意力主干网络(基于 `Cloudflare/clef-flash` 初始化的 `Qwen3.5-9B` 架构)及 116M 参数的交叉注意力联合决策头(`JointSchemaHead`)组成。给定任意 JSON 或多模态状态以及一组类型化决策定义(多选分类 `choice`、二元真值判别 `noul` 与序数量表评分 `score`),模型仅通过单次 Prefill 前向传播(`output_tokens = 0`)即可同步输出全部问题与候选项的校准概率分布。训练阶段,我们在复合序数 KL 散度、累积分布距离(离散分级概率评分 RPS)与期望值均方误差目标下,联合优化全部 32 层中的 104 个注意力投影矩阵及决策头参数,并将增量权重原地合并至主干张量。在涵盖 109 个参评模型条目的三项公开评测基准中:FastBrain-9B 在 `safeagents-judge-bench`(v2)7 类违规判别赛道位列 **第 1 / 8**(Macro-F1 `0.751`,准确率 `0.858`,较 27.4B 参数的 `Cloudflare/clef` 高出 18.8 个百分点),并在二元拦截赛道实现 `82/82` 越界轨迹全检出(AUROC `0.985`);在 `LocalLLaMA/typed-decisions` 零样本总榜位列 **第 7 / 43**(开源权重模型 **第 5 / 36**,$\le$9B 开源模型 **第 2 / 19**),中位推理延迟为 `50 ms`(较 31B 模型 `Quyet-1.0-Large` 快 66.7 倍),在冻结全部模型权重仅拟合先验与温度参数的设定下位列 `train` 校准榜 **并列第 12(列表第 14 / 51)**(准确率 `0.7970`,KL 散度 `0.0816`);在 Cloudflare Decision Index(v0.2.1)10 任务套件上取得 **80.55%** 的宏平均分,位列全量级 **第 2 / 7**、$\le$9B 量级 **10 / 10 任务单项第 1**,并在其中 5 项任务上超越 27.4B 的 `Cloudflare/clef`。
23
 
24
  ---
25
 
@@ -39,18 +39,20 @@ FastBrain-9B 主干网络包含 32 层混合序列建模模块(`9.30B` 参数
39
 
40
  ### 2.2 联合 Schema 决策头与向量化 CUDA 算子
41
 
42
- `JointSchemaHead`(`116M` 参数,独立存储于 `joint_head.safetensors`)从主干末层隐藏状态 $H \in \mathbb{R}^{T \times d}$ 中提取由 `<|question_start|>`/`<|question_end|>` 与 `<|option_start|>`/`<|option_end|>` 界定的片段表征,经问题-选项交叉注意力交互后映射为三类基础决策原语的标量对数几率:
43
- 1. **`choice`**($K$ 分类离散分布):输出归一化概率向量 $\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}$。
44
- 2. **`noul`**(二元真值判别):输出命题成立概率 $\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})$。
45
- 3. **`score`**(定义在有序分值 $v_1 < \dots < v_K$ 上的量表评估):同时输出各等级概率 $\hat{p} \in \Delta^{K-1}$ 与数学期望得分 $\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k$。
46
 
47
- 在参考运行时 `joint_schema_model.py` 中,针对 CUDA 设备实现了融合算子路径 `_forward_fast_cuda`:利用基于 `torch.cumsum` 的 $\mathcal{O}(1)$ 前缀和差分替代逐片段 Python 循环完成变长片段均值池化,并通过 `cu_seqlens` 紧凑打包调用变长 FlashAttention(`torch.ops.aten._flash_attention_forward`)替代稠密掩码注意力。在 NVIDIA RTX PRO 6000 Blackwell 上,该实现将决策头单次前向耗时由 `3.27 ms` 降至 `1.72 ms`(**1.90 倍加速**),且保持严格数值等价(最大 Logit 绝对偏差 `max_logit_diff = 0.00000`)。
48
 
49
  ### 2.3 序数 KL 蒸馏与编译期 Schema 拓扑校准
50
 
51
- 标准交叉熵或普通多类 KL 散度将有序评定量表视为无序独立类别,对相邻等级误差与跨端极端误差施加同等惩罚。FastBrain-9B 在 `TypeScript/FastBrain-Instruct` 数据集上采用复合分布匹配损失进行训练,联合优化软标签 KL 散度、离散分级概率评分(预测累积分布 $\hat{F}_k = \sum_{j \le k} \hat{p}_j$ 与目标累积分布 $F_k$ 之间的二阶距离)以及 `score` 题型的期望值均方误差:
52
 
53
- $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.$$
 
 
54
 
55
  在推理阶段,`joint_schema_model.py` 提供可选的编译期 Schema 拓扑校准函数(`apply_schema_aware_calibration`),通过解析候选项的语法结构特征(如 `<tool_call>` 调用声明、澄清追问子句及越界拒识选项)修正判别边际,抑制工具过度触发(Over-Triggering)与开放集误接收。
56
 
@@ -65,27 +67,27 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
65
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 7 类违规判别 ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 个决策模型 (+2 基线) | **0.751** Macro-F1 / **0.858** Acc | **1 / 8** | `Cloudflare/clef` (27.4B, `0.563`), `vllm-sr/d3-mini` (1.8B, `0.512`), `Cloudflare/clef-flash` (9.4B, `0.462`), `clef-omni` (30B, `0.437`) |
66
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 二元安全护栏 ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 个决策模型 (+2 基线) | **0.985** AUROC / **82/82** 轨迹检出<br>(`0.814` 默认;**0.938** 阈值校准) | **3 / 8** AUROC<br>**1 / 8** (并列) 轨迹 | `Cloudflare/clef` (27.4B, 默认 Bal. Acc `0.808`), `clef-flash` (`0.967` AUROC, `81/82`), `clef-omni` (`0.946` AUROC, `81/82`), `d1-3B` (`68/82`) |
67
  | [`safeagents-judge-bench` (v1)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v1) — 二元与 7 分类 ([PR #1](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/discussions/1)) | 9 个决策模型 (+2 基线) | **0.974** Bal. Acc / **0.850** Cat. F1 | **1 / 9** (双赛道第 1) | `Cloudflare/clef-omni` (30B, `0.845` / `0.327`), `Cloudflare/clef` (27.4B, `0.718` / `0.425`), `clef-flash` (`0.721` / `0.345`) |
68
- | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — 零样本评测 ([PR #58](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/58)) | 43 个模型 (36 开源, 19 个 $\le$9B 开源) | **0.7525** Acc (`0.7625` 并列口径)<br>`0.1508` KL · `0.0795` Brier · **50 ms** | **7 / 43** 总榜<br>**5 / 36** 开源 · **2 / 19** $\le$9B | `vllm-sr/d3` (27B, `0.749`), `Elephant 1` (9B, `0.748`, `352 ms`), `Darwin-27B` (`0.743`), `Cloudflare/clef` (27B, `0.728`), `clef-flash` (`0.710`) |
69
- | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — `train` 校准 ([PR #57](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/57)) | 51 个微调与校准模型 | **0.7970** Acc (`0.8070` 并列口径)<br>**0.0816** KL · **0.0437** Brier · **50 ms** | **并列 12** (列表 14 / 51)<br>冻结权重 $\le$9B 及 KL **第 1** | 冻结全部模型权重(`no weights changed`);高于 `Rev-gen1-midi` (25.6B, `0.7945`)、`OpenDecider-small-td` (4B LoRA, `0.792`)、`OpenDecider-medium-td` (30B LoRA, `0.788`) |
70
- | Cloudflare Decision Index (`v0.2.1`) — 10 任务套件 | 7 个模型 (0.4B–27.4B) | **80.55%** 宏平均 | **2 / 7** 全量级总榜<br>$\le$9B 量级 **10 / 10 单项第 1** | `Jev` (27B, `80.32%`), `Kev 9B` (`76.51%`), `Clef-flash` (9.4B, `74.96%`), `DiffGemma` (26B, `69.14%`);5 / 10 项任务高于 `27.4B Clef` |
71
 
72
  ---
73
 
74
  ### 3.1 智能体工具调用安全审计评测 (`safeagents-judge-bench`)
75
 
76
- [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) 用于评估决策模型对真实编码智能体(运行于 Terminal-Bench 2.1、OpenThoughts-TBLite 与 Terminal-Bench 4)在沙箱中被静态扫描器 `atif-scan` 标记的可疑工具调用的审计能力。`v2` 测试集包含来自 55 个任务的 $N = 661$ 次工具调用(其中 `536` 次为分布在 `82` 条违规轨迹中的恶意 `block` 调用,`125` 次为正常 `allow` 调用)。每次调用以零样本单次请求形式同时询问一个二元 `noul` 拦截问题与一个 7 类 `choice` 违规归因问题(`legit`、`eval_gaming`、`harness_introspection`、`credential_misuse`、`exfiltration`、`sandbox_escape`、`other_overreach`)。表 2 中的 `v2` 官方结果由基准维护者在 NVIDIA A100-SXM4-80GB(HF Jobs)上独立运行并提交合入([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1))。
77
 
78
- 在 7 类违规归因赛道上,FastBrain-9B 取得 `0.751` 的 Macro-F1 与 `0.858` 的分类准确率(`567/661`),位列全部参评模型第 1:相较于参数量约为其 3 倍的 27.4B 模型 `Cloudflare/clef`(Macro-F1 `0.563`,准确率 `0.737`),分别高出 `+18.8` 与 `+12.1` 个百分点;相较于同尺寸基座 `Cloudflare/clef-flash`(Macro-F1 `0.462`,准确率 `0.576`),分别提升 `+28.9` 与 `+28.2` 个百分点。分项指标上,`harness_introspection`($n=272$)F1 达 `0.945`(召回率 `0.978`),`eval_gaming`($n=245$)F1 达 `0.828`(召回率 `0.853`),合法调用 `legit`($n=125$)精确率达 `0.975`。
79
 
80
- 在二元安全拦截赛道上,FastBrain-9B 的 AUROC 为 `0.985`,恶意调用总召回率为 `0.996`(`534/536`),全部 `82/82` 条越界轨迹均在首次违规调用处即被捕获(`median_bad_calls_before_catch = 0.0`),其中 `harness_introspection`(`272/272`)、`credential_misuse`(`15/15`)与 `sandbox_escape`(`4/4`)三类召回率均为 `1.000`。在未经验证集调整的默认判别阈值 $P(\text{block}) \ge 0.50$ 下,平衡准确率为 `0.814`(误拦率 `0.368`,优于 27.4B `Cloudflare/clef` 的平衡准确率 `0.808` 与误拦率 `0.384`)。由于连续概率排序区分度保持在 `0.985` AUROC,将工作点阈值设定为 $P(\text{block}) \ge 0.961$ 时,平衡准确率提升至 `0.938`,误拦率降至 `0.056`(`7/125`),同时仍保持 `82/82` 越界轨迹全检出。表 2 同时列出了 `v1` 版本($N = 562$,78 条轨迹)的官方归档结果,FastBrain-9B 在该版本二元与多分类双赛道均位居第 1 / 9。
81
 
82
- **表 2:`harness-hub/safeagents-judge-bench` 零样本评测结果对比(`v2` 测试集 $N = 661$,82 条越界轨迹;附 `v1` 归档对照 $N = 562$)。**
83
 
84
  | 模型 | 参数量 | 7 分类 Macro-F1 ↑ | 7 分类准确率 ↑ | AUROC ↑ | 平衡准确率 ↑ | 恶意召回率 ↑ | 误拦率 ↓ | 越界轨迹检出 ↑ | p50 延迟(测试硬件) |
85
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
86
- | **TypeScript/FastBrain-9B** (`v2`, 默认 $t = 0.50$) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
87
- | **TypeScript/FastBrain-9B** (`v2`, 校准 $t = 0.961$) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
88
- | **TypeScript/FastBrain-9B** (`v1` 官方榜, $N = 562$) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
89
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
90
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
91
  | Cloudflare/clef-flash (基座) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
@@ -100,11 +102,11 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
100
 
101
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) 评测集覆盖智能体轨迹可观测性(`agent_trace_observability`)、客户服务(`customer_service`)、供应商发票核验(`invoice_processing`)与安全事件响应(`security_incidents`)四类企业工作流,测试集包含 400 个独立案例、共 2,000 个类型化子问题(涵盖 `noul`、`choice` 与 `score`),并以多教师模型集成的软标签分布作为真值基准。官方仓库存在两种并列真值处理脚本:`bench/score_with_jobs.py` 对真值分布中概率并列最高的选项按首个索引判对(`g.index(max(g))`),而 `scripts/score_model.py` 将 20 道并列题中的任一最高分选项均计为正确;两种脚本下的 KL 散度与 Brier 评分完全一致。我们在表 3 中同时报告两种口径的准确率,并在 [`eval_submissions/`](./eval_submissions/) 中公开原始预测文件。
102
 
103
- 在零样本总榜($N = 43$ 个参评模型)中,FastBrain-9B 取得 `0.7525`(并列口径 `0.7625`)的准确率、`0.1508` 的 KL 散度、`0.0795` 的 Brier 评分与 `0.085` 的期望校准误差(ECE),在 RTX PRO 6000(SGLang)上的 p50 延迟为 `50 ms`。该成绩位列全量零样本榜 **第 7 / 43**、开源权重模型 **第 5 / 36**、$\le$9B 开源模型 **第 2 / 19** 以及全部 `Clef` 架构模型 **第 1**,较 9.4B 基座 `Cloudflare/clef-flash`(准确率 `0.7100`,KL `0.2093`)提升 `+4.25` 个百分点,较 27.4B `Cloudflare/clef`(准确率 `0.7275`,KL `0.1907`)提升 `+2.50` 个百分点。在零样本准确率高于 `0.700` 的全部 28 个模型中,FastBrain-9B 具有最低的 p50 推理延迟(`50 ms`,对比 9B `Amos 1 medium` 的 `118 ms`、27B `Amos 1 large` 的 `357 ms`、`meraGPT Decider 1` 的 `526 ms` 以及 31B `Quyet-1.0-Large` 的 `3,335 ms`)。若直接评测未经后验规则调整的序数 KL 蒸馏原生头,零样本准确率为 `0.7735`(KL `0.1039`,Brier `0.0582`)。
104
 
105
- 在使用 `train` 划分(1,200 个样本)的独立榜单($N = 51$ 个模型)上,在**完全冻结主干与决策头权重(`no weights changed`)**、仅拟合逐问题对数先验偏置与标量温度参数的前提下,FastBrain-9B 取得 `0.7970`(并列口径 `0.8070`)的准确率、`0.0816` 的 KL 散度与 `0.0437` 的 Brier 评分,位列 **并列第 12(列表第 14 / 51)**,并在冻结权重模型中取得最优的 KL 分布校准度。
106
 
107
- **表 3:`LocalLLaMA/typed-decisions` 测试集($N = 400$ 案例,2,000 次决策)评测结果。`/` 前后分别对应 `score_with_jobs.py` 与 `score_model.py` 口径。**
108
 
109
  | 模型 | 参数量 | 评测设定 | 总体准确率 ↑ | KL 散度 ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | 客服 ↑ | 发票 ↑ | 安全 ↑ | p50 延迟 |
110
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -126,9 +128,9 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
126
 
127
  ### 3.3 多领域结构化决策基准 (Cloudflare Decision Index `v0.2.1`)
128
 
129
- 在涵盖 10 个评测数据集、共 $N = 22,427$ 条请求的 **Cloudflare Decision Index(`v0.2.1`)** 套件(覆盖 AST 函数调用、20 候选工具检索重排、多步状态 API 调用、细粒度金融意图识别、开放集越界拒识、推理密集型检索、电商相关性分级、多探针钓鱼网站鉴别与工具调用触发治理)上,FastBrain-9B 的无加权 10 任务宏平均得分为 **80.55%**(表 4,图 1–3)。
130
 
131
- 在 $\le$9B 参数量级(对比 `Clef-flash` 9.4B、`Kev 9B` 与 `Laya` 0.4B)中,FastBrain-9B 在全部 **10 / 10 个子基准上均取得第 1**,宏平均分较同架构基座 `Clef-flash`(`74.96%`)提升 `+5.59` 个百分点,较 `Kev 9B`(`76.51%`)提升 `+4.04` 个百分点。增幅主要集中于对边界校准敏感的开放集拒识与安全治理任务:`CLINC150+OOS` 提升 `+31.67` 个百分点(`98.44%` 对比 `66.77%`),`When2Call` 提升 `+15.55` 个百分点(`81.13%` 对比 `65.58%`),`PhishNChips` 提升 `+6.80` 个百分点(`78.75%` 对比 `71.95%`)。跨全部参数规模(最高 27.4B)比较,FastBrain-9B 总宏平均位列 **第 2 / 7**,高于 `27B Jev`(`80.32%`)与 `26B-A4B DiffusionGemma Jev`(`69.14%`),并在 `BFCL`、`API-Bank`、`CLINC150+OOS`、`Home Appliances` 与 `When2Call` 五个子基准上高于 `27.4B Clef`(`81.81%`)。
132
 
133
  ![图 1:Cloudflare Decision Index v0.2.1 总体宏平均与参数效率对比](./assets/benchmark_overview.png)
134
  *图 1. Cloudflare Decision Index(v0.2.1,22,427 条请求)10 任务宏平均分与模型参数规模关系。*
@@ -139,7 +141,7 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
139
  ![图 3:置信度校准曲线与开放集越界拒识分析](./assets/decision_calibration_breakdown.png)
140
  *图 3. 置信度可靠性图、开放集越界意图拒识(`CLINC150+OOS`)与工具触发边际分析(`When2Call`)。*
141
 
142
- **表 4:Cloudflare Decision Index(`v0.2.1`,10 任务,$N = 22,427$)逐项评测结果。粗体表示全参数规模最高值,下划线表示 $\le$9B 量级最高值。**
143
 
144
  | 评测基准(指标) | **FastBrain-9B** (本工作) | Clef-flash (基座) | Kev 9B | Clef | Jev | DiffGemma | Laya |
145
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -162,7 +164,7 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
162
 
163
  ### 4.1 不同推理引擎性能对比
164
 
165
- 表 5 展示了在同一张 NVIDIA RTX PRO 6000 Blackwell Server Edition GPU 上,SGLang(集成 `JointSchemaPooler` 的 `lmsysorg/sglang:dev-clef`)、`llama.cpp`(`b11430` `ClefModel` 计算图,含 `Q8_0` 与 `Q4_K_M` 两种精度)以及原生 PyTorch(`joint_schema_model.py`)在五类典型长度负载下的端到端延迟、Prefill 吞吐率与显存占用。由于全部请求的生成步数均为零(`output_tokens = 0`),推理耗时完全取决于 Prefill 计算与片段池化效率;在批量 $B=4$ 并发设定下,SGLang 吞吐达 `19,319 tok/s`(每秒完成 `131.2` 个独立结构化决策)。
166
 
167
  **表 5:NVIDIA RTX PRO 6000 Blackwell Server Edition(G4)上三类推理后端的延迟与吞吐实测。**
168
 
@@ -172,23 +174,23 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
172
  | 峰值显存占用 (16K 上下文) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
173
  | 兼容 16 GiB T4 显卡 | 否 (BF16 TP=1) | 是 (`9.03 GiB`, 4K) | **是 (`6.78 GiB`, 4K)** | 是 (`6.8 GiB`, NF4) |
174
  | 相对 PyTorch BF16 Top-1 一致率 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (基准) |
175
- | 1. 短文本意图路由 (293 tok, $K=2$) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
176
- | 2. Typed Decisions (736 tok, $K=5$) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
177
- | 3. When2Call 工具治理 (1,068 tok, $K=1$) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
178
- | 4. PhishNChips 安全检测 (1,303 tok, $K=9$) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
179
- | 5. SafeAgents 审计 (2,217 tok, $K=2$) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
180
- | 批量 $B=4$ Typed Decisions (2,944 tok, 20 题) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
181
 
182
  ### 4.2 混合精度 GGUF 量化保真度分析
183
 
184
  若对交叉注意力决策头或线性注意力递归状态直接施加均匀 4-bit 量化,细粒度选项间微小的对数几率差值易受量化噪声干扰。因此在 `FastBrain-9B-Q4_K_M.gguf`(`6.73 GB`)构建方案中,仅对前馈网络(FFN)使用 4-bit K-quants,而将全部 `dec.*` 决策头张量、`ssm_*` Gated DeltaNet 状态参数以及 `attn_*` 投影矩阵保持为 `Q8_0` 精度。在涵盖 10 类任务的分层验证子集(`203` 个多约束请求,共 `1,473` 道独立评分题;表 6)上,`Q8_0.gguf`(`9.66 GB`)在 `1,471/1,473` 道题上与原生 `bfloat16` 给出完全一致的 Top-1 决策(一致率 `99.86%`,预测概率平均绝对误差 `0.00203`,10 任务均分差异 `-0.03` 个百分点);`Q4_K_M.gguf` 保持了 `98.78%` 的 Top-1 决策一致率(概率 MAE `0.00387`,校准后均分差异 `-0.20` 个百分点)。表 7 给出了各精度版本在不同硬件(含 NVIDIA Tesla T4 `16 GiB` 实测)上的资源需求。
185
 
186
- **表 6:10 任务分层子集($N = 1,473$ 道评分题)上 GGUF 量化模型的决策一致性与精度保持率。**
187
 
188
  | 评测指标 / 子任务 | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` 决策头) |
189
  | :--- | :---: | :---: | :---: |
190
- | Top-1 决策一致率 ($N = 1,473$) | 100.00% (基准) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
191
- | 预测概率平均绝对误差 ($\|\hat{p}_q - \hat{p}_{\text{bf16}}\|$) | 0.00000 | 0.00203 | 0.00387 |
192
  | 10 任务宏平均(未校准原生输出) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
193
  | 10 任务宏平均(Schema 拓扑校准) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
194
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
@@ -204,7 +206,7 @@ $$\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{
204
  | PyTorch `load_in_4bit=True` (NF4 + `fp16` 头) | ~5.5 GB (内存) | 6.20 – 6.80 GiB | — | 8–16 GB CUDA 显卡 (Colab T4, 消费级 GPU) |
205
  | `FastBrain-9B-Q8_0.gguf` | 9.66 GB | 9.03 – 12.55 GiB | 349.3 ms / 1,149.8 ms | 12–16 GB 显卡 (Tesla T4, RTX 4070 Ti), Apple Silicon (16 GB+) |
206
  | PyTorch `load_in_8bit=True` (INT8 + `bf16` 头) | ~9.6 GB (内存) | ~11.20 GiB | — | 12–16 GB CUDA 显卡 |
207
- | 原生 `bfloat16` (`safetensors` / `BF16.gguf`) | 18.82 / 18.16 GB | 20.30 – 24.70 GiB | — | $\ge$24 GB 显卡 (NVIDIA L4, A100, H100, RTX 3090/4090, PRO 6000) |
208
 
209
  ---
210
 
 
19
 
20
  ## 摘要
21
 
22
+ 将自回归语言模型用于封闭 Schema 分类、工具路由与安全合规校验等结构化决策任务时,逐 Token 解码机制不仅引入与决策复杂度无关的序列生成延迟,还会因表面词形切分导致类别概率分布失准。本文发布 **FastBrain-9B**,一个总参数量为 9.41B 的非自回归结构化决策模型,由 32 层混合 Gated DeltaNet 线性注意力与全注意力主干网络(基于 `Cloudflare/clef-flash` 初始化的 `Qwen3.5-9B` 架构)及 116M 参数的交叉注意力联合决策头(`JointSchemaHead`)组成。给定任意 JSON 或多模态状态以及一组类型化决策定义(多选分类 `choice`、二元真值判别 `noul` 与序数量表评分 `score`),模型仅通过单次 Prefill 前向传播(`output_tokens = 0`)即可同步输出全部问题与候选项的校准概率分布。训练阶段,我们在复合序数 KL 散度、累积分布距离(离散分级概率评分 RPS)与期望值均方误差目标下,联合优化全部 32 层中的 104 个注意力投影矩阵及决策头参数,并将增量权重原地合并至主干张量。在涵盖 109 个参评模型条目的三项公开评测基准中:FastBrain-9B 在 `safeagents-judge-bench`(v2)7 类违规判别赛道位列 **第 1 / 8**(Macro-F1 `0.751`,准确率 `0.858`,较 27.4B 参数的 `Cloudflare/clef` 高出 18.8 个百分点),并在二元拦截赛道实现 `82/82` 越界轨迹全检出(AUROC `0.985`);在 `LocalLLaMA/typed-decisions` 零样本总榜位列 **第 7 / 43**(开源权重模型 **第 5 / 36**,≤9B 开源模型 **第 2 / 19**),中位推理延迟为 `50 ms`(较 31B 模型 `Quyet-1.0-Large` 快 66.7 倍),在冻结全部模型权重仅拟合先验与温度参数的设定下位列 `train` 校准榜 **并列第 12(列表第 14 / 51)**(准确率 `0.7970`,KL 散度 `0.0816`);在 Cloudflare Decision Index(v0.2.1)10 任务套件上取得 **80.55%** 的宏平均分,位列全量级 **第 2 / 7**、≤9B 量级 **10 / 10 任务单项第 1**,并在其中 5 项任务上超越 27.4B 的 `Cloudflare/clef`。
23
 
24
  ---
25
 
 
39
 
40
  ### 2.2 联合 Schema 决策头与向量化 CUDA 算子
41
 
42
+ `JointSchemaHead`(`116M` 参数,独立存储于 `joint_head.safetensors`)从主干末层隐藏状态 \(H \in \mathbb{R}^{T \times d}\) 中提取由 `<|question_start|>`/`<|question_end|>` 与 `<|option_start|>`/`<|option_end|>` 界定的片段表征,经问题-选项交叉注意力交互后映射为三类基础决策原语的标量对数几率:
43
+ 1. **`choice`**(\(K\) 分类离散分布):输出归一化概率向量 \(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\)。
44
+ 2. **`noul`**(二元真值判别):输出命题成立概率 \(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\)。
45
+ 3. **`score`**(定义在有序分值 \(v_1 < \dots < v_K\) 上的量表评估):同时输出各等级概率 \(\hat{p} \in \Delta^{K-1}\) 与数学期望得分 \(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\)。
46
 
47
+ 在参考运行时 `joint_schema_model.py` 中,针对 CUDA 设备实现了融合算子路径 `_forward_fast_cuda`:利用基于 `torch.cumsum` 的 \(\mathcal{O}(1)\) 前缀和差分替代逐片段 Python 循环完成变长片段均值池化,并通过 `cu_seqlens` 紧凑打包调用变长 FlashAttention(`torch.ops.aten._flash_attention_forward`)替代稠密掩码注意力。在 NVIDIA RTX PRO 6000 Blackwell 上,该实现将决策头单次前向耗时由 `3.27 ms` 降至 `1.72 ms`(**1.90 倍加速**),且保持严格数值等价(最大 Logit 绝对偏差 `max_logit_diff = 0.00000`)。
48
 
49
  ### 2.3 序数 KL 蒸馏与编译期 Schema 拓扑校准
50
 
51
+ 标准交叉熵或普通多类 KL 散度将有序评定量表视为无序独立类别,对相邻等级误差与跨端极端误差施加同等惩罚。FastBrain-9B 在 `TypeScript/FastBrain-Instruct` 数据集上采用复合分布匹配损失进行训练,联合优化软标签 KL 散度、离散分级概率评分(预测累积分布 \(\hat{F}_k = \sum_{j \le k} \hat{p}_j\) 与目标累积分布 \(F_k\) 之间的二阶距离)以及 `score` 题型的期望值均方误差:
52
 
53
+ $$
54
+ \mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
55
+ $$
56
 
57
  在推理阶段,`joint_schema_model.py` 提供可选的编译期 Schema 拓扑校准函数(`apply_schema_aware_calibration`),通过解析候选项的语法结构特征(如 `<tool_call>` 调用声明、澄清追问子句及越界拒识选项)修正判别边际,抑制工具过度触发(Over-Triggering)与开放集误接收。
58
 
 
67
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 7 类违规判别 ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 个决策模型 (+2 基线) | **0.751** Macro-F1 / **0.858** Acc | **1 / 8** | `Cloudflare/clef` (27.4B, `0.563`), `vllm-sr/d3-mini` (1.8B, `0.512`), `Cloudflare/clef-flash` (9.4B, `0.462`), `clef-omni` (30B, `0.437`) |
68
  | [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 二元安全护栏 ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 个决策模型 (+2 基线) | **0.985** AUROC / **82/82** 轨迹检出<br>(`0.814` 默认;**0.938** 阈值校准) | **3 / 8** AUROC<br>**1 / 8** (并列) 轨迹 | `Cloudflare/clef` (27.4B, 默认 Bal. Acc `0.808`), `clef-flash` (`0.967` AUROC, `81/82`), `clef-omni` (`0.946` AUROC, `81/82`), `d1-3B` (`68/82`) |
69
  | [`safeagents-judge-bench` (v1)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v1) — 二元与 7 分类 ([PR #1](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/discussions/1)) | 9 个决策模型 (+2 基线) | **0.974** Bal. Acc / **0.850** Cat. F1 | **1 / 9** (双赛道第 1) | `Cloudflare/clef-omni` (30B, `0.845` / `0.327`), `Cloudflare/clef` (27.4B, `0.718` / `0.425`), `clef-flash` (`0.721` / `0.345`) |
70
+ | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — 零样本评测 ([PR #58](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/58)) | 43 个模型 (36 开源, 19 个 ≤9B 开源) | **0.7525** Acc (`0.7625` 并列口径)<br>`0.1508` KL · `0.0795` Brier · **50 ms** | **7 / 43** 总榜<br>**5 / 36** 开源 · **2 / 19** ≤9B | `vllm-sr/d3` (27B, `0.749`), `Elephant 1` (9B, `0.748`, `352 ms`), `Darwin-27B` (`0.743`), `Cloudflare/clef` (27B, `0.728`), `clef-flash` (`0.710`) |
71
+ | [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — `train` 校准 ([PR #57](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/57)) | 51 个微调与校准模型 | **0.7970** Acc (`0.8070` 并列口径)<br>**0.0816** KL · **0.0437** Brier · **50 ms** | **并列 12** (列表 14 / 51)<br>冻结权重 ≤9B 及 KL **第 1** | 冻结全部模型权重(`no weights changed`);高于 `Rev-gen1-midi` (25.6B, `0.7945`)、`OpenDecider-small-td` (4B LoRA, `0.792`)、`OpenDecider-medium-td` (30B LoRA, `0.788`) |
72
+ | Cloudflare Decision Index (`v0.2.1`) — 10 任务套件 | 7 个模型 (0.4B–27.4B) | **80.55%** 宏平均 | **2 / 7** 全量级总榜<br>≤9B 量级 **10 / 10 单项第 1** | `Jev` (27B, `80.32%`), `Kev 9B` (`76.51%`), `Clef-flash` (9.4B, `74.96%`), `DiffGemma` (26B, `69.14%`);5 / 10 项任务高于 `27.4B Clef` |
73
 
74
  ---
75
 
76
  ### 3.1 智能体工具调用安全审计评测 (`safeagents-judge-bench`)
77
 
78
+ [`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) 用于评估决策模型对真实编码智能体(运行于 Terminal-Bench 2.1、OpenThoughts-TBLite 与 Terminal-Bench 4)在沙箱中被静态扫描器 `atif-scan` 标记的可疑工具调用的审计能力。`v2` 测试集包含来自 55 个任务的 \(N = 661\) 次工具调用(其中 `536` 次为分布在 `82` 条违规轨迹中的恶意 `block` 调用,`125` 次为正常 `allow` 调用)。每次调用以��样本单次请求形式同时询问一个二元 `noul` 拦截问题与一个 7 类 `choice` 违规归因问题(`legit`、`eval_gaming`、`harness_introspection`、`credential_misuse`、`exfiltration`、`sandbox_escape`、`other_overreach`)。表 2 中的 `v2` 官方结果由基准维护者在 NVIDIA A100-SXM4-80GB(HF Jobs)上独立运行并提交合入([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1))。
79
 
80
+ 在 7 类违规归因赛道上,FastBrain-9B 取得 `0.751` 的 Macro-F1 与 `0.858` 的分类准确率(`567/661`),位列全部参评模型第 1:相较于参数量约为其 3 倍的 27.4B 模型 `Cloudflare/clef`(Macro-F1 `0.563`,准确率 `0.737`),分别高出 `+18.8` 与 `+12.1` 个百分点;相较于同尺寸基座 `Cloudflare/clef-flash`(Macro-F1 `0.462`,准确率 `0.576`),分别提升 `+28.9` 与 `+28.2` 个百分点。分项指标上,`harness_introspection`(\(n=272\))F1 达 `0.945`(召回率 `0.978`),`eval_gaming`(\(n=245\))F1 达 `0.828`(召回率 `0.853`),合法调用 `legit`(\(n=125\))精确率达 `0.975`。
81
 
82
+ 在二元安全拦截赛道上,FastBrain-9B 的 AUROC 为 `0.985`,恶意调用总召回率为 `0.996`(`534/536`),全部 `82/82` 条越界轨迹均在首次违规调用处即被捕获(`median_bad_calls_before_catch = 0.0`),其中 `harness_introspection`(`272/272`)、`credential_misuse`(`15/15`)与 `sandbox_escape`(`4/4`)三类召回率均为 `1.000`。在未经验证集调整的默认判别阈值 \(P(\text{block}) \ge 0.50\) 下,平衡准确率为 `0.814`(误拦率 `0.368`,优于 27.4B `Cloudflare/clef` 的平衡准确率 `0.808` 与误拦率 `0.384`)。由于连续概率排序区分度保持在 `0.985` AUROC,将工作点阈值设定为 \(P(\text{block}) \ge 0.961\) 时,平衡准确率提升至 `0.938`,误拦率降至 `0.056`(`7/125`),同时仍保持 `82/82` 越界轨迹全检出。表 2 同时列出了 `v1` 版本(\(N = 562\),78 条轨迹)的官方归档结果,FastBrain-9B 在该版本二元与多分类双赛道均位居第 1 / 9。
83
 
84
+ **表 2:`harness-hub/safeagents-judge-bench` 零样本评测结果对比(`v2` 测试集 \(N = 661\),82 条越界轨迹;附 `v1` 归档对照 \(N = 562\))。**
85
 
86
  | 模型 | 参数量 | 7 分类 Macro-F1 ↑ | 7 分类准确率 ↑ | AUROC ↑ | 平衡准确率 ↑ | 恶意召回率 ↑ | 误拦率 ↓ | 越界轨迹检出 ↑ | p50 延迟(测试硬件) |
87
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
88
+ | **TypeScript/FastBrain-9B** (`v2`, 默认 \(t = 0.50\)) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
89
+ | **TypeScript/FastBrain-9B** (`v2`, 校准 \(t = 0.961\)) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
90
+ | **TypeScript/FastBrain-9B** (`v1` 官方榜, \(N = 562\)) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
91
  | Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
92
  | vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
93
  | Cloudflare/clef-flash (基座) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
 
102
 
103
  [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) 评测集覆盖智能体轨迹可观测性(`agent_trace_observability`)、客户服务(`customer_service`)、供应商发票核验(`invoice_processing`)与安全事件响应(`security_incidents`)四类企业工作流,测试集包含 400 个独立案例、共 2,000 个类型化子问题(涵盖 `noul`、`choice` 与 `score`),并以多教师模型集成的软标签分布作为真值基准。官方仓库存在两种并列真值处理脚本:`bench/score_with_jobs.py` 对真值分布中概率并列最高的选项按首个索引判对(`g.index(max(g))`),而 `scripts/score_model.py` 将 20 道并列题中的任一最高分选项均计为正确;两种脚本下的 KL 散度与 Brier 评分完全一致。我们在表 3 中同时报告两种口径的准确率,并在 [`eval_submissions/`](./eval_submissions/) 中公开原始预测文件。
104
 
105
+ 在零样本总榜(\(N = 43\) 个参评模型)中,FastBrain-9B 取得 `0.7525`(并列口径 `0.7625`)的准确率、`0.1508` 的 KL 散度、`0.0795` 的 Brier 评分与 `0.085` 的期望校准误差(ECE),在 RTX PRO 6000(SGLang)上的 p50 延迟为 `50 ms`。该成绩位列全量零样本榜 **第 7 / 43**、开源权重模型 **第 5 / 36**、≤9B 开源模型 **第 2 / 19** 以及全部 `Clef` 架构模型 **第 1**,较 9.4B 基座 `Cloudflare/clef-flash`(准确率 `0.7100`,KL `0.2093`)提升 `+4.25` 个百分点,较 27.4B `Cloudflare/clef`(准确率 `0.7275`,KL `0.1907`)提升 `+2.50` 个百分点。在零样本准确率高于 `0.700` 的全部 28 个模型中,FastBrain-9B 具有最低的 p50 推理延迟(`50 ms`,对比 9B `Amos 1 medium` 的 `118 ms`、27B `Amos 1 large` 的 `357 ms`、`meraGPT Decider 1` 的 `526 ms` 以及 31B `Quyet-1.0-Large` 的 `3,335 ms`)。若直接评测未经后验规则调整的序数 KL 蒸馏原生头,零样本准确率为 `0.7735`(KL `0.1039`,Brier `0.0582`)。
106
 
107
+ 在使用 `train` 划分(1,200 个样本)的独立榜单(\(N = 51\) 个模型)上,在**完全冻结主干与决策头权重(`no weights changed`)**、仅拟合逐问题对数先验偏置与标量温度参数的前提下,FastBrain-9B 取得 `0.7970`(并列口径 `0.8070`)的准确率、`0.0816` 的 KL 散度与 `0.0437` 的 Brier 评分,位列 **并列第 12(列表第 14 / 51)**,并在冻结权重模型中取得最优的 KL 分布校准度。
108
 
109
+ **表 3:`LocalLLaMA/typed-decisions` 测试集(\(N = 400\) 案例,2,000 次决策)评测结果。`/` ��后分别对应 `score_with_jobs.py` 与 `score_model.py` 口径。**
110
 
111
  | 模型 | 参数量 | 评测设定 | 总体准确率 ↑ | KL 散度 ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | 客服 ↑ | 发票 ↑ | 安全 ↑ | p50 延迟 |
112
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
128
 
129
  ### 3.3 多领域结构化决策基准 (Cloudflare Decision Index `v0.2.1`)
130
 
131
+ 在涵盖 10 个评测数据集、共 \(N = 22,427\) 条请求的 **Cloudflare Decision Index(`v0.2.1`)** 套件(覆盖 AST 函数调用、20 候选工具检索重排、多步状态 API 调用、细粒度金融意图识别、开放集越界拒识、推理密集型检索、电商相关性分级、多探针钓鱼网站鉴别与工具调用触发治理)上,FastBrain-9B 的无加权 10 任务宏平均得分为 **80.55%**(表 4,图 1–3)。
132
 
133
+ 在 ≤9B 参数量级(对比 `Clef-flash` 9.4B、`Kev 9B` 与 `Laya` 0.4B)中,FastBrain-9B 在全部 **10 / 10 个子基准上均取得第 1**,宏平均分较同架构基座 `Clef-flash`(`74.96%`)提升 `+5.59` 个百分点,较 `Kev 9B`(`76.51%`)提升 `+4.04` 个百分点。增幅主要集中于对边界校准敏感的开放集拒识与安全治理任务:`CLINC150+OOS` 提升 `+31.67` 个百分点(`98.44%` 对比 `66.77%`),`When2Call` 提升 `+15.55` 个百分点(`81.13%` 对比 `65.58%`),`PhishNChips` 提升 `+6.80` 个百分点(`78.75%` 对比 `71.95%`)。跨全部参数规模(最高 27.4B)比较,FastBrain-9B 总宏平均位列 **第 2 / 7**,高于 `27B Jev`(`80.32%`)与 `26B-A4B DiffusionGemma Jev`(`69.14%`),并在 `BFCL`、`API-Bank`、`CLINC150+OOS`、`Home Appliances` 与 `When2Call` 五个子基准上高于 `27.4B Clef`(`81.81%`)。
134
 
135
  ![图 1:Cloudflare Decision Index v0.2.1 总体宏平均与参数效率对比](./assets/benchmark_overview.png)
136
  *图 1. Cloudflare Decision Index(v0.2.1,22,427 条请求)10 任务宏平均分与模型参数规模关系。*
 
141
  ![图 3:置信度校准曲线与开放集越界拒识分析](./assets/decision_calibration_breakdown.png)
142
  *图 3. 置信度可靠性图、开放集越界意图拒识(`CLINC150+OOS`)与工具触发边际分析(`When2Call`)。*
143
 
144
+ **表 4:Cloudflare Decision Index(`v0.2.1`,10 任务,\(N = 22,427\))逐项评测结果。粗体表示全参数规模最高值,下划线表示 ≤9B 量级最高值。**
145
 
146
  | 评测基准(指标) | **FastBrain-9B** (本工作) | Clef-flash (基座) | Kev 9B | Clef | Jev | DiffGemma | Laya |
147
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
164
 
165
  ### 4.1 不同推理引擎性能对比
166
 
167
+ 表 5 展示了在同一张 NVIDIA RTX PRO 6000 Blackwell Server Edition GPU 上,SGLang(集成 `JointSchemaPooler` 的 `lmsysorg/sglang:dev-clef`)、`llama.cpp`(`b11430` `ClefModel` 计算图,含 `Q8_0` 与 `Q4_K_M` 两种精度)以及原生 PyTorch(`joint_schema_model.py`)在五类典型长度负载下的端到端延迟、Prefill 吞吐率与显存占用。由于全部请求的生成步数均为零(`output_tokens = 0`),推理耗时完全取决于 Prefill 计算与片段池化效率;在批量 \(B=4\) 并发设定下,SGLang 吞吐达 `19,319 tok/s`(每秒完成 `131.2` 个独立结构化决策)。
168
 
169
  **表 5:NVIDIA RTX PRO 6000 Blackwell Server Edition(G4)上三类推理后端的延迟与吞吐实测。**
170
 
 
174
  | 峰值显存占用 (16K 上下文) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
175
  | 兼容 16 GiB T4 显卡 | 否 (BF16 TP=1) | 是 (`9.03 GiB`, 4K) | **是 (`6.78 GiB`, 4K)** | 是 (`6.8 GiB`, NF4) |
176
  | 相对 PyTorch BF16 Top-1 一致率 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (基准) |
177
+ | 1. 短文本意图路由 (293 tok, \(K=2\)) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
178
+ | 2. Typed Decisions (736 tok, \(K=5\)) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
179
+ | 3. When2Call 工具治理 (1,068 tok, \(K=1\)) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
180
+ | 4. PhishNChips 安全检测 (1,303 tok, \(K=9\)) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
181
+ | 5. SafeAgents 审计 (2,217 tok, \(K=2\)) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
182
+ | 批量 \(B=4\) Typed Decisions (2,944 tok, 20 题) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
183
 
184
  ### 4.2 混合精度 GGUF 量化保真度分析
185
 
186
  若对交叉注意力决策头或线性注意力递归状态直接施加均匀 4-bit 量化,细粒度选项间微小的对数几率差值易受量化噪声干扰。因此在 `FastBrain-9B-Q4_K_M.gguf`(`6.73 GB`)构建方案中,仅对前馈网络(FFN)使用 4-bit K-quants,而将全部 `dec.*` 决策头张量、`ssm_*` Gated DeltaNet 状态参数以及 `attn_*` 投影矩阵保持为 `Q8_0` 精度。在涵盖 10 类任务的分层验证子集(`203` 个多约束请求,共 `1,473` 道独立评分题;表 6)上,`Q8_0.gguf`(`9.66 GB`)在 `1,471/1,473` 道题上与原生 `bfloat16` 给出完全一致的 Top-1 决策(一致率 `99.86%`,预测概率平均绝对误差 `0.00203`,10 任务均分差异 `-0.03` 个百分点);`Q4_K_M.gguf` 保持了 `98.78%` 的 Top-1 决策一致率(概率 MAE `0.00387`,校准后均分差异 `-0.20` 个百分点)。表 7 给出了各精度版本在不同硬件(含 NVIDIA Tesla T4 `16 GiB` 实测)上的资源需求。
187
 
188
+ **表 6:10 任务分层子集(\(N = 1,473\) 道评分题)上 GGUF 量化模型的决策一致性与精度保持率。**
189
 
190
  | 评测指标 / 子任务 | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` 决策头) |
191
  | :--- | :---: | :---: | :---: |
192
+ | Top-1 决策一致率 (\(N = 1,473\)) | 100.00% (基准) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
193
+ | 预测概率平均绝对误差 (\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\)) | 0.00000 | 0.00203 | 0.00387 |
194
  | 10 任务宏平均(未校准原生输出) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
195
  | 10 任务宏平均(Schema 拓扑校准) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
196
  | ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
 
206
  | PyTorch `load_in_4bit=True` (NF4 + `fp16` 头) | ~5.5 GB (内存) | 6.20 – 6.80 GiB | — | 8–16 GB CUDA 显卡 (Colab T4, 消费级 GPU) |
207
  | `FastBrain-9B-Q8_0.gguf` | 9.66 GB | 9.03 – 12.55 GiB | 349.3 ms / 1,149.8 ms | 12–16 GB 显卡 (Tesla T4, RTX 4070 Ti), Apple Silicon (16 GB+) |
208
  | PyTorch `load_in_8bit=True` (INT8 + `bf16` 头) | ~9.6 GB (内存) | ~11.20 GiB | — | 12–16 GB CUDA 显卡 |
209
+ | 原生 `bfloat16` (`safetensors` / `BF16.gguf`) | 18.82 / 18.16 GB | 20.30 – 24.70 GiB | — | ≥24 GB 显卡 (NVIDIA L4, A100, H100, RTX 3090/4090, PRO 6000) |
210
 
211
  ---
212