You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

tiny-aya-global-sqlcf-grpo-c

LoRA adapter (r=16) for CohereLabs/tiny-aya-global: GRPO, 60 steps, counterfactual-DB reward R_cf_mean (condition C).

Lexsi Labs AI Safety & Alignment Hackathon, Track 3 (PS #6 data and database agents + #9 reward design and reward-hacking defences), team Teen_khapeetars: Shreshth Rai (lead, @hi-phi-r25i), Rishii Kumar Singh (@LogicPalette), Aksh Aggarwal (@AkshAgg).

Collections: models, data · Demo · Data · Adapters: SFT, GRPO-B, GRPO-C · Code · Write-up

Status (gate G3): null: no gain over SFT (p = 0.21) and below GRPO-B; reported as a null.

Test set: 331 items x 8 samples, T=0.7, fresh databases from a different generator; "verified" = correct on the original and all 4 counterfactual DBs.

Metric Value (95% CI)
Verified accuracy 9.0% [7.9, 10.1]
Single-DB accuracy 14.0% [12.8, 15.4]
Silent-error rate 61.2% [59.3, 63.0]
Held-out family F4, verified 8.8% [7.5, 10.2]

Use

Accept the base model's licence first, then:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c")
tok = AutoTokenizer.from_pretrained("sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c")

It is a multi-turn SQL agent with one run_sql tool, prompted and parsed the way AgentTune renders Tiny Aya (see sqlcf/agent.py and evaluate.py gen in the repo). vLLM LoRA fails on Tiny Aya's tied embeddings, so for vLLM merge first: python train.py merge --init <this adapter> --out merged (fp32 merge, saved fp16).

Licence

This adapter is a derivative of CohereLabs/tiny-aya-global and is released under the same terms:

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c

Adapter
(388)
this model

Dataset used to train sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c

Space using sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c 1

Collection including sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c