Papers
arxiv:2610.26190

VEILED-SH: Vulnerability Expression through Indirect Language Evaluation Dataset (Self-Harm)

Published on Oct 19
Authors:
,
,
,
,
,
,
,
,
,

Abstract

VEILED-SH is a benchmark for assessing AI responses to indirect distress expressions across cultural and application contexts using practitioner-calibrated profiles.

AI systems may miss indirect, culturally situated distress or provide support mismatched to the application context. Existing benchmarks do not jointly address cultural variation, application context, and practitioner disagreement. We introduce VEILED-SH, a benchmark and construction methodology for evaluating single-turn responses to indirect expressions of suicide, self-harm, and distress. Its 10,003 prompts span counselor chat, forum posts, platform search, and triage, drawing on Singapore-grounded sources and public datasets. An expert-validated multi-label taxonomy distinguishes intent, communicative function, and response behavior. Six automated profiles, individually calibrated to three peer-support practitioners and three clinicians, retain separate reference policies for proxy-alignment scoring. Evaluating five systems on a coverage-enriched sample reveals variation in proxy alignment across profile-generated reference policies, even for the same system responses.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.26190
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.26190 in a model README.md to link it from this page.

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.26190 in a Space README.md to link it from this page.

Collections including this paper 1