Informace o publikaci
CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings
| Autoři | |
|---|---|
| Rok publikování | 2026 |
| Druh | Stať ve sborníku |
| Konference | Proceedings of the 35th ACM International Conference on Information and Knowledge Management (CIKM '26) |
| Fakulta / Pracoviště MU | |
| Citace | |
| www | Code repository containing the benchmark, probe training pipeline, and reports |
| Doi | https://doi.org/10.1145/3799682.3840044 |
| Klíčová slova | patient-facing AI; oncology; guardrails; linear probing; embeddings |
| Přiložené soubory | |
| Popis | Patient-facing AI assistants promise valuable support to patients, but incoming queries can pose medical risks. To create guardrails, we work with oncologists to define three ordinal risk axes: Medical Urgency, Psychological Urgency, and Topic Sensitivity. We propose Clinical Guardrail Probes (CG-Probes) to measure the risks from query embeddings. We probe for each axis in the normalized embedding space of frozen embedders via the difference-in-means method, treating each axis as a potential linear direction. To train the probes, we cluster 79,658 Czech oncology search queries with BERTopic and use these clusters to generate pairs of queries with contrastive risk levels via few-shot prompting. We evaluate the approach on 200 queries (90 real, 110 synthetic), each graded by two oncologists, against two open-weight LLMs and a frontier LLM. We find that urgency-based axes are recoverable as linear directions, and the probes are competitive with open-weight LLMs (no significant differences in quadratic-weighted ?) at a fraction of the latency. Each axis yields a scalar score that clinicians can inspect and use to set escalation thresholds. The pipeline requires only search logs, axis definitions, and black-box access to the embedding model, suggesting transferability across healthcare domains. Robust validation on new queries and axes remains future work. |
| Související projekty: |