Hybrid Decision Making via Conformal VLM-generated Guidance
Debodeep Banerjee, Burcu Sayin, Stefano Teso, Andrea Passerini
TLDR
ConfGuide is a new AI system that provides succinct, targeted guidance for human decision-making, improving clarity in complex tasks like medical diagnosis.
Key contributions
- Introduces ConfGuide, a novel Learning to Guide (LtG) approach for hybrid decision making.
- Generates succinct and targeted AI guidance, addressing the issue of information overload.
- Employs conformal risk control to select outcomes, ensuring a capped false negative rate.
- Demonstrated effectiveness on a real-world multi-label medical diagnosis task.
Why it matters
ConfGuide improves hybrid decision-making by providing succinct, targeted AI guidance, reducing cognitive load for human users. This addresses the critical issue of information overload in AI-assisted tasks, making complex decisions, like medical diagnoses, more manageable and accurate.
Original Abstract
Building on recent advances in AI, hybrid decision making (HDM) holds the promise of improving human decision quality and reducing cognitive load. We work in the context of learning to guide (LtG), a recently proposed HDM framework in which the human is always responsible for the final decision: rather than suggesting decisions, in LtG the AI supplies (textual) guidance useful for facilitating decision making. One limiting factor of existing approaches is that their guidance compounds information about all possible outcomes, and as a result it can be difficult to digest. We address this issue by introducing ConfGuide, a novel LtG approach that generates more succinct and targeted guidance. To this end, it employs conformal risk control to select a set of outcomes, ensuring a cap on the false negative rate. We demonstrate our approach on a real-world multi-label medical diagnosis task. Our empirical evaluation highlights the promise of ConfGuide.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.