Co-Refine: AI-Powered Tool Supporting Qualitative Analysis
Athikash Jeyaganthan, Kai Xu, Franziska Becker, Steffen Koch
TLDR
Co-Refine is an AI tool that provides real-time feedback on coding consistency in qualitative analysis, preventing temporal drift.
Key contributions
- Introduces Co-Refine, an AI-augmented platform for continuous, grounded feedback on qualitative coding consistency.
- Detects 'temporal drift' in code interpretations without disrupting the researcher's workflow.
- Utilizes a three-stage audit pipeline combining deterministic embedding metrics and LLM-grounded verdicts.
- Generates code definitions from past patterns, creating a deepening feedback loop for improved analysis.
Why it matters
Qualitative analysis often struggles with inconsistent code interpretations over time, impacting research credibility. Co-Refine addresses this by providing real-time, AI-powered consistency checks. This ensures more reliable and rigorous qualitative research, making it a valuable tool for social scientists.
Original Abstract
Qualitative coding relies on a researcher's application of codes to textual data. As coding proceeds across large datasets, interpretations of codes often shift (temporal drift), reducing the credibility of the analysis. Existing Computer-Assisted Qualitative Data Analysis (CAQDAS) tools provide support for data management but offer no workflow for real-time detection of these drifts. We present Co-Refine, an AI-augmented qualitative coding platform that delivers continuous, grounded feedback on coding consistency without disrupting the researcher's workflow. The system employs a three-stage audit pipeline: Stage 1 computes deterministic embedding-based metrics for mathematical consistency; Stage 2 grounds LLM verdicts within $\pm0.15$ of the deterministic scores; and Stage 3 produces code definitions from previous patterns to create a deepening feedback loop. Co-Refine demonstrates that deterministic scoring can effectively constrain LLM outputs to produce reliable, real-time audit signals for qualitative analysis.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.