MGTEVAL: An Interactive Platform for Systemtic Evaluation of Machine-Generated Text Detectors
Yuanfan Li, Qi Zhou, Chengzhengxu Li, Zhaohan Zhang, Chenxu Zhao + 3 more
TLDR
MGTEVAL is an interactive platform designed for the systematic and reproducible evaluation of machine-generated text detectors.
Key contributions
- Provides MGTEVAL, an extensible platform for systematic and reproducible evaluation of MGT detectors.
- Organizes evaluation into Dataset Building, Attack, Detector Training, and Performance Evaluation.
- Supports custom MGT generation, 12 text attacks, and unified detector training interfaces.
- Features both command-line and Web interfaces for user-friendly, code-free experimentation.
Why it matters
Existing evaluations of machine-generated text detectors are fragmented, making comparisons and reproductions difficult. MGTEVAL addresses this by offering a unified, systematic platform. This platform standardizes the evaluation process, enabling researchers to build custom benchmarks, test robustness against various attacks, and compare detectors effectively.
Original Abstract
We present MGTEVAL, an extensible platform for systematic evaluation of Machine-Generated Text (MGT) detectors. Despite rapid progress in MGT detection, existing evaluations are often fragmented across datasets, preprocessing, attacks, and metrics, making results hard to compare and reproduce. MGTEVAL organizes the workflow into four components: Dataset Building, Dataset Attack, Detector Training, and Performance Evaluation. It supports constructing custom benchmarks by generating MGT with configurable LLMs, applying 12 text attacks to test sets, training detectors via a unified interface, and reporting effectiveness, robustness, and efficiency. The platform provides both command-line and Web-based interfaces for user-friendly experimentation without code rewriting.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.