Volume 14, Number 5

Identifying Text Classification Failures in Multilingual AI-Generated Content

  Authors

Raghav Subramaniam, Independent Researcher

  Abstract

With the rising popularity of generative AI tools, the nature of apparent classification failures by AI content detection softwares, especially between different languages, must be further observed. This paper aims to do this through testing OpenAI’s “AI Text Classifier” on a set of human and AI-generated texts in English, German, Arabic, Hindi, Chinese, and Swahili. Given the unreliability of existing tools for detection of AI-generated text, it is notable that specific types of classification failures often persist in slightly different ways when various languages are observed: misclassification of human-written content as “AI-generated” and vice versa may occur more frequently in specific language content than others. Our findings indicate that false negative labelings are more likely to occur in English, whereas false positives are more likely to occur in Hindi and Arabic. There was an observed tendency for other languages to not be confidently labeled at all.

  Keywords

Artificial Intelligence, AI Detection, Generative AI, GPT, Natural Language Processing