Robustness Analysis on Natural Language Processing Based AI Q&A Robots

Yuan, Chengxiang and Xue, Mingfu and Zhang, Lingling and Wu, Heyi (2019) Robustness Analysis on Natural Language Processing Based AI Q&A Robots. In: Machine Learning and Intelligent Communications. 4th International Conference, MLICOM 2019, Nanjing, China, August 24–25, 2019, Proceedings.

[thumbnail of 34659.pdf] PDF
34659.pdf

Download (1MB)

Abstract

Recently, the natural language processing (NLP) based intelligent question and answering (Q&A) robots have been used in a wide range of applications, such as smart assistant, smart customer service, government business. However, the robustness and security issues of these NLP based artificial intelligence (AI) Q&A robots have not been studied yet. In this paper, we analyze the robustness problems in current Q&A robots, which include four aspects: (1) semantic slot settings are incomplete; (2) sensitive words are not filtered efficiently and completely; (3) Q&A robots return the search results directly; (4) unsatisfactory matching algorithms and inappropriate matching threshold settings. Then, we design and implement two types of evaluation tests, bad language and user’s typos, to evaluate the robustness of several state-of-the-art Q&A robots. Experiment results show that these common inputs (bad language and user’s typos) can successfully make these Q&A robots malfunction, denial of service, or speaking dirty words. Besides, we also propose possible countermeasures to enhance the robustness of these Q&A robots. To the best of the authors’ knowledge, this is the first work on analyzing the robustness and security problems of intelligent Q&A robots. This work can hopefully help provide guidelines to design robust and secure Q&A robots.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Date Deposited: 04 Mar 2026 12:35
Last Modified: 20 Nov 2026 11:12
URI: https://eprints.eai.eu/id/eprint/25318

Actions (login required)

View Item
View Item