A Semi-Automatic Light-Weight Approach Towards Data Generation for a Domain-Specific FAQ Chatbot Using Human-in-the-Loop

Anum Afzal, Tao Xiang, Florian Matthes

Research output: Contribution to journalConference articlepeer-review

Abstract

Employees at large companies tend to have longer waiting times if they need company-specific information and similarly someone on the other end needs to manually address those queries. Most companies are trying to incorporate LLM-powered conversational agents to make this processing faster but often struggle to find appropriate training data, especially domain-specific data. This paper introduces a semi-automatic approach for generating domain-specific training data while leveraging a domain-expert as a human-in-the-loop for quality control. We test this approach on a HR use-case of a large organization through a retrieval-based question-answering pipeline. Additionally, we also test the effect of long context on the performance of the FAQ chat for which we employ LongT5, an Efficient Transformer. Our experiments using LongT5 show that the inclusion of the generated training data improves the performance of the FAQ chatbot during inference.

Original languageEnglish
Pages (from-to)42-49
Number of pages8
JournalInternational Conference on Agents and Artificial Intelligence
Volume3
DOIs
StatePublished - 2024
Event16th International Conference on Agents and Artificial Intelligence, ICAART 2024 - Rome, Italy
Duration: 24 Feb 202426 Feb 2024

Keywords

  • Efficient Transformer
  • FAQ Chatbot
  • NLG Evaluation
  • Natural Language Generation
  • Training Data

Fingerprint

Dive into the research topics of 'A Semi-Automatic Light-Weight Approach Towards Data Generation for a Domain-Specific FAQ Chatbot Using Human-in-the-Loop'. Together they form a unique fingerprint.

Cite this