Customizing Context: Discovering the Optimal Integration of Context Data to Elevate ML-Driven Automated Support Ticket Classification

Context-aware Machine Learning (ML) is a well-established subfield of ML research. Here, the idea is to consider the context of an ML problem to enhance model performance. The decisive factor of context-aware ML is the precise implementation of representative context data. Still, the field of ML-driven support ticket classification has yet only scratched the potential of context awareness. Traditional text-only classification often lacks accuracy, posing a challenge to the deployment of ML-automated support ticket classifiers. To address this, we explore four approaches in this paper, integrating structured contextual data with textual descriptions. We found that the correct incorporation of context data significantly enhances accuracy. We evaluate the effect of text embedding on model performance, highlighting the need for thoughtful data integration strategies. We demonstrate the effectiveness of context-aware approaches and explore alternative text preprocessing techniques. In closing, we discuss our findings - especially the impact of data handling, data imbalance, and interpretability on the overall automation project.