Artificial Intelligence (AI) systems, including robotic assistants and voice interfaces, are increasingly present in public spaces such as airports, train stations, hospitals, and government offices, offering new opportunities to enhance user safety and comfort. Detecting aggression and hate speech in audio data from such locations is critical to protecting users and creating safe spaces. Previous approaches in this area have mainly relied on classification using classical Machine Learning (ML) methods and Deep Learning (DL) techniques, primarily focused on pure text analysis. However, most systems do not take into account real-world challenges such as background noise, incomplete utterances, or transcription errors resulting from automatic speech recognition. In this work, we propose a comprehensive approach that exploits the potential of Large Language Models (LLMs) to classify aggressive content in degraded audio transcriptions. We create a realistic test environment by converting tagged social media posts with offensive content into recordings with simulated noise and compare the performance of the models under different data quality conditions. Our research provides practical guidance on how to build more resilient and accessible voice moderation systems using LLM.