FireRedChat – A Full-Duplex Voice Interaction System Launched by Xiaohongshu

What is FireRedChat？

FireRedChat is a full-duplex voice interaction system developed by the Xiaohongshu Intelligent Audio Team. It enables real-time bidirectional communication with controllable interruption capability. Built on a modular architecture, the system includes components such as a transcription control module, interaction module, and dialogue manager, supporting both cascade and semi-cascade architectures for flexible deployment. FireRedChat leverages the LiveKit RTC Server for real-time communication, combined with an AI-Agent Bot Server that handles intelligent agent responses, and provides a WebUI interface for user interaction. It is equipped with a Redis Server for multi-node management, as well as TTS and ASR Servers for text-to-speech synthesis and automatic speech recognition, respectively.

Key Features of FireRedChat

Full-Duplex Voice Interaction:
Enables real-time, two-way communication between users and AI agents, allowing both parties to speak simultaneously with controllable interruption, significantly improving conversational fluidity.

Privacy Protection and On-Premises Deployment:
Supports complete self-hosting without reliance on external APIs, ensuring data security and allowing users full control over their deployment environment.

Modular Design:
Composed of multiple modules—such as transcription control, interaction module, and dialogue manager—supporting flexible cascade and semi-cascade configurations for easy customization and scalability.

Low-Latency Communication:
Achieves near-industrial-grade low latency through LiveKit RTC Server, ensuring smooth, real-time interactions.

Voice Activity Detection and Semantic Analysis:
Implements streaming personalized Voice Activity Detection (pVAD) and End-of-Turn (EoT) semantic detection to suppress background noise, accurately segment the main speaker’s voice, and improve interruption success rate and conversational naturalness.

Technical Principles of FireRedChat

Real-Time Communication:
Uses LiveKit RTC Server as the core infrastructure to enable low-latency, real-time audio/video communication and multi-user interaction.

Intelligent Agent Response:
Employs an AI-Agent Bot Server to process user input and generate intelligent, natural voice responses using natural language processing technologies.

Speech Recognition and Synthesis:
Integrates an ASR Server for automatic speech recognition (converting user speech into text) and a TTS Server for speech synthesis (converting AI responses into natural speech).

Voice Activity Detection:
Utilizes streaming personalized Voice Activity Detection (pVAD) to accurately identify the main speaker’s voice while suppressing background noise and irrelevant speech.

Semantic End-of-Turn Detection:
Uses semantic analysis to determine when a user has finished speaking, reducing misjudgments caused by pauses and improving conversational flow.

Modular Architecture:
Built from multiple independent modules that collaborate seamlessly, supporting flexible cascade and semi-cascade deployments for easy scalability and maintenance.

Data Persistence and Multi-Node Hosting:
Employs a Redis Server for data persistence across instances and supports multi-node hosting, ensuring high availability and stability.

Project Resources

GitHub Repository: https://github.com/FireRedTeam/FireRedChat
arXiv Paper: https://arxiv.org/pdf/2509.06502
Online Demo: https://fireredteam.github.io/demos/firered_chat

Application Scenarios of FireRedChat

Intelligent Customer Service:
Provides real-time voice support to users, enabling rapid responses to customer inquiries and improving service efficiency and satisfaction.

Virtual Assistants:
Serves as the core of voice interaction in smart home and smart office scenarios, enabling device control and information retrieval.

Education:
Facilitates real-time voice interaction in online learning environments, enhancing the teaching and learning experience.

Finance:
Used in financial consulting and trading assistance scenarios to offer secure and efficient voice-based services.

Healthcare:
Supports medical consultation and remote diagnosis, improving accessibility and efficiency in healthcare services through natural voice interaction.

Public Services:
Applied in government hotlines and civic service systems to provide intelligent voice services and enhance administrative efficiency.