Riffusion

Riffusion is a free and open-source real-time music and audio generation library with stable diffusion. Users only need to input a music description, and the AI can generate music in the corresponding style. This open-source project was launched by Seth Forsgren and Hayk Martiros. The underlying principle of Riffusion is to fine-tune spectrograms based on Stable Diffusion (a text-to-image generation model). A spectrogram is a visual representation of audio that shows the amplitude of different frequencies over time. Then, high-fidelity spectrogram images are converted into audio.