AudioCraft

AudioCraft is a unified open-source code base from Meta AI for generative audio tasks, including music generation, sound effect generation, and audio compression. It simplifies the design of generative audio models by using a single autoregressive language model (LM) operating over streams of compressed discrete audio tokens. These tokens are learned via the EnCodec neural audio codec, which maps raw audio waveforms into one or more parallel streams of discrete tokens. The LM recursively models these tokens, and the generated tokens are decoded back into audio waveforms by the EnCodec decoder. Conditioning models, such as pretrained text encoders, can be used to control generation, enabling text-to-audio applications.

Key Features

• Single code base supporting multiple audio generation tasks.
• Efficient modeling of long-term dependencies in audio sequences.
• High-quality audio output from raw waveforms.

Use Cases

• Text-to-sound generation with AudioGen.
• Text-to-music generation with MusicGen.

Who It’s For

AI researchers and developers working on generative audio models.

Key Benefits

• Single code base for multiple audio tasks
• Simplified generative model design
• High-quality audio generation
• Efficient modeling of long-term dependencies