We’ve previously discussed “Google Announces MusicLM—Generating Music from Text Prompts” here on 2023-01-28. Now a group of researchers have announced “SingSong: Generating musical accompaniments from singing”. A vocalist can just sing, with the artificial intelligence supplying the music. Here is the abstract.
We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create music featuring their own voice. To accomplish this, we build on recent developments in musical source separation and audio generation. Specifically, we apply a state-of-the-art source separation algorithm to a large corpus of music audio to produce aligned pairs of vocals and instrumental sources. Then, we adapt AudioLM (Borsos et al., 2022) – a state-of-the-art approach for unconditional audio generation – to be suitable for conditional “audio-to-audio” generation tasks, and train it on the source-separated (vocal, instrumental) pairs. In a pairwise comparison with the same vocal inputs, listeners expressed a significant preference for instrumentals generated by SingSong compared to those from a strong retrieval baseline.
Sound examples at this https URL.
Many examples are at the linked page. In particular, try the 30 second clips for examples how the tool might be used. Unfortunately, I am presently managing the site from a tablet, which does not allow uploading audio clips, so you’ll have to go to the authors’ site to listen to them.