Is Disentanglement enough? On Latent Representations for Controllable Music Generation

Plenty of automatic music generation models have been created recently; however, in most cases, the end-user has little to no control over the generation process. The field of representation learning may be promising for enabling users to manipulate one or more attributes (for instance, rhythm or scale) of the generated data.

Image credit: Tomgally via Wikimedia

A recent study looks at supervised disentangled representation learning methods, which have not been yet systematically evaluated. In the disentangled representations, individual factors of variation are separated. The changes to a single factor in the data lead to changes in a single factor of the representation.

Several supervised methods are compared in their controllability. The results show that supervised learning methods can achieve a high degree of disentanglement. Nevertheless, the degree of controllability depends both on the chosen method and musical attribute to be controlled.

Improving controllability or the ability to manipulate one or more attributes of the generated data has become a topic of interest in the context of deep generative models of music. Recent attempts in this direction have relied on learning disentangled representations from data such that the underlying factors of variation are well separated. In this paper, we focus on the relationship between disentanglement and controllability by conducting a systematic study using different supervised disentanglement learning algorithms based on the Variational Auto-Encoder (VAE) architecture. Our experiments show that a high degree of disentanglement can be achieved by using different forms of supervision to train a strong discriminative encoder. However, in the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes. To this end, we also propose methods and metrics to help evaluate the quality of a latent space with respect to the afforded degree of controllability.

Research paper: Pati, A. and Lerch, A., “Is Disentanglement enough? On Latent Representations for Controllable Music Generation”, 2021. Link:

More to track …
Bulletin Observer Science and Technology

[Advertisement Plan]

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest