Music is a fundamental part of the human experience. It’s a universal language through which we channel emotions, tell stories, and connect with others. Whether through the simple act of finger-tapping, humming a tune, or playing a symphony, musical expression is intrinsic to who we are. Yet, the journey from artistic intent to musical realization can be challenging, even for experienced musicians.
This is where music technology becomes essential. However, while current systems offer remarkable capabilities, they are often complex and require significant technical expertise, creating barriers for many who wish to express themselves musically. My research explores artificial intelligence that can help artists explore and express their creativity more easily, bridging the gap between creative intent and its sonic realization. Specifically, I study deep generative models for intuitive musical audio generation and manipulation. These tools simplify the intricate processes of music production and enrich human-machine collaboration, empowering artists and novices to express their creativity more freely.
Selected Publications
J Nistal, M Pasini, C Aouameur, M Grachten, S Lattner. “Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models” Proceedings of the International Society for Music Information Retrieval (ISMIR) 2024
J Nistal, S Lattner, G Richard. “DrumGAN: Synthesis of drum sounds with timbral feature conditioning using Generative Adversarial Networks.” International Society for Music Information Retrieval (ISMIR)
J Nistal, S Lattner, G Richard. “Comparing representations for audio synthesis using generative adversarial networks.” 2020 28th European Signal Processing Conference (EUSIPCO), 161-165
J. Nistal, C. Aouameur, I Velarde, S Lattner . “DrumGAN VST: A Plugin for Drum Sound Analysis/Synthesis with Autoencoding Generative Adversarial Networks”. ICML2022 Machine Learning for Audio Synthesis (MLAS) Workshop
J Nistal, M Pasini, S Lattner. “Improving Musical Accompaniment Co-creation via Diffusion Transformers” Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation