Warsaw scientist investigates applications of AI and deep neural networks to create and analyse music

What if instruments could learn to play on their own? Would music composed this way differ from the compositions written by humans? Can we teach machines to understand music created by man? Dr. Mateusz Modrzejewski from the Division of Computer Graphics at the Faculty of Electronics and Information Technology of the Warsaw University of Technology is looking for answers to these music information retrieval questions.

'In my research I combine my two greatest passions: music and issues relating to deep neural networks, which form the basis of machine learning and the use of artificial intelligence', explains Dr. Mateusz Modrzejewski, quoted in the Warsaw University of Technology press release. 'As a jazz drummer with vast performance experience, I know and understand the structure of music. As an engineer, I write it down not only as musical notes, but I also present it in a computer language. This is an excellent entry data collection for machine processing, analysis and composing music by AI algorithms'.

According to the release, Dr. Modrzejewski’s research focuses around the use of deep neural networks, including GAN-type recurrent and generative networks, as well as state-of-the-art representations based on the transfer learning approach.

'New methods are prepared with the use of large data sets in both audio and symbolic formats, such as MIDI (Musical Instrument Digital Interface). Once individual pieces are processed using AI algorithms, it is possible to generate original music that meets specific objectives and to accurately and precisely classify existing music according to given criteria, such as the identification of the singer, mood and instruments, or the recognition of a musical genre. The proprietary solutions were presented at the Artificial Intelligence and Soft Computing conference', the Warsaw University of Technology reports.

'Music is highly abstract communication, deeply rooted in human intelligence and sensitivity. In the case of generating music, we emphasise the compositional aspect - we code musical notes as expressive textural representations or in graphic form known as piano roll. Artificial intelligence becomes the ally of a human composer, providing new melodies, structures and solutions. We support human intelligence and creativity through AI techniques', Dr Modrzejewski points out

'In the case of analysis, we use the techniques of digital signal processing and we investigate representations learned by deep neural networks. These representations allow machines to organically understand the content contained in music - this also includes recommending new music to listeners, taking into account their personalised preferences', the researcher adds.

Dr. Modrzejewski develops his original concepts under the supervision of Professor Przemysław Rokita at the Division of Computer Graphics, Faculty of Electronics and Information Technology, Warsaw University of Technology. According to the release, he is also a drummer who has performed at many festivals and musical events in Poland, as well as in China, Vietnam, Germany, Scotland, England, Estonia, Ukraine. In 2020 he was awarded the Kultura w Sieci grant by the Polish Minister of Culture and National Heritage. Last year, as a member of Majka Jeżowska’s band, he was nominated for the Fryderyk award for the “Live at Pol’and’ rock” album.

PAP - Science in Poland

agt/ kap/

tr. RL