Balint Laczko

• Laczko, Balint; Rognes, Marie Elisabeth & Jensenius, Alexander Refsum (2025). Image Sonification as Unsupervised Domain Transfer. I McArthur, Angela; Matthews, Emma-Kate & Holberton, Tom (Red.), Proceedings of the 17th International Symposium on Computer Music Multidisciplinary Research. The Laboratory PRISM ¡°Perception, Representations, Image, Sound, Music¡±. ISSN 9791097498061. s. 596¨C607. doi: 10.5281/zenodo.17497987. Vis sammendrag

  The process of image sonification maps visual features into perceived auditory features. Most established sonification methods rely on identifying salient visual features in the input data and then mapping their distribution to a proportional distribution of auditory features. However, this approach requires both domain expertise and manual feature engineering. Here, we propose a new method of image sonification, leveraging recent advances in representation learning and domain transfer. Our approach introduces a pair of variational auto-encoder models that learn disentangled latent representations of the images and sounds, respectively, and a separate network that maps between these representations. The resulting sonification system encodes images into the latent space and then decodes them as sounds. Both representations and their mapping are learned in an entirely unsupervised manner. When evaluating the system in an interactive real-time setting, we observed that the model successfully learned disentangled representations of image and sound factors in our synthetic datasets.

• Laczko, Balint & Jensenius, Alexander Refsum (2025). Pixasonics: An Image Sonification Toolbox for Python. I Ziemer, Tim (Red.), Proceedings of the 30th International Conference on Auditory Display (ICAD 2025). The International Community for Auditory Display. Vis sammendrag

  Pixasonics is a new Python library for interactive image analysis and exploration through image sonification. It uses real-time audio and visualization to help uncover patterns in image data. With Pixasonics, users can launch one or more small web applications (running in a Jupyter Notebook), probe image data using various feature extraction methods, and map those feature vectors to synthesis parameters. The target users are researchers interested in exploring image and volumetric data and creative users who want an intuitive tool for experimental sound design. Pixasonics¡¯ design aims to strike a balance between an easy-to-use web application with minimal boilerplate code necessary and a library that can be integrated into more advanced workflows. Real-time exploration is at the heart, but it can also be used to script non-real-time sonifications of large datasets. This paper presents Pixasonics, its structure, interface, and advanced features, and discusses preliminary feedback from biology researchers and music technologists.

• Guo, Jinyue; Christodoulou, Anna-Maria; Laczko, Balint & Glette, Kyrre (2024). LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search. I Li, Xiaodong & Handl, Julia (Red.), GECCO '24 Companion: Proceedings of the Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 9798400704956. s. 667¨C670. doi: 10.1145/3638530.3654432. Vis sammendrag

  Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians.

• Laczko, Balint & Jensenius, Alexander Refsum (2024). Synth Maps: Mapping The Non-Proportional Relationships Between Synthesizer Parameters and Synthesized Sound. I Ziemer, Tim; Kantan, Prithvi Ravi; Chabot, Samuel & Braasch, Jonas (Red.), Proceedings of the 29th International Conference on Auditory Display (ICAD 2024). The International Community for Auditory Display. ISSN 9798991456203. s. 181¨C184. doi: 10.5281/zenodo.11237788. Fulltekst i vitenarkiv Vis sammendrag

  Parameter Mapping (PM) is probably the most used design approach in sonification. However, the relationship between a synthesizer¡¯s input parameters and the perceptual distribution of its output sounds might not be proportional, limiting its ability to convey relationships within the source data in the sound. This study evaluates a basic Frequency Modulation (FM) synthesis module with perceptually motivated descriptors, measures of spectral energy distribution, and latent embeddings of pre-trained audio representation models. We demonstrate how these metrics do not indicate straightforward relationships between synthesis parameters and perceived sound. This is done using interactive audiovisual scatter plots¡ªSynth Maps¡ªthat can be used to explore the sound distribution of the synthesizer and qualitatively evaluate how well the different representations align with human perception. Link to the code and the interactive Synth Maps are available.

• Laczko, Balint (2024). The Hum: 3D Audiovisual Live Performance for Projection and Spatial Audio. I Ziemer, Tim; Kantan, Prithvi Ravi; Chabot, Samuel & Braasch, Jonas (Red.), Proceedings of the 29th International Conference on Auditory Display (ICAD 2024). The International Community for Auditory Display. ISSN 9798991456203. s. 201¨C203. Fulltekst i vitenarkiv Vis sammendrag

  The Hum is a scalable audiovisual live performance instrument. It is a complete world full of sounding objects that interact with each other. It is based on a fictional story. The Hum currently contains around five thousand musical phrases. The sound dataset was then analysed with a bespoke set of descriptors, and the resulting representation was projected into 3D space using UMAP. The live performance involves exploring this 3D space both in terms of real-time visuals, and in a spatial audio setup. At its core the performance is the sonification of the various timbral and musical relationships of the sound dataset, expressed by various interactions triggered between neighboring sounds or using the whole dataset. The interactions may evoke associations of weather (blowing wind, rain or chain lightning). The structure of the performance follows a 3-stage plan, that is realized via improvisation. Thus the duration of the performance is flexible and can last between 15 to 30 minutes. The Hum is realized entirely in Max/MSP.

• Laczko, Balint & Jensenius, Alexander Refsum (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. I Kantan, Prithvi Ravi; Paisa, Razvan & Willemsen, Silvin (Red.), Proceedings of the Nordic Sound and Music Computing Conference. Aalborg Universitetsforlag. Fulltekst i vitenarkiv Vis sammendrag

  The paper presents the Musical Gestures Toolbox (MGT) for Python, a collection of modules targeted at researchers working with video recordings. The toolbox includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes. The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

Se alle arbeider i NVA

• Laczko, Balint; Rognes, Marie Elisabeth & Jensenius, Alexander Refsum (2025). Poster for "Image Sonification as Unsupervised Domain Transfer". doi: 10.5281/zenodo.17513165. Vis sammendrag

  The process of image sonification maps visual features into perceived auditory features. Most established sonification methods rely on identifying salient visual features in the input data and then mapping their distribution to a proportional distribution of auditory features. However, this approach requires both domain expertise and manual feature engineering. Here, we propose a new method of image sonification, leveraging recent advances in representation learning and domain transfer. Our approach introduces a pair of variational auto-encoder models that learn disentangled latent representations of the images and sounds, respectively, and a separate network that maps between these representations. The resulting sonification system encodes images into the latent space and then decodes them as sounds. Both representations and their mapping are learned in an entirely unsupervised manner. When evaluating the system in an interactive real-time setting, we observed that the model successfully learned disentangled representations of image and sound factors in our synthetic datasets.

• Laczko, Balint (2025). Presentation of PhD project: Perceptually Aligned Deep Image Sonification. doi: https:/comma.eecs.qmul.ac.uk/creative-audio-synthesis-and-interfaces-workshop/. Vis sammendrag

  Imaging technology has dramatically expanded our understanding of biological systems. This overabundance of images has come with unique problems, such as visual overload, which can potentially obscure data relationships and induce eye fatigue or divert vision from important tasks. Image sonification offers potential solutions to these problems by channeling data into the auditory domain, leveraging our natural pattern recognition skills through hearing. In my PhD project I have been exploring the potential of Machine Learning in solving the two fundamental challenges of image sonification: the perceptually aligned representations of images and sounds, and the cross-modal mapping between them. In this talk I will present my journey through timbre spaces, Differentiable DSP, and cross-modal domain transfer in search of new methods for image sonification.

• Laczko, Balint (2025). C2HO Workshop on Image Sonification with Pixasonics. doi: https:/www.hf.uio.no/imv/english/research/networks/creative-computing-hub-oslo/pages/c2ho-workshops/image-sonification-workshop.html. Vis sammendrag

  How can we turn images into sound? And what can we learn about those images by listening? Why listen instead of looking? These are some questions B¨¢lint Laczk¨®'s research on image sonification aims to find answers to. B¨¢lint has been researching applications of image sonification in biology and medical research and started working on a toolbox for Python, which he will be presenting during this workshop. The toolbox originates from bio-medical research, but it can also be used creatively for sound design. Bring your laptop with Python (or Anaconda) installed, and some images you'd like to squeeze some sounds out of!

• Laczko, Balint (2025). Insight Through Sound: Image Sonification in Biology Research. doi: https:/www.uio.no/forskning/satsinger/livsvitenskap/livsvitenskapskonferansen/2025/sidearrangementer/workshop/insight-through-sound-image-sonification-in-biolog.html.
• Laczko, Balint (2025). Oral presentation on ICAD 2025 about Pixasonics: An Image Sonification Toolbox for Python. Vis sammendrag

  Pixasonics is a new Python library for interactive image analysis and exploration through image sonification. It uses real-time audio and visualization to help uncover patterns in image data. With Pixasonics, users can launch one or more small web applications (running in a Jupyter Notebook), probe image data using various feature extraction methods, and map those feature vectors to synthesis parameters. The target users are researchers interested in exploring image and volumetric data and creative users who want an intuitive tool for experimental sound design. Pixasonics¡¯ design aims to strike a balance between an easy-to-use web application with minimal boilerplate code necessary and a library that can be integrated into more advanced workflows. Real-time exploration is at the heart, but it can also be used to script non-real-time sonifications of large datasets. This paper presents Pixasonics, its structure, interface, and advanced features, and discusses preliminary feedback from biology researchers and music technologists.

• Jensenius, Alexander Refsum & Laczko, Balint (2024). Video Visualization. Vis sammendrag

  This workshop is targeted at students and researchers working with video recordings. You will learn to use MG Toolbox, a Python package with numerous tools for visualizing and analyzing video recordings. This includes visualization techniques such as motion videos, motion history images, and motiongrams; techniques that, in different ways, allow for looking at video recordings from different temporal and spatial perspectives. It also includes some basic computer vision analysis, such as extracting quantity and centroid of motion, and using such features in analysis.MG Toolbox for Python is a collection of high-level modules for generating all of the above-mentioned visualizations and analyses. This toolbox was initially developed to analyze music-related body motion but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, psychology, medicine, and educational sciences.

• Laczko, Balint & Jensenius, Alexander Refsum (2024). Poster for "Synth Maps: Mapping The Non-Proportional Relationships Between Synthesizer Parameters and Synthesized Sound". Vis sammendrag

  Parameter Mapping (PM) is probably the most used design approach in sonification. However, the relationship between a synthesizer¡¯s input parameters and the perceptual distribution of its output sounds might not be proportional, limiting its ability to convey relationships within the source data in the sound. This study evaluates a basic Frequency Modulation (FM) synthesis module with perceptually motivated descriptors, measures of spectral energy distribution, and latent embeddings of pre-trained audio representation models. We demonstrate how these metrics do not indicate straightforward relationships between synthesis parameters and perceived sound. This is done using interactive audiovisual scatter plots¡ªSynth Maps¡ªthat can be used to explore the sound distribution of the synthesizer and qualitatively evaluate how well the different representations align with human perception. Link to the code and the interactive Synth Maps are available.

• Laczko, Balint (2024). Two-part guest lecture about spatial audio and Ambisonics for MCT students.
• Laczko, Balint (2023). Workshop on Spatial Audio and Ambisonics.
• Laczko, Balint (2023). Online guest lecture about The Hum - a real-time 3D audiovisual performance in Max.
• Laczko, Balint (2023). Two-part guest lecture about spatial audio and Ambisonics for MCT students.
• Laczko, Balint (2023). Guest lecture about granular synthesis with onset detection in Max.
• Outa, Amani al; Kn?velsrud, Helene; Laczko, Balint & Jensenius, Alexander Refsum (2022). Winner of RRI-inspired transdisciplinary side quest call. [Internet]. Centre for Digital Life Norway. Vis sammendrag

  Centre for Digital Life Norway (DLN) is excited to congratulate the team behind the project ¡°The autophagic symphony ¨C Unveiling the final rhythm¡± as winner of DLN¡¯s RRI-inspired transdisciplinary side quest call.

• Laczko, Balint & Jensenius, Alexander Refsum (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. Vis sammendrag

  The paper presents the Musical Gestures Toolbox (MGT) for Python, a collection of modules targeted at researchers working with video recordings. The toolbox includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes. The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

Se alle arbeider i NVA