Abstract
Image Sonification as Unsupervised Cross-Modal Domain Transfer
Parameter Mapping (PM) is the most used design pattern in sonification. However, the relationship between a synth's input parameters and the perceptual distribution of its output sounds might not be proportional, limiting PM's ability to transfer relationships from the source data domain to sound. PM also requires prior knowledge of the features and parameters involved in the sonification mapping. This can result in sonification applications that are highly specific and do not generalize well.
Recent advancements in representation learning offer new ways to capture high-level features of data that are otherwise impossible to find in imperative programming. A new area of representation learning focuses on establishing mappings between the learned distributions of latent variable models, allowing to transfer relationships between domains in an unsupervised manner.
This study explores how we can apply representation learning and unsupervised cross-modal domain transfer in image sonification, and overcome the limitations of PM.
Motion capture of cells in 3D
How does mammalian life evolve, from a fertilized egg to a fully developed organism? This intriguing question has baffled us for centuries, and despite intensive research efforts over the years, it is still far from understood. Especially, we lack a good description of the critical gastrulation stage, where a ball of cells starts to elongate because (a) cells divide preferentially in one direction, (b) because different types of cells divide at different rates, or (c) because cells coordinate their motion and migrate collectively. We set out to quantify the relative importance of these complementary mechanisms and how they shape early embryos. To establish a controlled experimental protocol, we use stem cell derived embryo models (so-called gastruloids), and we image them in 3D using our home-built light-sheet microscope. We couple our experiments with numerical simulations to understand how the cells orchestrate their motion.
The project is funded by the convergence environment ITOM under UiO:LifeScience.
Bio
Balint Laczko is a researcher, music technologist and composer. His technological background includes audiovisual analysis, computer vision, and interactive systems development. His artistic background includes classical music, electronic music with emphasis on 3D audio and data-driven audiovisual art.
balintlaczko.com
Joachim is an experimental fluid mechanician dedicated to everything that flows. He received his Ph.D. in microfluidics from the University of Oslo and was a postdoc at Stanford University and at UCSB to learn about interfacial flows and miscible fluids. He is now back at UiO, where he studies how tissue flows in developing embryos shape early mammalian development. Finally, Joachim is devoted to teaching and outreach and uses “kitchen flows” to communicate with a wider audience.