Our paper “Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching” (with Tomasz Kusmierczyk, Marcelo Hartmann, and Arto Klami) has been accepted in the Journal of Machine Learning Research (JMLR). The paper can be found at this link https://jmlr.org/papers/v24/21-0623.html
– Preprint: https://lnkd.in/dwf6WUXe
– Code: https://lnkd.in/d5TDyPz4
In this work, we develop a theoretical and algorithmic methodology based on the prior predictive distribution that can be helpful in the specification and choice of prior parameters **before** training the model or doing posterior inference, focused on the matrix factorization case. This prior predictive analysis of matrix factorization allows us to obtain closed-form equations for the dimension of the latent space, as well as relations with the mean and variance of the prior. It works for different choices of prior and likelihoods. Having a closed-form equation for the number of dimensions in such a case has been an open problem, and now we have a solution.
Furthermore, we developed a gradient-based optimization procedure that seeks to find prior parameters to match “virtual statistics” (moments induced by sampling and marginalizing the latent variable from the generative model before fitting any data) with fixed values.
This is the last chapter of my thesis, developed during the research visit to the University of Helsinki, hosted by Arto Klami’s Multi-Source Probabilistic Inference group. I am very thankful to my supervisors Helge Langseth and Heri Ramampiaro for their support in developing my research, to António Góis for the helpful discussions, and to Alina Karakanta for the detailed feedback on the paper.
Eliezer Silva 