Last year I had the opportunity to attend this great summer school in the beautiful and lovely city of Lisbon. It was a great week together with a lot of interesting and intelligent people, all of them interested in the amazing and exciting area of machine learning and NLP. I liked it so much last year that I decided to come back this year to volunteer as an assistant in the summer school. Today was the -1 day, where we organized some of the registration stuff, welcomed some student and had some beers. Looks like it will be, again, a great time here in Lisbon
On February I visited Cambridge to attend WSDM Doctoral Consortium. It happened during the first day of the conference, in parallel to some tutorials. It was a great time, we had excellent discussions about our projects with senior researchers and fellow Ph.D. candidates. Here is a photo for the posterity.
And the program: http://www.wsdm-conference.org/2017/doctoral-consortium/
Lately, I have been working with Poisson Matrix Factorization models and
at some point a needed to work a lower bound for . After seeing some people using this lower bound without a good explanation, I decided to write this blog post. Also, this is included as an appendix to my ECML-PKDD 2017 paper about poisson factorizatiom model for recommendation.
The function is a concave function, which means that:
By induction this property can be generalized to any convex combination of ( with ):
Now with the a random variable we can create a similar convex combination by multiplying and dividing each random variable by and apply the sum of of expectation property:
If we want a tight lower bound we should use Lagrange multipliers to choose the set of that maximize the lower-bound given that they should sum to 1.
The final formula for is exactly the same that we can find for the parameters of the the Multinomial distribution of the auxiliary variables in a Poisson model with rate parameter as sum of Gamma distributed latent variables. Also using this optimal we can show a tight bound without the auxiliary variables.
This results in:
We have a paper accepted at ECML-PKDD 2017: “Content-Based Social Recommendation with Poisson Matrix Factorization” (Eliezer de Souza da Silva, Helge Langseth and Heri Ramampiaro). This is our first full paper resulting from our research on Poisson factorization and integration of multiple sources of information in a single recommendation model. If you have interest on the paper please email me and I will be happy to discuss.
(in the same series HMM (part I): recurrence equations for filtering and prediction)
Consider a Hidden Markov Model (HMM) with hidden states (for ), initial probability , observed states , transition probability and observation model . This model can be factorized as
. We will use the notation to represent the set .
In this post we will present the details of the method to find the smoothing distribution of a HMM, given a set of observations :
Our starting point is the marginal probability of given all the observations .
There is an algorithm to generate Dirichlet samples using a sampler for Gamma distribution for any and . We will generate Gamma distributed variables , for , and do the following variable transformation to get Dirichlet samples . First we should demonstrate that this transformation results in Dirichlet distributed samples.
Consider the following tranformation , where and . We can rewrite this transformation as , where and . Also we can imediatly calculate the inverse transformation , with . From the transformation definition we know that , implying that and .
In Recommender Systems design we are faced with the following problem: given incomplete information about users preference, content information, user-items rating and contextual information, learn the user preference and suggest new items for users based on features as:
- previous pattern of items preference of the user;
- preference of users with similar rating pattern;
- contextual information: location, time (of the day, week, month, year), social network.
This is usually formulated as a matrix completion problem using Matrix Factorization techniques to offer a optimal solution. Is this case latent features for users and item are inferred from the observed/rated items for each user, and from this latent features the missing entries are estimated. One major modelling tool for this problem is probabilistic modelling, and there are many proposals in the literature of different Probabillistic Matrix Factorization approaches. We will briefly discuss some of this models, starting with the seminal paper: Probabilistic Matrix Factorization (PMF) – [Salakhutdinov and Mnih, 2008, NIPS]. Continue reading “Probabilistic models for Recommender systems (part I): Probabilistic Matrix Factorization”
This semester I will be attending the doctoral course MA8702 – Advanced Modern Statistical Methods with the excellent Prof. Håvard Rue. It will be course about statistical models defined over sparse structures (chains and graphs). We will start with Hidden Markov Chains and after go to Gaussian Markov Random Fields, Latent Gaussian Models and approximate inference with Integrated Nested Laplace Approximation (INLA). All this models are interesting for my research objective of developing sound latent models for recommender systems and I am really happy of taking this course with this great teacher and researcher. So, I will try to cover some of the material of the course, starting from what we saw in the first lecture: exact recurrence for Hidden Markov Chains and dynamic programming. In other words, general equations for predictions, filtering, smoothing, sampling, mode and marginal likelihood calculation of state-space model with latent variables. We will start by introduction the general model and specifying how to obtain the prediction and filtering equation.
- Markovian property: , with
- are observed and are latent, so is always known.
- If we know than no other variable will add any information to the conditional distribution of .
So, some of you may have heard about the free e-books frenzy at Springer (unfortunately, not working any more ). I jumped on the train of people trying to find the goodies on the website and ended up developing some tools to help me with that. Even after the freebies ended I kept on improving this tool because it is useful also in my day-to-day job of searching for bibliography and storing those PDF’s. So this script can basically navigate through a search result (passed to the script as a link to the search result) and download all books available (even if it consist of many pages results). This is also useful for downloading all books from a series (for example, springer series in statistics). In the future I intend to update the script including download of chapters of books and research articles. This script is specially useful if your university have access to Springer Books. (full code in https://gist.github.com/zehsilva/d890837cdfd4a20ee8a5#file-download-books-from-rss-springer-py) Continue reading “Script for searching and web crawling Springer Books”
IPython NBViewer: http://nbviewer.ipython.org/gist/zehsilva/18c28992796d9be99e16
A couple of days ago I was thinking about the possibility of generating GIF animations using matplotlib and visualizing this animations directly in IPython notebooks. So I did a quick search to find possible solutions, and found one solution to embed videos, and other to convert matplotlib animations to gif, so I combined both solution in a third solution converting the animation to GIF using imagemagick (you will need to have it installed in your computer), enconding the resulting file in a base64 string and embedding this in a img tag. Using IPython.display HTML class this resulting string will be displayed in the notebook (and saved with the notebook).
I will briefly explain the elements of my solution.