Evaluating IR Metrics for Top-N Recommendation

This page contain links to the source code and instructions to reproduce the experiments published in:

Daniel Valcarce, Alejandro Bellogín, Javier Parapar, Pablo Castells: On the Robustness and Discriminative Power of IR Metrics for Top-N Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, Vancouver, Canada, 2-7 October, 2018. DOI 10.1145/3240323.3240347.

Source Code

Check out this Github repo.

Slides

Available on SlideShare: slides link.

Poster

Available on SlideShare: poster link.

Data

We ran 21 recommender systems on three datasets (BeerAdvocate, LibraryThing and MovieLens 1M). The output of these recommenders was evaluated using rec_eval tool. We also measured statistically significant improvements using permutation test. The output of both tools can be found here.

Authors

Daniel Valcarce · Information Retrieval Lab · University of A Coruña
Alejandro Bellogín · Information Retrieval Group · Universidad Autónoma de Madrid
Javier Parapar · Information Retrieval Lab · University of A Coruña
Pablo Castells · Information Retrieval Group · Universidad Autónoma de Madrid