An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Last update: Dec 02, 2022

Related tags

Deep Learning recsys_metrics

Overview

recsys_metrics

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Overview • Installation • How to use • Benchmark • Citation •

Overview

Highlights

Efficient (vectorized) implementations over mini-batches
Standard RecSys metrics: precision, recall, map, mrr, hr, ndcg
Beyond-accuracy metrics: e.g. coverage, diversity, novelty, etc.
All metrics support a top-k argument.

Why do we need `recsys_metrics`?

Installation

You can install recsys_metrics from PyPI:

pip install recsys_metrics

Or you can also install the latest version from source:

pip install git+https://github.com/zuoxingdong/[email protected]

Note that we support Python 3.7+ only.

How to use

Let us take Hit Rate (HR) to illustrate how to use this library:

preds = torch.tensor([
    [.5, .3, .1],
    [.3, .4, .5]
])
target = torch.tensor([
    [0, 0, 1],
    [0, 1, 1]
])
hit_rate(preds, target, k=1, reduction='mean')

>> tensor(0.5000)

The first example in the batch does not have a hit (i.e. top-1 item is not a relevant item) and second example in the batch gets a hit (i.e. top-1 item is a relevant item). Thus, we have a hit-rate of 0.5.

The API of other metrics are of the same format.

Benchmark

Metrics	Single Example	Mini-Batch
Precision
Recall
MAP
MRR
HR
NDCG

Citation

This work is inspired by Torchmetrics from PyTorchLightning Team.

Please use this bibtex if you want to cite this repository in your publications:

@misc{recsys_metrics,
      author = {Zuo, Xingdong},
      title = {recsys_metrics: An efficient PyTorch implementation of the evaluation metrics in recommender systems.},
      year = {2021},
      publisher = {GitHub},
      journal = {GitHub repository},
      howpublished = {\url{https://github.com/zuoxingdong/recsys_metrics}},
    }

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

NVIDIA Merlin NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs. It enables data scientists, machine

419 Jan 3, 2023

This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

71 Dec 30, 2022

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking Datasets You can download datasets that have been pre-pr

25 May 29, 2022

Active and Sample-Efficient Model Evaluation

Active Testing: Sample-Efficient Model Evaluation Hi, good to see you here! 👋 This is code for "Active Testing: Sample-Efficient Model Evaluation". P

19 Oct 30, 2022

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

引言感谢苏神带来的模型，原文地址：https://spaces.ac.cn/archives/8877 如何运行对应模型EfficientGlobalPoi

40 Dec 14, 2022

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

[ICLR 2021] "UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems" by Jiayi Shen, Haotao Wang*, Shupeng Gui*, Jianchao Tan, Zhangyang Wang, and Ji Liu

39 Dec 3, 2022

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering r

1.2k Dec 21, 2022

A python library for implementing a recommender system

python-recsys A python library for implementing a recommender system. Installation Dependencies python-recsys is build on top of Divisi2, with csc-pys

1.5k Dec 17, 2022

A TikTok-like recommender system for GitHub repositories based on Gorse

GitRec GitRec is the missing recommender system for GitHub repositories based on Gorse. Architecture The trending crawler crawls trending repositories

337 Jan 4, 2023

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Related tags

Overview

Overview

Highlights

Why do we need recsys_metrics?

Installation

How to use

Benchmark

Citation

You might also like...

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

Active and Sample-Efficient Model Evaluation

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

A python library for implementing a recommender system

A TikTok-like recommender system for GitHub repositories based on Gorse

Releases(v0.0.3)

v0.0.3(Aug 1, 2022)

v0.0.2(Jun 14, 2022)

Owner

Xingdong Zuo

Implementation based on Paper - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Fast RFC3339 compliant Python date-time library

A curated list and survey of awesome Vision Transformers.

Annotate with anyone, anywhere.

RINDNet: Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth, in ICCV 2021 (oral)

TVNet: Temporal Voting Network for Action Localization

The Official PyTorch Implementation of DiscoBox.

A library for implementing Decentralized Graph Neural Network algorithms.

TSIT: A Simple and Versatile Framework for Image-to-Image Translation

MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images

Implementation of "Efficient Regional Memory Network for Video Object Segmentation" (Xie et al., CVPR 2021).

The official codes of "Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners".

To provide 100 JAX exercises over different sections structured as a course or tutorials to teach and learn for beginners, intermediates as well as experts

VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning

Minecraft Hack Detection With Python

Simple tool to combine(merge) onnx models. Simple Network Combine Tool for ONNX.

Extremely simple and fast extreme multi-class and multi-label classifiers.

Implementation of the final project of the course DDA6309 Probabilistic Graphical Model

Simple helper library to convert a collection of numpy data to tfrecord, and build a tensorflow dataset from the tfrecord.

Deep GPs built on top of TensorFlow/Keras and GPflow

Why do we need `recsys_metrics`?