Negative-Sampling-Paper

This repository collects 100 papers related to negative sampling methods, covering multiple research fields such as Recommendation Systems (RS), Computer Vision (CV)，Natural Language Processing (NLP) and Contrastive Learning (CL).

Existing negative sampling methods can be roughly divided into five categories: Static Negative Sampling, Hard Negative Sampling, Adversarial Sampling, Graph-based Sampling and Additional data enhanced Sampling.

• Category

  • Static Negative Sampling
  • Hard Negative Sampling
  • Adversarial Sampling
  • Graph-based Sampling
  • Additional data enhanced Sampling

• Future Outlook

  • False Negative Problem
  • Curriculum Learning
  • Negative Sampling Ratio
  • Debiased Sampling
  • Non-Sampling

Category

Static Negative Sampling

• BPR: Bayesian Personalized Ranking from Implicit Feedback. UAI(2009) [RS] [PDF]

• Real-Time Top-N Recommendation in Social Streams. RecSys(2012) [RS] [PDF]

• Distributed Representations of Words and Phrases and their Compositionality. NIPS(2013) [NLP] [PDF]

• word2vec Explained: Deriving Mikolov et al.'s Negative-Sampling Word-Embedding Method. arXiv(2014) [NLP] [PDF]

• Deepwalk: Online learning of social representations. KDD(2014) [GRL] [PDF]

• LINE: Large-scale Information Network Embedding. WWW(2015) [GRL] [PDF]

• Context- and Content-aware Embeddings for Query Rewriting in Sponsored Search. SIGIR(2015) [NLP] [PDF]

• node2vec: Scalable Feature Learning for Networks. KDD(2016) [NLP] [PDF]

• Fast Matrix Factorization for Online Recommendation with Implicit Feedback. SIGIR(2016) [RS] [PDF]

• Word2vec applied to Recommendation: Hyperparameters Matter. RecSys(2018) [RS] [PDF]

• General Knowledge Embedded Image Representation Learning. TMM(2018) [CV] [PDF]

• Alleviating Cold-Start Problems in Recommendation through Pseudo-Labelling over Knowledge Graph. WSDM(2021) [RS] [PDF]

Hard Negative Sampling

• Example-based learning for view-based human face detection. TPAMI(1998) [CV] [PDF]

• Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model. T-NN(2008) [NLP] [PDF]

• Optimizing Top-N Collaborative Filtering via Dynamic Negative Item Sampling. SIGIR(2013) [RS] [PDF]

• Bootstrapping Visual Categorization With Relevant Negatives. TMM(2013) [CV] [PDF]

• Improving Pairwise Learning for Item Recommendation from Implicit Feedback. WSDM(2014) [RS] [PDF]

• Improving Latent Factor Models via Personalized Feature Projection for One Class Recommendation. CIKM(2015) [RS] [PDF]

• Noise-Contrastive Estimation for Answer Selection with Deep Neural Networks. CIKM(2016) [NLP] [PDF]

• RankMBPR: Rank-aware Mutual Bayesian Personalized Ranking for Item Recommendation. WAIM(2016) [RS] [PDF]

• Training Region-Based Object Detectors With Online Hard Example Mining. CVPR(2016) [CV] [PDF]

• Hard Negative Mining for Metric Learning Based Zero-Shot Classification. ECCV(2016) [ML] [PDF]

• Vehicle detection in aerial images based on region convolutional neural networks and hard negative example mining. Sensors(2017) [CV] [PDF]

• WalkRanker: A Unified Pairwise Ranking Model with Multiple Relations for Item Recommendation. AAAI(2018) [RS] [PDF]

• Bootstrapping Entity Alignment with Knowledge Graph Embedding. IJCAI(2018) [KGE] [PDF]

• Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors. CVPR(2018) [CV] [PDF]

• NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding. ICDE(2019) [KGE] [PDF]

• Meta-Transfer Learning for Few-Shot Learning. CVPR(2019) [CV] [PDF]

• ULDor: A Universal Lesion Detector for CT Scans with Pseudo Masks and Hard Negative Example Mining. ISBI(2019) [CV] [PDF]

• Distributed representation learning via node2vec for implicit feedback recommendation. NCA(2020) [NLP] [PDF]

• Simplify and Robustify Negative Sampling for Implicit Collaborative Filtering. arXiv(2020) [RS] [PDF]

• Hard Negative Mixing for Contrastive Learning. arXiv(2020) [CL] [PDF]

• Bundle Recommendation with Graph Convolutional Networks. SIGIR(2020) [RS] [PDF]

• Supervised Contrastive Learning. NIPS(2020) [CL] [PDF]

• Curriculum Meta-Learning for Next POI Recommendation. KDD(2021) [RS] [PDF]

• Boosting the Speed of Entity Alignment 10×: Dual Attention Matching Network with Normalized Hard Sample Mining. WWW(2021) [KGE] [PDF]

• Hard-Negatives or Non-Negatives? A Hard-Negative Selection Strategy for Cross-Modal Retrieval Using the Improved Marginal Ranking Loss. ICCV(2021) [CV] [PDF]

Adversarial Sampling

• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks. NIPS(2015) [CV] [PDF]

• IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models. SIGIR(2017) [IR] [PDF]

• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. AAAI(2017) [NLP] [PDF]

• KBGAN: Adversarial Learning for Knowledge Graph Embeddings. NAACL(2018) [KGE] [PDF]

• Neural Memory Streaming Recommender Networks with Adversarial Training. KDD(2018) [RS] [PDF]

• GraphGAN: Graph Representation Learning with Generative Adversarial Nets. AAAI(2018) [GRL] [PDF]

• CFGAN: A Generic Collaborative Filtering Framework based on Generative Adversarial Networks. CIKM(2018) [RS] [PDF]

• Adversarial Contrastive Estimation. ACL(2018) [NLP] [PDF]

• Incorporating GAN for Negative Sampling in Knowledge Representation Learning. AAAI(2018) [KGE] [PDF]

• Exploring the potential of conditional adversarial networks for optical and SAR image matching. IEEE J-STARS(2018) [CV] [PDF]

• Deep Adversarial Metric Learning. CVPR(2018) [CV] [PDF]

• Adversarial Detection with Model Interpretation. KDD(2018) [ML] [PDF]

• Adversarial Sampling and Training for Semi-Supervised Information Retrieval. WWW(2019) [IR] [PDF]

• Deep Adversarial Social Recommendation. IJCAI(2019) [RS] [PDF]

• Adversarial Learning on Heterogeneous Information Networks. KDD(2019) [HIN] [PDF]

• Regularized Adversarial Sampling and Deep Time-aware Attention for Click-Through Rate Prediction. CIKM(2019) [RS] [PDF]

• Adversarial Knowledge Representation Learning Without External Model. IEEE Access(2019) [KGE] [PDF]

• Adversarial Binary Collaborative Filtering for Implicit Feedback. AAAI(2019) [RS] [PDF]

• ProGAN: Network Embedding via Proximity Generative Adversarial Network. KDD(2019) [GRL] [PDF]

• Generating Fluent Adversarial Examples for Natural Languages. ACL(2019) [NLP] [PDF]

• IPGAN: Generating Informative Item Pairs by Adversarial Sampling. TNLLS(2020) [RS] [PDF]

• Contrastive Learning with Adversarial Examples. arXiv(2020) [CL] [PDF]

• PURE: Positive-Unlabeled Recommendation with Generative Adversarial Network. KDD(2021) [RS] [PDF]

• Negative Sampling for Knowledge Graph Completion Based on Generative Adversarial Network. ICCCI(2021) [KGE] [PDF]

• Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation. arXiv(2021) [NLP] [PDF]

• Adversarial Feature Translation for Multi-domain Recommendation. KDD(2021) [RS] [PDF]

• Adversarial training regularization for negative sampling based network embedding. Information Sciences(2021) [GRL] [PDF]

• Adversarial Caching Training: Unsupervised Inductive Network Representation Learning on Large-Scale Graphs. TNNLS(2021) [GRL] [PDF]

• A Robust and Generalized Framework for Adversarial Graph Embedding. arxiv(2021) [GRL] [PDF]

• Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation. ICCV(2021) [CV] [PDF]

Graph-based Sampling

• ACRec: a co-authorship based random walk model for academic collaboration recommendation. WWW(2014) [RS] [PDF]

• GNEG: Graph-Based Negative Sampling for word2vec. ACL(2018) [NLP] [PDF]

• Graph Convolutional Neural Networks for Web-Scale Recommender Systems. KDD(2018) [RS] [PDF]

• SamWalker: Social Recommendation with Informative Sampling Strategy. WWW(2019) [RS] [PDF]

• Understanding Negative Sampling in Graph Representation Learning. KDD(2020) [GRL] [PDF]

• Reinforced Negative Sampling over Knowledge Graph for Recommendation. WWW(2020) [RS] [PDF]

• MixGCF: An Improved Training Method for Graph Neural Network-based Recommender Systems. KDD(2021) [RS] [PDF]

• SamWalker++: recommendation with informative sampling strategy. TKDE(2021) [RS] [PDF]

• DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN. CIKM(2021) [RS] [PDF]

Additional data enhanced Sampling

• Leveraging Social Connections to Improve Personalized Ranking for Collaborative Filtering. CIKM(2014) [RS] [PDF]

• Social Recommendation with Strong and Weak Ties. CIKM(2016) [RS] [PDF]

• Bayesian Personalized Ranking with Multi-Channel User Feedback. RecSys(2016) [RS] [PDF]

• Joint Geo-Spatial Preference and Pairwise Ranking for Point-of-Interest Recommendation. ICTAI(2017) [RS] [PDF]

• A Personalised Ranking Framework with Multiple Sampling Criteria for Venue Recommendation. CIKM(2017) [RS] [PDF]

• An Improved Sampling for Bayesian Personalized Ranking by Leveraging View Data. WWW(2018) [RS] [PDF]

• Reinforced Negative Sampling for Recommendation with Exposure Data. IJCAI(2019) [RS] [PDF]

• Geo-ALM: POI Recommendation by Fusing Geographical Information and Adversarial Learning Mechanism. IJCAI(2019) [RS] [PDF]

• Bayesian Deep Learning with Trust and Distrust in Recommendation Systems. WI(2019) [RS] [PDF]

• Socially-Aware Self-Supervised Tri-Training for Recommendation. arXiv(2021) [RS] [PDF]

• DGCN: Diversified Recommendation with Graph Convolutional Networks. WWW(2021) [RS] [PDF]

Future Outlook

False Negative Problem

• Incremental False Negative Detection for Contrastive Learning. arXiv(2021) [CL] [PDF]

• Graph Debiased Contrastive Learning with Joint Representation Clustering. IJCAI(2021) [GRL & CL] [PDF]

• Relation-aware Graph Attention Model With Adaptive Self-adversarial Training. AAAI(2021) [KGE] [PDF]

Curriculum Learning

• On The Power of Curriculum Learning in Training Deep Networks. ICML(2016) [CV] [PDF]

• Graph Representation with Curriculum Contrastive Learning. IJCAI(2021) [GRL & CL] [PDF]

Negative Sampling Ratio

• Are all negatives created equal in contrastive instance discrimination. arXiv(2020) [CL] [PDF]

• SimpleX: A Simple and Strong Baseline for Collaborative Filtering. CIKM(2021) [RS] [PDF]

• Rethinking InfoNCE: How Many Negative Samples Do You Need. arXiv(2021) [CL] [PDF]

Debiased Sampling

• Debiased Contrastive Learning. NIPS(2020) [CL] [PDF]

• Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems. KDD(2021) [RS] [PDF]

Non-Sampling

• Beyond Hard Negative Mining: Efficient Detector Learning via Block-Circulant Decomposition. ICCV(2013) [CV] [PDF]

• Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation. AAAI(2020) [RS] [PDF]

• Efficient Non-Sampling Knowledge Graph Embedding. WWW(2021) [KGE] [PDF]