A PyTorch implementation of "Predict then Propagate: Graph Neural Networks meet Personalized PageRank" (ICLR 2019).

Last update: Dec 30, 2022

Overview

APPNP

⠀

A PyTorch implementation of Predict then Propagate: Graph Neural Networks meet Personalized PageRank (ICLR 2019).

Abstract

Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success. However, these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood cannot be easily extended. In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank. We utilize this propagation procedure to construct personalized propagation of neural predictions (PPNP) and its approximation, APPNP. Our model's training time is on par or faster and its number of parameters on par or lower than previous models. It leverages a large, adjustable neighborhood for classification and can be combined with any neural network. We show that this model outperforms several recently proposed methods for semi-supervised classification on multiple graphs in the most thorough study done so far for GCN-like models.

A PyTorch and Tensorflow implementation is awailable [here.].

This repository provides a PyTorch implementation of PPNP and APPNP as described in the paper:

Predict then Propagate: Graph Neural Networks meet Personalized PageRank. Johannes Klicpera, Aleksandar Bojchevski, Stephan Günnemann. ICLR, 2019. [Paper]

Requirements

The codebase is implemented in Python 3.5.2. package versions used for development are just below.

networkx          2.4
tqdm              4.28.1
numpy             1.15.4
pandas            0.23.4
texttable         1.5.0
scipy             1.1.0
argparse          1.1.0
torch             1.1.0
torch-scatter     1.4.0
torch-sparse      0.4.3
torch-cluster     1.4.5
torch-geometric   1.3.2
torchvision       0.3.0

Datasets

The code takes the **edge list** of the graph in a csv file. Every row indicates an edge between two nodes separated by a comma. The first row is a header. Nodes should be indexed starting with 0. A sample graph for `Cora` is included in the `input/` directory. In addition to the edgelist there is a JSON file with the sparse features and a csv with the target variable.

The **feature matrix** is a sparse binary one it is stored as a json. Nodes are keys of the json and feature indices are the values. For each node feature column ids are stored as elements of a list. The feature matrix is structured as:

{ 0: [0, 1, 38, 1968, 2000, 52727],
  1: [10000, 20, 3],
  2: [],
  ...
  n: [2018, 10000]}

The target vector is a csv with two columns and headers, the first contains the node identifiers the second the targets. This csv is sorted by node identifiers and the target column contains the class meberships indexed from zero.

NODE ID	Target
0	3
1	1
2	0
3	1
...	...
n	3

Options

Training an APPNP/PPNP model is handled by the src/main.py script which provides the following command line arguments.

Input and output options

  --edge-path       STR    Edge list csv.         Default is `input/cora_edges.csv`.
  --features-path   STR    Features json.         Default is `input/cora_features.json`.
  --target-path     STR    Target classes csv.    Default is `input/cora_target.csv`.

Model options

  --seed              INT     Random seed.                   Defailt is 42.
  --model             STR     Model exact or approximate.    Default is `exact`.
  --iterations        INT     APP iterations.                Default is 10.
  --alpha             FLOAT   Teleport parameter.            Default is 0.1
  --epochs            INT     Number of training epochs.     Default is 2000.
  --early-stopping    INT     Early stopping rounds.         Default is 5.
  --training-size     INT     Training set size.             Default is 1500.
  --test-size         INT     Test set size.                 Default is 500.
  --learning-rate     FLOAT   Adam learning rate.            Default is 0.01
  --dropout           FLOAT   Dropout rate value.            Default is 0.5
  --lambd             FLOAT   Rgularization parameter.       Default is 0.005.
  --layers            LST     Layer sizes in first layers.   Default is [64, 64].

Examples

The following commands learn a neural network and score on the test set. Training a model on the default dataset.

python src/main.py

Training a PPNP model for a 100 epochs.

python src/main.py --epochs 100

Training an APPNP model.

python src/main.py --model approximate

Increasing the learning rate and the dropout.

python src/main.py --learning-rate 0.1 --dropout 0.9

License

GNU License

You might also like...

A PyTorch implementation of SlowFast based on ICCV 2019 paper "SlowFast Networks for Video Recognition"

SlowFast A PyTorch implementation of SlowFast based on ICCV 2019 paper SlowFast Networks for Video Recognition. Requirements Anaconda PyTorch conda in

8 Dec 23, 2022

Unofficial PyTorch Implementation of AHDRNet (CVPR 2019)

AHDRNet-PyTorch This is the PyTorch implementation of Attention-guided Network for Ghost-free High Dynamic Range Imaging (CVPR 2019). The official cod

4 Sep 8, 2022

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biom

59 Nov 24, 2022

Pytorch implementation of BRECQ, ICLR 2021

BRECQ Pytorch implementation of BRECQ, ICLR 2021 @inproceedings{ li&gong2021brecq, title={BRECQ: Pushing the Limit of Post-Training Quantization by Bl

148 Dec 28, 2022

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

512x512 flowers after 12 hours of training, 1 gpu 256x256 flowers after 12 hours of training, 1 gpu Pizza 'Lightweight' GAN Implementation of 'lightwe

1.5k Jan 2, 2023

This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).

DCL-PyTorch Pytorch implementation for the Dynamic Concept Learner (DCL). More details can be found at the project page. Framework Grounding Physical

31 Jan 6, 2023

An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Neural Attention Distillation This is an implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep

84 Jan 4, 2023

Official PyTorch implementation of "Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets" (ICLR 2021)

Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets This is the official PyTorch implementation for the paper Rapid Neural A

48 Dec 26, 2022

PyTorch implementation of Wide Residual Networks with 1-bit weights by McDonnell (ICLR 2018)

1-bit Wide ResNet PyTorch implementation of training 1-bit Wide ResNets from this paper: Training wide residual networks for deployment using a single

122 Dec 7, 2022

Comments

About training

Hi, thanks for you sharing your nice work. I have a question when I run your code. Maybe I misunderstand the main point of this paper. It seems that when you train you model, you do not apply the graph information, which means your network is a normal fully connected network. After training, you apply PageRank to the output of the network when evaluate the model. Is my understanding correct?

opened by Jianjin123 4

Releases(v_00001)

v_00001(Jun 5, 2021)

"Predict then Propagate: Graph Neural Networks meet Personalized PageRank" (ICLR 2019).
Source code(tar.gz)
Source code(zip)

A PyTorch implementation of "Predict then Propagate: Graph Neural Networks meet Personalized PageRank" (ICLR 2019).

Related tags

Overview

APPNP

Abstract

Requirements

Datasets

Options

Input and output options

Model options

Examples

You might also like...

A PyTorch implementation of SlowFast based on ICCV 2019 paper "SlowFast Networks for Video Recognition"

Unofficial PyTorch Implementation of AHDRNet (CVPR 2019)

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Pytorch implementation of BRECQ, ICLR 2021

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).

An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Official PyTorch implementation of "Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets" (ICLR 2021)

PyTorch implementation of Wide Residual Networks with 1-bit weights by McDonnell (ICLR 2018)

Comments

About training

Releases(v_00001)

v_00001(Jun 5, 2021)

Owner

Benedek Rozemberczki

Code for ACL 21: Generating Query Focused Summaries from Query-Free Resources

This tutorial aims to learn the basics of deep learning by hands, and master the basics through combination of lectures and exercises

Code for paper Novel View Synthesis via Depth-guided Skip Connections

A PyTorch Implementation of "Neural Arithmetic Logic Units"

[TIP 2020] Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion

Pose Transformers: Human Motion Prediction with Non-Autoregressive Transformers

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

An implementation of Equivariant e2 convolutional kernals into a convolutional self attention network, applied to radio astronomy data.

code for "Self-supervised edge features for improved Graph Neural Network training",

A little Python application to auto tag your photos with the power of machine learning.

Another pytorch implementation of FCN (Fully Convolutional Networks)

Code and experiments for "Deep Neural Networks for Rank Consistent Ordinal Regression based on Conditional Probabilities"

Plaything for Autistic Children (demo for PaddlePaddle/Wechaty/Mixlab project)

Official Implementation (PyTorch) of "Point Cloud Augmentation with Weighted Local Transformations", ICCV 2021

GLODISMO: Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

The code written during my Bachelor Thesis "Classification of Human Whole-Body Motion using Hidden Markov Models".

Simple-Neural-Network From Scratch in Python

Attack classification models with transferability, black-box attack; unrestricted adversarial attacks on imagenet

AttGAN: Facial Attribute Editing by Only Changing What You Want (IEEE TIP 2019)

A Robust Unsupervised Ensemble of Feature-Based Explanations using Restricted Boltzmann Machines