Generative Adversarial Networks(GANs)

1. Vanilla GAN
2. ClusterGAN

Vanilla GAN

Model Structure

Final Generator Structure

• A MLP with 2 hidden layers of hidden_size=1024

• A LeakyReLU of slope=0.2 is used for each layer for activation

• nn.Sequential(
      nn.Linear(self.n_z, self.hidden_size),
      nn.LeakyReLU(0.2),
  
      nn.Linear(self.hidden_size, self.hidden_size),
      nn.LeakyReLU(0.2),
  
      nn.Linear(self.hidden_size, self.hidden_size),
      nn.LeakyReLU(0.2),
  
      nn.Linear(self.hidden_size, 784),
      nn.Tanh(),
  )

Final Discriminator Structure

• A MLP with 2 hidden layers of hidden_size=1024

• A LeakyReLU of slope=0.2 is used for each layer for activation

• A Dropout of rate=0.3 is used for each layer

• nn.Sequential(
      nn.Linear(self.n_input, self.hidden_size),
      nn.LeakyReLU(0.2),
      nn.Dropout(0.3),
  
      nn.Linear(self.hidden_size, self.hidden_size),
      nn.LeakyReLU(0.2),
      nn.Dropout(0.3),
  
      nn.Linear(self.hidden_size, self.hidden_size),
      nn.LeakyReLU(0.2),
      nn.Dropout(0.3),
  
      nn.Linear(self.hidden_size, 1),
      nn.Sigmoid(),
  )

I tried several settings of Generator and Discriminator for 200 epochs to determine the final structures, including activations, Dropout rate, and # of hidden layers.

Training Losses

Trained for 500 epochs on a GPU

Generated Results During Training

Trained for 500 epochs on a GPU

ClusterGAN

TODO

Reference

• [GPAM+14] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
• [MALK19] Sudipto Mukherjee, Himanshu Asnani, Eugene Lin, and Sreeram Kannan. Clustergan: Latent space clustering in generative adversarial networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4610–4617, 2019.
• https://github.com/eriklindernoren/PyTorch-GAN from PapersWithCode