Graph neural networks
A birth of a paper
GNN is a new hot topic. I am studying this new topic and I will update what I feel helpful here to share my learning process.
Nice introduction videos
very clear explained. After watching the video, I have a very clear picture about what is GNN.
This is a pretty nice talk by Alexander Gaunt from Microsoft Research.
I have a Masters and PhD in experimental quantum physics from the University of Cambridge and am a Junior Research Fellow at Trinity College. During my PhD I developed a method for trapping and studying cold atomic clouds in holograms, culminating in publications in Science, Nature and Nature Physics. I was an early adopter of CUDA for scientific simulations, which lead to an internship and postdoc positions in the MIP group at MSRC.
Formal academic lectures
The above talk is delivered by a research scientist from NEC. This talk is very clear and informative. It should be a must-see talk although it is about 1 and a half hours long.
Libraries that might be useful
if you are a TF BOY, please check the graph nets developed by Deep mind.
Get started by developing your first GNN project
This video is a great explanation about how to get started.
I download the data and code from here
As the original includes too many files, I created a separate one and put it in a repo in my github. You can get my version from here.
git clone https://github.com/liketheflower/gnn.git
The project Jupyter notebook is here
The training process and final prediction can be found here:
If you wanna fully understand the system, please check out the following code. It has fewer nodes and you can print out all the intermediate values to have a better understanding of the model.
Permalink Dismiss GitHub is home to over 40 million developers working together to host and review code, manage…
Tutorial from DGL is a good alternative to build a first GNN model
First practice of using VGAE (April 20, 2020)
reproduce the experiment result of paper Variational Graph Auto-Encoders
The Cora dataset consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words
Seven classes are
Detail about this dataset
THE DIRECTORY CONTAINS TWO FILES:
The .content file contains descriptions of the papers in the following format:
<paper_id> <word_attributes>+ <class_label>
The first entry in each line contains the unique string ID of the paper followed by binary values indicating whether each word in the vocabulary is present (indicated by 1) or absent (indicated by 0) in the paper. Finally, the last entry in the line contains the class label of the paper.
The .cites file contains the citation graph of the corpus. Each line describes a link in the following format:
<ID of cited paper> <ID of citing paper>
Each line contains two paper IDs. The first entry is the ID of the paper being cited and the second ID stands for the paper which contains the citation. The direction of the link is from right to left. If a line is represented by “paper1 paper2” then the link is “paper2->paper1”.
Visualization of the built graph
The graph has
I did a visualization by modifying the code from the link mentioned before. Pretty beautiful, right?
- Model summary
The model used in this project contains 2 GCN layers and one decoder (InnerProductDecoder)
In_features is 1433 as “The dictionary consists of 1433 unique words”
Detail of the model is described in the Variational Graph Auto-Encoders paper.
The  is this paper Semi-Supervised Classification with Graph Convolutional Networks.
Output of the model by pytorch
- Training loss curve
In the original code
L2 loss is calculated between estimated adj_logits and adj. However, the estimated adj_logits are from a linear activation function which means the scope of the value is from -inf to +inf.
loss = loss_function(adj_logits, adj, pos_weight=pos_weight)
I change it by passing it to a sigmoid function and it is better.
def f(x):return torch.sigmoid(x)loss = loss_function(f(adj_logits), adj, pos_weight=pos_weight)
I am on the way to move forward fast. If you are also interested in this topic, drop me comments. Let’s improve together. Cheers.
Zero to hero plan
Read basic tutorials
Do a simple project by using GNN
- Read 50 GNN related papers (Not yet)
- Fully understand two other people’s projects (Yes)
- Do my own project (Yes)
- Publish a paper by using GNN (On the way, the paper is ready. Cheers !)
My advisors are Richard Zemel and Raquel Urtasun. My M.Phil. advisor was Jiaya Jia at the Chinese University of Hong…
links provided in the article.