This document presents a novel graph neural network (GNN) convolutional layer based on Auto-Regressive Moving Average (ARMA) filters. The ARMA layer aims to address limitations of existing GNN layers that use polynomial filters by providing a more flexible frequency response with fewer parameters. It models graph signals using parallel stacks of recurrent operations to approximate high-order neighborhoods efficiently. Experimental results show the ARMA layer outperforms other GNN architectures on tasks like node classification, graph signal classification, and graph regression. Future work could explore incorporating text and content metadata into graph convolutional models.

1. Joo-Ho Lee
School of Computer Science and Information Engineering,
The Catholic University of Korea
E-mail: jooho414@gmail.com
2023-07-10

2. 1
Introduction
Problem Statement
• Such a filter selectively shrinks or amplifies the Fourier coefficients of the graph signal (an instance of the
node features) and then maps the node features to a new space
• To avoid the expensive spectral decomposition and projection in the frequency domain, state-of-the-art
GNNs implement graph filters as low-order polynomials that are learned directly in the node domain

3. 2
Introduction
Problem Statement
• Polynomial filters have a finite impulse response and perform a weighted moving average filtering of graph
signals on local node neighborhoods, allowing for fast distributed implementations such as those based on
Chebyshev polynomials and Lanczos iterations

4. 3
Introduction
Problem Statement
• Polynomial filters have limited modeling capabilities and, due to their smoothness, can not model sharp
changes in the frequency response
• Crucially, polynomials with high degree are necessary to reach high-order neighborhoods, but they tend to
be more computationally expensive and, most importantly, overfit the training data making the model
sensitive to changes in the graph signal or the underlying graph structure

5. 4
Introduction
Problem Statement
• A more versatile class of filters is the family of Auto-Regressive Moving Average filters (ARMA), which offer a
larger variety of frequency responses and can account for higher-order neighborhoods compared to
polynomial filters with the same number of parameters

6. 5
Introduction
Contribution
• They address the limitations of existing graph convolutional layers inspired by polynomial filters and
propose a novel GNN convolutional layer based on ARMA filters
• ARMA layer implements a non-linear and trainable graph filter that generalizes the convolutional layers
based on polynomial filters and provides the GNN with enhanced modeling capability, thanks to a flexible
design of the filter’s frequency response
• The ARMA layer captures global graph structures with fewer parameters, overcoming GNNs with
convolutional ARMA filters the limitations of GNNs based on high-order polynomial filters
• Results show that a GNN equipped with ARMA layers outperforms GNNs with polynomial filters in every
downstream task

7. 6
Method
Architecture

8. 7
Method
The ARMA neural network layer
• Graph Convolutional Skip (GCS) layer

9. 8
Method
The ARMA neural network layer
• The output of the AMRA convolutional layer

10. 9
Experiment
Datasets

11. 10
Experiment
Node ClassificatIon

12. 11
Experiment
Graph signal classification

13. 12
Experiment
Graph Classification

14. 13
Experiment
Graph Regression

15. 14
Conclusion
• This paper introduced the ARMA layer, a novel graph convolutional layer based on a rational graph filter
• The ARMA layer models more expressive filter responses and can account for larger neighborhoods compared
to GNN layers based on polynomial filters of the same order
• ARMA layer consists of parallel stacks of recurrent operations, which approximate a graph filter with an arbitrary
order K by means of efficient sparse tensor multiplications
• The experiments showed that the proposed ARMA layer outperforms existing GNN architectures, including those
based on polynomial filters and other more complex models, on a large variety of graph machine learning tasks

16. 15
Future work
• As for the future work, it might be worthwhile to investigate text-augmented graphs where texts are associated
not only for the item nodes but also for the user nodes as well
• CoRGi can naturally operate on this setting because the personalized attention mechanism considers
directionality and the message floating from user to item can be different from that floating from item to user
• Furthermore, another potential future direction is to work on content metadata outside of texts: for example,
pixel-based images