Download Convolutional neural network of Graphs without any a

PhD topic - GREYC (Caen, France), - 2017
Convolutional neural network of Graphs
without any a priori on their structures
Objectives: Numerous problems (drug property prediction, IP networks, social networks,. . . ) involve
data which do not lie on an Eucidean space but which are efficiently represented through graph data
structures (often with attributes on nodes and/or edges). The richness of this type of data combined with
Big data aspects requires the design of efficient machine learning techniques. Recently, numerous results
have proven that deep learning methods constitute a very powerful tool for the analysis of Euclidean
data available in large numbers. In particular, Convolutional Neural Networks (CNN) [1] allow to
extract statistically significant patterns from large set of data and this property allows them to improve
significantly the pattern recognition tasks on Image, Sound and Video Media [2].
Recently the scientific community in Signal processing and Machine learning have demonstrated a big
interest in the generalization of this kind of neural network to graph data [3]. This problem is far from
being trivial since the basic operations of convolution, of coarsening and information sharing between
several layers are well defined only for regular grids. These last points make the adaptation of CNN to
graphs particularly complex. We may distinguish two main streams among methods extending CNN to
graphs. Indeed, these two types of methods consider two different problems.
The first type of methods attempts to analyses signals on graphs with a fixed structure. The majority
of methods recently proposed belong to this familly of methods [4, 5, 6]. The main drawback of these
methods is that they are based on a spectral formulation of the convolution which is relative to the
Fourier transform on graphs [7] and which is thus valid only for the considered graph. The spectral
model learn on one graph can not be easily applied on an other graph having a different Fourier basis.
This last point constitute a major drawback.
The second type of method attempt to characterized directly the structure of graphs and hence to
define CNN on graphs without apriori on their structure. Such a problem is usually considered by using
manifold learning methods [8] or the characterization of structural pattern composing the graph [9]. For
example, given a collection of graphs we would like to learn a classification function on these graphs (in
order, for example, to categorize them). Such a function should be sufficiently robust to be applied on
unknown graphs with a topology which may be quite different from the ones of the learning set (node
sets of graphs are not necessarily in correspondence). Very few approaches have considered this problem
with CNN on graphs. The majority of these methods use a normalization step which allows to come back
to a regular 1D or 2D grid [10, 11] hence loosing a large part of the structural information of graphs.
Other approaches are based on local geometric patches when a manifold structure may be attached to
the graph [12] which is not always true.
Within this thesis we will consider this second type of problems and we will try to avoid any apriori
on the structure of the graphs which may be presented to a CNN. In order to reach this, we will consider
different research direction combining graph kernels notions [13], the notion of supergraph [14] of a graph
database, the notions of coarsening and pooling by weighed aggregation [15]. The applications of these
results will be focused on the prediction of the property of molecular graphs for symbolic graphs and the
classification of images for graphs with numerical attributes.
Supervision:
Collaboration:
Sébastien Bougleux ([email protected]), Luc Brun ([email protected])
Olivier Lézoray ([email protected])
Candidate Profile:
Master 2 level. Good skills in Programming (C,C++, Matlab). Skills in Graph
theory and/or Machine learning and/or optimisation would be apreciated.
References
[1] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”
Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, November 1998.
[2] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444,
05 2015.
[3] Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst, “Geometric
deep learning: going beyond euclidean data,” CoRR, vol. abs/1611.08097, 2016.
[4] Mikael Henaff, Joan Bruna, and Yann LeCun, “Deep convolutional networks on graph-structured data,”
CoRR, vol. abs/1506.05163, 2015.
[5] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst, “Convolutional neural networks on graphs
with fast localized spectral filtering,” CoRR, vol. abs/1606.09375, 2016.
[6] Michael Edwards and Xianghua Xie,
abs/1609.08965, 2016.
“Graph based convolutional neural network,”
CoRR, vol.
[7] D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal
processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,”
IEEE Signal Process. Mag., vol. 30, no. 3, pp. 83–98, 2013.
[8] Mikhail Belkin and Partha Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Computation, vol. 15, no. 6, pp. 1373–1396, 2003.
[9] Nino Shervashidze, S. V. N. Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten M. Borgwardt,
“Efficient graphlet kernels for large graph comparison,” in Proceedings of the Twelfth International Conference
on Artificial Intelligence and Statistics, AISTATS 2009, Clearwater Beach, Florida, USA, April 16-18, 2009,
2009, pp. 488–495.
[10] Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov, “Learning convolutional neural networks for
graphs,” in Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York
City, NY, USA, June 19-24, 2016, 2016, pp. 2014–2023.
[11] Shaosheng Cao, Wei Lu, and Qiongkai Xu, “Deep neural networks for learning graph representations,” in
Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix,
Arizona, USA., 2016, pp. 1145–1152.
[12] Jonathan Masci, Davide Boscaini, Michael M. Bronstein, and Pierre Vandergheynst, “Geodesic convolutional
neural networks on riemannian manifolds,” in 2015 IEEE International Conference on Computer Vision
Workshop, ICCV Workshops 2015, Santiago, Chile, December 7-13, 2015, 2015, pp. 832–840.
[13] John Shawe-Taylor and Nello Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press,
2004.
[14] Horst Bunke, P. Foggia, C. Guidobaldi, and M. Vento, Graph Clustering Using the Weighted Minimum
Common Supergraph, pp. 235–246, Springer Berlin Heidelberg, Berlin, Heidelberg, 2003.
[15] Cédric Chevalier and Ilya Safro, “Learning and intelligent optimization,” chapter Comparison of Coarsening
Schemes for Multilevel Graph Partitioning, pp. 191–205. Springer-Verlag, Berlin, Heidelberg, 2009.