To investigate on the possibility of extending the computational capabilities
of neural networks and related machine learning approaches for the treatment
of structured domains (sequences, trees and graphs), making particular attention
to adaptive, constructive and contextual models and to the study of their theoretical properties.
This is a traditonal research aim of the CIML group in Pisa, with origins since the 90s. CIML members and collaborators were among the pioneers in the processing of structures by Recursive Neural Networks and are active since then for the progress of the field
continuosly developing theoretical analyis, new models, and many applications, including new approches for the chemical domain.
The basic of Recursive Neural Networks for adaptive processing of trees, with some historical notes, can be found in RNN-Wikipedia
The recent advancements include:
Supervised neural networks for structures: the proposers have investigated both
theoretical issues and real-world applications. They have proposed a first approach
to deal with contextual information in structured domains by Recursive Neural Networks
(RNN). The proposed model, i.e. Contextual Recursive Cascade Correlation (CRCC),
a generalization of the Recursive Cascade Correlation (RCC) model, is able to partially
remove the causality assumption by exploiting contextual information. They formally
characterize the properties of CRCC showing that it is able to compute contextual transductions
and also some causal supersource transductions that RCC cannot compute.
Unsupervised recursive networks for structures: They define also a general recursive framework for unsupervised processing of structured data.
This general framework offers a uniform notation for training mechanisms of different
models and insights into theoretical issues from the SOM literature to the structure
Further working in progress on machine learning for structured domains by Neural Networks
and Kernel based methods. In particular, the NN4G models extend the input domain of Neural Networks to general classes of graphs by exploiting contextual information on a costructive and adaptive approach.
Reservoir Computing and Generative approches: the extensions of such paradigms to structured domains are ongoing research of CIML.
Contact: Alessio Micheli (Associate Professor) ¦ HomePage ¦ E-mail
See collaborators in CIML People Page
Publications on this Topic (selection restricted to recent developments on model studies)
A. Micheli. “Neural Network for Graphs: A Contextual Constructive Approach”,
IEEE Transactions on Neural Networks, Vol. 20 (3): 498 - 511, March 2009. ISSN 1045-9227.
A. Micheli, D. Sona, A. Sperduti. "Contextual Processing of
Structured Data by Recursive Cascade Correlation."
IEEE Transactions on Neural Networks. Vol. 15, n. 6, Pages 1396- 1410,
November 2004. ISSN 1045-9227.
B. Hammer, A. Micheli, A. Sperduti. "Universal Approximation Capability
of Cascade Correlation for Structures." Neural Computation, Vol. 17,
Issue 5 - May 2005, Pages 1109-1159, MIT press. ISSN 0899-7667.
A. Micheli, F. Portera, A. Sperduti. "A Preliminary Empirical Comparison
of Recursive Neural Networks and Tree Kernel Methods on Regression Tasks
for Tree Structured Domains." Neurocomputing, Elsevier. Volume 64, Pages 73-92,
March 2005 ©2005 Elsevier B.V, ISSN 0925-2312.
B. Hammer, A. Micheli, A. Sperduti, M. Strickert. "A General Framework
for Unsupervised Processing of Structured Data." Neurocomputing, Elsevier.
Volume 57, Pages 3-35, March 2004. Imprint: Elsevier ISSN 0925-2312 (Selected
from the contribution at ESANN2002).
B. Hammer, A. Micheli, A. Sperduti, M. Strickert. "Recursive Self-organizing
Network Models." Neural Networks, Elsevier Science. Vol. 17, Issues 8-9,
Pages 1061-1085, October-November 2004, Available online since 8 October 2004
(www.sciencedirect.com) © 2004 Published by Elsevier Ltd. Imprint:
Pergamon. ISSN 0893-6080.
A. Micheli. "Recursive Processing of Structured Domains in Machine Learning."
PhD Thesis, TD-13/03. Scuola di Dottorato "Galileo Galilei". Dipartimento di
Informatica, Università di Pisa, Italy, December 2003.
B. Hammer, A. Micheli, A. Sperduti. "A General Framework for Self-Organizing
Structure Processing Neural Networks", TR-03-04, Dipartimento di Informatica,
Università di Pisa, February 2003.
A. Micheli, D. Sona, A. Sperduti. "A Note on Formal Determination of Context
in Contextual Recursive Cascade Correlation Networks", TR-04-04, Dipartimento di
Informatica, Università di Pisa, January 2004.
B. Hammer, P. Tino, A. Micheli. "A Mathematical Characterization of the
Architectural Bias of Recursive Models", Technical report 252- 2004 -
Universitat Osnabruck- Germany.
B. Hammer, A. Micheli, A. Sperduti. "Adaptive Contextual Processing of Structured
Data by Recursive Neural Networks: A Survey of Computational Properties." To appear
as chapter in book: .Perspectives of Neural-Symbolic Integration. Springer series:
'Studies in Computational Intelligence', Springer Verlag. ISSN 1860-949X.