Abstract
This paper addresses the issue of building a part-based representation of a dataset of images. More precisely, we look for a non-negative, sparse decomposition of the images on a reduced set of atoms, in order to unveil a morphological and explainable structure of the data. Additionally, we want this decomposition to be computed online for any new sample that is not part of the initial dataset. Therefore, our solution relies on a sparse, non-negative auto-encoder, where the encoder is deep (for accuracy) and the decoder shallow (for explainability). This method compares favorably to the state-of-the-art online methods on two benchmark datasets (MNIST and Fashion MNIST) and on a hyperspectral image, according to classical evaluation measures and to a new one we introduce, based on the equivariance of the representation to morphological operators.
References
[1] Jesús Angulo and Santiago Velasco-Forero. Sparse mathematical morphology using non-negative matrix factorization. In Pierre Soille, Martino Pesaresi, and Georgios K. Ouzounis, editors, 10th International Symposium on Mathematical Morphology and Its Application to Signal and Image Processing (ISMM), volume LNCS 6671, pages 1–12, 2011.10.1007/978-3-642-21569-8_1Search in Google Scholar
[2] Babajide O. Ayinde and Jacek M. Zurada. Deep learning of constrained autoencoders for enhanced understanding of data. CoRR, abs/1802.00003, 2018.Search in Google Scholar
[3] Vasileios Charisopoulos and Petros Maragos. Morphological perceptrons: Geometry and training algorithms. In Jesús Angulo, Santiago Velasco-Forero, and Fernand Meyer, editors, 13th International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing (ISMM), volume LNCS 10225, pages 3–15. Springer International Publishing, 2017.10.1007/978-3-319-57240-6_1Search in Google Scholar
[4] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. CoRR, abs/1606.03657, 2016.Search in Google Scholar
[5] Jan Chorowski and Jacek M. Zurada. Learning understandable neural networks with nonnegative weight constraints. IEEE Transactions on Neural Networks and Learning Systems, 26(1):62–69, 2015.10.1109/TNNLS.2014.2310059Search in Google Scholar PubMed
[6] David Donoho and Victoria Stodden. When does non-negative matrix factorization give correct decomposition into parts? In Advances in Neural Information Processing Systems 16 (NIPS), pages 1141–1148, 2004.Search in Google Scholar
[7] Alexey Dosovitskiy, Jost Tobias Springenberg, and Thomas Brox. Learning to generate chairs with convolutional neural networks. CoRR, abs/1411.5928, 2014.10.1109/CVPR.2015.7298761Search in Google Scholar
[8] Alaa El Khatib, Shimeng Huang, Ali Ghodsi, and Fakhri Karray. Nonnegative matrix factorization using autoencoders and exponentiated gradient descent. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2018.10.1109/IJCNN.2018.8489242Search in Google Scholar
[9] Gianni Franchi, Amin Fehri, and Angela Yao. Deep morphological networks. Pattern Recognition, 102:107246, 2020.10.1016/j.patcog.2020.107246Search in Google Scholar
[10] Ehsan Hosseini-Asl, Jacek M. Zurada, and Olfa Nasraoui. Deep learning of part-based representation of data using sparse autoencoders with nonnegativity constraints. IEEE Transactions on Neural Networks and Learning Systems, 27(12):2486–2498, 2016.10.1109/TNNLS.2015.2479223Search in Google Scholar PubMed
[11] Patrik O. Hoyer. Non-negative matrix factorization with sparseness constraints. CoRR, cs.LG/0408058, 2004.Search in Google Scholar
[12] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR, abs/1502.03167, 2015.Search in Google Scholar
[13] Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010.Search in Google Scholar
[14] Honglak Lee, Chaitanya Ekanadham, and Andrew Y. Ng. Sparse deep belief net model for visual area v2. In John C. Platt, Daphne Koller, Yoram Singer, and Sam T. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 873–880. Curran Associates, Inc., 2008.Search in Google Scholar
[15] Andre Lemme, René Felix Reinhart, and Jochen J. Steil. Online learning and generalization of parts-based image representations by non-negative sparse autoencoders. Neural Networks, 33:194–203, 2012.10.1016/j.neunet.2012.05.003Search in Google Scholar PubMed
[16] Andrew L. Maas. Rectifier nonlinearities improve neural network acoustic models. In International Conference on Machine Learning, page 3, 2013.Search in Google Scholar
[17] Julien Mairal, Francis R. Bach, and Jean Ponce. Sparse modeling for image and vision processing. CoRR, abs/1411.3230, 2014.10.1561/9781680830095Search in Google Scholar
[18] Petros Maragos and R Schafer. Morphological skeleton representation and coding of binary images. IEEE Transactions on Acoustics, Speech, and Signal Processing, 34(5):1228–1244, 1986.10.1109/TASSP.1986.1164959Search in Google Scholar
[19] Martino Pesaresi and Jón A. Benediktsson. A new approach for the morphological segmentation of high-resolution satellite imagery. IEEE Transactions on Geoscience and Remote Sensing, 39(2):309–320, 2001.10.1109/36.905239Search in Google Scholar
[20] Bastien Ponchon, Santiago Velasco-Forero, Samy Blusseau, Jesús Angulo, and Isabelle Bloch. Part-based approximations for morphological operators using asymmetric auto-encoders. In Bernhard Burgeth, Andreas Kleefeld, Benoît Naegel, Nicolas Passat, and Benjamin Perret, editors, 14th International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing (ISMM), volume LNCS 11564, pages 323–334. Springer International Publishing, 2019.10.1007/978-3-030-20867-7_25Search in Google Scholar
[21] Gerhard X. Ritter and Peter Sussner. An introduction to morphological neural networks. In 13th International Conference on Pattern Recognition, volume 4, pages 709 – 717, 1996.10.1109/ICPR.1996.547657Search in Google Scholar
[22] Yucong Shen, Xin Zhong, and Frank Y Shih. Deep morphological neural networks. arXiv preprint arXiv:1909.01532, 2019.Search in Google Scholar
[23] Frank Y Shih, Yucong Shen, and Xin Zhong. Development of deep learning framework for mathematical morphology. International Journal of Pattern Recognition and Artificial Intelligence, 33(06):1954024, 2019.10.1142/S0218001419540247Search in Google Scholar
[24] Pierre Soille. Morphological image analysis: principles and applications. Springer Science & Business Media, 2013.Search in Google Scholar
[25] Keiji Tanaka. Columns for complex visual object features in the inferotemporal cortex: clustering of cells with similar but slightly different stimulus selectivities. Cerebral Cortex, 13 1:90–9, 2003.Search in Google Scholar
[26] Fabian J Theis, Kurt Stadlthanner, and Toshihisa Tanaka. First results on uniqueness of sparse non-negative matrix factorization. In 13th IEEE European Signal Processing Conference, pages 1–4, 2005.Search in Google Scholar
[27] Marcos Eduardo Valle. Reduced dilation-erosion perceptron for binary classification. Mathematics, 8(4):512, 2020.10.3390/math8040512Search in Google Scholar
[28] Santiago Velasco-Forero and Jesús Angulo. Non-negative sparse mathematical morphology. In Advances in Imaging and Electron Physics, volume 202, chapter 1, pages 1 – 37. Elsevier Inc.Academic Press, 2017.10.1016/bs.aiep.2017.07.001Search in Google Scholar
[29] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747, 2017.Search in Google Scholar
[30] Yonghua Yin and Erol Gelenbe. Non-negative autoencoder with simplified random neural network. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–6, 2019.Search in Google Scholar
[31] Li Zhang and Yaping Lu. Comparison of auto-encoders with different sparsity regularizers. In 2015 International Joint Conference on Neural Networks (IJCNN), pages 1–5, 2015.10.1109/IJCNN.2015.7280364Search in Google Scholar
[32] Yunxiang Zhang, Samy Blusseau, Santiago Velasco-Forero, Isabelle Bloch, and Jesús Angulo. Max-plus operators applied to filter selection and model pruning in neural networks. In Bernhard Burgeth, Andreas Kleefeld, Benoît Naegel, Nicolas Passat, and Benjamin Perret, editors, 14th International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing (ISMM), volume LNCS 11564, pages 310–322. Springer International Publishing, 2019.10.1007/978-3-030-20867-7_24Search in Google Scholar
© 2020 Samy Blusseau et al., published by Sciendo
This work is licensed under the Creative Commons Attribution 4.0 International License.
