Rémi Flamary

Professional website

Home

photo

I am associate professor at Nice-Sophia Antipolis University in the Departement of Electronics and in the Lagrange Laboratory. This laboratory is part of the Observatoire de la Côte d'Azur. I was previously a PhD student and teaching assistant at the LITIS Laboratory and my PhD advisor was Alain Rakotomamonjy at Rouen University.

On this website, you can find a list of my publications and download the corresponding software/code. Some of my french teaching material is also available.

Research Interests

  • Machine learning and statistical signal processing
    • Classification, supervised learning
    • Kernel methods, Support Vector Machines
    • Optimization with sparsity, variable selection, mixed norms, non convex regularization
    • Feature learning, data representation, kernel learning
    • Convolutional neural networks, filter learning, image reconstruction
    • Optimal transport, domain adaptation
  • Applications
    • Biomedical engineering, Brain-Computer Interfaces
    • Remote sensing and hyperspectral Imaging
    • Astronomical image processing

Wordcloud of my research interests.

Recent work

R. Flamary, C. Févotte, N. Courty, V. Emyia, "Optimal spectral transportation with application to music transcription", Neural Information Processing Systems (NIPS), 2016.
Abstract: Many spectral unmixing methods rely on the non-negative decomposition of spectral data onto a dictionary of spectral templates. In particular, state-of-the-art music transcription systems decompose the spectrogram of the input signal onto a dictionary of representative note spectra. The typical measures of fit used to quantify the adequacy of the decomposition compare the data and template entries frequency-wise. As such, small displacements of energy from a frequency bin to another as well as variations of timber can disproportionally harm the fit. We address these issues by means of optimal transportation and propose a new measure of fit that treats the frequency distributions of energy holistically as opposed to frequency-wise. Building on the harmonic nature of sound, the new measure is invariant to shifts of energy to harmonically-related frequencies, as well as to small and local displacements of energy. Equipped with this new measure of fit, the dictionary of note templates can be considerably simplified to a set of Dirac vectors located at the target fundamental frequencies (musical pitch values). This in turns gives ground to a very fast and simple decomposition algorithm that achieves state-of-the-art performance on real musical data.
BibTeX:
@inproceedings{flamary2016ost,
author = {Flamary, Remi and Févotte, Cédric and Courty, N. and  Emyia, Valentin},
title = {Optimal spectral transportation with application to music transcription},
booktitle = { Neural Information Processing Systems (NIPS)},
year = {2016}
}
M. Perrot, N. Courty, R. Flamary, A. Habrard, "Mapping estimation for discrete optimal transport", Neural Information Processing Systems (NIPS), 2016.
Abstract: We are interested in the computation of the transport map of an Optimal Transport problem. Most of the computational approaches of Optimal Transport use the Kantorovich relaxation of the problem to learn a probabilistic coupling but do not address the problem of learning the transport map linked to the original Monge problem. Consequently, it lowers the potential usage of such methods in contexts where out-of-samples computations are mandatory. In this paper we propose a new way to jointly learn the coupling and an approximation of the transport map. We use a jointly convex formulation which can be efficiently optimized. Additionally, jointly learning the coupling and the transport map allows to smooth the result of the Optimal Transport and generalize it on out-of-samples examples. Empirically, we show the interest and the relevance of our method in two tasks: domain adaptation and image editing.
BibTeX:
@inproceedings{perrot2016mapping,
author = {Perrot, M. and Courty, N. and Flamary, R. and Habrard, A.},
title = {Mapping estimation for discrete optimal transport},
booktitle = {Neural Information Processing Systems (NIPS)},
year = {2016}
}
N. Courty, R. Flamary, D. Tuia, A. Rakotomamonjy, "Optimal transport for domain adaptation", Pattern Analysis and Machine Intelligence, IEEE Transactions on , 2016.
Abstract: Domain adaptation is one of the most challenging tasks of modern data analytics. If the adaptation is done correctly, models built on a specific data representations become more robust when confronted to data depicting the same semantic concepts (the classes), but observed by another observation system with its own specificities. Among the many strategies proposed to adapt a domain to another, finding domain-invariant representations has shown excellent properties, as a single classifier can use labelled samples from the source domain under this representation to predict the unlabelled samples of the target domain. In this paper, we propose a regularized unsupervised optimal transportation model to perform the alignment of the representations in the source and target domains. We learn a transportation plan matching both PDFs, which constrains labelled samples in the source domain to remain close during transport. This way, we exploit at the same time the few labeled information in the source and distributions of the input/observation variables observed in both domains. Experiments in toy and challenging real visual adaptation examples show the interest of the method, that consistently outperforms state of the art approaches.
BibTeX:
@article{courty2016optimal,
author = { Courty, N. and Flamary, R.  and Tuia, D. and Rakotomamonjy, A.},
title = {Optimal transport for domain adaptation},
journal = { Pattern Analysis and Machine Intelligence, IEEE Transactions on },
year = {2016}
}
S. Canu, R. Flamary, D. Mary, "Introduction to optimization with applications in astronomy and astrophysics", Mathematical tools for instrumentation and signal processing in astronomy, 2016.
Abstract: This chapter aims at providing an introduction to numerical optimization with some applications in astronomy and astrophysics. We provide important preliminary definitions that will guide the reader towards different optimization procedures. We discuss three families of optimization problems and describe numerical algorithms allowing, when this is possible, to solve these problems. For each family, we present in detail simple examples and more involved advanced examples. As a final illustration, we focus on two worked-out examples of optimization applied to astronomical data. The first application is a supervised classification of RR-Lyrae stars. The second one is the denoising of galactic spectra formulated by means of sparsity inducing models in a redundant dictionary.
BibTeX:
@incollection{canu2016introduction,
author = { Canu, Stephane, and Flamary, Remi and Mary, David},
title = {Introduction to optimization with applications in astronomy and astrophysics},
booktitle = { Mathematical tools for instrumentation and signal processing in astronomy},
editor = { {Mary, David and Flamary, Remi, and Theys, Celine, and Aime, Claude}},
year = {2016}
}
I. Harrane, R. Flamary, C. Richard, "Doubly partial-diffusion LMS over adaptive networks", Asilomar Conference on Signals, Systems and Computers (ASILOMAR), 2016.
Abstract: Diffusion LMS is an efficient strategy for solving distributed optimization problems with cooperating agents. Nodes are interested in estimating the same parameter vector and exchange information with their neighbors to improve their local estimates. However, successful implementation of such applications depends on a substantial amount of communication resources. In this paper, we introduce diffusion algorithms that have a significantly reduced communication load without compromising performance. We also perform analyses in the mean and mean-square sense. Simulations results are provided to confirm the theoretical findings.
BibTeX:
@inproceedings{harrane2016doubly,
author = {Harrane, Ibrahim and Flamary, R. and Richard, C.},
title = {Doubly partial-diffusion LMS over adaptive networks},
booktitle = {Asilomar Conference on Signals, Systems and Computers (ASILOMAR)},
year = {2016}
}
I. Harrane, R. Flamary, C. Richard, "Toward privacy-preserving diffusion strategies for adaptation and learning over networks", European Conference on Signal Processing (EUSIPCO), 2016.
Abstract: Distributed optimization allows to address inference problems in a decentralized manner over networks, where agents can exchange information with their neighbors to improve their local estimates. Privacy preservation has become an important issue in many data mining applications. It aims at protecting the privacy of individual data in order to prevent the disclosure of sensitive information during the learning process. In this paper, we derive a diffusion strategy of the LMS type to solve distributed inference problems in the case where agents are also interested in preserving the privacy of the local measurements. We carry out a detailed mean and mean-square error analysis of the algorithm. Simulations are provided to check the theoretical findings.
BibTeX:
@inproceedings{haranne2016toward,
author = {Harrane, I. and Flamary, R. and Richard, C.},
title = {Toward privacy-preserving diffusion strategies for adaptation and learning over networks},
booktitle = {European Conference on Signal Processing (EUSIPCO)},
year = {2016}
}
D. Tuia, R. Flamary, M. Barlaud, "Non-convex regularization in remote sensing", Geoscience and Remote Sensing, IEEE Transactions on, 2016.
Abstract: In this paper, we study the effect of different regularizers and their implications in high dimensional image classification and sparse linear unmixing. Although kernelization or sparse methods are globally accepted solutions for processing data in high dimensions, we present here a study on the impact of the form of regularization used and its parametrization. We consider regularization via traditional squared (l2) and sparsity-promoting (l1) norms, as well as more unconventional nonconvex regularizers (lp and Log Sum Penalty). We compare their properties and advantages on several classification and linear unmixing tasks and provide advices on the choice of the best regularizer for the problem at hand. Finally, we also provide a fully functional toolbox for the community
BibTeX:
@article{tuia2016nonconvex,
author = {Tuia, D. and  Flamary, R. and Barlaud, M.},
title = {Non-convex regularization in remote sensing},
journal = {Geoscience and Remote Sensing, IEEE Transactions on},
year = {2016}
}
R. Flamary, A. Rakotomamonjy, G. Gasso, "Importance Sampling Strategy for Non-Convex Randomized Block-Coordinate Descent", IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), 2015.
Abstract: As the number of samples and dimensionality of optimization problems related to statistics and machine learning explode, block coordinate descent algorithms have gained popularity since they reduce the original problem to several smaller ones. Coordinates to be optimized are usually selected randomly according to a given probability distribution. We introduce an importance sampling strategy that helps randomized coordinate descent algorithms to focus on blocks that are still far from convergence. The framework applies to problems composed of the sum of two possibly non-convex terms, one being separable and non-smooth. We have compared our algorithm to a full gradient proximal approach as well as to a randomized block coordinate algorithm that considers uniform sampling and cyclic block coordinate descent. Experimental evidences show the clear benefit of using an importance sampling strategy.
BibTeX:
@inproceedings{flamary2015importance,
author = {Flamary, R. and Rakotomamonjy, A. and  Gasso, G.},
title = {Importance Sampling Strategy for Non-Convex Randomized Block-Coordinate Descent},
booktitle = {IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)},
year = {2015}
}

News

POT Python Optimal Transport library

2016-11-07

We proposed recently a general purpose Python library for Optimal Transport called POT. The library is available on Github and can be easily installed using PyPI. The toolbox implement a number of solvers from the image and machine learning literature (see README and the Documentation for more details).

We also give several examples of the potential uses of OT in the form of Python scripts and Python notebook that show the toolbox in use without requiring Python.

Here is a list of the Python notebooks if you want a quick look:

Feel free to use and contribute to the library.

Two papers in optimal transport accepted at NIPS 2016

2016-08-04

My collaborators and I have been accepted to present the following two papers at NIPS 2016

R. Flamary, C. Févotte, N. Courty, V. Emyia, "Optimal spectral transportation with application to music transcription", Neural Information Processing Systems (NIPS), 2016.

Abstract: Many spectral unmixing methods rely on the non-negative decomposition of spectral data onto a dictionary of spectral templates. In particular, state-of-the-art music transcription systems decompose the spectrogram of the input signal onto a dictionary of representative note spectra. The typical measures of fit used to quantify the adequacy of the decomposition compare the data and template entries frequency-wise. As such, small displacements of energy from a frequency bin to another as well as variations of timber can disproportionally harm the fit. We address these issues by means of optimal transportation and propose a new measure of fit that treats the frequency distributions of energy holistically as opposed to frequency-wise. Building on the harmonic nature of sound, the new measure is invariant to shifts of energy to harmonically-related frequencies, as well as to small and local displacements of energy. Equipped with this new measure of fit, the dictionary of note templates can be considerably simplified to a set of Dirac vectors located at the target fundamental frequencies (musical pitch values). This in turns gives ground to a very fast and simple decomposition algorithm that achieves state-of-the-art performance on real musical data.
BibTeX:
@inproceedings{flamary2016ost,
author = {Flamary, Remi and Févotte, Cédric and Courty, N. and  Emyia, Valentin},
title = {Optimal spectral transportation with application to music transcription},
booktitle = { Neural Information Processing Systems (NIPS)},
editor = {},
year = {2016}
} 

M. Perrot, N. Courty, R. Flamary, A. Habrard, "Mapping estimation for discrete optimal transport", Neural Information Processing Systems (NIPS), 2016.

Abstract: We are interested in the computation of the transport map of an Optimal Transport problem. Most of the computational approaches of Optimal Transport use the Kantorovich relaxation of the problem to learn a probabilistic coupling but do not address the problem of learning the transport map linked to the original Monge problem. Consequently, it lowers the potential usage of such methods in contexts where out-of-samples computations are mandatory. In this paper we propose a new way to jointly learn the coupling and an approximation of the transport map. We use a jointly convex formulation which can be efficiently optimized. Additionally, jointly learning the coupling and the transport map allows to smooth the result of the Optimal Transport and generalize it on out-of-samples examples. Empirically, we show the interest and the relevance of our method in two tasks: domain adaptation and image editing.
BibTeX:
@inproceedings{perrot2016mapping,
author = {Perrot, M. and Courty, N. and Flamary, R. and Habrard, A.},
title = {Mapping estimation for discrete optimal transport},
booktitle = {Neural Information Processing Systems (NIPS)},
editor = {},
year = {2016}
} 

Feel free to come and see us at our posters, we will have real life demonstrations of audio musical annotation and seamless copy in images.

Helava award of the best paper in ISPRS Journal period 2012-2015

2016-07-06

Our paper has been selected for the Helava Award, i.e. best paper in the ISPRS Journal of Photogrammetry and Remote Sensing for the 2012-2015 period.

D. Tuia, R. Flamary, N. Courty, "Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions", ISPRS Journal of Photogrammetry and Remote Sensing, 2015.

Abstract: In this paper, we tackle the question of discovering an effective set of spatial filters to solve hyperspectral classification problems. Instead of fixing a priori the filters and their parameters using expert knowledge, we let the model find them within random draws in the (possibly infinite) space of possible filters. We define an active set feature learner that includes in the model only features that improve the classifier. To this end, we consider a fast and linear classifier, multiclass logistic classification, and show that with a good representation (the filters discovered), such a simple classifier can reach at least state of the art performances. We apply the proposed active set learner in four hyperspectral image classification problems, including agricultural and urban classification at different resolutions, as well as multimodal data. We also propose a hierarchical setting, which allows to generate more complex banks of features that can better describe the nonlinearities present in the data.
BibTeX:
@article{tuia2015multiclass,
author = {Tuia, D. and Flamary, R. and  Courty, N.},
title = {Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
editor = {},
year = {2015}
} 

It is a great honor for us and I will be present at the ISPRS Congress 2016 on July 12 to receive the prize on behalf of all authors. This is a joint work with Devis Tuia and Nicolas Courty.