An Optimal Transport Perspective on the Approximation of Mixture Densities

Many real world problems deal with data generated from sources of which laws are usually unknown. In order to make predictions or to infer the corresponding behavior, suitable probabilistic models are often sought to characterize such systems. Nonetheless, when the underlying nature is complex, simple models might result to be unsatisfactory, and it is necessary to resort to more descriptive forms; an efficient, yet powerful, alternative is to consider mixture models which, by combining simpler elements, allow to characterize particularly complex behaviors. However, when the number of mixture parameters becomes significantly large, such models can become computationally intractable and approximations have to be introduced. The goal of this dissertation is to address the mixture reduction problem in an \textit{Occam's razor} perspective, that is by finding good trade-offs between representation complexity and accuracy. The proposed methodology is general, but, for the goals of this work, the focus will be posed on target tracking in clutter problems, where the optimal Bayesian recursion leads to an unbounded increase in the number of mixture components, making corresponding algorithms intractable, and on clustering problems, especially in high dimensional settings, where finding a suitable number of representatives is a non-trivial task. Given a mixture model with many components, the corresponding reduction problem consists in finding another mixture, possessing a significantly less components, which is similar, in some sense, to the original one. The first step to address this problem is to have a measure of how dissimilar two mixtures are; in the literature many dissimilarity measures have been proposed, each of which possessing its own features, but a discussion regarding the corresponding peculiarities has been rarely addressed. Moreover, many of those dissimilarities are analytically intractable when applied to mixtures, leading to the reasonable approach of resorting either to tractable approximations, or to employ an ensemble of measures in the same reduction pipeline in order to ease the computations. In this work, a mixture reduction scheme is defined consistent when the same dissimilarity measure is employed for each of the steps involved in the process; in this regard, by exploiting the Optimal Transport theory, it is possible to formulate a consistent reduction framework which is also capable, in a specific case, to deal with the corresponding model selection problem, that is to provide automatically a suitable number of components for the reduced order model; choosing the order of the reduced model can be an impactful choice, since overly simple approximations can lead to a large bias of the representations, and overly complex alternatives can be computationally burdensome. In this dissertation, the optimal transport theory will serve as a systematic approach to solve all-round the mixture reduction problem, by providing all the tools necessary to reduce and refine a given mixture model.