Remote sensing data processing has long been conducted at pixel level, since times when objects of interest were way smaller or at most comparable to a pixel size. The significant developments on the spatial resolution front have led to the emergence of object-based image analysis, that no longer handles every pixel independently, but in contextual groups. In particular, multiscale models such as hierarchical representations (or trees) have been proposed and widely acknowledged as the appropriate solution since they enable us to model efficiently the relations between different image objects at multiple detail levels. In fact, depending on the required level of analysis, it is no longer uncommon for image regions to require multiple labels; e.g. a region might be identified as a road at a fine scale, as a residential area at an intermediate scale, or even as town or city at a coarse scale. These labels might be known a priori (supervised classification) or not (unsupervised classification or segmentation). While hierarchical representations inherently encode structural and rich information at several scales, its processing is merely a succession of monoscale analysis, either in an unsupervised fashion where the objective is to produce segmentations, or in a supervised one where classification maps with a set of labels (or a taxonomy of labels) are produced. No scientific solution allows for now a multiscale classification of the multiscale representation.
In the same time, in the machine learning community has emerged new tools to deal with hierarchical representations but they have barely been exploited in the remote sensing community. Structured prediction deals with the prediction of a structured output rather than a single label, which is relevant is our case as the structure of the data inherently defines a semantic taxonomy of labels. While supervised classification of flat labels (i.e. labels with no dependence between each others) is a marked up research field, hierarchical classification deals with labels organized in a hierarchy. This paradigm suits particularly to multiscale remote sensing data as it allows one to fully take advantage of the inherent hierarchical nature of the representation. Nevertheless, there exists no method that enables the labeling of the nodes of a tree with a hierarchy of labels. Few examples on the literature consider the classification of entire trees by a hierarchy of labels but no individual nodes. Another family of methods focuses to the classification of nodes of the tree, spreading labels from nodes to other nodes. This latter problem is called theNode Classification or Graph Labeling and arises naturally in many real-life problems that have inherently a network structure, but they do not consider any structured prediction.
The objective of the PhD is to define a new formulation of the structured classification of structured data problem, in which the labels are themselves structured. Two directions will be explored:
- the prerequisite of classification is the ability to measure in a relevant way the similarity between two nodes or (pieces of) trees. Optimal transport (OT) have inspired a number of recent breakthroughs in machine learning because of its capacity to compare efficiently empirical distributions. The lead of relying on an OT based-formulation of a distance, that deals inherently with structured or hierarchical outputs, shall be explored.
- there is nowadays no way around learning end-to-end solutions that rely on nodes or graph convolution. Solutions that rely on convolutional networks shall be considered.
The solutions shall be able to be embedded in either a supervised (where some labels are available -- also known as the node classification problem) or unsupervised (as in the community clustering solutions) approach. A particular emphasis will be put on the development of efficient solutions, able to deal with large datasets.
From an application point of view, a special emphasis will be given on remote sensing datasets, but the solution should be applicable to other application domains. Targeted publications will be conferences and journals from both the machine learning and the remote sensing communities.