Figure 1: Sparse top-down information used to complement local boundary evidence.

Figure 1: Sparse top-down information used to complement local boundary evidence.

The Kreshuk group develops machine-learning-based methods and tools for automatic segmentation, classification and analysis of biological images.

Previous and current research

Machine learning is advancing the state-of-the-art in image analysis more rapidly than ever before: for many problems in natural image analysis, automated methods are now approaching parity with humans. One of the major advantages of learning-based approaches is their general applicability: tailoring to a particular problem is performed by providing suitable training data, while the core of the algorithm remains unchanged. To bring these methods to life scientists without computer vision expertise, we have developed a toolkit for interactive learning and segmentation (ilastik).

While the algorithms in ilastik generalise to provide user-friendly solutions for a wide array of image analysis problems, the most challenging bioimage datasets require a tailored approach. In particular, we have addressed the problem of automatic reconstruction of neural circuits from 3D electron microscopy data. The circuit reconstruction involves segmentation of neurons along with detection of synaptic contacts and prediction of their directions. Here, the most challenging aspect is the difference in scale: while single synapses are visible only at electron microscopy resolution, neurons have to be reconstructed across whole brain regions. Methods we have developed combine convolutional neural networks with post-processing by probabilistic graphical models to achieve state-of-the-art results for most neural segmentation challenges.

Future projects and goals

All machine learning algorithms require user guidance at the training stage, but deep learning – the driver of the current computer vision revolution – is even more annotation hungry. This problem is especially acute in biological imaging, where annotation of ground-truth data cannot easily be outsourced to non-experts, and changes in experimental conditions can require retraining. Besides the annotation burden, the training process itself depends upon non-trivial expertise in the choice and tuning of hyperparameters. Our group will develop new training strategies for convolutional neural networks which will allow for simple, ilastik-style interactive training on biological images. On the application side we plan to address the problem of integrating data from different imaging modalities, such as data from correlative microscopy experiments.