Feature-based multi-resolution registration of immunostained serial sections

Highlights

• We non-rigidly register immunohistological serial sections of human specimen.

• Using feature detection and matching we iteratively compute non-rigid deformations.

• Vascular networks in spleen and bone marrow are shown on a level not possible before.

• A quantitative evaluation of our method shows its efficiency.

Abstract

The form and exact function of the blood vessel network in some human organs, like spleen and bone marrow, are still open research questions in medicine. In this paper, we propose a method to register the immunohistological stainings of serial sections of spleen and bone marrow specimens to enable the visualization and visual inspection of blood vessels. As these vary much in caliber, from mesoscopic (millimeter-range) to microscopic (few micrometers, comparable to a single erythrocyte), we need to utilize a multi-resolution approach.

Our method is fully automatic; it is based on feature detection and sparse matching. We utilize a rigid alignment and then a non-rigid deformation, iteratively dealing with increasingly smaller features. Our tool pipeline can already deal with series of complete scans at extremely high resolution, up to 620 megapixels. The improvement presented increases the range of represented details up to smallest capillaries. This paper provides details on the multi-resolution non-rigid registration approach we use. Our application is novel in the way the alignment and subsequent deformations are computed (using features, i.e. “sparse”). The deformations are based on all images in the stack (“global”).

We also present volume renderings and a 3D reconstruction of the vascular network in human spleen and bone marrow on a level not possible before. Our registration makes easy tracking of even smallest blood vessels possible, thus granting experts a better comprehension.

A quantitative evaluation of our method and related state of the art approaches with seven different quality measures shows the efficiency of our method. We also provide z-profiles and enlarged volume renderings from three different registrations for visual inspection.

Introduction

On the mesoscopic scale, structures like blood vessels or nerves may span over millimeters in biological specimens and thus pose a significant problem for medical imaging. In this context large dimensions are as important as fine details. Such details can only be revealed by microscopy. We propose a “sparse” registration process to solve a fundamental problem concerning 3D reconstruction in microscopic anatomy on a mesoscopic scale. This problem is relevant in immunohistological investigations necessitating serial sections.

Immunohistological staining (immunostaining) is the technique of choice to discriminate cell types in human organ specimens. However, the staining does not penetrate specimens thicker than 50 µm. Such a thickness is also actually too large for transmitted light microscopy. With respect to human specimens, two choices are possible at this point: confocal microscopy or serial sections. Confocal microscopy has the limitations that it cannot be applied to thicker sections and that the sections cannot be stored for longer periods of time. The thickness of a single serial section is 5–10 µm. These sections can be easily stored, can be repeatedly inspected, and allow good penetration of the immunohistological staining solutions. The resolution of a common scanning microscope is 0.25–0.3 µm/pixel. The only drawback is that the limited information in a single section prevents a mesoscopic overview. However, with the modern advances in conventional scanning microscopes, series of high-resolution digital images consisting of up to several hundreds of single sections can be acquired. These images need to be registered. The special problem of this registration is the non-linear distortion of the sections. These distortions are inevitably associated with the sectioning process, because the embedding materials for the organ specimens are relatively soft. This happens in every immunohistological serial section and necessitates the matching of large and small landmarks. We use “sparse,” off-the-shelf feature detection combined with novel matching and warping in non-rigid transform instead of image-based methods and minimize the global distortion over the whole series of sections.

This work focuses on multi-resolution non-rigid registration of serial sections. “Multi-resolution” applies to the varying size of the features used, not to an image pyramid. The non-rigid registration is only a part of the larger production pipeline consisting of image acquisition, registration, segmentation, 3D reconstruction, and visualization.

We present an approach for a “sparse,” feature detection-based multi-resolution registration of microscopic images. Feature-based rigid registration does not pose a significant problem, but it is not sufficient for our purposes, as already detailed earlier (Ulrich et al., 2014). The same publication introduced a section-wide spline-based non-rigid registration that allowed to successfully register complete serial sections at their maximal available resolution.

However, further processing stages uncovered a central problem of “simple” non-rigid registration. Previously, we used only large features. Now, we iterate over multiple feature sizes in the non-rigid registration part. We register the ROIs after a coarse non-rigid registration using the method Ulrich et al. (2014) used for human spleen. We stress that using the “first stage” method alone, or even iterating it again on a ROI yields worse results than our new method. Section 5 details on this.

This paper presents the iterated method consisting of feature detection, matching, filtering, and “warping.” The latter is the process of adapting the B-spline control point grid for actual image (un-)distortion based on similarity of features with close spatial position. This is done at once for all images in the stack. The final step, applying the computed transformation to the actual images, is quite straightforward and is not regarded here in detail. We show the final result of our imaging pipeline in Fig. 9. We compare our approach to the state of the art (Ulrich, Lobachev, Steiniger, Guthe, 2014, Klein, Staring, Murphy, Viergever, Pluim, 2010) using visual methods like volume renderings and z-stacks (Fig. 7, Fig. 8, Fig. 9, Fig. 10, Fig. 11). Further, we quantitatively evaluate our method and related approaches using seven quality measures (Fig. 12, Fig. 13, Fig. 14, Fig. 15).

Investigating micro-vessel density and distribution is relevant in a wide range of experimental therapies, especially in tumor treatment. Fully-automatic image registration on mesoscopic and microscopic scale is instrumental in such research. The main contribution of our paper is the registration based on a pyramid of decreasing sizes of features with a gradually refined control point grid. The “search radius” for spacial location of the features also declines. The result is the non-rigid registration process that fits not only the large features, but also smaller ones. Our larger features are still too small to be the large landmarks, required for, e.g. the method of Bagci et al. (2012). Smaller features represent smaller blood vessels and capillaries, or other microscopic-scale entities of the serial sections. Our registration and undistortion of serial sections allow to answer current research questions like:

• “What is the branching pattern of larger blood vessels?”

• “Where is the capillary network localized and do regions without capillaries exist?”

• “Do special cells exist in the vicinity of the capillaries?”