Medical Image Analysis - Articles in Press

Understanding the Phase Contrast Optics to Restore Artifact-free Microscopy Images for Segmentation - Accepted Manuscript

Zhaozheng Yin, Takeo Kanade, Mei Chen — 2012-02-06

Graphical abstractHighlights: ► Understand the phase contrast optics and its artifacts. ► Derive the computational imaging model for phase contrast optics. ► Design an effective algorithm to restore artifact-free phase contrast images. ► Facilitate high-performance microscopy image analysis such as cell segmentation.Abstract: Phase contrast, a noninvasive microscopy imaging technique, is widely used to capture time-lapse images to monitor the behavior of transparent cells without staining or altering them. Due to the optical principle, phase contrast microscopy images contain artifacts such as the halo and shade-off that hinder image segmentation, a critical step in automated microscopy image analysis. Rather than treating phase contrast microscopy images as general natural images and applying generic image processing techniques on them, we propose to study the optical properties of the phase contrast microscope to model its image formation process. The phase contrast imaging system can be approximated by a linear imaging model. Based on this model and input image properties, we formulate a regularized quadratic cost function to restore artifact-free phase contrast images that directly correspond to the specimen’s optical path length. With artifacts removed, high quality segmentation can be achieved by simply thresholding the restored images. The imaging model and restoration method are quantitatively evaluated on microscopy image sequences with thousands of cells captured over several days. We also demonstrate that accurate restoration lays the foundation for high performance in cell detection and tracking.

Brain Tissue Segmentation in MR Images based on a Hybrid of MRF and Social Algorithms - Accepted Manuscript

Sahar Yousefi, Reza Azmi, Morteza Zahedi — 2012-02-02

Graphical abstract: Effective abnormality detection and diagnosis in Magnetic Resonance Images (MRIs) requires a robust segmentation strategy. Since manual segmentation is a so time-consuming task which engages valuable human resources, automatic MRI segmentation have been received an enormous amount of attention. For this purpose various techniques have been applied however Markov Random Field (MRF) based algorithms have produced better results in noisy images compared to other methods. MRF seeks for a label field which minimizes energy function. Traditional minimization method, simulated annealing (SA), uses Monte Carlo simulation to access the minimum solution with heavy computation burden. For this reason MRFs never be used in real time processing environments. This paper proposed a novel method based on MRF and hybrid of social algorithms contains ant colony optimization (ACO) and Gossiping algorithm. Combining ACO with Gossiping algorithm assists ants to find the better path using the information of their neighbors. Therefore, this interaction causes the algorithm converges to optimum solution faster. Several experiments on phantom and real images were performed. Results indicate the proposed algorithm outperforms the traditional MRF in speed and accuracy.Highlights: ► This article presents a novel unsupervised MRF-based model for MRI segmentation. ► The proposed method uses ACO and gossiping algorithm to speed up the classic MRF. ► Tackling Gossiping algorithm, the proposed method assists ants in smart decision. ► The IBSR and Brainweb datasets have been used for testing the method. ► Results. ► demonstrate the novel method outperforms other models in speed and quality.Abstract: Effective abnormality detection and diagnosis in Magnetic Resonance Images (MRIs) requires a robust segmentation strategy. Since manual segmentation is a time-consuming task which engages valuable human resources, automatic MRI segmentations received an enormous amount of attention. For this goal, various techniques have been applied. However, Markov Random Field (MRF) based algorithms have produced reasonable results in noisy images compared to other methods. MRF seeks a label field which minimizes an energy function. The traditional minimization method, simulated annealing (SA), uses Monte Carlo simulation to access the minimum solution with heavy computation burden. For this reason, MRFs are rarely used in real time processing environments. This paper proposed a novel method based on MRF and a hybrid of social algorithms that contain an ant colony optimization (ACO) and a Gossiping algorithm which can be used for segmenting single and multispectral MRIs in real time environments. Combining ACO with the Gossiping algorithm helps find the better path using neighborhood information. Therefore, this interaction causes the algorithm to converge to an optimum solution faster. Several experiments on phantom and real images were performed. Results indicate that the proposed algorithm outperforms the traditional MRF and hybrid of MRF-ACO in speed and accuracy.

Nonparametric Bayesian inference of the fiber orientation distribution from diffusion-weighted MR images - Accepted Manuscript

Enrico Kaden, Frithjof Kruggel — 2012-02-02

Graphical abstract: Bayesian density estimation allows us to disentangle three crossing fiber bundles in the centrum semiovale.Highlights: ► The spherical convolution model is reformulated by adopting a measure-theoretic approach. ► The fiber orientation density is represented by a Dirichlet process mixture, which may be used for realistic simulations. ► The inverse problem is solved by nonparametric Bayesian statistics under a Rician noise model. ► The density estimation framework is demonstrated with HARDI datasets.Abstract: Diffusion MR imaging provides a unique tool to probe the microgeometry of nervous tissue and to explore the wiring diagram of the neural connections noninvasively. Generally, a forward model is established to map the intra-voxel fiber architecture onto the observable diffusion signals, which is reformulated in this article by adopting a measure-theoretic approach. However, the inverse problem, i.e., the spherical deconvolution of the fiber orientation density from noisy MR measurements, is ill-posed. We propose a nonparametric representation of the tangential distribution of the nerve fibers in terms of a Dirichlet process mixture. Given a second-order approximation of the impulse response of a fiber segment, the specified problem is solved by Bayesian statistics under a Rician noise model, using an adaptive reversible jump Markov chain Monte Carlo sampler. The density estimation framework is demonstrated by various experiments with a diffusion MR dataset featuring high angular resolution, uncovering the fiber orientation field in the cerebral white matter of the living human brain.

A Probabilistic Framework for Image Information Fusion with An Application to Mammographic Analysis - Accepted Manuscript

Marina Velikova, Peter J.F. Lucas, Maurice Samulski, Nico Karssemeijer — 2012-01-27

Graphical abstract: Highlights: ► We develop a multi-stage probabilistic framework for multi-view image analysis. ► The framework is computationally efficient and quite general to apply to any domain. ► A sound unbiased evaluation procedure for multi-stage models is proposed. ► Comparison with previous mammographic CAD systems show improved performance.Abstract: The recent increased interest in information fusion methods for solving complex problem, such as in image analysis, is motivated by the wish to better exploit the multitude of information, available from different sources, to enhance decision-making. In this paper, we propose a novel method, that advances the state of the art of fusing image information from different views, based on a special class of probabilistic graphical models, called causal independence models. The strength of this method is its ability to systematically and naturally capture uncertain domain knowledge, while performing information fusion in a computationally efficient way. We examine the value of the method for mammographic analysis and demonstrate its advantages in terms of explicit knowledge representation and accuracy (increase of at least 6.3% and 5.2% of true positive detection rates at 5% and 10% false positive rates) in comparison with previous single-view and multi-view systems, and benchmark fusion methods such as naïve Bayes and logistic regression.

A CANDLE for a deeper in vivo insight - Uncorrected Proof

2012-01-20

Graphical abstract: Optic chiasm between 275μm and 320μmHighlights: ► A new method for the denoising of confocal and multiphoton images is proposed. ► An extensive validation demonstrates the high denoising performance of our filter. ► The deeper in vivo imaging capabilities enabled by our filter are presented.Abstract: A new Collaborative Approach for eNhanced Denoising under Low-light Excitation (CANDLE) is introduced for the processing of 3D laser scanning multiphoton microscopy images. CANDLE is designed to be robust for low signal-to-noise ratio (SNR) conditions typically encountered when imaging deep in scattering biological specimens. Based on an optimized non-local means filter involving the comparison of filtered patches, CANDLE locally adapts the amount of smoothing in order to deal with the noise inhomogeneity inherent to laser scanning fluorescence microscopy images. An extensive validation on synthetic data, images acquired on microspheres and in vivo images is presented. These experiments show that the CANDLE filter obtained competitive results compared to a state-of-the-art method and a locally adaptive optimized non-local means filter, especially under low SNR conditions (PSNR<8dB). Finally, the deeper imaging capabilities enabled by the proposed filter are demonstrated on deep tissue in vivo images of neurons and fine axonal processes in the Xenopus tadpole brain.

Mass preserving image registration for lung CT - Uncorrected Proof

2012-01-16

Graphical abstract: Improvement of mass preserving image registration for four datasets with increasing differences in lung volume.Highlights: ► Preservation of mass is a plausible model of lung parenchyma. ► Mass preserving model simulates lung tissue density change related to the change in regional ventilation. ► Incorporating mass preserving model into an image registration leads to more accurate results.Abstract: This paper presents a mass preserving image registration algorithm for lung CT images. To account for the local change in lung tissue intensity during the breathing cycle, a tissue appearance model based on the principle of preservation of total lung mass is proposed. This model is incorporated into a standard image registration framework with a composition of a global affine and several free-form B-Spline transformations with increasing grid resolution. The proposed mass preserving registration method is compared to registration using the sum of squared intensity differences as a similarity function on four groups of data: 44 pairs of longitudinal inspiratory chest CT scans with small difference in lung volume; 44 pairs of longitudinal inspiratory chest CT scans with large difference in lung volume; 16 pairs of expiratory and inspiratory CT scans; and 5 pairs of images extracted at end exhale and end inhale phases of 4D-CT images. Registration errors, measured as the average distance between vessel tree centerlines in the matched images, are significantly lower for the proposed mass preserving image registration method in the second, third and fourth group, while there is no statistically significant difference between the two methods in the first group. Target registration error, assessed via a set of manually annotated landmarks in the last group, was significantly smaller for the proposed registration method.

Cardiac MR perfusion image processing techniques: A survey - Corrected Proof

2012-01-11

Graphical abstract: Highlights: ► Survey of semi- and fully automatic image processing methods for cardiac MR perfusion quantification. ► Classification of image processing methods based on their registration, segmentation and multimodality fusion and visualization algorithms. ► Investigation of the advantages and drawbacks of the surveyed methods and their robustness to acquisition artifacts. ► Extensive literature overview of the last 20 years.Abstract: First-pass cardiac MR perfusion (CMRP) imaging has undergone rapid technical advancements in recent years. Although the efficacy of CMRP imaging in the assessment of coronary artery diseases (CAD) has been proven, its clinical use is still limited. This limitation stems, in part, from manual interaction required to quantitatively analyze the large amount of data. This process is tedious, time-consuming, and prone to operator bias. Furthermore, acquisition and patient related image artifacts reduce the accuracy of quantitative perfusion assessment. With the advent of semi- and fully automatic image processing methods, not only the challenges posed by these artifacts have been overcome to a large extent, but a significant reduction has also been achieved in analysis time and operator bias. Despite an extensive literature on such image processing methods, to date, no survey has been performed to discuss this dynamic field. The purpose of this article is to provide an overview of the current state of the field with a categorical study, along with a future perspective on the clinical acceptance of image processing methods in the diagnosis of CAD.

A fast analysis method for non-invasive imaging of blood flow in individual cerebral arteries using vessel-encoded arterial spin labeling angiography - Accepted Manuscript

2011-12-29

Graphical abstract: Highlights: ► Non-invasive vessel selective MR angiography. ► Fast Bayesian analysis for artery flow contributions. ► Robust treatment of imperfect artery location specification e.g. due to patient movement. ► Robust to clinical scenarios e.g. occluded arteries.Abstract: Arterial spin labeling (ASL) MRI offers a non-invasive means to create blood-borne contrast in vivo for dynamic angiographic imaging. By spatial modulation of the ASL process it is possible to uniquely label individual arteries over a series of measurements, allowing each to be separately identified in the resulting angiographic images. This separation requires appropriate analysis for which a general Bayesian framework has previously been proposed. Here this framework is adapted for clinical dynamic angiographic imaging. This specifically addresses the issues of computational speed of the algorithm and the robustness required to deal with real patient data. An algorithm is proposed that can incorporate planning information about the arteries being imaged whilst adapting for subsequent patient movement. A fast maximum a priori solution is adopted and shown to be only marginally less accurate than Monte Carlo sampling under simulation. The final algorithm is demonstrated on in vivo data with analysis on a time scale of the order of 10 minutes, from both a healthy control and a patient with a vertebro-basilar occlusion.

Nonlinear dimensionality reduction combining MR imaging with non-imaging information - Corrected Proof

2011-12-26

Graphical abstract: The top figure shows a 2D manifold embedding of images acquired from subjects with AD (squares) and healthy controls (circles) using pairwise image similarities with Laplacian Eigenmaps. The bottom figure shows an embedding obtained with the proposed method incorporating metadata (color-coded) into the manifold learning process. A better separation between the two groups can be obtained when considering both measures. Misclassified subjects are marked by a black outline.Highlights: ► We propose an extension to nonlinear dimensionality reduction with Laplacian Eigenmaps. ► MR imaging information is combined with clinical metadata. ► The representation of subjects in the low-dimensional space is interpreted as biomarker. ► We use this biomarker to classify AD subjects from healthy controls. ► We evaluate the method on the ADNI study.Abstract: We propose a framework for the extraction of biomarkers from low-dimensional manifolds representing inter-subject brain variation. Manifold coordinates of each image capture information about structural shape and appearance and, when a phenotype exists, about the subject’s clinical state. Our framework incorporates subject meta-information into the manifold learning step. Apart from gender and age, information such as genotype or a derived biomarker is often available in clinical studies and can inform the classification of a query subject. Such information, whether discrete or continuous, is used as an additional input to manifold learning, extending the Laplacian Eigenmap objective function and enriching a similarity measure derived from pairwise image similarities. The biomarkers identified with the proposed method are data-driven in contrast to a priori defined biomarkers derived from, e.g., manual or automated segmentations. They form a unified representation of both the imaging and non-imaging measurements, providing a natural use for data analysis and visualization. We test the method to classify subjects with Alzheimer’s Disease (AD), mild cognitive impairment (MCI) and healthy controls enrolled in the ADNI study. Non-imaging metadata used are ApoE genotype, a risk factor associated with AD, and the CSF-concentration of , an established biomarker for AD. In addition, we use hippocampal volume as a derived imaging-biomarker to enrich the learned manifold. Our classification results compare favorably to what has been reported in a recent meta-analysis using established neuroimaging methods on the same database.

3D Segmentation of SBFSEM Images of Neuropil by a Graphical Model over Supervoxel Boundaries - Accepted Manuscript

2011-12-21

Graphical abstract: Highlights: ► We propose a supervoxel merging method for segmenting 3D SBFSEM images of neuropil. ► All merging decisions are coupled in a graphical model. ► Knowledge about correct segments is incorporated in higher-order potentials. ► These potentials are learned automatically from training data. ► The scope of this improvement is demonstrated on the benchmark dataset E1088.Abstract: The segmentation of large volume images of neuropil acquired by serial sectioning electron microscopy is an important step towards the 3D reconstruction of neural circuits. The only cue provided by the data at hand are boundaries between otherwise indistinguishable objects. This indistinguishability, combined with the boundaries becoming very thin or faint in places, makes the large body of work on region based segmentation methods inapplicable. On the other hand, boundary-based methods that exploit purely local evidence do not reach the extremely high accuracy required by the application domain that cannot tolerate the global topological errors arising from false local decisions.As a consequence, we propose a supervoxel merging method which arrives at itsdecisions in a non-local fashion, by posing and approximately solving ajoint combinatorial optimization problem over all faces between supervoxels. The use of supervoxels allows the extraction of expressive geometric features. These are used by the higher-order potentials in a graphical model that assimilate knowledge about the geometry of neural surfaces by automated training on a gold standard.The scope of this improvement is demonstrated on the benchmark dataset E1088 (Helmstaedter et al.,2011) of 7.5 billion voxels from the inner plexiform layer of rabbit retina. We provide C++ source code for annotation, geometry extraction, training and inference.

Manifold learning for image-based breathing gating in ultrasound and MRI - Corrected Proof

Christian Wachinger, Mehmet Yigitsoy, Erik-Jan Rijkhorst, Nassir Navab — 2011-12-09

Graphical abstract: Highlights: ► Automatic, image-based gating in ultrasound and MRI. ► Application of Laplacian eigenmaps for gating. ► Creation of 4D ultrasound data with wobbler transducer. ► Validation by comparison to alternative gating approaches.Abstract: Respiratory motion is a challenging factor for image acquisition and image-guided procedures in the abdominal and thoracic region. In order to address the issues arising from respiratory motion, it is often necessary to detect the respiratory signal. In this article, we propose a novel, purely image-based retrospective respiratory gating method for ultrasound and MRI. Further, we apply this technique to acquire breathing-affected 4D ultrasound with a wobbler probe and, similarly, to create 4D MR with a slice stacking approach. We achieve the gating with Laplacian eigenmaps, a manifold learning technique, to determine the low-dimensional manifold embedded in the high-dimensional image space. Since Laplacian eigenmaps assign to each image frame a coordinate in low-dimensional space by respecting the neighborhood relationship, they are well suited for analyzing the breathing cycle. We perform the image-based gating on several 2D and 3D ultrasound datasets over time, and quantify its very good performance by comparing it to measurements from an external gating system. For MRI, we perform the manifold learning on several datasets for various orientations and positions. We achieve very high correlations by a comparison to an alternative gating with diaphragm tracking.

Multi-modal registration of speckle-tracked freehand 3D ultrasound to CT in the lumbar spine - Corrected Proof

Andrew Lang, Parvin Mousavi, Sean Gill, Gabor Fichtinger, Purang Abolmaesumi — 2011-08-29

Graphical abstract: A method for registration of speckle-tracked freehand 3D ultrasound to preoperative CT volumes of the spine is proposed. The method corrects the significant drift error in speckle-tracked freehand ultrasound volume reconstruction. Results of phantoms and lamb cadaver experiments demonstrate registration errors consistently below 3mm.Highlights: ► A method for registration of speckle-tracked 3D ultrasound to CT is proposed. ► Registration is also used to reduce drift error in speckle-tracked 3D ultrasound. ► Phantom and ex vivo results show reduction in registration error by over 50%. ► Consistent registration error below 3mm is achieved.Abstract: A method for registration of speckle-tracked freehand 3D ultrasound (US) to preoperative CT volumes of the spine is proposed. We register the US volume to the CT volume by creating individual US “sub-volumes”, each consisting of a small section of the entire US volume. The registration proceeds incrementally from the beginning of the US volume to the end, registering every sub-volume, where each sub-volume contains overlapping images with the previous sub-volume. Each registration is performed by generating simulated US images from the CT volume. As a by-product of our registration, the significant drift error common in speckle-tracked US volumes is corrected for. Results are validated through a phantom study of plastic spine phantoms created from clinical patient CT data as well as an animal study using a lamb cadaver. Results demonstrate that we were able to successfully register a speckle-tracked US volume to CT volume in four out of five phantoms with a success rate of greater than 98%. The final error of the registered US volumes decreases by over 50 percent from the speckle tracking error to consistently below 3mm. Studies on the lamb cadaver showed a mean registration error consistently below 2mm.

Prostate biopsy tracking with deformation estimation - Corrected Proof

Michael Baumann, Pierre Mozer, Vincent Daanen, Jocelyne Troccaz — 2011-05-18

Graphical abstract: Prostate biopsy core and cancer grade map realized using 3D US image registration based prostate biopsy core tracking with deformation estimation.Highlights: ► Prostate biopsy core localization with deformation estimation. ► US image registration based prostate tracking. ► Prostate biopsy core and cancer maps. ► Targeting suspicious lesions in MR images during TRUS prostate biopsies.Abstract: Transrectal biopsies under 2D ultrasound (US) control are the current clinical standard for prostate cancer diagnosis. The isoechogenic nature of prostate carcinoma makes it necessary to sample the gland systematically, resulting in a low sensitivity. Also, it is difficult for the clinician to follow the sampling protocol accurately under 2D US control and the exact anatomical location of the biopsy cores is unknown after the intervention. Tracking systems for prostate biopsies make it possible to generate biopsy distribution maps for intra- and post-interventional quality control and 3D visualisation of histological results for diagnosis and treatment planning. They can also guide the clinician toward non-ultrasound targets. In this paper, a volume-swept 3D US based tracking system for fast and accurate estimation of prostate tissue motion is proposed. The entirely image-based system solves the patient motion problem with an a priori model of rectal probe kinematics. Prostate deformations are estimated with elastic registration to maximize accuracy. The system is robust with only 17 registration failures out of 786 (2%) biopsy volumes acquired from 47 patients during biopsy sessions. Accuracy was evaluated to 0.76±0.52mm using manually segmented fiducials on 687 registered volumes stemming from 40 patients. A clinical protocol for assisted biopsy acquisition was designed and implemented as a biopsy assistance system, which allows to overcome the draw-backs of the standard biopsy procedure.

Context specific descriptors for tracking deforming tissue - Corrected Proof

Peter Mountney, Guang-Zhong Yang — 2011-05-16

Graphical abstract: An adaptive intra-operative tissue deformation tracking framework based on context specific descriptors incorporating both geometric and appearance information that is robust to drift, occlusion, artefacts and changes in orientation and scale.Highlights: ► A tracking framework is proposed which uses context specific information. ► Tissue tracking is posed as a classification problem. ► Training data is generated online and with geometric and appearance models. ► Classifier training is improved using log likelihood ratios. ► It outperforms four existing trackers and is robust to deformation, rotation, scale and smoke.Abstract: In minimally invasive surgery, deployment of motion compensation, dynamic active constraints and adaptive intra-operative guidance require accurate estimation of deforming tissue in 3D. To this end, the use of vision-based techniques is advantageous in that it does not require the integration of additional hardware to the existing surgical settings. Deformation can be recovered by tracking features on the surface of the tissue. Existing methods are mostly based on ad hoc machine vision techniques that have generally been developed for rigid scenes or use pre-determined models with parameters fine tuned to specific surgical settings. In this work, we propose a novel tracking technique based on a context specific feature descriptor. The descriptor can adapt to its surroundings and identify the most discriminate information for tracking. The feature descriptor is represented as a decision tree and the tracking process is formulated as a classification problem for which log likelihood ratios are used to improve classifier training. A recursive tracking algorithm obtains examples of tissue deformation used to train the classifier. Additional training data is generated by geometric and appearance modelling. Experimental results have shown that the proposed context specific descriptor is robust to drift, occlusion, and changes in orientation and scale. The performance of the algorithm is compared with existing tracking algorithms and validated with both simulated and in vivo datasets.

Development and comparison of new hybrid motion tracking for bronchoscopic navigation - Corrected Proof

2010-12-14

Graphical abstract: The proposed hybrid motion prediction method comprises main three stages: feature-based motion prediction, Kalman filtering-based scale factor estimation, and intensity-based image registration. Inputs to the tracking method are bronchoscopic videos and CT datasets. Outputs of the tracking method are a series of the bronchoscope pose parameters (camera trajectories) or virtual bronchoscopic images corresponding to real bronchoscopic frames.Research highlights: ► Development of navigated bronchoscopy for diagnosis and treatment of lung cancer. ► Synchronization between 3D pre-built model and 2D bronchoscopic camera views. ► A new hybrid tracking method bases on SIFT, Kalman filter, and image registration. ► Our approach provides a more accurate and robustness bronchoscope tracking.Abstract: This paper presents a new hybrid camera motion tracking method for bronchoscopic navigation combining SIFT, epipolar geometry analysis, Kalman filtering, and image registration. In a thorough evaluation, we compare it to state-of-the-art tracking methods. Our hybrid algorithm for predicting bronchoscope motion uses SIFT features and epipolar constraints to obtain an estimate for inter-frame pose displacements and Kalman filtering to find an estimate for the magnitude of the motion. We then execute bronchoscope tracking by performing image registration initialized by these estimates. This procedure registers the actual bronchoscopic video and the virtual camera images generated from 3D chest CT data taken prior to bronchoscopic examination for continuous bronchoscopic navigation. A comparative assessment of our new method and the state-of-the-art methods is performed on actual patient data and phantom data. Experimental results from both datasets demonstrate a significant performance boost of navigation using our new method. Our hybrid method is a promising means for bronchoscope tracking, and outperforms other methods based solely on Kalman filtering or image features and image registration.

MR to ultrasound registration for image-guided prostate interventions - Corrected Proof

2010-12-14

Graphical abstract: Deformable registration between MR and ultrasound images of prostate using a statistical motion model and a novel probabilistic model-to-image registration algorithm.Research highlights: ► Ultrasound-probe-induced prostate motion is an important source of registration error. ► A biomechanically-informed statistical shape model constrains allowed deformations. ► Surface normal alignment is a robust and efficient approach to image registration. ► Deformable registration enables fast and accurate data fusion during prostate interventions.Abstract: A deformable registration method is described that enables automatic alignment of magnetic resonance (MR) and 3D transrectal ultrasound (TRUS) images of the prostate gland. The method employs a novel “model-to-image” registration approach in which a deformable model of the gland surface, derived from an MR image, is registered automatically to a TRUS volume by maximising the likelihood of a particular model shape given a voxel-intensity-based feature that represents an estimate of surface normal vectors at the boundary of the gland. The deformation of the surface model is constrained by a patient-specific statistical model of gland deformation, which is trained using data provided by biomechanical simulations. Each simulation predicts the motion of a volumetric finite element mesh due to the random placement of a TRUS probe in the rectum. The use of biomechanical modelling in this way also allows a dense displacement field to be calculated within the prostate, which is then used to non-rigidly warp the MR image to match the TRUS image. Using data acquired from eight patients, and anatomical landmarks to quantify the registration accuracy, the median final RMS target registration error after performing 100 MR–TRUS registrations for each patient was 2.40mm.

Reconstruction of a 3D surface from video that is robust to missing data and outliers: Application to minimally invasive surgery using stereo and mono endoscopes - Corrected Proof

2010-12-13

Graphical abstract: 3D structure is recovered from a moving endoscope in the presence of image noise, missing data and even outliers. It can provide a wide field-of-view and be used for 3D–3D registration of the anatomy to preoperative imaging data for use in image guided minimally invasive surgery.Research highlights: ► 3D structure is recovered from an endoscope and can provide a wide field-of-view. ► A robust strategy based on the trifocal tensor is proposed to remove outliers. ► Evolutionary agent algorithm is applied to improve the robustness of data filling.Abstract: Minimally invasive surgery (MIS) offers great benefits to patients compared with open surgery. Nevertheless during MIS surgeons often need to contend with a narrow field-of-view of the endoscope and obstruction from other surgical instruments. He/she may also need to relate the surgical scene to information derived from previously acquired 3D medical imaging. We thus present a new framework to reconstruct the 3D surface of an internal organ from endoscopic images which is robust to measurement noise, missing data and outliers. This can provide 3D surface with a wide field-of-view for surgeons, and it can also be used for 3D–3D registration of the anatomy to pre-operative CT/MRI data for use in image guided interventions. Our proposed method first removes most of the outliers using an outlier removal method that is based on the trilinear constraints over three images. Then data that are missing from one or more of the video images (missing data) and 3D structure are recovered using the structure from motion (SFM) technique. Evolutionary agents are applied to improve both the efficiency of data recovery and robustness to outliers. Furthermore, an incremental bundle adjustment strategy is used to refine the camera parameters and 3D structure and produce a more accurate 3D surface. Experimental results with synthetic data show that the method is able to reconstruct surfaces in the presence of feature tracking errors (up to 5 pixel standard deviation) and a large amount of missing data (up to 50%). Experiments on a realistic phantom model and in vivo data further demonstrate the good performance of the proposed approach in terms of accuracy (1.7mm residual phantom surface error) and robustness (50% missing data rate, and 20% outliers in in vivo experiments).

Statistical modeling and recognition of surgical workflow - Corrected Proof

2010-12-09

Graphical abstract: .Research highlights: ► Statistical workflow modeling from demonstrated surgeries. ► DTW/HMM based methods for on-line and off-line surew representation of interventions in terms of multidimensional time-series formed by synchronized signals acquired over time. We then introduce methods based on Dynamic Time Warping and Hidden Markov Models to analyze and process this data. This results in workflow models combining low-level signals with high-level information such as predefined phases, which can be used to detect actions and trigger an event. Two methods are presented to train these models, using either fully or partially labeled training surgeries. Results are given based on tool usage recordings from sixteen laparoscopic cholecystectomies performed by several surgeons.

An integrated diagnosis and therapeutic system using intra-operative 5-aminolevulinic-acid-induced fluorescence guided robotic laser ablation for precision neurosurgery - Corrected Proof

2010-11-29

Graphical abstract: Research highlights: ► Development of an integrated diagnosis and therapeutic system for precision malignant gliomas resection. ► Concept proof of the integration of diagnostic and therapeutic techniques. ► 5-Aminolevulinic-acid (5-ALA)-induced fluorescence based intra-operative automatic tumor diagnosis technique. ► High-precision robotic laser ablation therapeutic technique with automatic focusing and robotic scanning mechanism. ► Improved accuracy of the fluorescent measurement of the tumor using high-precision spectral analysis.Abstract: We have developed an integrated diagnosis and therapeutic system for precision malignant gliomas resection during neurosurgery. A combination of three-dimensional (3-D) magnetic resonance imaging (MRI) navigation and 5-aminolevulinic acid (5-ALA)-induced fluorescence based intra-operative tumor diagnosis technique has been incorporated into a robotic laser ablation neurosurgery system with an automatic focusing and robotic scanning mechanism. 5-ALA is a non-fluorescent prodrug that leads to intracellular accumulation of fluorescent protoporphyrins IX (PpIX) in malignant glioma. The PpIX tends to accumulate in pathological lesions, and emits red fluorescence when excited by blue light. This fluorescence is illuminated with laser excitation, enables intra-operative identification of the position of a tumor and provides guidance for resection with laser photocoagulation. The information provided by the MRI is enhanced by the intra-operative 5-ALA fluorescence data, and this enhanced information is integrated into a robotic laser ablation system. The accuracy of the fluorescent measurement of the tumor is improved using high-precision spectral analysis. The fluorescence assists in the detection of malignant brain tumors intra-operatively and improves their removal rate.

Computer assisted reconstruction of complex proximal humerus fractures for preoperative planning - Corrected Proof

2010-09-30

Graphical abstract: Research highlights: ► Semi-automatic fracture reconstruction in two consecutive registration steps. ► Efficient contralateral matching without relying on initial fragment positions. ► Robust pairwise registration of fracture surfaces and global multipiece alignment. ► The complex planning task can be performed in a reasonable time on the GPU.Abstract: Operative treatment of displaced fractures of the proximal humerus is among the most difficult problems in orthopedic shoulder surgery. An accurate preoperative assessment of fragment displacement is crucial for a successful joint restoration. We present a computer assisted approach to precisely quantify these displacements. The bone is virtually reconstructed by multi-fragment alignment. In case of largely displaced pieces, a reconstruction template based on the contralateral humerus is incorporated in the algorithm to determine the optimal assembly. Cadaver experiments were carried out to evaluate our approach. All cases could be successfully reconstructed with little user interaction, and only requiring a few minutes of processing time. On average, the reassembled bone geometries resulted in a translational displacement error of 1.3±0.4mm and a rotational error of 3.4±2.2°, respectively.

Intra-operative 3D guidance and edema detection in prostate brachytherapy using a non-isocentric C-arm - Corrected Proof

2010-08-16

Graphical abstract: Research highlights: ► The proposed system is the first of a kind that makes intra-operative detection of edema possible. ► It achieves a significantly more homogeneous seed distribution at low cost. ► The approach has the potential to affect a paradigm shift in clinical practice.Abstract: Purpose: Brachytherapy (radioactive seed insertion) has emerged as one of the most effective treatment options for patients with prostate cancer, with the added benefit of a convenient outpatient procedure. The main limitation in contemporary brachytherapy is faulty seed placement, predominantly due to the presence of intra-operative edema (tissue expansion). Though currently not available, the capability to intra-operatively monitor the seed distribution, can make a significant improvement in cancer control. We present such a system here.Methods: Intra-operative measurement of edema in prostate brachytherapy requires localization of inserted radioactive seeds relative to the prostate. Seeds were reconstructed using a typical non-isocentric C-arm, and exported to a commercial brachytherapy treatment planning system. Technical obstacles for 3D reconstruction on a non-isocentric C-arm include pose-dependent C-arm calibration; distortion correction; pose estimation of C-arm images; seed reconstruction; and C-arm to TRUS registration.Results: In precision-machined hard phantoms with 40–100 seeds and soft tissue phantoms with 45–87 seeds, we correctly reconstructed the seed implant shape with an average 3D precision of 0.35mm and 0.24mm, respectively. In a DoD Phase-1 clinical trial on six patients with 48–82 planned seeds, we achieved intra-operative monitoring of seed distribution and dosimetry, correcting for dose inhomogeneities by inserting an average of over four additional seeds in the six enrolled patients (minimum 1; maximum 9). Additionally, in each patient, the system automatically detected intra-operative seed migration induced due to edema (mean 3.84mm, STD 2.13mm, Max 16.19mm).Conclusions: The proposed system is the first of a kind that makes intra-operative detection of edema (and subsequent re-optimization) possible on any typical non-isocentric C-arm, at negligible additional cost to the existing clinical installation. It achieves a significantly more homogeneous seed distribution, and has the potential to affect a paradigm shift in clinical practice. Large scale studies and commercialization are currently underway.

Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine - Corrected Proof

2010-08-05

Abstract: We present a groupwise US to CT registration algorithm for guiding percutaneous spinal interventions. In addition, we introduce a comprehensive validation scheme that accounts for changes in the curvature of the spine between preoperative and intraoperative imaging. In our registration methodology, each vertebra in CT is treated as a sub-volume and transformed individually. A biomechanical model is used to constrain the displacement of the vertebrae relative to one another. The sub-volumes are then reconstructed into a single volume. During each iteration of registration, an US image is simulated from the reconstructed CT volume and an intensity-based similarity metric is calculated with the real US image. Validation studies are performed on CT and US images from a sheep cadaver, five patient-based phantoms designed to preserve realistic curvatures of the spine and a sixth patient-based phantom where the curvature of the spine is changed between preoperative and intraoperative imaging.For datasets where the spine curve between two imaging modalities was artificially perturbed, the proposed methodology was able to register initial misalignments of up to 20mm with a success rate of 95%. For the phantom with a physical change in the curvature of the spine introduced between the US and CT datasets, the registration success rate was 98.5%. Finally, the registration success rate for the sheep cadaver with soft-tissue information was 87%. The results demonstrate that our algorithm allows for robust registration of US and CT datasets, regardless of a change in the patients pose between preoperative and intraoperative image acquisitions.

Gaze-Contingent Motor Channelling, haptic constraints and associated cognitive demand for robotic MIS - Corrected Proof

2010-08-02

Graphical abstract: Research highlights: ► Human visual system coordinates are directly transformed into motor coordinates. ► Gaze-Contingent Motor Channelling improves motor targeting and task performance. ► Gaze-Contingent Haptic Constraints for dynamic setting and updating of haptic constraints. ► fNIRS striking differences between expert/non-expert during Gaze-Contingent manipulation.Abstract: The success of MIS is coupled with an increasing demand on surgeons’ manual dexterity and visuomotor coordination due to the complexity of instrument manipulations. The use of master–slave surgical robots has avoided many of the drawbacks of MIS, but at the same time, has increased the physical separation between the surgeon and the patient. Tissue deformation combined with restricted workspace and visibility of an already cluttered environment can raise critical issues related to surgical precision and safety. Reconnecting the essential visuomotor sensory feedback is important for the safe practice of robot-assisted MIS procedures. This paper introduces a novel gaze-contingent framework for real-time haptic feedback and virtual fixtures by transforming visual sensory information into physical constraints that can interact with the motor sensory channel. We demonstrate how motor tracking of deforming tissue can be made more effective and accurate through the concept of Gaze-Contingent Motor Channelling. The method is also extended to 3D by introducing the concept of Gaze-Contingent Haptic Constraints where eye gaze is used to dynamically prescribe and update safety boundaries during robot-assisted MIS without prior knowledge of the soft-tissue morphology. Initial validation results on both simulated and robot assisted phantom procedures demonstrate the potential clinical value of the technique. In order to assess the associated cognitive demand of the proposed concepts, functional Near-Infrared Spectroscopy is used and preliminary results are discussed.

CranialVault and its CRAVE tools: A clinical computer assistance system for deep brain stimulation (DBS) therapy - Corrected Proof

2010-08-02

Abstract: A number of methods have been developed to assist surgeons at various stages of deep brain stimulation (DBS) therapy. These include construction of anatomical atlases, functional databases, and electrophysiological atlases and maps. But, a complete system that can be integrated into the clinical workflow has not been developed. In this paper we present a system designed to assist physicians in pre-operative target planning, intra-operative target refinement and implantation, and post-operative DBS lead programming. The purpose of this system is to centralize the data acquired a the various stages of the procedure, reduce the amount of time needed at each stage of the therapy, and maximize the efficiency of the entire process. The system consists of a central repository (CranialVault), of a suite of software modules called CRAnialVault Explorer (CRAVE) that permit data entry and data visualization at each stage of the therapy, and of a series of algorithms that permit the automatic processing of the data. The central repository contains image data for more than 400 patients with the related pre-operative plans and position of the final implants and about 10,550 electrophysiological data points (micro-electrode recordings or responses to stimulations) recorded from 222 of these patients. The system has reached the stage of a clinical prototype that is being evaluated clinically at our institution. A preliminary quantitative validation of the planning component of the system performed on 80 patients who underwent the procedure between January 2009 and December 2009 shows that the system provides both timely and valuable information.

Robotic Tissue Tracking for Beating Heart Mitral Valve Surgery - Accepted Manuscript

Shelten G. Yuen, Nikolay V. Vasilyev, Pedro J. del Nido, Robert D. Howe — 2010-07-08

Abstract: The rapid motion of the heart presents a significant challenge to the surgeon during intracardiac beating heart procedures. We present a 3D ultrasound-guided motion compensation system that assists the surgeon by synchronizing instrument motion with the heart. The system utilizes the fact that certain intracardiac structures, like the mitral valve annulus, have trajectories that are largely constrained to translation along one axis. This allows the development of a real-time 3D ultrasound tissue tracker that we integrate with a 1 degree-of-freedom (DOF) actuated surgical instrument and predictive filter to devise a motion tracking system adapted to mitral valve annuloplasty. In experiments demonstrate that the system provides highly accurate tracking (1.0 mm error) with 70% less error than manual tracking attempts.

Fast virtual deployment of self-expandable stents: Method and in vitro evaluation for intracranial aneurysmal stenting - Corrected Proof

2010-05-12

Abstract: Introduction: Minimally invasive treatment approaches, like the implantation of percutaneous stents, are becoming more popular every day for the treatment of intracranial aneurysms. The outcome of such treatments is related to factors like vessel and aneurysm geometry, hemodynamic conditions and device design. For this reason, having a tool for assessing stenting alternatives beforehand is crucial.Methodology: The Fast Virtual Stenting (FVS) method, which provides an estimation of the configuration of intracranial stents when released in realistic geometries, is proposed in this paper. This method is based on constrained simplex deformable models. The constraints are used to account for the stent design. An algorithm for its computational implementation is also proposed. The performance of the proposed methodology was contrasted with real stents released in a silicone phantom.Results: In vitro experiments were performed on the phantom where a contrast injection was performed. Subsequently, corresponding Computational Fluid Dynamics (CFD) analyzes were carried out on a digital replica of the phantom with the virtually released stent. Virtual angiographies are used to compare in vitro experiments and CFD analysis. Contrast time–density curves for in vitro and CFD data were generated and used to compare them.Conclusions: Results of both experiments resemble very well, especially when comparing the contrast density curves. The use of FVS methodology in the clinical environment could provide additional information to clinicians before the treatment to choose the therapy that best fits the patient.

A review of 3D/2D registration methods for image-guided interventions - Corrected Proof

P. Markelj, D. Tomaževič, B. Likar, F. Pernuš — 2010-04-14

Abstract: Registration of pre- and intra-interventional data is one of the key technologies for image-guided radiation therapy, radiosurgery, minimally invasive surgery, endoscopy, and interventional radiology. In this paper, we survey those 3D/2D data registration methods that utilize 3D computer tomography or magnetic resonance images as the pre-interventional data and 2D X-ray projection images as the intra-interventional data. The 3D/2D registration methods are reviewed with respect to image modality, image dimensionality, registration basis, geometric transformation, user interaction, optimization procedure, subject, and object of registration.