T. Liu, C. Jones, M. Seyedhosseini, T. Tasdizen.
A modular hierarchical approach to 3D electron microscopy image segmentation, In Journal of Neuroscience Methods, Vol. 226, No. 15, pp. 88--102. April, 2014.
The study of neural circuit reconstruction, i.e., connectomics, is a challenging problem in neuroscience. Automated and semi-automated electron microscopy (EM) image analysis can be tremendously helpful for connectomics research. In this paper, we propose a fully automatic approach for intra-section segmentation and inter-section reconstruction of neurons using EM images. A hierarchical merge tree structure is built to represent multiple region hypotheses and supervised classification techniques are used to evaluate their potentials, based on which we resolve the merge tree with consistency constraints to acquire final intra-section segmentation. Then, we use a supervised learning based linking procedure for the inter-section neuron reconstruction. Also, we develop a semi-automatic method that utilizes the intermediate outputs of our automatic algorithm and achieves intra-segmentation with minimal user intervention. The experimental results show that our automatic method can achieve close-to-human intra-segmentation accuracy and state-of-the-art inter-section reconstruction accuracy. We also show that our semi-automatic method can further improve the intra-segmentation accuracy.
Keywords: Image segmentation, Electron microscopy, Hierarchical segmentation, Semi-automatic segmentation, Neuron reconstruction
A. J. Perez, M. Seyedhosseini, T. J. Deerinck, E. A. Bushong, S. Panda, T. Tasdizen, M. H. Ellisman.
A workflow for the automatic segmentation of organelles in electron microscopy image stacks, In Frontiers in Neuroanatomy, Vol. 8, No. 126, 2014.
Electron microscopy (EM) facilitates analysis of the form, distribution, and functional status of key organelle systems in various pathological processes, including those associated with neurodegenerative disease. Such EM data often provide important new insights into the underlying disease mechanisms. The development of more accurate and efficient methods to quantify changes in subcellular microanatomy has already proven key to understanding the pathogenesis of Parkinson's and Alzheimer's diseases, as well as glaucoma. While our ability to acquire large volumes of 3D EM data is progressing rapidly, more advanced analysis tools are needed to assist in measuring precise three-dimensional morphologies of organelles within data sets that can include hundreds to thousands of whole cells. Although new imaging instrument throughputs can exceed teravoxels of data per day, image segmentation and analysis remain significant bottlenecks to achieving quantitative descriptions of whole cell structural organellomes. Here, we present a novel method for the automatic segmentation of organelles in 3D EM image stacks. Segmentations are generated using only 2D image information, making the method suitable for anisotropic imaging techniques such as serial block-face scanning electron microscopy (SBEM). Additionally, no assumptions about 3D organelle morphology are made, ensuring the method can be easily expanded to any number of structurally and functionally diverse organelles. Following the presentation of our algorithm, we validate its performance by assessing the segmentation accuracy of different organelle targets in an example SBEM dataset and demonstrate that it can be efficiently parallelized on supercomputing resources, resulting in a dramatic reduction in runtime.
A. Perez, M. Seyedhosseini, T. Tasdizen, M. Ellisman.
Automated workflows for the morphological characterization of organelles in electron microscopy image stacks (LB72), In The FASEB Journal, Vol. 28, No. 1 Supplement LB72, April, 2014.
Advances in three-dimensional electron microscopy (EM) have facilitated the collection of image stacks with a field-of-view that is large enough to cover a significant percentage of anatomical subdivisions at nano-resolution. When coupled with enhanced staining protocols, such techniques produce data that can be mined to establish the morphologies of all organelles across hundreds of whole cells in their in situ environments. Although instrument throughputs are approaching terabytes of data per day, image segmentation and analysis remain significant bottlenecks in achieving quantitative descriptions of whole cell organellomes. Here we describe computational workflows that achieve the automatic segmentation of organelles from regions of the central nervous system by applying supervised machine learning algorithms to slices of serial block-face scanning EM (SBEM) datasets. We also demonstrate that our workflows can be parallelized on supercomputing resources, resulting in a dramatic reduction of their run times. These methods significantly expedite the development of anatomical models at the subcellular scale and facilitate the study of how these models may be perturbed following pathological insults.
M. Seyedhosseini, T. Tasdizen.
Disjunctive normal random forests, In Pattern Recognition, September, 2014.
We develop a novel supervised learning/classification method, called disjunctive normal random forest (DNRF). A DNRF is an ensemble of randomly trained disjunctive normal decision trees (DNDT). To construct a DNDT, we formulate each decision tree in the random forest as a disjunction of rules, which are conjunctions of Boolean functions. We then approximate this disjunction of conjunctions with a differentiable function and approach the learning process as a risk minimization problem that incorporates the classification error into a single global objective function. The minimization problem is solved using gradient descent. DNRFs are able to learn complex decision boundaries and achieve low generalization error. We present experimental results demonstrating the improved performance of DNDTs and DNRFs over conventional decision trees and random forests. We also show the superior performance of DNRFs over state-of-the-art classification methods on benchmark datasets.
Keywords: Random forest, Decision tree, Classifier, Supervised learning, Disjunctive normal form
M. Seyedhosseini, T. Tasdizen.
Scene Labeling with Contextual Hierarchical Models, In CoRR, Vol. abs/1402.0595, 2014.
Scene labeling is the problem of assigning an object label to each pixel. It unifies the image segmentation and object recognition problems. The importance of using contextual information in scene labeling frameworks has been widely realized in the field. We propose a contextual framework, called contextual hierarchical model (CHM), which learns contextual information in a hierarchical framework for scene labeling. At each level of the hierarchy, a classifier is trained based on downsampled input images and outputs of previous levels. Our model then incorporates the resulting multi-resolution contextual information into a classifier to segment the input image at original resolution. This training strategy allows for optimization of a joint posterior probability at multiple resolutions through the hierarchy. Contextual hierarchical model is purely based on the input image patches and does not make use of any fragments or shape examples. Hence, it is applicable to a variety of problems such as object segmentation and edge detection. We demonstrate that CHM outperforms state-of-the-art on Stanford background and Weizmann horse datasets. It also outperforms state-of-the-art edge detection methods on NYU depth dataset and achieves state-of-the-art on Berkeley segmentation dataset (BSDS 500).
T. Tasdizen, M. Seyedhosseini, T. Liu, C. Jones, E. Jurrus.
Image Segmentation for Connectomics Using Machine Learning, In Computational Intelligence in Biomedical Imaging, Edited by Suzuki, Kenji, Springer New York, pp. 237--278. 2014.
Reconstruction of neural circuits at the microscopic scale of individual neurons and synapses, also known as connectomics, is an important challenge for neuroscience. While an important motivation of connectomics is providing anatomical ground truth for neural circuit models, the ability to decipher neural wiring maps at the individual cell level is also important in studies of many neurodegenerative diseases. Reconstruction of a neural circuit at the individual neuron level requires the use of electron microscopy images due to their extremely high resolution. Computational challenges include pixel-by-pixel annotation of these images into classes such as cell membrane, mitochondria and synaptic vesicles and the segmentation of individual neurons. State-of-the-art image analysis solutions are still far from the accuracy and robustness of human vision and biologists are still limited to studying small neural circuits using mostly manual analysis. In this chapter, we describe our image analysis pipeline that makes use of novel supervised machine learning techniques to tackle this problem.
C. Jones, T. Liu, M. Ellisman, T. Tasdizen.
Semi-Automatic Neuron Segmentation in Electron Microscopy Images Via Sparse Labeling, In Proceedings of the 2013 IEEE 10th International Symposium on Biomedical Imaging (ISBI), pp. 1304--1307. April, 2013.
We introduce a novel method for utilizing user input to sparsely label membranes in electron microscopy images. Using gridlines as guides, the user marks where the guides cross the membrane to generate a sparsely labeled image. We use a best path algorithm to connect each of the sparse membrane labels. The resulting segmentation has a significantly better Rand error than automatic methods while requiring as little as 2\% of the image to be labeled.
C. Jones, M. Seyedhosseini, M. Ellisman, T. Tasdizen.
Neuron Segmentation in Electron Microscopy Images Using Partial Differential Equations, In Proceedings of 2013 IEEE 10th International Symposium on Biomedical Imaging (ISBI), pp. 1457--1460. April, 2013.
In connectomics, neuroscientists seek to identify the synaptic connections between neurons. Segmentation of cell membranes using supervised learning algorithms on electron microscopy images of brain tissue is often done to assist in this effort. Here we present a partial differential equation with a novel growth term to improve the results of a supervised learning algorithm. We also introduce a new method for representing the resulting image that allows for a more dynamic thresholding to further improve the result. Using these two processes we are able to close small to medium sized gaps in the cell membrane detection and improve the Rand error by as much as 9\% over the initial supervised segmentation.
T. Liu, M. Seyedhosseini, M. Ellisman, T. Tasdizen.
Watershed Merge Forest Classification for Electron Microscopy Image Stack Segmentation, In Proceedings of the 2013 International Conference on Image Processing, 2013.
Automated electron microscopy (EM) image analysis techniques can be tremendously helpful for connectomics research. In this paper, we extend our previous work  and propose a fully automatic method to utilize inter-section information for intra-section neuron segmentation of EM image stacks. A watershed merge forest is built via the watershed transform with each tree representing the region merging hierarchy of one 2D section in the stack. A section classifier is learned to identify the most likely region correspondence between adjacent sections. The inter-section information from such correspondence is incorporated to update the potentials of tree nodes. We resolve the merge forest using these potentials together with consistency constraints to acquire the final segmentation of the whole stack. We demonstrate that our method leads to notable segmentation accuracy improvement by experimenting with two types of EM image data sets.
M. Seyedhosseini, M. Ellisman, T. Tasdizen.
Segmentation of Mitochondria in Electron Microscopy Images using Algebraic Curves, In Proceedings of the 2013 IEEE 10th International Symposium on Biomedical Imaging (ISBI), pp. 860--863. 2013.
High-resolution microscopy techniques have been used to generate large volumes of data with enough details for understanding the complex structure of the nervous system. However, automatic techniques are required to segment cells and intracellular structures in these multi-terabyte datasets and make anatomical analysis possible on a large scale. We propose a fully automated method that exploits both shape information and regional statistics to segment irregularly shaped intracellular structures such as mitochondria in electron microscopy (EM) images. The main idea is to use algebraic curves to extract shape features together with texture features from image patches. Then, these powerful features are used to learn a random forest classifier, which can predict mitochondria locations precisely. Finally, the algebraic curves together with regional information are used to segment the mitochondria at the predicted locations. We demonstrate that our method outperforms the state-of-the-art algorithms in segmentation of mitochondria in EM images.
M. Seyedhosseini, T. Tasdizen.
Multi-class Multi-scale Series Contextual Model for Image Segmentation, In IEEE Transactions on Image Processing, Vol. PP, No. 99, 2013.
Contextual information has been widely used as a rich source of information to segment multiple objects in an image. A contextual model utilizes the relationships between the objects in a scene to facilitate object detection and segmentation. However, using contextual information from different objects in an effective way for object segmentation remains a difficult problem. In this paper, we introduce a novel framework, called multi-class multi-scale (MCMS) series contextual model, which uses contextual information from multiple objects and at different scales for learning discriminative models in a supervised setting. The MCMS model incorporates cross-object and inter-object information into one probabilistic framework and thus is able to capture geometrical relationships and dependencies among multiple objects in addition to local information from each single object present in an image. We demonstrate that our MCMS model improves object segmentation performance in electron microscopy images and provides a coherent segmentation of multiple objects. By speeding up the segmentation process, the proposed method will allow neurobiologists to move beyond individual specimens and analyze populations paving the way for understanding neurodegenerative diseases at the microscopic level.
M. Seyedhosseini, M. Sajjadi, T. Tasdizen.
Image Segmentation with Cascaded Hierarchical Models and Logistic Disjunctive Normal Networks, In Proceedings of the IEEE International Conference on Computer Vison (ICCV 2013), pp. (accepted). 2013.
Contextual information plays an important role in solving vision problems such as image segmentation. However, extracting contextual information and using it in an effective way remains a difficult problem. To address this challenge, we propose a multi-resolution contextual framework, called cascaded hierarchical model (CHM), which learns contextual information in a hierarchical framework for image segmentation. At each level of the hierarchy, a classifier is trained based on downsampled input images and outputs of previous levels. Our model then incorporates the resulting multi-resolution contextual information into a classifier to segment the input image at original resolution. We repeat this procedure by cascading the hierarchical framework to improve the segmentation accuracy. Multiple classifiers are learned in the CHM; therefore, a fast and accurate classifier is required to make the training tractable. The classifier also needs to be robust against overfitting due to the large number of parameters learned during training. We introduce a novel classification scheme, called logistic disjunctive normal networks (LDNN), which consists of one adaptive layer of feature detectors implemented by logistic sigmoid functions followed by two fixed layers of logical units that compute conjunctions and disjunctions, respectively. We demonstrate that LDNN outperforms state-of-theart classifiers and can be used in the CHM to improve object segmentation performance.
E.V.R. DiBella, T. Tasdizen.
Edge enhanced spatio-temporal constrained reconstruction of undersampled dynamic contrast enhanced radial MRI, In Proceedings of the 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 704--707. 2012.
There are many applications in MRI where it is desirable to have high spatial and high temporal resolution. This can be achieved by undersampling of k-space and requires special techniques for reconstruction. Even if undersampling artifacts are removed, sharpness of the edges can be a problem. We propose a new technique that uses the gradient from a reference image to improve the quality of the edges in the reconstructed image along with a spatio-temporal constraint to reduce aliasing artifacts and noise. The reference is created from undersampled dynamic data by combining several adjacent frames. The method was tested on undersampled radial DCE MRI data with little respiratory motion. The proposed method was compared to reconstruction using the spatio-temporal constrained reconstruction. Sharper edges and an increase in the contrast was observed by using the proposed method.
L. Hogrebe, A.R.C. Paiva, E. Jurrus, C. Christensen, M. Bridge, L. Dai, R.L. Pfeiffer, P.R. Hof, B. Roysam, J.R. Korenberg, T. Tasdizen.
Serial section registration of axonal confocal microscopy datasets for long-range neural circuit reconstruction, In Journal of Neuroscience Methods, Vol. 207, No. 2, pp. 200--210. 2012.
In the context of long-range digital neural circuit reconstruction, this paper investigates an approach for registering axons across histological serial sections. Tracing distinctly labeled axons over large distances allows neuroscientists to study very explicit relationships between the brain's complex interconnects and, for example, diseases or aberrant development. Large scale histological analysis requires, however, that the tissue be cut into sections. In immunohistochemical studies thin sections are easily distorted due to the cutting, preparation, and slide mounting processes. In this work we target the registration of thin serial sections containing axons. Sections are first traced to extract axon centerlines, and these traces are used to define registration landmarks where they intersect section boundaries. The trace data also provides distinguishing information regarding an axon's size and orientation within a section. We propose the use of these features when pairing axons across sections in addition to utilizing the spatial relationships among the landmarks. The global rotation and translation of an unregistered section are accounted for using a random sample consensus (RANSAC) based technique. An iterative nonrigid refinement process using B-spline warping is then used to reconnect axons and produce the sought after connectivity information.
S.K. Iyer, T. Tasdizen, E.V.R. DiBella.
Edge Enhanced Spatio-Temporal Constrained Reconstruction of Undersampled Dynamic Contrast Enhanced Radial MRI, In Magnetic Resonance Imaging, Vol. 30, No. 5, pp. 610--619. 2012.
Dynamic contrast-enhanced magnetic resonance imaging (MRI) is a technique used to study and track contrast kinetics in an area of interest in the body over time. Reconstruction of images with high contrast and sharp edges from undersampled data is a challenge. While good results have been reported using a radial acquisition and a spatiotemporal constrained reconstruction (STCR) method, we propose improvements from using spatially adaptive weighting and an additional edge-based constraint. The new method uses intensity gradients from a sliding window reference image to improve the sharpness of edges in the reconstructed image. The method was tested on eight radial cardiac perfusion data sets with 24 rays and compared to the STCR method. The reconstructions showed that the new method, termed edge-enhanced spatiotemporal constrained reconstruction, was able to reconstruct images with sharper edges, and there were a 36\%±13.7\% increase in contrast-to-noise ratio and a 24\%±11\% increase in contrast near the edges when compared to STCR. The novelty of this paper is the combination of spatially adaptive weighting for spatial total variation (TV) constraint along with a gradient matching term to improve the sharpness of edges. The edge map from a reference image allows the reconstruction to trade-off between TV and edge enhancement, depending on the spatially varying weighting provided by the edge map.
Keywords: MRI, Reconstruction, Edge enhanced, Compressed sensing, Regularization, Cardiac perfusion
E. Jurrus, S. Watanabe, R.J. Giuly, A.R.C. Paiva, M.H. Ellisman, E.M. Jorgensen, T. Tasdizen.
Semi-Automated Neuron Boundary Detection and Nonbranching Process Segmentation in Electron Microscopy Images, In Neuroinformatics, pp. (published online). 2012.
Neuroscientists are developing new imaging techniques and generating large volumes of data in an effort to understand the complex structure of the nervous system. The complexity and size of this data makes human interpretation a labor-intensive task. To aid in the analysis, new segmentation techniques for identifying neurons in these feature rich datasets are required. This paper presents a method for neuron boundary detection and nonbranching process segmentation in electron microscopy images and visualizing them in three dimensions. It combines both automated segmentation techniques with a graphical user interface for correction of mistakes in the automated process. The automated process first uses machine learning and image processing techniques to identify neuron membranes that deliniate the cells in each two-dimensional section. To segment nonbranching processes, the cell regions in each two-dimensional section are connected in 3D using correlation of regions between sections. The combination of this method with a graphical user interface specially designed for this purpose, enables users to quickly segment cellular processes in large volumes.
T. Liu, E. Jurrus, M. Seyedhosseini, M. Ellisman, T. Tasdizen.
Watershed Merge Tree Classification for Electron Microscopy Image Segmentation, In Proceedings of the 21st International Conference on Pattern Recognition (ICPR), pp. 133--137. 2012.
Automated segmentation of electron microscopy (EM) images is a challenging problem. In this paper, we present a novel method that utilizes a hierarchical structure and boundary classification for 2D neuron segmentation. With a membrane detection probability map, a watershed merge tree is built for the representation of hierarchical region merging from the watershed algorithm. A boundary classifier is learned with non-local image features to predict each potential merge in the tree, upon which merge decisions are made with consistency constraints to acquire the final segmentation. Independent of classifiers and decision strategies, our approach proposes a general framework for efficient hierarchical segmentation with statistical learning. We demonstrate that our method leads to a substantial improvement in segmentation accuracy.
A.R.C. Paiva, T. Tasdizen.
Fingerprint Image Segmentation using Data Manifold Characteristic Features, In International Journal of Pattern Recognition and Artificial Intelligence, Vol. 26, No. 4, pp. (23 pages). 2012.
Automatic fingerprint identification systems (AFIS) have been studied extensively and are widely used for biometric identification. Given its importance, many well-engineered methods have been developed for the different stages that encompass those systems. The first stage of any such system is the segmentation of the actual fingerprint region from the background. This is typically achieved by classifying pixels, or blocks of pixels, based on a set of features. In this paper, we describe novel features for fingerprint segmentation that express the underlying manifold topology associated with image patches in a local neighborhood. It is shown that fingerprint patches seen in a high-dimensional space form a simple and highly regular circular manifold. The characterization of the manifold topology suggests a set of optimal features that characterize the local properties of the fingerprint. Thus, fingerprint segmentation can be formulated as a classification problem based on the deviation from the expected topology. This leads to features that are more robust to changes in contrast than mean, variance and coherence. The superior performance of the proposed features for fingerprint segmentation is shown in the eight datasets from the 2002 and 2004 Fingerprint Verification Competitions.
Keywords: Fingerprint segmentation, manifold characterization, feature extraction, dimensionality reduction
N. Ramesh, B. J. Dangott, M. Salama, T. Tasdizen.
Segmentation and Two-Step Classification of White Blood Cells in Peripheral Blood Smear, In Journal of Pathology Informatics, Vol. 3, No. 13, 2012.
An automated system for differential white blood cell (WBC) counting based on morphology can make manual differential leukocyte counts faster and less tedious for pathologists and laboratory professionals. We present an automated system for isolation and classification of WBCs in manually prepared, Wright stained, peripheral blood smears from whole slide images (WSI). Methods: A simple, classification scheme using color information and morphology is proposed. The performance of the algorithm was evaluated by comparing our proposed method with a hematopathologist's visual classification. The isolation algorithm was applied to 1938 subimages of WBCs, 1804 of them were accurately isolated. Then, as the first step of a two-step classification process, WBCs were broadly classified into cells with segmented nuclei and cells with nonsegmented nuclei. The nucleus shape is one of the key factors in deciding how to classify WBCs. Ambiguities associated with connected nuclear lobes are resolved by detecting maximum curvature points and partitioning them using geometric rules. The second step is to define a set of features using the information from the cytoplasm and nuclear regions to classify WBCs using linear discriminant analysis. This two-step classification approach stratifies normal WBC types accurately from a whole slide image. Results: System evaluation is performed using a 10-fold cross-validation technique. Confusion matrix of the classifier is presented to evaluate the accuracy for each type of WBC detection. Experiments show that the two-step classification implemented achieves a 93.9\% overall accuracy in the five subtype classification. Conclusion: Our methodology achieves a semiautomated system for the detection and classification of normal WBCs from scanned WSI. Further studies will be focused on detecting and segmenting abnormal WBCs, comparison of 20x and 40x data, and expanding the applications for bone marrow aspirates.
N. Ramesh, M.E. Salama, T. Tasdizen.
Segmentation of Haematopoeitic Cells in Bone Marrow Using Circle Detection and Splitting Techniques, In 9th IEEE International Symposium on Biomedical Imaging (ISBI), pp. 206--209. 2012.
Bone marrow evaluation is indicated when peripheral blood abnormalities are not explained by clinical, physical, or laboratory findings. In this paper, we propose a novel method for segmentation of haematopoietic cells in the bone marrow from scanned slide images. Segmentation of clumped cells is a challenging problem for this application. We first use color information and morphology to eliminate red blood cells and the background. Clumped haematopoietic cells are then segmented using circle detection and a splitting algorithm based on the detected circle centers. The Hough Transform is used for circle detection and to find the number and positions of circle centers in each region. The splitting algorithm is based on detecting the maximum curvature points, and partitioning them based on information obtained from the centers of the circles in each region. The performance of the segmentation algorithm for haematopoietic cells is evaluated by comparing our proposed method with a hematologist's visual segmentation in a set of 3748 cells.