Collaborative Research: SCH: Geometry and Topology for Interpretable and Reliable Deep Learning in Medical Imaging

Award Number and Duration

NSF IIS 2205418 (University of Utah)

NSF IIS 2205417 (University of Virginia)

September 1, 2022 to August 31, 2026 (Estimated)

PI and Point of Contact

Bei Wang (Utah PI)
Associate Professor
School of Computing and Scientific Computing and Imaging Institute
University of Utah
beiwang AT sci.utah.edu
homepage

Tom Preston Fletcher (UVA PI)
Associate Professor
Department of Electrical and Computer Engineering and Department of Computer Science
Associate director, Center for Engineering in Medicine
University of Virgina
ptf8v AT virginia.edu
homepage
Companion collaborative project NSF IIS 2205417.

Collaborators

Jonathan C. Garneau, MD (UVA, Senior Personnel)
Assistant Professor
Division of Head and Neck Oncologic and Microvascular Surgery
University of Virgina
homepage

Overview

Deep learning models are being developed for safety-critical applications, such as health care, autonomous vehicles, and security. Their impressive performance has the potential to make profound impacts on human lives. For example, deep neural networks (DNNs) in medical imaging have been shown to have impressive diagnostic capabilities, often near that of expert radiologists. However, deep learning has not made it into standard clinical care, primarily due to a lack of understanding of why a model works and why it fails. The goal of this project is to develop methods for making machine learning models interpretable and reliable, and thus bridge the trust gap to make machine learning translatable to the clinic. This project achieves this goal through investigation of the mathematical foundations -- specifically the geometry and topology -- of DNNs. Based on these mathematical foundations, this project will develop computational tools that will improve the interpretability and reliability of DNNs. The methods developed in this project will be broadly applicable wherever deep learning is used, including health care, security, computer vision, natural language processing, etc.

The power of a deep neural network lies in its hidden layers, where the network learns internal representations of input data. This research project centers around the hypothesis that geometry and topology provide critical tools for analyzing the internal representations of DNNs. The first goal of this project is to develop a rigorous mathematical and algorithmic foundation for describing the geometry and topology of a neural network's internal representations and then design efficient algorithms for geometric and topological computations necessary to explore these spaces. The next aim of this project is to apply these tools to improve the interpretability of deep learning. This will be done by linking a model's internal representation with interpretable and trusted features and by interactive visualization that explores the landscape of a model's internal representation. The next goal of this project focuses on model reliability, where geometry and topology will be used for failure identification, mitigation, and prevention. Finally, this project will test the developed techniques for reliable and interpretable neural networks in a real-world setting to aid expert oncologists in predicting patient outcomes in head and neck cancers, e.g., whether a tumor will metastasize.

Publications and Manuscripts

Presentations, Educational Development and Broader Impacts

1. Bei Wang Keynote Talk, TDA Week, Japan, July 31 - August 4, 2023.

2. Bei Wang Invited Talk, International Forum at ChinaVis, July 21, 2023.

3. Tom Preston Fletcher Invited Talk, Institute for Mathematical and Statistical Innovation (IMSI), Workshop on Object Oriented Data Analysis in Health Sciences, July 10-14, 2023.

4. Bei Wang Invited Talk, Oxford Applied Topology Seminar, Centre for Topological Data Analysis (Oxford, Liverpool, and Durham), UK, June 16, 2023.

5. Bei Wang, Session Chairs, full paper session and Young Researchers Forum at the International Symposium on Computational Geometry (SOCG), June 10-15, 2023.

6. Workshop: Dagstuhl Seminar: Topological Data Analysis and Applications, May 7-12, 2023.
   Organizers: Bei Wang, Ulrich Bauer, Vijay Natarajan.

7. Bei Wang Invited Talk, Colorado State University Topology Seminar, April 18, 2023.

8. Bei Wang Invited Talk, Northeastern Topology Seminar, April 11, 2023.

9. Bei Wang Invited Talk, Institute for Mathematical and Statistical Innovation (IMSI), Randomness in Topology and its Applications workshop, March 21, 2023.

10. Bei Wang Keynote Talk, Machine Learning on Higher-Order Structured data (ML-HOS) Workshop at ICDM 2022. Hypergraph Co-Optimal Transport, November 28, 2022.

11. Bei Wang Invited Talk, Stochastic Seminar, Department of Mathematics, University of Utah, November 4, 2022.

12. Bei Wang Invited Talk, Mini Symposium on Statistics and Machine Learning in Topological and Geometric Data Analysis at SIAM Conference on Mathematics of Data Science (MDS22), September 29, 2022.

Students

Aman Shrivistava (CS PhD), University of Virginia.

Yinzhu Jin (CS PhD), University of Virginia.

Zhichao Xu (CS PhD, Spring 2023 - present), University of Utah.

Acknowledgement

This material is based upon work supported or partially supported by the National Science Foundation.

Any opinions, findings, and conclusions or recommendations expressed in this project are those of author(s) and do not necessarily reflect the views of the National Science Foundation.

Web page last update: July 5, 2023.