\documentclass[12pt,twocolumn,fullpage]{article}
\usepackage[usenames]{color}
\usepackage{fullpage}
\usepackage{graphicx}
\usepackage{url}
\title{Volume Visualization}

% highlight things that we need to revisit
\newcommand{\todo}[1]{\textcolor{red}{\textit{#1}}}
% how we are naming the DBPs
\newcommand{\WinslowDBP}[0]{\textit{Winslow}~}
\newcommand{\EMDBP}[0]{\textit{EM}~}

\begin{document}

\section{Large-Scale Data Visualization}

Data being acquired and refined by a group at Johns Hopkins University
(JHU) creates a new analysis and visualization challenge that many in
the biomedical domain are experiencing.  They have developed a complex,
novel process based on 2-photon microscopy which can image organs at
unprecedented levels of detail, and are using it to investigate cardiac
conditions and treatments, such as cardiac ablation.
Figure~\ref{fig:jhu-correction} shows a couple examples of the
resolutions obtained from their imaging process, along with some
post-processing we have done in an effort to draw out some of the
features.  However, due to the intricacy of the operation, it takes
almost 3 weeks to go from a specimen to a complete, imaged heart from
which they can gain insight.  A full week of this process is dedicated
to processing and filtering the data to reconstruct a single coherent
volume from a multitude of independent samples.  Worse, the data are
so large that most existing visualization software is incapable of
visualizing these data.

The groups standing solution was---after all the aforementioned effort
to acquire these high-resolution scans---to downsample the data into a
size which is amenable to current visualization software.  They turned
to us when they heard about our ImageVis3D tool, whose out-of-core
processing capabilities decouple data size from available memory,
allowing terabytes of data to be visualized on a single
workstation~\cite{Fogal:2010:Tuvok}.  Unfortunately our initial forays
into volume visualization with these data exhibit artifacts which
inhibit one's ability to pull out
features of interest, as demonstrated by Figure~\ref{fig:jhu-diagonal}.

The current volume reconstruction process is based on the algorithm of
Gopinath et al.~\cite{Gopinath:2008:Reconstruction}, using local window
information from the scans so that the solution remains computationally
tractable.  Our collaborators note
\begin{quote}
  Our application to a range of 70 $\mu$m and the inherently large
  light scattering effect of cardiac tissue being imaged in our work
  created a more significant attenuation than originally considered in
  the design of [Gopinath et al.\cite{Gopinath:2008:Reconstruction}].
\end{quote}
In context, portions of the heart are shaved off and the remaining
heart volume is imaged.  As a thick volume, energy is absorbed
when it hits the tissue, but it also penetrates the volume and is
absorbed and scattered at varying rates, based on the type of cells
which it intersects.  As such, the effect is one of a gradual as
opposed to immediate attenuation, and features much larger than the
desired slice thickness will accumulate in a single imaging.  The
reconstruction process aims to minimize this effect, however as noted
by the collaborator above, this process is leaves much to be desired.

The group's data issues are representative of a growing trend in
large-scale biomedical data acquisition: large-scale acquisition
requires computationally intense reconstruction processes which can
take as much time as the initial scanning process.  For many, the
subsequent analysis processes take more time than the data acquisition
itself, and thus they become the bottleneck in developing and testing
biomedical hypotheses.

Increasingly, this and subsequent visualization software must be
specifically architected to handle the scale of the data acquired.

\section{Volume Visualization}

We propose the research and development of biomedical and data analysis
software which can help to alleviate the data-intensive science
problems that groups like the one at JHU are experiencing.  As leaders
in the field of ray-guided volume
visualization~\cite{Hadwiger:2012:Guided, Fogal:2013:Analysis}, we are
well-poised to tackle this challenge.

Ray-guided volume visualization has the potential to starkly reduce the
amount
of data which must be processed~\cite{Crassin:2009:Gigavoxels,
Hadwiger:2012:Guided, Fogal:2013:Analysis} to create a visualization.
In 2012, Hadwiger, Pfister, et al. demonstrated that analysis can be
fruitfully
applied \emph{during} rendering, if appropriately guided by what the
user needs to see.  Figure~\ref{fig:vishuman-debug} illustrates this
concept: here, the 13 gigabyte Visible Human male has less than 500
megabytes of data visible at any one time.  By explicitly considering
the viewport we are outputting to, data can be reduced substantially,
as shown in
Figure~\ref{fig:rmi-resolution}.

We intend to include visualization much earlier in the processing
pipeline.  By using the visual analysis step to guide the
reconstruction, we can produce tools which are able to provide a visual
representation of the data \emph{as} scanning proceeds, due to the data
reduction implied by these ray-guided techniques.

\section{Volume Rendering}
\label{sec:volren}

Data sizes have been rising for years, and the prevalence of new
scanning technologies has quickened this pace.  While traditional
scanning technologies such as CT and MRI continue to improve their
effective resolution, the largest data explosion has occured with
newer technologies such as electromagnetic imaging.  These new data
acquisition strategies, such as those employed in the \WinslowDBP and \EMDBP
driving biological problems (DBPs), are generating tens-of-gigabytes to
terabytes of data, presenting massive data processing, data management,
and visualization challenges.  Unfortunately, this growth in data
has outpaced even the largest workstations, necessitating the use of
distributed memory, multiresolution, out-of-core, or a combination of
these processing techniques.

\begin{figure}
  \centering
  \includegraphics[width=\linewidth]{images/vishuman-debug}
  %\includegraphics[width=\linewidth]{images/draft}

  \caption{Per-ray behavior while rendering a large biomedical data
  set.  Green represents space which is skipped.  In red regions, dense
  sampling was required. Blue areas indicate that a ray terminated very
  quickly.  Most of the rendering is blue or green, indicating that
  very little work needed to be done.}
  \label{fig:vishuman-debug}
\end{figure}

Volume visualization is no exception to this trend.  State of the art
volume rendering techniques rely on what we call \textit{ray-guided}
techniques.  The idea behind such an approach is that data sampling
occurs \emph{during} rendering, which ensures that sampling is
restricted to parts of the volume which will be visible in the final
image~\cite{Crassin:2009:Gigavoxels, Hadwiger:2012:Guided,
Fogal:2013:Analysis}.  Taking the renderer's sampling needs into
account has a profound impact on the amount of data needed to produce
a rendering, as demonstrated in Figure~\ref{fig:vishuman-debug}.
For Big Data, this consideration can make or break an interactive
visualization system.

\begin{figure}
  \centering
  \includegraphics[width=\linewidth]{images/rmi-resolution-bricks}

  \caption{Amount of data needed, per-frame, to generate volume
  renderings of a Richtmyer-Meshkov instability.  Output-sensitive
  adaptive sampling can significantly effect the footprint of data
  needed for a visualization.}

  \label{fig:rmi-resolution}

\end{figure}

Existing ray-guided approaches are, however, dependent on a lengthy
preprocess which limits their applicability as data sizes grow.
The purpose of this preprocess is twofold: the first is data
reorganization, which moves data around so that it is stored in the
way it will be accessed; this can have a large effect on the time
it takes to access data.  The second is metadata computation, which
allows software to avoid loading data if it does not meet certain
characteristics, with often profound repercussions in the amount of
data to be visualized.  While the authors of this proposal are at the
forefront of this research
area~\cite{Hadwiger:2012:Guided, Fogal:2013:Analysis,
Valerio:Something}, we must admit that the large advantages a
preprocess gives comes at too great a cost to be considered for the
data sizes we are discussing here.

We propose a new processing pipeline, based around the idea that
\textit{sampling} is an inherent component of any visualization-based
system.  In our new pipeline, we have divorced the sampling stage
from its traditional shackles within `rendering' and given it proper
consideration as its own stage.  The salient characteristic of a system
designed with this idea in mind is that processing occurs \emph{only}
in regions which will effect the current visualization.  In some cases,
this means that large portions of a dataset need never be processed.
More importantly, this puts a bound on the amount of per-frame work
which must be done.

\begin{figure}
  \centering
  \includegraphics[width=\linewidth]{images/new-pipeline}
  %\includegraphics[width=\linewidth]{images/draft}

  \caption{A modified visualization pipeline scheme which performs
  sampling as a separate step.  Considering sampling earlier can have
  large ramifications on the set of data which must proceed through the
  rest of the pipeline.}

\end{figure}

Applied to ray-guided volume rendering techniques, this means
that the work traditionally performed during preprocessing would now
occur in a demand-driven manner.  The computation of metadata will be
performed when the data are first requested; similarly, data can be
reorganized and placed in a cache, so that subsequent accesses are
considerably faster.  Many of the image processing techniques required
for the \WinslowDBP DBP could be performed dynamically as well.

\section{\WinslowDBP}

The data being acquired and refined by a group headed by Raimond
Winslow at Johns Hopkins University (JHU) is representative of a new
challenge in volume visualization.  The JHU group has invested much
to acquire scans of heart data at unprecedented levels of resolution.
The hope is that the sheer scale of these data will enable a whole new
level of insights into heart conditions and their treatments, such
as cardiac ablation.  Unfortunately, due to limitations in current
biomedical visualization and analysis tools, the group cannot visualize
the data they are acquiring.  The groups' current solution is to
downsample the data to much smaller sizes---negating the purpose of
acquiring the high-resolution scans in the first place.

\begin{figure*}
  \centering
  \includegraphics[width=\linewidth]{images/histeq-lowres}
  %\includegraphics[width=\linewidth]{images/heq-very-lowres}

  \caption{Preliminary image processing effects on heart data from JHU.
  Features are made more prominent and given more consistent values,
  aiding in the task of transfer function design.}

  \label{fig:jhu-correction}
\end{figure*}

Furthermore, the data require extensive image processing to be able
to pull out features of interest.  The present reconstruction process
involves a series of corrections by Chow et al [cite] and Gopinath et
al [cite], which account for vignetting artifacts as well as signal
attenuation.  Unfortunately, a robust solution to issues such as the
scattering and attenuation of light through the volume has not been
possible, due to the computational intensity of the problem.  The
current pipeline therefore uses local region information and does not
account for potentially deep penetration and scattering of light.

\begin{figure}
  \centering
  \includegraphics[width=\linewidth]{images/heart_2}

  \caption{Heart data from the \WinslowDBP collaboration with a
  diagonal cut on one end.  Collaborators wish to visualize the
  striations visible in the transverse plane across the coronal or
  sagittal planes, however the noise overcomes the signal as the
  cutting plane's angle of incidence increases.}

  \label{fig:jhu-diagonal}
\end{figure}

Some image processing techniques, such as histogram equalization, can
mitigate these issues (see Figure~\ref{fig:jhu-correction}).  However,
we have not yet been able to fully draw out the desired features
in these data.  Figure~\ref{fig:jhu-diagonal} details a cut diagonal
cut of one such data set.  Our collaborators would like to follow the
structures visible here coronally through the volume, but they are
washed out as the angle of incidence is increased.  Coupled with the
computational complexity of the existing solution, it is clear that
a new visual reconstruction process would be a boon to this work's
progress.

% Furthermore, the data require extensive image processing to be able to
% pull out features of interest.  Imperfections in the scanning process
% result in minor image variation from slice to slice: intensity values
% within the data have only an approximate correlation to physical
% meaning.  This creates a number of problems when visualizing the data,
% as it is impossible for the eye to correlate data along a feature of
% interest.  Fortunately, the issue can be mitigated through the use of
% some image processing techniques, as shown in
% Figure~\ref{fig:jhu-correction}.
% 
% The current preprocess consists of a histogram equalization stage and
% a `despeckling' process, which uses a model of the noise in the data
% to remove the extraneous 1-voxel outliers which are present due to
% the scanning process.  We envision that this preprocess will grow in
% complexity, as we have only just begun to understand the intricacies
% in these data.  Yet the image processing pipeline is already extremely
% lengthy: to process a single heart scan in our current pipeline might
% take months.

We will address this issue on numerous fronts.  First, we are porting
the existing algorithms from GNU Octave---an image processing packages
which is easily programmable but generally lacking in performance---to
custom C-based code which is considerably faster.  Second, we are
parallelizing the input so that numerous slices can be processed at
once.  However, we consider our most promising approach to be based on
integrating this processing pipeline with our visualization tools.

As outlined in Section~\ref{sec:volren}, the data accessed in a typical
visualization is relatively small.  We therefore endeavor to use
the rendering process to guide which subsets of the data should be
processed. Through the use of the aforementioned ray-guided techniques,
we can identify which regions need this image processing pipeline
applied.  With the combined effect of faster hand-coded algorithms
and selectively processing only the data needed for a visualization
session, we should be able to provide a responsive visualization system
with no need for a preprocess.

% problem: we need the cumulative distribution function of all images
% to do the histogram equalization, which in turn implies we need to
% touch every pixel. however, i've seen that if I just use a CDF from
% an image or two, we still get very good results; hell, we could
% probably just do image normalization (linear CDF) and just be fine.
% thus, let's calculate the CDF on the fly based on what we see. it'll
% mean that we can't cache bricks---it's likely that the second time
% we request a brick we'll have a `better' CDF that we want to correct
% with, but that will be okay as long as our processing is fast enough.

\bibliographystyle{acm}
\bibliography{vr}
\end{document}
