PROJECT PLAN: 3d Annotation Project

AUTHOR: Jenny Simpson

ISSUE: 2

REVISION: 2

DATE: 5/16/2003

DEVELOPERS

Primary:

Janna Balling (SCI)

Jason Lauman (PICC)

Jenny Simpson (SCI)

Collaborators:

Kurt Albertine (PICC)

Richard Coffey (SCI)

"Bo" Kenneth Foreman (Anatomy)

Patricia Goede (PICC)

Chris Johnson (SCI)

Greg Jones (SCI)

David Morton (Anatomy)

PROJECT DESCRIPTION

Introduction:

The tentative overall goal for this project is to provide a tool that does the following (some of this is already accomplished by existing software which we will use):

· Reads CT scanned data into a 3d rendering system.

· Allows exploration of the 3d data with user-controlled viewing parameters.

· Includes an authoring system for adding 3d annotations to the model.

· Plugs into a common presentation tool such as Flash so that annotated models can be demonstrated in a classroom setting and used by students for self-learning.

· Has an end-user interface that includes multi-modal images and expert knowledge along with 3d annotated images.

The central piece of the Summer 2003 portion of this project will be the 3d annotation

authoring system. The last two steps will be beyond the scope of the summer

project, but offers opportunities for further collaboration between PICC and SCI.

We will use a CT scan of a scapula as our test dataset. We will use Vtk as our renderering engine. Our testbed will be an anatomy teaching environment.

Problem statement:

Our research has shown that while there are some existing tools that address these issues

(specifically Voxel-Man), the available software provides tools for viewing pre-annotated data but does not provide tools for authoring new annotations for models. This authoring component is critical to our users because of the flexibility and control it affords them.

Objectives:

The primary goal of the Scapula Atlas Project (SAP) is to

The secondary goal of the Scapula Atlas Project is to develop 1) an authoring

tool capable of annotating 3D representations of anatomic structures with the

Overall:

· PICC/SCI - Form a collaborative relationship between the Scientific Computing and Imaging (SCI) Institute, represented by Associate Director Dr. Greg Jones, and the Program in Imaging Communication and Collaboration (PICC) group represented by Dr. Kurt Albertine.

· SCI - Provide some sort of 3d annotation that can be run over the AG.

· Jason - Participate in the aforementioned collaboration and apply pieces from other projects that could be useful for this one.

· Jenny - Build a product that is a usable component of a larger system that is extensible. Produce a paper from the project that can be used to fulfill the senior project requirements for the CS department.

· Janna - Same as above, but her senior project is for the Bioengineering department.

· Bo & Dave - Gain useful technical tools for teaching anatomy that can be used for both teaching and for their graduate research.

Technical:

· Provide a 3d annotation authoring tool for instructors in class (Fall 2003)

· Provide a 3d interactive viewer with 3d annotations for students (Spring 2003?)

Requirements / Constraints:

1. Annotation Tool

· Provide full interactive viewing of 3d models (instructors have to view their models in order to annotate them).

· Provides fairly high level of detail in models.

· Provide ability to add 3d annotations to a model.

· Must be able to read in new data sets in some reasonable format (Dicom, STL).

· Runs locally (doesn't need to run over server).

· Runs on a windows box -- that's what instructors are comfortable with.

· Needs to be efficient enough to provide interactive 3d viewing when run locally on a medium to high-end desktop machine.

· Open source.

· Provides some interface for mouse interaction.

2. Viewing & Interaction Tool

Server:

· Runs on Windows box.

· Serves up 2d images to client based on client control.

· Can serve up images at an interactive rate to an arbitrary number of individually controlled clients over the internet.

· Has some way of storing an annotated 3d model -OR- a set of annotated 2d images.

Methods:

The developers involved will interact via e-mail and person-to-person interaction. Since the project is highly collaborative and integrated, all major design and implementation issues will be discussed thoroughly among developers. Regular meetings will be held and deadlines set to ensure timely completion of the application.

Costs:

Janna will be funded through the a grant. Jenny will not be funded for the project, due to conflicts with CS Senior Project requirements.

PLATFORM

Windows (server, client, and local viewer)

IMPLEMENTATION DETAILS

Overall Steps:

1. Provide a 3d annotation authoring tool for instructors in class (Fall 2003)

2. Provide a 3d interactive viewer with 3d annotations for students (Spring 2003?)

Intermediate Steps:

1) (done) Perform CT scans of select set of bones for the testbed case. (May, week 1)

2) (done) Read volume data into a 3d rendering system (i.e. Vtk).

ü (done) Get up and running with Vtk

ü (done) Finish reading User’s Guide (due Wed., May 14, 6pm)

ü (done) Obtain existing dataset in Vtk format

ü (done) Render aforementioned dataset using Vtk (due Fri., May 16, 12pm)

· (done) Obtain existing CT scanned dataset

· (done) Render aforementioned dataset using Vtk

3) (done) Develop a data format to represent 3d annotations that correspond to 3d

models. The format will be abstracted from the

renderer so that the format could theoretically be used by any renderer

capable of 3d

annotations.

· (ongoing) Compile information about what annotations should be included and how they should be characterized based on existing standards for 2d/3d annotation as well as input from Bo and Dave

4) (75% done) Build a set of 3d models in Vtk, one for each type of annotation required. Test each type of annotation in various scenarios with a real model.

5) Enable the renderer to read the aforementioned format and render

the 3d annotations described therein.

a. (done) Write a reader/writer for the annotation file that converts information into annotation “objects” with all related information and then puts information back in the file.

b. (done) Write routines to convert annotation objects into Vtk geometry etc.

c. Extensively test the rendering of the system.

6) Enable users to add 3d annotations to the model by using an authoring

system which converts graphical representations of annotations into

information stored in a flat file of the format described above.

7) Write a server that sends the rendering engine data and gets a series of 2d images in return. The data that the server sends the rendering engine includes the dataset, annotation data file, and viewing parameters.

8) Connect the server to a common presentation tool such as Flash so that

annotated models can be demonstrated in a classroom setting and can be

used by

students for self-learning.

9) Develop a centralized server that runs the system so that the system

can be accessed remotely. (This is not a small problem!)

10) And the list of possibilities goes on...

Notes:

We will use CT scans of several shoulder bones as our datasets. The CT

scans will be extremely valuable in and of themselves.

The sub-project which we aim to complete by September 2003 consists of

steps 1-5 (maybe others, but we're trying to be conservative).

TARGET COMPLETION DATES

Step 1: May, week 1

Step 2: May, weeks 2-3

Step 3: May, weeks 1-4

Step 4: June, weeks 1-4, July, weeks 1-2

Step 5a: June, weeks 1-2

Step 5b: July, weeks 3-4, August, week 1

Step 5c: August, weeks 2-4

Deployment of application: September, 2003

OPEN QUESTIONS

· How does Flash deal with JPG images?

· How are annotations added by the user on the Flash side?

· Is it possible to do what we are trying to do with Flash? If not, the project is not workable.

· If we complete the application as designed, will it satisfy the needs of users? What reason do we have to believe that it will satisfy their requirements? Should we write a requirements document or project proposal to send to them?

· How are we going to do 3d annotations? What is the standard way for doing them? How will we make this intuitive enough that users are comfortable transitioning from 2d to 3d annotations?

· What specific annotations do we need?

· What is the pipeline that typical 3d medical data goes through, from being scanned to being rendered?

· Is there an existing product that basically does all of this? What about Swift3d?

· What resolution will the images be? Can we control the resolution? What resolution do we want?

· Does vtk read dicom format directly?

· Will our end product be general and reusable?

· Do students need *full* 3d interaction, or just access to key views?

· Are we going to get the efficiency we need from polygonal or volume-based rendering? Would image-based rendering be more efficient?

· How long will the CT scan take?

· How are the test annotation files generated?

CONTACT INFORMATION

Kurt Albertine <Kurt.Albertine@hsc.utah.edu>

Janna Balling <balling@sci.utah.edu>

Richard Coffey <richardc@cs.utah.edu>

"Bo" Kenneth Foreman <Kenneth.Foreman@hsc.utah.edu>

Patricia Goede <Patricia.Goede@hsc.utah.edu>

Chris Johnson <crj@sci.utah.edu>

Greg Jones <gjones@sci.utah.edu>

Jason Lauman <Jason.Lauman@hsc.utah.edu>

David Morton <David.Morton@hsc.utah.edu>

Jenny Simpson <simpson@cs.utah.edu>