Medical Imaging Interaction Toolkit  2023.04.00
Medical Imaging Interaction Toolkit
Technical design of QmitkSegmentation

Introduction

QmitkSegmentation was designed for the liver resection planning project "ReLiver". The goal was a stable, well-documented, extensible, and testable re-implementation of a functionality called "ERIS", which was used for manual segmentation in 2D slices of 3D or 3D+t images. Re-implementation was chosen because it seemed to be easier to write documentation and tests for newly developed code. In addition, the old code had some design weaknesses (e.g. a monolithic class), which would be hard to maintain in the future.

By now Segmentation is a well tested and easily extensible vehicle for all kinds of interactive segmentation applications. A separate page describes how you can extend Segmentation with new tools in a shared object (DLL): How to extend the Segmentation view with external tools.

Overview of tasks

We identified the following major tasks:

  1. Management of images: what is the original patient image, what images are the active segmentations?
  2. Management of drawing tools: there is a set of drawing tools, one at a time is active, that is, someone has to decide which tool will receive mouse (and other) events.
  3. Drawing tools: each tool can modify a segmentation in reaction to user interaction. To do so, the tools have to know about the relevant images.
  4. Slice manipulation: drawing tools need to have means to extract a single slice from an image volume and to write a single slice back into an image volume.
  5. Interpolation of unsegmented slices: some class has to keep track of all the segmentations in a volume and generate suggestions for missing slices. This should be possible in all three orthogonal slice direction.
  6. Undo: Slice manipulations should be undoable, no matter whether a tool or the interpolation mechanism changed something.
  7. GUI: Integration of everything.

Classes involved

The above blocks correspond to a number of classes. Here is an overview of all related classes with their responsibilities and relations:


  1. Management of images: mitk::ToolManager has a set of reference data (original images) and a second set of working data (segmentations). mitk::Tool objects know a ToolManager and can ask the manager for the currently relevant images. GUI and non-GUI classes are coupled by itk::Events (non-GUI to GUI) and direct method calls (GUI to non-GUI).
  2. Management of drawing tools: As a second task, ToolManager manages all available tools and makes sure that one at a time is able to receive MITK events. The GUI for selecting tools is implemented in QmitkToolSelectionBox.
  3. Drawing tools: Drawing tools all inherit from mitk::Tool, which is a mitk::StateMachine. There is a number of derivations from Tool, each offering some helper methods for specific sub-classes, like manipulation of 2D slices. Tools are instantiated through the itk::ObjectFactory, which means that there is also one factory for each tool (e.g. mitk::AddContourToolFactory). For the GUI representation, each tool has an identification, consisting of a name and an icon (XPM). The actual drawing methods are mainly implemented in mitk::SegTool2D (helper methods) and its sub-classes for region growing, freehand drawing, etc.
  4. Slice manipulation: There are two ways to manipulate slices inside a 3D image volume. mitk::ExtractImageFilter retrieves a single 2D slice from a 3D volume. mitkVtkImageOverwrite replaces a slice inside a 3D volume with a second slice. These classes are used extensively by most of the tools to fulfill their task.
  5. Interpolation of unsegmented slices: There are two classes involved in interpolation: mitk::SegmentationInterpolationController knows a mitk::Image (the segmentation) and scans its contents for slices with non-zero pixels. It keeps track of changes in the image and is always able to tell, which neighbors of a slice (in the three orthogonal slice directions) contain segmentations. The class also performs this interpolation for single slices on demand. Again, we have a second class responsible for the GUI: QmitkSlicesInterpolator enables/disables interpolation and offers to accept interpolations for one or all slices.
  6. Undo: Undo functionality is implemented using mitk::DiffSliceOperation. mitk::SegTool2D::WriteSliceToVolume stores the original image and the modified slice to the undo stack as an mitk::DiffSliceOperation. When the user requests undo, this mitk::DiffSliceOperation will be executed by a singleton class DiffSliceOperationApplier. The operation itself observes the image, which it refers to, for itk::DeleteEvent, so no undo operation will be executed on/for images that have already been destroyed.
  7. GUI: The top-level GUI is the view QmitkSegmentation, which is very thin in comparison to ERIS. There are separate widgets for image and tool selection, for interpolation. Additionaly, there are some methods to create, delete, crop, load and save segmentations.