Medical Imaging Interaction Toolkit  2021.02.99-89548602
Medical Imaging Interaction Toolkit
The Segmentation View
segmentation-dox.svg
Icon of the segmentation view

Some of the features described below are closed source additions to the open source toolkit MITK and are not available in every application.

Overview

Segmentation is the act of partitioning an image into subsets by either manual or automated delineation to create i.e. a distinction between foreground and background. The MITK segmentation plugin allows you to create segmentations of anatomical and pathological structures in medical images. The plugin consists of a number of views:

In this documentation, the features and usage of the segmentation view will be described. For an introduction to the segmentation utilities and clipping plane views, please be referred to the respective documentation pages.

QmitkSegmentationPlugin_Overview.png
Segmentation plugin overview

Preferences

The segmentation plugin offers a number of preferences which can be set via the MITK Workbench application preferences:

QmitkSegmentationPreferences.png
Segmentation preferences
  • Slim view: Allows you to show or hide the tool button description of the segmentation view
  • 2D display: Specify whether the segmentation is drawn as outline or as a transparent overlay
  • 3D display: Activate 3D volume rendering for your segmentation
  • Data node selection mode: If activated the segmentation image is automatically chosen from the data manager selection.
  • Smoothed surface creation: Set certain smoothing parameters for surface creation

Technical Issues

The segmentation plugin makes a number of assumptions:

  • Images must be 2D, 3D, or 3D+t.
  • Images must be single-values, i.e. CT, MRI or "normal" ultrasound. Images from color doppler or photographic (RGB) images are only partially supported (please be aware that some tools might not be compatible with this image type).
  • Segmentations are handled as binary images of the same extent as the original image.

Data Selection & Creating New Segmentations

To select a reference image for the segmentation, click on the Patient Image selection widget and choose a suitable image from the selection available in the data manager. By default the auto selection mode is enabled (see Preferences).
Once a patient image is selected, a new segmentation can be created on this reference image by clicking the New... button to the right of the Segmentation selection widget. An input field will appear which allows you to set the name and display color of the segmentation. Notice that the input field suggests names once you start typing and that it also suggests colors for known organ names. Once generated the segmentation will be added with "binary mask" icon to the data manager as sub-node of the reference image. This item is automatically selected for you, allowing you to start editing the new segmentation right away.

Selecting Segmentations for Editing

Alternatively to creating a new segmentation, an existing one can be edited as well. As you might have segmented multiple structures within a single image, the application needs to know which of them to use for editing. For that you click the segmentation selection widget and a selection field will open, containing all suitable segmentations for the parent dataset available in the data manager.

Segmentation Tool Overview

MITK offers a comprehensive set of slice-based 2D and (semi-)automated 3D segmentation tools. The manual 2D tools require some user interaction and can only be applied to a single image slice whereas the 3D tools operate on the whole image. The 3D tools usually only require a small amount of user interaction, i.e. placing seed points or setting/adjusting parameters. You can switch between the different toolsets by selecting the 2D or 3D tab in the segmentation view.

QmitkSegmentation_ToolOverview.png
An overview of the existing 2D and 3D tools in MITK.

2D Segmentation Tools

With 2D manual contouring you define which voxels are part of the segmentation and which ones are not. This allows you to create segmentations of any structures of interest in an image. You can also use manual contouring to correct segmentations that result from sub-optimal automatic methods. The drawback of manual contouring is that you might need to define contours on many 2D slices. However, this is mitigated by the interpolation feature, which will make suggestions for a segmentation.

Selecting Editing Tools

To start using one of the editing tools, click its button from the displayed toolbox. The selected editing tool will be active and its corresponding button will stay pressed until you click the button again. Selecting a different tool also deactivates the previous one.
If you have to delineate a lot of images, shortcuts to switch between tools becomes convenient. For that, just hit the first letter of each tool to activate it (A for Add, S for Subtract, etc.).

Using Editing Tools

All of the editing tools work by the same principle: you use the mouse (left button) to click anywhere in a 2D window (any of the orientations axial, sagittal, or coronal), move the mouse while holding the mouse button and release to finish the editing action. Multi-step undo and redo is fully supported by all editing tools. Use the application-wide undo button in the toolbar to revert erroneous actions.

Remark: If you are familiar with the MITK Workbench, you know that clicking and moving the mouse in any of the 2D render windows will move around the crosshair that defines what part of the image is displayed. This behavior is disabled whilst any of the manual segmentation tools are active- otherwise you might have a hard time concentrating on the contour you are drawing.

Add and Subtract Tools

QmitkSegmentation_IMGIconAddSubtract.png
Add and Subtract Tools

Use the left mouse button to draw a closed contour. When releasing the mouse button, the contour will be added (Add tool) to or removed (Subtract tool) from the current segmentation. Adding and subtracting voxels can be iteratively repeated for the same segmentation. Holding CTRL / CMD while drawing will invert the current tool's behavior (i.e. instead of adding voxels, they will be subtracted).

Paint and Wipe Tools

QmitkSegmentation_IMGIconPaintWipe.png
Paint and Wipe Tools

Use the Size slider to change the radius of the round paintbrush tool. Move the mouse in any 2D window and press the left button to draw or erase pixels. Holding CTRL / CMD while drawing will invert the current tool's behavior (i.e. instead of painting voxels, they will be wiped).

Region Growing Tool

QmitkSegmentation_IMGIconRegionGrowing.png
Region Growing Tool

Click at one point in a 2D slice widget to add an image region to the segmentation with the region growing tool. Region Growing selects all pixels around the mouse cursor that have a similar gray value as the pixel below the mouse cursor. This enables you to quickly create segmentations of structures that have a good contrast to surrounding tissue. The tool operates based on the current level window, so changing the level window to optimize the contrast for the ROI is encouraged. Moving the mouse up/down is different from left/right: Moving up the cursor while holding the left mouse button widens the range for the included grey values; moving it down narrows it. Moving the mouse left and right will shift the range. The tool will select more or less pixels, corresponding to the changing gray value range.

Fill Tool

QmitkSegmentation_IMGIconFill.png
Fill Tool

Left-click inside a segmentation with holes to completely fill all holes. Left-click inside a hole to fill only this specific hole.

Erase Tool

QmitkSegmentation_IMGIconErase.png
Erase Tool

This tool removes a connected part of pixels that form a segmentation. You may use it to remove single segmentations (left-click on specific segmentation) or to clear a whole slice at once (left-click outside a segmentation).

Live Wire Tool

QmitkSegmentation_IMGIconLiveWire.png
Live Wire Tool

The Live Wire Tool acts as a magnetic lasso with a contour snapping to edges of objects.

QmitkSegmentation_IMGLiveWireUsage.PNG
Steps for using the Live Wire Tool
  • (1) To start the tool you have to double-click near the edge of the object you want to segment. The initial anchor point will snap to the edge within a 3x3 region.
  • (2) Move the mouse. You don't have trace the edge of the object. The contour will automatically snap to it.
  • (3) To fix a segment you can set anchor points by single left mouse button click.
  • (4) Go on with moving the mouse and setting anchor points.
  • (5) To close the contour double-click on the initial anchor point.
  • (6) After closing, the contour can be edited by moving, inserting and deleting anchor points.

The contour will be transferred to its binary image representation by deactivating the tool.

2D Fast Marching Tool

QmitkSegmentation_IMG2DFastMarchingUsage.png
2D Fast Marching Tool

Provides a fast marching based 2D interaction segmentation tool. You start with setting seed points in an image slice. Via several sliders you can adapt parameters and see the fast marching result instantly.

2D and 3D Interpolation

Creating segmentations using 2D manual contouring for large image volumes may be very time-consuming, because structures of interest may cover a large range of slices. The segmentation view offers two helpful features to mitigate this drawback:

  • 2D Interpolation
  • 3D Interpolation

The 2D Interpolation creates suggestions for a segmentation whenever you have a slice that

  • has got neighboring slices with segmentations (these do not need to be direct neighbors but could also be a couple of slices away) AND
  • is completely clear of a manual segmentation, i.e. there will be no suggestion if there is even only a single pixel of segmentation in the current slice.
QmitkSegmentation_2DInterpolation.png
2D Interpolation Usage

Interpolated suggestions are displayed as outlines, until you confirm them as part of the segmentation. To confirm single slices, click the Confirm for single slice button below the toolbox. You may also review the interpolations visually and then accept all of them at once by selecting Confirm for all slices.

The 3D interpolation creates suggestions for 3D segmentations. That means if you start contouring, from the second contour onwards, the surface of the segmented area will be interpolated based on the given contour information. The interpolation works with all available manual tools. Please note that this is currently a pure mathematical interpolation, i.e. image intensity information is not taken into account. With each further contour the interpolation result will be improved, but the more contours you provide the longer the recalculation will take. To achieve an optimal interpolation result and in this way a most accurate segmentation you should try to describe the surface with sparse contours by segmenting in arbitrary oriented planes. The 3D interpolation is not meant to be used for parallel slice-wise segmentation, but rather segmentations in i.e. the axial, coronal and sagittal plane.

QmitkSegmentation_3DInterpolationWrongRight.png
3D Interpolation Usage

You can accept the interpolation result by clicking the Confirm-button below the tool buttons. In this case the 3D interpolation will be deactivated automatically so that the result can be post-processed without any interpolation running in the background.

Additional to the surface, black contours are shown in the 3D render window, which mark all the drawn contours used for the interpolation. You can navigate between the drawn contours by clicking on the corresponding position nodes in the data manager which are stored as sub-nodes of the selected segmentation. If you do not want to see these nodes just uncheck the Show Position Nodes checkbox and these nodes will be hidden.

If you want to delete a drawn contour we recommend to use the Erase-Tool since Redo/Undo is not yet working for 3D interpolation. The current state of the 3D interpolation can be saved across application restart. For that, just click on save project during the interpolation is active. After restarting the application and load your project you can click on "Reinit Interpolation" within the 3D interpolation GUI area.

3D Segmentation Tools

The 3D tools operate on the whole image and require usually a small amount of interaction like placing seed-points or specifying certain parameters. All 3D tools provide an immediate segmentation feedback, which is displayed as a transparent green overlay. For accepting a preview you have to press the Confirm button of the selected tool. The following 3D tools are available:

3D Threshold Tool

The thresholding tool simply applies a 3D threshold to the patient image. All pixels with values equal or above the selected threshold are labeled as part of the segmentation. You can change the threshold by either moving the slider of setting a certain value in the spinbox.

QmitkSegmentation_3DThresholdTool.png
3D Threshold tool

3D Upper/Lower Threshold Tool

The Upper/Lower Thresholding tool works similar to the simple 3D threshold tool but allows you to define an upper and lower threshold. All pixels with values within this threshold interval will be labeled as part of the segmentation.

QmitkSegmentation_3DULThresholdTool.png
3D Upper/Lower Threshold tool

3D Otsu Tool

The 3D Otsu tool provides a more sophisticated thresholding algorithm. It allows you to define a number of regions. Based on the image histogram the pixels will then be divided into different regions. The more regions you define the longer the calculation will take. You can select afterwards which of these regions you want to confirm as segmentation.

QmitkSegmentation_3DOtsuTool.png
3D Otsu tool

3D Fast Marching Tool

The 3D Fast Marching tool works similar to the 2D pendant but on the whole image. Depending on your image size the calculation might take some time. You can interactively set the parameters of the algorithm via the GUI. The resulting segmentation will be automatically updated.

QmitkSegmentation_3DFMTool.png
3D Fast Marching tool

3D Region Growing Tool

The 3D Region Growing tool works similar to the 2D pendant. At the beginning you have to place a seedpoint and define a threshold interval. If you press Run Segmentation a preview is calculated. By moving the Adapt region growing slider you can interactively adapt the segmentation result.

QmitkSegmentation_3DRGTool.png
3D Region Growing tool

3D Watershed Tool

This tool provides a watershed based segmentation algorithm. For a detailed explanation of the parameters level and threshold, please be referred to https://itk.org/Doxygen/html/classitk_1_1WatershedImageFilter.html . Remark: The tool is (due to its implementation) faster if you use lower levels. So in doubt you should start with high levels (supposed to undersegment) and then lower the levels until you are happy.

QmitkSegmentation_3DWatershedTool.png
3D Watershed tool

Picking Tool

The Picking tool allows you to select islands within your segmentation. This is especially useful if e.g. a thresholding provided you with several areas within your image but you are just interested in one special region.

QmitkSegmentation_PickingTool.png
Picking tool

Additional things you can do with segmentations

Segmentations are never an end in themselves. Consequently, the segmentation view adds a couple of "post-processing" actions, accessible through the context-menu of the data manager.

QmitkSegmentation_IMGDataManagerContextMenu.png
Context menu items for segmentations.
  • Create polygon model applies the marching cubes algorithm to the segmentation. This polygon model can be used for visualization in 3D or other applications such as stereolithography (3D printing).
  • Create smoothed polygon model uses smoothing in addition to the marching cubes algorithm, which creates models that do not follow the exact outlines of the segmentation, but look smoother.
  • Autocrop can save memory. Manual segmentations have the same extent as the patient image, even if the segmentation comprises only a small sub-volume. This invisible and meaningless margin is removed by autocropping.

Segmentation of 3D+t images

For segmentation of 3D+t images, some tools give you the option to choose between creating dynamic and static masks.

  • Dynamic masks can be created on each time frame individually.
  • Static masks will be defined on one time frame and will be the same for all other time frames.

In general, segmentation is applied on the time frame that is selected when execution is performed. If you alter the time frame, the segmentation preview is adapted.

Technical Information for Developers

For technical specifications see Technical design of QmitkSegmentation and for information on the extensions of the tools system How to extend the Segmentation view with external tools .