Segmentation tool integrating the nnInteractive deep learning model. More...

#include <mitknnInteractiveTool.h>

Inheritance diagram for mitk::nnInteractiveTool:

[legend]

Collaboration diagram for mitk::nnInteractiveTool:

[legend]

Classes
struct	CUDADeviceInfo
	Information about a CUDA-capable GPU device. More...

struct	SupportedInteractions
	Which interaction types the running session's model checkpoint supports. More...

Public Types
using	InteractorMap = std::unordered_map< nnInteractive::InteractionType, std::unique_ptr< nnInteractive::Interactor > >
	Map type associating InteractionType keys to Interactor instances. More...

Public Types inherited from mitk::SegWithPreviewTool
enum class	LabelTransferScope { ActiveLabel , SelectedLabels , AllLabels }
	Defines which labels from the preview are transferred on confirmation. More...

enum class	LabelTransferMode { MapLabel , AddLabel }
	Defines how labels are transferred from preview to the working image. More...

using	SelectedLabelVectorType = std::vector< Label::PixelType >
	Type for the vector of selected label values. More...

Public Types inherited from mitk::Tool
typedef mitk::Label::PixelType	DefaultSegmentationDataType

Public Types inherited from mitk::EventStateMachine
typedef itk::SmartPointer< StateMachineState >	StateMachineStateType

Public Member Functions
	mitkClassMacro (nnInteractiveTool, SegWithPreviewTool) static Pointer New()

const char *	GetName () const override
	Returns the display name of this tool. More...

us::ModuleResource	GetIconResource () const override
	Returns the icon resource for this tool. More...

void	Deactivated () override
	Called when the tool is deactivated. More...

const InteractorMap &	GetInteractors () const
	Returns a reference to the map of all interactors. More...

const nnInteractive::Interactor *	GetInteractor (nnInteractive::InteractionType interactionType) const
	Returns the interactor for a specific interaction type. More...

void	EnableInteractor (nnInteractive::InteractionType nextInteractionType, nnInteractive::PromptType promptType)
	Enables an interactor for the given interaction and prompt type. More...

void	DisableInteractor (std::optional< nnInteractive::InteractionType > interactionType=std::nullopt)
	Disables an interactor. More...

void	ResetInteractions ()
	Resets all interactors and clears the current preview. More...

bool	HasInteractions () const
	Checks whether any interactor has recorded interactions. More...

void	UndoLastInteraction ()
	Reverts the most recent interaction (single-level undo). More...

bool	CanUndo () const
	Returns whether an interaction is currently available to undo. More...

bool	SupportsUndo () const
	Returns whether the running session's backend supports undo. More...

bool	GetAutoZoom () const
	Returns whether auto-zoom is enabled. More...

void	SetAutoZoom (bool autoZoom)
	Enables or disables auto-zoom. More...

bool	GetAutoRefine () const
	Returns whether auto-refine is enabled. More...

void	SetAutoRefine (bool autoRefine)
	Enables or disables auto-refine. More...

std::optional< nnInteractive::Backend >	GetBackend () const
	Returns the currently active computation backend. More...

std::string	GetVirtualEnvName () const
	Returns the name of the Python virtual environment used. More...

bool	CreatePythonContext ()
	Creates and activates a Python context with a virtual environment. More...

PythonContext *	GetPythonContext () const
	Returns the current Python context. More...

bool	IsInstalled () const
	Checks whether the nnInteractive Python package is installed. More...

bool	IsLocalInferenceAvailable () const
	Checks whether local (in-process) inference is available. More...

bool	GetCUDADeviceInfo (CUDADeviceInfo &info) const
	Queries CUDA device information via PyTorch. More...

void	StartSession ()
	Starts an nnInteractive inference session in the configured mode. More...

void	EndSession ()
	Ends the current nnInteractive inference session. More...

void	AbortSession ()
	Tears down a remote session whose connection was lost. More...

bool	IsSessionRunning () const
	Checks whether an nnInteractive session is currently running. More...

bool	IsRemoteSession () const
	Returns whether the running session is a remote (server) session. More...

void	Heartbeat ()
	Sends a single keep-alive heartbeat for a running remote session. More...

int	GetHeartbeatIntervalMs () const
	Returns the interval, in milliseconds, at which Heartbeat() should be called for the running remote session. More...

SupportedInteractions	GetSupportedInteractions () const
	Returns which interaction types the running session supports. More...

std::optional< std::string >	GetModelLicense () const
	Returns the running session's model checkpoint license, if any. More...

void	InitializeSessionWithMask (Image *mask)
	Initializes or reinitializes the session with an existing mask. More...

Public Member Functions inherited from mitk::SegWithPreviewTool
	mitkClassMacro (SegWithPreviewTool, Tool)

void	Activated () override
	Called when the tool gets activated. More...

void	Deactivated () override
	Called when the tool gets deactivated. More...

mitk::Label::AlgorithmType	GetAlgorithmType () const override
	Preview tools are SEMIAUTOMATIC by default (the user supplies seeds/parameters). Fully automatic preview tools (e.g. TotalSegmentator) override this again to AUTOMATIC. More...

void	ConfirmSegmentation ()
	Confirms the current preview segmentation and transfers it to the working image. More...

virtual void	SetCreateAllTimeSteps (bool _arg)
	If true, the preview is computed for all time steps at once. More...

virtual bool	GetCreateAllTimeSteps ()

virtual void	CreateAllTimeStepsOn ()

virtual void	CreateAllTimeStepsOff ()

virtual void	SetKeepActiveAfterAccept (bool _arg)
	If true, the tool stays active after the segmentation is accepted. More...

virtual bool	GetKeepActiveAfterAccept ()

virtual void	KeepActiveAfterAcceptOn ()

virtual void	KeepActiveAfterAcceptOff ()

virtual void	SetIsTimePointChangeAware (bool _arg)
	If true, the tool reacts to time point changes and updates the preview. More...

virtual bool	GetIsTimePointChangeAware ()

virtual void	IsTimePointChangeAwareOn ()

virtual void	IsTimePointChangeAwareOff ()

virtual void	SetResetsToEmptyPreview (bool _arg)
	If true, the preview is reset to empty on activation. More...

virtual bool	GetResetsToEmptyPreview ()

virtual void	ResetsToEmptyPreviewOn ()

virtual void	ResetsToEmptyPreviewOff ()

virtual void	SetUseSpecialPreviewColor (bool _arg)
	If true, a special color is used for the preview rendering. More...

virtual bool	GetUseSpecialPreviewColor ()

virtual void	UseSpecialPreviewColorOn ()

virtual void	UseSpecialPreviewColorOff ()

virtual void	SetRequestDeactivationConfirmation (bool _arg)
	If true, a confirmation dialog is shown before deactivating the tool. More...

virtual bool	GetRequestDeactivationConfirmation ()

virtual void	RequestDeactivationConfirmationOn ()

virtual void	RequestDeactivationConfirmationOff ()

Color	GetSpecialPreviewColor () const
	Returns the special preview color. More...

void	SetMergeStyle (MultiLabelSegmentation::MergeStyle mergeStyle)
	Sets how the preview content is merged into the working image upon confirmation. More...

virtual MultiLabelSegmentation::MergeStyle	GetMergeStyle ()

void	SetOverwriteStyle (MultiLabelSegmentation::OverwriteStyle overwriteStyle)
	Sets whether locked labels should be overwritten during transfer. More...

virtual MultiLabelSegmentation::OverwriteStyle	GetOverwriteStyle ()

void	SetLabelTransferScope (LabelTransferScope labelTransferScope)
	Sets which labels from the preview are transferred on confirmation. More...

virtual LabelTransferScope	GetLabelTransferScope ()

void	SetSelectedLabels (const SelectedLabelVectorType &labelsToTransfer)
	Sets the labels to transfer from preview to working image on confirmation. More...

virtual SelectedLabelVectorType	GetSelectedLabels ()

void	SetLabelTransferMode (LabelTransferMode labelTransferMode)
	Sets how labels are transferred from preview to the working image. More...

virtual LabelTransferMode	GetLabelTransferMode ()

bool	CanHandle (const BaseData referenceData, const BaseData workingData) const override
	Checks if a tool can handle the given reference data and optional working data. More...

void	UpdatePreview (bool ignoreLazyPreviewSetting=false)
	Triggers the update of the preview segmentation. More...

bool	IsUpdating () const

std::string	GetCurrentSegmentationName ()
	Returns the name of the currently selected segmentation node. More...

virtual DataNode *	GetTargetSegmentationNode () const
	Returns the currently selected segmentation node. More...

MultiLabelSegmentation *	GetTargetSegmentation () const

MultiLabelSegmentation *	GetPreviewSegmentation ()

const MultiLabelSegmentation *	GetPreviewSegmentation () const

DataNode *	GetPreviewSegmentationNode ()

void	SetPreviewOpacity (float opacity)

virtual float	GetPreviewOpacity () const

void	SetPreviewVisibility (bool visible)

virtual bool	GetPreviewVisibility () const

Public Member Functions inherited from mitk::Tool
	mitkClassMacro (Tool, EventStateMachine)

virtual us::ModuleResource	GetCursorIconResource () const
	Returns the cursor icon of the tool wrapped by a usModuleResource. More...

virtual bool	IsEligibleForAutoInit () const
	Returns whether this tool is eligible for auto-initialization by ToolManager. More...

virtual void	InitializeStateMachine ()
	Initializes the state machine for this tool based on the interaction type. More...

virtual itk::Object::Pointer	GetGUI (const std::string &toolkitPrefix, const std::string &toolkitPostfix)
	Interface for GUI creation. More...

virtual NodePredicateBase::ConstPointer	GetReferenceDataPreference () const
	Returns a predicate describing valid reference data for this tool. More...

virtual NodePredicateBase::ConstPointer	GetWorkingDataPreference () const
	Returns a predicate describing valid working data for this tool. More...

Public Member Functions inherited from mitk::EventStateMachine
	mitkClassMacro (EventStateMachine, InteractionEventHandler)

bool	LoadStateMachine (const std::string &filename, const us::Module *module=nullptr)
	Load a state machine pattern from an XML resource file. More...

bool	HandleEvent (InteractionEvent event, DataNode dataNode)
	Process an incoming interaction event. More...

void	EnableUndo (bool enable)
	Enable or disable the undo mechanism for this state machine. More...

void	EnableInteraction (bool enable)
	Enable or disable the state machine. More...

Public Member Functions inherited from mitk::InteractionEventHandler
	mitkClassMacroItkParent (InteractionEventHandler, itk::Object)

bool	SetEventConfig (const std::string &filename, const us::Module *module=nullptr)
	Loads a configuration from an XML resource. More...

bool	SetEventConfig (const EventConfig &config)
	Loads a configuration from an EventConfig object. More...

EventConfig	GetEventConfig () const
	Returns the current configuration. More...

bool	AddEventConfig (const std::string &filename, const us::Module *module=nullptr)
	This method extends the configuration. More...

bool	AddEventConfig (const EventConfig &config)
	This method extends the configuration. The configuration from the EventConfig object is loaded and only the ones conflicting are replaced by the new one. This way several configurations can be combined. More...

Public Member Functions inherited from mitk::InteractionEventObserver
	InteractionEventObserver ()

virtual	~InteractionEventObserver ()

void	Disable ()
	Disable this observer so it no longer receives notifications. More...

void	Enable ()
	Enable this observer so it receives notifications. More...

bool	IsEnabled () const
	Check whether this observer is currently enabled. More...

Static Public Member Functions
static const std::vector< std::pair< std::string, std::string > > &	GetSessionDefiningPreferences ()
	The preferences baked into a session at initialization, each paired with its default value (as stored). More...

Public Attributes
Message1< bool >	ConfirmCleanUpEvent
	Event triggered when the tool requests cleanup confirmation. More...

Message	PreviewUpdatedEvent
	Event triggered after the preview has been updated from a user interaction. More...

Message	DeactivatedEvent
	Event triggered at the end of Deactivated(). More...

Message	SessionEndedEvent
	Event triggered when the Python session has ended. More...

Message	SessionExpiredEvent
	Event triggered when a remote session was lost server-side. More...

Public Attributes inherited from mitk::Tool
Message	GUIProcessEventsMessage
	To let GUI process new events (e.g. qApp->processEvents() ) More...

Message1< std::string >	ErrorMessage
	To send error messages (to be shown by some GUI) More...

Message1< bool >	CurrentlyBusy
	To send whether the tool is busy (to be shown by some GUI) More...

Message1< std::string >	GeneralMessage
	To send general messages (to be shown by some GUI) More...

Protected Member Functions
	nnInteractiveTool ()
	Default constructor. Initializes interactors and connects events. More...

	~nnInteractiveTool () override
	Destructor. Disconnects interactor events. More...

void	SetToolManager (ToolManager *toolManager) override
	Sets the ToolManager and forwards it to all interactors. More...

void	DoUpdatePreview (const Image inputAtTimeStep, const Image oldSegAtTimeStep, MultiLabelSegmentation *previewImage, TimeStepType timeStep) override
	Updates the preview segmentation based on the latest interaction. More...

void	OnTimePointChanged () override
	Tears the running session down when the current time step no longer matches the time step the session was bound to. More...

void	Notify (InteractionEvent *event, bool isHandled) override
	Forwards unhandled interaction events to the enabled interactor. More...

void	ConfirmCleanUp () override
	Called when a confirmed segmentation needs cleanup. More...

void	SetPreviewLabel (MultiLabelSegmentation::LabelValueType value, const Color &color)
	Configures a label in the preview segmentation with a given value and color. More...

Protected Member Functions inherited from mitk::SegWithPreviewTool
	SegWithPreviewTool (bool lazyDynamicPreviews=false)

	SegWithPreviewTool (bool lazyDynamicPreviews, const char interactorType, const us::Module interactorModule=nullptr)

	~SegWithPreviewTool () override

const char *	GetGroup () const override
	Name of a group. More...

void	EnsureTargetSegmentationNodeInDataStorage () const

virtual void	InitiateToolByInput ()

virtual void	UpdatePrepare ()

virtual void	UpdateCleanUp ()

virtual void	PreparePreviewToResultTransfer (const LabelMappingType &labelMapping)
	Called before the preview content is transferred to the segmentation on confirmation. More...

const Image *	GetSegmentationInput () const

const Image *	GetReferenceData () const

void	ResetPreviewNode ()

void	ResetPreviewContent ()

void	ResetPreviewContentAtTimeStep (unsigned int timeStep)

TimePointType	GetLastTimePointOfUpdate () const

MultiLabelSegmentation::LabelValueType	GetActiveLabelValueOfPreview () const

virtual Label::PixelType	GetUserDefinedActiveLabel () const

virtual void	SetWorkingPlaneGeometry (PlaneGeometry *_arg)

virtual const PlaneGeometry *	GetWorkingPlaneGeometry ()

bool	ConfirmBeforeDeactivation () override
	Method call to invoke a dialog box just before exiting. More...

Protected Member Functions inherited from mitk::Tool
ToolManager *	GetToolManager () const

mitk::DataStorage *	GetDataStorage () const

void	ConnectActionsAndFunctions () override

	Tool (const char , const us::Module interactorModule=nullptr)

	~Tool () override

void	Notify (InteractionEvent *interactionEvent, bool isHandled) override
	Called by the Dispatcher to notify this observer about an InteractionEvent. More...

bool	FilterEvents (InteractionEvent , DataNode ) override

Protected Member Functions inherited from mitk::EventStateMachine
	EventStateMachine ()

	~EventStateMachine () override

void	AddActionFunction (const std::string &action, const ActionFunctionDelegate &delegate)

void	AddConditionFunction (const std::string &condition, const ConditionFunctionDelegate &delegate)

StateMachineState *	GetCurrentState () const

void	ResetToStartState ()
	ResetToStartState Reset state machine to it initial starting state. More...

virtual bool	CheckCondition (const StateMachineCondition &condition, const InteractionEvent *interactionEvent)

virtual void	ExecuteAction (StateMachineAction action, InteractionEvent interactionEvent)

void	SetMouseCursor (const char *xpm[], int hotspotX, int hotspotY)
	Sets the specified mouse cursor. More...

void	ResetMouseCursor ()
	Resets the mouse cursor to its original state. More...

StateMachineTransition *	GetExecutableTransition (InteractionEvent *event)
	Returns the executable transition for the given event. More...

Protected Member Functions inherited from mitk::InteractionEventHandler
	InteractionEventHandler ()

	~InteractionEventHandler () override

PropertyList::Pointer	GetAttributes () const

std::string	MapToEventVariant (InteractionEvent *interactionEvent)

virtual void	ConfigurationChanged ()

Additional Inherited Members
Protected Types inherited from mitk::SegWithPreviewTool
using	LabelMappingType = std::vector< std::pair< Label::PixelType, Label::PixelType > >

Protected Types inherited from mitk::EventStateMachine
typedef MessageAbstractDelegate2< StateMachineAction , InteractionEvent , void >	ActionFunctionDelegate

typedef MessageAbstractDelegate1< const InteractionEvent *, bool >	ConditionFunctionDelegate

Static Protected Member Functions inherited from mitk::SegWithPreviewTool
static Image::ConstPointer	GetImageByTimeStep (const Image *image, TimeStepType timestep)

static Image::Pointer	GetImageByTimeStep (Image *image, TimeStepType timestep)

static Image::ConstPointer	GetImageByTimePoint (const Image *image, TimePointType timePoint)

static void	TransferLabelInformation (const LabelMappingType &labelMapping, const mitk::MultiLabelSegmentation source, mitk::MultiLabelSegmentation target)
	Copies label information (not pixel content) from one segmentation to another. More...

Protected Attributes inherited from mitk::SegWithPreviewTool
ToolCommand::Pointer	m_ProgressCommand

Protected Attributes inherited from mitk::Tool
std::string	m_EventConfig
	Let subclasses change their event configuration. More...

Protected Attributes inherited from mitk::EventStateMachine
bool	m_IsActive

UndoController *	m_UndoController

bool	m_UndoEnabled

Detailed Description

Segmentation tool integrating the nnInteractive deep learning model.

This tool extends SegWithPreviewTool to provide AI-assisted interactive segmentation powered by the nnInteractive Python package. Users can provide prompts through various interaction types (points, boxes, scribbles, contours) to guide the segmentation model.

The tool manages:

A set of Interactor instances (one per InteractionType) for user input
A Python session that hosts the nnInteractive inference model
A virtual environment for Python dependency isolation
Backend selection (CUDA GPU or CPU)

The typical lifecycle is:

Create a Python context via CreatePythonContext()
Start a session via StartSession() (downloads and loads the model)
Enable an interactor via EnableInteractor() for user interaction
The tool automatically updates the preview segmentation on each interaction
End the session via EndSession() or tool deactivation

See also: SegWithPreviewTool, PythonContext, nnInteractive::Interactor, QmitknnInteractiveToolGUI

Definition at line 56 of file mitknnInteractiveTool.h.

Member Typedef Documentation

◆ InteractorMap

using mitk::nnInteractiveTool::InteractorMap = std::unordered_map<nnInteractive::InteractionType, std::unique_ptr<nnInteractive::Interactor> >

Map type associating InteractionType keys to Interactor instances.

Definition at line 119 of file mitknnInteractiveTool.h.

Constructor & Destructor Documentation

◆ nnInteractiveTool()

mitk::nnInteractiveTool::nnInteractiveTool ( )

protected

Default constructor. Initializes interactors and connects events.

◆ ~nnInteractiveTool()

mitk::nnInteractiveTool::~nnInteractiveTool ( )

overrideprotected

Destructor. Disconnects interactor events.

Member Function Documentation

◆ AbortSession()

void mitk::nnInteractiveTool::AbortSession ( )

Tears down a remote session whose connection was lost.

Disables the active interactor, ends the (now dead) Python session, clears all interactions/prompts and the preview, and refreshes the render windows. SessionEndedEvent (emitted by the internal EndSession) lets the GUI revert its session-dependent controls so the user only has to click Initialize to reconnect.

Warning: Must be called from a clean call stack (e.g. deferred from the GUI), never from within an interactor's event handling, because it disables and resets the interactors.

See also: SessionExpiredEvent, EndSession()

◆ CanUndo()

bool mitk::nnInteractiveTool::CanUndo ( ) const

Returns whether an interaction is currently available to undo.

true after a new interaction has been applied, until it is undone or the interactions are reset. Cheap (no Python call); reflects the tool's record of the last applied interaction.

Returns: true if UndoLastInteraction() would revert something.

See also: UndoLastInteraction(), SupportsUndo()

◆ ConfirmCleanUp()

void mitk::nnInteractiveTool::ConfirmCleanUp ( )

overrideprotectedvirtual

Called when a confirmed segmentation needs cleanup.

Emits the ConfirmCleanUpEvent to notify the GUI.

Reimplemented from mitk::SegWithPreviewTool.

◆ CreatePythonContext()

bool mitk::nnInteractiveTool::CreatePythonContext ( )

Creates and activates a Python context with a virtual environment.

The virtual environment name is determined by GetVirtualEnvName().

Returns: true if the Python context was successfully created, false if an error occurred.

See also: GetPythonContext(), PythonContext

◆ Deactivated()

void mitk::nnInteractiveTool::Deactivated ( )

overridevirtual

Called when the tool is deactivated.

Disables any active interactor, resets all interactions, and ends the Python session. Emits DeactivatedEvent at the end so observers can run cleanup that must not happen from the destructor path (e.g. removing unused auto-created labels in the GUI).

See also: DeactivatedEvent

Reimplemented from mitk::Tool.

◆ DisableInteractor()

void mitk::nnInteractiveTool::DisableInteractor ( std::optional< nnInteractive::InteractionType > interactionType = std::nullopt )

Disables an interactor.

If a specific interaction type is provided, that interactor is disabled. Otherwise, the currently enabled interactor (if any) is disabled.

Parameters

[in] interactionType The interaction type to disable. If std::nullopt, the currently enabled interactor is disabled.

See also: EnableInteractor()

◆ DoUpdatePreview()

void mitk::nnInteractiveTool::DoUpdatePreview	(	const Image *	inputAtTimeStep,
		const Image *	oldSegAtTimeStep,
		MultiLabelSegmentation *	previewImage,
		TimeStepType	timeStep
	)

overrideprotectedvirtual

Updates the preview segmentation based on the latest interaction.

Dispatches the latest interaction data to the nnInteractive Python model and writes the prediction result into the preview image.

Parameters

[in]	inputAtTimeStep	The input image at the current time step.
[in]	oldSegAtTimeStep	The previous segmentation at the current time step (unused).
[in,out]	previewImage	The preview segmentation to update with the prediction result.
[in]	timeStep	The current time step.

Implements mitk::SegWithPreviewTool.

◆ EnableInteractor()

void mitk::nnInteractiveTool::EnableInteractor	(	nnInteractive::InteractionType	nextInteractionType,
		nnInteractive::PromptType	promptType
	)

Enables an interactor for the given interaction and prompt type.

Any other currently enabled interactor is automatically disabled first. Only one interactor may be active at a time.

Parameters

[in]	nextInteractionType	The interaction type to enable.
[in]	promptType	The prompt type (positive or negative) for the interaction.

See also: DisableInteractor()

◆ EndSession()

void mitk::nnInteractiveTool::EndSession ( )

Ends the current nnInteractive inference session.

For a local session it cleans up the Python session, releases model resources, and optionally empties the CUDA cache. For a remote session it releases the server lease (best-effort and idempotent) so the slot frees up for other users. This is a no-op if no session is running.

See also: StartSession(), IsSessionRunning()

◆ GetAutoRefine()

bool mitk::nnInteractiveTool::GetAutoRefine ( ) const

Returns whether auto-refine is enabled.

When auto-refine is enabled, predictions are automatically run on initial segmentation masks provided via InitializeSessionWithMask().

Returns: true if auto-refine is enabled, false otherwise.

See also: SetAutoRefine()

◆ GetAutoZoom()

bool mitk::nnInteractiveTool::GetAutoZoom ( ) const

Returns whether auto-zoom is enabled.

When auto-zoom is enabled, the nnInteractive model automatically zooms into the region of interest during inference.

Returns: true if auto-zoom is enabled, false otherwise.

See also: SetAutoZoom()

◆ GetBackend()

std::optional<nnInteractive::Backend> mitk::nnInteractiveTool::GetBackend ( ) const

Returns the currently active computation backend.

The backend is determined during StartSession() based on hardware availability and user preferences.

Returns: The active Backend, or std::nullopt if no session has been started.

◆ GetCUDADeviceInfo()

bool mitk::nnInteractiveTool::GetCUDADeviceInfo ( CUDADeviceInfo & info ) const

Queries CUDA device information via PyTorch.

Checks for CUDA availability and retrieves device properties from the GPU device specified in the application preferences.

Parameters

[out] info The CUDADeviceInfo struct to populate with device information.

Precondition: A Python context must have been created via CreatePythonContext().

Returns: true if a CUDA device was found and info was populated, false otherwise.

◆ GetHeartbeatIntervalMs()

int mitk::nnInteractiveTool::GetHeartbeatIntervalMs ( ) const

Returns the interval, in milliseconds, at which Heartbeat() should be called for the running remote session.

Derived from the server-provided liveness timeout: half of it, but never less often than every 5 seconds (max(5 s, liveness / 2)), mirroring the client library's own cadence. Returns 0 for local sessions, when no session is running, or when the server has disabled the liveness timeout (no heartbeat needed). The GUI starts a timer only when this is greater than zero.

See also: Heartbeat()

◆ GetIconResource()

us::ModuleResource mitk::nnInteractiveTool::GetIconResource ( ) const

overridevirtual

Returns the icon resource for this tool.

Loads the "AI.svg" icon from the MitkSegmentation module resources.

Returns: The icon as a us::ModuleResource.

Reimplemented from mitk::Tool.

◆ GetInteractor()

const nnInteractive::Interactor* mitk::nnInteractiveTool::GetInteractor ( nnInteractive::InteractionType interactionType ) const

Returns the interactor for a specific interaction type.

Parameters

[in] interactionType The interaction type to look up.

Returns: Pointer to the corresponding Interactor.

Exceptions

std::out_of_range if the interaction type is not registered.

◆ GetInteractors()

const InteractorMap& mitk::nnInteractiveTool::GetInteractors ( ) const

Returns a reference to the map of all interactors.

Returns: A const reference to the InteractorMap containing all registered interactors.

◆ GetModelLicense()

std::optional<std::string> mitk::nnInteractiveTool::GetModelLicense ( ) const

Returns the running session's model checkpoint license, if any.

Reads the nnInteractive session's license attribute, populated identically for local sessions (first line of the checkpoint's LICENSE file, or a fallback for the official model) and remote sessions (mirrored from the server's /capabilities response). The special value "!!MISSING!!" means the license is unknown. Returns std::nullopt when no session is running or no license is reported. This is a cached local attribute read (no network call).

See also: GetSupportedInteractions()

◆ GetName()

const char* mitk::nnInteractiveTool::GetName ( ) const

overridevirtual

Returns the display name of this tool.

Returns: The string "nnInteractive".

Implements mitk::Tool.

◆ GetPythonContext()

PythonContext* mitk::nnInteractiveTool::GetPythonContext ( ) const

Returns the current Python context.

Returns: Pointer to the PythonContext, or nullptr if none has been created yet.

See also: CreatePythonContext()

◆ GetSessionDefiningPreferences()

static const std::vector<std::pair<std::string, std::string> >& mitk::nnInteractiveTool::GetSessionDefiningPreferences ( )

static

The preferences baked into a session at initialization, each paired with its default value (as stored).

Single source of truth shared by the tool and its GUI:

StartSession() seeds these so the stored values match what the session actually uses. Otherwise the first preferences "OK" after a fresh install writes a default into a never-stored key, which mitk::Preferences reports as a value change and which the GUI would act on by ending the session.
The GUI ends a running session whenever one of these keys changes, since the session can no longer match the new settings.

Live or GUI-only settings (auto-zoom, auto-refine, automation, shortcut labels, install mode) are deliberately excluded.

◆ GetSupportedInteractions()

SupportedInteractions mitk::nnInteractiveTool::GetSupportedInteractions ( ) const

Returns which interaction types the running session supports.

Reads the session's capability metadata. If no session is running, all interactions are reported as supported (the defaults).

Returns: A SupportedInteractions struct.

See also: SupportedInteractions

◆ GetVirtualEnvName()

std::string mitk::nnInteractiveTool::GetVirtualEnvName ( ) const

Returns the name of the Python virtual environment used.

The virtual environment name matches the tool's display name ("nnInteractive").

Returns: The virtual environment name as a string.

◆ HasInteractions()

bool mitk::nnInteractiveTool::HasInteractions ( ) const

Checks whether any interactor has recorded interactions.

Also returns true if an initial segmentation mask has been set via InitializeSessionWithMask().

Returns: true if any interactions or initial mask exist, false otherwise.

◆ Heartbeat()

void mitk::nnInteractiveTool::Heartbeat ( )

Sends a single keep-alive heartbeat for a running remote session.

Remote nnInteractive sessions are reaped server-side after a liveness timeout unless the client periodically proves it is alive. The client library ships its own background heartbeat daemon, but that Python thread cannot run while MITK is idle: the embedded interpreter holds the GIL on the Qt main thread between PythonContext::Execute() calls, so the daemon never gets to beat. This method drives the heartbeat from the Qt event loop instead and must be called periodically (see GetHeartbeatIntervalMs) by the GUI for as long as a remote session is running.

A transient transport error is tolerated (the next beat retries), matching the library's own heartbeat loop. A definitive lease expiry emits SessionExpiredEvent so the GUI tears the dead session down. This is a no-op for local sessions or when no session is running, and never throws (it is invoked from a Qt timer slot).

See also: GetHeartbeatIntervalMs(), SessionExpiredEvent

◆ InitializeSessionWithMask()

void mitk::nnInteractiveTool::InitializeSessionWithMask ( Image * mask )

Initializes or reinitializes the session with an existing mask.

Provides an existing segmentation mask as an initial segmentation to the nnInteractive model, allowing users to refine an existing segmentation. The mask replaces any previous initial segmentation and triggers a preview update.

Parameters

[in] mask Pointer to the mask Image to use as initial segmentation. If nullptr, this is a no-op.

See also: ResetInteractions()

◆ IsInstalled()

bool mitk::nnInteractiveTool::IsInstalled ( ) const

Checks whether the nnInteractive Python package is installed.

Uses Python's importlib to check for the presence of the nnInteractive package in the active virtual environment.

Precondition: A Python context must have been created via CreatePythonContext().

Returns: true if the package is installed, false otherwise.

◆ IsLocalInferenceAvailable()

bool mitk::nnInteractiveTool::IsLocalInferenceAvailable ( ) const

Checks whether local (in-process) inference is available.

Local inference is provided by the full "nnInteractive" distribution (which installs PyTorch and the inference backend); the lightweight client-only "nninteractive-client" distribution shares the same "nnInteractive" import namespace but offers remote sessions only. Detection reads the installed distribution metadata rather than importing the package, so it stays cheap and, crucially, has no import side effects: importing an nnInteractive submodule would map native libraries from the virtual environment and block a later in-place update on Windows.

Precondition: A Python context must have been created via CreatePythonContext().

Returns: true if local inference is available (full install), false for a client-only install.

◆ IsRemoteSession()

bool mitk::nnInteractiveTool::IsRemoteSession ( ) const

Returns whether the running session is a remote (server) session.

Returns: true if the current session was started in remote mode (i.e. it talks to an nninteractive-server), false for a local in-process session or when no session is running.

See also: StartSession()

◆ IsSessionRunning()

bool mitk::nnInteractiveTool::IsSessionRunning ( ) const

Checks whether an nnInteractive session is currently running.

Returns: true if a Python context exists and contains a valid session variable, false otherwise.

◆ mitkClassMacro()

mitk::nnInteractiveTool::mitkClassMacro	(	nnInteractiveTool	,
		SegWithPreviewTool
	)

◆ Notify()

void mitk::nnInteractiveTool::Notify	(	InteractionEvent *	event,
		bool	isHandled
	)

overrideprotectedvirtual

Forwards unhandled interaction events to the enabled interactor.

Parameters

[in]	event	The interaction event to process.
[in]	isHandled	Whether the event has already been handled.

Implements mitk::InteractionEventObserver.

◆ OnTimePointChanged()

void mitk::nnInteractiveTool::OnTimePointChanged ( )

overrideprotectedvirtual

Tears the running session down when the current time step no longer matches the time step the session was bound to.

The Python inference model is intrinsically 3D, so each session is bound to a single 3D slice extracted at the time step that was active when StartSession() ran. If the current image or segmentation time step diverges from that bound time step, all unconfirmed interactions are discarded, the preview is cleared, and the session is ended. This override deliberately does not chain to the base implementation, whose lazy-preview UpdatePreview() call would invoke DoUpdatePreview() against a stale Python binding.

Reimplemented from mitk::SegWithPreviewTool.

◆ ResetInteractions()

void mitk::nnInteractiveTool::ResetInteractions ( )

Resets all interactors and clears the current preview.

All interaction data (points, boxes, scribbles, contours) is removed and the Python session's interaction state is reset. Also clears any initial segmentation mask.

◆ SetAutoRefine()

void mitk::nnInteractiveTool::SetAutoRefine ( bool autoRefine )

Enables or disables auto-refine.

Parameters

[in] autoRefine true to enable auto-refine, false to disable.

See also: GetAutoRefine()

◆ SetAutoZoom()

void mitk::nnInteractiveTool::SetAutoZoom ( bool autoZoom )

Enables or disables auto-zoom.

If a session is currently running, the setting is applied immediately to the Python session.

Parameters

[in] autoZoom true to enable auto-zoom, false to disable.

Note: Auto-zoom is automatically disabled when falling back to CPU backend.

See also: GetAutoZoom()

◆ SetPreviewLabel()

void mitk::nnInteractiveTool::SetPreviewLabel	(	MultiLabelSegmentation::LabelValueType	value,
		const Color &	color
	)

protected

Configures a label in the preview segmentation with a given value and color.

Creates the label if it does not yet exist, updates its color, and sets it as the active label.

Parameters

[in]	value	The label pixel value.
[in]	color	The color to assign to the label.

◆ SetToolManager()

void mitk::nnInteractiveTool::SetToolManager ( ToolManager * toolManager )

overrideprotectedvirtual

Sets the ToolManager and forwards it to all interactors.

Parameters

[in] toolManager Pointer to the ToolManager to associate with this tool and its interactors.

Reimplemented from mitk::Tool.

◆ StartSession()

void mitk::nnInteractiveTool::StartSession ( )

Starts an nnInteractive inference session in the configured mode.

In local mode it downloads/loads the model checkpoint and initializes the inference session on a CUDA or CPU backend (determined from CUDA device availability and user preferences). In remote mode it claims a session on the configured nninteractive-server, which provides the model and compute. In both modes the reference image is bound to the session.

If a session is already running, it is ended first.

Precondition: A Python context must have been created via CreatePythonContext().; A reference image must be available through the ToolManager.

Exceptions

mitk::Exception if the session setup fails (local or remote).

See also: EndSession(), IsSessionRunning(), IsRemoteSession()

◆ SupportsUndo()

bool mitk::nnInteractiveTool::SupportsUndo ( ) const

Returns whether the running session's backend supports undo.

Reads the session's supports_undo capability (present on both local and remote sessions). Older nnInteractive versions without undo report false, letting the GUI keep the Undo control disabled. Returns false when no session is running.

See also: UndoLastInteraction()

◆ UndoLastInteraction()

void mitk::nnInteractiveTool::UndoLastInteraction ( )

Reverts the most recent interaction (single-level undo).

Calls the nnInteractive session's undo(), which restores the prediction to the state before the last interaction (point, box, scribble, lasso, or initial-segmentation mask). On success, the visualization of the undone prompt is removed and the preview is refreshed from the restored target buffer.

nnInteractive supports only single-level undo: after one undo, nothing is undoable again until a new interaction is added. This is a no-op when no session is running or CanUndo() is false.

See also: CanUndo(), SupportsUndo()

Member Data Documentation

◆ ConfirmCleanUpEvent

Message1<bool> mitk::nnInteractiveTool::ConfirmCleanUpEvent

Event triggered when the tool requests cleanup confirmation.

Emitted when the segmentation is confirmed and the tool needs to clean up interactor data. The boolean parameter indicates whether the cleanup was confirmed.

Definition at line 507 of file mitknnInteractiveTool.h.

◆ DeactivatedEvent

Message mitk::nnInteractiveTool::DeactivatedEvent

Event triggered at the end of Deactivated().

Emitted after the tool has released interactors and ended its Python session. GUI code can subscribe to run teardown logic that must happen on a proper user-initiated deactivation (as opposed to the GUI's Qt destructor, which may run during application shutdown when observers and widgets are in a partially-destructed state).

Definition at line 527 of file mitknnInteractiveTool.h.

◆ PreviewUpdatedEvent

Message mitk::nnInteractiveTool::PreviewUpdatedEvent

Event triggered after the preview has been updated from a user interaction.

Emitted at the end of DoUpdatePreview() when the update was driven by an enabled interactor (i.e., the user placed a point/box/scribble/lasso) and the prediction result has been written to the preview image.

Not emitted for initial-mask-based updates or reset operations.

Definition at line 517 of file mitknnInteractiveTool.h.

◆ SessionEndedEvent

Message mitk::nnInteractiveTool::SessionEndedEvent

Event triggered when the Python session has ended.

Emitted from EndSession() after the Python session has been torn down. Fires both on a normal Deactivated() (in which case DeactivatedEvent follows) and when the session is auto-ended because the current image or segmentation time step changed during a 4D session. GUI code can subscribe to revert session-dependent UI controls to their pre-init state.

Definition at line 538 of file mitknnInteractiveTool.h.

◆ SessionExpiredEvent

Message mitk::nnInteractiveTool::SessionExpiredEvent

Event triggered when a remote session was lost server-side.

Emitted while the (now dead) remote session is still nominally running, when a remote operation or the heartbeat failed because the lease expired or the server is at capacity / unreachable. Teardown has NOT happened yet: the GUI must defer it, calling AbortSession() on the next event-loop tick (teardown must not run inside an interactor's event handling), which then ends the session and fires SessionEndedEvent. Do not assume the session is already ended when this fires.

See also: SessionEndedEvent, AbortSession()

Definition at line 552 of file mitknnInteractiveTool.h.

The documentation for this class was generated from the following file:

mitknnInteractiveTool.h

Classes

Public Types

Public Member Functions

Static Public Member Functions

Public Attributes

Protected Member Functions

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ InteractorMap

Constructor & Destructor Documentation

◆ nnInteractiveTool()

◆ ~nnInteractiveTool()

Member Function Documentation

◆ AbortSession()

◆ CanUndo()

◆ ConfirmCleanUp()

◆ CreatePythonContext()

◆ Deactivated()

◆ DisableInteractor()

◆ DoUpdatePreview()

◆ EnableInteractor()

◆ EndSession()

◆ GetAutoRefine()

◆ GetAutoZoom()

◆ GetBackend()

◆ GetCUDADeviceInfo()

◆ GetHeartbeatIntervalMs()

◆ GetIconResource()

◆ GetInteractor()

◆ GetInteractors()

◆ GetModelLicense()

◆ GetName()

◆ GetPythonContext()

◆ GetSessionDefiningPreferences()

◆ GetSupportedInteractions()

◆ GetVirtualEnvName()

◆ HasInteractions()

◆ Heartbeat()

◆ InitializeSessionWithMask()

◆ IsInstalled()

◆ IsLocalInferenceAvailable()

◆ IsRemoteSession()

◆ IsSessionRunning()

◆ mitkClassMacro()

◆ Notify()

◆ OnTimePointChanged()

◆ ResetInteractions()

◆ SetAutoRefine()

◆ SetAutoZoom()

◆ SetPreviewLabel()

◆ SetToolManager()

◆ StartSession()

◆ SupportsUndo()

◆ UndoLastInteraction()

Member Data Documentation

◆ ConfirmCleanUpEvent

◆ DeactivatedEvent

◆ PreviewUpdatedEvent

◆ SessionEndedEvent

◆ SessionExpiredEvent