InteractiveSegmenter

public final class InteractiveSegmenter

Performs interactive segmentation on images.

Note that, in addition to the standard segmentation API ERROR(/segment) that takes an input image and returns the outputs, but involves deep copy of the returns, InteractiveSegmenter also supports the callback API, ERROR(/segmentWithResultListener), which allows you to access the outputs through zero copy. Set OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions properly to use the callback API.

The API expects a TFLite model with,TFLite Model Metadata.. The model expects input with 4 channels, where the first 3 channels represent RGB image, and the last channel represents the user's region of interest.

• Input image MPImage
  • The image that image segmenter runs on.
• Input roi InteractiveSegmenter.RegionOfInterest
  • Region of interest based on user interaction.
• Output ImageSegmenterResult ImageSegmenterResult
  • An ImageSegmenterResult containing segmented masks.

Nested Classes

class	InteractiveSegmenter.InteractiveSegmenterOptions		Options for setting up an InteractiveSegmenter.
class	InteractiveSegmenter.RegionOfInterest		The Region-Of-Interest (ROI) to interact with.

Public Methods

static InteractiveSegmenter	createFromOptions(Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions) Creates an InteractiveSegmenter instance from an InteractiveSegmenter.InteractiveSegmenterOptions.
ImageSegmenterResult	segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi) Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e.
ImageSegmenterResult	segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions) Performs segmentation on the provided single image, given user's region-of-interest.
void	segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi) Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e.
void	segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions) Performs segmentation on the provided single image given user's region-of-interest, and provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.

Inherited Methods

From class com.google.mediapipe.tasks.vision.core.BaseVisionTaskApi

void	close() Closes and cleans up the MediaPipe vision task.
static long	generateResultTimestampMs(RunningMode runningMode, Packet packet) Generates the timestamp of a vision task result object from the vision task running mode and the output packet.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.AutoCloseable

abstract void

close()

Public Methods

public static InteractiveSegmenter createFromOptions (Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)

Creates an InteractiveSegmenter instance from an InteractiveSegmenter.InteractiveSegmenterOptions.

Parameters

context	an Android ERROR(/Context).
segmenterOptions	an InteractiveSegmenter.InteractiveSegmenterOptions instance.

Throws

	if there is an error during InteractiveSegmenter creation.

public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi)

Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied. TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
roi	a InteractiveSegmenter.RegionOfInterest object to represent user interaction.

Throws

	if there is an internal error. Or if InteractiveSegmenter is created with a OutputHandler.ResultListener.

public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)

Performs segmentation on the provided single image, given user's region-of-interest. TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
roi	a InteractiveSegmenter.RegionOfInterest object to represent user interaction.
imageProcessingOptions	the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.

Throws

IllegalArgumentException	if the ImageProcessingOptions specify a region-of-interest.
	if there is an internal error. Or if InteractiveSegmenter is created with a OutputHandler.ResultListener.

public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi)

Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied, and provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.

TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
roi	a InteractiveSegmenter.RegionOfInterest object to represent user interaction.

Throws

IllegalArgumentException	if the ImageProcessingOptions specify a region-of-interest.
	if there is an internal error. Or if InteractiveSegmenter is not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions.

public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)

Performs segmentation on the provided single image given user's region-of-interest, and provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.

TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
roi	a InteractiveSegmenter.RegionOfInterest object to represent user interaction.
imageProcessingOptions	the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.

Throws

IllegalArgumentException	if the ImageProcessingOptions specify a region-of-interest.
	if there is an internal error. Or if InteractiveSegmenter is not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions.