Image Embedder

The MediaPipe Image Embedder task lets you create a numeric representation of an image, which is useful in accomplishing various ML-based image tasks. This functionality is frequently used to compare the similarity of two images using mathematical comparison techniques such as Cosine Similarity. This task operates on image data with a machine learning (ML) model as static data or a continuous stream, and outputs a numeric representation of the image data as a list of high-dimensional feature vectors, also known as embedding vectors, in either floating-point or quantized form.

Open in Web DemosEdit in StackBlitz

Quickstart

Install the SDK for your platform, prepare input data, choose a running mode, and run inference.

Web

npmnpm install @mediapipe/tasks-visionWeb demo·Edit in StackBlitz·Setup guide

Image

IMAGEStateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
import { FilesetResolver, ImageEmbedder } from "@mediapipe/tasks-vision";

const vision = await FilesetResolver.forVisionTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
const imageEmbedder = await ImageEmbedder.createFromOptions(vision, {
  baseOptions: { modelAssetPath: "mobilenet_v3_small.tflite", delegate: "GPU" },
  runningMode: "IMAGE",
  l2Normalize: true,
});

// Synchronous single-frame call (stateless across images)
const image = document.getElementById("image") as HTMLImageElement;
const otherImage = document.getElementById("other-image") as HTMLImageElement;
const result = imageEmbedder.embed(image);
const otherResult = imageEmbedder.embed(otherImage);
const similarity = ImageEmbedder.cosineSimilarity(result.embeddings[0], otherResult.embeddings[0]);

Video

VIDEOEvery frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
import { FilesetResolver, ImageEmbedder } from "@mediapipe/tasks-vision";

const vision = await FilesetResolver.forVisionTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
const imageEmbedder = await ImageEmbedder.createFromOptions(vision, {
  baseOptions: { modelAssetPath: "mobilenet_v3_small.tflite", delegate: "GPU" },
  runningMode: "VIDEO",
  l2Normalize: true,
});

// Synchronous across decoded frames with monotonic timestamps (never drops frames)
for (const { frame, timestampMs } of decodedVideoFrames) {
  const result = imageEmbedder.embedForVideo(frame, timestampMs);
}

Livestream

VIDEO + rAFLive camera · latest frameWeb tasks run synchronously—pair runningMode: "VIDEO" with requestAnimationFrame so each paint inspects only the latest camera frame (skipping intermediate frames when busy).
import { FilesetResolver, ImageEmbedder } from "@mediapipe/tasks-vision";

const vision = await FilesetResolver.forVisionTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
// On Web, live camera loops pair runningMode: "VIDEO" with requestAnimationFrame
const imageEmbedder = await ImageEmbedder.createFromOptions(vision, {
  baseOptions: { modelAssetPath: "mobilenet_v3_small.tflite", delegate: "GPU" },
  runningMode: "VIDEO",
  l2Normalize: true,
});

const video = document.getElementById("webcam") as HTMLVideoElement;
video.srcObject = await navigator.mediaDevices.getUserMedia({ video: true });
await video.play();
let lastVideoTime = -1;

function renderLoop() {
  if (video.readyState >= 2 && video.currentTime !== lastVideoTime) {
    lastVideoTime = video.currentTime;
    const result = imageEmbedder.embedForVideo(video, performance.now());
    console.log(result.embeddings[0]?.floatEmbedding);
  }
  requestAnimationFrame(renderLoop);
}
renderLoop();

Android (Kotlin)

Gradleimplementation("com.google.mediapipe:tasks-vision:latest.release")Sample code·Setup guide

Image

IMAGEStateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedder

val baseOptions = BaseOptions.builder()
    .setModelAssetPath("mobilenet_v3_small.tflite")
    .build()
val options = ImageEmbedder.ImageEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.IMAGE)
    .setL2Normalize(true)
    .setQuantize(true)
    .build()

val imageEmbedder = ImageEmbedder.createFromOptions(context, options)

// Synchronous single-image call (no temporal tracking across calls)
val mpImage = BitmapImageBuilder(bitmap).build()
val otherMpImage = BitmapImageBuilder(otherBitmap).build()
val result = imageEmbedder.embed(mpImage)
val otherResult = imageEmbedder.embed(otherMpImage)
val similarity = ImageEmbedder.cosineSimilarity(
    result.embeddingResult().embeddings().first(),
    otherResult.embeddingResult().embeddings().first()
)

Video

VIDEOEvery frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedder

val baseOptions = BaseOptions.builder()
    .setModelAssetPath("mobilenet_v3_small.tflite")
    .build()
val options = ImageEmbedder.ImageEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.VIDEO)
    .setL2Normalize(true)
    .setQuantize(true)
    .build()

val imageEmbedder = ImageEmbedder.createFromOptions(context, options)

// Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
val mpImage = BitmapImageBuilder(frameBitmap).build()
val result = imageEmbedder.embedForVideo(mpImage, frameTimestampMs)

Livestream

LIVE_STREAMAsync · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
import android.os.SystemClock
import android.util.Log
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedder

val baseOptions = BaseOptions.builder()
    .setModelAssetPath("mobilenet_v3_small.tflite")
    .build()
val options = ImageEmbedder.ImageEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.LIVE_STREAM)
    .setL2Normalize(true)
    .setQuantize(true)
    .setResultListener { result, _ -> val embedding = result.embeddingResult().embeddings().firstOrNull() }
    .setErrorListener { error -> Log.e("MediaPipe", error.message.orEmpty()) }
    .build()

val imageEmbedder = ImageEmbedder.createFromOptions(context, options)

// Async: automatically drops incoming camera frames when busy so results stay current
imageEmbedder.embedAsync(mpImage, SystemClock.uptimeMillis())

Android (Java)

Gradleimplementation 'com.google.mediapipe:tasks-vision:latest.release'Sample code·Setup guide

Image

IMAGEStateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.vision.core.RunningMode;
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedder;
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedderResult;

BaseOptions baseOptions = BaseOptions.builder()
    .setModelAssetPath("mobilenet_v3_small.tflite")
    .build();
ImageEmbedder.ImageEmbedderOptions options = ImageEmbedder.ImageEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.IMAGE)
    .setL2Normalize(true)
    .setQuantize(true)
    .build();

ImageEmbedder imageEmbedder = ImageEmbedder.createFromOptions(context, options);

// Synchronous single-image call (no temporal tracking across calls)
MPImage mpImage = new BitmapImageBuilder(bitmap).build();
MPImage otherMpImage = new BitmapImageBuilder(otherBitmap).build();
ImageEmbedderResult result = imageEmbedder.embed(mpImage);
ImageEmbedderResult otherResult = imageEmbedder.embed(otherMpImage);
double similarity = ImageEmbedder.cosineSimilarity(
    result.embeddingResult().embeddings().get(0),
    otherResult.embeddingResult().embeddings().get(0)
);

Video

VIDEOEvery frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.vision.core.RunningMode;
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedder;
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedderResult;

BaseOptions baseOptions = BaseOptions.builder()
    .setModelAssetPath("mobilenet_v3_small.tflite")
    .build();
ImageEmbedder.ImageEmbedderOptions options = ImageEmbedder.ImageEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.VIDEO)
    .setL2Normalize(true)
    .setQuantize(true)
    .build();

ImageEmbedder imageEmbedder = ImageEmbedder.createFromOptions(context, options);

// Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
MPImage mpImage = new BitmapImageBuilder(frameBitmap).build();
ImageEmbedderResult result = imageEmbedder.embedForVideo(mpImage, frameTimestampMs);

Livestream

LIVE_STREAMAsync · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
import android.os.SystemClock;
import android.util.Log;
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.vision.core.RunningMode;
import com.google.mediapipe.tasks.vision.imageembedder.ImageEmbedder;

BaseOptions baseOptions = BaseOptions.builder()
    .setModelAssetPath("mobilenet_v3_small.tflite")
    .build();
ImageEmbedder.ImageEmbedderOptions options = ImageEmbedder.ImageEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.LIVE_STREAM)
    .setL2Normalize(true)
    .setQuantize(true)
    .setResultListener((result, inputImage) -> { var embeddings = result.embeddingResult().embeddings(); })
    .setErrorListener(error -> Log.e("MediaPipe", String.valueOf(error.getMessage())))
    .build();

ImageEmbedder imageEmbedder = ImageEmbedder.createFromOptions(context, options);

// Async: automatically drops incoming camera frames when busy so results stay current
imageEmbedder.embedAsync(mpImage, SystemClock.uptimeMillis());

iOS

SwiftPMhttps://github.com/google-ai-edge/mediapipe → MediaPipeTasksVisionPackage.swift·Sample code·Setup guide

Image

IMAGEStateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
import Foundation
import MediaPipeTasksVision

guard let modelPath = Bundle.main.path(forResource: "mobilenet_v3_small", ofType: "tflite") else { return }

let options = ImageEmbedderOptions()
options.baseOptions.modelAssetPath = modelPath
options.runningMode = .image
options.l2Normalize = true
options.quantize = true

let imageEmbedder = try ImageEmbedder(options: options)

// Synchronous single-image call (stateless across images)
let mpImage = try MPImage(uiImage: image)
let otherMpImage = try MPImage(uiImage: otherImage)
let result = try imageEmbedder.embed(image: mpImage)
let otherResult = try imageEmbedder.embed(image: otherMpImage)
if let firstEmbed = result.embeddingResult.embeddings.first,
   let secondEmbed = otherResult.embeddingResult.embeddings.first {
    let similarity = try ImageEmbedder.cosineSimilarity(embedding1: firstEmbed, embedding2: secondEmbed)
}

Video

VIDEOEvery frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
import Foundation
import MediaPipeTasksVision

guard let modelPath = Bundle.main.path(forResource: "mobilenet_v3_small", ofType: "tflite") else { return }

let options = ImageEmbedderOptions()
options.baseOptions.modelAssetPath = modelPath
options.runningMode = .video
options.l2Normalize = true
options.quantize = true

let imageEmbedder = try ImageEmbedder(options: options)

// Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
let mpImage = try MPImage(pixelBuffer: pixelBuffer)
let result = try imageEmbedder.embed(videoFrame: mpImage, timestampInMilliseconds: timestampMs)

Livestream

LIVE_STREAMAsync · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
import Foundation
import MediaPipeTasksVision

final class ImageEmbedderStreamService: NSObject {
    private var imageEmbedder: ImageEmbedder?

    func setUp() throws {
        guard let modelPath = Bundle.main.path(
            forResource: "mobilenet_v3_small",
            ofType: "tflite"
        ) else { return }

        let options = ImageEmbedderOptions()
        options.baseOptions.modelAssetPath = modelPath
        options.runningMode = .liveStream
        options.l2Normalize = true
        options.quantize = true
        options.imageEmbedderLiveStreamDelegate = self

        imageEmbedder = try ImageEmbedder(options: options)
    }

    func embedAsync(mpImage: MPImage, timestampMs: Int) throws {
        // Async: automatically drops incoming camera frames when busy so results stay current
        try imageEmbedder?.embedAsync(image: mpImage, timestampInMilliseconds: timestampMs)
    }
}

extension ImageEmbedderStreamService: ImageEmbedderLiveStreamDelegate {
    func imageEmbedder(
        _ imageEmbedder: ImageEmbedder,
        didFinishEmbedding result: ImageEmbedderResult?,
        timestampInMilliseconds: Int,
        error: (any Error)?
    ) {
        guard let result else { return }
        DispatchQueue.main.async {
            let embedding = result.embeddingResult.embeddings.first
        }
    }
}

Python

pippip install mediapipeSample code·Setup guide

Image

IMAGEStateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
import mediapipe as mp

BaseOptions = mp.tasks.BaseOptions
ImageEmbedder = mp.tasks.vision.ImageEmbedder
ImageEmbedderOptions = mp.tasks.vision.ImageEmbedderOptions
VisionRunningMode = mp.tasks.vision.RunningMode

options = ImageEmbedderOptions(
    base_options=BaseOptions(model_asset_path="mobilenet_v3_small.tflite"),
    running_mode=VisionRunningMode.IMAGE,
    l2_normalize=True,
    quantize=True,
)

with ImageEmbedder.create_from_options(options) as embedder:
    # Synchronous single-image call (stateless across images)
    mp_image = mp.Image.create_from_file("input.jpg")
    other_image = mp.Image.create_from_file("other.jpg")
    result = embedder.embed(mp_image)
    other_result = embedder.embed(other_image)
    similarity = ImageEmbedder.cosine_similarity(
        result.embeddings[0], other_result.embeddings[0]
    )

Video

VIDEOEvery frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
import mediapipe as mp

BaseOptions = mp.tasks.BaseOptions
ImageEmbedder = mp.tasks.vision.ImageEmbedder
ImageEmbedderOptions = mp.tasks.vision.ImageEmbedderOptions
VisionRunningMode = mp.tasks.vision.RunningMode

options = ImageEmbedderOptions(
    base_options=BaseOptions(model_asset_path="mobilenet_v3_small.tflite"),
    running_mode=VisionRunningMode.VIDEO,
    l2_normalize=True,
    quantize=True,
)

with ImageEmbedder.create_from_options(options) as embedder:
    # Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
    for mp_image, timestamp_ms in decoded_video_frames:
        result = embedder.embed_for_video(mp_image, timestamp_ms)

Livestream

LIVE_STREAMAsync · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
import mediapipe as mp

BaseOptions = mp.tasks.BaseOptions
ImageEmbedder = mp.tasks.vision.ImageEmbedder
ImageEmbedderOptions = mp.tasks.vision.ImageEmbedderOptions
VisionRunningMode = mp.tasks.vision.RunningMode

def on_result(result, output_image: mp.Image, timestamp_ms: int):
    print(f"Embedding dimension at {timestamp_ms} ms: {len(result.embeddings[0].embedding)}")

options = ImageEmbedderOptions(
    base_options=BaseOptions(model_asset_path="mobilenet_v3_small.tflite"),
    running_mode=VisionRunningMode.LIVE_STREAM,
    l2_normalize=True,
    quantize=True,
    result_callback=on_result,
)

with ImageEmbedder.create_from_options(options) as embedder:
    # Async: automatically drops incoming camera frames when busy so results stay current
    mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb_frame)
    embedder.embed_async(mp_image, timestamp_ms)

Task details

This section describes the capabilities, inputs, outputs, and configuration options of this task.

Features

  • Input image processing - Processing includes image rotation, resizing, normalization, and color space conversion.
  • Region of interest - Performs embedding on a region of the image instead of the whole image.
  • Embedding similarity computation - Built-in utility function to compute the cosine similarity between two feature vectors.
  • Quantization - Supports scalar quantization for the feature vectors.
Inputs
The Image Embedder accepts an input of one of the following data types:
  • Still images
  • Decoded video frames
  • Live video feed

See Prepare vision input data for MPImage, mp.Image, and browser media examples.

Outputs
The Image Embedder outputs a list of embeddings consisting of:
  • Embedding: the feature vector itself, either in floating-point form or scalar-quantized.
  • Head index: the index for the head that produced this embedding.
  • Head name (optional): the name of the head that produced this embedding.

Configuration options

This task has the following configuration options:

Web

Option Name Description Value Range Default Value
runningMode Sets the running mode for the task:

IMAGE: Single image inputs.
VIDEO: Decoded video frames and live camera streams.		 {IMAGE, VIDEO} IMAGE
l2Normalize Whether to normalize the returned feature vector with L2 norm. Use only if the model does not already include a built-in L2_NORMALIZATION op. Boolean false
quantize Whether to scalar-quantize the returned embedding to bytes (assumes unit-norm vectors in [-1.0, 1.0]; pair with l2Normalize if needed). Boolean false

Android

Option Name Description Value Range Default Value
runningMode Sets the running mode for the task:

IMAGE: Single image inputs.
VIDEO: Decoded frames of a video.
LIVE_STREAM: Livestream of camera input data (requires asynchronous listener/delegate).		 {IMAGE, VIDEO, LIVE_STREAM} IMAGE
l2Normalize Whether to normalize the returned feature vector with L2 norm. Use only if the model does not already include a built-in L2_NORMALIZATION op. Boolean false
quantize Whether to scalar-quantize the returned embedding to bytes (assumes unit-norm vectors in [-1.0, 1.0]; pair with l2Normalize if needed). Boolean false
resultListener Asynchronous callback for LIVE_STREAM mode results. OutputHandler.ResultListener<ImageEmbedderResult, MPImage> Not set
errorListener Optional error listener for LIVE_STREAM mode. ErrorListener Not set

iOS

Option Name Description Value Range Default Value
runningMode Sets the running mode for the task:

IMAGE: Single image inputs.
VIDEO: Decoded frames of a video.
LIVE_STREAM: Livestream of camera input data (requires asynchronous listener/delegate).		 {IMAGE, VIDEO, LIVE_STREAM} IMAGE
l2Normalize Whether to normalize the returned feature vector with L2 norm. Use only if the model does not already include a built-in L2_NORMALIZATION op. Boolean false
quantize Whether to scalar-quantize the returned embedding to bytes (assumes unit-norm vectors in [-1.0, 1.0]; pair with l2Normalize if needed). Boolean false
imageEmbedderLiveStreamDelegate Asynchronous delegate for LIVE_STREAM mode results (invoked on a serial background queue). ImageEmbedderLiveStreamDelegate Not set

Python

Option Name Description Value Range Default Value
running_mode Sets the running mode for the task:

IMAGE: Single image inputs.
VIDEO: Decoded frames of a video.
LIVE_STREAM: Livestream of camera input data (requires result_callback).		 {IMAGE, VIDEO, LIVE_STREAM} IMAGE
l2_normalize Whether to normalize the returned feature vector with L2 norm. Use only if the model does not already include a built-in L2_NORMALIZATION op. Boolean False
quantize Whether to scalar-quantize the returned embedding to bytes (assumes unit-norm vectors in [-1.0, 1.0]; pair with l2_normalize if needed). Boolean False
result_callback Asynchronous callback function for LIVE_STREAM mode results. Callable Not set

Task output

Each ImageEmbedderResult returns an embeddings list (one per embedding head), and the cosineSimilarity (cosine_similarity) utility compares two embeddings in [-1.0, 1.0]:

Field Type / Range Description
floatEmbedding Float32Array / float[] High-dimensional float feature vector (embedding in Python; L2-normalized when l2Normalize is enabled).
quantizedEmbedding Uint8Array / byte[] Scalar-quantized 8-bit feature vector when quantize is enabled.
headIndex int Zero-based index of the embedding head (head_index in Python).
headName string Optional name of the embedding head (head_name in Python).

The following example shows a 1024-dimensional float feature vector returned for an input image:

{
  embeddings: [
    {
      headIndex: 0,
      floatEmbedding: [0.0142, -0.0318, 0.0521, 0.0089, -0.0412 /* ... 1024 floats */]
    }
  ]
}

Models

Image Embedder pipeline overview

The Image Embedder requires an image embedding model to be downloaded and stored in your project directory. Start with the default, recommended model for your target platform when you start developing with this task. The other available models typically make trade-offs between performance, accuracy, resolution, and resource requirements, and in some cases, include additional features.

MobileNetV3 model

This model family uses a MobileNet V3 architecture and was trained using ImageNet data. This model uses a multiplier of 0.75 for the depth (number of features) in the convolutional layers to tune the accuracy-latency trade off. In addition, MobileNet V3 comes in two different sizes, small and large, to adapt the network to low or high resource use cases.

Model name Input shape Quantization type Versions
MobileNet-V3 (small) 224 x 224 None (float32) Latest
MobileNet-V3 (large) 224 x 224 None (float32) Latest

Task benchmarks

Here are the task benchmarks for the whole pipeline based on the preceding pre-trained models. The latency result is the average latency on Pixel 6 using CPU / GPU.

Model name Device CPU latency GPU latency
MobileNet-V3 (small) Pixel 6 3.94ms 7.83ms
MobileNet-V3 (large) Pixel 6 9.75ms 9.08ms