Text-to-speech guide for iOS and macOS

The MediaPipe Text-to-Speech task lets you synthesize natural-sounding speech audio from text on iOS (16.0+) and macOS (13.0+) using the Kokoro-82M neural TTS model. These instructions show you how to use the Text-to-Speech in Swift applications.

For more information about the capabilities, .litertlm model bundles, and supported voices of this task, see the Overview.

Code example

The MediaPipe Tasks example code is a basic implementation of a Text-to-Speech app for iOS. The example synthesizes the text that you enter with a voice that you choose, either as a complete utterance or as a stream of audio chunks, and plays the audio as it arrives.

You can use the app as a starting point for your own iOS app, or refer to it when modifying an existing app. You can refer to the Text-to-Speech example code on GitHub.

Download the code

The following instructions show you how to create a local copy of the example code using the git command line tool.

To download the example code:

  1. Clone the git repository using the following command:

    git clone https://github.com/google-ai-edge/mediapipe-samples

  2. Optionally, configure your git instance to use sparse checkout, so you have only the files for the Text-to-Speech example app:

    cd mediapipe-samples
git sparse-checkout init --cone
git sparse-checkout set examples/text_to_speech/ios

After creating a local version of the example code, add the model to the app as described in the example's README. Then open the project in Xcode and run the app. For instructions, see the Setup guide for iOS.

Key components

The following files contain the main code for the Text-to-Speech example app:

  • TTSViewModel.swift: Creates the Text-to-Speech task, lists the available voices, and runs unary and streaming synthesis.
  • TTSAudioPlayer.swift: Plays the synthesized audio with AVAudioEngine and can export it as a WAV file.
  • ContentView.swift: Implements the SwiftUI user interface.

Setup

This section describes key steps for setting up your Xcode or Swift Package Manager (SPM) project to use Text-to-Speech. For general information on setting up your development environment for MediaPipe tasks, see the Setup guide for iOS.

Dependencies

Text-to-Speech uses the MediaPipeTasksAudio library, which you can install using Swift Package Manager (SPM).

To add the MediaPipeTasksAudio library to your Xcode project:

  1. With your project open in Xcode, select File > Add Package Dependencies....
  2. In the search field in the upper right, enter the repository URL: https://github.com/google-ai-edge/mediapipe
  3. Select the mediapipe package and choose your dependency rule.
  4. Click Add Package, then select MediaPipeTasksAudio and add it to your application target.

Then import the library in your Swift source files:

import MediaPipeTasksAudio

Model

The MediaPipe Text-to-Speech task requires the kokoro_int8_gpu.litertlm model bundle. See the task overview Models section for details.

Bundle kokoro_int8_gpu.litertlm in your app resources (or download it into your app's Application Support / Documents container) and pass its path to TTSBaseOptions(modelAssetPath:):

guard let modelPath = Bundle.main.path(forResource: "kokoro_int8_gpu", ofType: "litertlm") else {
    fatalError("kokoro_int8_gpu.litertlm not found in app bundle.")
}

Create the task

Use createTextToSpeech(options:) (or TextToSpeech(options:)) with TextToSpeechOptions, which takes a TTSBaseOptions with the model path and delegate:

import Foundation
import MediaPipeTasksAudio

let cacheDir = FileManager.default.temporaryDirectory
    .appendingPathComponent("mediapipe_tts_cache").path

let options = TextToSpeechOptions(
    baseOptions: TTSBaseOptions(
        modelAssetPath: modelPath,
        delegate: .cpu
    ),
    defaultVoiceId: "af_heart",
    speakingRate: 1.0,
    targetBucket: 64,
    cacheDirectory: cacheDir
)

let tts = try createTextToSpeech(options: options)

Configuration options

This task has the following configuration options for iOS and macOS apps:

Task options (TextToSpeechOptions)

Option Name Description Value Range Default Value
baseOptions.modelAssetPath Path (or URL on Web) to the single .litertlm model bundle containing the Kokoro-82M acoustic and vocoder graphs, phonetic lexicons, and voice embeddings. String / Path Required
baseOptions.delegate Hardware acceleration delegate used by the LiteRT runtime. When set to GPU, the Kokoro acoustic stage executes on CPU (XNNPACK) and the neural vocoder executes on GPU (OpenCL/Metal/WebGPU) with CPU fallback. .cpu, .gpu .cpu
speakingRate Speaking rate multiplier applied to synthesized Kokoro audio (1.0 represents normal speaking speed). Float (0.1, 4.0] 1.0
targetBucket Target sequence-length bucket for the Kokoro acoustic/vocoder graph (for example, 64, 128, or 256 tokens). Smaller buckets reduce synthesis latency for short sentences; 0 uses the model's default capacity. 0, 64, 128, 256 0
defaultVoiceId Default Kokoro voice profile identifier (for example, "af_heart", "ef_dora", or "zf_xiaobei"). Can be overridden per utterance using SynthesisOptions. String (Optional) "af_heart"
defaultLocale Optional default BCP 47 language tag (for example, "en-US", "es", "zh-CN"). When omitted, the task automatically infers the language from the selected Kokoro voice prefix. BCP 47 String (Optional) Inferred from voice
cacheDirectory Writable directory path used to persist unpacked phonetic dictionary assets and LiteRT XNNPACK weight caches across runs for faster cold starts. Directory Path (Optional) ""

Per-utterance options (SynthesisOptions)

Option Name Description Default Value
voiceId Overrides the voice profile for a single synthesize call (for example, switching from "af_heart" to "ef_dora" in under 1 ms without reloading model weights). null / None (uses task default)
locale Optional BCP 47 locale override for a single utterance. Leave unset (null / None) when switching voices so the engine automatically resolves the voice's target language from its prefix. null / None (inferred from voiceId)

Inspect available voices

Call listVoices() to query all Kokoro voices packaged inside the loaded .litertlm bundle:

let voices = try tts.listVoices()
for voice in voices {
    print("\(voice.voiceId) (\(voice.displayName)) - locale=\(voice.locale), \(voice.sampleRateHz) Hz")
}

Run the task

You can call the non-blocking synthesizeAsync(_:options:) method from Swift structured concurrency, or call synthesize(_:options:) synchronously on a background dispatch queue.

Async and synchronous synthesis

// Non-blocking async call (recommended):
let audioResult = try await tts.synthesizeAsync(
    "Hello from MediaPipe Text to Speech on Apple platforms!"
)

// Synchronous blocking call (run off the main thread):
let syncResult = try tts.synthesize("Synthesized synchronously on a worker thread.")

Fast bilingual voice switching

Pass SynthesisOptions(voiceId:) to switch between bundled Kokoro voices across languages in < 1 ms. Leave locale as nil so the engine automatically resolves the target language from the voice prefix:

// Synthesize English ("af_heart") followed by Spanish ("ef_dora"):
let enResult = try await tts.synthesizeAsync(
    "Hello! Let's switch to Spanish.",
    options: SynthesisOptions(voiceId: "af_heart")
)

let esResult = try await tts.synthesizeAsync(
    "Hola desde MediaPipe Text to Speech en iOS y macOS.",
    options: SynthesisOptions(voiceId: "ef_dora")
)

Streaming speech synthesis

For lower Time-To-First-Audio (TTFA) on multi-sentence text, iterate over tts.synthesizeStream(_:options:), which returns an AsyncThrowingStream<AudioChunk, Error>. Each AudioChunk (samples, sampleRateHz, chunkIndex, isFinal) is yielded as soon as its clause finishes vocoding, so you can schedule it onto AVAudioPlayerNode immediately:

for try await chunk in tts.synthesizeStream(
    "Streaming synthesis plays the first clause immediately. Subsequent clauses synthesize in the background.",
    options: SynthesisOptions(voiceId: "af_heart")
) {
    try playAudioResult(
        AudioResult(samples: chunk.samples, sampleRateHz: chunk.sampleRateHz),
        playerNode: playerNode,
        engine: audioEngine
    )
}

When streaming tokens incrementally from an on-device LLM, open a TTSStream with tts.openStream(options:), push text fragments with pushText(_:), mark end-of-input with finishInput(), and pull synthesized AudioChunks with readNext():

let stream = try tts.openStream(options: SynthesisOptions(voiceId: "af_heart"))
defer { stream.close() }

try stream.pushText("Hello from an ")
try stream.pushText("on-device LLM! ")
try stream.pushText("Each completed clause streams out.")
try stream.finishInput()

while let chunk = try stream.readNext() {
    try playAudioResult(
        AudioResult(samples: chunk.samples, sampleRateHz: chunk.sampleRateHz),
        playerNode: playerNode,
        engine: audioEngine
    )
}

Barge-in cancellation

If the user interrupts speech playback ("barge-in"), you can abort an in-flight call immediately:

  • Breaking out of for try await chunk in tts.synthesizeStream(...) or canceling the enclosing Swift Task automatically cancels and closes the underlying TTSStream using onTermination.
  • synthesizeAsync(_:options:) uses withTaskCancellationHandler internally: canceling the Swift Task automatically calls tts.cancel() and throws CancellationError.
  • You can also call tts.cancel() (or stream.cancel()) directly from any thread to abort an in-flight synthesis pass immediately.
let synthesisTask = Task {
    for try await chunk in tts.synthesizeStream("Long utterance that may be interrupted by the user...") {
        try playAudioResult(
            AudioResult(samples: chunk.samples, sampleRateHz: chunk.sampleRateHz),
            playerNode: playerNode,
            engine: audioEngine
        )
    }
}

// On user barge-in:
synthesisTask.cancel() // Immediately cancels the TTSStream

Handle and play results

synthesize and synthesizeAsync return an AudioResult struct containing:

  • samples: [Float] array of mono PCM samples in [-1.0, 1.0]
  • sampleRateHz: Int (24000 Hz for Kokoro-82M)

You can schedule the PCM samples onto an AVAudioEngine / AVAudioPlayerNode for playback:

import AVFoundation

func playAudioResult(_ result: AudioResult, playerNode: AVAudioPlayerNode, engine: AVAudioEngine) throws {
    guard let format = AVAudioFormat(
        standardFormatWithSampleRate: Double(result.sampleRateHz),
        channels: 1
    ),
    let buffer = AVAudioPCMBuffer(
        pcmFormat: format,
        frameCapacity: AVAudioFrameCount(result.samples.count)
    ) else { return }

    buffer.frameLength = buffer.frameCapacity
    if let channelData = buffer.floatChannelData?[0] {
        result.samples.withUnsafeBufferPointer { src in
            channelData.initialize(from: src.baseAddress!, count: src.count)
        }
    }

    if !engine.isRunning {
        try engine.start()
    }
    playerNode.scheduleBuffer(buffer, completionHandler: nil)
    playerNode.play()
}