# Media responses let your Actions play audio content with a playback duration
longer than the 240-second limit of [SSML](https://developers.google.com/assistant/conversational/ssml). Media responses work on both
audio-only devices and devices that can display visual content. On a display,
media responses are accompanied by a visual component with media controls and
(optionally) a still image.

When defining a media response, use a [candidate](https://developers.google.com/assistant/conversational/prompts#candidates) with both the
`RICH_RESPONSE` and `LONG_FORM_AUDIO` surface capabilities so that Google
Assistant only returns the rich response on supported devices. You can only use
one rich response per `content` object in a prompt.

Audio for playback must be in a correctly formatted MP3 file. MP3 files must
be hosted on a web server and be publicly available through an HTTPS URL. Live
streaming is only supported for the MP3 format.
![Example of media response on smart display](https://developers.google.com/static/assistant/conversational/images/smart-display/media.png) **Figure 1.** Example of a media response on smart display

## Behavior

![Example of a media response on a smartphone](https://developers.google.com/static/assistant/conversational/images/media.png) **Figure 2.** Example of a media response on smartphone

The primary component of a media response is the single-track card. The card
allows the user to do the following:

- Replay the last 10 seconds
- Skip forward 30 seconds
- View the total length of the media content
- View a progress indicator for media playback
- View the elapsed playback time

Media responses support the following audio controls for voice interactions, all
of which are handled by Google Assistant:

- *"Ok Google, play."*
- *"Ok Google, pause."*
- *"Ok Google, stop."*
- *"Ok Google, start over."*

Users can also control the volume by saying phrases like, *"Hey Google, turn the
volume up."* or *"Hey Google, set the volume to 50 percent."* Intents in your
Action take precedence if they handle similar training phrases. Let Assistant
handle these user requests unless your Action has a specific reason to.

### Behavior on Android phones

On Android phones, media controls are also available while the phone is locked.
Media controls also appear in the notification area, and users can see media
responses when any of the following conditions are met:

- Google Assistant is in the foreground, and the phone screen is on.
- The user leaves Google Assistant while audio is playing and returns to Google Assistant within 10 minutes of playback completion. On returning to Google Assistant, the user sees the media card and suggestion chips.

> [!NOTE]
> **Note:** Your webhook won't receive callbacks from Google Assistant if the user leaves the Google Assistant screen during media playback and media playback finishes while the user is away. Include suggestion chips to allow users to continue interacting with your Action when they return to the Google Assistant screen.

## Properties

Media responses have the following properties:

| Property | Type | Requirement | Description |
|---|---|---|---|
| `media_type` | [`MediaType`](https://developers.google.com/assistant/actionssdk/reference/rest/Shared.Types/MediaType) | Required | Media type of the provided response. Return `MEDIA_STATUS_ACK` when acknowledging a media status. |
| `start_offset` | string | Optional | Seek position to start the first media track. Provide the value in seconds, with fractional seconds expressed at no more than nine decimal places, and end in the suffix "s". For example, 3 seconds and 1 nanosecond is expressed as "3.000000001s". > [!NOTE] > **Note:** Currently unavailable on iOS. |
| `optional_media_controls` | array of [`OptionalMediaControls`](https://developers.google.com/assistant/actionssdk/reference/rest/Shared.Types/OptionalMediaControls) | Optional | Opt-in property to receive callbacks when a user changes their media playback status (such as by pausing or stopping media playback). |
| `media_objects` | array of [`MediaObject`](https://developers.google.com/assistant/actionssdk/reference/rest/Shared.Types/EventHandler#mediaobject) | Required | Represents the media objects to include in the prompt. When acknowledging a media status with `MEDIA_STATUS_ACK`, do not provide media objects. |
| `first_media_object_index` | integer | Optional | 0-based index of the first `MediaObject` in `media_objects` to play. If unspecified, zero, or out-of-bounds, playback starts at the first `MediaObject`. > [!NOTE] > **Note:** Currently unavailable on iOS. |
| `repeat_mode` | [`RepeatMode`](https://developers.google.com/assistant/actionssdk/reference/rest/Shared.Types/RepeatMode) | Optional | Repeat mode for the list of media objects. > [!NOTE] > **Note:** Currently unavailable on iOS. |

### Sample code

### YAML

```yaml
candidates:
  - first_simple:
      variants:
        - speech: This is a media response.
    content:
      media:
        start_offset: 2.12345s
        optional_media_controls:
          - PAUSED
          - STOPPED
        media_objects:
          - name: Media name
            description: Media description
            url: 'https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3'
            image:
              large:
                url: 'https://storage.googleapis.com/automotive-media/album_art.jpg'
                alt: Jazz in Paris album art
        media_type: AUDIO
```

### JSON

```
{
  "candidates": [
    {
      "first_simple": {
        "variants": [
          {
            "speech": "This is a media response."
          }
        ]
      },
      "content": {
        "media": {
          "start_offset": "2.12345s",
          "optional_media_controls": [
            "PAUSED",
            "STOPPED"
          ],
          "media_objects": [
            {
              "name": "Media name",
              "description": "Media description",
              "url": "https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
              "image": {
                "large": {
                  "url": "https://storage.googleapis.com/automotive-media/album_art.jpg",
                  "alt": "Jazz in Paris album art"
                }
              }
            }
          ],
          "media_type": "AUDIO"
        }
      }
    }
  ]
}
```

### Node.js

```javascript
// Media response
app.handle('media', (conv) => {
  conv.add('This is a media response');
  conv.add(new Media({
    mediaObjects: [
      {
        name: 'Media name',
        description: 'Media description',
        url: 'https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3',
        image: {
          large: JAZZ_IN_PARIS_IMAGE,
        }
      }
    ],
    mediaType: 'AUDIO',
    optionalMediaControls: ['PAUSED', 'STOPPED'],
    startOffset: '2.12345s'
  }));
});
```

### JSON

```
{
  "session": {
    "id": "session_id",
    "params": {},
    "languageCode": ""
  },
  "prompt": {
    "override": false,
    "content": {
      "media": {
        "mediaObjects": [
        {
          "name": "Media name",
          "description": "Media description",
          "url": "https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
          "image": {
            "large": {
              "alt": "Jazz in Paris album art",
              "height": 0,
              "url": "https://storage.googleapis.com/automotive-media/album_art.jpg",
              "width": 0
            }
          }
        }
        ],
        "mediaType": "AUDIO",
        "optionalMediaControls": [
          "PAUSED",
          "STOPPED"
        ]
      }
    },
    "firstSimple": {
      "speech": "This is a media response",
      "text": "This is a media response"
    }
  }
}
```

## Receiving media status

During or after media playback for a user, Google Assistant can generate media
status events to inform your Action of playback progress. Handle these status
events in your webhook code to appropriately route users when they pause, stop,
or finish media playback.

Google Assistant returns a status event from the following list based on media
playback progress and user queries:

- **`FINISHED`:** User completed media playback (or skips to next piece of media) and transition is not to a conversation exit. This status also maps to the `MEDIA_STATUS_FINISHED` system intent.
- **`PAUSED`:** User paused media playback. Opt in to receiving this status event with the `optional_media_controls` property. This status also maps to the `MEDIA_STATUS_PAUSED` system intent.
- **`STOPPED`:** User stopped or exited media playback. Opt in to receiving this status event with the `optional_media_controls` property. This status also maps to the `MEDIA_STATUS_STOPPED` system intent.
- **`FAILED`:** Media playback failed. This status also maps to the `MEDIA_STATUS_FAILED` system intent.

During media playback, a user might provide a query that can be interpreted as
both a media pause and stop event (like "stop", "cancel", or "exit"). In that
situation, Assistant provides the `actions.intent.CANCEL` system intent to your
Action, generates a media status event with the "STOPPED" status value, and
exits your Action completely.

When Assistant generates a media status event with the `PAUSED` or `STOPPED`
status value, respond with a media response that contains only an
acknowledgement (of type `MEDIA_STATUS_ACK`).

> [!NOTE]
> **Note:** Developers are restricted from responding to media pause and stop events with a media response, except to acknowledge receipt.

### Media progress

The current media playback progress is available in the
[`context.media.progress`](https://developers.google.com/assistant/conversational/reference/rest/v1/TopLevel/fulfill#context) field for webhook requests. You
can use the media progress as a start time offset to resume playback at the
point where media playback ended. To apply the start time offset to a media
response, use the `start_offset` property.

> [!NOTE]
> **Tip:** Retrieving media progress is not limited to media status events. For instance, you can access media progress from `context.media.progress` for a no-match intent that occurs during media playback.

### Sample code

### Node.js

```javascript
// Media status
app.handle('media_status', (conv) => {
  const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
  switch(mediaStatus) {
    case 'FINISHED':
      conv.add('Media has finished playing.');
      break;
    case 'FAILED':
      conv.add('Media has failed.');
      break;
    case 'PAUSED' || 'STOPPED':
      if (conv.request.context) {
        // Persist the media progress value
        const progress = conv.request.context.media.progress;
      }
      // Acknowledge pause/stop
      conv.add(new Media({
        mediaType: 'MEDIA_STATUS_ACK'
        }));
      break;
    default:
      conv.add('Unknown media status received.');
  }
});
```

## Return a playlist

You can add more than one audio file in your response to create a playlist.
When the first track is finished playing, the next track plays automatically,
and this continues until each track has played. Users can also
press the **Next** button on screen, or say *"Next"* or something similar to skip to the
next track.

The **Next** button is disabled on the last track of the playlist. However,
if you enable looping mode, the playlist begins again from the
first track. To learn more about looping mode, see
[Implement looping mode](https://developers.google.com/assistant/conversational/prompts-media#implement_looping_mode).

To create a playlist, include more than one `MediaObject` in the `media_objects`
array. The following code snippet shows a prompt that returns a playlist of
three tracks:

```
{
  "candidates": [
    {
      "content": {
        "media": {
          "media_objects": [
            {
              "name": "1. Jazz in Paris",
              "description": "Song 1 of 3",
              "url": "https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
              "image": {
                "large": {
                  "url": "https://storage.googleapis.com/automotive-media/album_art.jpg",
                  "alt": "Album cover of an ocean view",
                  "height": 1600,
                  "width": 1056
                }
              }
            },
            {
              "name": "2. Jazz in Paris",
              "description": "Song 2 of 3",
              "url": "https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
              "image": {
                "large": {
                  "url": "https://storage.googleapis.com/automotive-media/album_art.jpg",
                  "alt": "Album cover of an ocean view",
                  "height": 1600,
                  "width": 1056
                }
              }
            },
            {
              "name": "3. Jazz in Paris",
              "description": "Song 3 of 3",
              "url": "https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
              "image": {
                "large": {
                  "url": "https://storage.googleapis.com/automotive-media/album_art.jpg",
                  "alt": "Album cover of an ocean view",
                  "height": 1600,
                  "width": 1056
                }
              }
            }
          ],
        }
      }
    }
  ]
}
```

## Implement looping mode

Looping mode allows you to provide an audio response that automatically
repeats. You can use this mode to repeat
a single track or to loop through a playlist. If the user says *"Next"* or
something similar for a single looped track, the song begins again. For looped
playlists, a user saying *"Next"* starts the next track in the playlist.

> [!NOTE]
> **Note:** Looping mode is currently unavailable on iOS.

To implement looping mode, add the [`repeat_mode`](https://developers.google.com/assistant/conversational/prompts-media#MediaResponseProperties)
field to your prompt and set its value to `ALL`. This addition allows your media
response to loop to the beginning of the first media object when the end of the
last media object is reached.

The following code snippet shows a prompt that returns a looping track:

```
{
  "candidates": [
    {
      "content": {
        "media": {
          "media_objects": [
            {
              "name": "Jazz in Paris",
              "description": "Single song (repeated)",
              "url": "https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3",
              "image": {
                "large": {
                  "url": "https://storage.googleapis.com/automotive-media/album_art.jpg",
                  "alt": "Album cover of an ocean view",
                  "height": 1600,
                  "width": 1056
                }
              }
            }
          ],
          "repeat_mode": "ALL"
        }
      }
    }
  ]
}
```