Join us live on October 8th for the virtual Google Assistant Developer Day. Register now.

Foreground app invocation

With App Actions, users use Google Assistant to deep link directly into specific app destinations when they say something like, "Hey Google, order me a ride on ExampleApp." In situations where a user already has your app open, you can improve the experience by supporting foreground app invocation.

Foreground app invocation allows built-in intents to be matched without requiring the app name to be mentioned while a specific activity is in the device foreground.

For example, a user has your rideshare app in the foreground and says or types, "Order me a ride to Mountain View." to Google Assistant. Your app uses this input to set the destination field to Mountain View. Then, when the user says or types, "Order me a ride from SFO," your app can additionally set the pickup location while preserving the state of your app.

Limitations

Foreground app invocation is only available for the following BIIs:

To get locale support and example queries related to foreground app invocation for a specific BII, see the built-in intent reference.

Support foreground app invocation

Adding support for foreground app invocation involves adding fulfillment behavior based on a desired foreground activity. When that activity is in the foreground and the App Action is invoked, Assistant passes the deep link intent with an additional flag to that activity so your app can update its state.

To implement foreground app invocation for a built-in intent, do the following:

  1. In your actions.xml file, add a <fulfillment> tag to the built-in intent you want to support foreground app invocation. Fulfillments resolve from top to bottom, so define the fulfillment with foreground app invocation above your other fulfillments to give it priority.
  2. In the <fulfillment> tag, add the requiredForegroundActivity attribute.
  3. Set the requiredForegroundActivity attribute to the activity that you want to require in the foreground for foreground app invocation. Specify the activity without any class abbreviations using your app package name, followed by a forward slash (/), followed by the activity name: APP_PACKAGE_NAME/ACTIVITY_NAME
  4. In the foreground activity you specified for requiredForegroundActivity, implement the onNewIntent() method to handle a deep link intent with the SINGLE_TOP flag set. Assistant passes the deep link intent with this flag as fulfillment when the specified activity is in the foreground.

Treat calls to onNewIntent() as updates to your foreground activity, and manage state in that activity using parameters extracted from the deep link. By doing so, you improve the experience for users who want to issue multiple queries to fill out multiple fields (like "Order me a ride to SFO" followed by "Order me a ride from Mountain View").

If your app uses a router activity to allow all external deep links trigger a single gatekeeping router activity, see Handling router activities below.

To try out your foreground app invocation, first open your app to the activity you listed as the required foreground activity. Then, open Assistant and provide a query that corresponds to the built-in intent you implemented (optionally without mentioning the app name itself). When successful, your app updates itself based on your query while maintaining state and without restarting the activity.

Example actions.xml

Here's an example of adding the requiredForegroundActivity attribute to a fulfillment:

<!-- actions.xml -->
<action intentName="actions.intent.CREATE_TAXI_RESERVATION">
  <!-- Trigger with foreground app invocation if MainActivity is in the foreground. -->
  <fulfillment fulfillmentMode="actions.fulfillment.DEEPLINK"
               urlTemplate="app://book_ride_update{?dropoff_location,pickup_location}"
               requiredForegroundActivity="com.example/com.example.app.MainActivity">
    <parameter-mapping intentParameter="taxiReservation.dropoffLocation.name"
                       urlParameter="dropoff_location" />
    <parameter-mapping intentParameter="taxiReservation.pickupLocation.name"
                       urlParameter="pickup_location" />
  </fulfillment>

  <!-- This won't trigger if MainActivity is in the foreground. -->
  <fulfillment fulfillmentMode="actions.fulfillment.DEEPLINK"
               urlTemplate="app://book_ride_update{?dropoff_location,pickup_location}">
    <parameter-mapping intentParameter="taxiReservation.dropoffLocation.name"
                       urlParameter="dropoff_location" />
    <parameter-mapping intentParameter="taxiReservation.pickupLocation.name"
                       urlParameter="pickup_location" />
  </fulfillment>
</action>

Handling router activities

Some apps handle all external deep links using a single gatekeeping router activity. The router activity then starts the appropriate business logic activity (after any checks and validations) and returns the business logic activity to the foreground.

Triggering a deep link can cause the router activity to be added to the top of the task stack above the foreground activity. For apps that use a router activity, you must make sure that the router activity delivers the intent sent by Google Assistant to the current foreground activity instance. How you achieve this requirement changes depending on where your router activity starts.

If your router starts in the same task stack as your business logic activity, forward the intent using the bitwise OR of SINGLE_TOP, CLEAR_TOP, and NEW_TASK:

Intent.FLAG_ACTIVITY_SINGLE_TOP | Intent.FLAG_ACTIVITY_CLEAR_TOP | Intent.FLAG_ACTIVITY_NEW_TASK

If your router starts in a separate task stack from your business logic activity, forward the intent with the SINGLE_TOP flag to the business logic activity instead.

User permissions

In device settings, users must enable the 'Use text from screen'
            for foreground app invocation to work.

In order for foreground app invocation to work for a user, the user must have the Use text from screen device setting enabled. The location and exact name of this setting can vary by OEM or device manufacturer. For instance, the device setting might instead be called Use screen context on a user's device.

To reach this Android setting on a Pixel phone, first go to Settings > Apps & notifications > Default apps > Assist app. Then, on the Assist & voice input screen, enable Use text from screen.