Build Actions for Google Assistant using Actions Builder (Level 2)

1. Overview

Warning: Conversational Actions were deprecated on June 13, 2023. For more information, see Conversational Actions sunset.

The Google Assistant developer platform lets you create software to extend the functionality of Google Assistant, a virtual personal assistant, across more than 1 billion devices, including smart speakers, phones, cars, TVs, headphones, and more. Users engage Assistant in conversation to get things done, such as buying groceries or booking a ride. As a developer, you can use the Assistant developer platform to easily create and manage delightful and effective conversational experiences between users and your own third-party fulfillment service.

This codelab covers intermediate-level concepts for developing with Google Assistant and builds on the Action created in the Level 1 codelab. It's strongly recommended that you complete the Level 1 codelab before starting this one.

The Action you build in this codelab tells users their fortune for their quest in a mythical land, Gryffinberg, based on the aid they choose.

What you'll build

In this codelab, you build a sophisticated Conversational Action with the following functions:

• Collects data from the user and, depending on the value, modifies the conversational prompts
• Responds with follow-up questions to further the conversation
• Creates a game loop so a user can interact with the Action again after receiving a fortune

Before you start building, you can interact with the live Action on your Google Assistant-enabled device by saying "Hey Google, talk to Fate and Fortune". The default path through this Action for a returning user looks like the following interaction:

What you'll learn

• How to use slots to gather data from the user
• How to use conditions to add logic to a scene
• How to add a game loop
• How to add a supportive path

What you'll need

The prerequisites for this codelab include the following:

• A web browser, such as Google Chrome
• A completed codelab Level 1 Actions project ( Build Actions for Google Assistant using Actions Builder Level 1)

Familiarity with JavaScript (ES6) is strongly recommended, although not required, to understand the fulfillment code for this codelab.

2. Continue building conversational interface

In the first codelab, you created a simple Conversational Action with a single scene, Start.

In this codelab, you extend your Action's conversation. In the following sections, you configure your Action to do the following:

• Transition to a new Fortune scene when the user wants to hear their fortune
• Ask the user which aid they want to choose for their journey
• Deliver a customized fortune based on the user's choice

Create Fortune scene

In this section, you create the Fortune scene and define how the user transitions to it during the conversation.

To create a new scene called Fortune, follow these steps:

1. Open your codelab level 1 Actions project.
2. Click Develop in the navigation bar.
3. Under Scenes, click the Start scene.
4. Click the yes intent (the When yes is matched box) to open the options.
5. Clear Send prompts to remove the prompt.
6. In the Transition section, click the drop-down menu, click in to the text box, and type Fortune.
7. Click Add. This creates a new scene called Fortune. It also adds a transition from the Start scene to the Fortune scene when the user wants to hear their fortune.

Define conversational logic for Fortune scene

In this codelab, you configure your Fortune scene to ask the user, "What do you choose to help you on your quest, a dragon, a translator, or a compass?" You can use a capability called slot filling to gather the necessary information from the user before proceeding.

Key terms:

• Slot filling: A function of conversational logic that allows you to collect a set of values (slots) from the user through natural, back-and-forth conversation without having to create multiple intents.
• Slots: Data that you want to extract from user input.

Your Action provides fortunes for three aids: a dragon, translator, and compass. To configure your Action to identify these three options in a user's input, you must create a new type.

Key terms:

• Types: Allow you to configure the Assistant natural-language understanding (NLU) to extract structured data from user input.

You can use types within a scene's slot filling stage to define the information you want from the user. When the NLU engine detects a slot match in user input, it extracts the slot as a typed parameter, so you can carry out logic with it in a scene.

Create available_options type

In this section, you create a new type called available_options, which specifies the three options the users can choose (dragon, translator, and compass) in response to the prompt. You also define a few synonyms for these options in case a user says something similar. In a later section, you add the available_options type to a slot to specify that you want to obtain the user's choice.

To create the available_options type, follow these steps:

1. In the navigation bar, click Types.
2. Click + (plus sign), type available_options, and press Enter.
3. Click available_options to open the options.

Types are configured as key-value pairs of information, where the key is the name of the type and the values are synonyms for that key. When you define the key, it is automatically added as a value.

To add the three options the user can choose, follow these steps:

1. Scroll to the Add entries section.
2. In the New entry field, type dragon and press Enter. This action creates a dragon key.
3. Type hydra in the Add values field and press Enter to add it as a value (synonym). Then, repeat this step for the value lizard.
4. Add the rest of the keys and corresponding values:

• translator | translator, communicator, machine, decoder, translate
• compass | compass, direction, guide

5. Click Save.

Your Action now understands that the available_options are dragon, translator, and compass, and can also recognize a few corresponding synonyms.

Configure slot filling

Next, you need to configure slot filling in the Fortune scene. To configure the slot-filling logic, follow these steps:

1. In the navigation bar, under Scenes, click Fortune.
2. In the Fortune scene, Click the + (plus sign) for Slot filling.
3. In the Enter slot name field, add chosenOptions as the slot name.
4. In the Select type drop-down list, select available_options as the slot type.
5. Select the This slot is required checkbox.

6. Select Send prompts, and add the following message and suggestion chips:

candidates:
  - first_simple:
      variants:
        - speech: >-
            What do you choose to help you on your quest, a dragon, a
            translator, or a compass?
    suggestions:
      - title: 'Dragon'
      - title: 'Translator'
      - title: 'Compass'

7. Click Save.

You've now added the available_options type to the slot, which tells your Action the information you need to gather from the user (their choice of aid) before proceeding. You also configured a prompt within the slot, which is added to the prompt queue when the user reaches the slot-filling stage of the scene.

Note that when you named the slot chosenOptions, the Customize slot value writeback field is updated with the same name ($session.params.chosenOptions). You can access this parameter by that name in Actions Builder and in your fulfillment through the client library.

Configure scene.slots.status == "FINAL" condition

When you add a slot, the condition scene.slots.status == "FINAL" is automatically added to the condition list.

Key terms:

• Conditions: Let you check session, user, or slot data for a condition to be met.

The condition scene.slots.status == "FINAL" checks for slot filling to be complete. When all the slots are filled, the condition can trigger a webhook, transition to a new scene, or add prompts to the prompt queue.

In this section, you configure scene.slots.status == "FINAL" to add a prompt to the prompt queue once the slots are filled.

To add this prompt to the FINAL condition, follow these steps:

1. Click scene.slots.status == "FINAL" to open the options window.
2. Select Send prompts and add the following prompt:

candidates:
  - first_simple:
      variants:
        - speech: You picked $session.params.chosenOptions.

3. Click Save.

Test your Action in the simulator

At this point, you have defined which options the user should select to fill the slot. After obtaining this information from the user, your Action should provide a prompt referencing the specific option they've chosen.

To test your Action, follow these steps:

1. In the navigation bar, click Test.
2. Click or type Talk to my test app in the Input field and press Enter.
3. Type Yes in the Input field and press Enter. (Alternatively, you can click the Yes suggestion chip.)

If you receive a prompt indicating that the Action doesn't understand your input when you say, click, or type Yes, check that you made the slot required.

4. Click, type, or say dragon. You should receive the prompt "You picked dragon."

In the next section, you customize the prompts for each aid the user can select.

Customize prompts using conditions

In this section, you add conditions for each option the user can choose and add a custom prompt for each condition.

Customize the dragon fortune

To update the condition and customize the prompt for when a user chooses "dragon", follow these steps:

1. Click Develop in the navigation bar.
2. In the navigation bar, click the Fortune scene.
3. Click scene.slots.status == "FINAL" to open the options window.
4. Update the condition statement to: scene.slots.status == "FINAL" && session.params.chosenOptions == "dragon"
5. Select Send prompts.
6. Update the prompt with the following fortune in the code editor:

candidates:
  - first_simple:
      variants:
        - speech: >-
            The people of Gryffinberg will be awestruck by the beauty and power
            of the ancient dragon. Much to your dismay, the townspeople fall
            into dispute over who will receive the honor of riding the dragon
            first. You return home from your quest without everlasting glory or
            a dragon.

7. Click Save.

Now, when a user says "dragon" or something that sounds similar, your Action provides a fortune based on that selection. Next, you add the remaining two selections.

Customize the translator fortune

To add the condition and customize the prompt for when a user chooses "translator", follow these steps:

1. Click the + (plus sign) next to Condition.
2. Add scene.slots.status == "FINAL" && session.params.chosenOptions == "translator" to the else if field.
3. Select Send prompts.
4. Add the following prompt in your code editor:

candidates:
  - first_simple:
      variants:
        - speech: >-
            With the help of the translator, the rival factions in Gryffinberg
            are finally able to communicate with each other and resolve their
            disputes. You will complete your quest to restore peace in the town.
            The translator will be used on many subsequent journeys across the
            earth. After its work is done, it retires honorably to a premier
            location in the Gryffinberg History Museum.

4. Click Save.

Customize the compass fortune

To add the condition and customize the prompt for when a user chooses "compass", follow these steps:

1. Click the + (plus sign) next to Condition.
2. Add scene.slots.status == "FINAL" && session.params.chosenOptions == "compass" to the else if text box.
3. Select Send prompts.
4. Add the following prompt in the code editor:

candidates:
  - first_simple:
      variants:
        - speech: >-
            The compass will help you find the mystical and ancient Library of
            Gryffinberg. Among its infinite stacks of dusty books, you find one
            entitled "Wisdom of the Ages". By the time you've read the
            50,000-page tome, the townspeople have forgotten their problems. You
            will write a second edition of "Wisdom of the Ages", but have
            limited commercial success.

5. Click Save.

Test your Action in the simulator

At this point, your Action should provide a customized fortune for the user based on the option they select.

To test your Action, follow these steps:

1. In the navigation bar, click Test.
2. Type Talk to my test app in the Input field and press Enter.
3. Type Yes in the Input field and press Enter. Alternatively, click the Yes suggestion chip.
4. Click, type, or say Translator.

You should receive the appropriate fortune for the "translator" option.

3. Add game loop

In this section, you configure your Action so that the user can select another option and hear a different fortune after making a selection. This change is similar to a "Do you want to play again?" message at the end of a game. To build this loop, you can reuse the previously created yes and no intents, and add them to a new scene called Again.

Create Again scene

In this section, you create a new Again scene and add a prompt that asks the user if they'd like to select a different option.

To create the Again scene, follow these steps:

1. Click Develop in the navigation bar.
2. Click the + (plus sign) under Scenes.
3. Type Again and press Enter.
4. Click the Again scene in the navigation bar.
5. Click the + (plus sign) next to On enter.
6. Select Send prompts, and add the following prompt and suggestion chips:

candidates:
  - first_simple:
      variants:
        - speech: >-
            That is what I see for you. Would you like to choose a different option and
            explore another future?
    suggestions:
      - title: 'Yes'
      - title: 'No'

7. Click Save.

Add transition from Fortune to Again scene

After the user receives their fortune, the conversation needs to transition to the new Again scene.

To add a transition from the Fortune scene to the Again scene, follow these steps:

1. Click the Fortune scene.
2. Click the first condition (scene.slots.status == "FINAL" && session.params.chosenOptions == "dragon") to open the options window.
3. Scroll and select Again under Transition.
4. Click Save.
5. Click the second condition to open the options window.
6. Scroll and select Again under Transition.
7. Click Save.
8. Click the third condition to open the options window.
9. Scroll and select Again under Transition.
10. Click Save.

Test your Action in the simulator

At this point, your Action should provide the following prompt to the user after they receive their fortune: "That is what I see for you. Would you like to choose a different option and explore another future?"

To test your Action, follow these steps:

1. In the navigation bar, click Test.
2. Type Talk to my test app in the Input field and press Enter.
3. Type Yes in the Input field and press Enter. Alternatively, click the Yes suggestion chip.
4. Click, type, or say dragon.

You should receive the fortune for the dragon option and the Again prompt.

Add intents and transition to Again scene

In this section, you add yes and no intents to the Again scene so your Action understands if the user wants to choose a new option or not. You also add the appropriate transitions for the yes and no intents. The yes intent transitions to the Fortune scene, while the no intent transitions to the system scene End conversation.

To add intents and transitions to the Again scene, follow these steps:

1. Click Develop in the navigation bar.
2. Click the Again scene.
3. Click the + (plus sign) next to User intent handling.
4. Select yes from the intent drop-down menu.
5. Select Fortune from the Transition drop-down menu.
6. Click Save.

7. Click the + (plus sign) next to User intent handling.
8. Select no from the intent drop-down menu.
9. Select End conversation from the Transition drop-down menu.
10. Select Send prompts and add the following prompt in the code editor:

candidates:
  - first_simple:
      variants:
        - speech: >-
            It pleases me that you are satisfied with your choice. Best of luck on your quest. Farewell.

11. Click Save.

Test your Action in the simulator

Your Action should now understand if the user wants to choose a new option or end the conversation.

To test the yes intent, follow these steps:

1. In the navigation bar, click Test.
2. Type Talk to my test app in the Input field and press Enter.
3. Type Yes in the Input field and press Enter. Alternatively, click the Yes suggestion chip.
4. Click, type, or say one of the options.
5. Type Yes in the Input field and press Enter.

You should receive the prompt, "What do you choose to help you on your quest, a dragon, a translator, or a compass?"

To test the no intent, follow these steps:

1. Click, type, or say one of the options.
2. Type No in the input field and press Enter.

You should receive the End conversation prompt: "It pleases me that you are satisfied with your choice. Best of luck on your quest. Farewell."

4. Add a supportive path

You've now built the main path that most users take in your Action. However, the user could respond to the prompt from the Fortune scene, "What do you choose to help you on your quest, a dragon, a translator, or a compass?", with a choice that is not one of the provided options.

In this section, you configure your Action to understand when a user chooses "magic", "money", "horse", or "phone", and to prompt the user to select from one of the original three choices when they choose one of these options. To configure this logic, you need to create a new type that contains these other choices and a new intent, other_option, that is matched when a user says one of these options. You also need to annotate training phrases within the other_option intent to identify and extract intent parameters.

Key terms:

• Intent parameters: Typed data (specific words or phrases) that you want to extract from user input. To create a parameter, you annotate training phrases within an intent.
• Annotation: The process of specifying types that you want to extract from training phrases. If you configure a type before creating training phrases that include the type, this happens automatically.

When the Assistant's natural-language processing (NLU) engine detects a parameter match in user input, it extracts the value as a typed parameter so you can carry out logic with it in a scene. In this codelab, you configure your Action to extract the aid the user chooses and refer to that choice in a prompt.

An earlier section discussed slot filling, which collects information from the user that is needed to proceed. The data you collect from the user in scenes is referred to as slots. Intent parameters differ slightly in that they are extracted from user input after an intent is matched; however, intent parameters can also fill slots.

Create unavailable_options type

You can now create an unavailable_options type that contains a variety of different options so your Action can identify that data in a user's input.

To create the unavailable_options type, follow these steps:

1. Click Develop in the navigation bar.
2. Click the + (plus sign) under Types.
3. Type unavailable_options and press Enter.
4. Click unavailable_options to open the options.
5. Enter the following entries and corresponding values in the Add entries section:

horse	horse, stallion, steed
magic	magic, enchanted, spells
money	money, cash, gold
phone	phone, cell, apps

Your key-value table should look like the following:

6. Click Save.

Create other_option intent

Next, you create an intent called other_option and add training phrases that include the options in the unavailable_options type. This intent is matched when the user selects a choice contained within the unavailable_options type.

To create and configure the other_option intent, follow these steps:

1. Click the + (plus sign) under Custom Intents.
2. Type other_option and press Enter.
3. Click other_option to open the window.
4. Add the following training phrases and press Enter after each:

• I want to use spells
• I really really want to use a phone
• magic!
• cash
• I want to ride a horse

5. In the Add intent parameters section, update the parameter name to chosenUnavailableOption.
6. Click Save.

As you enter the training phrases, Actions Builder recognizes spells, phone, magic, cash, and horse from the unavailable_options type and automatically highlights (annotates) those words. Actions Builder automatically adds an intent parameter in the Add intent parameters section, as shown in the following image.

If the types aren't automatically annotated, highlight the words you want to annotate (spells, phone, magic, cash, and horse) and select the chosenUnavailableOption parameter. Under Data type, select unavailable_options.

The intent parameter allows you to extract the name of the option and use that option in a prompt.

Add other_option intent to Fortune scene

You now have an intent, other_option, that can handle a user specifying an option that isn't one of the original options. In this section, you add the other_option intent to the Fortune scene. You use the intent parameter to customize the prompt based on the user's input.

To add the other_option intent to the Fortune scene, follow these steps:

1. Click the Fortune scene.
2. Click the + (plus sign) next to User intent handling.
3. Select other_option from the intent drop-down menu.
4. Select Send prompts and add the following prompt:

candidates:
  - first_simple:
      variants:
        - speech: >-
            I have seen the future and a $intent.params.chosenUnavailableOption.original
            will not aid you on your journey. 

The expression $intent.params.chosenUnavailableOption refers to the intent parameter object and $intent.params.chosenUnavailableOption.original refers to that object's value. The original property refers to the raw input the user specifies.

5. Click Save.

When a user says an option listed in the unavailable_options type during the Fortune scene, the other_option intent is matched and adds a prompt to the prompt queue. Since there is no transition specified, the scene execution loop continues by reevaluating the conditions stage. The chosenOptions slot then adds its prompt to the prompt queue, and the prompt queue is delivered to the user.

Test your Action in the simulator

Your Action should now respond appropriately when a user selects one of the options listed in the unavailable_options type and specify which aid the user selected. Your Action should then prompt the user to pick one of the original choices (a dragon, translator, or compass).

To test your Action in the simulator, follow these steps:

1. In the navigation bar, click Test.
2. Type Talk to my test app in the Input field and press Enter.
3. Type Yes in the Input field and press Enter. Alternatively, click the Yes suggestion chip.
4. Type magic in the Input field and press Enter.

You may notice that the prompt doesn't sound correct when the user chooses "magic" because of the "a" article placed before it. You address this issue in the following sections.

Add unavailable_options handler

To place the "a" article before the appropriate choices from the unavailable_options type, you can configure an event handler in your fulfillment logic to check if the option the user chooses needs an "a" before it. First, you need to configure your Action to call the handler in the console.

To add the unavailable_options handler, follow these steps:

1. Click Develop in the navigation bar.
2. Click the Fortune scene.
3. Under User intent handling, click When other_option is matched to open the window.
4. Clear the Send prompts checkbox.
5. Select the Call your webhook checkbox.
6. Enter unavailable_options in the text box for the event handler.

7. Click Save.

Update and deploy fulfillment

Now that you've configured your Action to call the unavailable_options event handler, you can update the handler in your fulfillment and deploy it.

To update your fulfillment, follow these steps:

1. Click Webhook in the navigation bar.
2. Add the following code under the greeting handler:

app.handle('unavailable_options', conv => {
  const option = conv.intent.params.chosenUnavailableOption.original;
  const optionKey = conv.intent.params.chosenUnavailableOption.resolved;
  let message = 'I have seen the future and ';
  if(optionsNeedA.has(optionKey)){
    message = message + 'a ';
  }
  message = message + `${option} will not aid you on your journey. `;
  conv.add(message);
});

3. Add the following code under const app = conversation();:

const optionsNeedA = new Set();
optionsNeedA.add('horse').add('phone');

4. Click Save Fulfillment.
5. Click Deploy Fulfillment. When deployment is complete, a message above your editor reads Your Cloud Function deployment is up to date.

Understand the code

The unavailable_options handler does the following:

• Gets option data from the conv object and assigns option to the original property, which is the raw input from the user
• Assigns optionKey to the resolved property, which is the key for the unavailable_options type
• Checks if optionKey is one of the options that needs an "a"; if it is, constructs the message with an added "a"
• Adds the message via conv.add(message)

Test your Action in the simulator

Your Action should now adjust the prompt based on whether the user's choice from the unavailable_options type requires an "a" article before it.

To test your Action, follow these steps:

1. In the navigation bar, click Test.
2. Click or type Talk to my test app in the Input field and press Enter.
3. Type Yes in the Input field and press Enter. Alternatively, click the Yes suggestion chip.
4. Type magic in the Input field and press Enter.
5. Type horse in the Input field and press Enter.

Your Action should add the "a" article before the "horse" choice, while constructing the prompt without the "a" article for the "magic" choice.

Clean up your project [recommended]

To avoid incurring possible charges, it is recommended to remove projects that you don't intend to use. To delete the projects you created in this codelab, follow these steps:

1. To delete the Cloud Project and resources, complete the steps listed in the Shutting down (deleting) projects section.

Ensure that you select the correct project for deletion on the Google Cloud Platform Settings page.

2. Optional: To immediately remove your project from the Actions console, see Delete a project. If you don't complete this step, your project will automatically be removed after approximately 30 days.

5. Congratulations!

You've now covered the intermediate skills necessary to build Actions for Google Assistant.

What you covered

• How to develop Conversational Actions using the Node.js fulfillment library
• How to use slots to gather data from the user
• How to use conditions to add logic to the scene
• How to add a game loop
• How to add a supportive path

Learn more

Explore the following resources to learn more about building Actions for Google Assistant:

• Documentation for developing Actions for Google Assistant
• Actions on Google GitHub page for sample code and libraries
• The official Reddit community for developers working with Google Assistant
• Conversation design guidelines for best practices and guidelines regarding Conversational Actions

Follow @ActionsOnGoogle on Twitter to stay tuned to our latest announcements and tweet to #AoGDevs to share what you have built!

Feedback survey

Before you go, please fill out a brief survey about your experience.