Report State

Note: You must implement Report State in order to submit a smart home Action.

Report State is an important feature which lets the smart home Action proactively report the latest status of the user’s device back to Google’s Home Graph rather than waiting for a QUERY intent.

Report State reports to Google the states of user devices with the specified agentUserId associated with them (sent in the original SYNC request). When the Google Assistant wants to take an action that requires understanding the current state of a device, it can simply look up the state information in the Home Graph instead of issuing a QUERY intent to various third-party clouds prior to issuing the EXECUTE intent.

Without Report State, given lights from multiple providers in a living room, the command Ok Google, brighten my living room requires resolving multiple QUERY intents sent to multiple clouds, as opposed to simply looking up the current brightness values based on what has been previously reported. For the best user experience, the Google Assistant needs to have the current state of a device, without requiring a round-trip to the device.

Home Graph only stores the state that is sent with Report State. The state that is returned as the response to EXECUTE and QUERY intents is used only for speech responses to the user and are not stored in Home Graph. As a result, Report State should be called even if the new state of the device has already been returned in the response to an EXECUTE or QUERY intent. The ReportState API should also be called right after a SYNC intent. After a SYNC intent, new devices might have been added, and in order to set their initial states, a Report State call should follow.

Home Graph expects complete state data on a per-trait basis, as opposed to all state data for the device. For Google Smart Home and Home Graph, traits have state while devices do not. Home Graph updates states on a per-trait basis and overwrites all data for a given trait when a Report State request is sent. For example, if you are reporting state for the StartStop trait, the response needs to include values for both isRunning and isPaused.

If Report State is not implemented, the associated device will not be displayed on visual Assistant surfaces through the QUERY intent. The implementation of Report State is a requirement for the public launch of a Smart Home agent.

Implement Report State

To implement Report State, follow these steps:

• Enable the Google HomeGraph API
• Create a Service Account Key
• Call the API
• Handle the Disconnect intent

Enable the Google HomeGraph API

1. In the Cloud Platform Console, go to the Projects page. Select the project that matches your smart home project ID.
2. Enable the Google HomeGraph API

Create a Service Account Key

GCP Console

Note: Ensure that you are using the correct GCP project when performing these steps. This is the project that matches your smart home project ID.

1. In the GCP Console, go to the Create service account key page.

   Go to the Create Service Account Key page
2. From the Service account list, select New service account.
3. In the Service account name field, enter a name.
4. In the Service account ID field, enter a ID.

5. From the Role list, select Service Accounts > Service Account Token Creator.

6. For the Key type, select the JSON option.

7. Click Create. A JSON file that contains your key downloads to your computer.

Call the API

You have two options to call Report State:

• gRPC
• HTTP POST with a JSON Web Token (JWT)

Select an option from the tabs below:

gRPC

gRPC is a modern open source, high performance RPC framework that can run in any environment. For more information, see the gRPC site.

gRPC is supported on most modern programming languages. For most languages, the gRPC runtime can now be installed in a single step via native package managers such as npm for Node.js, gem for Ruby and pip for Python. Even though our Node, Ruby and Python runtimes are wrapped on gRPC’s C core, users now do not need to explicitly pre-install the C core library as a package in most Linux distributions. For Java, we have simplified the steps needed to add gRPC support to your build tools by providing plugins for Maven and Gradle.

1. To install gRPC for the language of your choice, see the following external links:
   • Runtime installations
   • OpenSSL: Statically Linked
   • Java Maven or Gradle dependencies
   • Eclipse or Netbeans IDE integration
2. Download and place the proto file according to documentation for your language.
3. Place the service account key in your project.
4. Proto Compiler should generate the necessary files for calling the gRPC service. See the generated packages and classes for Java below:
   

   

5. Implement the code to the gRPC service. See the Authentication page for more information.
6. Set the same `requestId` you received from the EXECUTE request (if any) and set the states. A Java example is given below:

   private HomeGraphApiServiceGrpc.HomeGraphApiServiceBlockingStub blockingStub;
   public void reportStateWithGrpc(String agentUserId) {
     GoogleCredentials creds;
     try {
       FileInputStream stream = new FileInputStream("key.json");
       creds = GoogleCredentials.fromStream(stream);
   
       ManagedChannel channel = ManagedChannelBuilder.forTarget("homegraph.googleapis.com").build();
   
       blockingStub = HomeGraphApiServiceGrpc.newBlockingStub(channel)
           // See https://grpc.io/docs/guides/auth.html#authenticate-with-google-3.
           .withCallCredentials(MoreCallCredentials.from(creds));
         ReportStateAndNotificationRequest request =
                 ReportStateAndNotificationRequest.newBuilder()
                     .setRequestId(requestId)
                     .setAgentUserId(agentUserId)
                     .setPayload(
                         StateAndNotificationPayload.newBuilder()
                             .setDevices(
                                 ReportStateAndNotificationDevice.newBuilder()
                                     .setStates(getStates())))
                     .build();
   
             System.out.printf("Calling ReportStateAndNotification with request "+ request);
             ReportStateAndNotificationResponse response = blockingStub.reportStateAndNotification(request);
       System.out.println(response.toString());
     } catch (IOException e) {
       e.printStackTrace();
     }
   }
   
   public Builder getStates() {
       // States
       Struct colorTemperature =
           Struct.newBuilder()
               .putFields("name", Value.newBuilder().setStringValue("Yellow").build())
               .putFields("temperature", Value.newBuilder().setNumberValue(24000).build())
               .build();
       Struct colorSpetrum =
           Struct.newBuilder()
               .putFields("name", Value.newBuilder().setStringValue("Red").build())
               .putFields("spectrumRGB", Value.newBuilder().setNumberValue(0xff0000).build())
               .build();
   
       Value device1States =
           Value.newBuilder()
               .setStructValue(
                   Struct.newBuilder()
                       .putFields("color", Value.newBuilder().setStructValue(colorTemperature).build())
                       .build())
               .build();
       Value device2States =
           Value.newBuilder()
               .setStructValue(
                   Struct.newBuilder()
                       .putFields("on", Value.newBuilder().setBoolValue(true).build())
                       .putFields("brightness", Value.newBuilder().setNumberValue(98.0).build())
                       .putFields("color", Value.newBuilder().setStructValue(colorSpetrum).build())
                       .putFields("thermostatMode", Value.newBuilder().setStringValue("heat").build())
                       .putFields(
                           "thermostatTemperatureSetpoint",
                           Value.newBuilder().setNumberValue(78).build())
                       .putFields(
                           "thermostatTemperatureAmbient",
                           Value.newBuilder().setNumberValue(68).build())
                       .putFields(
                           "thermostatTemperatureSetpointHigh",
                           Value.newBuilder().setNumberValue(99).build())
                       .putFields(
                           "thermostatTemperatureSetpointLow",
                           Value.newBuilder().setNumberValue(50).build())
                       .putFields(
                           "thermostatHumidityAmbient", Value.newBuilder().setNumberValue(45).build())
                       .build())
               .build();
       Builder states =
           Struct.newBuilder().putFields(deviceId1, device1States).putFields(deviceId2, device2States);
       return states;
   }
   
   

HTTP POST

1. Use the downloaded service account JSON file to create a JSON Web Token. For more information, see Authenticating Using a Service Account.
2. Construct a JWT payload where the iss field comes from the service account JSON file that you downloaded. The aud, iat and exp fields will be provided by your fulfillment at runtime and the scope needs to include homegraph. See the example below:

{
  "iss": "<service-account-email>",
  "scope": "https://www.googleapis.com/auth/homegraph",
  "aud": "https://accounts.google.com/o/oauth2/token",
  "iat": <current-time>,
  "exp": <current-time-plus-one-hour>
}

3. Sign the JWT payload with the private key from your service account.
4. Use the JWT to request an access token from https://accounts.google.com/o/oauth2/token.
5. Create the JSON request with the agentUserId and list of devices and their states. Use the same requestId you received from the EXECUTE request (if any). Here's a sample JSON request for Report State:

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "agentUserId": "1234",
  "payload": {
    "devices": {
      "states": {
        "1458765": {
          "on": true
        },
        "4578964": {
          "on": true,
          "isLocked": true
        }
      }
    }
  }
}

6. Combine the Report State JSON and the token in your HTTP POST request to the Google Home Graph endpoint. Here's an example of how to make the request in the command line using curl, as a test:

TOKEN=$ACCESS_TOKEN
DEVICE_STATE=`cat device-state.json`
curl -i -s -X POST -H "Authorization: Bearer $TOKEN" -H "X-GFE-SSL: yes" \
-H "Content-Type: application/json" \
-d "$DEVICE_STATE"  https://homegraph.googleapis.com/v1/devices:reportStateAndNotification

Code samples

Java

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.UnsupportedEncodingException;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClients;
import org.json.JSONException;
import org.json.JSONObject;

import com.google.auth.oauth2.ServiceAccountCredentials;

import io.jsonwebtoken.JwtBuilder;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;

public class TestReportStatePost {

  private static final String KEY_JSON = "";
  private static final String AGENT_USER_ID = "";

  public static void main(String[] args) throws IOException, JSONException {

    TestReportStatePost reportState = new TestReportStatePost();
    // create and sign jwt
    String jwt = reportState.getJwt();
    // get access token
    String token = reportState.getAccessToken(jwt);
    // call request sync
    reportState.callRS(token, AGENT_USER_ID);
  }

  private String getJwt() throws IOException {

    FileInputStream stream = new FileInputStream(KEY_JSON);
    ServiceAccountCredentials serviceAccount = ServiceAccountCredentials.fromStream(stream);
    JwtBuilder jwts = Jwts.builder();

    // set claims
    Map claims = new HashMap<>();
    claims.put("exp", System.currentTimeMillis() / 1000 + 3600);
    claims.put("iat", System.currentTimeMillis() / 1000);
    claims.put("iss", serviceAccount.getClientEmail());
    claims.put("aud", "https://accounts.google.com/o/oauth2/token");
    claims.put("scope", "https://www.googleapis.com/auth/homegraph");

    jwts.setClaims(claims).signWith(SignatureAlgorithm.RS256, serviceAccount.getPrivateKey());

    return jwts.compact();
  }

  private String getAccessToken(String jwt) throws JSONException, ClientProtocolException, IOException {
    Map m = new HashMap();
    m.put("grant_type", "urn:ietf:params:oauth:grant-type:jwt-bearer");
    m.put("assertion", jwt);
    // Request parameters and other properties.
    StringEntity params = new StringEntity(formEncode(m));

    HttpEntity entity = httpRequest("https://accounts.google.com/o/oauth2/token", jwt, params,
      "application/x-www-form-urlencoded");

    if (entity != null) {
      final StringBuilder out = readResponse(entity);
      String res = out.toString();
      if (res.indexOf("access_token") > 0) {
        String token = res.split(":")[1];
        token = token.substring(2, token.indexOf(",") - 1);
        return token;
      }
    }
    return "";
  }

  private void callRS(String token, String agentUserId) throws JSONException, ClientProtocolException, IOException {
    JSONObject json = prepareJson(agentUserId);

    StringEntity params = new StringEntity(json.toString());
    HttpEntity entity = httpRequest("https://homegraph.googleapis.com/v1/devices:reportStateAndNotification", token,
      params, "application/json");

    if (entity != null) {
      final StringBuilder out = readResponse(entity);
      System.out.println(out.toString());
    }
  }

  private JSONObject prepareJson(String agentUserId) throws JSONException {
    JSONObject json = new JSONObject();
    // Use the request id from execute request
    json.put("requestId", UUID.randomUUID());
    json.put("agentUserId", agentUserId);
    JSONObject payload = new JSONObject();
    JSONObject devices = new JSONObject();
    JSONObject states = new JSONObject();
    JSONObject device1 = new JSONObject();
    device1.put("on", true);
    device1.put("online", true);
    JSONObject device2 = new JSONObject();
    device2.put("on", true);
    device2.put("online", true);
    device2.put("locked", true);
    states.put("1458765", device1);
    states.put("4578964", device2);
    devices.put("states", states);
    payload.put("devices", devices);
    json.put("payload", payload);
    return json;
  }

  /*
   * Helper methods below
   */
  private HttpEntity httpRequest(String url, String token, StringEntity params, String type)
      throws UnsupportedEncodingException, IOException, ClientProtocolException {
    HttpClient httpclient = HttpClients.createDefault();
    HttpPost httppost = new HttpPost(url);
    httppost.setHeader("Authorization", "Bearer " + token);

    // Request parameters and other properties.
    httppost.addHeader("content-type", type);
    httppost.setEntity(params);

    // Execute and get the response.
    HttpResponse response = httpclient.execute(httppost);
    HttpEntity entity = response.getEntity();
    return entity;
  }

  private StringBuilder readResponse(HttpEntity entity) throws IOException, UnsupportedEncodingException {
    InputStream instream = entity.getContent();
    final int bufferSize = 1024;
    final char[] buffer = new char[bufferSize];
    final StringBuilder out = new StringBuilder();

    try {
      Reader in = new InputStreamReader(instream, "UTF-8");
      while (true) {
        int rsz = in.read(buffer, 0, buffer.length);
        if (rsz < 0)
          break;
        out.append(buffer, 0, rsz);
      }
    } finally {
      instream.close();
    }
    return out;
  }

  private String formEncode(Map m) {
    String s = "";
    for (String key : m.keySet()) {
      if (s.length() > 0)
        s += "&";
      s += key + "=" + m.get(key);
    }
    return s;
  }
}

Python

#!/usr/bin/env python

# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""A tool for calling HomeGraph API with a JWT signed by a Google API Service Account."""

import argparse
import time
import json
import io

import google.auth.crypt
import google.auth.jwt
import requests
from six.moves import urllib

def generate_jwt(service_account_file):
    """Generates a signed JSON Web Token using a Google API Service Account."""

    # Note: this sample shows how to manually create the JWT for the purposes
    # of showing how the authentication works, but you can use
    # google.auth.jwt.Credentials to automatically create the JWT.
    #   http://google-auth.readthedocs.io/en/latest/reference
    #   /google.auth.jwt.html#google.auth.jwt.Credentials

    signer = google.auth.crypt.RSASigner.from_service_account_file(
        service_account_file)

    now = int(time.time())
    expires = now + 3600  # One hour in seconds

    iss = ''

    with io.open(service_account_file, 'r', encoding='utf-8') as json_file:
        data = json.load(json_file)
        iss = data['client_email']

    payload = {
        'iat': now,
        'exp': expires,
        'aud': 'https://accounts.google.com/o/oauth2/token',
        'iss': iss,
        'scope': 'https://www.googleapis.com/auth/homegraph'
    }

    signed_jwt = google.auth.jwt.encode(signer, payload)

    return signed_jwt


def get_access_token(signed_jwt):
    url = 'https://accounts.google.com/o/oauth2/token'
    headers = {'Content-Type': 'application/x-www-form-urlencoded'}
    data = 'grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=' + signed_jwt

    response = requests.post(url, headers=headers, data=data)

    if response.status_code == requests.codes.ok:
        token_data = json.loads(response.text)
        return token_data['access_token']

    response.raise_for_status()
    return 'ERROR'

def report_state(access_token, report_state_file):
    url = 'https://homegraph.googleapis.com/v1/devices:reportStateAndNotification'
    headers = {
        'X-GFE-SSL': 'yes',
        'Authorization': 'Bearer ' + access_token
    }
    data = {}

    with io.open(report_state_file, 'r', encoding='utf-8') as json_file:
        data = json.load(json_file)

    response = requests.post(url, headers=headers, json=data)

    print 'Response: ' + response.text

    return response.status_code == requests.codes.ok

def main(service_account_file, report_state_file):
    signed_jwt = generate_jwt(service_account_file)
    print('signed JWT: ' + signed_jwt)

    access_token = get_access_token(signed_jwt)
    print('access token: ' + access_token)

    success = report_state(access_token, report_state_file)
    if success:
        print 'Report State has been done successfully.'
    else:
        print 'Report State failed. Please check the log above.'

if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument(
        'service_account_file',
        help='The path to your service account json file.')
    parser.add_argument(
        'report_state_file',
        help='The path to the json file containing the states you want to report.')

    args = parser.parse_args()

    main(args.service_account_file, args.report_state_file)

Alternatively, gRPC can be used to call the Report State. The public API and proto is available on https://homegraph.googleapis.com/$discovery/rest.

Node.js

The Actions on Google library for Node.js supports Report State over HTTP.

1. Place the downloaded service account JSON in your project directory.
2. Pass the file location into your smarthome constructor
3. Call the reportState method with your payload. It returns a Promise.

const {smarthome} = require('actions-on-google');
const app = smarthome({
  jwt: mySecretKeyJson
});
// ...
// Device state changed
app.reportState({
  requestId: '123ABC',
  agentUserId: 'user-123',
  payload: {
    devices: {
      states: {
        "light-123": {
          on: true
        }
      }
    }
  }
})
.then((res) => {
  // Report state was successful
})
.catch((res) => {
  // Report state failed
});

Handle the Disconnect intent

In the event of unlinking, user data is removed from the Home Graph and Report State calls will fail, resulting in a 404 error. When the user unlinks their account, similar to other intents, Google sends an action.devices.DISCONNECT intent to the fulfillment url.

{
    "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
    "inputs": [{
      "intent": "action.devices.DISCONNECT",
    }]
}

You should handle the DISCONNECT intent if your fulfillment is set to report state back (either through POST or gRPC). If it is not handled, your Action will continue reporting state and will receive an error back every time.

Test Report State

In order to get your action ready for certification, it is important to test report state.

Prerequisites

Before being able to test your action, you need your Service Account Key and your agentUserId. If you already have your Service Account Key and agentUserId, see Deploy the Report State Dashboard.

• Create a testing Service Account Key
• Retrieve your agentUserId

Create a testing Service Account Key

Note: For certification, this JSON key is different from your production JSON key. You should create a new temporary service account key for testing, which you can then delete once certification is complete.

GCP Console

Note: Ensure that you are using the correct GCP project when performing these steps. This is the project that matches your smart home project ID.

1. In the GCP Console, go to the Create service account key page.

   Go to the Create Service Account Key page
2. From the Service account list, select New service account.
3. In the Service account name field, enter a name.
4. In the Service account ID field, enter a ID.

5. From the Role list, select Service Accounts > Service Account Token Creator.

6. For the Key type, select the JSON option.

7. Click Create. A JSON file that contains your key downloads to your computer.

Retrieve your agentUserId

agentUserId

1. Replace the [e-mail] and your_project_id with valid values for your project and open the page:

   https://gala-demo.appspot.com/app?login_hint=[e-mail]&service_id=your_project_id_dev
   

   The page displays an Access Token. You can use this Access Token to retrieve your agentUserId.

2. Replace the your_access_token and your_fulfillment_URL with valid values for your project and run the curl command:

   curl -X POST -H"content-type: application/json" -d '{"requestId":"ff36a3cc-ec34-11e6-b1a0-64510650abcf","inputs":[{"intent":"action.devices.SYNC"}]}' -H "Authorization: Bearer your_access_token" your_fulfillment_URL' | grep agentUserId
   

Deploy the Report State dashboard

Once you have the Service Account Key and agentUserId for your project, download and deploy the latest version from Report State Dashboard. Once you have downloaded the latest version, follow the instructions from the included README.MD file.

After you have deployed the Report State dashboard, access the dashboard from the following URL (replace your_project_id with your project ID):

http://<your-project-id>.appspot.com

On the dashboard, do the following:

• Choose your Account Key File
• Add your Access Token
• Add your Fulfillment URL
• Add your agentUserId

Then, click List.

All of your devices are listed. Once the list is populated, you can use the Refresh button to update device states. If there is a device state change, the row is highlighted in green.

Error Responses

As you are implementing report state and request sync, there are several possible responses that you will receive from Google. These responses come in the form of HTTP status codes on the response.

• 200 - Success
• 400 - Failure: The 400 Bad Request Error is an HTTP response status code that indicates that the server was unable to process the request sent by the client due to invalid syntax. A couple common causes include malformed JSON or using null instead of "" for a string value.
• 404 - Failure: The requested resource could not be found but may be available in the future. Subsequent requests by the client are permissible. Typically, this means that we cannot find either the user (agentUserId) or the device. It may also mean that the user has not yet linked with Google, or you didn't send the agentUserId in the SYNC response.
• 429 - Failure: The user has sent too many sync requests in a given amount of time. The limit is simply one concurrent sync request per user at a time. We don't allow for concurrent requests to be made for the same user. This is for request sync only, not for report state. Report state does not have this limitation and will accept concurrent requests for the same device.