Geospatial developer guide for Unity (AR Foundation) targeting iOS

Learn how to use the Geospatial API in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

See the Introduction to the Geospatial API for more information about the Geospatial API.

If you're new to developing with ARCore, see Getting started for information about software and hardware requirements, prerequisities and other information specific to the platforms you are using.

To use the ARCore Geospatial API, your project must support AR Foundation and the ARCore Extensions for AR Foundation.

Set up a Google Cloud Project

To use the Visual Positioning System (VPS), your app needs to be associated with a Google Cloud Project that is enabled for the ARCore API.

You must enable the ARCore API in your Google Cloud Project. If you need to create the project, do the following:

  1. Visit Create a project in Google Cloud Platform.

  2. Enter an appropriate Project name, and choose a location for it.

  3. Click Create.

  4. In the sidebar, select APIs & Services, then Library.

  5. Search for the ARCore API, select it, and click Enable.

Set up authorization

To make Geospatial API calls to the VPS, your app needs authorization. Keyless authorization is preferred, but API Key authorization is also supported.

The default authentication strategy for new Unity projects built with ARCore SDK 1.24.0 or later is DoNotUse. This is to prevent apps from being built with unnecessary libraries.

Token (signed JWT) authorization

If you previously used an API key and no longer need it, delete it in the Google Cloud Platform Console and remove it from your app.

ARCore supports the authorization of API calls in iOS using a (JSON Web token). The token must be signed by a Google Service account.

In order to generate tokens for iOS, you must have an endpoint on your server that satisfies the following requirements:

  • Your own authorization mechanism must protect the endpoint.

  • The endpoint must generate a new token every time, such that:

    • Each user gets a unique token.
    • Tokens don’t immediately expire.

Create a service account and signing key

Follow these steps to create a Google service account and signing key:

  1. In the navigation menu of the Google Cloud Platform Console, go to APIs & Services > Credentials.

  2. Select the desired project, then click Create Credentials > Service account.

  3. Under Service account details, type a name for the new account, then click Create.

  4. On the Service account permissions page, go to the Select a role dropdown. Select Service Accounts > Service Account Token Creator, then click Continue.

  5. On the Grant users access to this service account page, click Done.

    This takes you back to APIs & Services > Credentials.

  6. On the Credentials page, scroll down to the Service Accounts section and click the name of the account you just created.

  7. On the Service account details page, scroll down to the Keys section and select Add Key > Create new key.

  8. Select JSON as the key type and click Create.

    This downloads a JSON file containing the private key to your machine. Store the downloaded JSON key file in a secure location.

Create tokens on your server

To create new tokens (JWTs) on your server, use the standard JWT libraries and the JSON file that you securely downloaded from your new service account.

Create tokens on your development machine

To generate JWTs on your development machine, use the following oauth2l command:

oauth2l fetch --cache "" --jwt --json $KEYFILE --audience "https://arcorecloudanchor.googleapis.com/"

Specifying an empty cache location using the --cache flag is necessary to ensure that a different token is produced each time. Be sure to trim the resulting string. Extra spaces or newline characters will cause the API to reject the token.

Sign the token

You must use the RS256 algorithm and the following claims to sign the JWT:

  • iss — The service account email address.
  • sub — The service account email address.
  • iat — The Unix time when the token was generated, in seconds.
  • expiat + 3600 (1 hour). The Unix time when the token expires, in seconds.
  • aud — The audience. The audience must be set to https://arcorecloudanchor.googleapis.com/.

Non-standard claims are not required in the JWT payload, though you may find the uid claim useful for identifying the corresponding user.

If you use a different approach to generate your JWTs, such as using a Google API in a Google-managed environment, make sure to sign your JWTs with the claims in this section. Above all, make sure that the audience is correct.

Pass the token in the ARCore session

Note the following when you pass a token into the session:

  • You must pass in a valid authorization token before attempting to host or resolve an anchor.

  • ARCore ignores authorization tokens that contain spaces or special characters.

  • ARCore ignores provided authorization tokens if the session is created with a valid API key.

    If you previously used an API key and no longer need it, delete it in the Google Cloud Platform Console and remove it from your app.

  • Tokens typically expire after one hour. If there is a possibility that your token may expire while in use, obtain a new token and call ARAnchorManager.SetAuthToken(string authToken) with the new token.

When you obtain a token, pass it into your ARCore session using ARAnchorManager.SetAuthToken():

// Designate the token to use when authenticating with the ARCore Cloud Anchor API
// on the iOS platform. This should be called each time the application's token is refreshed.
ARAnchorManager.SetAuthToken(string authToken);

Configure the project for JWT

  1. In Unity, go to Edit > Project Settings > XR > ARCore Extensions.

  2. Select the iOS Support Enabled checkbox.

  3. In the iOS Authentication Strategy drop-down, select Authentication Token.

  4. Under Optional Features, check the boxes for any of the APIs listed here that you are using in your project.

API key authorization

Follow these steps to obtain and add an API key to your project:

  1. Get the key from Google Cloud Platform, as described in the Google Cloud Platform Console Help Center.

  2. In Unity, go to Edit > Project Settings > XR > ARCore Extensions.

  3. Select the iOS Support Enabled checkbox.

  4. In the iOS Authentication Strategy drop-down, select API Key.

  5. In the iOS API Key field, paste the API key you obtained from Google Cloud Platform.

  6. Under Optional Features, check the boxes for any of the APIs listed here that you are using in your project.

Prompt user to allow usage of device data

Apps that use the ARCore Geospatial API must present the user with a prompt to acknowledge and allow the use of data from their device. See User privacy requirements for more information.

Check device compatibility

Not all devices that support ARCore also support the Geospatial API. To check the user's device for compatibility, call AREarthManager.IsGeospatialModeSupported(). If this returns FeatureSupported.Unsupported do not attempt to configure the session.

Ask user for location permissions at runtime

To enable Unity's location services in a script that triggers the runtime permission requests, do the following:

  1. In Project Settings > iOS > Other Settings > Location Usage Description, enter the name of the app that is requesting permissions.

  2. Enable Unity's location services to trigger the runtime permission request, as follows:

    public void OnEnable()
    {
        Input.location.Start();
    }
    
    public void OnDisable()
    {
        Input.location.Stop();
    }
    

    See Unity's LocationService documentation for more information.

Get a precise location

The CameraGeospatialPose describes a specific location, altitude, and compass heading relative to the Earth. It is managed in an AREarthManager object.

Check the tracking state

Geospatial values are only valid while TrackingState.EarthTrackingState is TrackingState.Tracking. Otherwise, TrackingState may be Limited or None. Always wrap your Geospatial API calls in a TrackingState control block, as shown below.

var earthTrackingState = EarthManager.EarthTrackingState;
if (earthTrackingState == TrackingState.Tracking)
{
     // Values obtained by the Geospatial API are valid as long as
     // earthTrackingState is TrackingState.Tracking.
     // Use Geospatial APIs in this block.
}

See also, AREarthManager.EarthState for other error states and conditions that may prevail. When TrackingState is Limited or None, AREarthManager.EarthState may describe the cause of this failure.

Obtain camera Geospatial pose

While the TrackingState is Tracking, request the camera's pose relative to the Earth:

var earthTrackingState = EarthManager.EarthTrackingState;
if (earthTrackingState == TrackingState.Tracking)
{
  // camera_geospatial_pose contains geodetic location, rotation, and
  // confidences values.
  var cameraGeospatialPose = EarthManager.CameraGeospatialPose;
}

This gives you a GeospatialPose, which contains the following information:

  • Location, expressed in latitude and longitude. An estimate of the location accuracy is also supplied.
  • Altitude, and an estimate of the altitude's accuracy.
  • Heading, an approximation of the direction the device is facing, and an estimate of the accuracy of the heading.

Adjust for pose accuracy

The accuracy of the pose from the VPS may vary, due to the availability of VPS data for the location, or due to temporal conditions at the location. Your app may have to make adjustments for the accuracy of the pose, as determined by the Geospatial API.

The Geospatial API provides an estimate for the accuracy of the latitude/longitude, altitude, and heading values returned from the VPS.

For example, if the heading value returned from GeospatialPose.Heading is 60 degrees, and the value from GeospatialPose.HeadingAccuracy is 10, there is a 68% probability that the VPS heading is within 10 degrees of the observed heading, as illustrated in the diagram on the left.

Heading accuracy

If the value from GeospatialPose.HeadingAccuracy is 15, there is a 68% chance that the true heading is within 15 degrees of 60 degrees, as shown in the diagram on the right. Note that the higher the value returned from GeospatialPose.HeadingAccuracy, the lower the accuracy of the heading value from GeospatialPose.Heading.

Similarly, GeospatialPose.HorizontalAccuracy reports the number of meters within which the true latitude/longitude value has a 68% probability of being within the given distance, and GeospatialPose.VerticalAccuracy reports the number of meters within which the true altitude value has a 68% probability of being within the given distance.

Enable the Geospatial API in your project

  1. Ensure that the project Assets folder contains an ARCoreExtensionsConfig scriptable object.

    To create one, in the Assets pane, right-click and select Create > ARCore Extensions > ARCore Extensions Config.

  2. Select the ARCoreExtensionsConfig scriptable object, and, in the Inspector window, for Geospatial Mode select Enabled from the drop-down menu.

  3. Configure the ARCore Extensions game object to use the ARCoreExtensionsConfig configuration:

    1. In Hierarchy pane locate the ARCore Extensions game object you created when you initially set up ARCore Extensions.

    2. Connect the ARCore Extensions Config field to the ARCoreExtensionsConfig scriptable object in your Assets folder.

Place a Geospatial anchor

When placing an anchor at the specified location and orientation relative to the Earth, latitude and longitude are defined by the WGS84 specification, and the altitude value is defined by the elevation above the WGS84 ellipsoid in meters. The rotation quaternion provided is with respect to an east-up-south (EUS) coordinate frame. An identity rotation has the anchor oriented such that X+ points to the east, Y+ points up, away from the center of the earth, and Z+ points to the south.

To create a Pose.rotation quaternion that has the +Z axis pointing in the same direction as the heading obtained from GeospatialPose, use the following formula.

Quaternion quaternion =
                Quaternion.AngleAxis(180f - (float)pose.Heading, Vector3.up);

Use ARAnchorManager.AddAnchor from ARAnchorManagerExtensions to create an ARGeospatialAnchor to anchor content to geographical coordinates that you specify.

if (earthTrackingState == TrackingState.Tracking)
{
  var anchor =
      AnchorManager.AddAnchor(
          latitude,
          longitude,
          altitude,
          quaternion);
  var anchoredAsset = Instantiate(GeospatialAssetPrefab, anchor.transform);
}

Calculate the altitude

Getting the altitude for placing an anchor is a bit tricky. You have to compare the altitude at the device's location against the altitude from the Geospatial API. There are two ways to determine the altitude:

  • If the anchor's location is near the user, consider using an altitude that's similar to the device's altitude.
  • Otherwise, consider getting the altitude value from the elevation in Google Maps.

Use the altitude from the camera's Geospatial pose

It may suffice to simply use the altitude you get from the Camera's GeospatialPose without cross-checking against data from another source such as the Maps API. See Obtain camera Geospatial pose, above.

If you can obtain the pose at the location in advance, by making your own local observation, you can use that data to cross-check the GeospatialPose obtained by your app for the user.

Get the elevation from Google Maps

The GeospatialPose reports altitude in meters above the WGS84 ellipsoid. Your app needs to compare the value from GeospatialPose.Altitude against the altitude at the device's location to determine an accurate altitude for the anchor.

You can get the latitude and longitude using Google Maps. Make sure to turn off Globe View first by navigating to Layers > More and deselecting the Globe View checkbox at the bottom left-hand corner of the screen. This will force a 2D perspective and eliminate possible errors that could come from an angled 3D view.

Once you have a a latitude and longitude, use the Elevation API to get an elevation based on the EGM96 specification. You must convert the Maps API EGM96 elevation to WGS84 for comparison against the GeospatialPose altitude. See the GeoidEval conversion tool that has both a command line and an HTML interface. The Maps API reports latitude and longitude according to the WGS84 specification out of the box.

API usage quota

The ARCore SDK limits API requests to the ARCore service to the following limits for each project that uses the ARCore SDK:

  • 1,000 sessions started per minute, or
  • 100,000 requests per minute

API requests in excess of these limits may result

in an EarthState.ErrorResourcesExhausted error and an unfulfilled request.