Cloud Anchors developer guide for ARCore Extensions targeting iOS

Important: The ARCore Cloud Anchor API requires the ARCore Extensions for AR Foundation. Make sure that you have Extensions installed and configured before proceeding.

Learn how to use Cloud Anchors in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

If you are new to Cloud Anchors, make sure that you understand how anchors and Cloud Anchors work.

Enable the ARCore API

Before using Cloud Anchors in your app, you must first enable the ARCore API in a new or existing Google Cloud Platform project. This service is responsible for hosting, storing, and resolving Cloud Anchors.

Authorize your app to call the ARCore API

You must authorize your app to call the ARCore API to host and resolve Cloud Anchors. Apps that host and resolve Cloud Anchors with a TTL greater than 1 day must use token (signed JWT) authorization.

Token (signed JWT) authorization

Use token (signed JWT) authorization to host and resolve Cloud Anchors with TTLs between 1 and 365 days. Currently, the only supported token type is a signed JSON Web Token (JWT). This token must be signed by a Google Service account.

1. Go to Edit > Project Settings > XR Plug-in Management > ARCore Extensions and check iOS Support Enabled.

   

2. From the iOS Authentication Strategy drop-down menu, select the Authentication Token option.

   

Server endpoint requirements

To generate tokens for iOS, you must have an endpoint on your server that satisfies the following requirements:

1. Your own authorization mechanism protects the endpoint.
2. The endpoint must generate a new token every time such that each user gets a unique token, and such that tokens don’t immediately expire.

Create a service account and signing key

Follow these steps to create a Google Service account and signing key.

1. In the navigation menu of the Google Cloud Platform Console, go to APIs & Services > Credentials.
2. Select the desired project and click Create Credentials > Service account.
3. Under Service account details, type a name for the new account and click Create.
4. On the Service account permissions page, go to the Select a role drop-down menu. Select Service Accounts > Service Account Token Creator and click Continue.
5. On the Grant users access to this service account page, click Done. This will take you back to APIs & Services > Credentials.
6. On the Credentials page, scroll down to the Service Accounts section and click the name of the account you just created.
7. On the Service account details page, scroll down to the Keys section and select Add Key > Create new key.
8. Select JSON as the key type and click Create. This downloads a JSON file containing the private key to your machine. Store the downloaded JSON key file in a secure location.

The JSON file contains a private key which must not be exposed to the public. Do not commit it to source code repositories like GitHub.

Create tokens on your server

To create new tokens (JWTs) on your server, use the standard JWT libraries and the JSON file that you securely downloaded from your new service account.

Create tokens on your development machine

To generate JWTs on your development machine, use the following oauth2l command:

// "oauth2l" uses a lowercase L, not a 1, as the last character.
// Specifying an empty cache location using the --cache flag is necessary
// to ensure that a different token is produced each time.
oauth2l fetch --cache "" --jwt --json $KEYFILE --audience "https://arcore.googleapis.com/"

Be sure to trim the resulting string. Extra spaces or newline characters will cause the API to reject the token.

Sign the token

To sign the JWT, use the RS256 algorithm and the following claims:

• iss — The service account email address.
• sub — The service account email address.
• iat — The Unix time when the token was generated, in seconds.
• exp — iat + 3600 (1 hour). The Unix time when the token expires, in seconds.
• aud — The audience. Must be set to https://arcore.googleapis.com/.

Non-standard claims are not required in the JWT payload, though you may find the uid claim useful for identifying the corresponding user.

If you use a different approach to generate your JWTs, such as using a Google API in a Google-managed environment, make sure to sign your JWTs with the claims in this section. Above all, make sure that the audience is correct.

Pass a token into the ARCore session

When you obtain a token, pass it into your ARCore session using ARAnchorManager.SetAuthToken():

// Designate the token to use when authorizing with the ARCore API
// on the iOS platform. The API should be called each time the application's token is refreshed.
ARAnchorManager.SetAuthToken(string authToken);

Note the following when you pass a token into the session:

• You must pass in a valid authorization token before attempting to host or resolve an anchor.
• ARCore ignores authorization tokens that contain spaces or special characters.
• ARCore ignores provided authorization tokens if the session is created with a valid API key. If you previously used an API key and no longer need it, we recommend deleting it in the Google Cloud Platform Console and removing it from your app after migrating users to the newest version.
• Tokens typically expire after one hour. If there is a possibility that your token may expire while in use, obtain a new token and call ARAnchorManager.SetAuthToken(string authToken) with the new token.

API key authorization

Use API key authorization to host and resolve Cloud Anchors with TTLs up to 24 hours (1 day).

The default authorization strategy for new Unity projects built with ARCore SDK 1.24.0 or later is DoNotUse. This is to prevent apps from being built with unnecessary libraries. If your app uses Cloud Anchors and is built using ARCore SDK 1.24.0 or later, you must manually enable authorization in Project Settings > XR Plug-in Management > ARCore Extensions.

1. Go to Edit > Project Settings > XR Plug-in Management > ARCore Extensions and check iOS Support Enabled.

   

2. From the iOS Authentication Strategy drop-down menu, select the API Key option.

   

3. Obtain an API key for this project from the Google Cloud Console.

4. Go to Edit > Project Settings > XR Plug-in Management > ARCore Extensions and add your API key to the Cloud Anchor API Keys field.

   

Enable Cloud Anchors functionality in your app

After authorizing your app to call the ARCore API, you must enable Cloud Anchors functionality in your app.

1. Navigate to Edit > Project Settings > XR Plug-In Management > ARCore Extensions. Make sure that iOS Support Enabled is selected.
2. Under Optional Features, select Cloud Anchors.

Enable Cloud Anchor capabilities in the session configuration

Once Geospatial functionality has been enabled in your app, enable Geospatial capabilities in your app’s AR session configuration so that it can communicate with the ARCore API:

1. Ensure that the project Assets folder contains an ARCoreExtensionsConfig scriptable object. To create one, right-click in the Assets pane and select Create > XR > ARCore Extensions Config.

2. Select the ARCoreExtensionsConfig scriptable object in your Assets folder and set the Cloud Anchor Mode to Enabled.

   

3. Configure the ARCore Extensions game object to use the ARCoreExtensionsConfig configuration. In the Hierarchy pane, locate the ARCore Extensions game object you created when you initially set up ARCore Extensions, and connect the ARCore Extensions Config field to the ARCoreExtensionsConfig scriptable object in your Assets folder.

   

Host a Cloud Anchor

Hosting starts with a call to ARAnchorManager.HostCloudAnchor(). ARCore will upload visual data, device poses, and the anchor pose to the ARCore API. The API then processes this information to construct a 3D feature map, ultimately returning a unique Cloud Anchor ID for the anchor to the device.

Note: Visual data uploaded to the cloud when hosting an anchor is discarded within 24 hours, and previously uploaded visual data is never sent to a user's device.

You can also extend the lifetime of a hosted anchor using the ARCore Cloud Anchor Management API.

Your app should follow these steps to complete hosting of a Cloud Anchor:

1. Call ARAnchorManager.HostCloudAnchor().
2. Continually check the state of the new anchor with ARCloudAnchor.cloudAnchorState() until the state is SUCCESS.
3. Check that cloudanchorid() returns a string identifier.
4. Share the identifier with other clients, and use it to resolve the Cloud Anchor with ARAnchorManagerExtensions.ResolveCloudAnchorId().

Check the mapping quality of feature points

ARCoreExtensions.FeatureMapQuality indicates the quality of feature points seen by ARCore in the preceding few seconds from a given camera pose. Cloud Anchors hosted using higher quality features are generally more accurately resolved. Use ARAnchorManagerExtensions.EstimateFeatureMapQualityForHosting() to obtain an estimation for the feature map quality for a given camera pose.

Value	Description
Insufficient	The quality of feature points identified from the pose in the preceding few seconds is low. This state indicates that ARCore will likely have more difficulty resolving the Cloud Anchor. Encourage the user to move the device so that the desired position of the Cloud Anchor that they wish to host can be viewed from different angles.
Sufficient	The quality of feature points identified from the pose in the preceding few seconds is likely sufficient for ARCore to successfully resolve a Cloud Anchor, although the accuracy of the resolved pose will likely be reduced. Encourage the user to move the device so that the desired position of the Cloud Anchor that they wish to host can be viewed from different angles.
Good	The quality of feature points identified from the pose in the preceding few seconds is likely sufficient for ARCore to successfully resolve a Cloud Anchor with a high degree of accuracy.

Resolve a previously hosted anchor

Call ARAnchorManagerExtensions.ResolveCloudAnchorId() to resolve a hosted Cloud Anchor. The ARCore API periodically compares visual features from the scene against the anchor’s 3D feature map to pinpoint the user's position and orientation relative to the anchor. When it finds a match, the API returns the pose of the hosted Cloud Anchor.

You can initiate resolves for multiple Cloud Anchors in sequence. Up to 40 Cloud Anchors can be resolved simultaneously.

Cancel or remove a Cloud Anchor

ARCloudAnchor.OnDestroy() is automatically called when the ARCloudAnchor component is removed from the game object containing it. This will detach and release the underlying native Cloud Anchor object.

Check the state of a Cloud Anchor

Use ARCloudAnchor.cloudAnchorState() to check the status of the hosted anchor or the resolve request, including errors.

Value	Description
ErrorResolvingCloudIdNotFound	Resolving failed because the ARCore API could not find the provided Cloud Anchor ID.
ErrorHostingDatasetProcessingFailed	Hosting failed because the server could not successfully process the dataset for the given anchor. Try again after the device has gathered more data from the environment.
ErrorHostingServiceUnavailable	The ARCore API was unreachable. This can happen for a number of reasons. The device might be in airplane mode or may not have a working Internet connection. The request sent to the server might have timed out with no response. There might be a bad network connection, DNS unavailability, firewall issues, or anything else that might affect the device's ability to connect to the ARCore API.
ErrorInternal	A hosting or resolving task for this anchor finished with an internal error. The app should not attempt to recover from this error.
ErrorNotAuthorized	The app cannot communicate with the ARCore API because of invalid authorization. Check Project Settings > XR > ARCore Extensions for a valid authorization strategy.
ErrorResolvingPackageTooNew	The Cloud Anchor could not be acquired because the ARCore Extensions package used to host the Cloud Anchor is newer than, and thus incompatible with, the version being used to acquire it.
ErrorResolvingPackageTooOld	The Cloud Anchor could not be acquired because the ARCore Extensions package used to host the Cloud Anchor is older than, and thus incompatible with, the version being used to acquire it.
ErrorResourceExhausted	The application has exhausted the request quota allotted to the given Google Cloud project. You should request additional quota for the ARCore API for your project from the Google Developers Console.
None	The Cloud Anchor is not ready to use.
Success	A hosting or resolving task for this anchor completed successfully.
TaskInProgress	A hosting or resolving task is in progress for this Cloud Anchor. Once the task completes in the background, the Cloud Anchor will get a new state after the next update.

API quotas for host and resolve requests

The ARCore API has the following quotas for request bandwidth:

Quota type	Maximum	Duration	Applies to
Number of anchors	unlimited	N/A	project
Anchor host requests	30	minute	IP address and project
Anchor resolve requests	300	minute	IP address and project

Best practices for a good user experience

Instruct users to do the following to ensure a good user experience on your app:

• Wait a few seconds after the session starts before attempting to host an anchor (by placing an object, etc.). This gives the tracking some time to stabilize.
• When selecting a location to host the anchor, try to find an area with visual features that are easily distinguishable from one another. For best results, avoid reflective surfaces or surfaces that lack visual features, such as blank white walls.

• Keep the camera trained on the center of interest and move the device around the center of interest to map the environment from different angles, maintaining roughly the same physical distance as you do so. This will help capture more visual data and make resolving more robust.

  Note: Moving around the center of interest is not the same as standing in one place and rotating, as someone might do when taking a panorama shot. Users should physically move around the environment.

• Make sure that there is sufficient lighting in the real-life environment while hosting and resolving Cloud Anchors.

Deprecation policy

• Apps built with ARCore SDK 1.12.0 or higher are covered by the Cloud Anchor API deprecation policy.
• Apps built with ARCore SDK 1.11.0 or lower are unable to host or resolve Cloud Anchors due to the SDK's use of an older, deprecated ARCore API.

What's next

• Check out the ARCore Extensions for AR Foundation reference documentation for more ways to use ARCore in your app.