Geospatial developer guide for iOS

Learn how to use the Geospatial API in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

If you want to run a sample app that demonstrates the functionality described here, see the ARCore Geospatial quickstart for iOS.

See the Introduction to the Geospatial API for more information about the Geospatial API.

If you're new to developing with ARCore, see Getting started for information about software and hardware requirements, prerequisities and other information specific to the platforms you are using.

Add the ARCore SDK to your app

Update the Podfile for your app to include the ARCore SDK and supported iOS versioning. To do this:

1. Add the following platform and pod to your project's Podfile:

   platform :ios, '11.0'
   pod 'ARCore/Geospatial', '~> ios_sdk_version'
   

   You may also specify platform :ios, '10.0', if you want to support iOS 10, but note that the Geospatial API will only function at runtime on iOS >= 11.

   Note: To ensure that your project includes the correct ARCore version, specify all three of the ARCore version digits in your pod. For example, use 1.31.0 instead of 1.31.

2. Open a Terminal window and run pod install from the folder where your Xcode project exists.
   

   This generates an .xcworkspace file that you use to build and run the app.

Be sure your development environment satisfies the ARCore SDK requirements, as described in the Quickstart.

Set up a Google Cloud Project

To use the Visual Positioning System (VPS), your app needs to be associated with a Google Cloud Project that is enabled for the ARCore API.

You must enable the ARCore API in your Google Cloud Project. If you need to create the project, do the following:

1. Visit Create a project in Google Cloud Platform.

2. Enter an appropriate Project name, and choose a location for it.

3. Click Create.

4. In the sidebar, select APIs & Services, then Library.

5. Search for the ARCore API, select it, and click Enable.

Set up authorization

To make Geospatial API calls to the VPS, your app needs authorization. You may use signed JSON Web Token (JWT) or API key authorization.

Token (signed JWT) authorization

If you previously used an API key and no longer need it, delete it in the Google Cloud Platform Console and remove it from your app.

ARCore supports the authorization of API calls in iOS using a (JSON Web token). The token must be signed by a Google Service account.

In order to generate tokens for iOS, you must have an endpoint on your server that satisfies the following requirements:

• Your own authorization mechanism must protect the endpoint.

• The endpoint must generate a new token every time, such that:

  • Each user gets a unique token.
  • Tokens don’t immediately expire.

Create a service account and signing key

Follow these steps to create a Google service account and signing key:

1. In the navigation menu of the Google Cloud Platform Console, go to APIs & Services > Credentials.

2. Select the desired project, then click Create Credentials > Service account.

3. Under Service account details, type a name for the new account, then click Create.

4. On the Service account permissions page, go to the Select a role dropdown. Select Service Accounts > Service Account Token Creator, then click Continue.

5. On the Grant users access to this service account page, click Done.

   This takes you back to APIs & Services > Credentials.

6. On the Credentials page, scroll down to the Service Accounts section and click the name of the account you just created.

7. On the Service account details page, scroll down to the Keys section and select Add Key > Create new key.

8. Select JSON as the key type and click Create.

   This downloads a JSON file containing the private key to your machine. Store the downloaded JSON key file in a secure location.

   Note: This file contains a private key which must not be exposed to the public. Do not commit it to source code repositories like GitHub.

Create tokens on your server

To create new tokens (JWTs) on your server, use the standard JWT libraries and the JSON file that you securely downloaded from your new service account.

Create tokens on your development machine

To generate JWTs on your development machine, use the following oauth2l command:

oauth2l fetch --cache "" --jwt --json $KEYFILE --audience "https://arcorecloudanchor.googleapis.com/"

Specifying an empty cache location using the --cache flag is necessary to ensure that a different token is produced each time. Be sure to trim the resulting string. Extra spaces or newline characters will cause the API to reject the token.

Sign the token

You must use the RS256 algorithm and the following claims to sign the JWT:

• iss — The service account email address.
• sub — The service account email address.
• iat — The Unix time when the token was generated, in seconds.
• exp — iat + 3600 (1 hour). The Unix time when the token expires, in seconds.
• aud — The audience. The audience must be set to https://arcorecloudanchor.googleapis.com/.

Non-standard claims are not required in the JWT payload, though you may find the uid claim useful for identifying the corresponding user.

If you use a different approach to generate your JWTs, such as using a Google API in a Google-managed environment, make sure to sign your JWTs with the claims in this section. Above all, make sure that the audience is correct.

Pass the token in the ARCore session

The ARCore session is the main entry point to the ARCore Geospatial API. To use the Geospatial API, you have to create a GARSessionConfiguration and set the geospatialMode property for it, as described in Configure an ARCore session in iOS.

1. Construct a session using sessionWithError: and pass in the token:

   NSError *error = nil;
   GARSession *session = [GARSession sessionWithError:&error];
   

   When authorizing with the ARCore SDK, the token must be a nonempty ASCII string with no spaces or control characters. ARCore will use this until you pass in another token.

   When you obtain a token, pass it into the session using setAuthToken:. Otherwise, the session will use the most recent valid authorization token that you passed in. Call this method each time you refresh your token:

   - (void)setAuthToken:(NSString *)authToken;
   

2. create a GARSessionConfiguration and set the geospatialMode property for it.

3. Use setConfiguration:error: (GARSession) to set the configuration.

Note the following when you pass a token into the session:

• If you have used an API key to create the session, ARCore will ignore the token and log an error.

  If you no longer need the API key, delete it in the Google Developers Console and remove it from your app.

• ARCore ignores tokens that contain spaces or special characters.

• Tokens typically expire after one hour. If there is a possibility that your token may expire while in use, obtain a new token and pass it to the API.

API key authorization

Get the API key

1. In your Google Cloud Project, obtain an API key as described in Creating an API key.

2. Copy the API key, as you will paste it, in a later step.

3. If you are creating an API key for your app's release version, edit the API key, and add restrictions, as described in Applying API key restrictions.

   You must also specify the bundle ID for restricted API keys.

Create the ARCore session with the API key

The ARCore session is the main entry point to the ARCore Geospatial API. To use the Geospatial API, you have to create a GARSessionConfiguration with the geospatialMode property, as described in Configure an ARCore session in iOS:

• create a GARSessionConfiguration and set the geospatialMode property for it.

• Use setConfiguration:error: (GARSession) to set the configuration.

In Xcode, in your app, add your API key to the GARSession. Paste the API key you copied from your Google Cloud Project, as in the following example:

self.garSession = [GARSession sessionWithAPIKey:@"your-api-key"
                               bundleIdentifier:nil
                                          error:&error];

Check device compatibility

Not all devices that support ARCore also support the Geospatial API, as described in the quickstart.

Use isGeospatialModeSupported: to check the device, as in the following:

if (![self.garSession isGeospatialModeSupported:GARGeospatialModeEnabled]) {
  [self setErrorStatus:@"GARGeospatialModeEnabled is not supported on this device."];
  return;
}

GARSessionConfiguration *configuration = [[GARSessionConfiguration alloc] init];
configuration.geospatialMode = GARGeospatialModeEnabled;
[self.garSession setConfiguration:configuration error:&error];
if (error) {
  [self setErrorStatus:[NSString stringWithFormat:@"Failed to configure GARSession: %d",
                                                  (int)error.code]];
  return;
}

Ask user for location permissions at runtime

Your app must request the following location permissions at runtime, before configuring the session:

• kCLAuthorizationStatusAuthorizedWhenInUse
• CLAccuracyAuthorizationFullAccuracy (Applies when iOS >= 14)

Get a precise location

The GARGeospatialTransform describes a specific location, altitude, and compass heading relative to the Earth. It is managed in an GAREarth object.

Geospatial values are only valid while the session is tracking and receiving frames that allow access to the camera image and device transform.

GARGeospatialTransform *geospatialTransform = garFrame.earth.cameraGeospatialTransform;

Adjust for transform accuracy

As noted in the quickstart, the accuracy of the transform from the VPS may vary, due to the availability of VPS data for the location, or due to temporal conditions at the location. Your app may have to make adjustments for the accuracy of the transform, as determined by the Geospatial API.

The GARGeospatialTransform provides estimates for the accuracy of the latitude/longitude, altitude, and heading values.

For example, if the heading value returned from GARGeospatialTransform.heading is 60 degrees, and the value from GARGeospatialTransform.headingAccuracy is 10, there is a 68% probability that the VPS heading is within 10 degrees of the observed heading, as illustrated in the diagram on the left.

If the value from GARGeospatialTransform.headingAccuracy is 15, there is a 68% chance that the true heading is within 15 degrees of 60 degrees, as shown in the diagram on the right. Note that the higher the value returned from GARGeospatialTransform.headingAccuracy, the lower the accuracy of the heading value from GARGeospatialTransform.heading.

Similarly, GARGeospatialTransform.horizontalAccuracy reports the number of meters within which the true latitude/longitude value has a 68% probability of being within the given distance, and GARGeospatialTransform.verticalAccuracy reports the number of meters within which the true altitude value has a 68% probability of being within the given distance.

Place a Geospatial anchor

When placing an anchor at the specified location and orientation relative to the Earth, latitude and longitude are defined by the WGS84 specification, and the altitude value is defined by the elevation above the WGS84 ellipsoid in meters. The rotation quaternion provided is with respect to an east-up-south (EUS) coordinate frame. An identity rotation has the anchor oriented such that X+ points to the east, Y+ points up, away from the center of the earth, and Z+ points to the south.

To create an EUS rotation quaternion that has the +Z axis pointing in the same direction as the heading obtained from GARGeospatialTransform, use the following formula. For the EUS rotation quaternion, qx is the X component, qy is the Y component, qz is the Z component, and qw is the W component:

{qx, qy, qz, qw} = {0, sin((pi - heading * pi / 180.0) / 2), 0, cos((pi - heading * pi / 180.0) / 2)}}

Use createAnchorWithCoordinate:altitude:eastUpSouthQAnchor:error: to anchor content to geographical coordinates that you specify.

  NSError *error = nil;
  [self.garSession createAnchorWithCoordinate:coordinate
                                     altitude:altitude
                           eastUpSouthQAnchor:eastUpSouthQAnchor
                                        error:&error];

Note: Creating anchors within 0.1 degrees of the North Pole or South Pole (90 degrees or -90 degrees) is not supported. If such an anchor is specified, createAnchorWithCoordinate:altitude:eastUpSouthQAnchor:error: will output GARSessionErrorCodeInvalidArgument, and the anchor will fail to be created.

Calculate the altitude

Getting the altitude for placing an anchor is a bit tricky. You have to compare the altitude at the device's location against the altitude from the Geospatial API. There are two ways to determine the altitude:

• If the anchor's location is near the user, consider using an altitude that's similar to the device's altitude.
• Otherwise, consider getting the altitude value from the elevation in Google Maps.

Use the altitude from the camera's Geospatial transform

It may suffice to simply use the altitude you get from the camera's GARGeospatialTransform without cross-checking against data from another source such as the Maps API. See Get a precise location, above.

If you can obtain the transform at the location in advance, by making your own local observation, you can use that data to cross-check the GARGeospatialTransform obtained by your app for the user.

Get the elevation from Google Maps

A GARGeospatialTransform reports altitude in meters above the WGS84 ellipsoid. Your app needs to compare the value from (CLLocationDistance) altitude against the altitude at the device's location to determine an accurate altitude for the anchor.

The Google Maps API can get the elevation, but the Maps API elevation is based on the EGM96 specification. Note that the Maps API reports latitude and longitude according to the WGS84 specification, but the elevation is based on EGM96. You must convert the Maps API EGM96 elevation to WGS84 for comparison against the GARGeospatialTransform altitude. See the GeoidEval conversion tool that has both a command line and an HTML interface.

API usage quota

The ARCore SDK limits API requests to the ARCore service to the following limits for each project that uses the ARCore SDK:

• 1,000 sessions started per minute, or
• 100,000 requests per minute

API requests in excess of these limits may result

in a GAREarthStateErrorResourceExhausted error and an unfulfilled request.