Geospatial developer guide for Android NDK (C)

Learn how to use the Geospatial API in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

If you want to run a sample app that demonstrates the functionality described here, see the ARCore Geospatial Quickstart for Android for Java. The sample app in the Quickstart is written in Java. The code samples in this document are for the same functionality written in C.

See the Introduction to the Geospatial API for more information about the Geospatial API.

If you're new to developing with ARCore, see Getting started for information about software and hardware requirements, prerequisities and other information specific to the platforms you are using.

Be sure your development environment satisfies the ARCore SDK requirements, as described in the Quickstart for Java.

Set up a Google Cloud Project

To use the Visual Positioning System (VPS), your app needs to be associated with a Google Cloud Project that is enabled for the ARCore API.

You must enable the ARCore API in your Google Cloud Project. If you need to create the project, do the following:

1. Visit Create a project in Google Cloud Platform.

2. Enter an appropriate Project name, and choose a location for it.

3. Click Create.

4. In the sidebar, select APIs & Services, then Library.

5. Search for the ARCore API, select it, and click Enable.

Set up authorization

To make Geospatial API calls to the VPS, your app needs authorization. Keyless authorization is preferred, but API Key authorization is also supported.

Keyless authorization

Get the signing key SHA-1 fingerprint, then plug it into your Google Cloud Project, and create an OAuth client ID.

Note that you need to register separately for all the different combinations of package name and signing key that you use: debug, release, etc.

If you are using the signing key for a pre-release or debug version of your app, or if your app uploads an APK instead of an AAB, get the signing key SHA-1 fingerprint from Android Studio, as follows:

1. In your Android Studio project, open the Gradle toolpane.

2. Navigate to <project-name> > work > Tasks > android.

3. Run the signingReport task.

4. Copy the SHA-1 fingerprint; you will paste it in a later step.

   For sample apps, use the debug fingerprint.

If you are using Play App Signing, where Google manages and protects your app's signing key for you, and uses it to sign optimized distribution APKs that are generated from your app bundle (AAB), get the SHA-1 fingerprint as described in Step 3: Register your app signing key with API providers.

API key authorization

If you use API key authorization, make sure that the key is either unrestricted, or that it allows the ARCore API (if restricted by API), or that it allows your app (if restricted by app). The API Key can list a collection of fingerprints, and must include the fingerprint for the release version, if it is a restricted API key.

1. In your Google Cloud Project, under APIs & Services, select Credentials.

2. In the top bar, click Create Credentials, and select API Key.

   Note that if you are using a restricted API key, you must edit the key to add a debug certificate fingerprint or a release certificate fingerprint to the API key.

3. Copy the API key that was created, as you will paste it in a later step.

4. In your Android Studio project, open app > src > AndroidManifest.xml.

5. In the AndroidManifest.xml file, in an <application> element, add a <meta-data> element with the API key as in the following example. You can paste the API key here, then copy the entire block.

    <meta-data
       android:name="com.google.android.ar.API_KEY"
       android:value="API_KEY"/>
   

   The value stored in com.google.android.ar.API_KEY authorizes this app.

Include required libraries

Include the Fused Location Provider for Android. ARCore uses this library to obtain the device's current location.

In your app's build.gradle, include the Play Services Location library.

dependencies {
  // Apps must declare play-services-location version >= 16.
  // In the following line, substitute `16 (or later)` with the latest version.
  implementation 'com.google.android.gms:play-services-location:16 (or later)'
}

Include required ProGuard rules

When minifying your app with Proguard, you should include the following ProGuard rules in the proguard-rules.pro configuration for your app.

Use the following rules when using keyless authorization:

# Keep classes required by the Geospatial API.
-keep class com.google.android.gms.auth.** { *; }
-keep class com.google.android.gms.location.** { *; }
-keep class com.google.android.gms.common.** { *; }
-keep class com.google.android.gms.tasks.** { *; }

Use the following rules when using API key authorization:

# Keep classes required by the Geospatial API.
-keep class com.google.android.gms.location.** { *; }
-keep class com.google.android.gms.common.** { *; }
-keep class com.google.android.gms.tasks.** { *; }

Prompt user to allow usage of device data

Apps that use the ARCore Geospatial API must present the user with a prompt to acknowledge and allow the use of data from their device. See User privacy requirements for more information.

Check device compatibility

Not all devices that support ARCore also support the Geospatial API, as described in the quickstart for Java.

To check the user's device for compatibility, call ArSession_isGeospatialModeSupported(). If this returns false do not attempt to configure the session (below), as doing so will cause the ArStatus to report an AR_ERROR_UNSUPPORTED_CONFIGURATION.

Ask user for location permissions at runtime

Your app must request location permissions at runtime.

To use the ARCore Geospatial API, your app needs to register the following extra permissions:

• ACCESS_FINE_LOCATION to accurately determine the user's location.

• ACCESS_COARSE_LOCATION for non-accurate determination of the user's location and to comply with user privacy requirements. However, the Geospatial API cannot be configured to work with coarse location, and API requests will fail when the user has set this permission. See below for more information.

• ACCESS_INTERNET to contact the ARCore Geospatial API service.

<manifest ... >
  <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
  <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
  <uses-permission android:name="android.permission.INTERNET" />
</manifest>

On devices that run Android version 12 or higher, users can request that your app have access to only approximate location information. To accommodate this request, your app must have the ACCESS_COARSE_LOCATION permission configured, along with ACCESS_FINE_LOCATION, as shown above. You must configure both location permissions.

However, when users specify coarse location, doing so prevents the Geospatial API from obtaining the precise location it requires. The Geospatial service will not allow itself to be configured if your app gives it only coarse location. Your app cannot use the Geospatial API with coarse location.

Configure the ARCore session

Before creating the session, change the ArGeospatialMode in your session configuration to AR_GEOSPATIAL_MODE_ENABLED:

// Create a session config.
ArConfig* ar_config = NULL;
ArConfig_create(ar_session, &ar_config);

// Enable Instant Placement mode.
ArConfig_setGeospatialMode(ar_session, ar_config, AR_GEOSPATIAL_MODE_ENABLED);
CHECK(ArSession_configure(ar_session, ar_config) == AR_SUCCESS);

// Release config resources.
ArConfig_destroy(ar_config);

While AR_GEOSPATIAL_MODE_ENABLED, the application is allowed to obtain geographical information from the Visual Positioning System (VPS).

Get a precise location

The ArGeospatialPose describes a specific location, altitude, and compass heading relative to the Earth. It is managed in an ArEarth object.

Check the tracking state

Geospatial values are only valid while ArEarthState is AR_EARTH_STATE_ENABLED, and ArTrackingState is AR_TRACKING_STATE_TRACKING. Otherwise, ArTrackingState may be AR_TRACKING_STATE_PAUSED or AR_TRACKING_STATE_STOPPED. Always wrap your Geospatial API calls in an ArTrackingState control block, as shown below.

ArEarth* ar_earth = NULL;
ArSession_acquireEarth(ar_session, &ar_earth);
if (ar_earth != NULL) {
  ArTrackingState earth_tracking_state = AR_TRACKING_STATE_STOPPED;
  ArTrackable_getTrackingState(ar_session, (ArTrackable*)ar_earth,
                               &earth_tracking_state);
  if (earth_tracking_state == AR_TRACKING_STATE_TRACKING) {
    // Values obtained by the Geospatial API are valid as long as ArEarth
    // has AR_TRACKING_STATE_TRACKING.
    // TODO: use Geospatial APIs in this block.
  }
}

Note that when ArTrackable_getTrackingState does not become AR_TRACKING_STATE_TRACKING, ArEarthState may contain the cause of this failure.

Obtain camera Geospatial pose

While the ArEarth object is tracking, request the camera's pose relative to Earth:

if (ar_earth != NULL) {
  ArTrackingState earth_tracking_state = AR_TRACKING_STATE_STOPPED;
  ArTrackable_getTrackingState(ar_session, (ArTrackable*)ar_earth,
                               &earth_tracking_state);
  if (earth_tracking_state == AR_TRACKING_STATE_TRACKING) {
    ArGeospatialPose* camera_geospatial_pose = NULL;
    ArGeospatialPose_create(ar_session, &camera_geospatial_pose);
    ArEarth_getCameraGeospatialPose(ar_session, ar_earth,
                                    camera_geospatial_pose);
    // camera_geospatial_pose contains geodetic location, rotation, and
    // confidences values.
    ArGeospatialPose_destroy(camera_geospatial_pose);
  }
}

This gives you a ArGeospatialPose, which contains the following information:

• Location, expressed in latitude and longitude. An estimate of the location accuracy is also supplied.
• Altitude, and an estimate of the altitude's accuracy.
• Heading, an approximation of the direction the device is facing, and an estimate of the accuracy of the heading.

Adjust for pose accuracy

As noted in the quickstart, the accuracy of the pose from the VPS may vary, due to the availability of VPS data for the location, or due to temporal conditions at the location. Your app may have to make adjustments for the accuracy of the pose, as determined by the Geospatial API.

The Geospatial API provides an estimate for the accuracy of the latitude/longitude, altitude, and heading values returned from the VPS.

For example, if the heading value returned from ArGeospatialPose_getHeading() is 60 degrees, and the value from ArGeospatialPose_getHeadingAccuracy() is 10, there is a 68% probability that the VPS heading is within 10 degrees of the observed heading, as illustrated in the diagram on the left.

If the value from ArGeospatialPose_getHeadingAccuracy() is 15, there is a 68% chance that the true heading is within 15 degrees of 60 degrees, as shown in the diagram on the right. Note that the higher the value returned from ArGeospatialPose_getHeadingAccuracy(), the lower the accuracy of the heading value from ArGeospatialPose_getHeading().

Similarly, ArGeospatialPose_getHorizontalAccuracy() reports the number of meters within which the true latitude/longitude value has a 68% probability of being within the given distance, and ArGeospatialPose_getVerticalAccuracy() reports the number of meters within which the true altitude value has a 68% probability of being within the given distance.

Place a Geospatial anchor

When placing an anchor at the specified location and orientation relative to the Earth, latitude and longitude are defined by the WGS84 specification, and the altitude value is defined by the elevation above the WGS84 ellipsoid in meters. The rotation quaternion provided is with respect to an east-up-south (EUS) coordinate frame. An identity rotation has the anchor oriented such that X+ points to the east, Y+ points up, away from the center of the earth, and Z+ points to the south.

To create an EUS rotation quaternion that has the +Z axis pointing in the same direction as the heading obtained from ArGeospatialPose, use the following formula. For the EUS rotation quaternion, qx is the X component, qy is the Y component, qz is the Z component, and qw is the W component:

{qx, qy, qz, qw} = {0, sin((pi - heading * M_PI / 180.0) / 2), 0, cos((pi - heading * M_PI / 180.0) / 2)}}

Use ArEarth_acquireNewAnchor() to anchor content to geographical coordinates that you specify.

float eus_quaternion_4[4] = {qx, qy, qz, qw};
if (ar_earth != NULL) {
  ArTrackingState earth_tracking_state = AR_TRACKING_STATE_STOPPED;
  ArTrackable_getTrackingState(ar_session, (ArTrackable*)ar_earth,
                               &earth_tracking_state);
  if (earth_tracking_state == AR_TRACKING_STATE_TRACKING) {
    ArAnchor* earth_anchor = NULL;
    CHECK(ArEarth_acquireNewAnchor(ar_session, ar_earth,
                                   /* Locational values */
                                   latitude, longitude, altitude,
                                   eus_quaternion_4, &earth_anchor));
    // TODO: Attach content to the anchor specified by geodetic location and
    // pose.
  }
}

Note: Creating anchors within 0.1 degrees of the North Pole or South Pole (90 degrees or -90 degrees) is not supported. If such an anchor is specified, ArEarth_acquireNewAnchor() will return an AR_ERROR_INVALID_ARGUMENT, and the anchor will fail to be created.

Calculate the altitude

Getting the altitude for placing an anchor is a bit tricky. You have to compare the altitude at the device's location against the altitude from the Geospatial API. There are two ways to determine the altitude:

• If the anchor's location is near the user, consider using an altitude that's similar to the device's altitude.
• Otherwise, consider getting the altitude value from the elevation in Google Maps.

Use the altitude from the camera's Geospatial pose

It may suffice to simply use the altitude you get from the Camera's ArGeospatialPose without cross-checking against data from another source such as the Maps API. See Obtain camera Geospatial pose, above.

If you can obtain the pose at the location in advance, by making your own local observation, you can use that data to cross-check the ArGeospatialPose obtained by your app for the user.

Get the elevation from Google Maps

The ArGeospatialPose reports altitude in meters above the WGS84 ellipsoid. Your app needs to compare the value from ArGeospatialPose_getAltitude() against the altitude at the device's location to determine an accurate altitude for the anchor.

You can get the latitude and longitude using Google Maps. Make sure to turn off Globe View first by navigating to Layers > More and deselecting the Globe View checkbox at the bottom left-hand corner of the screen. This will force a 2D perspective and eliminate possible errors that could come from an angled 3D view.

Once you have a a latitude and longitude, use the Elevation API to get an elevation based on the EGM96 specification. You must convert the Maps API EGM96 elevation to WGS84 for comparison against the GeospatialPose altitude. See the GeoidEval conversion tool that has both a command line and an HTML interface. The Maps API reports latitude and longitude according to the WGS84 specification out of the box.

API usage quota

The ARCore SDK limits API requests to the ARCore service to the following limits for each project that uses the ARCore SDK:

• 1,000 sessions started per minute, or
• 100,000 requests per minute

API requests in excess of these limits may result

in an AR_EARTH_STATE_ERROR_RESOURCE_EXHAUSTED error and an unfulfilled request.