Augmented Images developer guide for Android

Learn how to use Augmented Images in your own apps.

Create an image database

Each image database can store information for up to 1,000 images.

There two ways to create an AugmentedImageDatabase:

• Load a saved image database. Then optionally add more reference images.
• Create a new empty database. Then add reference images one at a time.

Load a saved image database

Use AugmentedImageDatabase.deserialize() to load an existing image database:

InputStream inputStream = context.getAssets().open("example.imgdb");
AugmentedImageDatabase imageDatabase = AugmentedImageDatabase.deserialize(inputStream);

Image databases can be created using the arcoreimg command line tool during development, or by calling AugmentedImageDatabase.serialize() on a database that contains that is loaded in memory.

Create a new empty database

To create an empty image database at runtime, use the no-arg AugmentedImageDatabase() constructor:

AugmentedImageDatabase imageDatabase = new AugmentedImageDatabase();

Add images to an existing database

Add images to your image database by calling AugmentedImageDatabase.addImage() for each image:

Bitmap bitmap;
try (InputStream inputStream = getAssets().open("dog.jpg")) {
  bitmap = BitmapFactory.decodeStream(inputStream);
} catch (IOException e) {
  Log.e(TAG, "I/O exception loading augmented image bitmap.", e);
}
int index = imageDatabase.addImage("dog", bitmap, imageWidthInMeters);

The returned indexes can later be used to identify which reference image was detected.

Enable image tracking

Configure your ARCore session to begin tracking images by setting the session config to one that is configured with the desired image database:

config.setAugmentedImageDatabase(imageDatabase);
session.configure(config);

During the session, ARCore looks for images by matching feature points from the camera image against those in the image database.

To get the matched images, poll for updated AugmentedImages in your frame update loop.

// Update loop, in onDrawFrame().
Frame frame = mSession.update();
Collection<AugmentedImage> updatedAugmentedImages =
  frame.getUpdatedTrackables(AugmentedImage.class);

for (AugmentedImage img : updatedAugmentedImages) {
  // Developers can:
  // 1. Check tracking state.
  // 2. Render something based on the pose, or attach an anchor.
  if (img.getTrackingState() == TrackingState.TRACKING) {
     // Use getTrackingMethod() to determine whether the image is currently
     // being tracked by the camera.
     if (getTrackingMethod() == TrackingMethod.LAST_KNOWN_POSE) {
       // The planar target is currently being tracked based on its last
       // known pose.
     } else    // (getTrackingMethod() == TrackingMethod.FULL_TRACKING)
     {
       // The planar target is being tracked using the current camera image.
     }
     // You can also check which image this is based on getName().
     if (img.getIndex() == dogIndex) {
       // TODO: Render a 3D version of a dog in front of img.getCenterPose().
     } else if (img.getIndex() == catIndex) {
       // TODO: Render a 3D version of a cat in front of img.getCenterPose().
     }
  }
}

Supporting different use cases

When ARCore detects an Augmented Image, it creates a Trackable for that Augmented Image and sets TrackingState to TRACKING and TrackingMethod to FULL_TRACKING. When the tracked image moves out of camera view, ARCore changes the TrackingMethod to LAST_KNOWN_POSE while continuing to provide the orientation and position of the image.

Your app should use these enumerators differently depending on the intended use case.

• Fixed images. Most use cases involving images that are fixed in place (that is, not expected to move) can simply use TrackingState to determine whether the image has been detected and whether its location is known. TrackingMethod can be ignored.

• Moving images. If your app needs to track a moving image, use both TrackingState and TrackingMethod to determine whether the image has been detected and whether its position is known.

Use case	Fixed image	Moving image
Example	A poster hung on a wall	An advertisement on the side of a bus
The pose can be considered valid when	TrackingState == TRACKING	TrackingState == TRACKING and TrackingMethod == FULL_TRACKING

Note: If an image in the camera view changes very rapidly to a second image that has roughly the same size and position, ARCore may erroneously set the TrackingMethod to FULL_TRACKING for both images and also update the anchor of the first Augmented Image to the position of the new image. For example, this could happen if you were pointing the camera at an image in a web browser, and then switched to another browser tab containing a second similarly sized and positioned image.