Learn how to use Augmented Images in your own apps.
Prerequisites
Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.
Create an image database
Each image database can store information for up to 1,000 images.
There two ways to create an
AugmentedImageDatabase
:
- Load a saved image database. Then optionally add more reference images.
- Create a new empty database. Then add reference images one at a time.
Load a saved image database
Use AugmentedImageDatabase.deserialize()
to load an existing image database:
Java
InputStream inputStream = context.getAssets().open("example.imgdb");
AugmentedImageDatabase imageDatabase = AugmentedImageDatabase.deserialize(session, inputStream);
Kotlin
val imageDatabase = context.assets.open("example.imgdb").use {
AugmentedImageDatabase.deserialize(session, it)
}
Image databases can be created using the arcoreimg
command line tool during development, or by calling
AugmentedImageDatabase.serialize()
on a database that contains that is loaded in memory.
Create a new empty database
To create an empty image database at runtime, use the AugmentedImageDatabase
constructor:
Java
AugmentedImageDatabase imageDatabase = new AugmentedImageDatabase(session);
Kotlin
val imageDatabase = AugmentedImageDatabase(session);
Add images to an existing database
Add images to your image database by calling
AugmentedImageDatabase.addImage()
for each image, specifying an optional widthInMeters
.
Java
Bitmap bitmap;
try (InputStream inputStream = getAssets().open("dog.jpg")) {
bitmap = BitmapFactory.decodeStream(inputStream);
} catch (IOException e) {
Log.e(TAG, "I/O exception loading augmented image bitmap.", e);
}
int index = imageDatabase.addImage("dog", bitmap, imageWidthInMeters);
Kotlin
val bitmap = assets.open("dog.jpg").use { BitmapFactory.decodeStream(it) }
val index = imageDatabase.addImage("dog", bitmap, imageWidthInMeters)
The returned indexes can later be used to identify which reference image was detected.
Enable image tracking
Configure your ARCore session to begin tracking images by setting the session config to one that is configured with the desired image database:
Java
Config config = new Config(session);
config.setAugmentedImageDatabase(imageDatabase);
session.configure(config);
Kotlin
val config = Config(session)
config.augmentedImageDatabase = imageDatabase
session.configure(config)
During the session, ARCore looks for images by matching feature points from the camera image against those in the image database.
To get the matched images, poll for updated AugmentedImage
s in your frame update loop.
Java
// Update loop, in onDrawFrame().
Frame frame = session.update();
Collection<AugmentedImage> updatedAugmentedImages =
frame.getUpdatedTrackables(AugmentedImage.class);
for (AugmentedImage img : updatedAugmentedImages) {
if (img.getTrackingState() == TrackingState.TRACKING) {
// Use getTrackingMethod() to determine whether the image is currently
// being tracked by the camera.
switch (img.getTrackingMethod()) {
case LAST_KNOWN_POSE:
// The planar target is currently being tracked based on its last
// known pose.
break;
case FULL_TRACKING:
// The planar target is being tracked using the current camera image.
break;
}
// You can also check which image this is based on img.getName().
if (img.getIndex() == dogIndex) {
// TODO: Render a 3D version of a dog in front of img.getCenterPose().
} else if (img.getIndex() == catIndex) {
// TODO: Render a 3D version of a cat in front of img.getCenterPose().
}
}
}
Kotlin
// Update loop, in onDrawFrame().
val frame = session.update()
val updatedAugmentedImages = frame.getUpdatedTrackables(AugmentedImage::class.java)
for (img in updatedAugmentedImages) {
if (img.trackingState == TrackingState.TRACKING) {
// Use getTrackingMethod() to determine whether the image is currently
// being tracked by the camera.
when (img.trackingMethod) {
AugmentedImage.TrackingMethod.LAST_KNOWN_POSE -> {
// The planar target is currently being tracked based on its last
// known pose.
}
AugmentedImage.TrackingMethod.FULL_TRACKING -> {
// The planar target is being tracked using the current camera image.
}
}
// You can also check which image this is based on AugmentedImage.getName().
when (img.index) {
dogIndex -> // TODO: Render a 3D version of a dog at img.getCenterPose().
catIndex -> // TODO: Render a 3D version of a cat at img.getCenterPose().
}
}
}
Supporting different use cases
When ARCore detects an Augmented Image, it creates a Trackable
for that
Augmented Image and sets TrackingState
to TRACKING
and TrackingMethod
to FULL_TRACKING
. When the tracked image moves out of camera view, ARCore
changes the TrackingMethod
to LAST_KNOWN_POSE
while continuing to provide the orientation and position of
the image.
Your app should use these enumerations differently depending on the intended use case.
Fixed images. Most use cases involving images that are fixed in place (that is, not expected to move) can simply use
TrackingState
to determine whether the image has been detected and whether its location is known.TrackingMethod
can be ignored.Moving images. If your app needs to track a moving image, use both
TrackingState
andTrackingMethod
to determine whether the image has been detected and whether its position is known.
Use case | Fixed image | Moving image |
---|---|---|
Example | A poster hung on a wall | An advertisement on the side of a bus |
The pose can be considered valid when |
TrackingState == TRACKING |
TrackingState == TRACKING
and TrackingMethod == FULL_TRACKING
|