Learn how to use Augmented Images in your own apps. ## Prerequisites Make sure that you understand [fundamental AR concepts](https://developers.google.com/ar/develop/fundamentals) and how to [configure an ARCore session](https://developers.google.com/ar/develop/java/session-config) before proceeding. ## Create an image database Each image database can store information for up to 1,000 images. There two ways to create an [`AugmentedImageDatabase`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImageDatabase): - **Load a saved image database**. Then optionally add more reference images. - **Create a new empty database**. Then add reference images one at a time. ### Load a saved image database Use [`AugmentedImageDatabase.deserialize()`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImageDatabase#deserialize(com.google.ar.core.Session,%20java.io.InputStream)) to load an existing image database: ### Java ```java AugmentedImageDatabase imageDatabase; try (InputStream inputStream = this.getAssets().open("example.imgdb")) { imageDatabase = AugmentedImageDatabase.deserialize(session, inputStream); } catch (IOException e) { // The Augmented Image database could not be deserialized; handle this error appropriately. } ``` ### Kotlin ```kotlin val imageDatabase = this.assets.open("example.imgdb").use { AugmentedImageDatabase.deserialize(session, it) } ``` Image databases can be created using the [`arcoreimg`](https://developers.google.com/ar/develop/java/augmented-images/arcoreimg) command line tool during development, or by calling [`AugmentedImageDatabase.serialize()`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImageDatabase#serialize(java.io.OutputStream)) on a database that contains that is loaded in memory. ### Create a new empty database To create an empty image database at runtime, use the [`AugmentedImageDatabase` constructor](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImageDatabase#AugmentedImageDatabase(com.google.ar.core.Session)): ### Java ```java AugmentedImageDatabase imageDatabase = new AugmentedImageDatabase(session); ``` ### Kotlin ```kotlin val imageDatabase = AugmentedImageDatabase(session) ``` ### Add images to an existing database Add images to your image database by calling [`AugmentedImageDatabase.addImage()`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImageDatabase#addImage(java.lang.String,%20android.graphics.Bitmap,%20float)) for each image, specifying an optional `widthInMeters`. ### Java ```java Bitmap bitmap; try (InputStream bitmapString = getAssets().open("dog.jpg")) { bitmap = BitmapFactory.decodeStream(bitmapString); } catch (IOException e) { // The bitmap could not be found in assets; handle this error appropriately. throw new AssertionError("The bitmap could not be found in assets.", e); } // If the physical size of the image is not known, use addImage(String, Bitmap) instead, at the // expense of an increased image detection time. float imageWidthInMeters = 0.10f; // 10 cm int dogIndex = imageDatabase.addImage("dog", bitmap, imageWidthInMeters); ``` ### Kotlin ```kotlin val bitmap = assets.open("dog.jpg").use { BitmapFactory.decodeStream(it) } // If the physical size of the image is not known, use addImage(String, Bitmap) instead, at the // expense of an increased image detection time. val imageWidthInMeters = 0.10f // 10 cm val dogIndex = imageDatabase.addImage("dog", bitmap, imageWidthInMeters) ``` The returned indexes can later be used to identify which reference image was detected. ## Enable image tracking Configure your ARCore session to begin tracking images by setting the session config to one that is configured with the desired image database: ### Java ```java Config config = new Config(session); config.setAugmentedImageDatabase(imageDatabase); session.configure(config); ``` ### Kotlin ```kotlin val config = Config(session) config.augmentedImageDatabase = imageDatabase session.configure(config) ``` During the session, ARCore looks for images by matching feature points from the camera image against those in the image database. To get the matched images, poll for updated [`AugmentedImage`s](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImage) in your frame update loop. ### Java ```java Collection updatedAugmentedImages = frame.getUpdatedTrackables(AugmentedImage.class); for (AugmentedImage img : updatedAugmentedImages) { if (img.getTrackingState() == TrackingState.TRACKING) { // Use getTrackingMethod() to determine whether the image is currently // being tracked by the camera. switch (img.getTrackingMethod()) { case LAST_KNOWN_POSE: // The planar target is currently being tracked based on its last // known pose. break; case FULL_TRACKING: // The planar target is being tracked using the current camera image. break; case NOT_TRACKING: // The planar target isn't been tracked. break; } // You can also check which image this is based on img.getName(). if (img.getIndex() == dogIndex) { // TODO: Render a 3D version of a dog in front of img.getCenterPose(). } else if (img.getIndex() == catIndex) { // TODO: Render a 3D version of a cat in front of img.getCenterPose(). } } } ``` ### Kotlin ```kotlin val updatedAugmentedImages = frame.getUpdatedTrackables(AugmentedImage::class.java) for (img in updatedAugmentedImages) { if (img.trackingState == TrackingState.TRACKING) { // Use getTrackingMethod() to determine whether the image is currently // being tracked by the camera. when (img.trackingMethod) { AugmentedImage.TrackingMethod.LAST_KNOWN_POSE -> { // The planar target is currently being tracked based on its last known pose. } AugmentedImage.TrackingMethod.FULL_TRACKING -> { // The planar target is being tracked using the current camera image. } AugmentedImage.TrackingMethod.NOT_TRACKING -> { // The planar target isn't been tracked. } } // You can also check which image this is based on AugmentedImage.getName(). when (img.index) { dogIndex -> TODO("Render a 3D version of a dog at img.getCenterPose()") catIndex -> TODO("Render a 3D version of a cat at img.getCenterPose()") } } } ``` ### Supporting different use cases When ARCore detects an Augmented Image, it creates a [`Trackable`](https://developers.google.com/ar/reference/java/com/google/ar/core/Trackable) for that Augmented Image and sets [`TrackingState`](https://developers.google.com/ar/reference/java/com/google/ar/core/TrackingState) to `TRACKING` and [`TrackingMethod`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImage.TrackingMethod) to `FULL_TRACKING`. When the tracked image moves out of camera view, ARCore changes the [`TrackingMethod`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImage.TrackingMethod) to `LAST_KNOWN_POSE` while continuing to provide the orientation and position of the image. Your app should use these enumerations differently depending on the intended use case. - **Fixed images** . Most use cases involving images that are fixed in place (that is, not expected to move) can simply use `TrackingState` to determine whether the image has been detected and whether its location is known. `TrackingMethod` can be ignored. - **Moving images** . If your app needs to track a moving image, use both `TrackingState` and `TrackingMethod` to determine whether the image has been detected and whether its position is known. | Use case | Fixed image | Moving image | | Example | A poster hung on a wall | An advertisement on the side of a bus | | The pose can be considered valid when | `TrackingState == TRACKING` | `TrackingState == TRACKING` and `TrackingMethod == FULL_TRACKING` | |---|---|---| > [!NOTE] > **Note:** If an image in the camera view changes very rapidly to a second image that has roughly the same size and position, ARCore may erroneously set the [`TrackingMethod`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedImage.TrackingMethod) to `FULL_TRACKING` for both images and also update the anchor of the first Augmented Image to the position of the new image. For example, this could happen if you were pointing the camera at an image in a web browser, and then switched to another browser tab containing a second similarly sized and positioned image. ## See also - The Augmented Images [sample projects](https://github.com/google-ar/arcore-android-sdk/tree/master/samples) in the ARCore SDK. - The [ARCore Augmented Images codelab](https://codelabs.developers.google.com/codelabs/augimg-intro).