**Platform-specific guides**

### Android (Kotlin/Java)

- [Understand the user's environment on Android SDK (Kotlin/Java)](https://developers.google.com/ar/develop/java/scene-semantics)

### Android NDK (C)

- [Understand the user's environment on Android SDK (C)](https://developers.google.com/ar/develop/c/scene-semantics)

### iOS

- [Understand the user's environment on iOS](https://developers.google.com/ar/develop/ios/scene-semantics)

### Unity (AR Foundation)

- [Understand the user's environment on Unity (AR Foundation)](https://developers.google.com/ar/develop/unity-arf/scene-semantics)

![Scene semantics](https://developers.google.com/static/ar/develop/scene-semantics/images/hero.jpg)

The Scene Semantics API enables developers to understand the scene surrounding the user, which is needed for many high-quality AR experiences. Built on an ML model, the Scene Semantics API provides real-time semantic information, which complements existing geometric information in ARCore.

Given an image of an outdoor scene, the API returns a label for each pixel across a set of useful semantic classes, such a sky, building, tree, road, sidewalk, vehicle, person, and more. In addition to pixel labels, the Scene Semantics API also offers confidence values for each pixel label and an easy-to-use way to query the prevalence of a given label in an outdoor scene.

Your browser does not support the video tag.

From left to right, examples of an input image, the semantic image of pixel labels, and the corresponding confidence image:

![Example of semantic images](https://developers.google.com/static/codelabs/arcore-scene-semantics-geospatial-depth/images/semantics-images.png)

With the Scene Semantics API, developers can identify specific scene components, such as roads and sidewalks to help guide a user through an unfamiliar city, people and vehicles to render occlusions on dynamic objects, sky to create a sunset at any time of the day, and buildings to modify their appearance and anchor virtual objects.

## Semantic labels and quality

The Scene Semantics API provides multiple labels, each with a corresponding quality or reliability. Generally, the ML model is better able to predict classes of larger, more common objects/surfaces than classes of smaller or more rare objects/surfaces. The classes can be grouped into the following quality tiers, ranked from higher to lower:

| Semantic label quality tiers ||
|---|---|
| Main scene components | - sky - building - tree - road - vehicle |
| Major scene details | - sidewalk - terrain - structure - water |
| Minor scene details | - object - person |

## Device compatibility

The Scene Semantics API shares the same list of supported devices as the Depth API. Please refer to the [ARCore supported devices](https://developers.google.com/ar/devices) page for an up-to-date-list of devices that support both APIs.

## Supported use cases

The Scene Semantics API is designed for use in the following scenarios:

1. **Outdoor scenes**: Supports outdoor scenes only and is not intended for indoor use cases.

2. **Portrait orientation**: Should only be used in the device's default orientation mode (i.e. portrait). The quality of semantic labels is not guaranteed for landscape mode.