GoogleARCore.Frame.CameraImage

Caution: The ARCore SDK for Unity is deprecated, and no longer supported in Unity 2020 and later. This SDK should only be used by developers working on existing projects which are unable to migrate to Unitys AR Foundation.

Key Point: Developers starting new projects should instead use the ARCore Extensions for AR Foundation with Unity.

Container for state related to the ARCore camera for the frame.

Summary

Properties
ImageDisplayUvs	static DisplayUvCoords Gets UVs that map the orientation and aspect ratio of the image returned by Frame.CameraImage.AcquireCameraImageBytes to that of the device's display.
ImageIntrinsics	static CameraIntrinsics Gets the unrotated and uncropped intrinsics for the image (CPU) stream.
Texture	static Texture Gets a texture used from the device's rear camera.
TextureDisplayUvs	static DisplayUvCoords Gets UVs that map the orientation and aspect ratio of Frame.CameraImage.Texture to those of the device's display.
TextureIntrinsics	static CameraIntrinsics Gets the unrotated and uncropped intrinsics for the texture (GPU) stream.

Public static functions
AcquireCameraImageBytes()	GoogleARCore.CameraImageBytes Attempts to acquire the camera image for CPU access that corresponds to the current frame.
GetCameraProjectionMatrix(float nearClipping, float farClipping)	Matrix4x4 Gets the projection matrix for the frame.
TransformCoordinate(Vector2 coordinate, DisplayUvCoordinateType sourceType, DisplayUvCoordinateType targetType)	Vector2 Transforms a coordinate between the source and target display UV coordinate types.
UpdateDepthTexture(ref Texture2D depthTexture)	DepthStatus Updates the input texture with the latest depth data from ARCore.
UpdateRawDepthConfidenceTexture(ref Texture2D confidenceTexture)	DepthStatus Attempts to acquire the confidence image corresponding to the raw depth image of the current frame.
UpdateRawDepthTexture(ref Texture2D depthTexture)	DepthStatus Attempts to acquire a "raw", mostly unfiltered, depth image that corresponds to the current frame.

Properties

ImageDisplayUvs

static DisplayUvCoords ImageDisplayUvs

Gets UVs that map the orientation and aspect ratio of the image returned by Frame.CameraImage.AcquireCameraImageBytes to that of the device's display.

ImageIntrinsics

static CameraIntrinsics ImageIntrinsics

Gets the unrotated and uncropped intrinsics for the image (CPU) stream.

Texture

static Texture Texture

Gets a texture used from the device's rear camera.

TextureDisplayUvs

static DisplayUvCoords TextureDisplayUvs

Gets UVs that map the orientation and aspect ratio of Frame.CameraImage.Texture to those of the device's display.

TextureIntrinsics

static CameraIntrinsics TextureIntrinsics

Gets the unrotated and uncropped intrinsics for the texture (GPU) stream.

Public static functions

AcquireCameraImageBytes

GoogleARCore.CameraImageBytes AcquireCameraImageBytes()

Attempts to acquire the camera image for CPU access that corresponds to the current frame.

Depending on device performance, this can fail for several frames after session start, and for a few frames at a time while the session is running.

Details
Returns	A CameraImageBytes struct with IsAvailable property set to true if successful and false if the image could not be acquired.

GetCameraProjectionMatrix

Matrix4x4 GetCameraProjectionMatrix(
  float nearClipping,
  float farClipping
)

Gets the projection matrix for the frame.

Details
Parameters	nearClipping The near clipping plane for the projection matrix. farClipping The far clipping plane for the projection matrix.
Returns	The projection matrix for the frame.

TransformCoordinate

Vector2 TransformCoordinate(
  Vector2 coordinate,
  DisplayUvCoordinateType sourceType,
  DisplayUvCoordinateType targetType
)

Transforms a coordinate between the source and target display UV coordinate types.

This can be used for the conversion of coordinates accessed in the same Unity update.

Details
Parameters	coordinate The coordinate to transform. sourceType The source type of the desired transformation matrix. targetType The target type of the desired transformation matrix.
Returns	A corresponding position in the target frame.

UpdateDepthTexture

DepthStatus UpdateDepthTexture(
  ref Texture2D depthTexture
)

Updates the input texture with the latest depth data from ARCore.

If there is no new data, or an error occurs, the contents of the texture will remain unchanged. See DepthStatus for a complete list of reasons.

The actual resolution of the depth image depends on the device and its display aspect ratio, with sizes typically around 160x120 pixels, with higher resolutions up to 640x480 on some devices. These sizes may change in the future.

Details
Parameters	depthTexture The texture to hold the depth data.
Returns	DepthStatus.Success if successful.

UpdateRawDepthConfidenceTexture

DepthStatus UpdateRawDepthConfidenceTexture(
  ref Texture2D confidenceTexture
)

Attempts to acquire the confidence image corresponding to the raw depth image of the current frame.

Each pixel is an 8-bit unsigned integer representing the estimated confidence of the corresponding pixel in the raw depth image. The confidence value is between 0 and 255, inclusive, with 0 representing the lowest confidence and 255 representing the highest confidence in the measured depth value. Pixels without a valid depth estimate have a confidence value of 0 and a corresponding depth value of 0 (see UpdateRawDepthTexture).

The actual resolution of the depth image depends on the device and its display aspect ratio, with sizes typically around 160x120 pixels, with higher resolutions up to 640x480 on some devices. These sizes may change in the future. The resolution of the textures set by UpdateDepthTexture, UpdateRawDepthTexture and UpdateRawDepthConfidenceTexture will be the same on a device.

Details
Parameters	confidenceTexture The texture to hold the depth data.
Returns	DepthStatus.Success if successful.

UpdateRawDepthTexture

DepthStatus UpdateRawDepthTexture(
  ref Texture2D depthTexture
)

Attempts to acquire a "raw", mostly unfiltered, depth image that corresponds to the current frame.

The raw depth image is sparse and does not provide valid depth for all pixels. Pixels without a valid depth estimate have a pixel value of 0 and a corresponding confidence value of 0 (see UpdateRawDepthConfidenceTexture).

The depth image has a single 16-bit plane at index 0, stored in little-endian format. Each pixel contains the distance in millimeters to the camera plane. Currently, the three most significant bits are always set to 000. The remaining thirteen bits express values ranging from 0 millimeters to 8191 millimeters.

The actual resolution of the depth image depends on the device and its display aspect ratio, with sizes typically around 160x120 pixels, with higher resolutions up to 640x480 on some devices. These sizes may change in the future. The resolution of the textures set by UpdateDepthTexture, UpdateRawDepthTexture and UpdateRawDepthConfidenceTexture will be the same on a device.

The output depth image can express depth values from 0 millimeters to 8191 millimeters. Optimal depth accuracy is achieved between 50 millimeters and 5000 millimeters from the camera. Error increases quadratically as distance from the camera increases. Depth captured on each frame is used to improve the accuracy for subsequent frames, reducing the error introduced by camera distance.

If an up-to-date depth image isn't ready for the current frame, a 3D reprojection of the most recent depth image to the current frame will be given instead. An up-to-date depth image should typically become available again within a few frames.

The timestamp of this image is equal to the timestamp of the latest camera image that contributed to the depth estimation. If the timestamp of the raw depth image is different from the timestamp of the frame, it means that the camera image of this frame did not take part in the depth estimation and the depth has been reprojected from a previous frame.

Details
Parameters	depthTexture The texture to hold the depth data.
Returns	DepthStatus.Success if successful.