AugmentedFace

public class AugmentedFace

Describes a face detected by ARCore and provides methods to access additional center and face region poses as well as face mesh related data.

Augmented Faces supports front-facing (selfie) camera only, and does not support attaching anchors nor raycast hit testing. Calling Trackable.createAnchor(Pose) will result in an IllegalStateException.

To use Augmented Faces, enable the feature in the session. This can be done at session creation time, or at any time during session runtime:

Session session = new Session(context, EnumSet.of(Session.Feature.FRONT_CAMERA));
 Config config = ...
 config.setAugmentedFaceMode(AugmentedFaceMode.MESH3D);
 session.configure(config);
 

When Augmented Face mode is enabled, ARCore updates the list of detected faces for each frame. Use Session.getAllTrackables(Class) and Trackable.getTrackingState() to get a list of faces that have valid meshes that can be rendered.

for (AugmentedFace face : session.getAllTrackables(AugmentedFace.class)) {
   if (face.getTrackingState() == TrackingState.TRACKING) {
     // Render face mesh ...
   }
 }
 

Faces provide static mesh data that does not change during the session, as well as pose and mesh data that is updated each frame:

// UVs and indices can be cached as they do not change during the session.
 FloatBuffer uvs = face.getMeshTextureCoordinates();
 ShortBuffer indices = face.getMeshTriangleIndices();
 // Center and region poses, mesh vertices, and normals are updated each frame.
 Pose facePose = face.getCenterPose();
 FloatBuffer faceVertices = face.getMeshVertices();
 FloatBuffer faceNormals = face.getMeshNormals();
 

Nested Classes

enum

AugmentedFace.RegionType

Defines face regions to query the pose for.

Public Methods

Anchor	createAnchor(Pose pose) Creates an anchor that is attached to this trackable, using the given initial pose in the world coordinate space.
boolean	equals(Object obj) Indicates whether some other object is a Trackable referencing the same logical trackable as this one.
Collection<Anchor>	getAnchors() Gets the Anchors attached to this trackable.
Pose	getCenterPose() Returns the pose of the center of the face, defined to have the origin located behind the nose and between the two cheek bones.
FloatBuffer	getMeshNormals() Returns a float buffer of 3D normals in (x, y, z) packing.
FloatBuffer	getMeshTextureCoordinates() Returns a float buffer of UV texture coordinates in (u, v) packing.
ShortBuffer	getMeshTriangleIndices() Returns a char buffer of triangles' indices in consecutive triplets.
FloatBuffer	getMeshVertices() Returns a float buffer of 3D vertices in (x, y, z) packing.
Pose	getRegionPose(AugmentedFace.RegionType regionType) Returns the pose of a region in world coordinates when the face trackable state is TRACKING.
TrackingState	getTrackingState() Gets this trackable's TrackingState.
int	hashCode() Returns a hash code value for the object.

Inherited Methods

From class java.lang.Object

Object	clone()
boolean	equals(Object arg0)
void	finalize()
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface com.google.ar.core.Trackable

abstract Anchor	createAnchor(Pose pose) Creates an anchor that is attached to this trackable, using the given initial pose in the world coordinate space.
abstract Collection<Anchor>	getAnchors() Gets the Anchors attached to this trackable.
abstract TrackingState	getTrackingState() Gets the TrackingState of this Trackable.

Public Methods

public Anchor createAnchor (Pose pose)

createAnchor

public Anchor createAnchor(
  Pose pose
)

Creates an anchor that is attached to this trackable, using the given initial pose in the world coordinate space. The type of trackable will determine the semantics of attachment and how the anchor's pose will be updated to maintain this relationship. Note that the relative offset between the pose of multiple anchors attached to a trackable may adjust slightly over time as ARCore updates its model of the world.

Details
Parameters	pose

public boolean equals (Object obj)

equals

public boolean equals(
  Object obj
)

Indicates whether some other object is a Trackable referencing the same logical trackable as this one.

Details
Parameters	obj the reference object with which to compare.
Returns	true if this object is the same as the obj argument; false otherwise.
See Also	hashCode() HashMap

public Collection<Anchor> getAnchors ()

getAnchors

public Collection<Anchor> getAnchors()

Gets the Anchors attached to this trackable.

public Pose getCenterPose ()

getCenterPose

public Pose getCenterPose()

Returns the pose of the center of the face, defined to have the origin located behind the nose and between the two cheek bones. Z+ is forward out of the nose, Y+ is upwards, and X+ is towards the left. The units are in meters. When the face trackable state is TRACKING, this pose is synced with the latest frame. When face trackable state is PAUSED, an identity pose will be returned.

Use getRegionPose(AugmentedFace.RegionType) to retrieve poses of specific regions of the face.

public FloatBuffer getMeshNormals ()

getMeshNormals

public FloatBuffer getMeshNormals()

Returns a float buffer of 3D normals in (x, y, z) packing.

Each (x, y, z) is a unit vector of the normal to the surface at each vertex. There is exactly one normal vector for each vertex. These normals are relative to the center pose of the face. When the face trackable state is TRACKING, this pose is synced with the latest frame. While face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns	a FloatBuffer of normals when TRACKING, or an empty FloatBuffer when PAUSED.

public FloatBuffer getMeshTextureCoordinates ()

getMeshTextureCoordinates

public FloatBuffer getMeshTextureCoordinates()

Returns a float buffer of UV texture coordinates in (u, v) packing.

There is a pair of texture coordinates for each vertex. These values do not change over a session, however while face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns	a FloatBuffer of texture coordinates when TRACKING, or an empty FloatBuffer when PAUSED.

public ShortBuffer getMeshTriangleIndices ()

getMeshTriangleIndices

public ShortBuffer getMeshTriangleIndices()

Returns a char buffer of triangles' indices in consecutive triplets.

Every three consecutive values are indices that represent a triangle. The vertex position and texture coordinates are mapped by the indices. The front face of each triangle is defined by the face where the vertices are in counter clockwise winding order. These values do not change, however while face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns	a ShortBuffer of triangle indices when TRACKING, or an empty ShortBuffer when PAUSED.

public FloatBuffer getMeshVertices ()

getMeshVertices

public FloatBuffer getMeshVertices()

Returns a float buffer of 3D vertices in (x, y, z) packing.

These vertices are relative to the center pose of the face with units in meters. When the face trackable state is TRACKING, this data is synced with the latest frame. While face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns	a FloatBuffer of vertices when TRACKING, or an empty FloatBuffer when PAUSED.

public Pose getRegionPose (AugmentedFace.RegionType regionType)

getRegionPose

public Pose getRegionPose(
  AugmentedFace.RegionType regionType
)

Returns the pose of a region in world coordinates when the face trackable state is TRACKING. When face trackable state is PAUSED, an identity pose will be returned.

Details
Parameters	regionType The face region for which to get the pose.
Returns	the Pose of the selected region when TRACKING, or an identity pose when PAUSED.

public TrackingState getTrackingState ()

getTrackingState

public TrackingState getTrackingState()

Gets this trackable's TrackingState.

public int hashCode ()

hashCode

public int hashCode()

Returns a hash code value for the object. This method is supported for the benefit of hash tables such as those provided by HashMap.

Details
Returns	a hash code value for this object.
See Also	equals(Object)