You can use ML Kit to detect faces in images and video.

<br />

> [!NOTE]
> **Note:** ML Kit iOS APIs only run on 64-bit devices. If you build your app with 32-bit support, check the device's architecture before using this API.

## Try it out

- Play around with [the sample app](https://github.com/googlesamples/mlkit/tree/master/ios/quickstarts/vision) to see an example usage of this API.
- Try the code yourself with the [codelab](https://codelabs.developers.google.com/codelabs/mlkit-ios).

## Before you begin

1. Include the following ML Kit pods in your Podfile:

   ```
   pod 'GoogleMLKit/FaceDetection', '8.0.0'
   ```
2. After you install or update your project's Pods, open your Xcode project using its `.xcworkspace`. ML Kit is supported in Xcode version 12.4 or greater.

## Input image guidelines

For face recognition, you should use an image with dimensions of at least 480x360 pixels.
For ML Kit to accurately detect faces, input images must contain faces
that are represented by sufficient pixel data. In general, each face you want
to detect in an image should be at least 100x100 pixels. If you want to detect
the contours of faces, ML Kit requires higher resolution input: each face
should be at least 200x200 pixels.

If you detect faces in a real-time application, you might also want
to consider the overall dimensions of the input images. Smaller images can be
processed faster, so to reduce latency, capture images at lower resolutions, but keep
in mind the above accuracy requirements and ensure that the
subject's face occupies as much of the image as possible. Also see
[tips to improve real-time performance](https://developers.google.com/ml-kit/vision/face-detection/ios.md.txt#performance_tips).

Poor image focus can also impact accuracy. If you don't get acceptable
results, ask the user to recapture the image.

The orientation of a face relative to the camera can also affect what facial
features ML Kit detects. See
[Face Detection Concepts](https://developers.google.com/ml-kit/vision/face-detection/face-detection-concepts.md.txt#landmarks).

## 1. Configure the face detector

Before you apply face detection to an image, if you want to change any of the face detector's default settings, specify those settings with a [`FaceDetectorOptions`](https://developers.google.com/ml-kit/reference/swift/mlkitfacedetection/api/reference/Classes/FaceDetectorOptions.md.txt) object. You can change the following settings:

<br />

| Settings ||
|---|---|
| `performanceMode` | `fast` (default) \| `accurate` Favor speed or accuracy when detecting faces. |
| `landmarkMode` | `none` (default) \| `all` Whether to attempt to detect the facial "landmarks"---eyes, ears, nose, cheeks, mouth---of all detected faces. |
| `contourMode` | `none` (default) \| `all` Whether to detect the contours of facial features. Contours are detected for only the most prominent face in an image. |
| `classificationMode` | `none` (default) \| `all` Whether or not to classify faces into categories such as "smiling" and "eyes open". |
| `minFaceSize` | `CGFloat` (default: `0.1`) Sets the smallest desired face size, expressed as the ratio of the width of the head to width of the image. |
| `isTrackingEnabled` | `false` (default) \| `true` Whether or not to assign faces an ID, which can be used to track faces across images. Note that when contour detection is enabled, only one face is detected, so face tracking doesn't produce useful results. For this reason, and to improve detection speed, don't enable both contour detection and face tracking. |

For example, build a [`FaceDetectorOptions`](https://developers.google.com/ml-kit/reference/swift/mlkitfacedetection/api/reference/Classes/FaceDetectorOptions.md.txt)
object like one of the following examples:

### Swift

```
// High-accuracy landmark detection and face classification
let options = FaceDetectorOptions()
options.performanceMode = .accurate
options.landmarkMode = .all
options.classificationMode = .all

// Real-time contour detection of multiple faces
// options.contourMode = .all
```

### Objective-C

```
// High-accuracy landmark detection and face classification
MLKFaceDetectorOptions *options = [[MLKFaceDetectorOptions alloc] init];
options.performanceMode = MLKFaceDetectorPerformanceModeAccurate;
options.landmarkMode = MLKFaceDetectorLandmarkModeAll;
options.classificationMode = MLKFaceDetectorClassificationModeAll;

// Real-time contour detection of multiple faces
// options.contourMode = MLKFaceDetectorContourModeAll;
```

## 2. Prepare the input image

To detect faces in an image, pass the image as a `UIImage` or a `CMSampleBufferRef` to the `FaceDetector` using either the `process(_:completion:)` or `results(in:)` method:

<br />

Create a [`VisionImage`](https://developers.google.com/ml-kit/reference/swift/mlkitvision/api/reference/Classes/VisionImage.md.txt) object using a `UIImage` or a
`CMSampleBuffer`.

If you use a `UIImage`, follow these steps:

- Create a [`VisionImage`](https://developers.google.com/ml-kit/reference/swift/mlkitvision/api/reference/Classes/VisionImage.md.txt) object with the `UIImage`. Make sure to specify the correct `.orientation`.

  ### Swift

  ```
  let image = VisionImage(image: UIImage)
  visionImage.orientation = image.imageOrientation
  ```

  ### Objective-C

  ```
  MLKVisionImage *visionImage = [[MLKVisionImage alloc] initWithImage:image];
  visionImage.orientation = image.imageOrientation;
  ```

If you use a `CMSampleBuffer`, follow these steps:

- Specify the orientation of the image data contained in the
  `CMSampleBuffer`.

  To get the image orientation:

  ### Swift

  ```
  func imageOrientation(
    deviceOrientation: UIDeviceOrientation,
    cameraPosition: AVCaptureDevice.Position
  ) -> UIImage.Orientation {
    switch deviceOrientation {
    case .portrait:
      return cameraPosition == .front ? .leftMirrored : .right
    case .landscapeLeft:
      return cameraPosition == .front ? .downMirrored : .up
    case .portraitUpsideDown:
      return cameraPosition == .front ? .rightMirrored : .left
    case .landscapeRight:
      return cameraPosition == .front ? .upMirrored : .down
    case .faceDown, .faceUp, .unknown:
      return .up
    }
  }
        
  ```

  ### Objective-C

  ```
  - (UIImageOrientation)
    imageOrientationFromDeviceOrientation:(UIDeviceOrientation)deviceOrientation
                           cameraPosition:(AVCaptureDevicePosition)cameraPosition {
    switch (deviceOrientation) {
      case UIDeviceOrientationPortrait:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationLeftMirrored
                                                              : UIImageOrientationRight;

      case UIDeviceOrientationLandscapeLeft:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationDownMirrored
                                                              : UIImageOrientationUp;
      case UIDeviceOrientationPortraitUpsideDown:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationRightMirrored
                                                              : UIImageOrientationLeft;
      case UIDeviceOrientationLandscapeRight:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationUpMirrored
                                                              : UIImageOrientationDown;
      case UIDeviceOrientationUnknown:
      case UIDeviceOrientationFaceUp:
      case UIDeviceOrientationFaceDown:
        return UIImageOrientationUp;
    }
  }
        
  ```
- Create a [`VisionImage`](https://developers.google.com/ml-kit/reference/swift/mlkitvision/api/reference/Classes/VisionImage.md.txt) object using the `CMSampleBuffer` object and orientation:

  ### Swift

  ```
  let image = VisionImage(buffer: sampleBuffer)
  image.orientation = imageOrientation(
    deviceOrientation: UIDevice.current.orientation,
    cameraPosition: cameraPosition)
  ```

  ### Objective-C

  ```
   MLKVisionImage *image = [[MLKVisionImage alloc] initWithBuffer:sampleBuffer];
   image.orientation =
     [self imageOrientationFromDeviceOrientation:UIDevice.currentDevice.orientation
                                  cameraPosition:cameraPosition];
  ```

## 3. Get an instance of FaceDetector

Get an instance of [`FaceDetector`](https://developers.google.com/ml-kit/reference/swift/mlkitfacedetection/api/reference/Classes/FaceDetector.md.txt):

### Swift

```
let faceDetector = FaceDetector.faceDetector(options: options)
```

### Objective-C

```
MLKFaceDetector *faceDetector = [MLKFaceDetector faceDetectorWithOptions:options];
      
```

<br />

## 4. Process the image

Then, pass the image to the `process()` method:

### Swift

```
weak var weakSelf = self
faceDetector.process(visionImage) { faces, error in
  guard let strongSelf = weakSelf else {
    print("Self is nil!")
    return
  }
  guard error == nil, let faces = faces, !faces.isEmpty else {
    // ...
    return
  }

  // Faces detected
  // ...
}
```

### Objective-C

```
[faceDetector processImage:image
                completion:^(NSArray<MLKFace *> *faces,
                             NSError *error) {
  if (error != nil) {
    return;
  }
  if (faces.count > 0) {
    // Recognized faces
  }
}];
```

## 5. Get information about detected faces

If the face detection operation succeeds, the face detector passes an array of [`Face`](https://developers.google.com/ml-kit/reference/swift/mlkitfacedetection/api/reference/Classes/Face.md.txt) objects to the completion handler. Each `Face` object represents a face that was detected in the image. For each face, you can get its bounding coordinates in the input image, as well as any other information you configured the face detector to find. For example:

<br />

### Swift

```
for face in faces {
  let frame = face.frame
  if face.hasHeadEulerAngleX {
    let rotX = face.headEulerAngleX  // Head is rotated to the uptoward rotX degrees
  }
  if face.hasHeadEulerAngleY {
    let rotY = face.headEulerAngleY  // Head is rotated to the right rotY degrees
  }
  if face.hasHeadEulerAngleZ {
    let rotZ = face.headEulerAngleZ  // Head is tilted sideways rotZ degrees
  }

  // If landmark detection was enabled (mouth, ears, eyes, cheeks, and
  // nose available):
  if let leftEye = face.landmark(ofType: .leftEye) {
    let leftEyePosition = leftEye.position
  }

  // If contour detection was enabled:
  if let leftEyeContour = face.contour(ofType: .leftEye) {
    let leftEyePoints = leftEyeContour.points
  }
  if let upperLipBottomContour = face.contour(ofType: .upperLipBottom) {
    let upperLipBottomPoints = upperLipBottomContour.points
  }

  // If classification was enabled:
  if face.hasSmilingProbability {
    let smileProb = face.smilingProbability
  }
  if face.hasRightEyeOpenProbability {
    let rightEyeOpenProb = face.rightEyeOpenProbability
  }

  // If face tracking was enabled:
  if face.hasTrackingID {
    let trackingId = face.trackingID
  }
}
```

### Objective-C

```
for (MLKFace *face in faces) {
  // Boundaries of face in image
  CGRect frame = face.frame;
  if (face.hasHeadEulerAngleX) {
    CGFloat rotX = face.headEulerAngleX;  // Head is rotated to the upward rotX degrees
  }
  if (face.hasHeadEulerAngleY) {
    CGFloat rotY = face.headEulerAngleY;  // Head is rotated to the right rotY degrees
  }
  if (face.hasHeadEulerAngleZ) {
    CGFloat rotZ = face.headEulerAngleZ;  // Head is tilted sideways rotZ degrees
  }

  // If landmark detection was enabled (mouth, ears, eyes, cheeks, and
  // nose available):
  MLKFaceLandmark *leftEar = [face landmarkOfType:FIRFaceLandmarkTypeLeftEar];
  if (leftEar != nil) {
    MLKVisionPoint *leftEarPosition = leftEar.position;
  }

  // If contour detection was enabled:
  MLKFaceContour *upperLipBottomContour = [face contourOfType:FIRFaceContourTypeUpperLipBottom];
  if (upperLipBottomContour != nil) {
    NSArray<MLKVisionPoint *> *upperLipBottomPoints = upperLipBottomContour.points;
    if (upperLipBottomPoints.count > 0) {
      NSLog("Detected the bottom contour of the subject's upper lip.")
    }
  }

  // If classification was enabled:
  if (face.hasSmilingProbability) {
    CGFloat smileProb = face.smilingProbability;
  }
  if (face.hasRightEyeOpenProbability) {
    CGFloat rightEyeOpenProb = face.rightEyeOpenProbability;
  }

  // If face tracking was enabled:
  if (face.hasTrackingID) {
    NSInteger trackingID = face.trackingID;
  }
}
```

### Example of face contours

When you have face contour detection enabled, you get a list of points for
each facial feature that was detected. These points represent the shape of the
feature. See [Face
Detection Concepts](https://developers.google.com/ml-kit/vision/face-detection/face-detection-concepts#contours) for details about how contours are
represented.

The following image illustrates how these points map to a face, click the
image to enlarge it:
[![example detected face contour mesh](https://developers.google.com/static/ml-kit/vision/face-detection/images/face_contours.svg)](https://developers.google.com/static/ml-kit/vision/face-detection/images/face_contours.svg)

## Real-time face detection

If you want to use face detection in a real-time application, follow these
guidelines to achieve the best framerates:

- Configure the face detector to use either
  face contour detection or classification and landmark detection, but not both:

  Contour detection  

  Landmark detection  

  Classification  

  Landmark detection and classification  

  Contour detection and landmark detection  

  Contour detection and classification  

  Contour detection, landmark detection, and classification  
- Enable `fast` mode (enabled by default).

- Consider capturing images at a lower resolution. However, also keep in mind
  this API's image dimension requirements.


- For processing video frames, use the `results(in:)` synchronous API of the detector. Call this method from the [`AVCaptureVideoDataOutputSampleBufferDelegate`](https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutputsamplebufferdelegate)'s [`captureOutput(_, didOutput:from:)`](https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutputsamplebufferdelegate/1385775-captureoutput) function to synchronously get results from the given video frame. Keep [`AVCaptureVideoDataOutput`](https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutput)'s [`alwaysDiscardsLateVideoFrames`](https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutput/1385780-alwaysdiscardslatevideoframes) as `true` to throttle calls to the detector. If a new video frame becomes available while the detector is running, it will be dropped.
- If you use the output of the detector to overlay graphics on the input image, first get the result from ML Kit, then render the image and overlay in a single step. By doing so, you render to the display surface only once for each processed input frame. See the [updatePreviewOverlayViewWithLastFrame](https://github.com/googlesamples/mlkit/blob/master/ios/quickstarts/vision/VisionExample/CameraViewController.swift) in the ML Kit quickstart sample for an example.