在 iOS 上使用 ML Kit 為圖片加上標籤

您可以使用 ML Kit 為圖片中辨識出的物件加上標籤。提供的預設模型 ML Kit 支援超過 400 個不同標籤。

立即試用

事前準備

  1. 在 Podfile 中加入下列 ML Kit Pod:
    pod 'GoogleMLKit/ImageLabeling', '15.5.0'
    
  2. 安裝或更新專案的 Pod 後,請使用 .xcworkspace。Xcode 12.4 以上版本支援 ML Kit。

您現在可以開始為圖片加上標籤。

1. 準備輸入圖片

使用 UIImageVisionImage CMSampleBuffer

如果您使用 UIImage,請按照下列步驟操作:

  • 使用 UIImage 建立 VisionImage 物件。請務必指定正確的 .orientation

    Swift

    let image = VisionImage(image: UIImage)
    visionImage.orientation = image.imageOrientation

    Objective-C

    MLKVisionImage *visionImage = [[MLKVisionImage alloc] initWithImage:image];
    visionImage.orientation = image.imageOrientation;

如果您使用 CMSampleBuffer,請按照下列步驟操作:

  • 指定 CMSampleBuffer

    如何取得圖片方向:

    Swift

    func imageOrientation(
      deviceOrientation: UIDeviceOrientation,
      cameraPosition: AVCaptureDevice.Position
    ) -> UIImage.Orientation {
      switch deviceOrientation {
      case .portrait:
        return cameraPosition == .front ? .leftMirrored : .right
      case .landscapeLeft:
        return cameraPosition == .front ? .downMirrored : .up
      case .portraitUpsideDown:
        return cameraPosition == .front ? .rightMirrored : .left
      case .landscapeRight:
        return cameraPosition == .front ? .upMirrored : .down
      case .faceDown, .faceUp, .unknown:
        return .up
      }
    }
          

    Objective-C

    - (UIImageOrientation)
      imageOrientationFromDeviceOrientation:(UIDeviceOrientation)deviceOrientation
                             cameraPosition:(AVCaptureDevicePosition)cameraPosition {
      switch (deviceOrientation) {
        case UIDeviceOrientationPortrait:
          return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationLeftMirrored
                                                                : UIImageOrientationRight;
    
        case UIDeviceOrientationLandscapeLeft:
          return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationDownMirrored
                                                                : UIImageOrientationUp;
        case UIDeviceOrientationPortraitUpsideDown:
          return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationRightMirrored
                                                                : UIImageOrientationLeft;
        case UIDeviceOrientationLandscapeRight:
          return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationUpMirrored
                                                                : UIImageOrientationDown;
        case UIDeviceOrientationUnknown:
        case UIDeviceOrientationFaceUp:
        case UIDeviceOrientationFaceDown:
          return UIImageOrientationUp;
      }
    }
          
  • 使用VisionImage CMSampleBuffer 物件和方向:

    Swift

    let image = VisionImage(buffer: sampleBuffer)
    image.orientation = imageOrientation(
      deviceOrientation: UIDevice.current.orientation,
      cameraPosition: cameraPosition)

    Objective-C

     MLKVisionImage *image = [[MLKVisionImage alloc] initWithBuffer:sampleBuffer];
     image.orientation =
       [self imageOrientationFromDeviceOrientation:UIDevice.currentDevice.orientation
                                    cameraPosition:cameraPosition];

2. 設定並執行映像檔標籤工具

如要為圖片中的物件加上標籤,請將 VisionImage 物件傳遞至 ImageLabelerprocessImage() 方法。

  1. 首先,請取得 ImageLabeler 的例項。

Swift

let labeler = ImageLabeler.imageLabeler()

// Or, to set the minimum confidence required:
// let options = ImageLabelerOptions()
// options.confidenceThreshold = 0.7
// let labeler = ImageLabeler.imageLabeler(options: options)

Objective-C

MLKImageLabeler *labeler = [MLKImageLabeler imageLabeler];

// Or, to set the minimum confidence required:
// MLKImageLabelerOptions *options =
//         [[MLKImageLabelerOptions alloc] init];
// options.confidenceThreshold = 0.7;
// MLKImageLabeler *labeler =
//         [MLKImageLabeler imageLabelerWithOptions:options];
  1. 接著,將圖片傳遞至 processImage() 方法:

Swift

labeler.process(image) { labels, error in
    guard error == nil, let labels = labels else { return }

    // Task succeeded.
    // ...
}

Objective-C

[labeler processImage:image
completion:^(NSArray *_Nullable labels,
            NSError *_Nullable error) {
   if (error != nil) { return; }

   // Task succeeded.
   // ...
}];

3. 取得加上標籤的物件相關資訊

如果圖片標籤成功,完成處理常式會收到 ImageLabel 物件。每個 ImageLabel 物件都代表 映像檔和映像檔版本基礎模型支援 400 多種標籤。 您可以取得每個標籤的文字說明以及 和比對結果的可信度分數例如:

Swift

for label in labels {
    let labelText = label.text
    let confidence = label.confidence
    let index = label.index
}

Objective-C

for (MLKImageLabel *label in labels) {
   NSString *labelText = label.text;
   float confidence = label.confidence;
   NSInteger index = label.index;
}

即時效能改善訣竅

如要在即時應用程式中為圖片加上標籤,請按照下列步驟操作: 實現最佳影格速率: