如何基于Tensorflow.js中的边界框裁剪脸部？

如何解决如何基于Tensorflow.js中的边界框裁剪脸部？

我需要裁剪在Blazeface模型中检测到的面部，然后将图像发送到我制作的自定义模型中。我已经使用边界框实现了人脸检测，但是被困在裁剪人脸上。

我具有地标的坐标以及bott omright和topLeft的坐标，但是我不知道该怎么做。在带有tensorflow的python中，它们存在的功能可以做到这一点，但是在tensorflow.js中我找不到任何东西。

在脸上渲染边界框

    const faces = await bfModel
      .estimateFaces(tensor,returnTensors)
      .catch(e => console.log(e));
    console.log(faces);

    // Faces is an array of objects
    if (!isEmpty(faces)) {
      setModelFaces({ faces });
    }

  const renderBoundingBoxes = () => {
    const { faces } = modelFaces;
    const scale = {
      height: styles.camera.height / tensorDims.height,width: styles.camera.width / tensorDims.width
    };
    const flipHorizontal = Platform.OS === "ios" ? false : true;
    if (!isEmpty(faces)) {
      return faces.map((face,i) => {
        const { topLeft,bottomright } = face;
        const bbLeft = topLeft.dataSync()[0] * scale.width;
        const BoxStyle = Object.assign({},styles.bBox,{
          left: flipHorizontal
            ? previewWidth - bbLeft - previewLeft
            : bbLeft + previewLeft,top: topLeft.dataSync()[1] * scale.height + 20,width:
            (bottomright.dataSync()[0] - topLeft.dataSync()[0]) * scale.width,height:
            (bottomright.dataSync()[1] - topLeft.dataSync()[1]) * scale.height
        });

        return <View style={BoxStyle}></View>;
        1;
      });
    }
  };

console.log的输出（面孔）

Array [
  Object {
    "bottomright": Tensor {
      "dataId": Object {},"dtype": "float32","id": 220600,"isdisposedInternal": false,"kept": false,"rankType": "1","scopeId": 426282,"shape": Array [
        2,],"size": 2,"strides": Array [],},"landmarks": Tensor {
      "dataId": Object {},"id": 220602,"rankType": "2","scopeId": 426286,"shape": Array [
        6,2,"size": 12,"strides": Array [
        2,"probability": Tensor {
      "dataId": Object {},"id": 220592,"scopeId": 426249,"shape": Array [
        1,"size": 1,"topLeft": Tensor {
      "dataId": Object {},"id": 220599,"scopeId": 426280,]

解决方法

可以使用tf.image.cropAndResize裁剪

图像。张量应为4d张量。如果图像是3d张量，则首先需要对其进行扩展。作物的预期高度和宽度应作为参数传递给copAndResize

boxes = tf.concat([topLeftTensor,bottomRightTensor]).reshape([-1,4])
crop = tf.image.cropAndResize(images,boxes,[0],[height,width])