通过 Mask-R CNN 从掩码中检测到错误的角

如何解决通过 Mask-R CNN 从掩码中检测到错误的角

我有一个应用程序，我使用 Mask-R CNN 训练模型来识别房屋图像中的门窗。该模型在线预测看不见的图像中的物体，根据这些预测，我们尝试使用定制的前端来适应新的门窗设计。为了使新设计适合检测到的掩码，我们尝试检测所述掩码的四个角。这些新计算的点将成为使用称为配置器的前端上传设计的角点。

下面是用于计算图像中预测的所有实例（掩码）的角点的代码。

import cv2
import numpy as np
from functools import cmp_to_key

r = results[0]  # r now contains all the masks predicted
mask = r['masks'].astype(np.float32)

for i in range(len(r['rois'])):
    if r['scores'][i] > 0.95:
        find_corners(mask[:,:,i],mask.shape)

def simplify_contour(contour,n_corners=4):
    """
    Binary searches best `epsilon` value to force contour
        approximation contain exactly `n_corners` points.

    :param contour: OpenCV2 contour.
    :param n_corners: Number of corners (points) the contour must contain.

    :returns: Simplified contour in successful case. Otherwise returns initial contour.
    """

    n_iter,max_iter = 0,100
    lb,ub = 0.,1.

    while True:
        n_iter += 1
        if n_iter > max_iter:
            return contour

        k = (lb + ub)/2.
        eps = k*cv2.arcLength(contour,True)
        approx = cv2.approxPolyDP(contour,eps,True)

        if len(approx) > n_corners:
            lb = (lb + ub)/2.
        elif len(approx) < n_corners:
            ub = (lb + ub)/2.
        else:
            return approx


def find_corners(mask,size):
    """
    Returns the 4 corners for the given door or window mask
    The points are normalized to be between 0..1
    The order must be
    d c
    a b
    -> [a,b,c,d]
    where the coordinate 0,0 is the lower left corner.
    :param mask:  float32 mask image
    :param size: tuple (x,y) the original image size
    :return: [{x: float,y: float}] point array
    """

    m = np.uint8(mask)
    # thresh = cv2.Canny(m,1)

    contours,h = cv2.findContours(m,1,2)

    for cnt in contours:
        approx = simplify_contour(cnt,4)
        if len(approx) == 4:
            p = [x[0] for x in approx]
            p_list.append(p)
            print(p)
            c = order_by_rows(p)
            return [{'x': float(x[0]) / size[1],'y': float(x[1]) / size[0]} for x in c]

    return None


def order_by_rows(points):
    """
    Orders the given points counter clockwise around their center
    Only works because there are 4 entries
    :param points: 4 points
    :return:  4 points ordered
    """
    c = sorted(points,key=cmp_to_key(compare))

    if c[0][0] > c[1][0]:
        tmp = c[0]
        c[0] = c[1]
        c[1] = tmp

    if c[2][0] < c[3][0]:
        tmp = c[2]
        c[2] = c[3]
        c[3] = tmp

    return c


def compare(a,b):
    if a[1] < b[1]:
        return -1
    if a[1] > b[1]:
        return 1
    return 0

问题在于，当新设计安装在这些角点上时，它们中的大多数都有些倾斜，使感兴趣的对象部分未被覆盖。以下是尝试重新设计窗口时预测的掩码图像和来自配置器前端的图像。