本文将介绍使用OpenCV实现多角度模板匹配的详细步骤 + 代码。

背景介绍

熟悉OpenCV的朋友肯定都知道OpenCV自带的模板匹配matchTemplate方法是不支持旋转的，也就是说当目标和模板有角度差异时匹配常常会失败，可能目标只是轻微的旋转，匹配分数就会下降很多，导致匹配精度下降甚至匹配出错。本文介绍基于matchTemplate + 旋转 + 金字塔下采样实现多角度的模板匹配，返回匹配结果(坐标、角度)。

实现思路

【1】如何适应目标的角度变化？我们可以将模板旋转，从0~360°依次匹配找到最佳的匹配位置；

【2】如何提高匹配速度？使用金字塔下采样，将模板和待匹配图均缩小后匹配；加大匹配搜寻角度的步长，比如从每1°匹配一次改为每5°匹配一次等。

实现步骤

【1】旋转模板图像。旋转图像本身比较简单，下面是代码：

# 图片旋转函数
def ImageRotate(img, angle):   # img:输入图片；newIm：输出图片；angle：旋转角度(°)
    height, width = img.shape[:2]  # 输入(H,W,C)，取 H，W 的值
    center = (width // 2, height // 2)  # 绕图片中心进行旋转
    M = cv.getRotationMatrix2D(center, angle, 1.0)
    image_rotation = cv.warpAffine(img, M, (width, height))
    return image_rotation

但需要注意，很多时候按照上面方法旋转时，会丢失模板信息产生黑边，这里是进行裁剪模板为圆形ROI

# 取圆形ROI区域函数：具体实现功能为输入原图，取原图最大可能的原型区域输出
def circle_tr(src):
    dst = np.zeros(src.shape, np.uint8)  # 感兴趣区域ROI
    mask = np.zeros(src.shape, dtype='uint8')  # 感兴趣区域ROI
    (h, w) = mask.shape[:2]
    (cX, cY) = (w // 2, h // 2)  # 是向下取整
    radius = int(min(h, w) / 2)
    cv.circle(mask, (cX, cY), radius, (255, 255, 255), -1)
    # 以下是copyTo的算法原理：
    # 先遍历每行每列（如果不是灰度图还需遍历通道，可以事先把mask图转为灰度图）
    for row in range(mask.shape[0]):
        for col in range(mask.shape[1]):
            # 如果掩图的像素不等于0，则dst(x,y) = scr(x,y)
            if mask[row, col] != 0:
                # dst_image和scr_Image一定要高宽通道数都相同，否则会报错
                dst[row, col] = src[row, col]
                # 如果掩图的像素等于0，则dst(x,y) = 0
            elif mask[row, col] == 0:
                dst[row, col] = 0
    return dst

【2】图像金字塔下采样。什么是图像金字塔？什么是上下采样？直接百度。

减小图像分辨率提高图像匹配速度，代码如下：

# 金字塔下采样
def ImagePyrDown(image,NumLevels):
    for i in range(NumLevels):
        image = cv.pyrDown(image)       #pyrDown下采样
    return image

【3】0~360°各角度匹配。旋转模板图像，依次调用matchTemplate在目标图中匹配，记录最佳匹配分数，以及对应的角度。旋转匹配代码：

# 旋转匹配函数（输入参数分别为模板图像、待匹配图像）
def RatationMatch(modelpicture, searchpicture):
    searchtmp = []
    modeltmp = []

    searchtmp = ImagePyrDown(searchpicture, 3)
    modeltmp = ImagePyrDown(modelpicture, 3)

    newIm = circle_tr(modeltmp)
    # 使用matchTemplate对原始灰度图像和图像模板进行匹配
    res = cv.matchTemplate(searchtmp, newIm, cv.TM_SQDIFF_NORMED)
    min_val, max_val, min_indx, max_indx = cv.minMaxLoc(res)
    location = min_indx
    temp = min_val
    angle = 0     # 当前旋转角度记录为0

    tic = time.time()
    # 以步长为5进行第一次粗循环匹配
    for i in range(-180, 181, 5):
        newIm = ImageRotate(modeltmp, i)
        newIm = circle_tr(newIm)
        res = cv.matchTemplate(searchtmp, newIm, cv.TM_SQDIFF_NORMED)
        min_val, max_val, min_indx, max_indx = cv.minMaxLoc(res)
        if min_val < temp:
            location = min_indx
            temp = min_val
            angle = i
    toc = time.time()
    print('第一次粗循环匹配所花时间为：' + str(1000*(toc - tic)) + 'ms')

    tic = time.time()
    # 在当前最优匹配角度周围10的区间以1为步长循环进行循环匹配计算
    for j in range(angle-5, angle+6):
        newIm = ImageRotate(modeltmp, j)
        newIm = circle_tr(newIm)
        res = cv.matchTemplate(searchtmp, newIm, cv.TM_SQDIFF_NORMED)
        min_val, max_val, min_indx, max_indx = cv.minMaxLoc(res)
        if min_val < temp:
            location = min_indx
            temp = min_val
            angle = j
    toc = time.time()
    print('在当前最优匹配角度周围10的区间以1为步长循环进行循环匹配所花时间为：' + str(1000*(toc - tic)) + 'ms')

    tic = time.time()
    # 在当前最优匹配角度周围2的区间以0.1为步长进行循环匹配计算
    k_angle = angle - 0.9
    for k in range(0, 19):
        k_angle = k_angle + 0.1
        newIm = ImageRotate(modeltmp, k_angle)
        newIm = circle_tr(newIm)
        res = cv.matchTemplate(searchtmp, newIm, cv.TM_SQDIFF_NORMED)
        min_val, max_val, min_indx, max_indx = cv.minMaxLoc(res)
        if min_val < temp:
            location = min_indx
            temp = min_val
            angle = k_angle
    toc = time.time()
    print('在当前最优匹配角度周围2的区间以0.1为步长进行循环匹配所花时间为：' + str(1000*(toc - tic)) + 'ms')

    # 用下采样前的图片来进行精匹配计算
    k_angle = angle - 0.1
    newIm = ImageRotate(modelpicture, k_angle)
    newIm = circle_tr(newIm)
    res = cv.matchTemplate(searchpicture, newIm, cv.TM_CCOEFF_NORMED)
    min_val, max_val, min_indx, max_indx = cv.minMaxLoc(res)
    location = max_indx
    temp = max_val
    angle = k_angle
    for k in range(1, 3):
        k_angle = k_angle + 0.1
        newIm = ImageRotate(modelpicture, k_angle)
        newIm = circle_tr(newIm)
        res = cv.matchTemplate(searchpicture, newIm, cv.TM_CCOEFF_NORMED)
        min_val, max_val, min_indx, max_indx = cv.minMaxLoc(res)
        if max_val > temp:
            location = max_indx
            temp = max_val
            angle = k_angle

    location_x = location[0] + 50
    location_y = location[1] + 50

    # 前面得到的旋转角度是匹配时模板图像旋转的角度，后面需要的角度值是待检测图像应该旋转的角度值，故需要做相反数变换
    angle = -angle

    match_point = {'angle': angle, 'point': (location_x, location_y)}
    return match_point

【4】标注匹配结果。根据模板图大小、匹配结果角度画出匹配框，代码如下：

# 画图
def draw_result(src, temp, match_point):
    cv.rectangle(src, match_point,
                  (match_point[0] + temp.shape[1], match_point[1] + temp.shape[0]),
                  (0, 255, 0), 2)
    cv.imshow('result', src)
    cv.waitKey()

【5】调用。对输入图像做预处理，并输出匹配到的结果，代码如下：

def get_realsense(src, temp):
    ModelImage = temp
    SearchImage = srcx
    ModelImage_edge = cv.GaussianBlur(ModelImage, (5, 5), 0)
    ModelImage_edge = cv.Canny(ModelImage_edge, 10, 200, apertureSize=3)
    SearchImage_edge = cv.GaussianBlur(SearchImage, (5, 5), 0)

    (h1, w1) = SearchImage_edge.shape[:2]
    SearchImage_edge = cv.Canny(SearchImage_edge, 10, 180, apertureSize=3)
    serch_ROIPart = SearchImage_edge[50:h1 - 50, 50:w1 - 50]  # 裁剪图像

    tic = time.time()
    match_points = RatationMatch(ModelImage_edge, serch_ROIPart)
    toc = time.time()
    print('匹配所花时间为：' + str(1000 * (toc - tic)) + 'ms')
    print('匹配的最优区域的起点坐标为：' + str(match_points['point']))
    print('相对旋转角度为：' + str(match_points['angle']))
    TmpImage_edge = ImageRotate(SearchImage_edge, match_points['angle'])
    cv.imshow("TmpImage_edge", TmpImage_edge)
    cv.waitKey()
    draw_result(SearchImage, ModelImage_edge, match_points['point'])
    return match_points

【6】举例演示。模板图从下图中截取并保存 template.png:

测试图像6张，匹配结果：

角度误差在正负1度左右。

后记

可以在此基础上添加匹配分数阈值和NMS实现多目标匹配。