优化距离矩阵计算

如何解决优化距离矩阵计算

我试图从前两个分量的傅里叶变换计算矩阵距离。矩阵是40k x 40k，并且执行的方式非常慢。有没有一种计算矩阵的方法是一种更高效，更快捷的方法？

import numpy as np
from scipy.linalg import dft

#Transform the data using Fourier Transform.
ft = norm_data.dot(dft(8).transpose())/sqrt(8)

def ft_distance_calc(x,y):
    temp = np.zeros((x,y))
    for i in range(x):
        for z in range(y):
            temp[i,z] = sqrt(np.square(abs(ft[i,0:2] - ft[z,0:2])).sum())
    return temp

ft_distance = ft_distance_calc(40000,40000) 

解决方法

广播如何

def dist_cal(x,y):
    return np.sqrt(np.square(ft[:x,None,:2]-ft[None,:y,:2]).sum(-1))

# test
a = ft_distance_calc(400,200)
b = dist_cal(400,200)

(np.abs(a-b) < 1e-6).all()
# True

,

您可以使用内置功能：

from scipy.spatial.distance import cdist
def ft_distance_calc_2(x,y):
    return cdist(ft[:x,0:2],ft[:y,0:2])

---

比较，使用benchit：

#OP's solution
def ft_distance_calc(x,y):
    temp = np.zeros((x,y))
    for i in range(x):
        for z in range(y):
            temp[i,z] = np.sqrt(np.square(abs(ft[i,0:2] - ft[z,0:2])).sum())
    return temp

#@Ehsan's solution
def ft_distance_calc_2(x,0:2])

#@Quang's solution
def dist_cal(x,:2]).sum(-1))

ft = np.random.rand(1000,2)
in_ = {n:[n,n] for n in [10,100,1000]}

好像ft_distance_calc_2是最快的。