这段代码是否可以在多个处理器上执行吗？

如何解决这段代码是否可以在多个处理器上执行吗？

下面的代码是否是跨多个处理器（dfA_Train和dfB_Train是Pandas DataFrames列表）并行执行 func 的正确方法？

from multiprocessing import Pool
import itertools

def func(dfA,dfB,param):
  L = []
  ...
  return L #L is a list of dictionaries

if __name__ == "__main__":
   dictList = []
   for param in paramGrid:
      with Pool(processes=8) as pool:
         rslts = pool.starmap(func,zip(dfA_Train,dfB_Train,itertools.repeat(param)))
      rslts = [item for subList in rslts for item in subList if item != []]
     _ = [dictList.append(item) for item in rslts]
   results = pd.DataFrame.from_dict(dictList) if dictList else None

似乎没有在超过1个proc中运行-单进程和多个proc之间没有加速。

解决方法

我相信multiprocessing.dummy仅根据下面的文档在内部调用多个线程。当我们寻找线程时，显然会调用一个进程，其中每个线程都由全局解释器锁控制。 multiprocessing.Pool()将是此处实现并行性的合适人选。与线程不同，Pool对象为每个任务分配单独的进程。例如，请考虑以下示例。

https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing.dummy

from multiprocessing import Pool

def worker_function(arg_1,arg_2):
    print('do some work with arg_1 and arg_2')

work = (["A",5],["B",2],["C",1],["D",3])
proc_pool = Pool(<desired no. of processes>)          # create a multi processing Pool
proc_pool.map(worker_function,work)                  # start parallel work