在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败错误信息其他信息工作示例

如何解决在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败错误信息其他信息工作示例

我一直在尝试使用 MSVC 编译器和英特尔 MKL 从 Windows 10 上的源代码编译 NumPy。我正在使用 Microsoft Visual Studio 2019 (16.8.4) 和 Intel MKL 2017.8.275 运行 Windows 10.0.18363。

我设法使用最新的 Python 和 NumPy 以最少的设置重现了该问题。

下载最新的 Python (3.9.1) 和最新的 NumPy (1.20.1) 源。
打开 VS 命令提示符，解压 Python 源代码，使用 PCbuild\build.bat 构建
运行 mklvars.bat intel64 以获得正确的环境变量集。

将英特尔编译器（ifort 所需）添加到 PATH：

set PATH=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64;%PATH%

创建一个虚拟环境，从 Python 构建中复制一些文件并激活虚拟环境：

copy Python\PCbuild\amd64\python39.dll venv\Scripts
copy Python\PC\pyconfig.h venv\Include

从源代码构建 NumPy 并安装：pip install . -v
尝试导入 NumPy：python -c "import numpy"

错误信息

Traceback (most recent call last):
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py",line 22,in <module>
    from . import multiarray
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py",line 12,in <module>
    from . import overrides
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py",line 7,in <module>
    from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "<string>",line 1,in <module>
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py",line 145,in <module>
    from . import core
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py",line 48,in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons,often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "C:\cygwin\home\user\numpy_clean_env\venv\Scripts\python.exe"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.

其他信息

MKL 库在编译期间被拾取，因为它返回：

FOUND:
        libraries = ['mkl_rt']
        library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib\\intel64']
        define_macros = [('SCIPY_MKL_H',None),('HAVE_CBLAS',None)]
        include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl','C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\include','C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib']

我尝试使用 Dependencies（Dependency Walker 的较新版本）分析 DLL 解析，但似乎 MKL DLL 加载正常。有一些 DLL 显示为未正确加载，但据我所知，这是由 Windows API 集的检查软件限制引起的，而不是由该 DLL 的实际问题引起的，所以我认为系统设置正确。以下是在分析 NOT_FOUND 时导致依赖项中出现 _multiarray_umath.pyd 的唯一 DLL 列表：

EMCLIENT.dll (NOT_FOUND) :
HvsiFileTrust.dll (NOT_FOUND) :
UpdateAPI.dll (NOT_FOUND) :
api-ms-win-core-comm-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-ansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-stringansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-winrt-string-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-0.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-1.dll (NOT_FOUND) :
api-ms-win-coreui-secruntime-l1-1-0.dll (NOT_FOUND) :
ext-ms-mf-pal-l2-1-0.dll (NOT_FOUND) :
ext-ms-onecore-appmodel-emclient-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-defaultdiscovery-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-orientation-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-shellchromeapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-deployment-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-usercontext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-viewscalefactor-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-audiocore-pal-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-com-suspendresiliency-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-core-winrt-remote-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dwmapi-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dxcore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-networking-wlanstorage-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-2.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-3.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-4.dll (NOT_FOUND) :
ext-ms-win-ras-tapi32-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ro-typeresolution-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-input-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-private-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-capauthz-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-chambers-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-knownfolderext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-shell32-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-shell-shlwapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-tabbedtitlebar-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellcom-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellfolders-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ui-viewmanagement-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-uiacore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-wer-xbox-l1-1-0.dll (NOT_FOUND) :
ext-ms-win32-subsystem-query-l1-1-0.dll (NOT_FOUND) :
ext-ms-windowscore-deviceinfo-l1-1-0.dll (NOT_FOUND) :

我也尝试遵循 this guide 并且我可以确认 os.path.exists('_multiarray_umath.pyd') 返回 True，但是 ctypes.WinDLL('_multiarray_umath.pyd') 无法加载 DLL（或其依赖项之一）。

工作示例

与上述步骤相同，但不执行 mklvars.bat，因此不链接到英特尔 MKL。

过去，此设置用于 Python 3.6、VS2015 和类似版本的英特尔 MKL。我可能遗漏了一些东西，欢迎提出任何建议。

解决方法

通过遵循 Numpy 邮件列表 (permalink) 上的建议解决。

Python >= 3.8 将不再使用 PATH 来解析依赖项扩展模块。使用 os.add_dll_directory(mkl_bin_path) https://docs.python.org/3/library/os.html#os.add_dll_directory 总共在导入 numpy 或将调用添加到 numpy包目录下的_distributor_init.py文件。

在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败 错误信息其他信息工作示例

如何解决在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败 错误信息其他信息工作示例