[废除, YOLO V5 V6.0 起, 自带 .pt 导出 .engine 功能] TensorRT 加速 YOLO V5 模型的推理

博文目录

---

YOLO V5 6.2 (注意选对 Tag)
TensorRTx For YOLO V5 6.2 (注意选对 Tag)

环境准备

YOLO V5 6.2

PyTorch Cuda

Python Apex YOLO v5 AI 自瞄 全过程记录

参考上文中 环境准备 YOLO V5 6.2 部分

准备代码

克隆仓库地址: git clone -b v6.2 https://github.com/ultralytics/yolov5.git

TensorRTx For YOLO V5 6.2

TensorRTx Tutorial
TensorRTx Tutorial RunOnWindows

OpenCV

官网 OpenCV Release
GitHub OpenCV Release
opencv-4.6.0-vc14_vc15.exe

安装即解压,到指定目录,将 C:\mrathena\develop\opencv-4.6.0\build\x64\vc15\bin 配置到环境变量 Path

Cuda

CUDA 下载

安装选项 选 推荐的 精简 安装

安装时会自动将部分组件集成到 Visual Studio 开发环境中,具体影响可以参考 这里,我并不使用 VS,所以选择打勾不安装

安装完成后,执行 nvcc --version 或 nvcc -V,如果输出如下信息,则安装成功

TensorRT

TensorRT 下载

执行 nvidia-smi,驱动版本是 522.25,可以支持到 CUDA 11.8. PyTorch 环境中 CUDA 选的是 11.6,综合下来,TensorRT 也选支持 11.6 的

dirent.h

Dirent 是一个编程接口，用于检索有关 C 和 C++ 语言中的文件和目录的信息。该项目为 Microsoft Visual Studio 提供了一个 Dirent 接口。

dirent.h for windows,put into tensorrtx/include,download from https://github.com/tronkko/dirent

准备代码

克隆仓库地址: git clone -b yolov5-v6.2 https://github.com/wang-xinyu/tensorrtx.git

从 .pt 生成 .wts

拷贝 TensorRTx/yolov5/gen_wts.py 到 YOlov5 环境中,在 conda 激活 yolo 环境后执行转换命令,生成 yolov5s.wts

cp {tensorrtx}/yolov5/gen_wts.py {ultralytics}/yolov5
cd {ultralytics}/yolov5
python gen_wts.py -w yolov5s.pt -o yolov5s.wts
// a file 'yolov5s.wts' will be generated.

构建并运行 TensorRTx/yolov5,C++

配置 CLion

C/C++ CLion Windows 开发环境搭建

CLion 开发 CUDA 应用,工具链必须使用 Visual Studio. 安装 Visual Studio 时必须选择 C++ 桌面开发里的 MSVC 和 Windows SDK

初始 CMakeLists.txt

cmake_minimum_required(VERSION 2.6)

project(yolov5)

add_deFinitions(-std=c++11)
add_deFinitions(-DAPI_EXPORTS)
option(CUDA_USE_STATIC_CUDA_RUNTIME OFF)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_BUILD_TYPE Debug)

find_package(CUDA required)

if(WIN32)
enable_language(CUDA)
endif(WIN32)

include_directories(${PROJECT_SOURCE_DIR}/include)
# include and link dirs of cuda and tensorrt,you need adapt them if yours are different
# cuda
include_directories(/usr/local/cuda/include)
link_directories(/usr/local/cuda/lib64)
# tensorrt
include_directories(/usr/include/x86_64-linux-gnu/)
link_directories(/usr/lib/x86_64-linux-gnu/)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -Ofast -g -Wfatal-errors -D_MWAITXINTRIN_H_INCLUDED")
cuda_add_library(myplugins SHARED yololayer.cu)
target_link_libraries(myplugins nvinfer cudart)

find_package(OpenCV)
include_directories(${OpenCV_INCLUDE_Dirs})

cuda_add_executable(yolov5 calibrator.cpp yolov5.cpp preprocess.cu)

target_link_libraries(yolov5 nvinfer)
target_link_libraries(yolov5 cudart)
target_link_libraries(yolov5 myplugins)
target_link_libraries(yolov5 ${OpenCV_LIBS})

add_executable(yolov5-cls calibrator.cpp yolov5_cls.cpp)

target_link_libraries(yolov5-cls nvinfer)
target_link_libraries(yolov5-cls cudart)
target_link_libraries(yolov5-cls myplugins)
target_link_libraries(yolov5-cls ${OpenCV_LIBS})

if(UNIX)
add_deFinitions(-O2 -pthread)
endif(UNIX)

修改 CMakeLists.txt 的过程

我使用的 IDE 是 CLion,它也是使用该目录自带的 CMakeLists.txt 来构建工程,Build

警告

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

意思是版本小于 2.8.12 的未来可能会移除 … 先改个 2.8.12

报错

TensorRTx Tutorial RunOnWindows

CMake Error at C:/mrathena/develop/clion-2021.2.4/bin/cmake/win/share/cmake-3.20/Modules/FindCUDA.cmake:859 (message):
  Specify CUDA_TOOLKIT_ROOT_DIR
Call Stack (most recent call first):
  CMakeLists.txt:11 (find_package)

意思是 find_package 找不到 CUDA 目录,需要我们指定 CUDA_TOOLKIT_ROOT_DIR,在它上一行添加

set(CUDA_TOOLKIT_ROOT_DIR "C:\\Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8")

又报错

CMake Error at C:/mrathena/develop/clion-2021.2.4/bin/cmake/win/share/cmake-3.20/Modules/FindCUDA.cmake:903 (find_program):
  Syntax error in cmake code at

    C:/mrathena/develop/clion-2021.2.4/bin/cmake/win/share/cmake-3.20/Modules/FindCUDA.cmake:903

  when parsing string

    CUDA_NVCC_EXECUTABLE;NAMES;nvcc;PATHS;C:\Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8;ENV;CUDA_PATH;ENV;CUDA_BIN_PATH;PATH_SUFFIXES;bin;bin64;NO_DEFAULT_PATH

  Invalid escape sequence \P
Call Stack (most recent call first):
  C:/mrathena/develop/clion-2021.2.4/bin/cmake/win/share/cmake-3.20/Modules/FindCUDA.cmake:913 (cuda_find_host_program)
  CMakeLists.txt:12 (find_package)

无效字符 \P,路径要用 / 而不是 \ 或 \\

补充

根据 教程,可以在 CMakeLists.txt 中打印一些信息

在 find_package(CUDA required) 下添加

message(STATUS "libraries: ${CUDA_LIBRARIES}")
message(STATUS "include path: ${CUDA_INCLUDE_Dirs}")

可以打印出

-- libraries: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/lib/x64/cudart.lib
-- include path: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/include

报错

CMake Error at C:/mrathena/develop/clion-2021.2.4/bin/cmake/win/share/cmake-3.20/Modules/CMakeDetermineCUDACompiler.cmake:174 (message):
  Failed to find nvcc.

  Compiler requires the CUDA toolkit.  Please set the CUDAToolkit_ROOT
  variable.
Call Stack (most recent call first):
  CMakeLists.txt:18 (enable_language)

找不到 nvcc,指定 CUDAToolkit_ROOT

https://www.jetbrains.com/help/clion/cuda-projects.html#set-nvcc

https://www.jetbrains.com/help/clion/2021.2/quick-tutorial-on-configuring-clion-on-windows.html#MSVC

https://visualstudio.microsoft.com/zh-hans/vs/older-downloads/

C:\mrathena\develop\clion-2021.2.4\bin\cmake\win\bin\cmake.exe -DCMAKE_BUILD_TYPE=Debug -DCMAKE_DEPENDS_USE_COMPILER=FALSE -G "CodeBlocks - NMake Makefiles" C:\mrathena\develop\workspace\clion\test
-- The C compiler identification is MSVC 19.29.30146.0
-- The CXX compiler identification is MSVC 19.29.30146.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - Failed
-- Check for working C compiler: C:/mrathena/develop/visual.studio-2019/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/mrathena/develop/visual.studio-2019/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - broken
CMake Error at C:/mrathena/develop/clion-2021.2.4/bin/cmake/win/share/cmake-3.20/Modules/CMakeTestCCompiler.cmake:66 (message):
  The C compiler

    "C:/mrathena/develop/visual.studio-2019/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/mrathena/develop/workspace/clion/test/cmake-build-debug/CMakeFiles/CMakeTmp
    
    Run Build Command(s):nmake -f Makefile /nologo cmTC_c73c3\fast && 	C:\mrathena\develop\visual.studio-2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\nmake.exe  -f CMakeFiles\cmTC_c73c3.dir\build.make /nologo -L                  CMakeFiles\cmTC_c73c3.dir\build
    Building C object CMakeFiles/cmTC_c73c3.dir/testCCompiler.c.obj
    	C:\mrathena\develop\clion-2021.2.4\bin\cmake\win\bin\cmake.exe -E cmake_cl_compile_depends --dep-file=CMakeFiles\cmTC_c73c3.dir\testCCompiler.c.obj.d --working-dir=C:\mrathena\develop\workspace\clion\test\cmake-build-debug\CMakeFiles\CMakeTmp --filter-prefix="" -- C:\mrathena\develop\visual.studio-2019\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\cl.exe @C:\Users\mrathena\AppData\Local\Temp\nm85F9.tmp
    Linking C executable cmTC_c73c3.exe
    	C:\mrathena\develop\clion-2021.2.4\bin\cmake\win\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\cmTC_c73c3.dir --rc=rc --mt=CMAKE_MT-NOTFOUND --manifests -- C:\mrathena\develop\visual.studio-2019\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\cmTC_c73c3.dir\objects1.rsp @C:\Users\mrathena\AppData\Local\Temp\nm8697.tmp
    RC Pass 1: command "rc /fo CMakeFiles\cmTC_c73c3.dir/manifest.res CMakeFiles\cmTC_c73c3.dir/manifest.rc" Failed (exit code 0) with the following output:
    系统找不到指定的文件。NMAKE : Fatal error U1077: “C:\mrathena\develop\clion-2021.2.4\bin\cmake\win\bin\cmake.exe”: 返回代码“0xffffffff”
    Stop.
    NMAKE : Fatal error U1077: “C:\mrathena\develop\visual.studio-2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\nmake.exe”: 返回代码“0x2”
    Stop.
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)


-- Configuring incomplete,errors occurred!
See also "C:/mrathena/develop/workspace/clion/test/cmake-build-debug/CMakeFiles/CMakeOutput.log".
See also "C:/mrathena/develop/workspace/clion/test/cmake-build-debug/CMakeFiles/CMakeError.log".

[Finished]