Skip to content

Commit 33d54ea

Browse files
zgjjazgjja
committed
[Refactor] 2 basic demos and all related documents
1 parent 3a40bbc commit 33d54ea

32 files changed

+1229
-980
lines changed

‎CMakeLists.txt

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
cmake_minimum_required(VERSION 3.14)
2+
3+
project(
4+
tensorrtx
5+
VERSION 0.1
6+
LANGUAGES C CXX CUDA)
7+
8+
set(TensorRT_7_8_10_TARGETS mlp lenet)
9+
10+
set(TensorRT_8_TARGETS)
11+
12+
set(TensorRT_10_TARGETS)
13+
14+
set(ALL_TARGETS ${TensorRT_7_8_10_TARGETS} ${TensorRT_8_TARGETS}
15+
${TensorRT_10_TARGETS})
16+
17+
foreach(sub_dir ${ALL_TARGETS})
18+
message(STATUS "Add subdirectory: ${sub_dir}")
19+
add_subdirectory(${sub_dir})
20+
endforeach()

‎README.md

Lines changed: 68 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ The basic workflow of TensorRTx is:
1616
## News
1717

1818
- `22 Oct 2024`. [lindsayshuo](https://github.com/lindsayshuo): YOLOv8-obb
19-
- `18 Oct 2024`. [zgjja](https://github.com/zgjja): Rafactor docker image.
19+
- `18 Oct 2024`. [zgjja](https://github.com/zgjja): Refactor docker image.
2020
- `11 Oct 2024`. [mpj1234](https://github.com/mpj1234): YOLO11
2121
- `9 Oct 2024`. [Phoenix8215](https://github.com/Phoenix8215): GhostNet V1 and V2.
2222
- `21 Aug 2024`. [Lemonononon](https://github.com/Lemonononon): real-esrgan-general-x4v3
@@ -38,7 +38,7 @@ The basic workflow of TensorRTx is:
3838
- [A guide for quickly getting started, taking lenet5 as a demo.](./tutorials/getting_started.md)
3939
- [The .wts file content format](./tutorials/getting_started.md#the-wts-content-format)
4040
- [Frequently Asked Questions (FAQ)](./tutorials/faq.md)
41-
- [Migrating from TensorRT 4 to 7](./tutorials/migrating_from_tensorrt_4_to_7.md)
41+
- [Migration Guide](./tutorials/migration_guide.md)
4242
- [How to implement multi-GPU processing, taking YOLOv4 as example](./tutorials/multi_GPU_processing.md)
4343
- [Check if Your GPU support FP16/INT8](./tutorials/check_fp16_int8_support.md)
4444
- [How to Compile and Run on Windows](./tutorials/run_on_windows.md)
@@ -47,21 +47,80 @@ The basic workflow of TensorRTx is:
4747

4848
## Test Environment
4949

50-
1. TensorRT 7.x
51-
2. TensorRT 8.x(Some of the models support 8.x)
50+
1. (**NOT recommended**) TensorRT 7.x
51+
2. (**Recommended**)TensorRT 8.x
52+
3. (**NOT recommended**) TensorRT 10.x
53+
54+
### Note
55+
56+
1. For history reason, some of the models are limited to specific TensorRT version, please check the README.md or code for the model you want to use.
57+
2. Currently, TensorRT 8.x has better compatibility and the most of the features supported.
5258

5359
## How to run
5460

55-
Each folder has a readme inside, which explains how to run the models inside.
61+
**Note**: this project support to build each network by the `CMakeLists.txt` in its subfolder, or you can build them together by the `CMakeLists.txt` on top of this project.
62+
63+
* General procedures before building and running:
64+
65+
```bash
66+
# 1. generate xxx.wts from https://github.com/wang-xinyu/pytorchx/tree/master/lenet
67+
# ...
68+
69+
# 2. put xxx.wts on top of this folder
70+
# ...
71+
```
72+
73+
* (*Option 1*) To build a single subproject in this project, do:
74+
75+
```bash
76+
## enter the subfolder
77+
cd tensorrtx/xxx
78+
79+
## configure & build
80+
cmake -S . -B build
81+
make -C build
82+
```
83+
84+
* (*Option 2*) To build many subprojects, firstly, in the top `CMakeLists.txt`, **uncomment** the project you don't want to build or not suppoted by your TensorRT version, e.g., you cannot build subprojects in `${TensorRT_8_Targets}` if your TensorRT is `7.x`. Then:
85+
86+
```bash
87+
## enter the top of this project
88+
cd tensorrtx
89+
90+
## configure & build
91+
# you may use "Ninja" rather than "make" to significantly boost the build speed
92+
cmake -G Ninja -S . -B build
93+
ninja -C build
94+
```
95+
96+
**WARNING**: This part is still under development, most subprojects are not adapted yet.
97+
98+
* run the generated executable, e.g.:
99+
100+
```bash
101+
# serialize model to plan file i.e. 'xxx.engine'
102+
build/xxx -s
103+
104+
# deserialize plan file and run inference
105+
build/xxx -d
106+
107+
# (Optional) check if the output is same as pytorchx/lenet
108+
# ...
109+
110+
# (Optional) customize the project
111+
# ...
112+
```
113+
114+
For more details, each subfolder may contain a `README.md` inside, which explains more.
56115

57116
## Models
58117

59118
Following models are implemented.
60119

61-
|Name | Description |
62-
|-|-|
63-
|[mlp](./mlp) | the very basic model for starters, properly documented |
64-
|[lenet](./lenet) | the simplest, as a "hello world" of this project |
120+
| Name | Description | Supported TensorRT Version |
121+
|---------------|---------------|---------------|
122+
|[mlp](./mlp) | the very basic model for starters, properly documented | 7.x/8.x/10.x |
123+
|[lenet](./lenet) | the simplest, as a "hello world" of this project | 7.x/8.x/10.x |
65124
|[alexnet](./alexnet)| easy to implement, all layers are supported in tensorrt |
66125
|[googlenet](./googlenet)| GoogLeNet (Inception v1) |
67126
|[inception](./inception)| Inception v3, v4 |

‎docker/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,11 @@ Change the `TAG` on top of the `.dockerfile`. Note: all images are officially ow
4949

5050
For more detail of the support matrix, please check [HERE](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html)
5151

52-
### How to customize opencv?
52+
### How to customize the opencv in the image?
5353

5454
If prebuilt package from apt cannot meet your requirements, please refer to the demo code in `.dockerfile` to build opencv from source.
5555

56-
### How to solve image build fail issues?
56+
### How to solve failiures when building image?
5757

5858
For *443 timeout* or any similar network issues, a proxy may required. To make your host proxy work for building env of docker, please change the `build` node inside docker-compose file like this:
5959
```YAML

‎docker/tensorrtx-docker-compose.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
services:
22
tensorrt:
3-
image: tensortx:1.0.0
3+
image: tensortx:1.0.1
44
container_name: tensortx
55
environment:
66
- NVIDIA_VISIBLE_DEVICES=all

‎docker/x86_64.dockerfile

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,16 @@ ENV DEBIAN_FRONTEND noninteractive
77
# basic tools
88
RUN apt update && apt-get install -y --fix-missing --no-install-recommends \
99
sudo wget curl git ca-certificates ninja-build tzdata pkg-config \
10-
gdb libglib2.0-dev libmount-dev \
10+
gdb libglib2.0-dev libmount-dev locales \
1111
&& rm -rf /var/lib/apt/lists/*
1212
RUN pip install --no-cache-dir yapf isort cmake-format pre-commit
1313

14+
## fix a potential pre-commit error
15+
RUN locale-gen "en_US.UTF-8"
16+
1417
## override older cmake
1518
RUN find /usr/local/share -type d -name "cmake-*" -exec rm -rf {} + \
16-
&& curl -fsSL "https://github.com/Kitware/CMake/releases/download/v3.29.0/cmake-3.29.0-linux-x86_64.sh" \
19+
&& curl -fsSL "https://github.com/Kitware/CMake/releases/download/v3.30.0/cmake-3.30.0-linux-x86_64.sh" \
1720
-o cmake.sh && bash cmake.sh --skip-license --exclude-subdir --prefix=/usr/local && rm cmake.sh
1821

1922
RUN apt update && apt-get install -y \

‎lenet/CMakeLists.txt

Lines changed: 43 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,43 @@
1-
cmake_minimum_required(VERSION 2.6)
2-
3-
project(lenet)
4-
5-
add_definitions(-std=c++11)
6-
7-
set(TARGET_NAME "lenet")
8-
9-
option(CUDA_USE_STATIC_CUDA_RUNTIME OFF)
10-
set(CMAKE_CXX_STANDARD 11)
11-
set(CMAKE_BUILD_TYPE Debug)
12-
13-
include_directories(${PROJECT_SOURCE_DIR}/include)
14-
# include and link dirs of cuda and tensorrt, you need adapt them if yours are different
15-
# cuda
16-
include_directories(/usr/local/cuda/include)
17-
link_directories(/usr/local/cuda/lib64)
18-
# tensorrt
19-
include_directories(/usr/include/x86_64-linux-gnu/)
20-
link_directories(/usr/lib/x86_64-linux-gnu/)
21-
22-
FILE(GLOB SRC_FILES ${PROJECT_SOURCE_DIR}/lenet.cpp ${PROJECT_SOURCE_DIR}/include/*.h)
23-
24-
add_executable(${TARGET_NAME} ${SRC_FILES})
25-
target_link_libraries(${TARGET_NAME} nvinfer)
26-
target_link_libraries(${TARGET_NAME} cudart)
27-
28-
add_definitions(-O2 -pthread)
29-
1+
cmake_minimum_required(VERSION 3.17.0)
2+
3+
project(
4+
lenet
5+
VERSION 0.1
6+
LANGUAGES C CXX CUDA)
7+
8+
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
9+
set(CMAKE_CUDA_ARCHITECTURES
10+
60
11+
70
12+
72
13+
75
14+
80
15+
86
16+
89)
17+
endif()
18+
19+
set(CMAKE_CXX_STANDARD 17)
20+
set(CMAKE_CXX_STANDARD_REQUIRED ON)
21+
set(CMAKE_CUDA_STANDARD 17)
22+
set(CMAKE_CUDA_STANDARD_REQUIRED ON)
23+
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
24+
set(CMAKE_INCLUDE_CURRENT_DIR TRUE)
25+
set(CMAKE_BUILD_TYPE
26+
"Debug"
27+
CACHE STRING "Build type for this project" FORCE)
28+
29+
option(CUDA_USE_STATIC_CUDA_RUNTIME "Use static cudaruntime library" OFF)
30+
31+
find_package(Threads REQUIRED)
32+
find_package(CUDAToolkit REQUIRED)
33+
34+
if(NOT TARGET TensorRT::TensorRT)
35+
include(FindTensorRT.cmake)
36+
else()
37+
message("TensorRT has been found, skipping for ${PROJECT_NAME}")
38+
endif()
39+
40+
add_executable(${PROJECT_NAME} lenet.cpp)
41+
42+
target_link_libraries(${PROJECT_NAME} PUBLIC Threads::Threads CUDA::cudart
43+
TensorRT::TensorRT)

‎lenet/FindTensorRT.cmake

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
cmake_minimum_required(VERSION 3.17.0)
2+
3+
set(TRT_VERSION
4+
$ENV{TRT_VERSION}
5+
CACHE STRING
6+
"TensorRT version, e.g. \"8.6.1.6\" or \"8.6.1.6+cuda12.0.1.011\"")
7+
8+
# find TensorRT include folder
9+
if(NOT TensorRT_INCLUDE_DIR)
10+
if(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
11+
set(TensorRT_INCLUDE_DIR
12+
"/usr/local/cuda/targets/aarch64-linux/include"
13+
CACHE PATH "TensorRT_INCLUDE_DIR")
14+
else()
15+
set(TensorRT_INCLUDE_DIR
16+
"/usr/include/x86_64-linux-gnu"
17+
CACHE PATH "TensorRT_INCLUDE_DIR")
18+
endif()
19+
message(STATUS "TensorRT: ${TensorRT_INCLUDE_DIR}")
20+
endif()
21+
22+
# find TensorRT library folder
23+
if(NOT TensorRT_LIBRARY_DIR)
24+
if(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
25+
set(TensorRT_LIBRARY_DIR
26+
"/usr/lib/aarch64-linux-gnu/tegra"
27+
CACHE PATH "TensorRT_LIBRARY_DIR")
28+
else()
29+
set(TensorRT_LIBRARY_DIR
30+
"/usr/include/x86_64-linux-gnu"
31+
CACHE PATH "TensorRT_LIBRARY_DIR")
32+
endif()
33+
message(STATUS "TensorRT: ${TensorRT_LIBRARY_DIR}")
34+
endif()
35+
36+
set(TensorRT_LIBRARIES)
37+
38+
message(STATUS "Found TensorRT lib: ${TensorRT_LIBRARIES}")
39+
40+
# process for different TensorRT version
41+
if(DEFINED TRT_VERSION AND NOT TRT_VERSION STREQUAL "")
42+
string(REGEX MATCH "([0-9]+)" _match ${TRT_VERSION})
43+
set(TRT_MAJOR_VERSION "${_match}")
44+
set(_modules nvinfer nvinfer_plugin)
45+
unset(_match)
46+
47+
if(TRT_MAJOR_VERSION GREATER_EQUAL 8)
48+
list(APPEND _modules nvinfer_vc_plugin nvinfer_dispatch nvinfer_lean)
49+
endif()
50+
else()
51+
message(FATAL_ERROR "Please set a environment variable \"TRT_VERSION\"")
52+
endif()
53+
54+
# find and add all modules of TensorRT into list
55+
foreach(lib IN LISTS _modules)
56+
find_library(
57+
TensorRT_${lib}_LIBRARY
58+
NAMES ${lib}
59+
HINTS ${TensorRT_LIBRARY_DIR})
60+
list(APPEND TensorRT_LIBRARIES ${TensorRT_${lib}_LIBRARY})
61+
endforeach()
62+
63+
# make the "TensorRT target"
64+
add_library(TensorRT IMPORTED INTERFACE)
65+
add_library(TensorRT::TensorRT ALIAS TensorRT)
66+
target_link_libraries(TensorRT INTERFACE ${TensorRT_LIBRARIES})
67+
68+
set_target_properties(
69+
TensorRT
70+
PROPERTIES C_STANDARD 17
71+
CXX_STANDARD 17
72+
POSITION_INDEPENDENT_CODE ON
73+
SKIP_BUILD_RPATH TRUE
74+
BUILD_WITH_INSTALL_RPATH TRUE
75+
INSTALL_RPATH "$\{ORIGIN\}"
76+
INTERFACE_INCLUDE_DIRECTORIES "${TensorRT_INCLUDE_DIR}")
77+
78+
unset(TRT_MAJOR_VERSION)
79+
unset(_modules)

‎lenet/README.md

Lines changed: 16 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,22 @@
11
# lenet5
22

3-
lenet5 is the simplest net in this tensorrtx project. You can learn the basic procedures of building tensorrt app from API. Including `define network`, `build engine`, `set output`, `do inference`, `serialize model to file`, `deserialize model from file`, etc.
3+
lenet5 is one of the simplest net in this repo. You can learn the basic procedures of building CNN from TensorRT API. This demo includes 2 major steps:
44

5-
## TensorRT C++ API
6-
7-
```
8-
// 1. generate lenet5.wts from https://github.com/wang-xinyu/pytorchx/tree/master/lenet
9-
10-
// 2. put lenet5.wts into tensorrtx/lenet
11-
12-
// 3. build and run
13-
14-
cd tensorrtx/lenet
15-
16-
mkdir build
5+
1. Build engine
6+
* define network
7+
* set input/output
8+
* serialize model to `.engine` file
9+
2. Do inference
10+
* load and deserialize model from `.engine` file
11+
* run inference
1712

18-
cd build
19-
20-
cmake ..
21-
22-
make
23-
24-
sudo ./lenet -s // serialize model to plan file i.e. 'lenet5.engine'
25-
26-
sudo ./lenet -d // deserialize plan file and run inference
13+
## TensorRT C++ API
2714

28-
// 4. see if the output is same as pytorchx/lenet
29-
```
15+
see [HERE](../README.md#how-to-run)
3016

3117
## TensorRT Python API
3218

33-
```
19+
```bash
3420
# 1. generate lenet5.wts from https://github.com/wang-xinyu/pytorchx/tree/master/lenet
3521

3622
# 2. put lenet5.wts into tensorrtx/lenet
@@ -39,9 +25,11 @@ sudo ./lenet -d // deserialize plan file and run inference
3925

4026
cd tensorrtx/lenet
4127

42-
python lenet.py -s # serialize model to plan file, i.e. 'lenet5.engine'
28+
# 4.1 serialize model to plan file, i.e. 'lenet5.engine'
29+
python lenet.py -s
4330

44-
python lenet.py -d # deserialize plan file and run inference
31+
# 4.2 deserialize plan file and run inference
32+
python lenet.py -d
4533

46-
# 4. see if the output is same as pytorchx/lenet
34+
# 5. (Optional) see if the output is same as pytorchx/lenet
4735
```

0 commit comments

Comments
 (0)