Tutorial - deploy YOLOv5 with ncnn

[nihui]

Tutorial - deploy YOLOv5 with ncnn

setup yolov5 pytorch

# checkout yolov5 v7.0 project
git clone https://github.com/ultralytics/yolov5
cd yolov5
git checkout v7.0

# install requirements
pip install -r requirements.txt --user

# download yolov5s weight
wget https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt

# test detection with pytorch weight, result saved to runs/detect/expN folder
python detect.py --source /home/nihui/nbs.jpg --weights yolov5s.pt --view-img

export torchscript and convert it to ncnn via pnnx

# export to torchscript, result saved to yolov5s.torchscript
python export.py --weights yolov5s.pt --include torchscript

# download latest pnnx from https://github.com/pnnx/pnnx/releases
wget https://github.com/pnnx/pnnx/releases/download/20230217/pnnx-20230217-ubuntu.zip
unzip pnnx-20230217-ubuntu.zip

# convert torchscript to pnnx and ncnn, result saved to yolov5s.ncnn.param yolov5s.ncnn.bin
./pnnx-20230217-ubuntu/pnnx yolov5s.torchscript inputshape=[1,3,640,640]

ncnn inference

// load ncnn model, auto-enable vulkan acceleration
ncnn::Net yolov5;
yolov5.opt.use_vulkan_compute = true;
yolov5.load_param("yolov5s.ncnn.param");
yolov5.load_model("yolov5s.ncnn.bin");

const int target_size = 640;
const float prob_threshold = 0.25f;
const float nms_threshold = 0.45f;

// load image, resize and pad to 640x640
cv::Mat bgr = cv::imread("/home/nihui/nbs.jpg", CV_LOAD_IMAGE_COLOR);
const int img_w = bgr.cols;
const int img_h = bgr.rows;

// solve resize scale
int w = img_w;
int h = img_h;
float scale = 1.f;
if (w > h)
{
    scale = (float)target_size / w;
    w = target_size;
    h = h * scale;
}
else
{
    scale = (float)target_size / h;
    h = target_size;
    w = w * scale;
}

// construct ncnn::Mat from image pixel data, swap order from bgr to rgb
ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR2RGB, img_w, img_h, w, h);

// pad to target_size rectangle
const int wpad = target_size - w;
const int hpad = target_size - h;
ncnn::Mat in_pad;
ncnn::copy_make_border(in, in_pad, hpad / 2, hpad - hpad / 2, wpad / 2, wpad - wpad / 2, ncnn::BORDER_CONSTANT, 114.f);

// apply yolov5 pre process, that is to normalize 0~255 to 0~1
const float norm_vals[3] = { 1 / 255.f, 1 / 255.f, 1 / 255.f };
in_pad.substract_mean_normalize(0, norm_vals);

// yolov5 model inference
ncnn::Extractor ex = yolov5.create_extractor();
ex.input("in0", in_pad);
ncnn::Mat out;
ex.extract("out0", out);

// the out blob would be a 2-dim tensor with w=85 h=25200
//
//        |cx|cy|bw|bh|box score(1)| per-class scores(80) |
//        +--+--+--+--+------------+----------------------+
//        |53|50|70|80|    0.11    |0.1 0.0 0.0 0.5 ......|
//   all /|  |  |  |  |      .     |           .          |
//  boxes |46|40|38|44|    0.95    |0.0 0.9 0.0 0.0 ......|
// (25200)|  |  |  |  |      .     |           .          |
//       \|  |  |  |  |      .     |           .          |
//        +--+--+--+--+------------+----------------------+
//

implement yolov5 post-process in cpp

struct Object
{
    cv::Rect_<float> rect;
    int label;
    float prob;
};

// enumerate all boxes
std::vector<Object> proposals;
for (int i = 0; i < out.h; i++)
{
    const float* ptr = out.row(i);

    const int num_class = 80;

    const float cx = ptr[0];
    const float cy = ptr[1];
    const float bw = ptr[2];
    const float bh = ptr[3];
    const float box_score = ptr[4];
    const float* class_scores = ptr + 5;

    // find class index with the biggest class score among all classes
    int class_index = 0;
    float class_score = -FLT_MAX;
    for (int j = 0; j < num_class; j++)
    {
        if (class_scores[j] > class_score)
        {
            class_score = class_scores[j];
            class_index = j;
        }
    }

    // combined score = box score * class score
    float confidence = box_score * class_score;

    // filter candidate boxes with combined score >= prob_threshold
    if (confidence < prob_threshold)
        continue;

    // transform candidate box (center-x,center-y,w,h) to (x0,y0,x1,y1)
    float x0 = cx - bw * 0.5f;
    float y0 = cy - bh * 0.5f;
    float x1 = cx + bw * 0.5f;
    float y1 = cy + bh * 0.5f;

    // collect candidates
    Object obj;
    obj.rect.x = x0;
    obj.rect.y = y0;
    obj.rect.width = x1 - x0;
    obj.rect.height = y1 - y0;
    obj.label = class_index;
    obj.prob = confidence;

    proposals.push_back(obj);
}

// sort all candidates by score from highest to lowest
qsort_descent_inplace(proposals);

// apply non max suppression
std::vector<int> picked;
nms_sorted_bboxes(proposals, picked, nms_threshold);

// collect final result after nms
std::vector<Object> objects;
const int count = picked.size();
objects.resize(count);
for (int i = 0; i < count; i++)
{
    objects[i] = proposals[picked[i]];

    // adjust offset to original unpadded
    float x0 = (objects[i].rect.x - (wpad / 2)) / scale;
    float y0 = (objects[i].rect.y - (hpad / 2)) / scale;
    float x1 = (objects[i].rect.x + objects[i].rect.width - (wpad / 2)) / scale;
    float y1 = (objects[i].rect.y + objects[i].rect.height - (hpad / 2)) / scale;

    // clip
    x0 = std::max(std::min(x0, (float)(img_w - 1)), 0.f);
    y0 = std::max(std::min(y0, (float)(img_h - 1)), 0.f);
    x1 = std::max(std::min(x1, (float)(img_w - 1)), 0.f);
    y1 = std::max(std::min(y1, (float)(img_h - 1)), 0.f);

    objects[i].rect.x = x0;
    objects[i].rect.y = y0;
    objects[i].rect.width = x1 - x0;
    objects[i].rect.height = y1 - y0;
}

// draw result on image
draw_objects(bgr, objects);

dynamic shape inference (optional but recommended for faster inference)

• Eliminate unnecessary calculations on image padding
  For non-square images, such as 640x480, you need to pad them to 640x640 first, and use 640x640 as the input size for inference. In fact, 1/4 of the areas are black parts, and these areas will also bring additional calculations.
  Dynamic shape inference can eliminate or minimize the calculation caused by padding.

• Eliminate calculations on candidate boxes with low scores
  In post-processing, both score and candidate box position generation require sigmoid operation, which is not very fast. Most of the anchor box scores are below the threshold. We can optimize the generation logic of the candidate box and calculate the offset and size of the candidate box only when the threshold is met.

• Reduce memory layout conversion and access overhead in post-processing
  Post-processing has some memory layout operators such as reshape, transpose, tensor split, concat, etc. These operators will cause more memory access overhead. We can write cpp code to implement the end-to-end post-processing main loop, avoiding these memory layout conversion operations as much as possible.

// extract intermediate blob before anchor box generation instead of the very last "out0"
// the three blob names can be easily discovered in netron visualizer
// they are the outputs of the last three convolution, before any memory layout transformation
// each out blob is the feat for a grid scale stride(8, 16, 32)
ncnn::Mat out0;
ncnn::Mat out1;
ncnn::Mat out2;
ex.extract("187", out0);
ex.extract("202", out1);
ex.extract("218", out2);

// we need to generate anchor box ourself
// the anchor setting is from yolov5/models/yolov5s.yaml
// there are three anchor types for different box size ratio

// stride 8           | // stride 16          | // stride 32
ncnn::Mat anchors(6); | ncnn::Mat anchors(6); | ncnn::Mat anchors(6);
anchors[0] = 10.f;    | anchors[0] = 30.f;    | anchors[0] = 116.f;
anchors[1] = 13.f;    | anchors[1] = 61.f;    | anchors[1] = 90.f;
anchors[2] = 16.f;    | anchors[2] = 62.f;    | anchors[2] = 156.f;
anchors[3] = 30.f;    | anchors[3] = 45.f;    | anchors[3] = 198.f;
anchors[4] = 33.f;    | anchors[4] = 59.f;    | anchors[4] = 373.f;
anchors[5] = 23.f;    | anchors[5] = 119.f;   | anchors[5] = 326.f;

// the out blob would be a 3-dim tensor with w=dynamic h=dynamic c=255=85*3
// we view it as [num_grid_x,num_grid_y,85,3] for 3 anchor ratio types

//
//            |<--   dynamic anchor grids     -->|
//            |   larger image yields more grids |
//            +-------------------------- // ----+
//           /| center-x                         |
//          / | center-y                         |
//         /  | box-w                            |
// anchor-0   | box-h                            |
//  +-----+   | box score(1)                     |
//  |     |   +----------------                  |
//  |     |   | per-class scores(80)             |
//  +-----+\  |   .                              |
//          \ |   .                              |
//           \|   .                              |
//            +-------------------------- // ----+
//           /| center-x                         |
//          / | center-y                         |
//         /  | box-w                            |
// anchor-1   | box-h                            |
//  +-----+   | box score(1)                     |
//  |     |   +----------------                  |
//  +-----+   | per-class scores(80)             |
//         \  |   .                              |
//          \ |   .                              |
//           \|   .                              |
//            +-------------------------- // ----+
//           /| center-x                         |
//          / | center-y                         |
//         /  | box-w                            |
// anchor-2   | box-h                            |
//  +--+      | box score(1)                     |
//  |  |      +----------------                  |
//  |  |      | per-class scores(80)             |
//  +--+   \  |   .                              |
//          \ |   .                              |
//           \|   .                              |
//            +-------------------------- // ----+
//

const int num_anchors = 3;
for (int q = 0; q < num_anchors; q++)
{
    for (int i = 0; i < num_grid_y; i++)
    {
        for (int j = 0; j < num_grid_x; j++)
        {
            // code body for determining if the anchor box at grid(i,j) and ratio box(q)
            // should be considered as the detected object
        }
    }
}

// the code body is mostly the same as static shape post-process
// in addition to adding the necessary sigmoid operations
// and calculating the box coordinates corresponding to the image from the anchor grid(i,j) + ratio box(q)

// find class index with max class score
int class_index = 0;
float class_score = -FLT_MAX;
for (int k = 0; k < num_class; k++)
{
    float score = feat_blob.channel(q * feat_offset + 5 + k).row(i)[j];
    if (score > class_score)
    {
        class_index = k;
        class_score = score;
    }
}

float box_score = feat_blob.channel(q * feat_offset + 4).row(i)[j];

// combined score = box score * class score
// apply sigmoid first to get normed 0~1 value
float confidence = sigmoid(box_score) * sigmoid(class_score); // <--- UPDATE

// filter candidate boxes with combined score >= prob_threshold
if (confidence < prob_threshold)
    continue;

// yolov5/models/yolo.py Detect forward
// y = x[i].sigmoid()
// y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i].to(x[i].device)) * self.stride[i]  # xy
// y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh

float dx = sigmoid(feat_blob.channel(q * feat_offset + 0).row(i)[j]); // <--- NEW
float dy = sigmoid(feat_blob.channel(q * feat_offset + 1).row(i)[j]); // <--- NEW
float dw = sigmoid(feat_blob.channel(q * feat_offset + 2).row(i)[j]); // <--- NEW
float dh = sigmoid(feat_blob.channel(q * feat_offset + 3).row(i)[j]); // <--- NEW

const float anchor_w = anchors[q * 2];     // <--- NEW
const float anchor_h = anchors[q * 2 + 1]; // <--- NEW

float cx = (dx * 2.f - 0.5f + j) * stride; // <--- NEW
float cy = (dy * 2.f - 0.5f + i) * stride; // <--- NEW
float bw = pow(dw * 2.f, 2) * anchor_w; // <--- NEW
float bh = pow(dh * 2.f, 2) * anchor_h; // <--- NEW

// transform candidate box (center-x,center-y,w,h) to (x0,y0,x1,y1)
float x0 = cx - bw * 0.5f;
float y0 = cy - bh * 0.5f;
float x1 = cx + bw * 0.5f;
float y1 = cy + bh * 0.5f;

// collect candidates
Object obj;
obj.rect.x = x0;
obj.rect.y = y0;
obj.rect.width = x1 - x0;
obj.rect.height = y1 - y0;
obj.label = class_index;
obj.prob = confidence;

objects.push_back(obj);

full cmake project and cpp source

project(yolov5)
# TODO change me to your ncnn installation path
set(ncnn_DIR "/home/nihui/osd/ncnn/build/install/lib64/cmake/ncnn")
find_package(ncnn REQUIRED)

add_executable(yolov5 yolov5.cpp)
target_link_libraries(yolov5 ncnn)

// Tencent is pleased to support the open source community by making ncnn available.
//
// Copyright (C) 2022 THL A29 Limited, a Tencent company. All rights reserved.
//
// Licensed under the BSD 3-Clause License (the "License"); you may not use this file except
// in compliance with the License. You may obtain a copy of the License at
//
// https://opensource.org/licenses/BSD-3-Clause
//
// Unless required by applicable law or agreed to in writing, software distributed
// under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.

#include "net.h"

#if defined(USE_NCNN_SIMPLEOCV)
#include "simpleocv.h"
#else
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#endif
#include <float.h>
#include <stdio.h>
#include <vector>

struct Object
{
    cv::Rect_<float> rect;
    int label;
    float prob;
};

static inline float intersection_area(const Object& a, const Object& b)
{
    cv::Rect_<float> inter = a.rect & b.rect;
    return inter.area();
}

static void qsort_descent_inplace(std::vector<Object>& faceobjects, int left, int right)
{
    int i = left;
    int j = right;
    float p = faceobjects[(left + right) / 2].prob;

    while (i <= j)
    {
        while (faceobjects[i].prob > p)
            i++;

        while (faceobjects[j].prob < p)
            j--;

        if (i <= j)
        {
            // swap
            std::swap(faceobjects[i], faceobjects[j]);

            i++;
            j--;
        }
    }

    #pragma omp parallel sections
    {
        #pragma omp section
        {
            if (left < j) qsort_descent_inplace(faceobjects, left, j);
        }
        #pragma omp section
        {
            if (i < right) qsort_descent_inplace(faceobjects, i, right);
        }
    }
}

static void qsort_descent_inplace(std::vector<Object>& faceobjects)
{
    if (faceobjects.empty())
        return;

    qsort_descent_inplace(faceobjects, 0, faceobjects.size() - 1);
}

static void nms_sorted_bboxes(const std::vector<Object>& faceobjects, std::vector<int>& picked, float nms_threshold, bool agnostic = false)
{
    picked.clear();

    const int n = faceobjects.size();

    std::vector<float> areas(n);
    for (int i = 0; i < n; i++)
    {
        areas[i] = faceobjects[i].rect.area();
    }

    for (int i = 0; i < n; i++)
    {
        const Object& a = faceobjects[i];

        int keep = 1;
        for (int j = 0; j < (int)picked.size(); j++)
        {
            const Object& b = faceobjects[picked[j]];

            if (!agnostic && a.label != b.label)
                continue;

            // intersection over union
            float inter_area = intersection_area(a, b);
            float union_area = areas[i] + areas[picked[j]] - inter_area;
            // float IoU = inter_area / union_area
            if (inter_area / union_area > nms_threshold)
                keep = 0;
        }

        if (keep)
            picked.push_back(i);
    }
}

static int detect_yolov5(const cv::Mat& bgr, std::vector<Object>& objects)
{
    // load ncnn model
    ncnn::Net yolov5;

    yolov5.opt.use_vulkan_compute = true;

    yolov5.load_param("yolov5s.ncnn.param");
    yolov5.load_model("yolov5s.ncnn.bin");

    const int target_size = 640;
    const float prob_threshold = 0.25f;
    const float nms_threshold = 0.45f;

    // load image, resize and pad to 640x640
    const int img_w = bgr.cols;
    const int img_h = bgr.rows;

    // solve resize scale
    int w = img_w;
    int h = img_h;
    float scale = 1.f;
    if (w > h)
    {
        scale = (float)target_size / w;
        w = target_size;
        h = h * scale;
    }
    else
    {
        scale = (float)target_size / h;
        h = target_size;
        w = w * scale;
    }

    // construct ncnn::Mat from image pixel data, swap order from bgr to rgb
    ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR2RGB, img_w, img_h, w, h);

    // pad to target_size rectangle
    const int wpad = target_size - w;
    const int hpad = target_size - h;
    ncnn::Mat in_pad;
    ncnn::copy_make_border(in, in_pad, hpad / 2, hpad - hpad / 2, wpad / 2, wpad - wpad / 2, ncnn::BORDER_CONSTANT, 114.f);

    // apply yolov5 pre process, that is to normalize 0~255 to 0~1
    const float norm_vals[3] = { 1 / 255.f, 1 / 255.f, 1 / 255.f };
    in_pad.substract_mean_normalize(0, norm_vals);

    // yolov5 model inference
    ncnn::Extractor ex = yolov5.create_extractor();
    ex.input("in0", in_pad);
    ncnn::Mat out;
    ex.extract("out0", out);

    // the out blob would be a 2-dim tensor with w=85 h=25200
    //
    //        |cx|cy|bw|bh|box score(1)| per-class scores(80) |
    //        +--+--+--+--+------------+----------------------+
    //        |53|50|70|80|    0.11    |0.1 0.0 0.0 0.5 ......|
    //   all /|  |  |  |  |      .     |           .          |
    //  boxes |46|40|38|44|    0.95    |0.0 0.9 0.0 0.0 ......|
    // (25200)|  |  |  |  |      .     |           .          |
    //       \|  |  |  |  |      .     |           .          |
    //        +--+--+--+--+------------+----------------------+
    //

    // enumerate all boxes
    std::vector<Object> proposals;
    for (int i = 0; i < out.h; i++)
    {
        const float* ptr = out.row(i);

        const int num_class = 80;

        const float cx = ptr[0];
        const float cy = ptr[1];
        const float bw = ptr[2];
        const float bh = ptr[3];
        const float box_score = ptr[4];
        const float* class_scores = ptr + 5;

        // find class index with the biggest class score among all classes
        int class_index = 0;
        float class_score = -FLT_MAX;
        for (int j = 0; j < num_class; j++)
        {
            if (class_scores[j] > class_score)
            {
                class_score = class_scores[j];
                class_index = j;
            }
        }

        // combined score = box score * class score
        float confidence = box_score * class_score;

        // filter candidate boxes with combined score >= prob_threshold
        if (confidence < prob_threshold)
            continue;

        // transform candidate box (center-x,center-y,w,h) to (x0,y0,x1,y1)
        float x0 = cx - bw * 0.5f;
        float y0 = cy - bh * 0.5f;
        float x1 = cx + bw * 0.5f;
        float y1 = cy + bh * 0.5f;

        // collect candidates
        Object obj;
        obj.rect.x = x0;
        obj.rect.y = y0;
        obj.rect.width = x1 - x0;
        obj.rect.height = y1 - y0;
        obj.label = class_index;
        obj.prob = confidence;

        proposals.push_back(obj);
    }

    // sort all candidates by score from highest to lowest
    qsort_descent_inplace(proposals);

    // apply non max suppression
    std::vector<int> picked;
    nms_sorted_bboxes(proposals, picked, nms_threshold);

    // collect final result after nms
    const int count = picked.size();
    objects.resize(count);
    for (int i = 0; i < count; i++)
    {
        objects[i] = proposals[picked[i]];

        // adjust offset to original unpadded
        float x0 = (objects[i].rect.x - (wpad / 2)) / scale;
        float y0 = (objects[i].rect.y - (hpad / 2)) / scale;
        float x1 = (objects[i].rect.x + objects[i].rect.width - (wpad / 2)) / scale;
        float y1 = (objects[i].rect.y + objects[i].rect.height - (hpad / 2)) / scale;

        // clip
        x0 = std::max(std::min(x0, (float)(img_w - 1)), 0.f);
        y0 = std::max(std::min(y0, (float)(img_h - 1)), 0.f);
        x1 = std::max(std::min(x1, (float)(img_w - 1)), 0.f);
        y1 = std::max(std::min(y1, (float)(img_h - 1)), 0.f);

        objects[i].rect.x = x0;
        objects[i].rect.y = y0;
        objects[i].rect.width = x1 - x0;
        objects[i].rect.height = y1 - y0;
    }

    return 0;
}

static inline float sigmoid(float x)
{
    return static_cast<float>(1.f / (1.f + exp(-x)));
}

static void generate_proposals(const ncnn::Mat& anchors, int stride, const ncnn::Mat& in_pad, const ncnn::Mat& feat_blob, float prob_threshold, std::vector<Object>& objects)
{
    // the out blob would be a 3-dim tensor with w=dynamic h=dynamic c=255=85*3
    // we view it as [grid_w,grid_h,85,3] for 3 anchor ratio types

    //
    //            |<--   dynamic anchor grids     -->|
    //            |   larger image yields more grids |
    //            +-------------------------- // ----+
    //           /| center-x                         |
    //          / | center-y                         |
    //         /  | box-w                            |
    // anchor-0   | box-h                            |
    //  +-----+   | box score(1)                     |
    //  |     |   +----------------                  |
    //  |     |   | per-class scores(80)             |
    //  +-----+\  |   .                              |
    //          \ |   .                              |
    //           \|   .                              |
    //            +-------------------------- // ----+
    //           /| center-x                         |
    //          / | center-y                         |
    //         /  | box-w                            |
    // anchor-1   | box-h                            |
    //  +-----+   | box score(1)                     |
    //  |     |   +----------------                  |
    //  +-----+   | per-class scores(80)             |
    //         \  |   .                              |
    //          \ |   .                              |
    //           \|   .                              |
    //            +-------------------------- // ----+
    //           /| center-x                         |
    //          / | center-y                         |
    //         /  | box-w                            |
    // anchor-2   | box-h                            |
    //  +--+      | box score(1)                     |
    //  |  |      +----------------                  |
    //  |  |      | per-class scores(80)             |
    //  +--+   \  |   .                              |
    //          \ |   .                              |
    //           \|   .                              |
    //            +-------------------------- // ----+
    //

    const int num_grid_x = feat_blob.w;
    const int num_grid_y = feat_blob.h;

    const int num_anchors = anchors.w / 2;

    const int num_class = feat_blob.c / num_anchors - 5;

    const int feat_offset = num_class + 5;

    // enumerate all anchor types
    for (int q = 0; q < num_anchors; q++)
    {
        const float anchor_w = anchors[q * 2];
        const float anchor_h = anchors[q * 2 + 1];

        for (int i = 0; i < num_grid_y; i++)
        {
            for (int j = 0; j < num_grid_x; j++)
            {
                // find class index with max class score
                int class_index = 0;
                float class_score = -FLT_MAX;
                for (int k = 0; k < num_class; k++)
                {
                    float score = feat_blob.channel(q * feat_offset + 5 + k).row(i)[j];
                    if (score > class_score)
                    {
                        class_index = k;
                        class_score = score;
                    }
                }

                float box_score = feat_blob.channel(q * feat_offset + 4).row(i)[j];

                // combined score = box score * class score
                // apply sigmoid first to get normed 0~1 value
                float confidence = sigmoid(box_score) * sigmoid(class_score);

                // filter candidate boxes with combined score >= prob_threshold
                if (confidence < prob_threshold)
                    continue;

                // yolov5/models/yolo.py Detect forward
                // y = x[i].sigmoid()
                // y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i].to(x[i].device)) * self.stride[i]  # xy
                // y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh

                float dx = sigmoid(feat_blob.channel(q * feat_offset + 0).row(i)[j]);
                float dy = sigmoid(feat_blob.channel(q * feat_offset + 1).row(i)[j]);
                float dw = sigmoid(feat_blob.channel(q * feat_offset + 2).row(i)[j]);
                float dh = sigmoid(feat_blob.channel(q * feat_offset + 3).row(i)[j]);

                float cx = (dx * 2.f - 0.5f + j) * stride;
                float cy = (dy * 2.f - 0.5f + i) * stride;

                float bw = pow(dw * 2.f, 2) * anchor_w;
                float bh = pow(dh * 2.f, 2) * anchor_h;

                // transform candidate box (center-x,center-y,w,h) to (x0,y0,x1,y1)
                float x0 = cx - bw * 0.5f;
                float y0 = cy - bh * 0.5f;
                float x1 = cx + bw * 0.5f;
                float y1 = cy + bh * 0.5f;

                // collect candidates
                Object obj;
                obj.rect.x = x0;
                obj.rect.y = y0;
                obj.rect.width = x1 - x0;
                obj.rect.height = y1 - y0;
                obj.label = class_index;
                obj.prob = confidence;

                objects.push_back(obj);
            }
        }
    }
}

static int detect_yolov5_dynamic(const cv::Mat& bgr, std::vector<Object>& objects)
{
    // load ncnn model
    ncnn::Net yolov5;

    yolov5.opt.use_vulkan_compute = true;

    yolov5.load_param("yolov5s.ncnn.param");
    yolov5.load_model("yolov5s.ncnn.bin");

    const int target_size = 640;
    const float prob_threshold = 0.25f;
    const float nms_threshold = 0.45f;

    // load image, resize and letterbox pad to multiple of max_stride
    const int img_w = bgr.cols;
    const int img_h = bgr.rows;
    const int max_stride = 64;

    // solve resize scale
    int w = img_w;
    int h = img_h;
    float scale = 1.f;
    if (w > h)
    {
        scale = (float)target_size / w;
        w = target_size;
        h = h * scale;
    }
    else
    {
        scale = (float)target_size / h;
        h = target_size;
        w = w * scale;
    }

    // construct ncnn::Mat from image pixel data, swap order from bgr to rgb
    ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR2RGB, img_w, img_h, w, h);

    // pad to target_size rectangle
    const int wpad = (w + max_stride - 1) / max_stride * max_stride - w;
    const int hpad = (h + max_stride - 1) / max_stride * max_stride - h;
    ncnn::Mat in_pad;
    ncnn::copy_make_border(in, in_pad, hpad / 2, hpad - hpad / 2, wpad / 2, wpad - wpad / 2, ncnn::BORDER_CONSTANT, 114.f);

    // apply yolov5 pre process, that is to normalize 0~255 to 0~1
    const float norm_vals[3] = { 1 / 255.f, 1 / 255.f, 1 / 255.f };
    in_pad.substract_mean_normalize(0, norm_vals);

    // yolov5 model inference
    ncnn::Extractor ex = yolov5.create_extractor();
    ex.input("in0", in_pad);

    ncnn::Mat out0;
    ncnn::Mat out1;
    ncnn::Mat out2;
    ex.extract("187", out0);
    ex.extract("202", out1);
    ex.extract("218", out2);

    std::vector<Object> proposals;

    // anchor setting from yolov5/models/yolov5s.yaml

    // stride 8
    {
        ncnn::Mat anchors(6);
        anchors[0] = 10.f;
        anchors[1] = 13.f;
        anchors[2] = 16.f;
        anchors[3] = 30.f;
        anchors[4] = 33.f;
        anchors[5] = 23.f;

        std::vector<Object> objects8;
        generate_proposals(anchors, 8, in_pad, out0, prob_threshold, objects8);

        proposals.insert(proposals.end(), objects8.begin(), objects8.end());
    }

    // stride 16
    {
        ncnn::Mat anchors(6);
        anchors[0] = 30.f;
        anchors[1] = 61.f;
        anchors[2] = 62.f;
        anchors[3] = 45.f;
        anchors[4] = 59.f;
        anchors[5] = 119.f;

        std::vector<Object> objects16;
        generate_proposals(anchors, 16, in_pad, out1, prob_threshold, objects16);

        proposals.insert(proposals.end(), objects16.begin(), objects16.end());
    }

    // stride 32
    {
        ncnn::Mat anchors(6);
        anchors[0] = 116.f;
        anchors[1] = 90.f;
        anchors[2] = 156.f;
        anchors[3] = 198.f;
        anchors[4] = 373.f;
        anchors[5] = 326.f;

        std::vector<Object> objects32;
        generate_proposals(anchors, 32, in_pad, out2, prob_threshold, objects32);

        proposals.insert(proposals.end(), objects32.begin(), objects32.end());
    }

    // sort all candidates by score from highest to lowest
    qsort_descent_inplace(proposals);

    // apply non max suppression
    std::vector<int> picked;
    nms_sorted_bboxes(proposals, picked, nms_threshold);

    // collect final result after nms
    const int count = picked.size();
    objects.resize(count);
    for (int i = 0; i < count; i++)
    {
        objects[i] = proposals[picked[i]];

        // adjust offset to original unpadded
        float x0 = (objects[i].rect.x - (wpad / 2)) / scale;
        float y0 = (objects[i].rect.y - (hpad / 2)) / scale;
        float x1 = (objects[i].rect.x + objects[i].rect.width - (wpad / 2)) / scale;
        float y1 = (objects[i].rect.y + objects[i].rect.height - (hpad / 2)) / scale;

        // clip
        x0 = std::max(std::min(x0, (float)(img_w - 1)), 0.f);
        y0 = std::max(std::min(y0, (float)(img_h - 1)), 0.f);
        x1 = std::max(std::min(x1, (float)(img_w - 1)), 0.f);
        y1 = std::max(std::min(y1, (float)(img_h - 1)), 0.f);

        objects[i].rect.x = x0;
        objects[i].rect.y = y0;
        objects[i].rect.width = x1 - x0;
        objects[i].rect.height = y1 - y0;
    }

    return 0;
}

static void draw_objects(const cv::Mat& bgr, const std::vector<Object>& objects)
{
    static const char* class_names[] = {
        "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light",
        "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow",
        "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee",
        "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard",
        "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple",
        "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch",
        "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone",
        "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear",
        "hair drier", "toothbrush"
    };

    cv::Mat image = bgr.clone();

    for (size_t i = 0; i < objects.size(); i++)
    {
        const Object& obj = objects[i];

        fprintf(stderr, "%d = %.5f at %.2f %.2f %.2f x %.2f\n", obj.label, obj.prob,
                obj.rect.x, obj.rect.y, obj.rect.width, obj.rect.height);

        cv::rectangle(image, obj.rect, cv::Scalar(255, 0, 0));

        char text[256];
        sprintf(text, "%s %.1f%%", class_names[obj.label], obj.prob * 100);

        int baseLine = 0;
        cv::Size label_size = cv::getTextSize(text, cv::FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseLine);

        int x = obj.rect.x;
        int y = obj.rect.y - label_size.height - baseLine;
        if (y < 0)
            y = 0;
        if (x + label_size.width > image.cols)
            x = image.cols - label_size.width;

        cv::rectangle(image, cv::Rect(cv::Point(x, y), cv::Size(label_size.width, label_size.height + baseLine)),
                      cv::Scalar(255, 255, 255), -1);

        cv::putText(image, text, cv::Point(x, y + label_size.height),
                    cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
    }

    cv::imshow("image", image);
    cv::waitKey(0);
}

int main(int argc, char** argv)
{
    if (argc != 2)
    {
        fprintf(stderr, "Usage: %s [imagepath]\n", argv[0]);
        return -1;
    }

    const char* imagepath = argv[1];

    cv::Mat m = cv::imread(imagepath, 1);
    if (m.empty())
    {
        fprintf(stderr, "cv::imread %s failed\n", imagepath);
        return -1;
    }

    std::vector<Object> objects;
    // detect_yolov5(m, objects);
    detect_yolov5_dynamic(m, objects);

    draw_objects(m, objects);

    return 0;
}

[YFforever2022]

yolov5-v7.0
python export.py --weights yolov5s.pt --include torchscript
使用命令行pnnx yolov5s.torchscript inputshape=[1,3,640,640]
得到了yolov5s.ncnn.param，它的内容是
7767517
210 240
Input in0 0 1 in0
Convolution conv_3 1 1 in0 1 0=32 1=6 11=6 12=1 13=2 14=2 2=1 3=2 4=2 5=1 6=3456
Swish silu_66 1 1 1 2
Convolution conv_4 1 1 2 3 0=64 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=18432
Swish silu_67 1 1 3 4
Split splitncnn_0 1 2 4 5 6
Convolution conv_5 1 1 6 7 0=32 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=2048
Swish silu_68 1 1 7 8
Split splitncnn_1 1 2 8 9 10
Convolution conv_6 1 1 10 11 0=32 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=1024
Swish silu_69 1 1 11 12
Convolution conv_7 1 1 12 13 0=32 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=9216
Swish silu_70 1 1 13 14
BinaryOp add_0 2 1 9 14 15 0=0
Convolution conv_8 1 1 5 16 0=32 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=2048
Swish silu_71 1 1 16 17
Concat cat_0 2 1 15 17 18 0=0
Convolution conv_9 1 1 18 19 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=4096
Swish silu_72 1 1 19 20
Convolution conv_10 1 1 20 21 0=128 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=73728
Swish silu_73 1 1 21 22
Split splitncnn_2 1 2 22 23 24
Convolution conv_11 1 1 24 25 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=8192
Swish silu_74 1 1 25 26
Split splitncnn_3 1 2 26 27 28
Convolution conv_12 1 1 28 29 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=4096
Swish silu_75 1 1 29 30
Convolution conv_13 1 1 30 31 0=64 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=36864
Swish silu_76 1 1 31 32
BinaryOp add_1 2 1 27 32 33 0=0
Split splitncnn_4 1 2 33 34 35
Convolution conv_14 1 1 35 36 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=4096
Swish silu_77 1 1 36 37
Convolution conv_15 1 1 37 38 0=64 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=36864
Swish silu_78 1 1 38 39
BinaryOp add_2 2 1 34 39 40 0=0
Convolution conv_16 1 1 23 41 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=8192
Swish silu_79 1 1 41 42
Concat cat_1 2 1 40 42 43 0=0
Convolution conv_17 1 1 43 44 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_80 1 1 44 45
Split splitncnn_5 1 2 45 46 47
Convolution conv_18 1 1 47 48 0=256 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=294912
Swish silu_81 1 1 48 49
Split splitncnn_6 1 2 49 50 51
Convolution conv_19 1 1 51 52 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=32768
Swish silu_82 1 1 52 53
Split splitncnn_7 1 2 53 54 55
Convolution conv_20 1 1 55 56 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_83 1 1 56 57
Convolution conv_21 1 1 57 58 0=128 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=147456
Swish silu_84 1 1 58 59
BinaryOp add_3 2 1 54 59 60 0=0
Split splitncnn_8 1 2 60 61 62
Convolution conv_22 1 1 62 63 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_85 1 1 63 64
Convolution conv_23 1 1 64 65 0=128 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=147456
Swish silu_86 1 1 65 66
BinaryOp add_4 2 1 61 66 67 0=0
Split splitncnn_9 1 2 67 68 69
Convolution conv_24 1 1 69 70 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_87 1 1 70 71
Convolution conv_25 1 1 71 72 0=128 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=147456
Swish silu_88 1 1 72 73
BinaryOp add_5 2 1 68 73 74 0=0
Convolution conv_26 1 1 50 75 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=32768
Swish silu_89 1 1 75 76
Concat cat_2 2 1 74 76 77 0=0
Convolution conv_27 1 1 77 78 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_90 1 1 78 79
Split splitncnn_10 1 2 79 80 81
Convolution conv_28 1 1 81 82 0=512 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=1179648
Swish silu_91 1 1 82 83
Split splitncnn_11 1 2 83 84 85
Convolution conv_29 1 1 85 86 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=131072
Swish silu_92 1 1 86 87
Split splitncnn_12 1 2 87 88 89
Convolution conv_30 1 1 89 90 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_93 1 1 90 91
Convolution conv_31 1 1 91 92 0=256 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=589824
Swish silu_94 1 1 92 93
BinaryOp add_6 2 1 88 93 94 0=0
Convolution conv_32 1 1 84 95 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=131072
Swish silu_95 1 1 95 96
Concat cat_3 2 1 94 96 97 0=0
Convolution conv_33 1 1 97 98 0=512 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=262144
Swish silu_96 1 1 98 99
Convolution conv_34 1 1 99 100 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=131072
Swish silu_97 1 1 100 101
Split splitncnn_13 1 2 101 102 103
Pooling maxpool2d_63 1 1 103 104 0=0 1=5 11=5 12=1 13=2 2=1 3=2 5=1
Split splitncnn_14 1 2 104 105 106
Pooling maxpool2d_64 1 1 106 107 0=0 1=5 11=5 12=1 13=2 2=1 3=2 5=1
Split splitncnn_15 1 2 107 108 109
Pooling maxpool2d_65 1 1 109 110 0=0 1=5 11=5 12=1 13=2 2=1 3=2 5=1
Concat cat_4 4 1 102 105 108 110 111 0=0
Convolution conv_35 1 1 111 112 0=512 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=524288
Swish silu_98 1 1 112 113
Convolution conv_36 1 1 113 114 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=131072
Swish silu_99 1 1 114 115
Split splitncnn_16 1 2 115 116 117
Interp upsample_123 1 1 117 118 0=1 1=2.000000e+00 2=2.000000e+00 6=0
Concat cat_5 2 1 118 80 119 0=0
Split splitncnn_17 1 2 119 120 121
Convolution conv_37 1 1 121 122 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_100 1 1 122 123
Convolution conv_38 1 1 123 124 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_101 1 1 124 125
Convolution conv_39 1 1 125 126 0=128 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=147456
Convolution conv_40 1 1 120 127 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_103 1 1 127 128
Swish silu_102 1 1 126 129
Concat cat_6 2 1 129 128 130 0=0
Convolution conv_41 1 1 130 131 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_104 1 1 131 132
Convolution conv_42 1 1 132 133 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=32768
Swish silu_105 1 1 133 134
Split splitncnn_18 1 2 134 135 136
Interp upsample_124 1 1 136 137 0=1 1=2.000000e+00 2=2.000000e+00 6=0
Concat cat_7 2 1 137 46 138 0=0
Split splitncnn_19 1 2 138 139 140
Convolution conv_43 1 1 140 141 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_106 1 1 141 142
Convolution conv_44 1 1 142 143 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=4096
Swish silu_107 1 1 143 144
Convolution conv_45 1 1 144 145 0=64 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=36864
Convolution conv_46 1 1 139 146 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_109 1 1 146 147
Swish silu_108 1 1 145 148
Concat cat_8 2 1 148 147 149 0=0
Convolution conv_47 1 1 149 150 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_110 1 1 150 151
Split splitncnn_20 1 2 151 152 153
Convolution conv_48 1 1 153 154 0=128 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=147456
Swish silu_111 1 1 154 155
Concat cat_9 2 1 155 135 156 0=0
Split splitncnn_21 1 2 156 157 158
Convolution conv_49 1 1 158 159 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=32768
Swish silu_112 1 1 159 160
Convolution conv_50 1 1 160 161 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=16384
Swish silu_113 1 1 161 162
Convolution conv_51 1 1 162 163 0=128 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=147456
Convolution conv_52 1 1 157 164 0=128 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=32768
Swish silu_115 1 1 164 165
Swish silu_114 1 1 163 166
Concat cat_10 2 1 166 165 167 0=0
Convolution conv_53 1 1 167 168 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_116 1 1 168 169
Split splitncnn_22 1 2 169 170 171
Convolution conv_54 1 1 171 172 0=256 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=589824
Swish silu_117 1 1 172 173
Concat cat_11 2 1 173 116 174 0=0
Split splitncnn_23 1 2 174 175 176
Convolution conv_55 1 1 176 177 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=131072
Swish silu_118 1 1 177 178
Convolution conv_56 1 1 178 179 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65536
Swish silu_119 1 1 179 180
Convolution conv_57 1 1 180 181 0=256 1=3 11=3 12=1 13=1 14=1 2=1 3=1 4=1 5=1 6=589824
Convolution conv_58 1 1 175 182 0=256 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=131072
Swish silu_121 1 1 182 183
Swish silu_120 1 1 181 184
Concat cat_12 2 1 184 183 185 0=0
Convolution conv_59 1 1 185 186 0=512 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=262144
MemoryData pnnx_53 0 1 187 0=2 1=20 11=20 2=3
MemoryData pnnx_54 0 1 188 0=2 1=20 11=20 2=3
MemoryData pnnx_55 0 1 189 0=2 1=40 11=40 2=3
MemoryData pnnx_56 0 1 190 0=2 1=40 11=40 2=3
MemoryData pnnx_58 0 1 191 0=2 1=80 11=80 2=3
MemoryData pnnx_60 0 1 192 0=2 1=80 11=80 2=3
Convolution conv_60 1 1 152 193 0=255 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=32640
Reshape view_128 1 1 193 194 0=80 1=80 11=85 2=3
Permute permute_134 1 1 194 195 0=3
Sigmoid sigmoid_0 1 1 195 196
Slice split_0 1 3 196 197 198 199 -23300=3,2,2,81 1=3
MemoryData pnnx_fold_580 0 1 200 0=1
Eltwise weighted_sum_0 2 1 197 192 201 0=1 -23301=2,2.000000e+00,1.000000e+00
BinaryOp mul_9 2 1 201 200 202 0=2
BinaryOp mul_10 1 1 198 203 0=2 1=1 2=2.000000e+00
UnaryOp pow_11 1 1 203 204 0=4
BinaryOp mul_12 2 1 204 191 205 0=2
Concat cat_13 3 1 202 205 199 206 0=3
Convolution conv_61 1 1 170 207 0=255 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=65280
Reshape view_129 1 1 207 208 0=40 1=40 11=85 2=3
Permute permute_135 1 1 208 209 0=3
Sigmoid sigmoid_1 1 1 209 210
Slice split_1 1 3 210 211 212 213 -23300=3,2,2,81 1=3
MemoryData pnnx_fold_616 0 1 214 0=1
Eltwise weighted_sum_1 2 1 211 190 215 0=1 -23301=2,2.000000e+00,1.000000e+00
BinaryOp mul_15 2 1 215 214 216 0=2
BinaryOp mul_16 1 1 212 217 0=2 1=1 2=2.000000e+00
UnaryOp pow_17 1 1 217 218 0=4
BinaryOp mul_18 2 1 218 189 219 0=2
Concat cat_14 3 1 216 219 213 220 0=3
Swish silu_122 1 1 186 221
Convolution conv_62 1 1 221 222 0=255 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=130560
Reshape view_130 1 1 222 223 0=20 1=20 11=85 2=3
Permute permute_136 1 1 223 224 0=3
Sigmoid sigmoid_2 1 1 224 225
Slice split_2 1 3 225 226 227 228 -23300=3,2,2,81 1=3
MemoryData pnnx_fold_652 0 1 229 0=1
Eltwise weighted_sum_2 2 1 226 188 230 0=1 -23301=2,2.000000e+00,1.000000e+00
BinaryOp mul_21 2 1 230 229 231 0=2
BinaryOp mul_22 1 1 227 232 0=2 1=1 2=2.000000e+00
UnaryOp pow_23 1 1 232 233 0=4
BinaryOp mul_24 2 1 233 187 234 0=2
Concat cat_15 3 1 231 234 228 235 0=3
Reshape view_133 1 1 206 236 0=85 1=19200
Reshape view_132 1 1 220 237 0=85 1=4800
Reshape view_131 1 1 235 238 0=85 1=1200
Concat cat_16 3 1 236 237 238 out0 0=0
使用完整的代码运行出错了

[YFforever2022]

D:\test>pnnx yolov5s.torchscript inputshape=[1,3,640,640]
pnnxparam = yolov5s.pnnx.param
pnnxbin = yolov5s.pnnx.bin
pnnxpy = yolov5s_pnnx.py
pnnxonnx = yolov5s.pnnx.onnx
ncnnparam = yolov5s.ncnn.param
ncnnbin = yolov5s.ncnn.bin
ncnnpy = yolov5s_ncnn.py
fp16 = 1
optlevel = 2
device = cpu
inputshape = [1,3,640,640]f32
inputshape2 =
customop =
moduleop =
############# pass_level0
inline module = models.common.Bottleneck
inline module = models.common.C3
inline module = models.common.Concat
inline module = models.common.Conv
inline module = models.common.SPPF
inline module = models.yolo.Detect
inline module = models.common.Bottleneck
inline module = models.common.C3
inline module = models.common.Concat
inline module = models.common.Conv
inline module = models.common.SPPF
inline module = models.yolo.Detect

---

############# pass_level1
no attribute value
no attribute value
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 1
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 1
unknown Parameter value kind prim::Constant of TensorType, t.dim = 1
############# pass_level2
############# pass_level3
############# pass_level4
############# pass_level5
############# pass_ncnn

[YFforever2022]

[Ss-shuang123]

@nihui 大佬可以分享一下你的yolov5s.ncnn.param和yolov5s.ncnn.bin吗，我也遇到同样问题，param内容和@YFforever2022 是一致的，蟹蟹

[Chikage0o0]

// extract intermediate blob before anchor box generation instead of the very last "out0"
// the three blob names can be easily discovered in netron visualizer
// they are the outputs of the last three convolution, before any memory layout transformation
// each out blob is the feat for a grid scale stride(8, 16, 32)

[Trytoz]

这是来自QQ邮箱的自动回复邮件。邮件我已收到。

[YFforever2022]

我已使用此方法解决https://zhuanlan.zhihu.com/p/606440867
来自https://github.com/triple-Mu/ncnn-examples

[BboyHanat]

使用这个方法里forward代码, 可以使用onnx2ncnn 转, 要onnx-simplify后的onnx模型(yolov5 里的那个export.py搞搞就好了), 我使用的mac
m1, 使用pnnx 转出来的ncnn模型结果一直有问题，但是可以转onnx也可以了。

[YFforever2022]

但是，使用这个方法导出的onnx模型感觉结构优化还不够好，之后转ncnn的推理速度有所下降

[BboyHanat]

但是，使用这个方法导出的onnx模型感觉结构优化还不够好，之后转ncnn的推理速度有所下降

是的，主要我使用mac m1, pnnx 转的模型输出一直有问题, 没找到原因。

[huangxiaoyuan]

1. 未修改forward（非动态），使用onnx2ncnn转出来的模型，进行检测，没有出现任何的提示；使用pnnx则没有问题。
2. 修改改forward，使用pnnx，检测后出现全是框。

[cqs-cc]

请问，20240410版本的ncnn交叉编译的头文件不包含以往版本的net.h，新版本是用哪个或者哪几个头文件替代net.h了呢

[ITCZhuxy]

有yolov5分类网络的代码吗

[cccober]

Due to version issues, there have been continuous error reports.