most time cost in NDArray GetData() func

[gxkevin]

My predict code is like this， I have 32 worker thread, each has 1 openmp thread, the mxnet engine is NaiveEngine , using the openblas, disable the mkldnn, running the code on the Skylatex machine, using the avx512, but it is not faster than the avx2, I don't know why?

And what's more, the naiveEngine means that all the funcs are sequential？(not asynchronous?)
In this condition, the WaitAll() func should removed?

what's more , I found that the most time is the NDArray.GetData(), about 17ms, and the Forward() cost 6ms, why GetData() cost so much time? How to increase the proformance?
the Avx512 doesn't work.

    dnn_model->model_data["data"].SyncCopyFromCPU(batch_data.data(), batch_size * fea_num);
    mxnet::cpp::NDArray::WaitAll();
    dnn_model->exec->Forward(false);
    mxnet::cpp::NDArray::WaitAll();
    uint32_t output_num = dnn_model->exec->outputs.size();
    for (uint32_t j = 0; j < b_num; j++) {
        for (uint32_t q_idx = 0; q_idx < output_num; q_idx++) {
            //cvm_q_t* item = result + ins_idx;
            //item->q[q_idx]  = dnn_model->exec->outputs[q_idx].GetData()[j];
            _result[ins_index][q_idx] = dnn_model->exec->outputs[q_idx].GetData()[j];
            CVM_DEBUG_LOG("ins_index:%u  q_idx %u q:%f", ins_index, q_idx, _result[ins_index][q_idx]);
        }
		ins_index += 1;
    }

[frankfliu]

@mxnet-label-bot add [question, performance]

[cyrusbehr]

I am wondering the same thing. Can we omit the WaitAll if we are running the NaiveEngine?