For inference, I have the same problem. The client send jpg to server, then the server use cv2 to do resize. When put the image data into the mx.nd.array, it's very slow. And the Utilization of GPU is low too. How to solve it? Thank you!

[Joke09]

Originally posted by @Joke09 in #7899 (comment)

[piyushghai]

Hi @Joke09 ,

When you put your image data into the NDArray, can you also try to copy it to GPU using as_in_context method before passing it to your model ?

http://mxnet.incubator.apache.org/test/api/python/ndarray.html#mxnet.ndarray.NDArray.as_in_context

[lanking520]

Hi @Joke09 thanks for your question. I think you might miss this: https://mxnet.incubator.apache.org/api/python/image/image.html please take a look

[Joke09]

Hi @Joke09 ,

When you put your image data into the NDArray, can you also try to copy it to GPU using as_in_context method before passing it to your model ?

http://mxnet.incubator.apache.org/test/api/python/ndarray.html#mxnet.ndarray.NDArray.as_in_context

Thank you! It works, But not much.
Before:
data = [mx.nd.array(im_arrays), mx.nd.array(im_infos)]
The shape of im_arrays is [8,3,896,1024]. It take 104ms.
After:
data = [mx.nd.array(im_arrays, ctx=mx.gpu(1)), mx.nd.array(im_infos)]
It take 68ms. I think it's not fast enough. And it use more memory in GPU.

[Joke09]

Hi @Joke09 thanks for your question. I think you might miss this: https://mxnet.incubator.apache.org/api/python/image/image.html please take a look

I think it works, when the image is in the disk. But image data is numpy array in this problem.
Sorry, I don't know how to use the image api for numpy array.

[shuokay]

convert numpy.array to raw binary and then image.decode

[Joke09]

convert numpy.array to raw binary and then image.decode

@shuokay Thank you very much!
Can you give me some code example?
I use like this:

im = cv2.read("a.jpg")
im = cv2.resize(im, None, None, fx=im_scale, fy=im_scale)
...
image = mx.img.imdecode(im)

But it's fail.

File "test.py", line 12, in
image = mx.img.imdecode(im)
File "/data/usr/incubator-mxnet/python/mxnet/image/image.py", line 142, in imdecode
return _internal._cvimdecode(buf, *args, **kwargs)
File "", line 36, in _cvimdecode
File "/data/usr/incubator-mxnet/python/mxnet/_ctypes/ndarray.py", line 98, in _imperative_invoke
stype=out_stypes[0])
File "/data/usr/incubator-mxnet/python/mxnet/ndarray/sparse.py", line 1185, in _ndarray_cls
raise Exception("unknown storage type: %s"%stype)
Exception: unknown storage type: -1

[Joke09]

I have this code:

data = [mx.nd.array(im_arrays), mx.nd.array(im_infos)]
data_shapes = [('data', im_arrays.shape), ('im_info', im_infos.shape)]
data_batch = mx.io.DataBatch(data=data, label=None, provide_data=data_shapes, provide_label=None)
self.module.forward(data_batch)

im_arrays is a numpy array and read from cv2. im_arrays.shape is [8,3,896,1024]
Can someone help me speed up this code with image api. I'm not familiar with Mxnet.

[lanking520]

Hi @Joke09 instead of trying to use cv2 to read, you should use this:
https://mxnet.incubator.apache.org/api/python/image/image.html#mxnet.image.imread
Which convert an image into NDArray directly

[Joke09]

Hi @Joke09 instead of trying to use cv2 to read, you should use this:
https://mxnet.incubator.apache.org/api/python/image/image.html#mxnet.image.imread
Which convert an image into NDArray directly

Hi @lanking520 , the client use cv2 to read and send the image data to the server. I can't control it.
I don't really want to write the image data to disk.

Another question:
the shape of data in mx.io.DataBatch must be [8,3,896,1024]. The im_infos.shape is (8, 3). They are both used for proposal in image detection. But image.imread and image.imdecode get only one NDArray and the shape is [3,896,1024]. How to get NDArray with the shape is [8,3,896,1024].

I have try this:

im_arrays = im_arrays.swapaxes(2, 3).swapaxes(1, 2)
#data = [mx.nd.array(im_arrays), mx.nd.array(im_infos)]
img_encodes = []
for im_array in im_arrays:
  img_encodes.append(cv2.imencode('.bmp', im_array)[1])
batch_images = [mx.img.imdecode(img_encode) for img_encode in img_encodes]
# note: I don't know how to convert this list of NDArray to an NDArray of batch image data.
data_batch = mx.io.DataBatch(data=data, label=None, provide_data=data_shapes, provide_label=None)
self.module.forward(data_batch)

I also try this:

im = cv2.imread("0.jpg")
start = time.time()
img_encode = cv2.imencode('.bmp', im)[1]
image = mx.img.imdecode(img_encode)
print(time.time() - start)
start = time.time()
image = mx.nd.array(im)
print(time.time() - start)

the fisrt print time is almost half of the second. Maybe the mx.nd.array is too slow.

[lanking520]

The first dim is the batch size which seemed you should set 8 images as a batch to feed them in. You can use nd.expand_dim to expand one more dimension on the image and use concat to concatenate images together.

[sandeep-krishnamurthy]

@Joke09 - Are you still facing the performance issue?