Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using GPU functionality with a custom docker image #806

Open
ashwinjoseph95 opened this issue Nov 18, 2024 · 2 comments
Open

Using GPU functionality with a custom docker image #806

ashwinjoseph95 opened this issue Nov 18, 2024 · 2 comments

Comments

@ashwinjoseph95
Copy link

Hi Community ,
I am new to the world of Dockers and would appreciate help to help me resolve an issue I am facing!

I tried your example here with ubuntu docker image it detects the nvidia drivers:
image

I am trying to setup a docker instance for me to carry out a calibration process. The command have been provided by the author too but when I try to execute it, apparantly GPU is not getting utilised though it is installed in the host system. The repo is:
https://github.com/koide3/direct_visual_lidar_calibration/tree/main

I am trying to run the calibration on the environemnt:ROS1 and Ubuntu 20
My commands are as follows:

# Define paths
bag_path=$(realpath /home/knadmin/Ashwin/Targetless_calib/livox_ros1)
preprocessed_path=$(realpath /home/knadmin/Ashwin/Targetless_calib/livox_ros1_preprocessed)

docker run \
  -it \
  --net host \
  --runtime=nvidia --gpus all  \
  -e DISPLAY=$DISPLAY \
  -v $HOME/.Xauthority:/root/.Xauthority \
  -v $bag_path:/tmp/input_bags \
  -v $preprocessed_path:/tmp/preprocessed \
  -v pip_cache:/root/.cache/pip \
  koide3/direct_visual_lidar_calibration:noetic /bin/bash nvidia-smi

but apparenty nvidia-smi is not getting recognized.
image

What should i do to rectify this issue? I understand this might be slightly off topic but I would be really helpful to udnerstand what is going wrong and how I can fix it. I checked the dockefile its already usign another required base image.

@yeongrokgim
Copy link

As a quick debug, try append following environment variables so that nvidia-container-runtime would inject GPU related resources.

@@ -1,10 +1,12 @@
 docker run \
   -it \
   --net host \
   --runtime=nvidia --gpus all  \
   -e DISPLAY=$DISPLAY \
+  -e NVIDIA_VISIBLE_DEVICES=all \
+  -e NVIDIA_DRIVER_CAPABILITIES=all \
   -v $HOME/.Xauthority:/root/.Xauthority \
   -v $bag_path:/tmp/input_bags \
   -v $preprocessed_path:/tmp/preprocessed \
   -v pip_cache:/root/.cache/pip \
   koide3/direct_visual_lidar_calibration:noetic /bin/bash nvidia-smi

@elezar
Copy link
Member

elezar commented Nov 29, 2024

@ashwinjoseph95 could it be that the image that you're trying to run does not have the path set up such that nvidia-smi injected by the NVIDIA Continer Toolkit can be located.

What is the output of the env command in the container? Whats happens when you run /usr/bin/nvidia-smi instead of nvidia-smi?

@yeongrokgim thanks for the tip. Note that the --gpus all flag should be setting NVIDIA_VISIBLE_DEVICES=all in the docker daemon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants