- This repo contains the work carried out as part of the Master's course Cognitive_Robotics at University of Groningen
- This repo contains a variation of the idea presented in Learning to See before Learning to Act paper, more details can be found in References
- Firstly, the training is done on passive vision task to learn to detect objects. We chose segmentation and in particular object grasp affordance segmentation instead of foreground segmentation as mentioned in the paper
- The trained passive vision task model is then transferred to learn an active vision task which is grasping
- The vision task used for learning is object part grasp affordance as segmentation
- Densenet-121 is the pre-trained model used
- Our code is in src/passive_task_segmentation/
- The dataset used for training passive vision task is UMD Grasp affordance segmentation RGBD dataset
- The model which was learnt in the passive vision task is transferred to initialize the learning of active vision task of grasping
- A major portion of the code for learning grasping task is borrowed from GGCNN repository
- The modified code is in src/active_task_grasping/
- The dataset used for training active vision task is Jacquard sample dataset
- The main page of Jacquard dataset main page
- The src/passive_task_segmentation/requirements.txt lists all requirements required for passive vision task
- To list all training options
python3 src/passive_task_segmentation/train.py --help
- To list all inference options
python3 src/passive_task_segmentation/infer.py --help
- To visualize data samples and evaluate the performance of the passive vision task, use the notebook in src/passive_task_segmentation/helper_notebooks/passive_task.ipynb
- The src/active_task_grasping/requirements.txt lists all requirements required for active vision task
- To list all training options
python3 src/active_task_grasping/train.py --help
- For evaluating the performance of the trained grasping model, we used simulated YCB objects in PyBullet framework
- The original repo is available here - https://github.com/SeyedHamidreza/cognitive_robotics_manipulation
- The modified repo used for evaluation is here - https://github.com/AbhishekRS4/cognitive_robotics_grasping_manipulation
- Abhishek Ramanathapura Satyanarayana - [email protected]
- Amit Bharti - [email protected]
- Isabelle Tilleman - [email protected]
- Nikos Douros - [email protected]