How to load custom dataset in coco format for training an instance segmentation model?

[jeanpat]

Hello,

I have a small dataset available on on kaggle. it consists in 125 images and their instance labels in coco format
How can I modify the given example to load my dataset?:

# 1. Create the DataModule
data_dir = icedata.pets.load_data()

datamodule = InstanceSegmentationData.from_folders(
    train_folder=data_dir,
    val_split=0.1,
    parser=partial(icedata.pets.parser, mask=True),
)

Thank you

[ethanwharris]

Hi @jeanpat thanks for the question! You can take a look at an example in our latest docs (not yet released) here: https://lightning-flash.readthedocs.io/en/latest/api/generated/flash.image.instance_segmentation.data.InstanceSegmentationData.html#flash.image.instance_segmentation.data.InstanceSegmentationData.from_coco

Note that you will need to install flash from github in order for this to work, which you can do with the following:

pip install 'git+https://github.com/PyTorchLightning/lightning-flash.git#egg=lightning-flash[image]' icevision

Hope that helps! 😃

[jeanpat]

Thank you!

[jeanpat]

I succeeded to load the dataset from the example code (great!):

notebook on kaggle, see version 2

The trainer complains a little bit:

/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/configuration_validator.py:122: UserWarning: You defined a `validation_step` but have no `val_dataloader`. Skipping val loop.
  rank_zero_warn("You defined a `validation_step` but have no `val_dataloader`. Skipping val loop.")
/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:117: UserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:413: UserWarning: The number of training samples (1) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  f"The number of training samples ({self.num_training_batches}) is smaller than the logging interval"

The training was very fast, I guess too fast:

Epoch 0: 100%
1/1 [00:05<00:00, 5.78s/it, loss=9.28, v_num=, train_loss=9.280]

This is a beginning