TwoStageDetector

Features

• Pure Pytorch Code
• Three Pooling Methods: roi-pooling, roi-align, roi-crop

Main Results

• dataset: kitti(val) use train.txt for training, use val.txt for test

methods	ap(moderate)
baseline	89.21%
semantic	89.72%
semantic(finetune)	89.82%
iou	88.02%

• dataset: kitti(test) use trainval.txt and use test.txt for test, AP is 89.24 at present.

The details about models can be seen on wiki page.

Set up the environment

• This repo has been test on Python3.5 and Pytorch 0.4.0

Prepare the data

• data structure should be like this:

Kitti
    object
        training
            image_2
            label_2
            train.txt
            val.txt

• modify the some paths in configs/*.json like as examples

Configs

It can be generated by python scripts for convenience. Of course you can modified it manually. but you need be careful to make sure the consistency of configuration.(e.g. the same postprocess in eval and train)

There are lots of options in config files,you can just use configs/refine_kitti_config.json for free

Training

• To train KITTI dataset, simply run:

CUDA_VISIBLE_DEVICES=1 python trainval_net.py --cuda \
--net iou_faster_rcnn \
--out_path /data/object/liangxiong/iou_exp \
--config configs/iou_kitti_config.json

• There are many options that can be used

  • --net: specify net arch, there are many models supported now,like fpn,cascade,faster-rcnn,ssd
  • --out_path: all output file will put there
  • --config: config file
  • --cuda: enable gpu
  • --model: load pretrained model if you have your own one

• To resume from checkpoint,run

CUDA_VISIBLE_DEVICES=1 python trainval_net.py --cuda \
 --net two_rpn \
 --out_path /data/object/liangxiong/two_rpn \
 --config configs/two_rpn_config.json
 --r True \
 --checkpoint 3257 \
 --checkepoch 1

modify the corresponding options for yourself

Test

for evaluation model in val dataset,run

CUDA_VISIBLE_DEVICES=1 python test_net.py --cuda \
--checkpoint 3257 \
--checkepoch 37 \
--net iou_faster_rcnn \
--load_dir /data/object/liangxiong/iou_exp

note that dont need to specify config file for test, the script will find it in the directory of experiments.

after that, results will be generated in 'result/data' directory

Visualization

After running test,result will be put in results/data/ directory. Make sure that the results file exsits in that directory first.

python vis_all.py

It will visualize all results in order

Evaluation

just run sh eval.sh, then you can get three 2D aps for easy,moderate,hard

Development

• The directory structure of project is like

build/
    -  xxxx_builder.py
core/
    -  models/
        - feature_extractors/
    -  similarity_cals/
    -  bbox_coders/
    -  losses/
    -  ops/
    target_assigner.py
    filler.py
    saver.py
    trainer.py
    tester.py
    analyzer.py

• structure description

  • modules for training(xxxx_builder)

    • optmizer, how to optimize
    • scheduler, how to change leanring rate,
    • summary_writer, how to visualize some data
    • saver, how to load and store model weights
    • trainer and tester, exploit all modules above to train and test

  • modules for model

    • target_assigner, used for assigning each bbox labels for training
    • similarity_cals, used for calculating similarity between bboxes and gt_boxes(e.g. calculating iou)
    • bbox_coder, used for representing bbox(can be 3d bbox)
    • model, (e.g. rpn_model.py, faster_rcnn_model.py)
    • sampler, all subsample methods, (e.g. balanced subsample, hard negative subsample,...)
    • feature_extractors, use classic backbone to extract features
    • matcher, used for matching bbox and gt box
    • filler, used for weights initialization
    • analyzer, used for get statistics of intermedia results

If you want to develop a custom model, inherit from core/model.py and realize some abstract functions like init_params,init_modules and others

If many classes can be selected, builder design pattern can be used, do the same thing like as build/