-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
staging-20230930_60k_basic_targetted_epoch37_Myria3DV3.4.0
Mount subdir instead of root cicd dir Correct default model name Correct default model name in config Add test set to the mix. config: set subtile_width to 0 fix: smaller batch size and prefetch factor to adapt to VM size fix: batch_size=1 fix: batch_size=2 Fix subtile_width to 50 and tile_width to 1000
- Loading branch information
1 parent
0c528d3
commit ecd8a44
Showing
4 changed files
with
99 additions
and
51 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| name: staging-model | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - 'staging*' | ||
|
|
||
| env: | ||
| nexus_server: 10.128.81.69:8082 | ||
| cicd_path: /var/data/cicd/CICD_github_assets/ | ||
| model_id: 20230930_60k_basic_targetted_epoch37_Myria3DV3.4.0 | ||
|
|
||
| jobs: | ||
| staging-model: | ||
| runs-on: self-hosted | ||
|
|
||
| steps: | ||
|
|
||
| - name: Checkout branch | ||
| uses: actions/checkout@v3 | ||
|
|
||
| - name: Build docker image | ||
| run: docker build -t myria3d . | ||
|
|
||
| - name: Inference on validation set of 20211001_building_val-lidar-prod-optimization | ||
| run: > | ||
| docker run | ||
| -v ${cicd_path}lidar_prod_optimization/:/lidar_prod_optimization/ | ||
| -v ${cicd_path}20211001_building_val-lidar-prod-optimization/:/data/ | ||
| --ipc=host | ||
| --shm-size=2gb | ||
| myria3d | ||
| python run.py | ||
| task.task_name=predict | ||
| predict.src_las=/data/val/*.las | ||
| predict.output_dir=/lidar_prod_optimization/${model_id}/preds-valset/ | ||
| predict.interpolator.probas_to_save=[building] | ||
| - name: Inference on test set of 20211001_building_val-lidar-prod-optimization | ||
| run: > | ||
| docker run | ||
| -v ${cicd_path}lidar_prod_optimization/:/lidar_prod_optimization/ | ||
| -v ${cicd_path}20211001_building_val-lidar-prod-optimization/:/data/ | ||
| --ipc=host | ||
| --shm-size=2gb | ||
| myria3d | ||
| python run.py | ||
| task.task_name=predict | ||
| predict.src_las=/data/test/*.las | ||
| predict.output_dir=/lidar_prod_optimization/${model_id}/preds-testset/ | ||
| predict.interpolator.probas_to_save=[building] | ||
| # NB: we run the new, tagged image at least once so that is it not prunned | ||
| # by mistake when using docker system prune. | ||
| - name: Tag the docker image with branch name | ||
| run: | | ||
| docker tag myria3d:latest myria3d:${{github.ref_name}} | ||
| docker run myria3d:${{github.ref_name}} bash | ||
| # get version number and date, to tag the image pushed to nexus | ||
| - name: get version number | ||
| id: tag | ||
| run: | | ||
| echo "::set-output name=version::$(docker run myria3d grep '__version__' package_metadata.yaml| cut -d\" -f2)" | ||
| echo "::set-output name=date::$(date '+%Y.%m.%d')" | ||
| - name: push branch docker on nexus (tagged with the branch name) | ||
| # we push on nexus an image from a branch when it's pushed | ||
| if: ((github.event_name == 'push') && (github.ref_name != 'main')) | ||
| run: | | ||
| docker tag myria3d $nexus_server/lidar_hd/myria3d:${{steps.tag.outputs.version}}-${{github.ref_name}} | ||
| docker login $nexus_server --username svc_lidarhd --password ${{ secrets.PASSWORD_SVC_LIDARHD }} | ||
| docker push $nexus_server/lidar_hd/myria3d:${{steps.tag.outputs.version}}-${{github.ref_name}} | ||
| - name: Clean dangling docker images | ||
| if: always() # always do it, even if something failed | ||
| run: docker system prune --force # remove dangling docker images, without asking user for confirmation |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file renamed
BIN
+13 MB
...roto151_V2.0_epoch_100_Myria3DV3.1.0.ckpt → ...asic_targetted_epoch37_Myria3DV3.4.0.ckpt
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters