Instructions for fresh environment setup

• Clone our repository: git clone [email protected]:niwa/kelp-detection.git
• Create an .env file (contains LINZ LDS keys) and put into the root directory of the 'kelp-detection' repository
• Create the conda environment from environment.yml in the repository root - conda env create -f environment.yml
• Launch jupyter lab from the repositoriy root directory in the kelp conda environment
• Open the notebooks/compare_test_sites.ipynb and run

Dataset update

This is done by running Python scripts in the scripts folder and also review results in Streamlit Dashboard and perform manual QA/QC.

These scripts expect the current working directory to be relative to the scripts folder. If running in Visual Studio Code in file->perferences->settings search 'Execute in File Path' and select checkbox, or navigate to the scripts folder in the terminal prior to running for the autogenerated data folder to be created in the expected location (nested within the repository).

Instructions for data update.

1. Create kelp detections all dates - run create_data_NZ_wide_two_pass.py
2. Launch kelp dashboard if not already running (see Running Kelp Dashboard section below)
3. Review 'All Dates' tab in the kelp dashboard for any anomalous dates to ignore.
4. Update sites_dates_to_ignore.json with dates to ignore. IMPORTANT - date format is YYYY-MM-DD
5. Create kelp detections quarterly averaged - create_data_NZ_wide_two_pass_quarterly.py
6. Create site-wide presence-absence maps & summary info - run summarise_NZ_wide_info.py and summarise_NZ_wide_info_quarterly.py
7. Review in the the kelp dashboard. If suitable for deployment to the external production website talk to Rose or Craig about updating in the deployment.

Running Kelp Dashboard

A dashboard for displaying Kelp extents across dates used for QA/QC and general review.

We do this using Streamlit. The code is located under the src folder.

Usage - Windows locally

1. Open a miniconda terminal and activate an environment with streamlit and streamlit-folium added.
2. Navigate to the repository main folder
3. execute streamlit run src/Home.py
4. Follow the printed Local URL.

Usage - LINUX on HPC

Either use launch_app.sh directly or use the code within to lauch using the commandline.

./launch_app.sh

Just follow the printed network URL.

Setup - Conda

Create environment

set +u
module load Miniforge3
source $(conda info --base)/etc/profile.d/conda.sh
conda config --add pkgs_dirs /nesi/nobackup/niwa03440/$USER/conda_pkgs
cd {PATH_TO_REPOSITORY}
conda env create -f streamlit.yml -p /nesi/project/niwa03440/conda/envs/streamlit
conda activate /nesi/project/niwa03440/conda/envs/streamlit
set -u

Activate environment

set +u
module load Miniforge3
source $(conda info --base)/etc/profile.d/conda.sh
conda activate /nesi/project/niwa03440/conda/envs/streamlit
set -u