Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifying the workflow to enable the use of different methods (LHS, Sobol) allowing for scenario discovery applications #13

Draft
wants to merge 35 commits into
base: envs
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
8e342fd
adding options to select different sampling method for scenario disco…
AgnesBelt Apr 15, 2024
c25f6dd
modifying snakemake file to enable the use of different sampling meth…
AgnesBelt Apr 15, 2024
a532005
changes in the sample.smk file to select different create_sample.py f…
AgnesBelt Apr 15, 2024
5f09fdc
adding different scripts for generating samples based on different me…
AgnesBelt Apr 15, 2024
b1ba801
changes in the snakemake file for calculating nr of model runs for LH…
AgnesBelt Apr 15, 2024
830941e
fixing calculation of model runs for Sobol method, N= replicates^2
AgnesBelt Apr 17, 2024
8e0cbdd
fixing the syntax for rule create_sample to allow for the use of diff…
AgnesBelt Apr 17, 2024
27e121a
modifying N definition for Sobol sample, N=replicates^2
AgnesBelt Apr 17, 2024
15b7006
changing name for sample.txt file to be non-Morris specific
AgnesBelt Apr 17, 2024
ca87810
started adding some documentation to the create_sample_LHS.py script
AgnesBelt Apr 17, 2024
43ace18
changed from morris_sample.txt to generic sample.txt in all inputs to…
AgnesBelt Apr 17, 2024
2c995f6
adding some documentation in the config file
AgnesBelt Apr 17, 2024
d8dacfc
testing some minor changes in the calculations of number of model run…
AgnesBelt Apr 17, 2024
277039a
changing from morris_sample.txt to generic sample.txt
AgnesBelt Apr 17, 2024
445704e
changing from morris_sample.txt to generic sample.txt for use with di…
AgnesBelt Apr 17, 2024
bad0398
testing out some options to fix bug: current issue is that the modelr…
AgnesBelt Apr 17, 2024
287031a
remove reference to morris_sample (replace with simple 'sample') for …
AgnesBelt Apr 17, 2024
1b987cb
adding fi at the end of the if/then option under create_sample rule t…
AgnesBelt Apr 17, 2024
955c9f5
removed test command that are not needed anymore
AgnesBelt Apr 17, 2024
19e6188
adding option to specify the starting year of interpolation for the u…
AgnesBelt Apr 18, 2024
e2c169b
modifying the script to generate the model run input data files to a…
AgnesBelt Apr 18, 2024
da0e419
adjusting the osemosys.smk rule to read in the user-specific start_ye…
AgnesBelt Apr 18, 2024
f36fa4d
fixing bug on create_modelrun.py to generate the model run input data…
AgnesBelt Apr 18, 2024
2d594a1
correcting message typo under calculate_SA_objective rule
AgnesBelt Apr 18, 2024
23aad32
fixing bug: changing input type from 'int' to 'int64' to allow readin…
AgnesBelt Apr 22, 2024
f28f090
removed the need for adding a YEAR into the parameters indexes for ca…
AgnesBelt Apr 22, 2024
1d75c6e
testing new changes also for Sobol method
AgnesBelt Apr 22, 2024
ab0a6f7
changing the deifnition of PARAMETERS as to take in all values input …
AgnesBelt May 2, 2024
14ec3a3
addressing minor issues as identified by Will's comments
AgnesBelt May 2, 2024
f550b28
updated instruction for Sobol sampling on the config.yaml file
AgnesBelt May 2, 2024
a7005bd
adding conda environments requisites to run the workflow
AgnesBelt Nov 22, 2024
71561a1
adding last changes to allow for definnying end year of interpolation…
AgnesBelt Nov 22, 2024
8d5de88
adding GLUCOSE-related files
AgnesBelt Nov 22, 2024
249a1ca
editing back the function apply_interploted_values to allow for uncer…
AgnesBelt Nov 26, 2024
ef6a960
latest settingd of the config file used for the runs on Nov 26, 2024
AgnesBelt Dec 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 19 additions & 8 deletions config/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,36 @@

# Populate the scenarios.csv file with a list of scenario names
# and path (description optional) to the model csv data
scenarios: config/scenarios.csv
scenarios: resources/glucose/scenarios_glucose_2024.csv
# scenarios: config/scenarios.csv

# Tell the workflow which model results to plot
results: config/results.csv
results: resources/glucose/results.csv
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to config/results.csv

# results: config/results.csv

# Filetype options: 'csv' or 'parquet' or 'feather'
filetype: csv

# Define the uncertain parameters used to define the Monte Carlo sample
parameters: config/parameters.csv
parameters: resources/glucose/parameters_glucose_2024_all.csv
# parameters: config/parameters.csv

# Define a start and end year for interpolation of the parameters
start_year: 2020
end_year: 2050

# Path to the OSeMOSYS model file
model_file: resources/osemosys_fast.txt
model_file: resources/glucose/osemosys_fast.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to resources/osemosys_fast.txt


# Choose a solver, choices: 'cbc' or 'gurobi' or 'cplex'
solver: cbc
solver: gurobi

# Choose a sampling method: 'Morris' or 'LHS' or 'Sobol'
method: Sobol

# Sampling - how large should the sample be?
replicates: 10
# Sampling - how large should the sample be?
# [NB: for Sobol, the nr of replicates will result in [(2^replicates)*(nr parameters +2)] nr of runs]
replicates: 6

# Scale Elementary Effects (True/False)
scale: False
Expand All @@ -29,4 +40,4 @@ scale: False
zip: True

# Only works for OSeMOSYS Global Naming Conventions (True/False)
re_share: True
re_share: False
234 changes: 234 additions & 0 deletions envs/snakemake_esom_gsa_env.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
name: snakemake_esom_gsa
channels:
- bioconda
- conda-forge
- defaults
dependencies:
- aiohttp=3.9.3=py312he37b823_1
- aiosignal=1.3.1=pyhd8ed1ab_0
- amply=0.1.6=pyhd8ed1ab_0
- appdirs=1.4.4=pyh9f0ad1d_0
- appnope=0.1.4=pyhd8ed1ab_0
- argparse-dataclass=2.0.0=pyhd8ed1ab_0
- asttokens=2.4.1=pyhd8ed1ab_0
- atk-1.0=2.38.0=hd03087b_2
- attrs=23.2.0=pyh71513ae_0
- boto=2.49.0=py_0
- boto3=1.34.75=pyhd8ed1ab_0
- botocore=1.34.75=pyge310_1234567_0
- brotli-python=1.1.0=py312h9f69965_1
- bz2file=0.98=py_0
- bzip2=1.0.8=h93a5062_5
- c-ares=1.28.1=h93a5062_0
- ca-certificates=2024.2.2=hf0a4a13_0
- cachetools=5.3.3=pyhd8ed1ab_0
- cairo=1.18.0=hd1e100b_0
- certifi=2024.2.2=pyhd8ed1ab_0
- cffi=1.16.0=py312h8e38eb3_0
- charset-normalizer=3.3.2=pyhd8ed1ab_0
- coin-or-cbc=2.10.10=hfaca913_0
- coin-or-cgl=0.60.7=hf050ae7_0
- coin-or-clp=1.17.8=h65c2c7c_0
- coin-or-osi=0.108.8=h19bcb3e_0
- coin-or-utils=2.11.9=h12587b1_0
- coincbc=2.10.10=0_metapackage
- comm=0.2.2=pyhd8ed1ab_0
- conda-inject=1.3.1=pyhd8ed1ab_0
- configargparse=1.7=pyhd8ed1ab_0
- connection_pool=0.0.3=pyhd3deb0d_0
- cryptography=42.0.5=py312h99f8e83_0
- datrie=0.8.2=py312h02f2b3b_7
- debugpy=1.8.1=py312h20a0b95_0
- decorator=5.1.1=pyhd8ed1ab_0
- docutils=0.20.1=py312h81bd7bf_3
- dpath=2.1.6=pyha770c72_0
- exceptiongroup=1.2.0=pyhd8ed1ab_2
- executing=2.0.1=pyhd8ed1ab_0
- expat=2.6.2=hebf3989_0
- font-ttf-dejavu-sans-mono=2.37=hab24e00_0
- font-ttf-inconsolata=3.000=h77eed37_0
- font-ttf-source-code-pro=2.038=h77eed37_0
- font-ttf-ubuntu=0.83=h77eed37_2
- fontconfig=2.14.2=h82840c6_0
- fonts-conda-ecosystem=1=0
- fonts-conda-forge=1=0
- freetype=2.12.1=hadb7bae_2
- fribidi=1.0.10=h27ca646_0
- frozenlist=1.4.1=py312he37b823_0
- gdk-pixbuf=2.42.11=h13c029f_0
- gettext=0.22.5=h8fbad5d_2
- gettext-tools=0.22.5=h8fbad5d_2
- giflib=5.2.2=h93a5062_0
- gitdb=4.0.11=pyhd8ed1ab_0
- gitpython=3.1.43=pyhd8ed1ab_0
- google-api-core=2.18.0=pyhd8ed1ab_0
- google-auth=2.29.0=pyhca7485f_0
- google-cloud-core=2.4.1=pyhd8ed1ab_0
- google-cloud-storage=2.16.0=pyhca7485f_0
- google-crc32c=1.1.2=py312h80a10bc_5
- google-resumable-media=2.7.0=pyhd8ed1ab_0
- googleapis-common-protos=1.63.0=pyhd8ed1ab_0
- graphite2=1.3.13=hebf3989_1003
- graphviz=9.0.0=h3face73_1
- grpcio=1.62.1=py312h17030e7_0
- gtk2=2.24.33=h7895bb2_4
- gts=0.7.6=he42f4ea_4
- harfbuzz=8.4.0=hbe0f7c0_0
- humanfriendly=10.0=pyhd8ed1ab_6
- icu=73.2=hc8870d7_0
- idna=3.6=pyhd8ed1ab_0
- immutables=0.20=py312h02f2b3b_1
- importlib-metadata=7.1.0=pyha770c72_0
- importlib_metadata=7.1.0=hd8ed1ab_0
- importlib_resources=6.4.0=pyhd8ed1ab_0
- ipykernel=6.29.3=pyh3cd1d5f_0
- ipython=8.22.2=pyh707e725_0
- jedi=0.19.1=pyhd8ed1ab_0
- jinja2=3.1.3=pyhd8ed1ab_0
- jmespath=1.0.1=pyhd8ed1ab_0
- jsonschema=4.21.1=pyhd8ed1ab_0
- jsonschema-specifications=2023.12.1=pyhd8ed1ab_0
- jupyter_client=8.6.1=pyhd8ed1ab_0
- jupyter_core=5.7.2=py312h81bd7bf_0
- lerc=4.0.0=h9a09cb3_0
- libabseil=20240116.1=cxx17_hebf3989_2
- libasprintf=0.22.5=h8fbad5d_2
- libasprintf-devel=0.22.5=h8fbad5d_2
- libblas=3.9.0=21_osxarm64_openblas
- libcblas=3.9.0=21_osxarm64_openblas
- libcrc32c=1.1.2=hbdafb3b_0
- libcxx=16.0.6=h4653b0c_0
- libdeflate=1.20=h93a5062_0
- libexpat=2.6.2=hebf3989_0
- libffi=3.4.2=h3422bc3_5
- libgd=2.3.3=hfdf3952_9
- libgettextpo=0.22.5=h8fbad5d_2
- libgettextpo-devel=0.22.5=h8fbad5d_2
- libgfortran=5.0.0=13_2_0_hd922786_3
- libgfortran5=13.2.0=hf226fd6_3
- libglib=2.80.1=h535f939_0
- libgrpc=1.62.1=h9c18a4f_0
- libiconv=1.17=h0d3ecfb_2
- libintl=0.22.5=h8fbad5d_2
- libintl-devel=0.22.5=h8fbad5d_2
- libjpeg-turbo=3.0.0=hb547adb_1
- liblapack=3.9.0=21_osxarm64_openblas
- liblapacke=3.9.0=21_osxarm64_openblas
- libopenblas=0.3.26=openmp_h6c19121_0
- libpng=1.6.43=h091b4b1_0
- libprotobuf=4.25.3=hbfab5d5_0
- libre2-11=2023.09.01=h7b2c953_2
- librsvg=2.58.0=hb3d354b_1
- libsodium=1.0.18=h27ca646_1
- libsqlite=3.45.2=h091b4b1_0
- libtiff=4.6.0=h07db509_3
- libwebp=1.4.0=h54798ee_0
- libwebp-base=1.4.0=h93a5062_0
- libxml2=2.12.6=h0d0cfa8_2
- libzlib=1.2.13=h53f4e23_5
- llvm-openmp=18.1.2=hcd81f8e_0
- markupsafe=2.1.5=py312he37b823_0
- matplotlib-inline=0.1.6=pyhd8ed1ab_0
- multidict=6.0.5=py312h670c8ac_0
- nbformat=5.10.3=pyhd8ed1ab_0
- ncurses=6.4.20240210=h078ce10_0
- nest-asyncio=1.6.0=pyhd8ed1ab_0
- numpy=1.26.4=py312h8442bc7_0
- openssl=3.3.0=h0d3ecfb_0
- packaging=24.0=pyhd8ed1ab_0
- pango=1.52.2=hb067d4f_0
- parso=0.8.3=pyhd8ed1ab_0
- pcre2=10.43=h26f9a81_0
- pexpect=4.9.0=pyhd8ed1ab_0
- pickleshare=0.7.5=py_1003
- pip=24.0=pyhd8ed1ab_0
- pixman=0.43.4=hebf3989_0
- pkgutil-resolve-name=1.3.10=pyhd8ed1ab_1
- plac=1.4.3=pyhd8ed1ab_0
- platformdirs=4.2.0=pyhd8ed1ab_0
- prompt-toolkit=3.0.42=pyha770c72_0
- proto-plus=1.23.0=pyhd8ed1ab_0
- protobuf=4.25.3=py312h1a610f3_0
- psutil=5.9.8=py312he37b823_0
- ptyprocess=0.7.0=pyhd3deb0d_0
- pulp=2.8.0=py312h81bd7bf_0
- pure_eval=0.2.2=pyhd8ed1ab_0
- pyasn1=0.5.1=pyhd8ed1ab_0
- pyasn1-modules=0.3.0=pyhd8ed1ab_0
- pycparser=2.22=pyhd8ed1ab_0
- pygments=2.17.2=pyhd8ed1ab_0
- pyopenssl=24.0.0=pyhd8ed1ab_0
- pyparsing=3.1.2=pyhd8ed1ab_0
- pysocks=1.7.1=pyha2e5f31_6
- python=3.12.2=hdf0ec26_0_cpython
- python-dateutil=2.9.0=pyhd8ed1ab_0
- python-fastjsonschema=2.19.1=pyhd8ed1ab_0
- python-tzdata=2024.1=pyhd8ed1ab_0
- python_abi=3.12=4_cp312
- pytz=2024.1=pyhd8ed1ab_0
- pyu2f=0.1.5=pyhd8ed1ab_0
- pyyaml=6.0.1=py312h02f2b3b_1
- pyzmq=25.1.2=py312h1edf716_0
- re2=2023.09.01=h4cba328_2
- readline=8.2=h92ec313_1
- referencing=0.34.0=pyhd8ed1ab_0
- requests=2.31.0=pyhd8ed1ab_0
- reretry=0.11.8=pyhd8ed1ab_0
- rpds-py=0.18.0=py312h77200ec_0
- rsa=4.9=pyhd8ed1ab_0
- s3transfer=0.10.1=pyhd8ed1ab_0
- setuptools=69.2.0=pyhd8ed1ab_0
- six=1.16.0=pyh6c4a22f_0
- smart_open=3.0.0=pyh9f0ad1d_0
- smmap=5.0.0=pyhd8ed1ab_0
- snakemake-interface-common=1.17.1=pyhdfd78af_0
- snakemake-interface-executor-plugins=9.1.0=pyhdfd78af_0
- snakemake-interface-report-plugins=1.0.0=pyhdfd78af_0
- snakemake-interface-storage-plugins=3.1.1=pyhdfd78af_0
- snakemake-minimal=8.10.4=pyhdfd78af_0
- stack_data=0.6.2=pyhd8ed1ab_0
- stopit=1.1.2=py_0
- tabulate=0.9.0=pyhd8ed1ab_1
- throttler=1.2.2=pyhd8ed1ab_0
- tk=8.6.13=h5083fa2_1
- toposort=1.10=pyhd8ed1ab_0
- tornado=6.4=py312he37b823_0
- traitlets=5.14.2=pyhd8ed1ab_0
- typing_extensions=4.10.0=pyha770c72_0
- tzdata=2024a=h0c530f3_0
- urllib3=2.2.1=pyhd8ed1ab_0
- wcwidth=0.2.13=pyhd8ed1ab_0
- wheel=0.43.0=pyhd8ed1ab_1
- wrapt=1.16.0=py312he37b823_0
- xz=5.2.6=h57fd34a_0
- yaml=0.2.5=h3422bc3_2
- yarl=1.9.4=py312he37b823_0
- yte=1.5.4=pyha770c72_0
- zeromq=4.3.5=hebf3989_1
- zipp=3.17.0=pyhd8ed1ab_0
- zlib=1.2.13=h53f4e23_5
- zstd=1.5.6=hb46c0d2_0
- pip:
- annotated-types==0.6.0
- contourpy==1.2.1
- cycler==0.12.1
- dill==0.3.8
- et-xmlfile==1.1.0
- flatten-dict==0.4.2
- fonttools==4.50.0
- kiwisolver==1.4.5
- matplotlib==3.8.3
- multiprocess==0.70.16
- networkx==3.2.1
- openpyxl==3.1.2
- otoole==1.1.0
- pandas==2.0.3
- pillow==10.3.0
- pydantic==2.6.4
- pydantic-core==2.16.3
- pydot==2.0.0
- python-graphviz==0.20.3
- salib==1.4.8
- scipy==1.13.0
- xlrd==2.0.1
prefix: /Users/agnesebeltramo/anaconda3/envs/snakemake_esom_gsa
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

AnnualEmissionLimit equal the GLUCOSE Baseline trajectory (high-value, increasing over modelling period) for the period 2010-2020
No EmissionPenalty
Add ModelPeriodEmissionLimit based on IPCC SR15 Estimated remaining carbon budget from beginning of 2020.

Loading