Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPC Branch Comparison #2

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
46ab1bd
shell script to push jobs
prakaa Nov 10, 2022
db4fb85
capture all decisions
prakaa Nov 10, 2022
dcb231e
add 2 hour horizon and tighten gap to 0.005
prakaa Nov 13, 2022
51fe08b
pbs scripts
prakaa Nov 13, 2022
7ae8f5b
shell script for queueing
prakaa Nov 13, 2022
9c8edc3
fix pbs names
prakaa Nov 13, 2022
de22633
do not capture all decisions - int32 truncation error
prakaa Nov 14, 2022
38a3f5d
add sleep to scripts to prevent simultaneous completion
prakaa Nov 14, 2022
13c75c0
refactor to allow CLI args
prakaa Nov 14, 2022
6a4a034
parse CLI args in pbs
prakaa Nov 14, 2022
1255f83
pass permutations via shell scripts for queueing pbs jobs
prakaa Nov 14, 2022
0bf1294
change qsub flags
prakaa Nov 14, 2022
da726c3
delimeter for julia calls
prakaa Nov 14, 2022
7ea46a2
try using qsub to pass vars
prakaa Nov 14, 2022
4e29716
correct qsub -v syntax
prakaa Nov 14, 2022
986724c
add CairoMakie to deps and to sysimage script
prakaa Nov 17, 2022
d7df4a5
add pbs for arb penalty
prakaa Nov 18, 2022
9f57367
add queueing shell script for throughpout penalty
prakaa Nov 18, 2022
76647b3
fix dir
prakaa Nov 18, 2022
0be7f4c
fix path
prakaa Nov 18, 2022
17ba302
fix names
prakaa Nov 18, 2022
ba37cf3
fix penalty costs submitted for simulation
prakaa Nov 20, 2022
22877b3
update julia version to match new katana version
prakaa Mar 14, 2023
2a8598a
correct rebase mistakes
prakaa Mar 23, 2023
38182e0
update Manifest
prakaa Mar 23, 2023
33acbba
add echos to notify submission
prakaa Mar 23, 2023
c4581c3
harmonise sims with master branch
prakaa Mar 23, 2023
cb010ef
fix HPC Manifest
prakaa Mar 23, 2023
2ff961e
HPC scripts for cap contract
prakaa Apr 3, 2023
c7832e3
fix typo in queue script
prakaa Apr 3, 2023
3271cd0
try absolute cap values and restrict modelled MW
prakaa Apr 4, 2023
d677a75
add discounting pbs and sh script
prakaa Apr 17, 2023
3ed3372
larger sleep time between sims
prakaa Apr 18, 2023
761ec84
original sleep time
prakaa Apr 18, 2023
e6933f8
update deps
prakaa Apr 18, 2023
9e738ca
fix toml
prakaa Jul 5, 2024
5378cc7
fix HPC manifest
prakaa Jul 5, 2024
77c70bd
pin progressmeter to 1.7.2
prakaa Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,063 changes: 420 additions & 643 deletions Manifest.toml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[compat]
julia = "1.9"
julia = "1.8"

[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_capcontracted_no_degradation/simulate_actual.jl $power $energy $cap
echo "Submitted ArbCapContracted Actual $power MW $energy MWh $cap Cap MW"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_capcontracted_no_degradation/simulate_forecast.jl $power $energy $cap
echo "Submitted ArbCapContracted Forecast $power MW $energy MWh $cap Cap MW"
11 changes: 11 additions & 0 deletions pbs/arbdiscounted_nodeg/simulate_arbdiscounted_nodeg_actual.pbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_discounted_no_degradation/simulate_actual.jl $power $energy $func
echo "Submitted ArbDiscounted Actual $power MW $energy MWh $func discounting"
11 changes: 11 additions & 0 deletions pbs/arbdiscounted_nodeg/simulate_arbdiscounted_nodeg_forecast.pbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_discounted_no_degradation/simulate_forecast.jl $power $energy $func
echo "Submitted ArbDiscounted Forecast $power MW $energy MWh $func discounting"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_throughputlimited_no_degradation/simulate_actual.jl $power $energy
echo "Submitted ArbThroughputLimited Actual $power MW $energy MWh"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_throughputlimited_no_degradation/simulate_forecast.jl $power $energy
echo "Submitted ArbThroughputLimited Forecast $power MW $energy MWh"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_throughputpenalty_no_degradation/simulate_actual.jl $power $energy $cost
echo "Submitted ArbThroughputPenalty Actual $power MW $energy MWh $cost AUD/MWH"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_throughputpenalty_no_degradation/simulate_forecast.jl $power $energy $cost
echo "Submitted ArbThroughputPenalty Forecast $power MW $energy MWh $cost AUD/MWH"
11 changes: 11 additions & 0 deletions pbs/standardarb_nodeg/simulate_standardarb_nodeg_actual.pbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_no_degradation/simulate_actual.jl $power $energy
echo "Submitted StandardArb Actual $power MW $energy MWh"
11 changes: 11 additions & 0 deletions pbs/standardarb_nodeg/simulate_standardarb_nodeg_forecast.pbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

#PBS -l select=1:ncpus=16:mem=60gb
#PBS -l walltime=12:00:00
#PBS -M [email protected]
#PBS -m ae

module add julia/1.8.2-ilp64
julia -e 'using Pkg;Pkg.activate("NEMStorageUnderUncertainty");Pkg.instantiate()'
julia --project="NEMStorageUnderUncertainty" -- NEMStorageUnderUncertainty/simulations/arbitrage_no_degradation/simulate_forecast.jl $power $energy
echo "Submitted StandardArb Forecast $power MW $energy MWh"
27 changes: 27 additions & 0 deletions queue_arbcapcontracted_nodeg.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/sh

# Simulate 4 different degrees of cap contracting
# 0 MW (equivalent to arbthroughputpenalty with penalty of 500,000 AUD/MWh)
# 25 MW
# 50 MW
# 100 MW
# 200 MW
# 400 MW
# 800 MW
# Restrict modelled power ratings to 25 MW, 100 MW, 400 MW

for energy in 100
do
for cap_mw in 0.0 25.0 50.0 100.0 200.0 400.0 800.0
do
for powerratio in 0.25 1 4
do
power=$(echo "$energy*$powerratio" | bc -l)
for file in pbs/arbcapcontracted_nodeg/*
do
qsub -v "power=$power,energy=$energy,cap=$cap_mw" $file
sleep 30s
done
done
done
done
19 changes: 19 additions & 0 deletions queue_arbdiscounted_nodeg.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/sh

# Simulate exp and hyp discounting for each power/energy combination

for energy in 100
do
for func in "hyp" "exp"
do
for powerratio in 0.125 0.25 0.5 1 2 4 8
do
power=$(echo "$energy*$powerratio" | bc -l)
for file in pbs/arbdiscounted_nodeg/*
do
qsub -v "power=$power,energy=$energy,func=$func" $file
sleep 30s
done
done
done
done
13 changes: 13 additions & 0 deletions queue_arbthroughputlimited_nodeg.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/sh
for energy in 100
do
for powerratio in 0.125 0.25 0.5 1 2 4 8
do
power=$(echo "$energy*$powerratio" | bc -l)
for file in pbs/arbthroughputlimited_nodeg/*
do
qsub -v "power=$power,energy=$energy" $file
sleep 1m
done
done
done
21 changes: 21 additions & 0 deletions queue_arbthroughputpenalty_nodeg.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/sh

# costs roughly reflect a range of build costs for BESS of varuious storage duration as outlined in AEMO I&A 2021 Workbook
# for 2020-2021, costs in $/kW are ~800 $/kW (1 hr storage) and ~3000 $/kW (8 hour storage)
# translating to energy capacity, 800,000 $/MWh and 375,000 $/MWh

for energy in 100
do
for cost in 200000.0 400000.0 600000.0 800000.0
do
for powerratio in 0.125 0.25 0.5 1 2 4 8
do
power=$(echo "$energy*$powerratio" | bc -l)
for file in pbs/arbthroughputpenalty_nodeg/*
do
qsub -v "power=$power,energy=$energy,cost=$cost" $file
sleep 30s
done
done
done
done
13 changes: 13 additions & 0 deletions queue_standardarb_nodeg.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/sh
for energy in 100
do
for powerratio in 0.125 0.25 0.5 1 2 4 8
do
power=$(echo "$energy*$powerratio" | bc -l)
for file in pbs/standardarb_nodeg/*
do
qsub -v "power=$power,energy=$energy" $file
sleep 1m
done
done
done