Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge dt changes #815

Merged
merged 221 commits into from
Jan 20, 2022
Merged
Show file tree
Hide file tree
Changes from 175 commits
Commits
Show all changes
221 commits
Select commit Hold shift + click to select a range
2832c85
Update SimulationResults to match newer interfaces
daniel-thom Dec 22, 2021
ce3020a
Update simulation results interfaces and tests
daniel-thom Dec 23, 2021
937bb90
Merge branch 'dev' into fix-results
daniel-thom Dec 23, 2021
20a0506
Merge branch 'dev' into fix-results
daniel-thom Dec 23, 2021
77f27cf
Fix duals results storage
daniel-thom Dec 24, 2021
9071a6b
Merge branch 'dev' into fix-results
daniel-thom Dec 24, 2021
f0a2f1b
Fix test
daniel-thom Dec 24, 2021
47a6249
Merge branch 'dev' into fix-results
daniel-thom Dec 27, 2021
8178375
correct error message printing
jd-lara Dec 27, 2021
a4d5c0f
add methods to initializar system state from emulator
jd-lara Dec 27, 2021
d1a6b68
use ordered dicts
jd-lara Dec 27, 2021
ce3d9de
minor fixes to models and sequence
jd-lara Dec 27, 2021
473e872
WIP: implement em models store data
jd-lara Dec 27, 2021
7851a29
Consoliate JuMP container conversions
daniel-thom Dec 28, 2021
2ef16cf
Fix array calculations
daniel-thom Dec 28, 2021
29d0834
WIP changes for emulation
jd-lara Dec 28, 2021
656ccdf
Merge pull request #807 from daniel-thom/fix-results
jd-lara Dec 28, 2021
8b3caf8
Merge pull request #813 from NREL-SIIP/jd/add_emulation_to_sim
jd-lara Dec 28, 2021
64a8070
fix emulation store
jd-lara Dec 28, 2021
01969f5
use get_column names
jd-lara Dec 29, 2021
696eff9
implement em_models for in memory store
jd-lara Dec 29, 2021
cffddc4
implement em_dataset for hdf5 store
jd-lara Dec 29, 2021
d96343f
update initialization of store
jd-lara Dec 29, 2021
310903b
fix use of calc dimensions
jd-lara Dec 29, 2021
0cc49d8
update interface functions
jd-lara Dec 29, 2021
3ae1ebb
add missing returns
jd-lara Dec 29, 2021
58632db
implement update initial conditions for em
jd-lara Dec 29, 2021
1390bdd
implement parameter update for em
jd-lara Dec 29, 2021
c476724
write em optimizer stats for in memory store
jd-lara Dec 29, 2021
bdd810c
generalize update_model!
jd-lara Dec 29, 2021
32536a9
add write resulting value
jd-lara Dec 29, 2021
4e9a8d6
add expressions to states
jd-lara Dec 29, 2021
ae2268d
implement missing methods
jd-lara Dec 29, 2021
ba987c3
add expression to value states
jd-lara Dec 29, 2021
a6c7968
check write before sending to store/state
jd-lara Dec 29, 2021
51c9339
update state methods
jd-lara Dec 29, 2021
74aaa9b
update time series interface
jd-lara Dec 30, 2021
c079de7
remove stale comments
jd-lara Dec 30, 2021
0c5e92d
add update function
jd-lara Dec 30, 2021
f2b9509
refactor store interface
jd-lara Dec 30, 2021
dff8853
fix tests
jd-lara Dec 30, 2021
35fcdb7
implement method for in memory store
jd-lara Dec 30, 2021
d204c04
add dispatch on the index type
jd-lara Dec 30, 2021
499ba7d
specialize on model type
jd-lara Dec 30, 2021
a3e9455
refactor store uses
jd-lara Dec 30, 2021
3ca4786
update jump utils to dispatch on array size
jd-lara Dec 30, 2021
0ff9993
use logging.error in test
jd-lara Dec 30, 2021
8acf8c5
whitespace change
jd-lara Dec 30, 2021
d335908
additional changes to store
jd-lara Dec 30, 2021
3caf711
remove stale code
jd-lara Dec 30, 2021
d970de7
remove a dual solve call
jd-lara Dec 30, 2021
cea1166
add @assertion in initial conditions handling
jd-lara Dec 30, 2021
4133644
abstract getter functions
jd-lara Dec 30, 2021
f44dbc1
use getter functions
jd-lara Dec 30, 2021
0991194
WIP add _update_state for em
jd-lara Dec 30, 2021
f5bc50b
record row
jd-lara Dec 30, 2021
477a47c
fix export tests
jd-lara Dec 30, 2021
a1fbe30
update to re-establish results tests
jd-lara Dec 30, 2021
de90abb
formatter
jd-lara Dec 30, 2021
87c269b
Update SimulationResults to match newer interfaces
daniel-thom Dec 22, 2021
de804f9
Update simulation results interfaces and tests
daniel-thom Dec 23, 2021
d179f52
Fix duals results storage
daniel-thom Dec 24, 2021
6f975c4
Fix test
daniel-thom Dec 24, 2021
4b4c37c
Consoliate JuMP container conversions
daniel-thom Dec 28, 2021
0a73ec6
Fix array calculations
daniel-thom Dec 28, 2021
017edd5
correct error message printing
jd-lara Dec 27, 2021
f894b75
add methods to initializar system state from emulator
jd-lara Dec 27, 2021
3c471b3
use ordered dicts
jd-lara Dec 27, 2021
8ea3cf4
minor fixes to models and sequence
jd-lara Dec 27, 2021
c52c38d
WIP: implement em models store data
jd-lara Dec 27, 2021
27d71e4
WIP changes for emulation
jd-lara Dec 28, 2021
920207e
fix emulation store
jd-lara Dec 28, 2021
b1e0ada
use get_column names
jd-lara Dec 29, 2021
af03b74
implement em_models for in memory store
jd-lara Dec 29, 2021
440fe5a
implement em_dataset for hdf5 store
jd-lara Dec 29, 2021
96f7dc1
update initialization of store
jd-lara Dec 29, 2021
b65fea3
fix use of calc dimensions
jd-lara Dec 29, 2021
4106792
update interface functions
jd-lara Dec 29, 2021
6e0585f
add missing returns
jd-lara Dec 29, 2021
4279b6c
implement update initial conditions for em
jd-lara Dec 29, 2021
ea32c27
implement parameter update for em
jd-lara Dec 29, 2021
4b55933
write em optimizer stats for in memory store
jd-lara Dec 29, 2021
c32d3f5
generalize update_model!
jd-lara Dec 29, 2021
1efb95b
add write resulting value
jd-lara Dec 29, 2021
e8942d4
add expressions to states
jd-lara Dec 29, 2021
1d6dccb
implement missing methods
jd-lara Dec 29, 2021
8bb6329
add expression to value states
jd-lara Dec 29, 2021
1c806a3
check write before sending to store/state
jd-lara Dec 29, 2021
1196cfd
update state methods
jd-lara Dec 29, 2021
c1960c0
update time series interface
jd-lara Dec 30, 2021
4d528ec
remove stale comments
jd-lara Dec 30, 2021
b0f7c82
add update function
jd-lara Dec 30, 2021
09967ce
refactor store interface
jd-lara Dec 30, 2021
afb4f94
fix tests
jd-lara Dec 30, 2021
be2127f
implement method for in memory store
jd-lara Dec 30, 2021
5c7dede
add dispatch on the index type
jd-lara Dec 30, 2021
4828d60
specialize on model type
jd-lara Dec 30, 2021
cf9052b
refactor store uses
jd-lara Dec 30, 2021
e6a6f74
update jump utils to dispatch on array size
jd-lara Dec 30, 2021
13350c5
use logging.error in test
jd-lara Dec 30, 2021
94e5b46
whitespace change
jd-lara Dec 30, 2021
b044e9f
additional changes to store
jd-lara Dec 30, 2021
849b514
remove stale code
jd-lara Dec 30, 2021
3c34054
remove a dual solve call
jd-lara Dec 30, 2021
a66d8d6
add @assertion in initial conditions handling
jd-lara Dec 30, 2021
3f2d42e
abstract getter functions
jd-lara Dec 30, 2021
ef5a72f
use getter functions
jd-lara Dec 30, 2021
bab606f
WIP add _update_state for em
jd-lara Dec 30, 2021
6369aaf
record row
jd-lara Dec 30, 2021
734c0e8
fix export tests
jd-lara Dec 30, 2021
556ccc9
update to re-establish results tests
jd-lara Dec 30, 2021
23b2b09
formatter
jd-lara Dec 30, 2021
8689e4b
Merge branch 'jd/merge_dt_changes' of https://github.com/NREL-SIIP/Po…
jd-lara Dec 30, 2021
e7838ca
remove unused multiplier
jd-lara Dec 30, 2021
89220c5
fix utils tests
jd-lara Dec 30, 2021
dac3222
fix tests simulation store
jd-lara Dec 30, 2021
0f1e563
update simulation results
jd-lara Dec 30, 2021
b3fd5c8
fix leaky returns
jd-lara Dec 30, 2021
8e7d2f1
fix leaky returns
jd-lara Dec 31, 2021
7265de2
use index in read results
jd-lara Dec 31, 2021
5016811
WIP updating state from em
jd-lara Dec 31, 2021
1d8f8cb
whitespace change
jd-lara Jan 3, 2022
c9cc6fb
remove stale code
jd-lara Jan 3, 2022
f7edfeb
remove stale code
jd-lara Jan 4, 2022
a48c03b
remove not used variable
jd-lara Jan 4, 2022
d4e9d7d
adding a new objective func struct
sourabhdalvi Jan 7, 2022
798be77
adding cost function parameters
sourabhdalvi Jan 7, 2022
23b01e8
adding PWL variables
sourabhdalvi Jan 7, 2022
f977cea
adding cost function update methods
sourabhdalvi Jan 7, 2022
15205fc
adding test for cost updates
sourabhdalvi Jan 7, 2022
c048525
bug fix in has_container_key func
sourabhdalvi Jan 7, 2022
450f380
correct key encoding
jd-lara Jan 10, 2022
04f7b57
correct implementation of has_container_key
jd-lara Jan 11, 2022
9e49e1f
refactor cost function build
jd-lara Jan 11, 2022
566f4ed
add missing method for sparse axis arrays
jd-lara Jan 11, 2022
ec13899
update cost function time multiplier
jd-lara Jan 11, 2022
e8b8cb2
comment out test
jd-lara Jan 11, 2022
ef1c595
cost function refactor
jd-lara Jan 11, 2022
780b0ea
call different method
jd-lara Jan 11, 2022
1894c17
add new method to replace expression values
jd-lara Jan 11, 2022
d713970
move method to correct file
jd-lara Jan 11, 2022
1e8d885
remove method from file
jd-lara Jan 11, 2022
65dae7b
refactor some expression methods
jd-lara Jan 11, 2022
e1d3e4b
formatter run
jd-lara Jan 11, 2022
443ff20
remove stale codes
jd-lara Jan 11, 2022
ca681f5
change method name
jd-lara Jan 11, 2022
25246cf
constant name
jd-lara Jan 11, 2022
ff5bc63
add specific method for reals
jd-lara Jan 11, 2022
8582565
WIP simulation methods implementation
jd-lara Jan 11, 2022
9309e41
Merge branch 'jd/merge_dt_changes' into sd/cost_parameters
jd-lara Jan 11, 2022
210e125
Merge pull request #816 from NREL-SIIP/sd/cost_parameters
jd-lara Jan 11, 2022
cbfc625
remove duplicate code
jd-lara Jan 11, 2022
bd77e48
add alternative constructor
jd-lara Jan 12, 2022
3e08c9b
add new write result method
jd-lara Jan 12, 2022
1f2c45e
updates to simulation state
jd-lara Jan 12, 2022
b8a5a51
WIP writing to store from system state
jd-lara Jan 12, 2022
17f9fed
add SequentialWriteDataFrame and tests
jd-lara Jan 12, 2022
5be8329
Draft update to restore realized variables
daniel-thom Jan 12, 2022
f2eea92
update get_name
jd-lara Jan 12, 2022
0f0c88e
fix method use
jd-lara Jan 13, 2022
a40ae17
add more methods
jd-lara Jan 13, 2022
ec57ec2
update methods for sequential dfs
jd-lara Jan 13, 2022
cc08637
add get_name again
jd-lara Jan 13, 2022
eeef1f4
add tests
jd-lara Jan 13, 2022
c7c2533
more updates to seq dfs
jd-lara Jan 13, 2022
2157f21
add more writing capabilities
jd-lara Jan 13, 2022
682106d
update the use of last_recorded value
jd-lara Jan 13, 2022
b8811d1
remove unexported methods
jd-lara Jan 13, 2022
d5d36d5
update value states
jd-lara Jan 13, 2022
dfc86fe
update emulation model
jd-lara Jan 13, 2022
e7c0c35
use SequentialWriteDataFrame in state
jd-lara Jan 13, 2022
4d2a1bc
use SequentialWriteDataFrame in store
jd-lara Jan 13, 2022
78f12ab
update changes in simulation
jd-lara Jan 13, 2022
38f30b8
improve time keeping
jd-lara Jan 13, 2022
92210fc
rename data frame
jd-lara Jan 13, 2022
ae161b3
update names
jd-lara Jan 13, 2022
13b5d16
env variable name fix
jd-lara Jan 13, 2022
de82794
formatter changes
jd-lara Jan 14, 2022
f4dd347
updates to store for emulation simulation
jd-lara Jan 14, 2022
b4ab87c
remoce unused code
jd-lara Jan 14, 2022
a6b38da
add additional fields and accessors
jd-lara Jan 14, 2022
4e93f7d
remove un used field
jd-lara Jan 14, 2022
dab1bdc
wip refactor all stores
jd-lara Jan 14, 2022
acdc789
fix updating logic
jd-lara Jan 14, 2022
fd74d7a
update emulation model to pass tests
jd-lara Jan 15, 2022
b96673e
fix decision model tests
jd-lara Jan 15, 2022
c2477b0
move method
jd-lara Jan 15, 2022
ba826ac
change method name
jd-lara Jan 15, 2022
927d781
address PR comments
jd-lara Jan 18, 2022
a149a86
correct use of setters
jd-lara Jan 18, 2022
533fa79
format change
jd-lara Jan 18, 2022
9591a4d
remove unnecessary method
jd-lara Jan 18, 2022
090c2ac
wip: merge behavior of result writing to em_data
jd-lara Jan 18, 2022
f2fbb26
Merge branch 'jd/merge_dt_changes' into realized-results
jd-lara Jan 18, 2022
1fd58bf
Merge branch 'jd/store_refactors' into realized-results
jd-lara Jan 18, 2022
aa7c3d0
Merge pull request #817 from daniel-thom/realized-results
jd-lara Jan 18, 2022
cff70fd
format change
jd-lara Jan 18, 2022
e7c3636
implement read em data
jd-lara Jan 18, 2022
382b1fb
update JuliaFormatter version
jd-lara Jan 19, 2022
de33ed1
refactor datasets
jd-lara Jan 19, 2022
e451468
add new utils
jd-lara Jan 19, 2022
1355c10
remove use of getter for timestamps
jd-lara Jan 19, 2022
f62c52a
add write/read methods for em_data in hdf store
jd-lara Jan 19, 2022
507e315
change recorder writes
jd-lara Jan 19, 2022
0ca98da
formatter run
jd-lara Jan 19, 2022
1eea2e6
update test
jd-lara Jan 19, 2022
cc33172
remove unnecessary use of Dates.Period
jd-lara Jan 20, 2022
81dc907
remove type annotation
jd-lara Jan 20, 2022
6d5094a
simplify conversion code
jd-lara Jan 20, 2022
c3354fd
fix docstring
jd-lara Jan 20, 2022
3a78bff
comment out invalid code
jd-lara Jan 20, 2022
0332fa1
formatter
jd-lara Jan 20, 2022
f3bcb90
update of get_XXX_values in dataset container
jd-lara Jan 20, 2022
e75c5af
revert change
jd-lara Jan 20, 2022
da902b7
create correct dataset
jd-lara Jan 20, 2022
7592f72
Update src/simulation/hdf_simulation_store.jl
jd-lara Jan 20, 2022
0db7cde
fix results tests
jd-lara Jan 20, 2022
2ab9d50
Merge pull request #818 from NREL-SIIP/jd/store_refactors
jd-lara Jan 20, 2022
43da263
delete throw
jd-lara Jan 20, 2022
e10ee0e
set keep in cache true
jd-lara Jan 20, 2022
8ea3c3c
fix return
jd-lara Jan 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 4 additions & 18 deletions src/PowerSimulations.jl
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,6 @@ export list_recorder_events
export show_recorder_events
export list_simulation_events
export show_simulation_events
export write_results
export write_to_CSV

## Enums
Expand Down Expand Up @@ -329,25 +328,12 @@ import LinearAlgebra
import JSON3
import PowerSystems
import InfrastructureSystems
# so that users have access to IS.Results interfaces
import InfrastructureSystems:
get_variables,
get_parameters,
get_total_cost,
write_results,
get_timestamp,
get_resolution,
get_name,
@assert_op,
list_recorder_events
import InfrastructureSystems: @assert_op, list_recorder_events, get_name
export get_name
jd-lara marked this conversation as resolved.
Show resolved Hide resolved
export get_model_base_power
export get_total_cost
export get_optimizer_stats
export get_timestamp
export get_timestamps
export get_resolution
export write_results
import PowerModels
import TimerOutputs
import ProgressMeter
Expand Down Expand Up @@ -433,9 +419,11 @@ include("core/initial_conditions.jl")
include("core/settings.jl")
include("core/cache_utils.jl")
include("core/optimizer_stats.jl")
include("core/extended_dataframe.jl")
include("core/value_states.jl")

include("core/optimization_container.jl")
include("core/store_common.jl")

# Order Required
include("initial_conditions/initial_condition_chronologies.jl")
Expand Down Expand Up @@ -474,7 +462,6 @@ include("simulation/simulation_models.jl")
include("simulation/simulation_state.jl")
include("simulation/initial_condition_update_simulation.jl")
include("simulation/simulation_store_params.jl")
include("simulation/simulation_store_common.jl")
include("simulation/hdf_simulation_store.jl")
include("simulation/in_memory_simulation_store.jl")
include("simulation/simulation_problem_results.jl")
Expand All @@ -489,15 +476,14 @@ include("devices_models/devices/common/range_constraint.jl")
include("devices_models/devices/common/add_variable.jl")
include("devices_models/devices/common/add_auxiliary_variable.jl")
include("devices_models/devices/common/add_constraint_dual.jl")
include("devices_models/devices/common/rating_constraints.jl")
include("devices_models/devices/common/rateofchange_constraints.jl")
include("devices_models/devices/common/duration_constraints.jl")
include("devices_models/devices/common/commitment_constraint.jl")
include("devices_models/devices/common/get_time_series.jl")

# Device Modeling components
include("devices_models/devices/interfaces.jl")
include("devices_models/devices/common/add_to_expression.jl")
include("devices_models/devices/common/set_expression.jl")
include("devices_models/devices/renewable_generation.jl")
include("devices_models/devices/thermal_generation.jl")
include("devices_models/devices/electric_loads.jl")
Expand Down
7 changes: 6 additions & 1 deletion src/core/abstract_simulation_store.jl
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,9 @@ abstract type SimulationStore end
# - write_result!
# - read_result!
# - write_optimizer_stats!
# - read_problem_optimizer_stats
# - read_optimizer_stats
# - get_dm_data
# - get_em_data
jd-lara marked this conversation as resolved.
Show resolved Hide resolved

get_dm_data(store::SimulationStore) = store.dm_data
get_em_data(store::SimulationStore) = store.em_data
2 changes: 2 additions & 0 deletions src/core/auxiliary_variables.jl
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,5 @@ struct TimeDurationOff <: AuxVariableType end

""" Auxiliary Variable for Thermal Generation Models that solve for power above min"""
struct PowerOutput <: AuxVariableType end

should_write_resulting_value(::Type{<:AuxVariableType}) = true
2 changes: 2 additions & 0 deletions src/core/constraints.jl
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,5 @@ function encode_key(key::ConstraintKey)
end

Base.convert(::Type{ConstraintKey}, name::Symbol) = ConstraintKey(decode_symbol(name)...)

should_write_resulting_value(::Type{<:ConstraintType}) = true
4 changes: 4 additions & 0 deletions src/core/expressions.jl
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,7 @@ struct ReserveRangeExpressionUB <: RangeConstraintUBExpressions end
struct ComponentActivePowerRangeExpressionUB <: RangeConstraintUBExpressions end
struct ComponentReserveUpBalanceExpression <: ExpressionType end
struct ComponentReserveDownBalanceExpression <: ExpressionType end

should_write_resulting_value(::Type{<:ExpressionType}) = false
should_write_resulting_value(::Type{<:CostExpressions}) = true
should_write_resulting_value(::Type{RawACE}) = true
189 changes: 189 additions & 0 deletions src/core/extended_dataframe.jl

Large diffs are not rendered by default.

Loading