Implements option to build individual/groups of States/Regions

.pre-commit-config.yaml

@@ -60,4 +60,15 @@ repos:
   rev: 24.8.0
   hooks:
   - id: black
+    language_version: python3.11
+    args: ["--line-length", "120"]
   - id: black-jupyter
+    language_version: python3.11
+    args: ["--line-length", "120"]
+
+
+- repo: https://github.com/pycqa/flake8
+  rev: 7.1.1
+  hooks:
+  - id: flake8
+    args: [--select=F841]    # F841 is the error code for unused variables

docs/source/config-configuration.md

@@ -215,7 +215,6 @@ Sector coupling studies are all under active development. More info to come!
 (clustering_cf)=
 ## `clustering`
 
-When clustering `aggregation_zones` defines the region boundaries which will be respected through the clustering process; State boarders, balancing authority regions, or REeDs shapes. This feature is important for imposing constraints (`opts`) which are defined over specific regions. For example, the data included in the model on interface transfer capacities are prepared for REeDs shapes but not states and BA regions. Moving forward we plan to use REeDs shapes as our default however we will maintain States and BA regions as well.
 
 Each clustering and interconnection option will have a different number of minimum nodes which can be clustered to, an error will be thrown in `cluster_network` notifying you of that number if you have selected a value too low.
 

docs/source/config-spatial.md

@@ -0,0 +1,66 @@
+# Spatial Configuration
+
+## Configuring Spatial Scope
+
+PyPSA-USA allows for flexible configuration of the spatial scope of your energy model, enabling you to define the geographical area and level of detail for your simulations. The **spatial scope** is determined by the `interconnect`, `model_topology` configuration settings.
+
+- `{interconnect}` is used to select which of the three asynchronous interconnection to model. You can select `western`, `eastern`, `texas`, or `usa` to model the entire US.
+- After selecting your `{interconnect}`, you can specify `model_topology: include:` to filter individual states or balancing authorities to be selected in your model.
+
+### Example: Modeling California
+
+To create a model that includes only California, you can specify the relevant ReEDS zone IDs (p8-11) as shown below. This will limit the spatial scope to the specified regions within California.
+
+```yaml
+model_topology:
+   include:
+      reeds_zone: ['p8', 'p9', 'p10', 'p11']
+```
+
+Alternatively, you can use the reeds_state: 'CA' option to achieve the same result by specifying the entire state.
+
+<!-- ### Custom Aggregation Example
+If you'd like to create custom nodal aggregations, you can use the model_topology: aggregate: option. For instance, to cluster Arizona, Nevada, and New Mexico into a single region (WECC_SW), you would configure the model as follows:
+
+
+```yaml
+model_topology:
+   aggregate:
+      reeds_state:
+         WECC_SW : ['AZ', 'NV', 'NM']
+```
+
+This configuration will dissolve the borders between these states, effectively creating a copper-plate network where resources can be clustered and shared across the region. -->
+
+## Configuring Spatial Resolution
+
+PyPSA-USA gives you control over the spatial resolution of your transmission and resource networks, allowing for detailed or aggregated views depending on your modeling needs.
+
+### Transmission Network Selection
+You can specify the transmission network you want to use by setting the model_topology: transmission_network: option. There are two available options:
+
+- 'tamu': The synthetic BE-TAMU nodal network.
+- 'reeds': The ReEDS zonal network.
+
+### Configuring Node Clusters
+When using the ReEDS network, you need to specify the number of clusters (nodes) for your modeled interconnection. The number of nodes for each zone is **detailed in the table below**. If you're working with custom configurations, PyPSA-USA will notify you during the cluster_network stage, indicating the correct number of nodes to set in the clusters configuration.
+
+### Resource Group Resolution
+PyPSA-USA allows you to independently configure the resolution of resource zones from the transmission network. You can control this using the simpl and clusters parameters in the configuration file.
+
+For example, if you want a transmission network with 10 nodes and a resource model with 100 nodes, you would configure it as follows:
+
+```yaml
+scenario:
+   clusters: [10m]
+   simpl: [100]
+```
+This setup results in a model with 10 transmission nodes and 100 distinct resource zones, allowing for more granular modeling of renewable resource distribution while keeping the transmission network simplified.
+
+
+```{eval-rst}
+.. csv-table::
+   :header-rows: 1
+   :widths: 22,22,33
+   :file: datatables/transmission_nodes.csv
+```

docs/source/configtables/clustering.csv

@@ -6,7 +6,6 @@ feature,str," {'solar+onwind-time', 'solar+onwind-cap', 'solar-time', 'solar-cap
 cluster_network:,,,
 algorithm,str,{'kmeans'},
 feature,str," {'solar+onwind-time', 'solar+onwind-cap', 'solar-time', 'solar-cap', 'solar+offwind-cap'}",For HAC clustering.
-aggregation_zones,str,"{'balancing_area', 'state', 'reeds_zone'}",Boundaries of GIS shapes that are to be respected in clustering. Retain if you would like to analyze expansion within a given zone.
 aggregation_strategies:,,,
 table --> {key},str,"{'mean','max','min',etc}","Specifiy the method of aggregating fields within the generators, buses tables. "
 focus_weights:,,,

docs/source/configtables/lines.csv

@@ -5,4 +5,3 @@ s_nom_max,MW,float,Global upper limit for the maximum capacity of each extendabl
 max_extension,MW,float,Upper limit for the extended capacity of each extendable line.
 length_factor,--,float,Correction factor to account for the fact that buses are *not* connected by lines through air-line distance.
 interface_transmission_limits, --, true or false, Activate the Interface Transmission Limits (ITL) zones limits.
-transport_model,--,true or false,Converts model from DC PF to Transport Model by replacing lines with Links. Link p_nom value uses NARIS network. Must be used with n_clusters == minimum # of Clusters.

docs/source/data-transmission.md

@@ -29,19 +29,6 @@ The **ReEDS zonal network** is a Balancing Authority resolution transmission net
 ![ReEDS_topology](./_static/networks/ReEDS_Topology.png)
 
 
-### Usage
-
-The TAMU network is the default transmission network in PyPSA-USA, you can modify it's resolution though the `simpl` and `clusters` wildcards in the configuration files.
-
-To use the ReEDS network in PyPSA-USA, you must enable the `links: transport_model` setting, and set the proper number of `cluster` nodes for your modeled interconnection. You can find the details on number of nodes in each zone in the table below.
-
-```{eval-rst}
-.. csv-table::
-   :header-rows: 1
-   :widths: 22,22,33
-   :file: datatables/transmission_nodes.csv
-```
-
 (transmission-data)=
 ### Data
 ```{eval-rst}

docs/source/index.md

@@ -67,8 +67,9 @@ data-transportation
 :maxdepth: 1
 :hidden:
 
-config-wildcards
+config-spatial
 config-configuration
+config-wildcards
 config-sectors
 ```
 

pyproject.toml

@@ -45,11 +45,6 @@ dev = [
 [tool.setuptools.package-dir]
 myproj = "workflow/scripts"
 
-[tool.black]
-target-version = ["py37"]
-line-length = 120
-skip-string-normalization = true
-
 [tool.ruff]
 line-length = 110
 target-version = "py311"

workflow/Snakefile

@@ -128,15 +128,17 @@ def electricity_figures(wildcards):
         # maps
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/maps/{figure}",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/maps/{figure}",
                 **config["scenario"],
                 figure=FIGURES_MAPS,
             )
         )
         # emissions
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/emissions/{figure}",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/emissions/{figure}",
                 **config["scenario"],
                 figure=FIGURES_EMISSIONS,
             )
@@ -153,7 +155,8 @@ def electricity_figures(wildcards):
         # system
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/system/{figure}",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/system/{figure}",
                 **config["scenario"],
                 figure=FIGURES_SYSTEM,
             )
@@ -172,7 +175,8 @@ def sector_figures(wildcards):
     if "G" in config["scenario"]["sector"].split("-"):
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/gas/{figure}",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/gas/{figure}",
                 **config["scenario"],
                 figure=FIGURES_SECTOR_NATURAL_GAS,
             )
@@ -213,7 +217,8 @@ def sector_figures(wildcards):
         )
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/{s}/capacity/{figure}.png",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/{s}/capacity/{figure}.png",
                 **config["scenario"],
                 figure=FIGURES_SECTOR_CAPACITY,
                 s=states,
@@ -229,15 +234,17 @@ def sector_figures(wildcards):
         )
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/{s}/loads/{figure}.png",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/{s}/loads/{figure}.png",
                 **config["scenario"],
                 figure=FIGURES_SECTOR_LOADS,
                 s=states,
             )
         )
         figs.append(
             expand(
-                RESULTS + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/system/loads/{figure}.png",
+                RESULTS
+                + "{interconnect}/figures/s{simpl}_c{clusters}/l{ll}_{opts}_{sector}/system/loads/{figure}.png",
                 **config["scenario"],
                 figure=FIGURES_SECTOR_LOADS,
             )
@@ -284,7 +291,8 @@ def sector_figures(wildcards):
 def validation_figures(wildcards):
     if config["run"]["validation"]:
         return expand(
-            RESULTS + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/{figure}",
+            RESULTS
+            + "{interconnect}/figures/s{simpl}_cluster_{clusters}/l{ll}_{opts}_{sector}/{figure}",
             **config["scenario"],
             figure=FIGURES_VALIDATE,
         )

workflow/envs/environment.yaml

@@ -4,20 +4,19 @@ channels:
 - bioconda
 
 dependencies:
-- python>=3.8
+- python==3.11.9
 - pip
 
-- pypsa>=0.26.1
-- atlite>=0.2.9
-- linopy==0.3.8
-
-- dask==2023.7.0
-- dask-core==2023.7.0
+- pypsa==0.30.2
+- atlite==0.2.14
+- linopy==0.3.14
 
 # Dependencies of the workflow itself
-- pandas>=2.1
-- xarray>=2023.11.0
+- pandas==2.2.2
+- pyarrow==16.1.0
+- xarray==2024.9.0
 - xlrd==2.0.1
+- dask>=2023.7.0
 - openpyxl==3.1.2
 - pycountry==22.3.5
 - seaborn==0.13.2
@@ -34,7 +33,9 @@ dependencies:
 - progressbar2==4.3.2
 - matplotlib==3.8.0
 - plotly==5.17.0
+- pyomo==6.6.1
 - graphviz
+- cyipopt
 
 # Keep in conda environment when calling ipython
 - ipython==8.16.1
@@ -44,11 +45,10 @@ dependencies:
 
 # GIS dependencies:
 - geopy==2.4.0
-- cartopy==0.22.0
+- cartopy==0.23.0
 - descartes==1.1.0
 - rasterio==1.3.8
-- geopandas>=0.11.0
-- geopandas-base==0.14.0
+- geopandas==1.0.1
 
 # TODO: check these dependencies
 - tqdm==4.66.1
@@ -63,3 +63,4 @@ dependencies:
   - gurobipy==11.0.3
   - highspy
   - duckdb==0.10.0
+  - flake8

workflow/repo_data/config/config.common.yaml

@@ -1,6 +1,3 @@
-
-countries: [US]
-
 foresight:  'perfect'
 
 # docs :
@@ -123,6 +120,20 @@ atlite:
       dx: 0.3
       dy: 0.3
 
+lines:
+  types: # All temporary values, need to be updated
+    115.: "Al/St 240/40 2-bundle 220.0"
+    138.: "Al/St 240/40 2-bundle 220.0"
+    161.: "Al/St 240/40 2-bundle 220.0"
+    230.: "Al/St 240/40 2-bundle 220.0"
+    345.: "Al/St 240/40 4-bundle 380.0"
+    500.: "Al/St 560/50 4-bundle 750.0"
+    765.: "Al/St 560/50 4-bundle 750.0"
+
+model_topology:
+  interface_transmission_limits: false
+  topological_boundaries: 'reeds_zone' # [reeds_zone]
+
 
 # docs :
 offshore_shape:

workflow/repo_data/config/config.default.yaml

@@ -18,6 +18,14 @@ scenario:
   sector: "" # G
   planning_horizons: [2030, 2040, 2050]
 
+# docs :
+model_topology:
+  transmission_network: 'reeds' # [reeds, tamu]
+  include: # mixed zone types not supported
+    reeds_zone: ['p62']
+    # reeds_state: ['CO']
+    # reeds_ba: ['ERCO']
+
 # docs :
 enable:
   build_cutout: false
@@ -31,7 +39,7 @@ snapshots:
 # docs :
 electricity:
   conventional_carriers: [nuclear, oil, OCGT, CCGT, coal, geothermal, biomass, waste] # Choose the conventional plant types to include in network
-  renewable_carriers: [onwind, offwind, offwind_floating, solar, hydro] # Choose the renewable plant types to include in network
+  renewable_carriers: [onwind, solar, hydro] # Choose the renewable plant types to include in network
   voltage_simplified: 230 #Voltage level to simplify network to in rule "simplify network"
   co2limit: 1.4728e+9 # 0.8 * 1.841e+9
   co2limit_enable: false # For sector coupled studies
@@ -57,7 +65,7 @@ electricity:
     H2: 168
 
   extendable_carriers:
-    Generator: [solar, onwind, offwind, offwind_floating, OCGT, CCGT, coal, nuclear, coal-95CCS, CCGT-95CCS, SMR] #include CCGT-CCS
+    Generator: [solar, onwind, OCGT, CCGT, coal, nuclear, coal-95CCS, CCGT-95CCS, SMR] #include CCGT-CCS
     StorageUnit: [4hr_battery_storage, 8hr_battery_storage] # [Xhr-battery-storage (2-10 hours)]
     Store: []
     Link: [] 
@@ -82,8 +90,6 @@ lines:
   s_nom_max: .inf
   max_extension: 2000 #MW
   length_factor: 1.25
-  interface_transmission_limits: false
-  transport_model: false
 
 # docs :
 links:
@@ -154,7 +160,6 @@ clustering:
   cluster_network:
     algorithm: kmeans # choose from: [hac, kmeans]
     feature: solar+onwind-time
-    aggregation_zones: 'reeds_zone' # [balancing_area, state, reeds_zone]
     exclude_carriers: []
     consider_efficiency_classes: false
   aggregation_strategies: