Preparation for public release

.github/workflows/docker.yml

@@ -0,0 +1,59 @@
+name: Build Docker Image
+
+on:
+  push:
+    branches: [main, dev, ci]
+    tags:
+      - 'v*'
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository_owner }}/demos
+
+jobs:
+  publish:
+    name: Build and publish Docker image
+    runs-on: ubuntu-latest
+    if: github.event_name != 'pull_request' && github.actor != 'dependabot[bot]'
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Log in to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Docker meta (tags)
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=sha,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
+            type=ref,event=branch
+            type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
+            type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
+
+      - run: echo "${{ steps.meta.outputs.tags }}"
+      - name: Build and push
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          push: true
+          platforms: linux/amd64
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max

README.md

@@ -1,34 +1,66 @@
 # Demographic Microsimulator (DEMOS)
 
-[![Docs](https://github.com/NREL/DEMOS_NREL/actions/workflows/docs.yml/badge.svg)](https://nrel.github.io/DEMOS_NREL/)
+[![Docs](https://github.com/NREL/DEMOS/actions/workflows/docs.yml/badge.svg)](https://nrel.github.io/DEMOS/)
+
+## Overview
+The Demographic Microsimulator (DEMOS) is an agent-based simulation framework used to model the evolution of population demographic characteristics and lifecycle events, such as education attainment, marital status, and other key transitions. DEMOS modules are designed to capture the interdependencies between short-term and long-term lifecycle events, which are often influential in downstream transportation and land-use modeling.
+
+A key feature of DEMOS is its ability to track changes in an agent’s demographic status from year *t* to year *t + 1*. This structure allows the model to evolve populations over any user-defined time horizon. As a result, DEMOS is well suited for analyzing medium- and long-term transportation-related decisions, including household vehicle transactions (e.g., purchasing, selling, or replacing vehicles) and work location choices.
+Core features of DEMOS include the modeling of more than ten lifecycle events, behaviorally realistic patterns informed by long-running panel data, explicit representation of interdependencies among lifecycle processes, and a flexible, modular simulation architecture.
+
+A technical memorandum describing DEMOS is available [here](./DEMOS_Technical_Memo.pdf). The memorandum provides an overview of the framework’s functionality, model structure, input and output data, and its applications in transportation planning and broader policy analysis contexts. Interested readers are also encouraged to consult the paper listed below for additional details on the DEMOS methodology.
+
+
+*Sun, Bingrong, Shivam Sharda, Venu M. Garikapati, Mohamed Amine Bouzaghrane, Juan Caicedo, Srinath Ravulaparthy, Isabel Viegas de Lima, Ling Jin, C. Anna Spurlock, and Paul Waddell. "Demographic Microsimulator for Integrated Urban Systems: Adapting Panel Survey of Income Dynamics to Capture the Continuum of Life." Transportation Research Record (2025): 03611981251333339.*
 
 ## Usage
-> A public Docker image of DEMOS has not been released. Please follow the `From Source` instructions.
 
-### Docker Container
-The docker image for demos is stored in `registry/demos:latest`. The input data and configuration file are fed to the container through volumes. Alternatively, we provide a `docker-compose` workflow that can be used.
+### Docker Compose (recommended)
+The latest docker image for demos is stored in `ghcr.io/nrel/demos:latest`. The input data and configuration file are fed to the container through volumes. Alternatively, we provide a `docker-compose` workflow that can be used.
+
+#### Prepare the configuration file and data folder
 
-For running the `docker-compose` workflow:
 ```bash
-DEMOS_CONFIG_PATH=<path-to-config> DEMOS_DATA_DIR=<path-to-data-dir> docker-compose up
+# Create a directory where to run DEMOS from
+mkdir demos
+cd demos
+
+# Create the configuration folder and retrieve an example configuration
+mkdir configuration
+cd configuration
+curl -L -o demos_config.toml https://raw.githubusercontent.com/nrel/DEMOS/main/configuration/demos_config_sfbay.toml
+
+# Create the data folder for the output to be stored
+cd ..
+mkdir data
+# Populate the data folder
+
+# Finally, retrieve the docker-compose.yml file
+curl -L -o docker-compose.yml https://raw.githubusercontent.com/nrel/DEMOS/main/docker-compose.yml
 ```
 
-By default `DEMOS_CONFIG_PATH` is set to `./demos_config.toml` and `DEMOS_DATA_DIR` is set to `./data`, so if `data` and `demos_config.toml` are part of the current directory, no additional input is needed.
-
-Alternatively,
+Now you can run docker as follows:
 ```bash
-docker run --volume <path-to-config>:/demos/config.toml:ro --volume <path-to-data-dir>:/demos/data --platform=linux/amd64 demos
+docker compose up
 ```
-
 #### IMPORTANT for MacOS and Windows users
 > Docker imposes a global limit on how much RAM containers can allocate. DEMOS easily surpases those limits, so in order to run DEMOS in Docker, users need to access the Docker Desktop GUI and `Preferences → Resources → Memory → Increase it (at least 16-20gb)`
 
-#### Building the docker image (development only)
+Documentation for custom data requirements, configuration and overall functionality of demos can be found [in the Docs](https://nrel.github.io/DEMOS/).
+
+## Other ways to run DEMOS
+
+If you need to change either the data or configuration path:
 ```bash
-docker build -t demos:0.0.1 --platform=linux/amd64 -f Dockerfile .
+DEMOS_CONFIG_PATH=<path-to-config> DEMOS_DATA_DIR=<path-to-data-dir> docker compose up
 ```
 
-### From Source
+Alternatively, if you prefer not to use docker compose, you can specify the ccreation of volumes to your data and configuration as follows:
+```bash
+docker run --volume <path-to-config>:/demos/config.toml:ro --volume <path-to-data-dir>:/demos/data --platform=linux/amd64 ghcr.io/nrel/demos:latest
+```
+
+### Running from Source
 
 1. Clone this repository
 	```
@@ -48,30 +80,3 @@ docker build -t demos:0.0.1 --platform=linux/amd64 -f Dockerfile .
 	conda activate demos-env
 	pip install .
 	```
-
-## Running DEMOS
-
-DEMOS requires a series of input tables. Example tables are provided [here](https://app.box.com/s/tox2nflumia2g4n6rk2i0navca9pskep) for internal NREL use. It is recommended to store all the input values in the folder `data` in root of the project, but absolute values can be used by specifying them in the configuration file. You may also refer to the data description [here](https://cloud.urbansim.com/docs/general/documentation/urbansim%20block%20model%20data.html)
-
-To run demos:
-```
-python simulate.py -cfg {configuration_file}
-```
-
-
-
-A default configuration file is provided in `configuration/demos_config.toml`. The `[[tables]]` entries outline tables to be loaded. For example, we can load the `persons` and `households` table from an H5 source:
-
-```toml
-[[tables]]
-file_type = "h5"
-table_name = "persons"
-filepath = "../data/custom_mpo_06197001_model_data.h5"
-h5_key = "persons"
-
-[[tables]]
-file_type = "h5"
-table_name = "households"
-filepath = "../data/custom_mpo_06197001_model_data.h5"
-h5_key = "households"
-```

demos/models/income_adjustment.py

@@ -43,7 +43,9 @@ def income_adjustment(persons, households, income_rates, year):
     start_time = time.time()
     # TODO: CountyID is not being updated by default
     # Update income according to county rate
-    income_rates['lcm_county_id'] = income_rates['lcm_county_id'].map(lambda x: f'{x:0>5}')
+    income_rates["lcm_county_id"] = income_rates["lcm_county_id"].map(
+        lambda x: f"{x:0>5}"
+    )
 
     persons.local.earning *= (
         1

demos/models/kids_moving.py

@@ -138,7 +138,14 @@ def run_and_calibrate_model(persons):
     demos_config: DEMOSConfig = get_config()
     module_config: KidsMovingModuleConfig = demos_config.kids_moving_module_config
 
-    child_relate = [2, 3, 4, 7, 9, 14] # This is more `dependent` because `child` is determined by age
+    child_relate = [
+        2,
+        3,
+        4,
+        7,
+        9,
+        14,
+    ]  # This is more `dependent` because `child` is determined by age
     target_share = module_config.calibration_target_share
 
     # Get model data
@@ -149,25 +156,29 @@ def run_and_calibrate_model(persons):
     kids_moving = model.predict(model_data).astype(int)
 
     # NOTE: This could be much easier if we set the age at 18 because we could use model_filters
-    adult_filter = (persons.age >= 18)
+    adult_filter = persons.age >= 18
     age_moved = persons.age.loc[kids_moving[kids_moving == 1].index]
-    adult_stay = (adult_filter & persons.relate.isin(child_relate)).sum() - (age_moved >=18).sum()
+    adult_stay = (adult_filter & persons.relate.isin(child_relate)).sum() - (
+        age_moved >= 18
+    ).sum()
     observed_share = adult_stay / adult_filter.sum()
-    error = (observed_share - target_share)
+    error = observed_share - target_share
 
     print("Calibrating Kids moving model")
     calibrate_iteration = 0
-    while abs(error) > module_config.calibration_tolerance: 
+    while abs(error) > module_config.calibration_tolerance:
         print(f"{calibrate_iteration} iteration error: {error}")
-        model.fitted_parameters[0] += np.log(observed_share/target_share)
+        model.fitted_parameters[0] += np.log(observed_share / target_share)
 
         kids_moving = model.predict(model_data).astype(int)
         age_moved = persons.age.loc[kids_moving[kids_moving == 1].index]
-        adult_stay = (adult_filter & persons.relate.isin(child_relate)).sum() - (age_moved >=18).sum()
+        adult_stay = (adult_filter & persons.relate.isin(child_relate)).sum() - (
+            age_moved >= 18
+        ).sum()
         observed_share = adult_stay / adult_filter.sum()
-        error = (observed_share - target_share)
+        error = observed_share - target_share
 
         calibrate_iteration += 1
     print(f"{calibrate_iteration} iteration error: {error}")
 
-    return kids_moving
+    return kids_moving

demos/models/marriage.py

@@ -201,8 +201,9 @@ def update_married_households_random(
             module_config.geoid_col,
         ].values
         households.local.loc[new_hh_ids, module_config.geoid_col] = new_hh_geoid
-    county_assignment = households.local.loc[all_df.loc[first_index & neither_head_index, "household_id"], 
-                                             "lcm_county_id"].values
+    county_assignment = households.local.loc[
+        all_df.loc[first_index & neither_head_index, "household_id"], "lcm_county_id"
+    ].values
     households.local.loc[new_hh_ids, "lcm_county_id"] = county_assignment
     ## Decide who is household head in the households where the head left
     head_left_index = (all_df.relate == 0) & (all_df.household_id != all_df.new_hh_id)
@@ -279,9 +280,7 @@ def update_divorce(persons, households, divorce_list, get_new_households):
         geoid_assignment = households.local.loc[
             old_household_id, module_config.geoid_col
         ].values
-        households.local.loc[new_households, module_config.geoid_col] = (
-            geoid_assignment
-        )
+        households.local.loc[new_households, module_config.geoid_col] = geoid_assignment
     county_assignment = households.local.loc[old_household_id, "lcm_county_id"].values
     households.local.loc[new_households, "lcm_county_id"] = county_assignment
 

demos/variables.py

@@ -632,6 +632,7 @@ def marital34(persons):
     p = persons.to_frame(columns=["MAR"])
     return p.isin([3, 4]).astype(int)
 
+
 ######## NOTE: Not needed for now
 # # PERSON VARIABLES
 # # -----------------------------------------------------------------------------------------

docker-compose.yml

@@ -3,7 +3,7 @@ services:
     build:
       context: .
       dockerfile: Dockerfile
-    image: demos:0.0.1
+    image: ghcr.io/${GITHUB_REPOSITORY_OWNER:-nrel}/demos
     tty: true
     platform: linux/amd64
     environment:

docs/source/pages/intro.md

@@ -1,6 +1,16 @@
 # Get Started
 
-DEMOS is a modular demographic microsimulator. It operates on tabular data representing agents or entities (primarily persons and households), and is configured via a simple TOML file. DEMOS can be run from source or using Docker for reproducibility.
+## Overview
+The Demographic Microsimulator (DEMOS) is an agent-based simulation framework used to model the evolution of population demographic characteristics and lifecycle events, such as education attainment, marital status, and other key transitions. DEMOS modules are designed to capture the interdependencies between short-term and long-term lifecycle events, which are often influential in downstream transportation and land-use modeling.
+
+A key feature of DEMOS is its ability to track changes in an agent’s demographic status from year *t* to year *t + 1*. This structure allows the model to evolve populations over any user-defined time horizon. As a result, DEMOS is well suited for analyzing medium- and long-term transportation-related decisions, including household vehicle transactions (e.g., purchasing, selling, or replacing vehicles) and work location choices.
+Core features of DEMOS include the modeling of more than ten lifecycle events, behaviorally realistic patterns informed by long-running panel data, explicit representation of interdependencies among lifecycle processes, and a flexible, modular simulation architecture.
+
+A technical memorandum describing DEMOS is available [here](https://github.com/NREL/DEMOS/blob/main/DEMOS_Technical_Memo.pdf). The memorandum provides an overview of the framework’s functionality, model structure, input and output data, and its applications in transportation planning and broader policy analysis contexts. Interested readers are also encouraged to consult the paper listed below for additional details on the DEMOS methodology.
+
+---
+
+DEMOS operates on tabular data representing agents or entities (primarily persons and households), and is configured via a simple TOML file. DEMOS can be run from source or using Docker for reproducibility.
 
 This document summarizes instructions to install, configure and run DEMOS. Sections 1-4 will help you correctly organize the data and configuration file, so we recommend reading them once before attempting to run DEMOS.
 
@@ -15,50 +25,56 @@ This document summarizes instructions to install, configure and run DEMOS. Secti
 
 ## 1. Installation
 
-### Using Docker (Recommended)
+### Docker Compose (recommended)
+The latest docker image for demos is stored in `ghcr.io/nrel/demos:latest`. The input data and configuration file are fed to the container through volumes. Alternatively, we provide a `docker-compose` workflow that can be used.
+
+#### Prepare the configuration file and data folder
 
+```bash
+# Create a directory where to run DEMOS from
+mkdir demos
+cd demos
+
+# Create the configuration folder and retrieve an example configuration
+mkdir configuration
+cd configuration
+curl -L -o demos_config.toml https://raw.githubusercontent.com/nrel/DEMOS/main/configuration/demos_config_sfbay.toml
+
+# Create the data folder for the output to be stored
+cd ..
+mkdir data
+# Populate the data folder
+
+# Finally, retrieve the docker-compose.yml file
+curl -L -o docker-compose.yml https://raw.githubusercontent.com/nrel/DEMOS/main/docker-compose.yml
+```
+
+Now you can run docker as follows:
+```bash
+docker compose up
+```
 > **Note:**  
 > Make sure the Docker Daemon is running. This changes from system to system but Docker Desktop should have a status flag indicating if the daemon is live, if Desktop is available
 
 <!-- **Important Note:**
 > While the pipeline to build a docker image is implemented, there is no public docker image available, please execute [from source](#From-Source) -->
 
-1. **Clone the repository**:
-    ```bash
-    git clone https://github.com/NREL/DEMOS_NREL.git
-    cd DEMOS_NREL
-    ```
-
-    **Build Docker Image** *(Development only)*
-    > NOTE: Ideally, our DEMOS Docker image is hosted in a public image registry so users will not need to build it themselves
-    ```bash
-    docker build -t demos:0.0.1 --platform=linux/amd64 -f Dockerfile .
-    ```
 
-1. **Run with Docker Compose**:
-    ```bash
-    docker compose up
-    ```
-
-    By default, this assumes that your config file is located in `./configuration/demos_config.toml` and the data folder is `./data`, with `./` being the root of the project (See the [file stucture section](#file-tree-structure-for-data-and-configuration) for details on how to organize the input data).
-    If you need to specify a different location for them, you can run:
 
-    ```bash
-    DEMOS_CONFIG_PATH=<path-to-config> DEMOS_DATA_DIR=<path-to-data-dir> docker compose up
-    ```
+By default, this assumes that your config file is located in `./configuration/demos_config.toml` and the data folder is `./data`, with `./` being the root of the project (See the [file stucture section](#file-tree-structure-for-data-and-configuration) for details on how to organize the input data).
+If you need to specify a different location for them, you can run:
 
-<!-- 1. **Or run with Docker directly**:
-    ```bash
-    docker run --volume <path-to-config>:/demos/config.toml:ro --volume <path-to-data-dir>:/demos/data --platform=linux/amd64 demos
-    ``` -->
+```bash
+DEMOS_CONFIG_PATH=<path-to-config> DEMOS_DATA_DIR=<path-to-data-dir> docker compose up
+```
 
 > **Note for MacOS/Windows:**  
 > Increase Docker's memory allocation to at least 16–20 GB via Docker Desktop:  
 > `Preferences → Resources → Memory`.
 
 ---
 
-### From Source
+### From Source (is you are not using Docker)
 
 1. **Clone the repository**:
     ```bash
@@ -79,14 +95,6 @@ This document summarizes instructions to install, configure and run DEMOS. Secti
     python simulate.py -cfg ../configuration/demos_config.toml
     ```
 
-### Compiling documentation (Optional but recommended)
-From the root of the project:
-```bash
-cd docs
-make html
-open build/html/index.html
-```
-
 
 ## 2. Preparing Your Configuration