[Refactor] Rename/define Scenario variables so they make sense when building tiered scenarios

[lmz]

User Story

As a travel modeler who is building networks, I would like to apply projects for a given set of tags without creating a new scenario instance.

Maybe this is already doable using scenario.add_project_cards_from_tags() -- what do these add methods do? Is there a list of cards that are waiting to be applied?

Maybe I'm looking for another layer of cards, like a card set, rather than just the single list in scenario.project_cards?

Right now, our notebook creates new scenario instances over and over just to apply cards for certain tags, which seems unncessary. (Also, this notebook has a huge memory footprint which likely is related...)

Priority

Level of Effort

Resolution Ideas

Project

MTC

Who should be involved?

Implementer: I am happy do this with some guidance
Commenters:
Users:
Reviewers:

Risk

Will this potentially break anything?

Tests

What are relevant tests or what tests need to be created in order to determine that this issue is complete?

[e-lo]

I think the functionality for tags is already there. Network Wrangler does something similar to git when adding new cards: first they are added and then they are applied (~ committed)

scenario.add_project_cards_from_tags() calls scenario.add_project_cards_from_file() which adds the project cards to scenario.project_cards instance.

Calling scenario.apply_all_proejcts() then applies all the projects in scenario.project_cards.

[e-lo]

Maybe I'm looking for another layer of cards, like a card set, rather than just the single list in scenario.project_cards?

Are you thinking similar to the "plans" we had in the og network wrangler 1.0?

[lmz]

Ahh got it. So I think I would like to do something like this
(Reference: https://github.com/BayAreaMetro/NetworkWrangler/blob/master/scripts/net_spec_blueprint.py
)

# Apply base roadway fixes first
scenario.add_project_cards_from_tags(folder=card_path, tags=['Managed Lanes', 'toll review', 'Exclude Trucks'])
scenario.apply_all_projects()
# [do checks or whatever, possibly save it out]
scenario.remove_all_projects()

scenario.add_project_cards_from_tags(folder=card_path, tags=['Major Transit links'])
scenario.apply_all_projects()
# [do checks or whatever, possibly save it out]
scenario.remove_all_projects()

# This is what will be most clear for planners, see net_spec_blueprint.py link above
PLAN_PROJECTS = collections.OrderedDict([
(2020: [
    'proj1',
    'proj2',
    'proj3']),
(2025: [
    'proj4',
    'proj5'])
])
 
for year in PLAN_PROJECTS.keys():
    scenario.add_project_cards_from_list(PLAN_PROJECTS[year])
    scenario.apply_all_projects()
    scenario.write(output_path[year])
    scenario.remove_all_projects()

[e-lo]

scenario.remove_all_projects()
--> not sure why this is necessary if you want to layer on?

[e-lo]

I think this should work - what is it not happy about?

for year in PLAN_PROJECTS.keys():
    scenario.add_project_cards_from_list(PLAN_PROJECTS[year])
    scenario.apply_all_projects()
    scenario.write(output_path[year])

[lmz]

Ahh... I was under the impression that rerunning scenario.apply_all_projects() without clearing them would apply again? Is it checking for ones already applied? Some projects are made to apply multiple times but most of them are not, so how are they distinguished?

[e-lo]

AH - looking at it more I think you are right. I guess we could do either:

1. add remove to your process

for year in PLAN_PROJECTS.keys():
    scenario.add_project_cards_from_list(PLAN_PROJECTS[year])
    scenario.apply_all_projects()
    scenario.write(output_path[year])
    scenario.remove_all_projects()

2. remove the project cards as default in apply_project()

Either would be straightforward to implement...

[lmz]

I think it would be intuitive to have scenario.apply_all_projects() remove the project from the project_cards list once it's applied.

scenario.apply_project() is a little trickier since it's taking the project card as an arg

(In the meantime, I am invoking scenario.remove_all_projects() after applying as a workaround.

[e-lo]

@lmz @DavidOry @i-am-sijia @kulshresthaa Thinking about this some more, I'd like to propose changing some attribute definitions 😨 😬 so we are:

• (A) super explicit about what wrangler is doing with each function/stage of network building and
• (B) can have the functionality that MTC (and likely other agencies) want without having to use confusing terminology (e.g. "remove_all projects" seems like it is "reversing" them from the network scenario)

Current Definitions

• Scenario.project_cards: Collection of Project card instances that have been either queued or applied; in proposed implementation order .
• Scenario.projects: projects that have been queued for
• Scenario.ordered_project_cards: filled when project
• Scenario.applied_projects :
• There are also two "state" variables: requisites_checked and conflicts_checked

Issues

1. There are meaningful changes to the definitions of what is in the variables above which can vary not only on the state variables, but also other activities.
2. Difficult to understand what each means
3. A lot of the "state" variable situations could be more easily taken care of in a @Property based variable definition.

Proposed Definitions

• Scenario.project_cards: Mapping[ProjectCard.name,ProjectCard] Storage of all project cards by name; helps prevent naming conflicts.
• Scenario._planned_projects: Collection[ProjectCard.name] Initial ordering of project cards which are awaiting to be queued for adding. Similar to a git working directory, project cards aren't recognized in this collection once they are moved to staging
• Scenario._queued_projects: Collection[ProjectCard.name] Projects which are "shovel ready": have had pre-requistes checked and done any required re-ordering. Similar to a git staging, project cards aren't recognized in this collection once they are moved to applied
• Scenario.queued_projects: @Property wrapper on _queued_projects which lazily queues (checks requirements/conflicts and orders) anything already in Scenario._queued_projects and anything in Scenario.planned_projects (and removes queued projects from planned_projects)
• Scenario.applied_projects: Collection[ProjectCard.name] Projects which have been applied to the network.
• No more "state" variables: requisites_checked and conflicts_checked: as this is the definitional difference between planned and queued
• Scenario.projects: Sum of applied, queued.

Corresponding function definitions would be:

• create_scenario(project_cards_list=project_cards_list) : adds project card instances to Scenario.project_cards and project card names to Scenario.planned_projects
• my_scenario.apply_all_projects(): applies Scenario.queued_projects, removing from queue once they have been applied and adding project name to Scenario.applied_projects`

Considerations:

• This makes the queuing implicit, not explicit...would we want to change that or is that an ok expected behavior?

[DavidOry]

I like it. Agree that Scenario.projects is ambiguous and @lmz's use case seems common. The fact that it wasn't obvious to her how to do this (apply projects, look at things, and then apply some more) is compelling motivation to change.

[e-lo]

Implemented.