Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config Settings and Documentation #758

Merged
merged 459 commits into from
Feb 1, 2024
Merged
Changes from 14 commits
Commits
Show all changes
459 commits
Select commit Hold shift + click to select a range
1681445
yaml.dump paths to strings
jpn-- May 5, 2023
aa1b058
logit configs in separate file [makedocs]
jpn-- May 5, 2023
9b78306
configs for trip dest
jpn-- May 8, 2023
69b93c8
cleanup
jpn-- May 8, 2023
a918c9c
deprecate DESTINATION_SAMPLE_SPEC and DESTINATION_SPEC
jpn-- May 8, 2023
18ec447
cfg for mandatory locations
jpn-- May 9, 2023
a349d93
shadow pricing cfg
jpn-- May 9, 2023
609086b
COEFFS file is optional
jpn-- May 9, 2023
0f9cb3b
common tour location component settings
jpn-- May 9, 2023
d382bb5
pop include_settings
jpn-- May 10, 2023
95c01ab
fix SHADOW_PRICE_METHOD
jpn-- May 10, 2023
cc452cc
fix bugs
jpn-- May 10, 2023
3fa10a6
repair
jpn-- May 10, 2023
45e057b
TourODSettings
jpn-- May 10, 2023
6fad550
robust write_spec
jpn-- May 10, 2023
4037af5
accessibility config
jpn-- May 12, 2023
ef4346a
tour mode
jpn-- May 15, 2023
ee71901
prefer float to str
jpn-- May 15, 2023
9806b3b
clean up
jpn-- May 16, 2023
7f4eb0b
overflow protection in utils_to_probs
jpn-- May 17, 2023
c40632a
only enable overflow_protect when not allow_zero_probs
jpn-- May 17, 2023
532e9e9
logsums w probs
jpn-- May 17, 2023
c6b15c7
tweaks in regression test due to overflow protection
jpn-- May 18, 2023
dad22d0
Merge branch 'develop' into overflow-protection
jpn-- May 19, 2023
52b8b79
Merge branch 'overflow-protection' into cfg-TourMode
jpn-- May 19, 2023
3a3088d
test logging
jpn-- May 25, 2023
f1e894d
annotate rng
jpn-- May 25, 2023
b483556
Merge branch 'upstream-develop' into generic-whale
jpn-- May 25, 2023
325eb3d
update semcog model
jpn-- May 25, 2023
939f323
docs for logging
jpn-- May 25, 2023
5d4c4db
fix semcog example test
jpn-- May 25, 2023
6b9d8b8
only one add_size_tables func
jpn-- May 25, 2023
d5fdfe1
better error msg
jpn-- May 25, 2023
c84925d
remove extra lines
jpn-- May 25, 2023
a2bb99d
multiple output dirs for tests
jpn-- May 25, 2023
f35fa59
push recoding
jpn-- Jun 1, 2023
1956e87
recoding both ways
jpn-- Jun 1, 2023
027a575
recoding on mp
jpn-- Jun 2, 2023
b851226
Merge branch 'fix-684' into cfg-TourMode
jpn-- Jun 2, 2023
dee5ea4
avoid error sorting on no index
jpn-- Jun 2, 2023
19bc4f7
validate settings
jpn-- Jun 2, 2023
099bcd0
public BaseLogitComponentSettings
jpn-- Jun 19, 2023
6160f4b
fix for chunking
jpn-- Jun 19, 2023
f626778
blacken
jpn-- Jun 19, 2023
d2992f0
remove vestigial log_df
jpn-- Jun 19, 2023
038a408
undo junk
jpn-- Jun 19, 2023
4963260
docstring
jpn-- Jun 19, 2023
a3e430d
write log to output dir
jpn-- Jun 20, 2023
11d2902
more docs on logging
jpn-- Jun 20, 2023
59e8dd7
bigger chunksize, too small times out
jpn-- Jun 20, 2023
ad9e23b
clean up when missing TAZ input table
jpn-- Jun 20, 2023
c592f94
historian
jpn-- Jun 20, 2023
ee7bc37
allow state to be None
jpn-- Jun 20, 2023
8435976
change recode_pipeline_columns default to false
jpn-- Jun 21, 2023
2f4bec5
update prototype_psrc_in_development
jpn-- Jun 21, 2023
5868bb3
fix cdap for sharrow
jpn-- Jun 21, 2023
4c4838c
import skim_dataset
jpn-- Jun 21, 2023
22eee5b
recode pipeline for mtc
jpn-- Jun 21, 2023
0f01901
Multiprocessing Performance
jpn-- Jun 22, 2023
a45304b
Arithmetic on Logical Values
jpn-- Jun 23, 2023
c8c492b
clarify
jpn-- Jun 23, 2023
318d7f7
Merge branch 'generic-whale' into cfg-TourMode
jpn-- Jun 26, 2023
831ec5c
annotations
jpn-- Jun 26, 2023
eb38e6b
config for joint tour freq
jpn-- Jun 26, 2023
9bfd5ee
TimeSettings
jpn-- Jun 30, 2023
1c065ac
allow deprecated hours
jpn-- Jun 30, 2023
84f1829
Merge branch 'orca-cleanup' into time-settings
jpn-- Aug 7, 2023
5b5e401
test commit
vivekyadav26 Sep 7, 2023
7300a90
updating work from home documentation
vivekyadav26 Sep 8, 2023
2c9b071
update model settings of work from home model
vivekyadav26 Sep 9, 2023
b392750
school and work location choice documentation
vivekyadav26 Sep 18, 2023
9fb33ec
auto ownership
vivekyadav26 Sep 19, 2023
5f6033d
accessibility documentation
vivekyadav26 Sep 19, 2023
089d9c8
format fix
vivekyadav26 Sep 19, 2023
c5f51d4
Merge branch 'time-settings' into cfg-doc-2
jpn-- Sep 19, 2023
4775eda
auto_ownership is a logit model
jpn-- Sep 19, 2023
7eb97db
docbuild uses pydantic 1.10
jpn-- Sep 19, 2023
617530a
add more components to docs
jpn-- Sep 19, 2023
4657ec3
ao docs [makedocs]
jpn-- Sep 19, 2023
3c45b16
Adding examples to accessibility
vivekyadav26 Sep 19, 2023
fbcc9a1
Merge branch 'cfg-doc-2' into cfg-documentation-vy
jpn-- Sep 19, 2023
372da21
updated transit pass subsidy model
vivekyadav26 Oct 24, 2023
fe60d15
Updating model components
vivekyadav26 Oct 24, 2023
fb3f72e
update telecommute frequency
vivekyadav26 Oct 24, 2023
ba9ec40
updating telecommute frequency and mandatory tour frequency
vivekyadav26 Oct 24, 2023
37eaa4f
update joint tout composition
vivekyadav26 Oct 24, 2023
7bf96fc
update joint tour participation. There is model_settings.get that nee…
vivekyadav26 Oct 24, 2023
dba895d
Update joint tour scheduling
vivekyadav26 Oct 24, 2023
84106a4
Update non mandatory tour frequency. Needs closer review
vivekyadav26 Oct 24, 2023
fa10214
update vehicle allocation and atwork subtour frequency
vivekyadav26 Oct 24, 2023
5fe2603
update several models
vivekyadav26 Oct 24, 2023
1866330
updated documentation for few models
navsarmajs Oct 27, 2023
187633f
Merge branch 'cfg-documentation-vy' into Branch_nav_cfg-documentation-vy
navsarmajs Oct 27, 2023
95d5a9a
Updating component documentation
vivekyadav26 Oct 27, 2023
a26a1e4
bug fix
vivekyadav26 Oct 27, 2023
7151fc9
fix bug
vivekyadav26 Oct 27, 2023
982973e
how to build docs [makedocs]
jpn-- Oct 27, 2023
2b667d1
Added component documentation for more models
navsarmajs Oct 30, 2023
3507e67
Merge branch 'cfg-documentation-vy' into Branch_nav_cfg-documentation-vy
navsarmajs Oct 30, 2023
66d6989
Documented more model componets and added to index toctree
navsarmajs Oct 30, 2023
257b5aa
Model Component Documentation
navsarmajs Oct 30, 2023
8fe0ea3
updating stop frequency, trip purpose and trip scheduling documentation
vivekyadav26 Oct 30, 2023
4df5b32
Merge branch 'cfg-documentation-vy' into cfg-documentation-navjs
navsarmajs Oct 30, 2023
db2a5c4
updating component documentation
vivekyadav26 Oct 30, 2023
f415d43
Update vehicle allocation and build docs [makedocs]
vivekyadav26 Oct 30, 2023
b10c538
documentation fixes [makedocs]
vivekyadav26 Oct 30, 2023
1f93167
minor fixes [makedocs]
vivekyadav26 Oct 30, 2023
2eb75f9
Added Documentation for more models
navsarmajs Oct 30, 2023
ba2d165
Adding annotate_tables settings class
vivekyadav26 Oct 30, 2023
246e611
Update cdap.md
vivekyadav26 Oct 31, 2023
195c1ed
Update disaggregate_accessibility.md
vivekyadav26 Oct 31, 2023
d990db3
Update initialize.md
vivekyadav26 Oct 31, 2023
83c1b2d
Update initialize_los.md
vivekyadav26 Oct 31, 2023
90bb7a2
Update mandatory_scheduling.md
vivekyadav26 Oct 31, 2023
e9acd33
Update mandatory_tour_frequency.md
vivekyadav26 Oct 31, 2023
2d558c4
Update telecommute_frequency.md
vivekyadav26 Oct 31, 2023
d9e7c93
Update telecommute_frequency.md
vivekyadav26 Oct 31, 2023
538f405
Update transit_pass_ownership.md
vivekyadav26 Oct 31, 2023
28b3fa3
Update transit_pass_subsidy.md
vivekyadav26 Oct 31, 2023
b0e6131
Update vehicle_type_choice.md
vivekyadav26 Oct 31, 2023
d5b24dd
Merge branch 'cfg-documentation-vy' into cfg-documentation-navjs
vivekyadav26 Oct 31, 2023
200d73d
Merge pull request #11 from camsys/cfg-documentation-navjs
vivekyadav26 Oct 31, 2023
0b7efcf
Updated documentation for joint tour compostion model
navsarmajs Oct 31, 2023
42abe71
Merge branch 'cfg-documentation-navjs' of https://github.com/camsys/a…
navsarmajs Oct 31, 2023
c9615ce
Added documentation for more models and minor bug fixes
navsarmajs Oct 31, 2023
5f16583
Update non_mandatory_scheduling.md
vivekyadav26 Nov 1, 2023
13cfa28
Merge pull request #12 from camsys/cfg-documentation-navjs
vivekyadav26 Nov 1, 2023
a24d8e4
minor edits [makedocs]
vivekyadav26 Nov 6, 2023
7f5693e
verbose action [makedocs]
jpn-- Oct 27, 2023
26d54c6
bump pytables to 3.9
jpn-- Nov 6, 2023
199ae2e
pin autodoc_pydantic<2.0
jpn-- Nov 6, 2023
ff87616
[makedocs]
jpn-- Nov 6, 2023
fc5d9a5
escorting
jpn-- Nov 6, 2023
73fc620
Merge pull request #13 from camsys/cfg-docs-jn-2
vivekyadav26 Nov 7, 2023
b40dffc
[makedocs]
vivekyadav26 Nov 7, 2023
34adea1
Merge branch 'cfg-documentation-vy' of https://github.com/camsys/acti…
vivekyadav26 Nov 7, 2023
4f5d56d
minor edits [makedocs]
vivekyadav26 Nov 7, 2023
4be2058
test initialize
vivekyadav26 Nov 7, 2023
fc17ced
test
vivekyadav26 Nov 7, 2023
501430f
AnnotateTableSettings
jpn-- Nov 7, 2023
86c3820
init tables things
jpn-- Nov 7, 2023
ac5ca47
bug fixes [makedocs]
vivekyadav26 Nov 7, 2023
8b9596c
Merge branch 'cfg-documentation-vy' into cfg-docs-jn-escort
jpn-- Nov 9, 2023
529a4df
documentation updates [makedocs]
vivekyadav26 Nov 9, 2023
dead81a
Merge commit '529a4df3f18c2f44d856b4339c7c391e503b7f67' into cfg-docs…
jpn-- Nov 9, 2023
237c0c6
more precise type
jpn-- Nov 9, 2023
9b8fd95
fix jtp
jpn-- Nov 9, 2023
2901363
adding size tables
vivekyadav26 Nov 9, 2023
2772d7e
read coefficients of mandatory tour frequency
vivekyadav26 Nov 9, 2023
7600776
Revert "read coefficients of mandatory tour frequency"
vivekyadav26 Nov 9, 2023
6f95526
read coeff
vivekyadav26 Nov 9, 2023
9eaebe4
NonMandatoryTourFrequencySettings as logit
jpn-- Nov 9, 2023
50bfe41
Merge branch 'cfg-docs-jn-escort' into cfg-documentation-vy
jpn-- Nov 9, 2023
3c4369c
drop dupe MandatoryTourFrequencySettings
jpn-- Nov 9, 2023
7985f2c
updating joint tour frequency composition
vivekyadav26 Nov 10, 2023
5e90e65
progress on tests
jpn-- Nov 10, 2023
23a9e72
mostly through test1
jpn-- Nov 10, 2023
4d05c28
fix summarize
jpn-- Nov 10, 2023
b184ae0
fix tour schedule test
jpn-- Nov 10, 2023
ca3a42b
3 zone settings for trip mode
jpn-- Nov 10, 2023
d711d5e
fix annotate_table if missing
jpn-- Nov 10, 2023
e6c1a69
various fixing
jpn-- Nov 10, 2023
958f335
TelecommuteFrequencySettings should be LogitComponentSettings
jpn-- Nov 10, 2023
0d25796
repair vehicle type settings
jpn-- Nov 10, 2023
3d512f6
fix - trip scheduling choice
vivekyadav26 Nov 10, 2023
3748f35
fixing
jpn-- Nov 10, 2023
245b872
comment out unused settings
jpn-- Nov 10, 2023
b40fd29
Initial commit for revised user guide, changes to user guide index an…
navsarmajs Nov 10, 2023
4993f76
Merge commit '3d512f64a1df63435429fedd586764522a994826' into cfg-docu…
jpn-- Nov 10, 2023
e4825e9
fix tests for mtc_extended
jpn-- Nov 10, 2023
a5f1476
fix arc
jpn-- Nov 11, 2023
7bce5f2
partial disagg accessblty
jpn-- Nov 11, 2023
16189f1
disagg acc doesn't crash but different results
jpn-- Nov 11, 2023
04c1d14
cdap settings
jpn-- Nov 11, 2023
4288459
trip purpose
jpn-- Nov 11, 2023
18f04ca
also constants
jpn-- Nov 11, 2023
1b75bef
Merge commit '460203c5b54c597e4416bbef30d67fc2a9da3ee8' into cfg-docu…
jpn-- Nov 12, 2023
da00756
updating auto ownership [makedocs]
vivekyadav26 Nov 13, 2023
4d7aac5
prefer float values so fractions are not swallowed
jpn-- Nov 13, 2023
eb05ab7
rollback to have AutoOwnership settings
jpn-- Nov 13, 2023
cfaa5f9
JointTourFreqCompSettings
jpn-- Nov 14, 2023
2c0b700
typos [makedocs]
jpn-- Nov 14, 2023
115f4d8
more doc updates
vivekyadav26 Nov 14, 2023
749ad0a
update cdap
vivekyadav26 Nov 14, 2023
5702e49
Merge branch 'cfg-documentation-vy' of https://github.com/camsys/acti…
vivekyadav26 Nov 14, 2023
6763bc7
clean up disagg accesbly
jpn-- Nov 14, 2023
f357115
format admonition [makedocs]
jpn-- Nov 14, 2023
7ff414e
fix disagg acc
jpn-- Nov 14, 2023
8a844df
Updates to modelsetup documentation page in user-guide
navsarmajs Nov 15, 2023
0bca9fa
Merge branch 'cfg-documentation-vy' into user-guide-doc-navjs
vivekyadav26 Nov 15, 2023
bdb379c
Updates to model run user guide documentation
navsarmajs Nov 16, 2023
18ce345
User guide documentation updates (Ways to run the model)
navsarmajs Nov 16, 2023
dffb45a
Completed documentation of ways to run section in user guide
navsarmajs Nov 16, 2023
d1fd8ae
User guide documentation updates to examples section and other minor …
navsarmajs Nov 18, 2023
b3e48f8
User guide documentation for visualization
navsarmajs Nov 18, 2023
6f1bba5
User guide doc updates (how it works section in progress)
navsarmajs Nov 18, 2023
9abafff
added Semcog model structure figure
navsarmajs Nov 18, 2023
0228785
text formatting
vivekyadav26 Nov 18, 2023
34b669b
Merge branch 'user-guide-doc-navjs' of https://github.com/camsys/acti…
vivekyadav26 Nov 18, 2023
a72289c
doc updates
vivekyadav26 Nov 18, 2023
a97aa5e
Minor bug fixes in user guide doc
navsarmajs Nov 19, 2023
dc8f356
User guide documentation updates - added more sections and minor bug …
navsarmajs Nov 20, 2023
bdd56c0
Adding zone system [makedocs]
vivekyadav26 Nov 20, 2023
c0f6a5e
User guide documentation, minor bug fixes and updates
navsarmajs Nov 20, 2023
0e2d6f8
Merge branch 'user-guide-doc-navjs' into cfg-documentation-vy
vivekyadav26 Nov 20, 2023
9cb9400
[makedocs]
vivekyadav26 Nov 20, 2023
70f24a5
Updates to model setup and anatomy user guide documentation
navsarmajs Nov 26, 2023
70570a1
Minor updates and bug fixes in Ways to Run Model user guide doc
navsarmajs Nov 26, 2023
871cee4
Updates to Model anatomy user doc
navsarmajs Nov 26, 2023
a9f9e13
Added flowchart in How it works section of user guide
navsarmajs Nov 26, 2023
2593477
Updates to howitworks section in user guide
navsarmajs Nov 26, 2023
fadc451
Minor updates to example models user guide
navsarmajs Nov 27, 2023
535a08b
Updating school escorting documentation
vivekyadav26 Nov 27, 2023
b827207
Minor updates to example models user guide - reorganization
navsarmajs Nov 27, 2023
169335b
Merge branch 'user-guide-doc-navjs' into cfg-documentation-vy
vivekyadav26 Nov 27, 2023
939e637
[makedocs]
vivekyadav26 Nov 27, 2023
b4fb5a5
Merge branch 'cfg-documentation-vy' into user-guide-doc-navjs
navsarmajs Nov 27, 2023
56843ac
User guide updates
navsarmajs Nov 27, 2023
25a4dab
Merge branch 'user-guide-doc-navjs' into cfg-documentation-vy
vivekyadav26 Nov 27, 2023
ff559a2
[makedocs]
vivekyadav26 Nov 27, 2023
5bec1f9
Removing tables as they are not being rendered properly [makedocs]
vivekyadav26 Nov 27, 2023
fe9f8e6
[makedocs]
navsarmajs Nov 28, 2023
a92cb66
[makedocs] Fixed broken image links in visualization user guide
navsarmajs Nov 28, 2023
fcc99f8
adding tables and removing models.rst [makedocs]
vivekyadav26 Nov 28, 2023
a46c737
error fix [makedocs]
vivekyadav26 Nov 28, 2023
1310dce
Merge branch 'develop' into cfg-documentation-vy
jpn-- Dec 5, 2023
12b8a49
fix estimation mode to use correct default SIZE_TERM_SELECTOR
jpn-- Dec 5, 2023
e99e564
drop buggy test for now
jpn-- Dec 6, 2023
03c3e38
Updates to user guide - Minor additions and bug fixes.
navsarmajs Dec 11, 2023
3799c08
Navigation bug fixes in Model Anatomy User guide
navsarmajs Dec 13, 2023
75703b1
[makedocs]
navsarmajs Dec 13, 2023
f4f4ba3
Minor bug fixes and user guide improvements
navsarmajs Dec 13, 2023
6b351a7
[makedocs]
navsarmajs Dec 15, 2023
e26e67f
Updates to users guide - model setup page
navsarmajs Dec 21, 2023
03d53d6
Merge branch 'develop' of https://github.com/camsys/activitysim into …
jpn-- Dec 22, 2023
3bbee7b
User guide updates - example models section
navsarmajs Dec 22, 2023
e4f3003
[makedocs] Updates to userguide documentation
navsarmajs Dec 22, 2023
bc88594
Merge branch 'cfg-documentation-vy' into user-guide-doc-navjs
vivekyadav26 Dec 28, 2023
aea3903
Merge pull request #14 from camsys/user-guide-doc-navjs
vivekyadav26 Dec 28, 2023
3e07dbe
[makedocs]
vivekyadav26 Dec 28, 2023
3358002
harmonize on consistent read_file_settings
jpn-- Jan 29, 2024
d70bc16
drop extra file load
jpn-- Jan 29, 2024
5e89e32
InitializeToursSettings
jpn-- Jan 29, 2024
1d9c4d6
use JointTourParticipationSettings
jpn-- Jan 29, 2024
486a4f3
use TourModeComponentSettings for logsum_settings
jpn-- Jan 29, 2024
7f4c1e1
remove old comment
jpn-- Jan 29, 2024
0fa85e2
add missing import
jpn-- Jan 29, 2024
25644f9
docstrings
jpn-- Jan 29, 2024
a5ee11b
bugfix
jpn-- Jan 29, 2024
b24874a
Merge branch 'develop' into cfg-documentation-vy
jpn-- Feb 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 1 addition & 12 deletions activitysim/abm/models/disaggregate_accessibility.py
Original file line number Diff line number Diff line change
@@ -91,7 +91,7 @@ def __init__(self, state: workflow.State, network_los, chunk_size):
add_size_tables = self.model_settings.get("add_size_tables", True)
if add_size_tables:
# warnings.warn(f"Calling add_size_tables from initialize will be removed in the future.", FutureWarning)
shadow_pricing._add_size_tables(
shadow_pricing.add_size_tables(
state, self.model_settings.get("suffixes"), scale=False
)

@@ -757,13 +757,6 @@ def compute_disaggregate_accessibility(

logsums["proto_disaggregate_accessibility"] = access_df

# Drop any tables prematurely created
for tablename in [
"school_destination_size",
"workplace_destination_size",
]:
state.drop_table(tablename)

for ch in list(state.get_rn_generator().channels.keys()):
state.get_rn_generator().drop_channel(ch)

@@ -775,10 +768,6 @@ def compute_disaggregate_accessibility(
for name in list(state.existing_table_status):
if name not in tables_prior:
state.drop_table(name)
# orca._TABLES.clear()
# for name, func in inject._DECORATED_TABLES.items():
# logger.debug("reinject decorated table %s" % name)
# orca.add_table(name, func)

# Inject accessibility results into pipeline
for k, df in logsums.items():
10 changes: 1 addition & 9 deletions activitysim/abm/tables/shadow_pricing.py
Original file line number Diff line number Diff line change
@@ -1285,20 +1285,12 @@ def load_shadow_price_calculator(
return spc


# first define add_size_tables as an orca step with no scale argument at all.
@workflow.step
def add_size_tables(
state: workflow.State,
disaggregate_suffixes: dict[str, Any],
scale: bool = True,
) -> None:
_add_size_tables(state, disaggregate_suffixes)


# then define _add_size_tables as a second method which also offers an optional
# default argument to not scale sizes. This is used only in disaggregate
# accessibility (for now) and is not called via orca. We need to do this to
# avoid having to create a new orca variable for the scale argument.
def _add_size_tables(state, disaggregate_suffixes, scale=True) -> None:
"""
inject tour_destination_size_terms tables for each model_selector (e.g. school, workplace)

3 changes: 2 additions & 1 deletion activitysim/cli/run.py
Original file line number Diff line number Diff line change
@@ -181,7 +181,8 @@ def inject_arg(name, value):
try:
state.load_settings()
except Exception as err:
logger.exception("erroro")
logger.exception(f"Error {err} in loading settings")
raise

if args.multiprocess:
if "configs_mp" not in state.filesystem.configs_dir:
13 changes: 11 additions & 2 deletions activitysim/core/configuration/top.py
Original file line number Diff line number Diff line change
@@ -146,7 +146,16 @@ class OutputTables(PydanticBase):


class MultiprocessStepSlice(PydanticBase, extra="forbid"):
"""Instructions on how to slice tables for each subprocess."""
"""
Instructions on how to slice tables for each subprocess.

.. versionchanged:: 1.3

In ActivitySim versions 1.2 and earlier, slicing instructions for
multiprocess steps allowed for an "except" instruction, which has
been renamed to be "exclude" to avoid problems from using a reserved
Python keyword.
"""

tables: list[str]
"""
@@ -174,7 +183,7 @@ class MultiprocessStepSlice(PydanticBase, extra="forbid"):
`tables`.

Note in ActivitySim versions 1.2 and earlier, this option was named "except"
instead of "exclude", but that is a reserved python keywork and cannot be
instead of "exclude", but that is a reserved python keyword and cannot be
used as a Pydantic field name.
"""

18 changes: 7 additions & 11 deletions activitysim/core/interaction_simulate.py
Original file line number Diff line number Diff line change
@@ -89,7 +89,6 @@ def eval_interaction_utilities(
trace_eval_results = None

with chunk.chunk_log(state, trace_label) as chunk_sizer:

assert len(spec.columns) == 1

# avoid altering caller's passed-in locals_d parameter (they may be looping)
@@ -105,7 +104,6 @@ def eval_interaction_utilities(
locals_d["df"] = df

if sharrow_enabled:

from .flow import apply_flow

spec_sh = spec.copy()
@@ -257,10 +255,8 @@ def to_series(x):

for expr, label, coefficient in zip(exprs, labels, spec.iloc[:, 0]):
try:

# - allow temps of form _od_DIST@od_skim['DIST']
if expr.startswith("_"):

target = expr[: expr.index("@")]
rhs = expr[expr.index("@") + 1 :]
v = to_series(eval(rhs, globals(), locals_d))
@@ -309,12 +305,10 @@ def to_series(x):
utility = (v * coefficient).astype("float")

if log_alt_losers:

assert ALT_CHOOSER_ID in df
max_utils_by_chooser = utility.groupby(df[ALT_CHOOSER_ID]).max()

if (max_utils_by_chooser < simulate.ALT_LOSER_UTIL).any():

losers = max_utils_by_chooser[
max_utils_by_chooser < simulate.ALT_LOSER_UTIL
]
@@ -333,7 +327,6 @@ def to_series(x):
utilities.utility.values[:] += utility

if trace_eval_results is not None:

# expressions should have been uniquified when spec was read
# (though we could do it here if need be...)
# expr = assign.uniquify_key(trace_eval_results, expr, template="{} # ({})")
@@ -353,6 +346,13 @@ def to_series(x):
logger.exception(
f"{trace_label} - {type(err).__name__} ({str(err)}) evaluating: {str(expr)}"
)
if isinstance(
err, AssertionError
) and "od pairs not in skim" in str(err):
logger.warning(
f"recode_pipeline_columns is set to {state.settings.recode_pipeline_columns}, "
f"you may want to check this"
)
raise err

if estimator:
@@ -482,7 +482,6 @@ def to_series(x):
timelogger.mark("sharrow interact trace", True, logger, trace_label)

if sharrow_enabled == "test":

try:
if sh_util is not None:
np.testing.assert_allclose(
@@ -735,7 +734,6 @@ def _interaction_simulate(
or (sharrow_enabled == "test")
or interaction_utilities is None
):

interaction_df = logit.interaction_dataset(
state,
choosers,
@@ -885,7 +883,6 @@ def interaction_simulate(
trace_choice_name=None,
estimator=None,
):

"""
Run a simulation in the situation in which alternatives must
be merged with choosers because there are interaction terms or
@@ -946,7 +943,6 @@ def interaction_simulate(
chunk_trace_label,
chunk_sizer,
) in chunk.adaptive_chunked_choosers(state, choosers, trace_label):

choices = _interaction_simulate(
state,
chooser_chunk,
157 changes: 157 additions & 0 deletions activitysim/core/test/test_logging.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# ActivitySim
# See full license in LICENSE.txt.
from __future__ import annotations

import logging
import textwrap

import pytest
import yaml

from activitysim.core import workflow


def close_handlers():
loggers = logging.Logger.manager.loggerDict
for name in loggers:
logger = logging.getLogger(name)
logger.handlers = []
logger.propagate = True
logger.setLevel(logging.NOTSET)


logging_config_content = {
"simple": """
---
logging:
version: 1
disable_existing_loggers: true
loggers:
activitysim:
level: DEBUG
handlers: [logfile, console]
propagate: false
handlers:
logfile:
class: logging.FileHandler
filename: activitysim.log
mode: w
formatter: simpleFormatter
level: NOTSET
console:
class: logging.StreamHandler
stream: ext://sys.stdout
formatter: simpleFormatter
level: WARNING
formatters:
simpleFormatter:
class: logging.Formatter
format: '%(levelname)s - %(name)s - %(message)s'
datefmt: '%d/%m/%Y %H:%M:%S'
...
""",
"functional": """
---
logging:
version: 1
disable_existing_loggers: true
loggers:
activitysim:
level: DEBUG
handlers: [logfile, console]
propagate: false
handlers:
logfile:
class: logging.FileHandler
filename:
get_log_file_path: 'activitysim_from_func.log'
mode: w
formatter: simpleFormatter
level: NOTSET
console:
class: logging.StreamHandler
stream: ext://sys.stdout
formatter: simpleFormatter
level: WARNING
formatters:
simpleFormatter:
class: logging.Formatter
format: '%(levelname)s - %(name)s - %(message)s'
datefmt: '%d/%m/%Y %H:%M:%S'
...
""",
"unsecure": """
---
logging:
version: 1
disable_existing_loggers: true
loggers:
activitysim:
level: DEBUG
handlers: [logfile, console]
propagate: false
handlers:
logfile:
class: logging.FileHandler
filename: !!python/object/apply:activitysim.core.config.log_file_path ['activitysim_unsecure.log']
mode: w
formatter: simpleFormatter
level: NOTSET
console:
class: logging.StreamHandler
stream: ext://sys.stdout
formatter: simpleFormatter
level: WARNING
formatters:
simpleFormatter:
class: logging.Formatter
format: '%(levelname)s - %(name)s - %(message)s'
datefmt: '%d/%m/%Y %H:%M:%S'
...
""",
}


@pytest.mark.parametrize("logging_yaml", logging_config_content.keys())
def test_config_logger(capsys, logging_yaml):

print(logging_config_content[logging_yaml])

state = workflow.State.make_temp()
state.filesystem.get_configs_dir()[0].joinpath("logging.yaml").write_text(
textwrap.dedent(logging_config_content[logging_yaml])
)

if logging_yaml == "unsecure":
with pytest.raises(yaml.constructor.ConstructorError):
state.logging.config_logger()
return

state.logging.config_logger()

logger = logging.getLogger("activitysim")

file_handlers = [h for h in logger.handlers if type(h) is logging.FileHandler]
assert len(file_handlers) == 1
asim_logger_baseFilename = file_handlers[0].baseFilename

logger.info("test_config_logger")
logger.info("log_info")
logger.warning("log_warn1")

out, err = capsys.readouterr()

assert "could not find conf file" not in out
assert "log_warn1" in out
assert "log_info" not in out

close_handlers()

logger = logging.getLogger(__name__)
logger.warning("log_warn2")

with open(asim_logger_baseFilename) as content_file:
content = content_file.read()
print(content)
assert "log_warn1" in content
assert "log_warn2" not in content
5 changes: 3 additions & 2 deletions activitysim/core/workflow/state.py
Original file line number Diff line number Diff line change
@@ -17,6 +17,7 @@
from pypyr.context import Context
from sharrow.dataset import construct as _dataset_construct

import activitysim.core.random
from activitysim.core.configuration import FileSystem, NetworkSettings, Settings
from activitysim.core.exceptions import StateAccessError
from activitysim.core.workflow.checkpoint import LAST_CHECKPOINT, Checkpoints
@@ -841,7 +842,7 @@ def extract(self, func):
add_injectable = set # legacy function name
"""Alias for :meth:`State.set`."""

def rng(self):
def rng(self) -> activitysim.core.random.Random:
if "prng" not in self._context:
self._initialize_prng()
return self._context["prng"]
@@ -863,7 +864,7 @@ def close_open_files(self):
file.close()
self.open_files.clear()

def get_rn_generator(self):
def get_rn_generator(self) -> activitysim.core.random.Random:
"""
Return the singleton random number object

Original file line number Diff line number Diff line change
@@ -189,3 +189,5 @@ household_median_value_of_time:
2: 8.81
3: 10.44
4: 12.86

recode_pipeline_columns: True
Loading