Skip to content

CB Meeting Notes

Jump to bottom
Ben Stabler edited this page Jun 2, 2021 · 59 revisions
Meeting Notes 06/02/21
  • Latest model summaries here, https://nbviewer.jupyter.org/github/SANDAG/ABM/blob/f613bca58d8dd76708863d9ae4ef3197ef8b1c6b/src/asim_cross_border/cross_border_validation.ipynb
  • Fixed several issues since we last talked: walk mode walk times, removed sample correction factor like previous model (but will add back later), corrected sentri shared by purpose, dropped the extra POE, corrected a couple incorrect coefficient labels
  • Results are looking better, but we still have a tour OD distribution issue....way too many short trips
  • Issue appears to be the piecewise distance terms in the destination choice models...they are not coded consistently
  • Will fix and re-run summaries. Also add backstopping of walk times if not in the MAZ to MAZ file. Will share results with SANDAG.
  • Need to add writing of estimation data bundle to tour OD model and move the trip destination sampling coefficients to a coefficient file.
  • Will clean-up the summaries notebook and check-in.
  • Then can get Hannah working on the estimation setup
Meeting Notes 04/22/21
  • latest code merged into sandag cross border fork
  • ran 100% sample and realized tour mode choice logsum was turned off which is also now fixed
  • Pre-sampling is implemented but needs debugging. As it currently exists for specific sub-models it takes a little more effort to port over to a new sub-model.
  • Enrique improved the runtimes for tour OD choice by editing some of the expressions. He will touch base with Max.
  • Model currently uses 40 time bins, need to move to using 48 half-hour bins and can reference SEMCOG for this where is is already implemented.
  • Border Wait time model will need inputs, Max to forward files to Hannah to format survey data.
  • Need to check the properties used for path calculations and check UECs for bugs.
    • max_paths_across_tap_sets is currently set to 3 but should be 4 and max_paths_per_tap_set is set to 1 but should be 2 in order to remain consistent with java code
  • Estimation works backwards from last model to first:
    • Start with trip mode choice estimation: requires creation of trip data tables from survey, will likely not give good results
    • Stop location choice constrained by tour mode, OD
    • Trip scheduling and purpose files to be prepared from survey
    • Stop frequency gets estimated
    • Tour mode choice gets estimated (conditional on lane crossing type)
    • Tour POE/destination choice gets estimated
  • CB model will only predict wait time by border crossing type, there is not a lane choice model. Likely, will have SENTRI/Ready pass ownership as an attribute so they see the pass lane wait time. There will not be a component to model pass ownership in the future as it is a border operations decision. i.e. increase in pass ownership affects number of lanes/congestion and lanes are not optimized for speed from CBP.
  • Project deadline current goal is end of Nov.
Meeting Notes 04/07/21
  • finishing merge of latest code into sandag cross border fork
  • after that, setup on Windows server, run 100% sample, fix any issues, report runtimes, add presampling to tour od model, and then validation
  • will do validation submodel by submodel against old model results
  • need to program border wait time function
  • will need to format survey - it is just tours and trips - to match asim format for estimation
  • 45 minutes for existing CB model and hoping for around 20 minutes for new model
  • we'll do a training on expression writing / tuning next Tuesday
    • expressions overview, including annotations
    • review ARC, SEMCOG, and Cross Border examples
  • border wait time model will have performance implications since it will be iterative
    • will iterate with tour OD model
    • will need to average to converge, like shadow price / MSA
    • will option to run or not and will usually be turned off
  • start working on the estimation plan now
Meeting Notes 03/24/21
Meeting Notes 03/10/21
  • Cross Border Wait Time updated in the Memo, this is up on box for review.
  • Now working on implementing trip scheduling model, which is the last model remaining to be implemented
  • Required a new submodel since the probability lookup table is by duration as opposed to the existing asim model which is by depart hour
  • Max/Joel to talk through any remaining questions related to the implementation
  • This is an improved version over the existing asim approach (but the DaySim approach is even better - it makes more sense and works better with DTA)
  • Next steps - get running all the way through, pull in upstream code revisions (including revising trip models for named coefficients and adding presampling to the destination choice models), run a full sample, validate results, review runtimes
  • Max update https://github.com/SANDAG/ABM/wiki/Cross-Border-Model-Implementation-Plan as needed once the software is up and running
  • Enrique running develop2 and experimenting with runtimes, settings, and data input issues
  • Will try more processors, different chunksizes, and SET MKL_THREADS=1 to get better runtimes
  • Enrique create issue for assertation issue
  • Asim accessibility calculator is MTC TM1's aggregate zone calculator and runs slow for SANDAG because of TVPB and lack of caching
  • We should build a better accessibility calculator - a disaggregate smart sampling approach like Joel suggested
  • The accessibility calculator is not needed for the cross border model but for ABM3
Meeting Notes 02/24/21
  • Cross Border Wait Time updated in the Memo, this is up on box for review.
  • Tour and trip mode choice model running. Tour mode choice calls trip mode choice if optionally configured. Trip mode choice calls TVPB. Trip mode choice also gets all three VOT skims and then picks the right one for each trip chooser based trip chooser VOT. Thus, no code changes required to support SANDAG's 4D skims.
  • Not sure yet about runtimes....just doing some basic testing for getting the code working.
  • Tour mode choice has a tourPurpose UEC bug and trip mode choice has a totalStops/Trips bug for correction
  • Next steps, work through remaining models with the hope to have the system running in a couple weeks
  • Then we'll pull in the location choice presampling update and do a step-by-step validation exercise
Meeting Notes 02/10/21
  • Cross Border Wait Time updated in the Memo, this is up on box for review.
  • Max and Ben are working on the tour mode choice logsum issue inside the tour mode choice software improvement. Currently ActivitySim doesn't have the plumbing for communication between trip and tour mode choice. This is necessary for the crossborder model because the tour mode is really the 'border crossing' mode and has different implication for the trip mode choice. i.e. Someone walking across the border (tour mode = walk) may be picked up, take transit, or a taxi. Currently the resident model doesn't deal with this as the trip modes are constrained by the tour mode. However, it seems more theoretically attractive to implement this feature in the resident model as well, since travelers who start a tour by walking may also take other modes throughout the tour. Max will take a look to see how much effort this would be to implement and we will revisit the issue. The implications on the run time also need to be considered. This is also something to bring up to the ASIM consortium.
  • There is still a memory issue while testing the 3 Zone Marin examply using SANDAG data. Enrique to forward some of the crash report/data and Ben and Max will take a look.
Meeting Notes 01/28/21
  • Updated model design, including border wait time model, memo posted on box account for review
  • The tour origin and destination choice model is now working and checked in to SANDAG's GitHub account
  • We're running a 5k sample of HHs for development and testing
  • Doyle is going to work on the taz/maz location sampling improvement and then Max will integrate it once it is ready
  • This will help significantly with runtime
  • The sample of alternatives correction factor is turned off in the Java UEC?
  • The mode choice logsum term is commented out until we get tour mode choice working
  • Next up is tour sampling, which is a probability lookup model. We're going to copy the existing trip scheduling model and rename as a new submodel, something like tour_scheduling_probabilitic
  • We may refactor the code so trip and tour can use the same underlying code
  • After that we'll do tour mode choice, which seems straightforward
  • Then we'll turn the logsum back on in OD choice and run a full sample
  • We'll use all time periods and multiprocessing too (and hopefully Jeff's taz/maz sampling code too)
  • And then we'll have a good idea about runtime performance
  • Then on to the trip based models
Meeting Notes 12/30/20
  1. Cross Border Wait Times
    • Model for border crossing wait times evaluated using only terms with high significance
    • Wait time (min) =  x1 * volume/total lanes (all stations) + x2 * volume/total lanes * OM(0,1) + x3 * volume/total lanes * TC(0,1,) + x4 + x5 * OM(0,1) + x6 * TC(0,1)
    • fit looks reasonable with observed data. Ready Lane wait time at San Ysidro is set to a constant value to compensate for negatively correlated data
    • Will do another estimation with a time of day factor.
Meeting Notes 12/14/20
  1. Cross Border Wait Times
    • Started to estimate a model with the wait time data for each lane type
    • Wait time =  x1 * volume/total lanes (all stations) + x2 * volume/total lanes * SY(0,1) + x3 * volume/total lanes * OM(0,1,) + x4 + x5 * SY(0,1) + x6 * OM(0,1)
    • X1 is estimate that considers all stations, x4 considers all stations, x2/x5 is added effect at San Ysidro and x3/x6 added effect at OM
    • Will swap San Ysidro for Tecate so San Ysdor is the base scenario
    • Underlying assumption is future POE will behave similarly to the base POE
    • Next step is to plot estimated wait times and make sure things are reasonable before testing specifications.
    • Otay Mesa East may have a fixed wait time/toll
    • The toll and coefficients can be coded in yaml file
  2. Survey Memo
    • RSG to review comments on Memo
    • RSG to add cross border wait time analysis to memo
  3. Implementation
    • Wiki has been updated
    • Max working to setup Activity Sim, getting familiar with inputs and correct columns
    • Decided not to add additional MAZs for POEs/Colonias
    • Utilities are stored in efficient way so they are fast to use in all downstream model components
Meeting Notes 12/02/20
  1. Housekeeping
    • Paperwork needed for Max Garder to work on project as part-time employee to RSG
    • Max needs access to GitHub Repo
  2. Survey Memo
    • Posted to Box under "03_Data_Processing\Deliverables\09.15.2020_CBM_v5.docx"
    • Section 2/3/4 are mostly filled out and the rest of the memo is a placeholder. This is for survey only.
    • Download Box Tools to edit in word, add comments
  3. Border Wait Time
    • Received updated data to include Nov. 15th and Nov 11th. Nov 11th is Veterans Day and will not be included.
    • Updated data did not significantly change previously seen relationships and will move ahead with the analysis, potentially removing some outliers for better behaving relationships
    • Processing rates from model were looked at by POE/Hour/Lane Type. This seems to match what we are seeing in the data. The model number of open lanes is dynamic while the processing rates by lane type do not change over time.
    • Stacked lanes are likely to be phased out of use.
  4. Software Implementation
    • Tour generation is using working using some dummy variables for tour share by lane type.
    • Year specific parameters should be called in a pre-processing script which writes a yaml file for Activity Sim to use
    • Pre-processing script will also be used when initializing Land use to describe the special geographies (Colonia, or POE vs MGRA)
    • When computing accessibility can use skim time instead of distance
    • Tour Origin and Destination: Colonia to POE part of the tour is more or less deterministic based on the population/employment characteristics. This is ultimately another field on the POE which can be changed to reflect future pop/emp changes south of the border.
Meeting Notes 11/18/20
  1. Survey Memo
    • Should be ready to send out later today.
  2. Border wait Time
    • Analysis of observed border wait times by POE/Lane/Hr by Previous hour and previous 2 hour volumes shared.
    • Missing Nov 11 and Nov 15, 2019 wait time data
    • Negative Correlation in Ready Lane at San Ysidro needs an explanation:
      • Real time adjustment of number of lanes?
      • Incomplete data?
      • Different traveler types in this lane?
      • Time of Day factors?
    • Note that different ports of entry have behaviorally different types of passengers and time of day may play a role.
    • Generally have good understanding of observed data and will begin thinking about model plan
      • Iterative demand simulation will add time to model runs
      • Which toggles are needed to describe POE and future policy attributes?
  3. Model Implementation
    • Max has joined the team and will be working on the software implementation
    • Changes that need to be made to deviate from ActivitySim
      • POE choice
      • Border Wait time
      • Space (Colonia/POE choice) Special zones can have a flag for POE/Colonia so they can be used for specific destination choice. It's better to use existing structure than to have to introduce new ones, but need to pay attention to how geographies are used.
Meeting Notes 11/04/20

  1. Survey Memo Update:

    • Survey Memo will be updated shortly.

  2. Cross Border Wait Time Analysis:

    • Want to know what variables we can use and what policy scenarios to test. The SR11 Model Runs were analyzed to look at border wait time as a function of operating hours/open lanes/volumes by hour and lane type. Pedestrian data is not included in the SR11 model runs
    • Intuitively, wait times should be sensitive to demand. It's possible to use a dynamic wait time by running the border crossing choice and a simple delay model in a loop. Average the delay results to guarantee a convergence (similar to shadow pricing) this way demand and number of lanes are somewhat dynamic.
    • This can be one model with interaction effects for Otay/Tecate to estimate average wait time across all modes and all ports with independent variables of number of passengers by mode, number of lanes, and time of day. Then a data driven factor can be used to get wait times for each lane type.
    • The model should recognize feedback between pedestrian and vehicle crossings such that pedestrian mode becomes more favorable with increased vehicle volumes.
    • For future scenarios, the minimum required input should be number of vehicle lanes.
    • We will investigate the observed data and see if some of these relationships of volumes and wait times are stable.
    • A proposal will be put together to review.

  3. Software Implementation Plan: https://github.com/SANDAG/ABM/wiki/Cross-Border-Model-Implementation-Plan

    • Space: Treat POEs and Colonias as additional MAZs, that way the data structures do not change. There will be a special connection or relationship for these zones with accessibilities that can be coded in the data file for them. Distance from POE to Colonia can be put in a MAZ file. Activity Sim takes multiple maz-maz cost files so we can include one for poe-maz and colonia-maz with costs.
    • This model will use features from ActivitySim including TNCs with future mobility forms as new modes.
    • Items to pay attention to:
      • Having different representation of space where external stations have unique mgra instead of finding mgra in region closest to external station (for walk access to transit). This would show up in the logsums used.
      • POE demand based wait time,
      • Computing accessibility by Colonia by POE, so we can code this in the data file
      • Initialize households model will create records instead of read them in.
    • Most model components use the existing functionality except tour origin/destination model. We will take the existing destination choice model and create new model from it. The alternative set will be generated using POE to SANDAG MAZ pairs, then it's basically the same code: sample without logsums, then use tour mode choice with logsums. The difference is the alternative set will be built with destination and origin pair combinations and will pick from combination set. The current code predicts border crossing mode. Tour scheduling happens first but if wait time is to affect people's border crossing, then a fixed arrival/departure should be used.
Meeting Notes 10/21/20

  1. Cross Border Survey:

    • We were able to use the older 2017 cross-border survey to estimate the percent of border crossings by Mexican residents versus San Diego county residents versus non-SD county US residents by POE and lane type/mode. We then apply the share of Mexican residents who return versus who do not return to the POE in the departure day to calculate total targets by market. So now we can calculate, for each POE and lane type/mode, targets for cross-border tours, internal (SD resident) – external (POE) tours, and external (POE)-external (non-POE) trips.
    • Now the expansion factors will be recalculated and summaries will be re-run and the memo will be updated accordingly.

  2. Border Wait Times:

    • Regression model will be estimated based on SR11 model runs
    • To start, we need a table of SR11 runs so each row is a scenario and columns are number of lanes, volume etc.
    • Operational characteristics should be tied to potential future policies
    • First we will describe what data is available. Border wait times used to be static but want to make this a function of some policy (how many booths are open, processing speed). Border wait time peak demand should be consistent with 30min time slice.
    • There should be an external property file to allow users to represent the future policy. This shouldn't get too specific as things like number of lanes by type can change throughout the day adjusted to demand. This would be something difficult to code to future scenarios.
Meeting Notes 10/07/20
  1. Survey Report Summary (Tim, True North)
    • SANDAG has read the True North summary report and has sent back all comments and feedback
    • True North will work on addressing all comments
    • Is there a summary between the 2010 and 2019 summary reports?
      • There isn't, but True North will incorporate it into the updated summary report
    • Outstanding questions on report summary
      • The main question is whether there is any data that can be used to expand key markets that were unaccounted for in survey?
      • True North developed weights, however expansion factors are needed to create the universe of travelers across different key markets
      • Key markets RSG is interested in
        • Number of US residents crossing the border
        • Number of MX residents crossing the border
        • Everyone else
          • US residents who are not SD County residents and who additionally do not make any stops within SD County
          • MX residents who do not make any stops within SD County
      • Per True North's e-mail response, it seems we are unable to obtain the above market distribution data
        • The main issue that was realized early on was that the CBP data provides crossing volumes by citizenship, not residency. The CBP crossing data showed there is a large amount of US residents who reside in Mexico, therefore citizenship is not an accurate indicator or residency.
        • The screening data from the 2019 cross border survey is also not a good indicator to reveal key market distributions
          • However, after talking with all the surveyors, the general trend revealed that the distribution of US residents to MX residents was about 1 to 5 (i.e. 20% US residents)
          • True North was planning on collecting actual distributions during the second round of data collection, but was unable to due to COVID-19
        • This data was collected for 2010
          • Tecate numbers seemed to have too many US residents
            • This was partly due to the low number of data collection there
            • Tecate crossers may be about 90% MX residents
      • Would the US/MX resident distribution be different between vehicles and pedestrians?
        • Could look at distributions of different lanes
        • Could also look at 2017 border crossing data
      • Negative trip duration record IDs were sent to True North
        • Quick review showed that there were in fact errors in the way respondents reported times
          • Should those be dropped or should True North make judgement calls on a case-by-case scenario?
            • Judgement call on case-by-case scenario with the possibility of dropping indiscernible records
      • Is bike percentage actually 12%?
        • Yes, San Ysidro specifically had a large bike share
          • True North will double check this though
  2. Data Processing Report (RSG)
    • Report on hold due to the issues related to pending market distribution data and expansion factors
    • Started creating tour length frequency distribution summaries by appending midday medium VOT distance skims to trip and tour files
  3. Model Estimation (Ben, RSG)
    • What scenario year should be used to start processing and formatting for ActivitySim?
      • 2016 seems like the best choice
        • Everything is on the T Drive on SANDAG servers
    • Will there be customization for the model? And if so, in what areas? Has the old cross border model been reviewed yet?
      • Started going through it all and started building an ActivitySim "skeleton"
      • Along the way making notes of what can stay, what will need to be updated and what will need to be added
    • The cross border model is a tour based model, so it starts with tours. ActivitySim, on the other hand, starts with household and persons and then adds tours and trips. Thoughts on how to address this?
      • Tours do have attributes, but they're not currenlty associated with households nor persons
        • Could potentially extend model to consider households and persons
      • Another option is that for every tour you create a household and assume everyone in that tour is part of said household and then persons can be created for each of those people in the tour
    • Given 3-zone system is work in progress, we can't go through the entire model yet, right?
      • We can, but it will be slow. The 3-zone system task works, it's just slow
    • SANDAG is interested in learning more about ActivitySim and would therefore be interested in being more involved in the model design process
  4. Action Items
    • @True North
      • To address SANDAG feedback on survey summary report
      • To incorporate 2010 and 2019 survey comparisons in updated report
      • To go through negative stop duration records
      • To double check high bike share at San Ysidro
    • @RSG
      • To work on finalizing data processing report
      • To continue working on model estimation/design
      • To include SANDAG in ActivitySim learning process
Meeting Notes 09/23/20
  1. Report, Summaries
    • Expansion factors revised; resulting summaries are more reasonable now.
    • For initial summaries, might be good to show old data compared to new data. Additionally, adding visuals will make it easier for readers to understand numbers.
    • Time bins
      • There are currently 40 time bins where bins correspond to half hour time segments throughout the day and first and last are aggregated
        • Might be a good idea to instead use 48 time bins corresponding to every single half hour time segment in a day and reduce confusion
  2. SANDAG Data
    • Tijuana Census Geography
      • Has been finalized and sent
    • SR-11 Model Skim Data
      • Has been finalized and sent
    • Mexico Population Forecast
      • Data is already available, however SANDAG is working on getting additional colonia level data
      • SANDAG will continue to provide updates on any progress
    • Data has been saved on T Drive and is ready for RSG to download
  3. Action Items
    • Finalize report
    • RSG to download data from T Drive
Meeting Notes 09/16/20
  1. Survey Weighting, Processing, and Report
    • Initial expansion was revised due to discrepancy between border survey and observed crossing volumes. Could have been leading to low expansions as we previously saw in first iteration of summaries.
    • Expansion factors were intially not considering party size (vehicle passengers). That was revised. Expansion factors now allow expansion for number of travelers / party size
    • We can utilize the 2019 cross border survey data to feed External-to-External model given there were respondents who were passing through the San Diego region
      • Can also utilize the data to improve the Interanal-to-External model for the San Diego region
    • Will have to check with True North on 100% pedestrian volumes
      • True North report has been sent to SANDAG, however not everyone has reviewed it yet. Will send to RSG anyway.
      • Meeting can be set up with True North if need be
    • Report will be finalized and reviewed internally (RSG)
      • Will send updates to SANDAG when they become available
  2. Data from SANDAG
    • Border Crossing Volumes by Mode
      • RSG has this dataset already
    • Border Crossings Volumes by Lane Type
      • RSG has this dataset already
    • Border Crossings - Pedestrian and Bus Passengers by Visa and POE
      • RSG has this dataset already
    • Border Wait Times - Northbound
      • RSG has this dataset already
    • Mexico Population Forecast
      • Will be sent out soon. Is almost ready.
    • Colonia Population
      • Still working on this
      • Ensenada, Rosarito, Tecate can be aggregated as large zones if smaller boundaries are not available
    • Skims
      • Will be sent out soon. Is almost ready.
  3. Model Estimation Plan
    • Plan is to finalize survey processing and finish report
    • Afterwards, we can start perparing inputs to model setup
    • Then, Blake can begin looking at Java code
    • Larch v Alogit
      • ActivitySim writes out all required information
      • Really just depends what the user is most familiar with
  4. 3-Zone System
    • Caching will be an important feature here
    • RSG can begin gathering things for this
      • SANDAG can help, but will need some guidance
      • Ben will document list of inputs required
  5. Action Items
    • @Cundo to finalize summaries and report
    • @Ben to document list of inputs required for 3-Zone System
Meeting Notes 08/26/20
  1. Data Processing Summaries
    • CBM Table 1
      • There is no documentation on expansion weights for previous CBM 2011 nor do we have the Python scripts used
        • Does SANDAG have those scripts? If so, please share with RSG
      • Cargo
        • Went down due to old survey including cargo as a trip purpose, while the new survey did not. Instead, the new survey only asks about cargo at the tour purpose level.
        • Will probably group cargo with other
      • Would be useful to create some documentation of how weights were created for 2019 border crossing data
    • CBM Table 3
      • Need to investigate further regarding transit increase. Do Mexican travelers leave their vehicles near the border and then walk across to transit?
      • Need to investigate drive-alone increase as well
    • TOD Tour Arrival and Departure
      • Need to compare to observed volumes
      • Early AM data is not showing up in survey.
        • This was expected given True North was concerned about safety during these times
        • May need to smooth this data out.
        • Alternatively, 3rd party data may be obtained to augment these uncaptured time segments. Such collected data could help with distribution
          • Wu will follow up with Grace to determine if there was 3rd party data collected
    • Party Size
      • Would be useful to see this broken up by trip mode
  2. SANDAG Datasets
    • Border Crossing Volumes by Mode
      • Data has been pulled using an API
      • There is a lot of overlap in the data. The dataset may be useful for QA/QC checks.
    • Border Crossings Volumes by Lane Type
      • This datset was sent to RSG already
    • Border Crossings Pedestrian and Bus Passengers by Visa and POE
      • This dataset was sent to RSG already
    • Borde Wait Times - Northbound
      • There are 3 datasets that comprise this data
      • This dataset was sent to RSG already
    • Mexico Land-Use
      • This is done. Baja California land-use was exported
      • The dataset is in CSV and shapefile format
      • The data is primarily land-use codes
        • Would such data be useful for RSG?
          • Potentially. Could inform POE entry
    • Tijuana Land-Use
      • SANDAG to provide colonia data with population estimates
      • Is it possible to get both travel times and distances (to POEs)?
        • Yes, should be possible
    • Transactional Vehicle by Vehicle Movements Across the Border
      • May not be useful; too much detail for model needs
    • SR-11 Model Skim Data
      • Skims are almost done that provide distances to POEs from different colonias in Tijuana
      • Rosarito, Tecate and Ensenada not currently included, but can add
  3. Action Items
    • @Cundo
      • to create documentation of how expansion weights were created for 2019 border crossing data
      • to investigate transit and drive-alone increase for CBM Table 3 summary
      • to compare TOD Tour Arrival/Departure numbers with observed volumes
      • to reformat TOD Tour Arrival/Departure summaries to match model half-hour time bins
      • will break up TOD Tour Arrival/Departure summaries by tour purpose
      • will break up TOD Tour Arrival/Departure summaries by port of entry
      • will break up Party Size summary by trip mode
      • to create distributions that are actually used in model such as trip lengths
      • to look into observed transit trips versus survey numbers
    • @Ben to create memo guidelines/list to document summary and data processing
    • @SANDAG to provide colonia, Tecate, Ensenada and Rosarito population, time and distance data to RSG by next Wednesday
    • @Neeco to add Rosarito, Tecate and Ensenada to SR-11 Model Skims
Meeting Notes 08/12/20
  1. Data Processing
    • Finalized trips and tour files, which consisted of:
      • Geocoding coordinates to MAZs and TAZs
      • Converting survey trip modes and purposes to match with those of the CBM Specification Report
      • Coded logic to determine tour purpose. Used logic explained in CBM Specification Report
    • In finalizing tour and trips files, a few responses were flagged where the issue that stood out the most was negative duration at a stop/activity.
      • RSG should send list of flagged records (and any other questions) to Grace so that they can be sent to True North
        • A meeting could also be set up to speak with True North
    • Summaries:
      • Recreated the summaries from the CBM Specification Report, however no expansion factors/weights were provided. Created expansion factors utilizing the crossing volumes data (from 8-day period in November 2019), however there were issues related to significantly lower volumes. Still exploring this.
      • Cundo and Binny to finalize summaries for next week
    • ActivitySim format
      • The data processing output files have thus far been formatted in a standard format. Once a more solid model design is establish, that format can easily be switched over to the ActivitySim format.
    • Once summaries are finalized, can start determining some targets (e.g. by TOD, POE, Purpose, Lane Crossing Type) and those should be sent to SANDAG so they can use for policy design/implementation
  2. True North
    • Report from survey is still pending, and is expected to be received this week
    • Report will be useful in determining more information about the weights provided in the survey data
  3. SANDAG Data Collection
    • SR-11
      • Ran SR-11 model and produced some skims. There are 24 skims in total (1 for each hour of the day).
      • They're large files. Neeco is still exploring the data
    • Need clarification on what zones correspond to POEs
      • They differ depending on whether it's the section of the POE that is for pedestrian crossings, vehicle crossings, or commercial vehicle crossings
      • Jielin to confirm and notify Wu
    • Mexican TAZ to Colonia conversion (Tijuana Land-Use dataset)
      • Working on getting shapefile for Colonias
      • Grace Chung provided a shapefile for Colonias (2015), AGEBS (2010), and Delegaciones (2013)
        • Jeff to start looking over these files
        • Asked for a newer (than 2015) shapefile for Colonias, but was told there is likely not significant change in geography
    • Population (Mexico Population Forecast dataset)
      • Zack has requested the data and list of other possible attributes. Hector followed up, but have not heard back from him likely because he hasn't heard back either
      • There was a dataset that was found that is for 2017 population data by colonia. The dataset has forecasted data for the year 2030
    • Border Crossings Volumes by Lane Type
      • Grace sent it over to Wu and Enrique
    • Broder Crossings Pedestrian and Bus Passengers by Visa and POE
      • Is it possible to prase out passenger by bus passengers? Grace?
    • Border Wait Times - Northbound
      • Pat has not reached out to neither Neeco or Enrique
      • Lisbeth to reach out to Pat and indicate data is needed within the next 2-week-timeframe
    • Mexico Land-Use
      • This is available now
      • Jeff to start exploring data
    • Transactional Vehicle by Vehicle Movements Across the Border
      • Need to verify if any of this data is confidential
        • Lisbeth to confirm with CBP
      • There are some codes in the data that Lisbeth has not been able to determine what they mean
        • Lisbeth to clarify with CBP
  4. Micromobility / Emerging Modes
    • Would like to incorporate micromobility and emerging modes in the model
    • Need to come back to how it is treated within ActivitySim given present Java code (i.e. CT-RAMP) limitations
      • Limitations such as:
        • No way to model microtransit
        • No differentiation between kiss-and-ride and park-and-ride
        • Cannot use e.g. car as egress from alighting transit station if car was used to access boarding transit station
    • Walk has been used as a surrogate for walking, microtransit and micromobility
  5. Action Items
    • @RSG to finalize summaries by next week
    • @RSG to send list of flagged survey records and any other questions that they may have for True North
    • @Neeco to finalize going over SR-11 skim data
    • @Jielin to clarify (to Wu) what zones correspond to what POE
    • @Jeff to start looking over Mexican geography shapefiles
    • @Jeff to start looking over Mexico land-use data
    • @Lisbeth to confirm confidentiality of CBP data
    • @Lisbeth to clarify unkown code meanings of CBP data
Meeting Notes 07/29/20
  1. Data Processing
    • After taking a closer look at the crossborder survey data, it was decided it is no longer necessary to use the SPA tool to process the data given each respondent was noted to only have 1 tour. The tour file (and others) can/will be created with scripting.
      • Person, Household, Place and Trips files have been created. Tour file is pending.
    • In addition to the above files, summaries were created for:
      • SENTRI vs Non-SENTRI distributions by POE
      • Income distribution by POE
      • Crossing volume distribution by POE
      • Tour purpose distribution by POE
    • The above summaries will be weighted by observed POE observed crossings (where applicable)
    • If any issues arise with the data, RSG should notify Grace and Grace will reach out to True North
    • Auto estimation:
      • Has the data processing work been generating data in ActivitySim format?
        • Not yet, has mainly been a summarization task
        • Once model design is determined, can begin formatting for ActivitySim
      • Would be good to take a look at recently updated ActivitySim estimation documentation to start thinking about how to format the data into ActivitySim format.
    • For created summaries, would be good to compare them to the data from the old model. These can be found in the report. SANDAG to send to RSG
  2. Datasets (Lisbeth)
    • Border Crossings Volumes by Mode
      • Grace to send BTS data to RSG
    • Border Crossings Volumes by Lane Type
      • Pending
    • Customs and Border Protection Data (sent by Grace in 7.29.20 e-mail)
      • The data is from a 9-day period in November of 2019
      • Grace will send this data once she receives it from True North
      • Can we have similar 2017 data?
        • Yes, Rick to send. Note that it does not contain citizenship information
    • Border Wait Times
      • Pat reached out to Wu's team and will coordinate with Enrique to summarize data
    • Mexico Land Use
      • Grace Chung currently compiling this dataset and will likely provide it soon
    • Tijuana Land Use
      • Grace Chung and Pat may be working on this dataset. Need to verify.
      • Things to confirm about this dataset:
        • In what geographic format is this data needed? Colonia or Super Colonia?
          • Colonia given it is more disaggregate and can determine data for the more aggregate Super Colonia geographic format
        • How soon is this data needed?
          • As soon as it becomes available.
          • Will likely not be able to provide data by this Friday. Lisbeth will keep Wu updated on whether it can be provided by next Friday.
        • Are there any other attributes needed in addition to population per Colonia?
          • Lisbeth to provide Wu with a list of available attributes and will work together to determine which attributes are desired.
    • Mexico Population Forecast
      • Requested colonia level, but is available in other geographic formats such as city and state
        • Not sure if colonia level available given previous data was not in said format
      • Zack will confirm what geographic format it is available in
    • Transactional Vehicle by Vehicle Movement Across the Border
      • This data comes from the lane sensor counters that counts every vehicle for every lane
      • Grace Chung has already obtained this data
      • There were questions about the meaning of certain fields
        • There is a data dictionary file that may contain information on field values
        • Rick to look for it and provide
    • SR-11 Model Skim Data
      • Enrique has already obtained this data
      • Neeco produced skims and converted to CSV format
      • The remaining task involves mapping data to colonias and then deriving time and distance from each colonia to each POE
        • Neeco and Jielin are working on this task
        • Data will likely be available by next Friday
  3. Action Items
    • @RSG to finalize data processing and begin formatting data into ActivitySim format
    • @SANDAG to send previous CBM report to pull summaries for comparison with 2019 survey data
    • @Grace to send RSG Border Crossing Volumes by Mode BTS data
    • @Rick to provide 2017 Customs and Border Protection (similar) data
    • @Grace to provide 2019 Customs and Border Protection data once it is finalized by consultant
    • @Enrique and Pat to coordinate on Border Wait Time data
    • @Grace Chung to provide Mexico Land Use data once finalized
    • @Lisbeth to provide Wu with a list of available attributes for Tijuana Land Use data
    • @Zack to confirm geographic format for Mexico Population Forecast data
    • @Rick to look for and provide data dictionary for Transactional Vehicle by Vehicle Movement Across the Border data
    • @Neeco and Jielin to provide SR-11 Model Skim data
Meeting Notes 07/15/20
  1. Data Processing
    • Planning on using Survey Processing Automation (SPA) Python tool to create data files required for the model
      • Tool reads-in a place file and generates tours, stops and trips files
    • Survey has been formatted into person, household and place file
      • Place file still requires further formatting to be compatible with SPA tool
    • Regarding trip/tour data for model
      • Border crossing location (i.e. port of entry(POE)) should be an attribute of the tour
      • A tour origin will always be one of the port of entries
      • Travel patterns in Mexico should be eliminated as they are not required/considered by the model
  2. Income
    • Imputing income data may no longer be necessary given previous model did not use income
      • Previous border survey also had a similar amount of missing income information
    • If income were to be included, it would have a limited amount of impact as it relates to value of time (VOT) calculation
    • Nevertheless, the model should be sensitive to tolling (especially for Otay East)
      • Otay East (tolled POE) will likely have users with higher reliability and VOT given they have opted to pay extra to use that POE
    • In the future, it's believed there will be an increase in SENTRI pass holders, however income will likely decrease as an indicator of SENTRI pass ownership
    • There is a need to create summaries to better understand income distribution
      • SENTRI vs Non-SENTRI distributions by POE
      • Income distribution by POE
      • Crossing volume distribution by POE
      • Tour purpose distribution by POE
  3. Additional Data
    • There are 2 main sources for additional data: SR-11 and Darlanne's database/list
      • Likely there will be more data available and gathered from Darlanne's list
      • Need to determine what data is available and/or how to obtain
    • Thus far, planning on gathering the following data
      • Colonia data
        • Population, employment, and other demographic socio-economic data
      • Colonia to POE skims
        • Each POE is a zone, so there are travel times from colonias to POE zone and also travel times from POE zone to San Diego (USA) TAZs
        • Need this data for the most current year given the border crossing survey was conducted in 2019
          • There is 2017 data available
          • There are also forecasted volumes for different years (for both build and no build scenarios)
      • Wait times by POE by lane type
      • Existing and future lane configurations
      • Crossing volumes (observed and forecasted) by mode, lane type and purpose
    • Wu to send data is becomes available
  4. POE MGRA Issues
    • POEs are tied to an MGRA in San Diego and has caused issues before
      • There may be a need to separate MAZs for external stations
      • The current problem is tied to allocating travel activity to large MAZs at border which causes e.g. issue for pedestrian crossers trying to access transit station (inaccurate/excessive walk access times)
        • Would be better to have separate MAZs for external stations (i.e. POEs) and place centroid at exact location of pedestrian crossing for increased accuracy
  5. Mobility Hubs
    • The model should consider micromobility, microtransit and/or bikes given there will be mobility hubs at the port of entries
      • Discussion tabled for next meeting
Meeting Notes 07/01/20
  1. Data
    • SANDAG had internal discussion on what data (presented by Darlanne) is available. Need to discuss further to determine what data is most importand and whether there is missing data for collection
    • RSG internally discussed and reviewed previous model design
      • Some questions about the border survey data came up and will be discussed with Tom from True North in tomorrow's meeting
      • US resident data is missing. Is this coming?
        • More data was expected to come in this month, however that data collection was halted due to COVID. If there is a window of opportunity between now and June 2021 where conditions are back to normal, then the pending data collection can be conducted
      • About 40% of income question was not answered. Planning to develop some machine learning imputation algorithm for this
      • Other data processing is being planned as well such as creating trips file, place file and linking trips
      • Likely that data requests will be made to collecte new data and update some old data
      • Need to determine base year for new model. Once that is determined, data collection could be needed
        • SANDAG leaning on using 2019 as base year given survey data was collected in 2019
        • Will need to create 2019 network
    • Two other sources of data: SR-11 and Border Data Plan
  2. SR-11
    • RSG reviewed GitHub
    • Is there a model design or report for this model?
      • No model design documentation, however there is an innovation document
      • Nico to send innovation documents to RSG. Includes chapters on model discussion and assumptions.
  3. Border Data Plan
    • There is an internal wiki available listing all data. There could be some overlap with SR-11.
    • Any data that RSG needs will, for the time being, require SANDAG staff to access and send
      • Wu to check with Darlanne if access to wiki and database is possible
    • Wu will meet with Darlanne to determine where Border related data is available
  4. Model Estimation
    • What will be used for model estimation? Auto or more traditional method of estimation?
      • Data needs to be processed and organized first. Then, need to decide on model design (sub-models), and finally, models need to be programmed to write out data for estimation.
      • Model design will likely be similar to previous model
  5. Action Items
    • @SANDAG to determine model base year
    • @Neeco to send SR-11 innovation documents to RSG
    • @Wu to check with Darlanne if Border Data Plan wiki and database can be shared with RSG
    • @RSG will meet with Tim from True North to discuss survey data
    • @RSG to process survey data
Meeting Notes 06/17/20
  1. Project Page Setup on GitHub Wiki
    • Cross Border Model (CBM) meeting notes will be hosted here in the SANDAG ABM GitHub Wiki repository
      • Is there a need to make a separate wiki repository for the CBM?
        • Probably not given the CBM will eventually be merged into ABM3. We should leave it as is for now.
    • There is now an asim-cross-border branch available for the CBM
    • @RSG to upload Scope of Work to the Project Page
  2. Progress on Kick-Off Meeting 'To-Do' Items
    • Project page setup has been implemented on SANDAG ABM GitHub Wiki
    • Grace has sent data
      • @RSG to look at data
    • @Wu has to check on progress of data from SR-11
      • @Rick will help with this
      • Data is spread across different locations. What data does RSG need and when?
        • Tijuana Network (TransCAD network for current and future)
          • This can be found in the GitHub site. @Rick to send to Ben
        • Wait time component information
          • Methodology and code
      • Rick will go over SR-11 GitHub page to go over structure and data (see item 8 below)
    • @Darlanne's team to work and provide border related data by the end of this month
    • Grace to set up meeting with Tim from True North.
      • Meeting has been set for Thursday July 2nd.
  3. Model Design
    • Some key components:
      • Border transportation infrastructure at POEs such as: managed lanes, pedestrian facilities, highways, and border crossing schematics and configurations
      • Mobility Hubs:
        • Land-Use will increase population density near mobility hubs. The model should reflect this densification.
        • Planners envision emerging technology implementation at mobility hubs. The model should consider emerging technology.
    • Unsure about incorporation of the following two components:
      • CBX. Will this be incorporated into the CBM?
        • Probably not given the scope of work does not forecast demand for the airport, rather it forecasts Mexican travel to San Diego County
        • Therefore, the CBX model will remain a separate model. Its travel demand is for a different group and purpose of travelers than typical travelers to POEs.
        • The data collected thus far does show a segregation of people crossing POEs and people accessing the CBX
          • Need to confirm this: For Tijuana airport access, 2/3 of travelers are using CBX will a 1/3 are crossing the border and then navigating to Tijuana airport.
      • Freight. Will this be incorporated into the CBM?
        • It seems this does not have the same level of complexity as the air travel component.
        • What does POE validation consist of?
  4. POE Validation
    • Validation considers the following data/information:
      • Time-of-Day
      • Vehicle type
        • Extends to serve mode e.g. walking, bus, personal vehicles, Commercial vehicles
      • Wait time
        • Influenced by traveler type and their respective documentation type (i.e. ready lane, sentri, medical lane)
      • Lane type
      • Traveler type
        • General
        • SENTRI
      • Trip/Travel purpose
  5. Travel Survey Data
    • There was an item in the travel survey related to purpose. Could that information help identify information on the Mexican commercial vehicle market?
      • No, did not ask about cargo
  6. Project Management Plan
    • @Ben to prepare this, create draft and send to team
    • Will update the Scope of Work
    • Will update schedule
  7. Data
    • @Cundo and @Binny to set up a time to go over data with Joel
      • Afterwards speak with Ben
  8. SR-11 GitHub Review (Rick Curry)
    • The SR-11 GitHub Repository is called sr11_ome_v2
    • Under docs
      • Typical user guide items: model components, how-to set up and run model, inputs, outputs, etc.
      • POE rate file
      • Bit of information on meta data
        • Lanes
        • Processing rates
        • Direction of travel (NB, SB)
    • Under scenario_inputs
      • Network
      • POE rate files for both no build and build scenarios (different POE schematics/configurations)
    • Under data_in
      • The port_configurations.csv file
        • For each POE, will show:
          • Port number
          • Vehicle type
          • Traveler type
          • Direction (NB, SB)
          • Default toll
          • Initial queue
            • POEs are never empty, so initial queue is added at the beginning of the day
          • Max wait time
    • Under calibration
      • @RSG to look go over data here and update to make sure it corresponds to what is needed for the model
    • Under powerbi and tableau
      • Scripts used to create outputs in both powerbi and tableu
    • Under model
      • Run scripts
      • POE choice model
    • Outputs
      • @Rick directed RSG to a directory with model runs to look through model setups and model outputs. Inputs can be found in the repository
      • What are the dimensions of POE wait time output files?
        • Hour
        • POE name
        • Traveler type
        • Direction
        • Sim volume
        • Sim wait time
        • Process volume
          • Volume that was processed is differentiated with the volume that arrived (i.e. sim volume)
        • Queue
          • Queue left over at the end of the day
        • Toll
          • At Otay Mesa East
  9. Action Items
    • @RSG to upload Scope of Work to project page
    • @RSG to look over data Grace sent
    • @Darlanne's team to work and provide border related data by the end of the month
    • @Ben to create project management plan draft, which includes updating scope of work and schedule
    • @Cundo and @Binny to set up meeting with Joel to go over data
    • @RSG to check calibration SR-11 repository data and make updates corresponding to CBM needs
Meeting Notes 06/10/20
  1. Introductions (All)
  2. Background & Objectives (Wu)
    • Cross border model created from a 2010 cross border survey
      • There was an update in 2019 to the cross-border survey
      • One of the tasks include updating the model based on this survey
    • Will be one of the first model components that will transfer over to ActivitySim
      • Goal is to make the model perform better (e.g. run time) than the java model
      • Hopefully, it will make debugging easier than the java-based model
  3. 2019 Crossborder Survey (Grace)
    • Caltrans grant funded
      • True North applied and won the RFP for the cross-border survey
    • 1900 travel diaries collected
      • This has been cleaned and is ready for hand-off
    • Goal was to get 7400 intercept surveys
      • RSG would be interested in all of these and all data available
      • No analysis has been done on these intercept surveys, so it is not clear how clean or valid the data is
        • True North utilized digital surveys for 2019 survey (as opposed to paper and pencil 2010 survey data) so this provided for less missing data
        • True North is working on report for the collected data and could be done by the end of the month
          • There will be comparisons between 2019 and 2010
    • @RSG check on confidentiality agreement
    • @Grace will setup a meeting with Tim (and Rick?) to discuss the survey
    • Data file have been provided in SPSS but can have True North convert them to CSV
    • @Grace share data with RSG
  4. Border Policies in Regional Plans (Hector)
    • Borders programs coordinates 2 groups that have bi-national coordination and participation
      • Common areas:
        • Housing balance
        • Transportation infrastructure
        • Energy supply
        • Freight
      • Currently working with Caltrans on Border Master Plan
      • California’s number 1 export market is Mexico
      • Currently working on covid-19 report, which unfortunately did not make any references to cross-border interactions
  • Border Policies in Regional Plans (Jennifer)
    • Most regional plans thus far have been focused on trolley line expansions and rapid bus services
    • New regional plans:
      • Better connectivity and transportation services
      • How to link communities to transportation network via mobility hubs
      • Heavy rail line providing connections from border to San Diego potentially grade separated (subway perhaps)
      • Airport-to-airport connection (CBX to SAN)
      • Provide second or third runway to SAN
      • CBX has been very successful
      • Rapid services with dedicated guideways
      • Enhance Otay Mesa POE access to transit station that could enhance pedestrian infrastructure
      • Developing a system of managed lanes for highways in the region
    • @Jennifer to send a list of projects to RSG
    • There is a need for a model to determine effects of these new regional plan items
    • New regional plan is highly focused at cross-border interactions
  • SR11, Data, and Modeling (Rick)
    • Current traffic and revenue studies
      • Tier 2 traffic and revenue studies recently completed by WSP
        • Include border model, forecasts, and data
      • Investment Grade
        • Will begin July or August of 2020
        • Heavy coordination will be required between RSG and this consultant
    • Project Overview
      • Sr-11 will be constructed toward the POE with construction to begin in 2020
      • Within POE, the site is designed that multiple configurations can be built
        • Interchangeable lanes
        • Way it’s being built allows for multiple configurations of regular vehicle and commercial vehicles
    • Data
      • Value of Time info
      • Forecast and border crossing volumes
      • PoV and CV
      • Cross border surveys
      • TJ network congestion
      • Wait times
      • Throughput volumes
      • Hours of operation
      • Processing rates
      • Traveler / lane type forecast
      • Future Tijuana facilities
      • Max/min lane configurations
      • More data to come
    • Physical Innovations
      • Otay Mesa can operate at different configurations
    • T&R Model
      • Tier 2 traffic and revenue model version 2.1
        • POE choice model added
      • Model available on Github
    • Model Requirements
      • Runs on TransCAD
      • Also utilizes R
    • Model Overview
      • POEs are modeled
      • Hourly time periods
      • Demand for passenger vehicles
      • Trucks
    • Model Flow Chart
      • 5 outer feedback loops
    • Models Inputs and Outputs
      • Inputs
        • Passenger and freight forecasts
        • Networks
        • Processing rates
        • Number of open lanes and hours of operations
        • Value of time
      • Outputs
        • Wait times
        • Lane configs
        • Queue lengths
    • Results
      • On Tableau, MS Power BI, and Excel
    • Model Overview
      • Strictly a vehicle-based model
      • How does toll of sr-11 impact POE demand?
        • ABM can help identify this
      • Overlap
        • Border data
        • Tj network
        • Zone system
        • Demand projections
        • Socio-economic data
        • Survey analysis
        • DES
        • POE choice model
      • Issues
        • Timing and Velocity
          • Projects are well aligned but velocity of projects are very different
          • Firm will continue to use WSP model with minor enhancements (e.g. looking at weekends)
    • Questions/Comments
      • RSG would like to use all data used thus far
        • Need to verify if data is shareable
        • There is a plan to have the data ready to share
      • Develop meta-model of disaggregate model?
        • Could reduce run time
        • This was something that had been proposed to reduce wait time portion of model
      • Is there a database of different runs with different scenarios?
        • Yes, but still building it out further
        • More scenarios will be run in the coming weeks
      • Why does it take a long time to run?
        • Largest problem is that with every little change to toll or adding a lane, really influences POE choice. Such changes really changes demand profile of model.
      • What other data will be purchased?
        • Data from INRIX on travel time and speed information in Tijuana
        • Street light data and OD information
        • All this will become clearer in the next week or so. Need to talk to consultant
        • Will probably be purchased up until January of 2021
  • Other border related data (Darlanne)
    • Border Data Acquisition Plan
      • Work in progress
      • Items in yellow are items that could be most useful to RSG; items with ‘*’ are readily available
    • @RSG to let Darlanne know what data has the most priority, @Darlanne share inventory table
    • Base year of the model still not certain
  1. ActivitySim (Ben)
  2. Scope of Work and Schedule (Ben/Joel)
    • Presentation
    • Need Data as soon as possible
    • Scheduled completion by end of 2021
    • Rather than use SANDAG’s Confluence for meeting minutes, use the Wiki so other partners have access
    • This model will not be ready for 2021 regional plan given the plan will be done by the end of year 2020 and this model will be done by end of 2021
    • After RSG proposes structure, SANDAG needs to review with planners to get go-ahead
  3. Action Items (All)
    • Create these meeting notes and a project page to start managing project notes
    • @Grace to send data and data dictionary to RSG
    • @Wu to check where data is from sr-11
    • @Need to check on data sharing agreement
    • @Set up meeting with Tim from True North
    • @RSG will start looking at things to develop design

User Guide

ABM2+

Appendices

Cross Border Model Update

ABM3 Progress

Clone this wiki locally