Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow partial reads and align with GTFS specifications for stop_lat, stop_long, and transfer_type #68

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
+50 −41
Open

Allow partial reads and align with GTFS specifications for stop_lat, stop_long, and transfer_type #68

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
744ed51
Loader will not fail on malformed records, instead will count and ski…
InterferencePattern Dec 15, 2022
5fbbd84
Added option to fail append_feed if bad records are found
InterferencePattern Dec 15, 2022
1baa416
Specified that stop_lat and stop_long are allowed to be empty
InterferencePattern Dec 15, 2022
e4acef5
Allowing transfer_type up to 5 and allowing None for stop_lat and sto…
InterferencePattern Dec 15, 2022
dab51b0
Changed return statement for conformity
InterferencePattern Dec 15, 2022
8e486e6
Changed f-strings for Python 2.7 compatibility
InterferencePattern Dec 20, 2022
c18a423
Made default value for ignoring corrupt lines False
InterferencePattern Jan 6, 2023
6b358a7
Added _validate_nullable_float_range function for lat and long valida…
InterferencePattern Jan 6, 2023
a4c679d
Simplified nullable float check to a single function
InterferencePattern Jan 6, 2023
f1f9b18
Clarified where lat and long are nullable for clarity
InterferencePattern Jan 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions pygtfs/loader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,15 @@ def append_feed(schedule, feed_filename, strip_fields=True,
read_records += 1
except:
skipped_records += 1
print(f"Failure while writing {record}")
print("Failure while writing {}".format(record))
if not ignore_failures:
raise
if i % chunk_size == 0 and i > 0:
schedule.session.flush()
sys.stdout.write('.')
sys.stdout.flush()
print(f'{read_records} records read for {gtfs_class}')
print(f'{skipped_records} records skipped for {gtfs_class}')
print('{0} records read for {1}'.format(read_records, gtfs_class))
InterferencePattern marked this conversation as resolved.
Show resolved Hide resolved
print('{0} records skipped for {1}'.format(skipped_records, gtfs_class))
schedule.session.flush()
schedule.session.commit()
# load many to many relationships
Expand Down