Skip to content

Fix local file loader kwargs bug#390

Merged
ppinchuk merged 4 commits intomainfrom
pp/fix_local_file_loader_kwargs_bug
Mar 11, 2026
Merged

Fix local file loader kwargs bug#390
ppinchuk merged 4 commits intomainfrom
pp/fix_local_file_loader_kwargs_bug

Conversation

@ppinchuk
Copy link
Copy Markdown
Collaborator

We were previously pulling from the file kwargs tuple instead of the processed file loader kwargs, so sometimes we were getting an attribute error. Update behavior to pull fro mthe processed dict, so that we can always be sure it's a dictionary

@ppinchuk ppinchuk self-assigned this Mar 11, 2026
@ppinchuk ppinchuk requested a review from castelao as a code owner March 11, 2026 16:48
@ppinchuk ppinchuk added bugfix Fixed a known bug p-critical Priority: critical topic-python-general Issues/pull requests related to python labels Mar 11, 2026
Copilot AI review requested due to automatic review settings March 11, 2026 16:48
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.52%. Comparing base (5126e36) to head (9b282eb).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #390   +/-   ##
=======================================
  Coverage   54.52%   54.52%           
=======================================
  Files          61       61           
  Lines        5652     5652           
  Branches      530      530           
=======================================
  Hits         3082     3082           
  Misses       2525     2525           
  Partials       45       45           
Flag Coverage Δ
unittests 54.52% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a bug in the Python processing pipeline where local file loader kwargs were sometimes accessed from an unprocessed (and not always dict-like) value, causing attribute errors. The update routes lookups through the already-configured loader kwargs to ensure consistent dictionary access.

Changes:

  • Update local_file_loader_kwargs to read pdf_read_kwargs / html_read_kwargs from the processed self.file_loader_kwargs dict.
  • Add read_html_file support to the local file loader kwargs.
  • Minor Rust test-sample string formatting adjustment in qualitative ordinance sample output.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
crates/compass/src/scraper/ordinance/qualitative.rs Adjusts CSV header string formatting in the qualitative sample generator.
compass/scripts/process.py Fixes local file loader kwargs lookup to use processed dict; adds read_html_file coroutine for local HTML reads.

@ppinchuk ppinchuk merged commit 5698a48 into main Mar 11, 2026
26 checks passed
@ppinchuk ppinchuk deleted the pp/fix_local_file_loader_kwargs_bug branch March 11, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Fixed a known bug p-critical Priority: critical topic-python-general Issues/pull requests related to python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants