Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update csv cleaner #8828

Merged
merged 5 commits into from
Feb 7, 2025
Merged

feat: Update csv cleaner #8828

merged 5 commits into from
Feb 7, 2025

Conversation

sjrl
Copy link
Contributor

@sjrl sjrl commented Feb 7, 2025

Related Issues

  • fixes #issue-number

Proposed Changes:

At the suggestion of @mathislucka added a few more options to make it versatile and more consistent with our other cleaners.
I added the options:

  • remove_empty_rows --> user can control whether to remove empty rows
  • remove_empty_columns --> user can control whether to remove empty columns
  • keep_id --> Also keeping of the old Document ID like we have in our DocumentCleaner

How did you test it?

Added more tests.

Notes for the reviewer

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@sjrl sjrl requested a review from a team as a code owner February 7, 2025 10:25
@sjrl sjrl requested review from vblagoje and removed request for a team February 7, 2025 10:25
@github-actions github-actions bot added topic:tests type:documentation Improvements on the docs labels Feb 7, 2025
@sjrl sjrl requested review from davidsbatista and removed request for vblagoje February 7, 2025 10:27
@sjrl sjrl requested a review from a team as a code owner February 7, 2025 10:30
@sjrl sjrl requested review from dfokina and removed request for a team February 7, 2025 10:30
@coveralls
Copy link
Collaborator

coveralls commented Feb 7, 2025

Pull Request Test Coverage Report for Build 13199036941

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.06%) to 92.709%

Totals Coverage Status
Change from base Build 13183889468: 0.06%
Covered Lines: 9053
Relevant Lines: 9765

💛 - Coveralls

@sjrl sjrl added this to the 2.10.0 milestone Feb 7, 2025
Copy link
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sjrl sjrl merged commit 35788a2 into main Feb 7, 2025
18 checks passed
@sjrl sjrl deleted the update-csv-cleaner branch February 7, 2025 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants