quiet option on imperfect tables

[Tim-at-AST]

I deal a lot with tables which have missing final fields (usually optional 'description' fields).

Currently I do xsv table file.csv to get a nice display of them, but I get a warning and stopped process when such rows are encountered.

CSV error: record 21 (line: 22, byte: 365): found record with 4 fields, but the previous record has 3 fields

I could go to the trouble of xsv fixlengths -l4 | xsv table but this involves me knowing in advance the number 4 (for example), and (to my naive mind) unnecessary processing.

Could xsv table somehow suppress the warning and force-continue the process, even without a perfect csv? Or must I really fix it first?

Currently the command columns -s, -te is seeming to be a preferred alternative.

[ajwinters]

I would like to +1 this issue. I run a select and join operation on 300+ million rows and recieve this error at random points. After looking at the specific lines and record I see that the field lengths are correct. And I tried pre-processing with awk and sed to fix the lengths, but I still get this issue. Also pre-processing with fixlengths command is incredibly slow once it gets to the presumed troubled lines/records.

[smillies]

+1 from me as well.
The error messages seem more or less random, at least in my case I could see no missing field from the incriminated lines.

[BurntSushi]

You guys are +1'ing the wrong issue. This issue isn't about an error message being reported incorrectly. It's about suppressing the error and continuing anyway.

If you are seeing an error message for a condition that did not occur, then please file a bug report with reproducible steps.

[smillies]

Well, yes. My problem is I can't give away the data with which it happened, and haven't yet succeeded in making them anonymous/random so I could provide you with a reproducible case. I'm not even completely sure that the error message is wrong, because xsv fixlengths makes it go away. But it may equally be possible that fixlengths is adding a spurious field. Perhaps my data have a quoting issue.

Meanwhile, it would be convenient if problematic records could just be skipped.

[BurntSushi]

This error doesn't have anything to do with missing or spurious records. It's about whether each record has a consistent number of fields.

Without a reproduction it will be difficult to track down your bug. My point is that there are two distinct issues here. Let's not conflate them.

[smillies]

Sorry, I wrote record in a few places were I meant field. I've edited my comments accordingly. I'll file another issue when I can create a reproducible case for the possibly wrong error message.

As for this issue, I'm still +1'ing it.