Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CsvParseStream: negative fieldsPerRecord doesn't work with skipFirstRow or columns #6434

Open
mfulton26 opened this issue Feb 20, 2025 · 0 comments
Labels
bug Something isn't working needs triage

Comments

@mfulton26
Copy link
Contributor

Describe the bug

According to fieldsPerRecord documentation, I should be able to parse CSV with variable length records. I can successfully do so but not when I use skipFirstRow and/or columns.

working example
import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
import { assertEquals } from "jsr:@std/[email protected]/equals";
import { assertType, IsExact } from "jsr:@std/[email protected]/types";

const source = ReadableStream.from([
  "name,age\n",
  "Alice,34\n",
  "Bob\n", // incomplete record
]);
const stream = source.pipeThrough(new CsvParseStream({ fieldsPerRecord: -1 }));
const result = await Array.fromAsync(stream);

assertEquals(result, [
  ["name", "age"],
  ["Alice", "34"],
  ["Bob"],
]);
assertType<IsExact<typeof result, string[][]>>(true);

Steps to Reproduce

  1. Create a script with the contents of any of the following examples

    not working with skipFirstRow
    import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
    import { assertEquals } from "jsr:@std/[email protected]/equals";
    import { assertType, IsExact } from "jsr:@std/[email protected]/types";
    
    const source = ReadableStream.from([
      "name,age\n",
      "Alice,34\n",
      "Bob\n", // incomplete record
    ]);
    const stream = source.pipeThrough(
      new CsvParseStream({
        fieldsPerRecord: -1,
        skipFirstRow: true,
      }),
    );
    const result = await Array.fromAsync(stream);
    
    assertEquals(result, [
      { name: "Alice", age: "34" },
      { name: "Bob", age: undefined },
    ]);
    assertType<IsExact<typeof result, Record<string, string | undefined>[]>>(true);
    not working with columns
    import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
    import { assertEquals } from "jsr:@std/[email protected]/equals";
    import { assertType, IsExact } from "jsr:@std/[email protected]/types";
    
    const source = ReadableStream.from([
      "Alice,34\n",
      "Bob\n", // incomplete record
    ]);
    const stream = source.pipeThrough(
      new CsvParseStream({
        fieldsPerRecord: -1,
        columns: ["name", "age"],
      }),
    );
    const result = await Array.fromAsync(stream);
    
    assertEquals(result, [
      { name: "Alice", age: "34" },
      { name: "Bob", age: undefined },
    ]);
    assertType<
      IsExact<typeof result, Record<"name" | "age", string | undefined>[]>
    >(true);
    not working with skipFirstRow and columns
    import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
    import { assertEquals } from "jsr:@std/[email protected]/equals";
    import { assertType, IsExact } from "jsr:@std/[email protected]/types";
    
    const source = ReadableStream.from([
      "name,age\n",
      "Alice,34\n",
      "Bob\n", // incomplete record
    ]);
    const stream = source.pipeThrough(
      new CsvParseStream({
        fieldsPerRecord: -1,
        skipFirstRow: true,
        columns: ["name", "age"],
      }),
    );
    const result = await Array.fromAsync(stream);
    
    assertEquals(result, [
      { name: "Alice", age: "34" },
      { name: "Bob", age: undefined },
    ]);
    assertType<
      IsExact<typeof result, Record<"name" | "age", string | undefined>[]>
    >(true);
  2. Execute the script with deno run

  3. See error:

    error: Uncaught (in promise) Error: Syntax error on line 3: The record has 1 fields, but the header has 2 fields
        throw new Error(
              ^
        at convertRowToObject (https://jsr.io/@std/csv/1.0.5/_io.ts:233:11)
        at CsvParseStream.#pull (https://jsr.io/@std/csv/1.0.5/parse_stream.ts:460:28)

Expected behavior

  1. The scripts should run successfully.
  2. TypeScript type checking should also pass.
  3. Record<T, string | undefined> or Partial<Record<T, string>> should be used instead of Record<T, string>

Environment

@mfulton26 mfulton26 added bug Something isn't working needs triage labels Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage
Projects
None yet
Development

No branches or pull requests

1 participant