Skip to content

nagasawaryoya/web-csv-toolbox

Β 
Β 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

npm version License: MIT PRs Welcome node version FOSSA Status

npm package minimized gzipped size GitHub code size in bytes npm codecov

🌐 web-csv-toolbox 🧰

A CSV Toolbox utilizing Web Standard APIs.

πŸ”—

GitHub npm API Reference Sponsor CodSpeed Badge

format: Biome test: Vitest build: Vite


Key Concepts ✨

  • 🌐 Web Standards first.
  • ❀️ TypeScript friendly & User friendly.
    • Fully typed and documented.
  • 0️⃣ Zero dependencies.
    • Using only Web Standards APIs.
  • πŸ’ͺ Property-based testing.
  • βœ… Cross-platform.
    • Works on browsers, Node.js, and Deno.

Key Features πŸ“—

  • 🌊 Efficient CSV Parsing with Streams
    • πŸ’» Leveraging the WHATWG Streams API and other Web APIs for seamless and efficient data processing.
  • 🎨 Flexible Source Support
    • 🧩 Parse CSVs directly from strings, ReadableStreams, or Response objects.
  • βš™οΈ Advanced Parsing Options: Customize your experience with various delimiters and quotation marks.
    • πŸ”„ Defaults to , and " respectively.
  • πŸ’Ύ Specialized Binary CSV Parsing: Leverage Stream-based processing for versatility and strength.
    • πŸ”„ Flexible BOM handling.
    • πŸ—œοΈ Supports various compression formats.
    • πŸ”€ Charset specification for diverse encoding.
  • πŸ“¦ Lightweight and Zero Dependencies: No external dependencies, only Web Standards APIs.
  • πŸ“š Fully Typed and Documented: Fully typed and documented with TypeDoc.
  • πŸš€ Using WebAssembly for High Performance: WebAssembly is used for high performance parsing. (Experimental)
    • πŸ“¦ WebAssembly is used for high performance parsing.

Installation πŸ“₯

With Package manager πŸ“¦

This package can then be installed using a package manager.

# Install with npm
$ npm install web-csv-toolbox
# Or Yarn
$ yarn add web-csv-toolbox
# Or pnpm
$ pnpm add web-csv-toolbox

From CDN (unpkg.com) 🌐

UMD Style πŸ”„

<script src="https://unpkg.com/web-csv-toolbox"></script>
<script>
const csv = `name,age
Alice,42
Bob,69`;

(async function () {
  for await (const record of CSV.parse(csv)) {
    console.log(record);
  }
})();
</script>

ESModule Style πŸ“¦

<script type="module">
import { parse } from 'https://unpkg.com/web-csv-toolbox?module';

const csv = `name,age
Alice,42
Bob,69`;

for await (const record of parse(csv)) {
  console.log(record);
}
</script>

Deno πŸ¦•

You can install and use the package by specifying the following:

import { parse } from "npm:web-csv-toolbox";

Usage πŸ“˜

Parsing CSV files from strings

import { parse } from 'web-csv-toolbox';

const csv = `name,age
Alice,42
Bob,69`;

for await (const record of parse(csv)) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files from ReadableStreams

import { parse } from 'web-csv-toolbox';

const csv = `name,age
Alice,42
Bob,69`;

const stream = new ReadableStream({
  start(controller) {
    controller.enqueue(csv);
    controller.close();
  },
});

for await (const record of parse(stream)) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files from Response objects

import { parse } from 'web-csv-toolbox';

const response = await fetch('https://example.com/data.csv');

for await (const record of parse(response)) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files with different delimiters and quotation characters

import { parse } from 'web-csv-toolbox';

const csv = `name\tage
Alice\t42
Bob\t69`;

for await (const record of parse(csv, { delimiter: '\t' })) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files with headers

import { parse } from 'web-csv-toolbox';

const csv = `Alice,42
Bob,69`;

for await (const record of parse(csv, { headers: ['name', 'age'] })) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Supported Runtimes πŸ’»

Works on Node.js

Versions Status
20.x βœ…
18.x βœ…

Works on Browser

OS Chrome FireFox Default
Windows βœ… βœ… βœ… (Edge)
macos βœ… βœ… ⬜ (Safari *)
Linux βœ… βœ… -

* To Be Tested: I couldn't launch Safari in headless mode on GitHub Actions, so I couldn't verify it, but it probably works.

Others

  • Verify that JavaScript is executable on the Deno. Deno CI

APIs πŸ§‘β€πŸ’»

High-level APIs πŸš€

These APIs are designed for Simplicity and Ease of Use, providing an intuitive and straightforward experience for users.

  • function parse(input[, options]): AsyncIterableIterator<CSVRecord>: πŸ“‘
    • Parses various CSV input formats into an asynchronous iterable of records.
  • function parse.toArray(input[, options]): Promise<CSVRecord[]>: πŸ“‘
    • Parses CSV input into an array of records, ideal for smaller data sets.

The input paramater can be a string, a ReadableStream of strings or Uint8Arrays, or a Uint8Array object, or a ArrayBuffer object, or a Response object.

Middle-level APIs 🧱

These APIs are optimized for Enhanced Performance and Control, catering to users who need more detailed and fine-tuned functionality.

  • function parseString(string[, options]): πŸ“‘
    • Efficient parsing of CSV strings.
  • function parseBinary(buffer[, options]): πŸ“‘
    • Parse CSV Binary of ArrayBuffer or Uint8Array.
  • function parseResponse(response[, options]): πŸ“‘
    • Customized parsing directly from Response objects.
  • function parseStream(stream[, options]): πŸ“‘
    • Stream-based parsing for larger or continuous data.
  • function parseStringStream(stream[, options]): πŸ“‘
    • Combines string-based parsing with stream processing.
  • function parseUint8ArrayStream(stream[, options]): πŸ“‘
    • Parses binary streams with precise control over data types.

Low-level APIs βš™οΈ

These APIs are built for Advanced Customization and Pipeline Design, ideal for developers looking for in-depth control and flexibility.

  • class LexerTransformer: πŸ“‘
    • A TransformStream class for lexical analysis of CSV data.
  • class RecordAssemblerTransformer: πŸ“‘
    • Handles the assembly of parsed data into records.

Experimental APIs πŸ§ͺ

These APIs are experimental and may change in the future.

Parsing using WebAssembly for high performance.

You can use WebAssembly to parse CSV data for high performance.

  • Parsing with WebAssembly is faster than parsing with JavaScript, but it takes time to load the WebAssembly module.
  • Supports only UTF-8 encoding csv data.
  • Quotation characters are only ". (Double quotation mark)
    • If you pass a different character, it will throw an error.
import { loadWASM, parseStringWASM } from "web-csv-toolbox";

// load WebAssembly module
await loadWASM();

const csv = "a,b,c\n1,2,3";

// parse CSV string
const result = parseStringToArraySyncWASM(csv);
console.log(result);
// Prints:
// [{ a: "1", b: "2", c: "3" }]
  • function loadWASM(): Promise<void>: πŸ“‘
    • Loads the WebAssembly module.
  • function parseStringToArraySyncWASM(string[, options]): CSVRecord[]: πŸ“‘
    • Parses CSV strings into an array of records.

Options Configuration πŸ› οΈ

Common Options βš™οΈ

Option Description Default Notes
delimiter Character to separate fields ,
quotation Character used for quoting fields "
headers Custom headers for the parsed records First row If not provided, the first row is used as headers

Advanced Options (Binary-Specific) 🧬

Option Description Default Notes
charset Character encoding for binary CSV inputs utf-8 See Encoding API Compatibility for the encoding formats that can be specified.
decompression Decompression algorithm for compressed CSV inputs See DecompressionStream Compatibility.
ignoreBOM Whether to ignore Byte Order Mark (BOM) false See TextDecoderOptions.ignoreBOM for more information about the BOM.
fatal Throw an error on invalid characters false See TextDecoderOptions.fatal for more information.

How to Contribute πŸ’ͺ

Star ⭐

The easiest way to contribute is to use the library and star repository.

Questions πŸ’­

Feel free to ask questions on GitHub Discussions.

Report bugs / request additional features πŸ’‘

Please register at GitHub Issues.

Financial Support πŸ’Έ

Please support kamiazya.

Even just a dollar is enough motivation to develop 😊

License βš–οΈ

This software is released under the MIT License, see LICENSE.

FOSSA Status

About

🌊A CSV Toolbox utilizing Web Standard APIs.

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 99.2%
  • Other 0.8%