jsonlines-web
Web stream based jsonlines decoder/encoder
-
✅ Deno -
✅ browser -
✅ Node.js
This library supports JSON in the following formats:
- Line-delimited JSON (JSONLinesParseStream)
- NDJSON
- JSON lines
- Record separator-delimited JSON (JSONLinesParseStream)
- Concatenated JSON (ConcatenatedJSONParseStream)
See wikipedia for the specifications of each JSON.
install or import
Deno
https://deno.land/x/jsonlines/ https://doc.deno.land/https://deno.land/x/jsonlines/mod.ts
import {
ConcatenatedJSONParseStream,
ConcatenatedJSONStringifyStream,
JSONLinesParseStream,
JSONLinesStringifyStream,
} from "https://deno.land/x/[email protected]/mod.ts";
browser
import {
ConcatenatedJSONParseStream,
ConcatenatedJSONStringifyStream,
JSONLinesParseStream,
JSONLinesStringifyStream,
} from "https://deno.land/x/[email protected]/js/mod.js";
Node.js
https://www.npmjs.com/package/jsonlines-web
npm install jsonlines-web
import {
ConcatenatedJSONParseStream,
ConcatenatedJSONStringifyStream,
JSONLinesParseStream,
JSONLinesStringifyStream,
} from "jsonlines-web";
// if you need
// import { TextDecoderStream, TextEncoderStream } from "node:stream/web";
// import { fetch } from "undici";
Usage
A working example can be found at ./testdata/test.ts.
How to parse JSON Lines
./json-lines.jsonl
{"some":"thing"}
{"foo":17,"bar":false,"quux":true}
{"may":{"include":"nested","objects":["and","arrays"]}}
import { JSONLinesParseStream } from "https://deno.land/x/[email protected]/mod.ts";
const { body } = await fetch(
"https://deno.land/x/[email protected]/testdata/json-lines.jsonl",
);
const readable = body!
.pipeThrough(new TextDecoderStream())
.pipeThrough(new JSONLinesParseStream());
for await (const data of readable) {
console.log(data);
}
How to parse json-seq
./json-seq.json-seq
�{"some":"thing\n"}
�{
"may": {
"include": "nested",
"objects": [
"and",
"arrays"
]
}
}
import { JSONLinesParseStream } from "https://deno.land/x/[email protected]/mod.ts";
const { body } = await fetch(
"https://deno.land/x/[email protected]/testdata/json-seq.json-seq",
);
const recordSeparator = "\x1E";
const readable = body!
.pipeThrough(new TextDecoderStream())
.pipeThrough(new JSONLinesParseStream({ separator: recordSeparator }));
for await (const data of readable) {
console.log(data);
}
How to parse concat-json
./concat-json.concat-json
{"foo":"bar"}{"qux":"corge"}{"baz":{"waldo":"thud"}}
import { ConcatenatedJSONParseStream } from "https://deno.land/x/[email protected]/mod.ts";
const { body } = await fetch(
"https://deno.land/x/[email protected]/testdata/concat-json.concat-json",
);
const readable = body!
.pipeThrough(new TextDecoderStream())
.pipeThrough(new ConcatenatedJSONParseStream());
for await (const data of readable) {
console.log(data);
}
How to stringify JSON Lines
import { readableStreamFromIterable } from "https://deno.land/[email protected]/streams/mod.ts";
import { JSONLinesStringifyStream } from "https://deno.land/x/[email protected]/mod.ts";
const file = await Deno.open(new URL("./tmp.concat-json", import.meta.url), {
create: true,
write: true,
});
readableStreamFromIterable([{ foo: "bar" }, { baz: 100 }])
.pipeThrough(new JSONLinesStringifyStream())
.pipeThrough(new TextEncoderStream())
.pipeTo(file.writable)
.then(() => console.log("write success"));
How to stringify json-seq
import { readableStreamFromIterable } from "https://deno.land/[email protected]/streams/mod.ts";
import { JSONLinesStringifyStream } from "https://deno.land/x/[email protected]/mod.ts";
const recordSeparator = "\x1E";
const file = await Deno.open(new URL("./tmp.concat-json", import.meta.url), {
create: true,
write: true,
});
readableStreamFromIterable([{ foo: "bar" }, { baz: 100 }])
.pipeThrough(new JSONLinesStringifyStream({ separator: recordSeparator }))
.pipeThrough(new TextEncoderStream())
.pipeTo(file.writable)
.then(() => console.log("write success"));
How to stringify concat-json
import { readableStreamFromIterable } from "https://deno.land/[email protected]/streams/mod.ts";
import { ConcatenatedJSONStringifyStream } from "https://deno.land/x/[email protected]/mod.ts";
const file = await Deno.open(new URL("./tmp.concat-json", import.meta.url), {
create: true,
write: true,
});
readableStreamFromIterable([{ foo: "bar" }, { baz: 100 }])
.pipeThrough(new ConcatenatedJSONStringifyStream())
.pipeThrough(new TextEncoderStream())
.pipeTo(file.writable)
.then(() => console.log("write success"));
note
This library contains ReadableStream.prototype[Symbol.asyncIterator] polyfills. Importing this library will automatically enable ReadableStream.prototype[Symbol.asyncIterator].
develop
need to manually deno task transpile before release.
5 Sep 11, 2022
7 Jun 19, 2022
5 Aug 2, 2022
85 Nov 27, 2022
4 Nov 13, 2022
12 Nov 11, 2022
122 Jan 6, 2023
7 Nov 20, 2022
13 Aug 30, 2022
25 Nov 1, 2022