Typesafe API for processing iterable data in TypeScript and JavaScript.

Overview

Stream API

Code Style: Google CI Pipeline

Type-safe API for processing iterable data in TypeScript and JavaScript similarly to Java 8 Stream API, LINQ or Kotlin Sequences. Unlike RxJS this library is not asynchronous, you get an immediate result without any subscription or await call.

If this library makes your life easier or your boss happier, and you want to support my work then you can always thank me with a free coffee.

donate

Getting started

npm install --save @szilanor/stream

Alternatively use Stream from CDN by adding this to your HTML:

<script src="https://unpkg.com/@szilanor/stream/"></script>

Classic Javascript solution

const result = [1, 2, 3].filter(x => x % 2 === 0).map(x => x * 2);

Stream API solution

Creating a Stream object

import {Stream, of, from} from '@szilanor/stream';

let stream: Stream<number>;
stream = new Stream([1, 2, 3]); // With constructor;
stream = of(1, 2, 3); // With the 'of' creator function
stream = from([1, 2, 3]); // With the 'from' creator function

Operations on stream entries for the same result

import {filter, map, compound} from '@szilanor/stream';

// Chaining pipes
stream = stream.pipe(filter(x => x % 2 === 0)).pipe(map(x => x * 2));

stream = stream.pipe(
  filter(x => x % 2 === 0),
  map(x => x * 2)
);

// Using the 'compound' operation
stream = stream.pipe(
  compound(
    filter(x => x % 2 === 0),
    map(x => x * 2)
  )
);

Process the stream for the same result

import {toArray} from '@szilanor/stream';

// Using for..of
let result = [];
for (let entry of stream) {
  result.push(entry);
}

// With a collector
result = stream.collect(toArray());

Why Stream API?

  • Can achieve faster results
/*
 * Since the API is using Javascript Iterables the operations are applied
 * to the entries one bye one in order. In this example we know the
 * answer after the second element, so it is unnecessary to map everything
 * first.
 */

let allEven: boolean;
const input = [1, 2, 3, 4, 5];

// Classic JS maps all the entries first then returns false
allEven = input.map(x => x + 1).every(x => x % 2 === 0);

// vs Stream API maps only the first element then returns false
allEven = from(input)
  .pipe(map(x => x + 1))
  .collect(all(x => x % 2 === 0));

Benchmark file based on this example

Classic JS x 1,833 ops/sec ±0.25% (94 runs sampled)
Stream API x 123,688 ops/sec ±1.19% (88 runs sampled)
Fastest is Stream API
  • More readable code
import {from, distinct, collect} from '@szilanor/stream';

// Filtering duplicates and group them by whether they are even or odd
const input = [1, 1, 1, 1, 2, 3, 4, 4, 5];

// Classic JS
const resultClassic: Map<string, number[]> = new Map<string, number[]>();
Array.from(new Set<number>(input)).forEach(x => {
  const key = x % 2 === 0 ? 'even' : 'odd';
  resultClassic.set(key, [...(resultClassic.get(key) || []), x]);
});

// Stream API
const resultStreamApi: Map<string, number[]> = from(input)
  .pipe(distinct())
  .collect(groupBy(x => (x % 2 === 0 ? 'even' : 'odd')));
  • You can create your own operators / collectors if you don't find what you need
import {CollectorFunction, OperationFunction} from '@szilanor/stream';

const myAwesomeCollector: CollectorFunction<unknown, unknown> = {
  /* your own implementation */
};
const myAwesomeOperation: OperationFunction<unknown, unknown> = {
  /* your own implementation */
};

const result = of(1, 2, 3)
  .pipe(myAwesomeOperation())
  .collect(myAwesomeCollector());
You might also like...

All five assignments and the final group project is done in class CSCI5410 (Serverless Data Processing) Fall 2021 of MACS at Dalhousie University.

Dalhousie University | Fall 2021 | CSCI5410 | SDP (Serverless Data Processing) All five assignments and the final group project is done in class CSCI5

Dec 26, 2021

A slick loader to use during your AJAX calls or data processing

Slick Loader A slick loader to use during your AJAX calls or data processing Doc Installation Simply import slick-loader into your HTML. link rel="st

Jan 21, 2022

ndarray/tensor data processing for modern browsers

nadder Easy n-dimensional data manipulation with NumPy syntax. Installation npm i nadder # or yarn add nadder, or pnpm add nadder Usage import { ndarr

Dec 23, 2022

This project is built with JavaScript, Webpack, HTML & CSS, Leaderboard api. When user clicks on Refresh button it hits the api and responds with the data, The user can also post data to the api

This project is built with JavaScript, Webpack, HTML & CSS, Leaderboard api. When user clicks on Refresh button it hits the api and responds with the data, The user can also post data to the api

leaderboad Description the project. this project is about the leaderboad i did during Microverse to build a website for adding Data to the API and fet

May 30, 2022

Minecraft 1.8.9 mod which steals the access token and more things from the targetted user and sends to a backend server for processing. Disclaimer: For educational purposes only.

Minecraft 1.8.9 mod which steals the access token and more things from the targetted user and sends to a backend server for processing. Disclaimer: For educational purposes only.

R.A.T Retrieve Access Token Check DxxxxY/TokenAuth to login into an MC account with a name, token and uuid combo. Features Grabs the username, uuid, t

Jan 9, 2023

A Node.js module for processing .aff files.

affutil.js 一个照抄 feightwywx/arcfutil aff模块 的 Node.js模块 in Typescript 可以进行aff格式字符串与Javascript对象之间的相互转换 为Note对象提供了有限的方法 恐狼是神,快去用arcfutil 用法 导入: In Node.j

Jun 9, 2022

Provides a KafkaJS-compatible handler for processing messages that facilitates publishing to delayed-retry or dead-letter topics

kafkajs-async-retry This module handles retries and dead-lettering for messages from a Kafka topic without blocking the processing of subsequent messa

Dec 5, 2022

A set of scripts to test markdown processing speeds in various site generators/frameworks

bench-framework-markdown A set of scripts to test markdown processing speeds in various site generators/frameworks. Read the blog post: Which Generato

Nov 3, 2022

It shows an effective way to correct bus arrival information using data analytics based on Amazon Serverless such as Kiness Data Stream, Kinesis Data Firehose, S3, and Lambda.

It shows an effective way to correct bus arrival information using data analytics based on Amazon Serverless such as Kiness Data Stream, Kinesis Data Firehose, S3, and Lambda.

Amazon Serverless를 이용한 실시간 버스 정보 수집 및 저장 본 github repository는 버스 정보를 주기적으로 수집하여 분석할 수 있도록, Amazon Serverless인 Amazon Kinesis Data Stream, Kinesis Data

Nov 13, 2022
Comments
  • Use

    Use "fairer" comparison to "Classical JS"

    The examples used for "Classical JS" seems to use code that is very slow and contains extensive typing and unneeded code.

    As an example, here's a benchmark using the second to last code example (concerning readability):

    Map x 636 ops/sec ±124.98% (5 runs sampled)
    Stream x 30,502 ops/sec ±15.18% (37 runs sampled)
    Object x 441,113 ops/sec ±19.31% (34 runs sampled)
    Reduce x 1,285,775 ops/sec ±5.84% (43 runs sampled)
    Object with For Loop x 443,450 ops/sec ±13.82% (43 runs sampled)
    Fastest is Reduce
    

    As it can be seen, the map from the example is very very slow. Using an object instead of map greatly improves performance and outperforms this library (as also indicated in #2 )

    REPL with the benchmark

    In terms of readability I'd like to suggest a more "cleaned up" comparison with less extensive typing and removing unneeded code so users can have a fair comparison between this library and "Classic JS". See following TS Playground as an example with different approaches to the problem from the example.

    Example TS Playground

    opened by PedroHase 1
  • Benchmark not correct(?)

    Benchmark not correct(?)

    When using the provided benchmark, I noticed that I could not reproduce the stated result. In my tests, this library always was the slowest.

    The results of my benchmark (I added a For Loop as comparison):

    Classic JS x 11,375,579 ops/sec ±10.54% (48 runs sampled)
    Stream API x 33,607 ops/sec ±25.26% (40 runs sampled)
    For loop x 34,567,827 ops/sec ±91.45% (48 runs sampled)
    Fastest is For loop
    

    Online REPL

    Did I maybe do something wrong in my setup? If not, I think it'd be good to either remove the benchmark or update them.

    opened by PedroHase 1
A workshop about JavaScript iteration protocols: iterator, iterable, async iterator, async iterable

JavaScript Iteration protocol workshop A workshop about JavaScript iteration protocols: iterator, iterable, async iterator, async iterable by @loige.

Luciano Mammino 96 Dec 20, 2022
Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

JavaScript-med-Ramverk-Laboration-3 Grupprojektet för kurserna Javascript med Ramverk och Agil Utveckling. Utvecklingsguide För information om hur utv

Svante Jonsson IT-Högskolan 3 May 18, 2022
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
TypeSafe MongoDB Atlas Data API SDK for Deno & Deno Deploy

Atlas SDK atlas_sdk is a TypeSafe MongoDB Atlas Data API SDK for Deno & Deno Deploy Links Docs Import Replace LATEST_VERSION with current latest versi

Erfan Safari 20 Dec 26, 2022
End-to-end typesafe APIs with tRPC.io in SvelteKit applications

✨ tRPC-SvelteKit End-to-end typesafe APIs with tRPC.io in SvelteKit applications. No code generation, run-time bloat, or build pipeline. ❤️ ???? See b

Ionut-Cristian Florescu 307 Dec 29, 2022
typesafe nodejs client for transit.land

TransitLand Graphql Client About Since the Transitland released its new and shiny GraphQL API, it is possible to query the API using GraphQL. The clie

ioki 5 Jan 9, 2023
Quick T3 Stack with SvelteKit for rapid deployment of highly performant typesafe web apps.

create-t3svelte-app Just Build npx create-t3svelte-app Outline Get Building npm yarn More Info ?? Early Version Note Prisma Requirements Available Tem

Zach 85 Dec 26, 2022
End-to-end typesafe APIs in Astro wesbites made easy

Astro x tRPC ?? End-to-end typesafe APIs in Astro wesbites made easy View Demo · Report Bug · Request Feature ?? Introducing astro-trpc astro-trpc is

Happydev 61 Dec 30, 2022
FormGear is a framework engine for dynamic form creation and complex form processing and validation for data collection.

FormGear is a framework engine for dynamic form creation and complex form processing and validation for data collection. It is designed to work across

Ignatius Aditya Setyadi 91 Dec 27, 2022