Draft specification for a proposed Array.fromAsync method in JavaScript.

Overview

Array.fromAsync for JavaScript

ECMAScript Stage-1 Proposal. J. S. Choi, 2021.

Why an Array.fromAsync method

Since its standardization in JavaScript, Array.from has become one of Array’s most frequently used built-in methods. However, no similar functionality exists for async iterators.

Such functionality would be useful for dumping the entirety of an async iterator into a single data structure, especially in unit tests or in command-line interfaces. (Several real-world examples are included in a following section.)

There is an it-all NPM library that performs only this task and which gets about 50,000 weekly downloads daily. This of course does not include any code that uses ad-hoc for awaitof loops with empty arrays:

const arr = [];
for await (const item of asyncItems) {
  arr.push(item);
}

Further demonstrating the demand for such functionality, several Stack Overflow questions have been asked by various developers, asking how to convert async iterators to arrays.

Description

(A formal draft specification is available.)

Similarly to Array.from, Array.fromAsync would be a static method of the Array built-in class, with one required argument and two optional arguments: (items, mapfn, thisArg).

But instead of converting an array-like object or iterable to an array, it converts an async iterable (or array-like object or iterable) to a promise that will resolve to an array.

async function * f () {
  for (let i = 0; i < 4; i++)
    yield i;
}

// Resolves to [0, 1, 2, 3].
await Array.fromAsync(f());

mapfn is an optional function to call on every item value. (Unlike Array.from, mapfn may be an async function. Whenever mapfn returns a promise, that promise will be awaited, and the value it resolves to is what is added to the final returned promise’s array. If mapfn’s promise rejects, then the final returned promise will also reject with that error.)

thisArg is an optional value with which to call mapfn (or undefined by default).

Like Array.from, Array.fromAsync is a generic factory method. It does not require that its this value be the Array constructor, and it can be transferred to or inherited by any other constructors that may be called with a single numeric argument.

Other proposals

Object.fromEntriesAsync

In the future, a complementary method could be added to Object.

Type Sync method Async method
Array from fromAsync
Object fromEntries fromEntriesAsync?

It is uncertain whether Object.fromEntriesAsync should be piggybacked onto this proposal or left to a separate proposal.

Async spread operator

In the future, standardizing an async spread operator (like [ 0, await ...v ]) may be useful. This proposal leaves that idea to a separate proposal.

Iterator helpers

The iterator-helpers proposal puts forward, among other methods, a toArray method for async iterators (as well as synchronous iterators). We could consider Array.fromAsync to be redundant with toArray.

However, Array.from already exists, and Array.fromAsync would parallel it. If we had to choose between asyncIterator.toArray and Array.fromAsync, we should prefer Array.fromAsync to asyncIterator.toArray for its parallelism with what already exists.

In addition, the iterator.toArray method already would duplicate Array.from for synchronous iterators. We consider duplication with an Array method as okay anyway. If duplication between syncIterator.toArray and Array.from is already okay, then duplication between asyncIterator.toArray and Array.fromAsync should also be okay.

Records and tuples

The record/tuple proposal puts forward two new data types with APIs that respectively resemble those of Array and Object. The Tuple constructor, too, would probably need an fromAsync method. Whether the Record constructor gets a fromEntriesAsync method depends on whether Object gets fromEntriesAsync.

Set and Map

There is a proposal for Set.from and Map.from methods. If this proposal is accepted before that proposal, then that proposal could also add corresponding fromAsync methods.

Real-world examples

Only minor formatting changes have been made to the status-quo examples.

Status quo With binding
const all = require('it-all');

// Add the default assets to the repo.
const results = await all(
  addAll(
    globSource(initDocsPath, {
      recursive: true,
    }),
    { preload: false },
  ),
);
const dir = results
  .filter(file =>
    file.path === 'init-docs')
  .pop()
print('to get started, enter:\n');
print(
  `\tjsipfs cat` +
  `/ipfs/${dir.cid}/readme\n`,
);

From ipfs-core/src/runtime/init-assets-nodejs.js.

// Add the default assets to the repo.
const results = await Array.fromAsync(
  addAll(
    globSource(initDocsPath, {
      recursive: true,
    }),
    { preload: false },
  ),
);
const dir = results
  .filter(file =>
    file.path === 'init-docs')
  .pop()
print('to get started, enter:\n');
print(
  `\tjsipfs cat` +
  `/ipfs/${dir.cid}/readme\n`,
);
const all = require('it-all');

const results = await all(
  node.contentRouting
    .findProviders('a cid'),
);
expect(results)
  .to.be.an('array')
  .with.lengthOf(1)
  .that.deep.equals([result]);

From js-libp2p/test/content-routing/content-routing.node.js.

const results = await Array.fromAsync(
  node.contentRouting
    .findProviders('a cid'),
);
expect(results)
  .to.be.an('array')
  .with.lengthOf(1)
  .that.deep.equals([result]);
async function toArray(items) {
  const result = [];
  for await (const item of items) {
    result.push(item);
  }
  return result;
}

it('empty-pipeline', async () => {
  const pipeline = new Pipeline();
  const result = await toArray(
    pipeline.execute(
      [ 1, 2, 3, 4, 5 ]));
  assert.deepStrictEqual(
    result,
    [ 1, 2, 3, 4, 5 ],
  );
});

From node-httptransfer/test/generator/pipeline.test.js.

it('empty-pipeline', async () => {
  const pipeline = new Pipeline();
  const result = await Array.fromAsync(
    pipeline.execute(
      [ 1, 2, 3, 4, 5 ]));
  assert.deepStrictEqual(
    result,
    [ 1, 2, 3, 4, 5 ],
  );
});
Comments
  • Accept non-iterator array-like inputs, like Array.from?

    Accept non-iterator array-like inputs, like Array.from?

    Array.from accepts non-iterable array-like inputs (objects with a length property and indexed elements).

    Should the inputs of Array.fromAsync be a superset of Array.from?

    question 
    opened by js-choi 23
  • What happens with rejections in synchronous iterables?

    What happens with rejections in synchronous iterables?

    Are synchronous iterables/array-likes being iterated synchronously or asynchronously as they would in a for await? What is the expectation for a rejection within a sync collection that settles before to other promises prior? In a for await it goes unhandled. Would Array.fromAsync() be able to account for that and reject its returned promise or would it to allow for the unhandled rejection to slip through?

    const delay = (ms, reject = false) => new Promise((r, j) => setTimeout(reject ? j : r, ms, ms));
    
    for await (let ms of [delay(1000), delay(3000), delay(2000, true)]) {
      console.log(ms) // 1000, (unhandled error 2000), 3000
    }
    
    documentation question 
    opened by senocular 19
  • Should it await `next.value` when not passing a mapper function?

    Should it await `next.value` when not passing a mapper function?

    Consider this async iterable:

    const it = {
      [Symbol.asyncIterator]() {
        return {
          async next() {
            if (i > 2) return { done: true };
            i++;
            return { value: Promise.resolve(i), done: false }
          }
        }
      }
    }
    

    Unless I'm reading the spec wrong, await Array.fromAsync(it) returns [Promise.resolve(1), Promise.resolve(2), Promise.resolve(3)] while await Array.fromAsync(it, x => x) returns [1, 2, 3].

    Currently, when using Array.from, x => x is the "default mapper function" (even if it's not specified as such): Array.from(something) is always equivalent to Array.from(something, x => x). However, there is no "default function" that can be passed to Array.fromAsync.

    If we changed Array.fromAsync(something) to await the next.value values, await Array.fromAsync(it) would always return the same result as await Array.fromAsync(it, x => x). In practice, it means making it behave like

    const arr = [];
    for await (const nextValue of it) arr.push(await nextValue);
    
    question 
    opened by nicolo-ribaudo 13
  • Add ecma262 references

    Add ecma262 references

    I was working on test262 for this proposal, and I figured it'll be convenient if we had links to references of abstract operations that are defined in ecma262 spec (Eg: CreateAsyncFromSyncIterator).

    This PR makes a small change to add those references.

    opened by Aditi-1400 6
  • Iterator helpers vs this proposal

    Iterator helpers vs this proposal

    Stage 2 iterator helpers proposal already contains %AsyncIteratorPrototype%.toArray method that does absolutely the same. What are the advantages has Array.asyncFrom?

    opened by zloirock 6
  • Sync iterator’s and array-like objects’ values are not awaited

    Sync iterator’s and array-like objects’ values are not awaited

    On 64a49214f6a7036e7622ff4b4b9f938e277ae7ae, @zloirock left this comment:

    this fallback will not properly work with sync iterators.

    await Array.fromAsync([Promise.resolve(1), 2, Promise.resolve(3)]); // => [Promise.resolve(1), 2, Promise.resolve(3)]
    

    Something like that could be simpler.

    Let usingAsyncIterator be ? GetMethod(asyncItems, @@asyncIterator).
    Let usingIterator be undefined.
    If usingAsyncIterator is undefined,
      Let usingIterator be ? GetMethod(asyncItems, @@Iterator).
      If usingIterator is undefined,
        Let usingIterator be ? %Array.prototype.values%.
    ...
    If usingAsyncIterator is not undefined,
      Let iteratorRecord be ? GetIterator(asyncItems, async, usingAsyncIterator).
    Else,
      Let syncIteratorRecord be ? GetIterator(asyncItems, sync, usingIterator).
      Let iteratorRecord be ? CreateAsyncFromSyncIterator(syncIteratorRecord).
    

    The problem is that the current specification will not properly await each value yielded from a synchronous iterator. (We do want await to be called on each value from a synchronous iterator—this is what for await does.)

    This problem is due to how the spec currently incorrectly creates an synchronous iterator. Step 3.e.iii assigns iteratorRecord to GetIterator(asyncItems, async, usingAsyncOrSyncIterator), which results in a synchronous iterator that does not call await on each of its yielded values. What we need to do is copy GetIterator when it is called with an async hint. We need to call CreateAsyncFromSyncIterator on the created sync iterator, which in turn will create an Async-from-Sync Iterator object, whose next method will call await on each of its values. (Thanks @bakkot for the help).

    I will fix this later.

    bug 
    opened by js-choi 4
  • Unscopables?

    Unscopables?

    Other recent proposals adding methods to Array.prototype have added the name to Array.prototype[@@unscopables] -- I sort of imagine fromAsync should get added too.

    opened by mgaudet 2
  • Wrap bare return values with Completion Records in abstract closure

    Wrap bare return values with Completion Records in abstract closure

    Resolves #31, at least temporarily. (Also affects #14.)

    Eventually we might want to make a change to match whatever approach proposal-iterator-helpers does; see tc39/proposal-iterator-helpers#218. But that should be able to be a mere editorial change.

    bug 
    opened by js-choi 2
  • Wiring of return value is underspecified

    Wiring of return value is underspecified

    fromAsync makes use of an Abstract Closure passed to AsyncFunctionStart. What is the return type of this closure? The use of ? implies it must be a Completion Record, but there's also a bare Return A. which returns a value not wrapped in a completion record. In normal built-in functions this would be implicitly wrapped in NormalCompletion by the rules in this clause, but a.) that doesn't apply within the Abstract Closure (rather, it's the thing which allows the final Return promiseCapability.[[Promise]] line to work and b.) that wouldn't do the right thing anyway, because the logic in AsyncBlockStart treats an invocation returning a normal completion as if it had explicitly returned undefined (which is how normal functions work) - an actual return would be a return completion.

    I see three possible ways forward:

    1. in the closure within fromAsync, explicitly wrap each Return _value_. as Return Completion Record { [[Type]]: ~return~, [[Value]]: _value_, [[Target]]: empty }
    2. in AsyncFunctionStart, have a different path for Abstract Closures vs built-in functions
    3. define "built-in async function" rigorously in a way which would allow you to write the steps of a built-in function, but using Await

    Of these I think the third option is cleanest, especially if we add more built-in async stuff (as the iterator helpers proposal will do).

    question 
    opened by bakkot 2
  • Async callbacks

    Async callbacks

    opened by zloirock 2
  • Relationship with iterator-helpers

    Relationship with iterator-helpers

    The exciting iterator-helpers proposal has a similar toArray method that overlaps with this. (toArray already overlaps with Array.from too.) See #1.

    My preference is to keep both toArray and Array.fromAsync (Array.from being a fait accompli). But we can explore this more before both proposals advance further. CC: @codehag

    question 
    opened by js-choi 2
  • update readme for stage 3

    update readme for stage 3

    As of the September meeting, this proposal is now at stage 3. See https://github.com/tc39/notes/blob/HEAD/meetings/2022-09/sep-14.md#arrayfromasync-for-stage-3.

    opened by michaelficarra 1
  • remove built-in async function infrastructure from this proposal

    remove built-in async function infrastructure from this proposal

    See https://github.com/tc39/proposal-iterator-helpers/pull/245 and https://github.com/tc39/ecma262/pull/2942. This will align our efforts and allow reviewers to focus on the actual content of this proposal.

    opened by michaelficarra 3
  • Spec Tidy Suggestion: Don't Double Construct A

    Spec Tidy Suggestion: Don't Double Construct A

    As written, the spec -may- double counstruct A:

    It will be constructed first as part of Step 3.{e,f}; then if iteratorRecord is undefined, it's constructed again in Step 3.k.{iv,v}.

    Probably that first construction could be sunk into 3.j.

    opened by mgaudet 0
  • Spec Issue: IteratorStep returns a promise for async iterators, leading to an infinite loop

    Spec Issue: IteratorStep returns a promise for async iterators, leading to an infinite loop

    In the published spec, Step 3.j.ii.3 and 4 are the termination condition of the iteration:

    • Step 3.j.ii.3 "Let next be ? Await(IteratorStep(iteratorRecord))."
    • Step 3.j.ii.4 "If next is false, then..."

    The problem is that IteratorStep doesn't properly flag termination on an async iterator; the return value of the IteratorNext in async iteration is a Promise, which doesn't have a "done" property, and so we always get the promise object from IteratorStep.

    I think the patch is actually relatively simple; Step 3.j.ii.4 should be expanded into the steps which have the effect of "if next.done is false:"

    opened by mgaudet 1
  • Sharing machinary with iterator-helpers toArray

    Sharing machinary with iterator-helpers toArray

    Hey, I was wondering, since Array.fromAsync(iter) is similar to AsyncIterator.from(iter).toArray() wouldn't it make sense if they shared the same spec text? That is one (for example fromAsync) is defined and the other is defined in terms of it?

    (I agree that it makes sense to have both as it's more ergonomic and people will likely reach out to both - though that's not the discussion here)

    question 
    opened by benjamingr 4
Owner
Ecma TC39
Ecma International, Technical Committee 39 - ECMAScript
Ecma TC39
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
An example of implementation of the Veriifiable Presentation Generation Service specification.

Verifiable Presentation Generation Service A plugin-based service that allows issuers to render verifiable presentations from templates, and store it

Verifiable Presentation Generation 5 Nov 16, 2022
An OpenAPI specification for the MagicBell API.

MagicBell's OpenAPI Specification This repository contains OpenAPI specifications for the MagicBell REST API. Changelog Files can be found in the /spe

MagicBell 5 Dec 15, 2022
🔎 (Draft!) VSCode extension to show the search results in a tree view

vscode-search-tree ?? (Draft!) VSCode extension to show the search results in a tree view The work on this extension is on-pause for now since VSCode

Oleksii Trekhleb 16 Sep 7, 2022
Multi-chain defi crypto sniper written in typescript/javascript. Fastest method of sniping with auto-sell and rug prevention features.

CryptoSniper Community Edition Multi-chain defi crypto sniper written in typescript/javascript. Fastest method of sniping with auto-sell and rug preve

null 18 Nov 3, 2022
Demonstrating the Dashboard++ method of organizing a vault in Obsidian

Information This repository is an Obsidian vault that demonstrates using the Dashboard++ method for organizing and navigating notes. For further detai

null 194 Jan 3, 2023
Disallow form tags without explicit method="post"

eslint-plugin-require-form-method-post Disallow form tags without explicit method="post" Prevents sensitive data appearing on URLs Allow form tags wit

Darwin Christopher Tantuco 2 Apr 24, 2022
An example on how to use Solana Wallet Adapter as a Web Authentication Method.

Solana Wallet Auth: A FullStack example This example uses Solana's wallet adapter to sign messages and verifies their signatures on the backend, allow

Kevin Rodríguez 19 Dec 20, 2022
TimezoneDB is an easy, cross-platform method of keeping track of others' timezones.

TimezoneDB TimezoneDB is an easy, cross-platform method of keeping track of others' timezones. This project is inspired by PronounDB, and we'd like to

Synapse Technologies, LLC 13 Nov 16, 2022
Quick access to Laravel's helper method documentation — right from Alfred.

alfred-laravel-helper-docs alfred-laravel-helper-docs offers you rapid access to documentation for Laravel's helper functions — including those for Co

Stephan Casas 9 Aug 11, 2022
Get a quick hash that uses the well-liked Bernstein "times 33" hash method and delivers a hex string.

short-hash-ts -> Get a quick hash that uses the well-liked Bernstein "times 33" hash method and delivers a hex string. Installation Install short-hash

Younis Rahman 3 Sep 4, 2022
Cookbook Method is the process of learning a programming language by building up a repository of small programs that implement specific programming concepts.

CookBook - Hacktoberfest Find the book you want to read next! PRESENTED BY What is CookBook? A cookbook in the programming context is collection of ti

GDSC-NITH 16 Nov 17, 2022
Analisador de números utilizando Array JavaScript com Html 5 e CSS 3

Olá pessal, tudo bem? :D Esse site foi desenvolvido para analisar números armazenados em um array chamado "dados[]". Temos dois botões um input e uma

Yuri Willian 0 Jan 6, 2022
🧰 Javascript array-like containers for multithreading

بسم الله الرحمن الرحيم Struct Vec ?? Javascript array-like containers for multithreading Efficiently communicating between js workers is a pain becaus

Mostafa Elbannan 19 Jun 23, 2022
Fundamentos, estruturas, função, objeto, array e etc...

JavaScript Fundamentos, estruturas, função, array e etc... Atividades praticas detalhadas e comentadas para todo mundo entender cada tag, bons estudos

Leonardo Madeira 6 Feb 27, 2022
Complete JavaScipt Array methods Cheatsheet 🚀

Javascript Array Cheatsheet ?? Click to download ☝ Important ?? There is no way this is perfect or include all the methods. I'll try to fix/add more m

Ayush 91 Dec 7, 2022
Hierarchical Converter for Array of Objects

Conversor Hierárquico para Array de Objetos - Hierarchical Converter to Array of Objects Docker-compose Cria a interface network e containers indicado

Victor Vinícius Eustáquio de Almeida 2 Jan 27, 2022
A simple module to get porperties of an array

A simple module to get statistical porperties of an array. Currently the possible properties are mean, stdev and variance.

null 4 Nov 29, 2022