Overview

• Expand capabilities to handle both async and sync iterables - Fixes #20

Open Issues

• [x] Duplicating the tests for the async versions doesn't feel right - Let me know if there is a better way to handle this with ava
• [x] There is a problem in the original functionality around the stop-on-error test that needs to be resolved
  • The test has a bug in that neither p-map nor the test mapper function, which is provided for that specific test and is not the common version, await the calling of the delay() function for item 2
  • This means that item 2 will only ever return a Promise, never a value
  • When this is fixed to await the function call, it causes the returned results to change from [1, 3] to [1, 3, 2]
  • Fixing this test makes it clear that the concurrency setup loop does not wait for mappers, so the only way to make this test fail would be to have enough items that eventually the initial iteration could not reach the last items in the test array before the 100 ms delay on the exception expires
  • With concurrency = 1 this test behaves as expected since there are not unlimited unawaited promises created
  • Let me know your thoughts on how to resolve this as it becomes more apparent with the behavior changes needed for asyncIterable (where the concurrency setup iteration of asyncIterable has to be awaited to prevent inifinte runners from being created)
  • The test, in the state it's in in this PR, does not actually demonstrate that stop on error works as intended

e.g. a few seconds into waiting for the promise to finish, we could increase concurrency due to CPU limit being lowered below 20%, to a concurrency of 5x or something. Basically dynamic concurrency

I bump my service which are using p-map for testing from version 4.0.0 to 5.0.0 and now I got the following issue during my test.

node version v14.16.0

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: .../node_modules/p-map/index.js require() of ES modules is not supported. require() of .../node_modules/p-map/index.js from ..../tests/contracts/step_definitions/test.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules. Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from .../node_modules/p-map/package.json.

is is a breaking change not documented? if yes how I can fix then

note I use pMap using import

import pMap from 'p-map';

Hello,

How to use this library with typescript? I'm getting this error:

Must use import to load ES Module: /home/omar/Desktop/node_modules/p-map/index.js\n' +
    'require() of ES modules is not supported.\n' +
    'require() of /home/omar/Desktop/node_modules/p-map/index.js from /home/omar/Desktop/src/resolvers/views.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.\n' +
    'Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/omar/Desktop/node_modules/p-map/package.json.

Here's my tsconfig:

{
    "compilerOptions": {
        "target": "ES2018",
        "module": "CommonJS",
        "lib": [
            "es2020",
            "esnext.asynciterable"
        ],
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true,
        "esModuleInterop": true,
        "outDir": "dist",
        "baseUrl": "src",
        "sourceMap": true,
        "skipLibCheck": true,
        "forceConsistentCasingInFileNames": true,
        "moduleResolution": "node",
        "removeComments": true,
    },
    "include": [
        "src/**/*.ts",
        "prisma/**/*.ts",
    ],
    "exclude": [
        "node_modules/**/*",
        "_warmup/**/*",
        ".vscode/**/*",
        ".husky/**/*",
        "dist/**/*"
    ],
}

According to the documentation, pMap returns

a Promise that is fulfilled when all promises in input and ones returned from mapper are fulfilled, or rejects if any of the promises reject.

But strictly speaking, it's

a Promise that is fulfilled when all promises in input and ones returned from mapper are fulfilled, or rejects if any of the promises reject, or mapper or options.concurrency are invalid, or mapper throws.

It's a subtle difference, especially if you're awaiting pMap or passing an async function as mapper.

Hi @sindresorhus, I love your p-* libs use, but I find that most often, I want to be building interesting promise chains.

getTimeNow().then( addTime("2 weeks")).then (lookupCalendar()).then(lookForConflicts).then(notify("email")) is the kind of promise based data-flow system that promises usually have me creating.

The liftFn provided here allows for that fluent-style usage of p-map, taking the static elements- the mapper & whatever options- and lifting them into a handler that is called for each input.

Promise.resolve([1,2,3]).then( pMap.liftFn( x=> x* 3).then( console.log) ought log [3,6,9].

This vs:

var input= Promise.resolve([1,2,3])
var output= pMap( input, x=> x* 3).then( console.log)

• Source iterator next() functions can throw
• There are 3 distinct cases where next() is called, one of which does not have a try/catch around it that will reject the outer promise, resulting in an unhandled promise rejection
• Note: in the current state, the 3rd test fails - I'm not sure how you want to handle this... a try/catch for the entire next() function would handle it... but there are other ways, such as a try/catch within the catch...

I encountered the need for p-flat-map. Here's what I came up with:

const pFlatMap = async <Element, NewElements>(
  input: Iterable<Element>,
  mapper: pMap.Mapper<Element, NewElements>,
  options?: pMap.Options
): Promise<FlatArray<NewElements[], 1>[]> => {
  const mapped: NewElements[] = await pMap(input, mapper, options)
  return mapped.flat()
}

Usage example:

const products = await pFlatMap(categories, listProductsForCategory, { concurrency: 50 })

Would it be possible to add this to the collection?

I happen to configure the concurrency option through JSON, but JSON cannot encode a Infinity value. Would you agree to taking 0 or -1 as an alias for Infinity?

Random undefined items are added to my mapped array with version 1.1.0. I don't have the same issue with version 1.0.0 or with bluebird. I've been able to consistently reproduce the issue with the following test. The same test works with concurrency 1 or concurrency >= myInput.length.

test('async with concurrency: 2', async t => {
	const myInput = [100, 200, 10, 36, 13, 45];
	const myMapper = value => {
		return new Promise(resolve => {
			setTimeout(() => resolve(value), value);
		});
	};

	t.deepEqual(await m(myInput, myMapper, {concurrency: 2}), myInput);
});

It is not obvious to see the extra undefined items with the ava output (I've never used ava before) but my mocha/chai version of the test is showing this output:

AssertionError: expected [ 100, 200, 10, , 36, , 13, , 45 ] to deeply equal [ 100, 200, 10, 36, 13, 45 ]
      + expected - actual

       [
         100
         200
         10
      -  [undefined]
         36
      -  [undefined]
         13
      -  [undefined]
         45
       ]

After upgrading to ^5.0.0, vite got an error:

browser-external:os:3 Uncaught Error: Module "os" has been externalized for browser compatibility and cannot be accessed in client code.
    at Object.get (browser-external:os:3:11)
    at index.js:6:27

After some digging, this is related to aggregate-error ^4.0.0's dep clean-stack ^4.0.0

import os from 'os' (index.js#L1)

I wonder could homedir be implemented in a browser-friendly way?

I don't know which repo is the most suitable place to open this issue, please transfer to the right place if needed.

Thank you!

Main differences from pMap():

• Skipped values are simply never yielded
• stopOnError is not available because this can now be done by the user themselves
• signal is also not available for the same reason
• Values that are yielded from the iterator, are immediately passed to the mapper and the returned values are yielded from the function in the order that they originally were yielded

If you're ok with this, I'll add docs and tests

Fixes https://github.com/sindresorhus/promise-fun/issues/21

Both, console.trace and throw new Error don't show the stack-trace beyond the point p-map got called. In other words, the functions that called before pMap, disappear from the stack trace. As a comparison, the p-map-series doesn't suffer from this issue. It does keep the stack-trace.

See the example below, if you run a function that runs the native Promise.all, the stack trace shows the function name - runPromiseNative. Same if you run the function runPromisePmapSeries. However, try to run runPromisePmap, and you'll see how the stack trace truncated.

I tried node v12.x and v14.x.

const pMap = require('p-map');

const pMapSeries = require('p-map-series');

async function promiseFn() {
  throw new Error('stop')
}

async function runPromiseNative() {
  await Promise.all([promiseFn()]).then(() => { console.log('completed') }).catch((err) => console.error(err));
}

async function runPromisePmap() {
  await pMap([promiseFn], fn => fn()).then(() => { console.log('completed') }).catch((err) => console.error(err));
}

async function runPromisePmapSeries() {
  await pMapSeries([promiseFn], fn => fn()).then(() => { console.log('completed') }).catch((err) => console.error(err));
}

// runPromiseNative();

runPromisePmap();

// runPromisePmapSeries();

Results when running runPromisePmap :

Error: stop
    at promiseFn (/Users/davidfirst/teambit/bit/pmap.js:6:9)
    at /Users/davidfirst/teambit/bit/pmap.js:14:33
    at /Users/davidfirst/teambit/bit/node_modules/p-map/index.js:57:28

Results when running runPromiseNative.

Error: stop
    at promiseFn (/Users/davidfirst/teambit/bit/pmap.js:6:9)
    at runPromiseNative (/Users/davidfirst/teambit/bit/pmap.js:10:22)
    at Object.<anonymous> (/Users/davidfirst/teambit/bit/pmap.js:21:1)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47

I wish I could stop the iteration early by returning something like pMap.stop (a Symbol) from the iterator.

This is basically the same request as https://github.com/sindresorhus/p-each-series/issues/3

For now, I have to throw an error on purpose just to break it. Thankfully, I don't need the return value of p-map anyway.

But with this feature, the parts of the array that weren't computed yet could be set to undefined, or pMap.uncomputed (another Symbol), or even be specified by a valueForUncomputedEarlyStop option.

Can I do a PR?

Do you have a module that calculates and exports a reasonable default value for concurrency? For example what is done in https://github.com/sindresorhus/got/pull/393. I'm planning to use p-map in some parts of nyc that can be async so I want to set a reasonable concurrency limit. I'd like to avoid calculating os.cpus().length multiple times and avoid duplicating the CI limiting logic.