A quick analysis with a flamegraph shows that safe-stable-stringify is the major bottleneck for this cache.

We should investigate if we could come up with a faster algorithm for hashing objects but with the same properties.

proposal for use of an external storage, with optional invalidation

TODO

• [x] test on storage/s
• [x] 100% coverage (and significant cases)
• [x] use an empty logger on storage
• [x] fix CI, add a redis instance
• [x] restore expiration logic on memory storage / do not use Date.now
• [x] expose explicit "invalidation" function
• [x] improve redis storage (pipeline etc)
• [x] ~~storage gc function~~
• [x] memory storage with sync deletes
• [x] redis storage with sync deletes
• [x] optional invalidation
• [x] benchmarks
• [x] redis references TTL
• [x] redis gc
• [x] "stale-on-error" > next
• [x] solve TODOs
• [x] check redis pipeline
• [x] memory references: ~~use a btree? a set?~~ arrays are good enough
• [x] example
  • [x] basic
  • [x] mjs
• [x] update documentation

as well as onDedupe, onHit, onMiss, we can have "onError" event

~~consider to implement also "stale on error" and relative options when an error occur, for example the first implementation can be serve the latest response for an amount of time~~

To implement invalidation, I need to remove ttl and clear the deduped promise is resolved

Note: tests where the cache is suppose to be used are not passing

If this has gone too far from the original purpose, I'm happy to start a new module

Good day, when setting the log level to level higher than debug in fastify, log.debug becomes undefined and causes a failure. I can submit a PR that checks for a level's existence and skips if it isn't there

Let's say we want to cache something like an access token where expiration is set from the server.

const cache = createCache();
cache.define('fetchSomething', fetchSomethingHandler);

async function fetchSomethingHandler() {
  const data =  { "token": "abc", "expiresInSeconds": 60 }

  // something like this
  cache.fetchSomething.ttl(data.expiresInSeconds)
  return data;
}

expiresInSeconds is changing every function call

This fixes a memory leak in the cache for dedupes. We were previously setting the value of each key to undefined instead of deleting it. This caused a slow leak in this package that ultimately begins causing performance issues.

Functions whose TTL is set to 0 will try to retrieve the cached value from storage, even when the value is never stored in the first place. We should bypass storage completely, to avoid the performance hit of the additional lookup.

The global cache TTL takes precedence over the more specific function TTL when the function TTL is 0. This happens in https://github.com/mcollina/async-cache-dedupe/blob/afdf82b7a6e7a2e51e596ac6f532f78f8c390381/src/cache.js#L103, since opts.ttl is falsy.

Please see the following example, where caching should be disabled for fetchSomething. As can be seen from the output, the function is called only once.

$ cat example.mjs
import { createCache } from 'async-cache-dedupe'

const cache = createCache({
  ttl: 5,
  storage: { type: 'memory' },
})

cache.define('fetchSomething', { ttl: 0 }, async (k) => {
  console.log('query', k)
  return { k }
})

await cache.fetchSomething(1)
await cache.fetchSomething(1)
await cache.fetchSomething(1)

$ node example.mjs 
query 1

I've tested against version 1.2.2 of async-cache-dedupe.

$ cat package.json 
{
  "name": "zero-ttl",
  "version": "1.0.0",
  "dependencies": {
    "async-cache-dedupe": "1.2.2"
  }
}

Bumps tap from 15.2.3 to 16.1.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps tap from 15.2.3 to 16.0.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

For some use cases where we don't necessarily need the latest data as soon as ttl is passed, it can be useful to be able to specify a separate interval for staleness so we can continue serving data that's considered stale while we trigger a refetch in the background.

This feature is similar in principle to #14, and can possibly share some implementation details.

Proposal:

Accept a new staleWhileRevalidate: number argument (or maybe staleWhileRefetch? since we're not necessarily going to be doing any revalidation here), specifying the staleness interval.

Between the ttl and the staleness interval, data is considered stale, and any requests for it will resolve immediately with stale data while triggering a deduped refetch of the data in the background. After the interval, stale data must not be used and requests for the data must await on the refetch of the new data before being served.

Defaulting to 0 should preserve existing behavior where no stale data is ever served.

Thoughts?

I have a bunch of higher priority customer-facing stuff I have to work on over the next few weeks before I'm going to be able to work on an optimization making use of this, but I'd be happy to take a crack at a PR if nobody has tackled it by then!

when an error occur, we can add the option "staleOnError" to serve the latest cached response, for example

const cache = createCache({ ttl: 60 })

cache.define('fetchUser', {
  staleOnError: 10
}, 
(id) => database.find({ table: 'users', where: { id }}))

note the error is caused by the defined function, so for example here the database may not respond

for the first version, I'd go with a simple time (in seconds), then we could add a function to add some logic later, I'm not sure at the moment

we must also renew the cache ttl for staling entries

this logic should go here https://github.com/mcollina/async-cache-dedupe/blob/main/src/cache.js#L247