Next-level mongoose caching layer with event based cache clearing

Overview

SpeedGoose

## About The Project

This project is a next-level mongoose caching library which is fully written in typescript. It's caching on two levels. Shared - with redis. And local inside memory. Supports all mongoose operations like find,findOne,count,aggregate... and others. Also supports lean queries. Why it is different?

  • It supports caching not only JSON objects in redis, but also whole Mongoose.Document instances in local memory to speed up code, and prevent unnecessary hydrations
  • It has auto-clearing ability based on mongoose events. So if the query was cached for some records, and in the meantime that records changes, all cached related results will be cleared.
  • Supports custom eventing. For example you wan't to remove given results from cache, but removal logic is not based on removing documents from db but rather field based (like deleted: true), then you can apply a wasRecordDeleted callback as an option for the plugin

(back to top)

Getting Started

This is an example of how you may give instructions on setting up your project locally. To get a local copy up and running follow these simple example steps.

Installation

npm install speedgoose
or 
yarn add speedgoose
  1. Simple wrap your mongoose with the library (required)
import {applySpeedGooseCacheLayer} from "speedgoose";
import mongoose from "mongoose";

applySpeedGooseCacheLayer(mongoose, {
  redisUri: process.env.REDIS_URI,
  redisIndex : process.env.REDIS_INDEX_DB
})
  1. To enable auto-clearing for given schema, just add plugin to it (required)
import {SpeedGooseCacheAutoCleaner} from "speedgoose";

Schema.plugin(SpeedGooseCacheAutoCleaner)
// additionaly you can pass options for example callback for setting record as deleted 
Schema.plugin(SpeedGooseCacheAutoCleaner, {wasRecordDeletedCallback} )

Usage

  1. With find, count etc...
model.find({}).cacheQuery()
model.find({}).sort({fieldA : 1}).cacheQuery()
model.find({}).lean().cacheQuery()
  1. With aggregation
model.aggregate([]).cachePipeline()

(back to top)

Roadmap

  • Add more examples
  • Deep hydration for nested documents
  • Cache-based population
  • Manual cache clearing for custom keys
  • Flowchart of logic
  • Tests
  • Multitenant (tenant field indicator) support
  • [ ]

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Contact

(back to top)

You might also like...

Promisified thunk with caching (pronounced "funky")

phunky phunky (promisified thunk, pronounced funky) is yet another thunk-inspired library, to delay a calculation until its result is needed, but also

Oct 14, 2022

A simple in-memory time-based cache for both objects and function execution.

What is this? A simple in-memory time-based cache for both objects and function execution. How do I install it? You can install it by using the follow

Dec 15, 2022

School App / Back-End with MongoDB / mongoose / Express / TS

TEST Api Dependencies El mati se la come es por eso que en 1998 la guerra fria se llevo a mas de la mitad del activo del pais "dependencies": { "axios

Jun 10, 2022

🚀 A mongoose plugin to monetize your apis in few lines of code

🚀 A mongoose plugin to monetize your apis in few lines of code

Stripe Mongoose Api Stripe Mongoose Api is a Mongoose plugin that simplifies building checkout and payment system for apis with stripe. It will provid

Dec 29, 2022

Breve explicacion de Mongoose, asi como un codigo con las operaciones CRUD

Breve explicacion de Mongoose, asi como un codigo con las operaciones CRUD

Introduccion a Mongoose ¿Qué es MongoDB? MongoDB es una base de datos NoSQL (Not Only SQL) y por ende no relacional que es utilizado para proyectos we

Mar 17, 2022

Uma Api dos 151 primeiros pokemons utilizando NodeJS, Typescript, Docker, MongoDB e Mongoose.

Uma Api dos 151 primeiros pokemons utilizando NodeJS, Typescript, Docker, MongoDB e Mongoose.

Pokedex API Tópicos Preview Sobre o Pokedex Api Tecnologias Instalação e uso Preview Clique aqui para ter acesso a um preview da Api. Sobre o Pokedex

Nov 27, 2022

Set up and build a Node.js REST API using Typescript, Express, Mongoose with a maintainable and scalable structure.

Introduction Create a maintainable and scalable Node.js REST API with TypeScript, Express and Mongoose. The project structure is based on MVC and foll

Nov 18, 2022

Create a maintainable and scalable Node.js GraphQL API with TypeScript, Express, Mongoose and Apollo Server.

Set up and build a Node.js GraphQL API using Typescript, Express, Mongoose with a maintainable and scalable structure

Nov 4, 2022

Mongoose Node.js Express TypeScript application boilerplate with best practices for API development.

Node TypeScript Boilerplate Mongoose The main purpose of this repository is to show a good end-to-end project setup and workflow for writing a Mongoos

Dec 13, 2022
Comments
  • No key found

    No key found

    Hey, I always get the following debug information:

      speedgoose:Visitor:cacheQuery  Reading cache for key  {"collection":"visitors","op":"findOne","options":{}} +0ms
      speedgoose:Visitor:cacheQuery  Key didn't exists in cache, retreaving value from database  {"collection":"visitors","op":"findOne","options":{}} +1ms
      speedgoose:Visitor:cacheQuery  Setting key in cache  {"collection":"visitors","op":"findOne","options":{}} +194ms
      speedgoose:Visitor:cacheQuery  Cache key set  {"collection":"visitors","op":"findOne","options":{}} +0ms
    

    In redis, I can't find any key - it seems like that the library doesn't add any key to the database.

    opened by dnish 11
  • Question about in-memory caching without the use of Redis

    Question about in-memory caching without the use of Redis

    Hi, I first wanted to say thank you for creating an alternative to https://github.com/boblauer/cachegoose that is actively maintained and has TS support.

    I've got a question about the applySpeedGooseCacheLayer function. The README mentions:

    It's caching on two levels. Shared - with redis. And local inside memory.

    I tried calling the applySpeedGooseCacheLayer function without providing a Redis URI, similarly to how the cachegoose function works:

    import mongoose from "mongoose";
    import { applySpeedGooseCacheLayer } from 'speedgoose';
    
    // . . .
    
    await applySpeedGooseCacheLayer(mongoose, {});
    

    but I get an error stating that the redisUri property is required (https://github.com/arqo123/speedgoose/blob/master/src/types/types.ts#L29): Screen Shot 2022-08-30 at 5 24 10 PM

    I'm a bit confused by this because I thought that this package and its behavior was based off cachegoose (but this is also just an assumption I'm making). In cachegoose, if you don't include a Redis URL, it defaults to in-memory caching: https://github.com/boblauer/cachegoose#usage. Screen Shot 2022-08-30 at 5 28 57 PM

    My questions are:

    • Is in-memory caching without the use of Redis supported by this package?
    • If not, will this be part of the future roadmap?

    Thank you!

    opened by victoria-miltcheva 4
  • About some update and delete operation in the code.

    About some update and delete operation in the code.

    Hello, I have a question to ask. I don't see the function of modifying or deleting the affected cache when updating records in the document and source code, For example, when I update a document, theoretically, I should clear the cache records of the document. There is no schema in the source code The function of post ('update ', xxx). In addition, if I use deleteMany or deleteOne to delete documents, the source code only executes schema Post ('remove ', xxx), but no schema Post ('deleteOne ', xxx), etc

    opened by YanxinNet 3
  • SpeedGooseCacheAutoCleaner causing multitenantKey error

    SpeedGooseCacheAutoCleaner causing multitenantKey error

    I'm getting an error when trying to do a findByIdAndUpdate on a schema I'd previously forgotten to add the SpeedGooseCacheAutoCleaner plugin to, and now when I try to include it, e..g:

    UserSchema.plugin(SpeedGooseCacheAutoCleaner);
    

    in my code when I do a findByIdAndUpdate, I get this error (locally) - I'd never set up anything relating to multitennancy on this project before, so I don't know why it's now complaining.

    TypeError: Cannot read properties of undefined (reading 'multitenantKey')
        at getRecordAffectedByAction ([root of project]/node_modules/speedgoose/lib/src/plugin/utils.js:22:77)
    

    The code I'm using to update:

        User.findByIdAndUpdate(
          { _id: req.user._id },
          {
            $set: {
              token: req.user.token,
              "section.override": valueFromUser,
            },
          },
          function (err) {
            if (err) {
              console.log(err);
              errors = [err];
              has_saved = false;
            }
          }
        );
    
    bug 
    opened by adjstreams 2
Owner
Arkadiusz Gil
Arkadiusz Gil
🪐 The IPFS gateway for NFT.Storage is not "another gateway", but a caching layer for NFTs that sits on top of existing IPFS public gateways.

nftstorage.link The IPFS gateway for nft.storage is not "another gateway", but a caching layer for NFT’s that sits on top of existing IPFS public gate

NFT.Storage 37 Dec 19, 2022
ZxCDDoS for education with LAYER 7, LAYER 4, AMP METHODS

?? ZxCDDoS: Release v1.0 - Free DDoS Panel ?? Terminal only accepts ANSI color. Username: admin Password: admin Language Logs Fixed L7 methods (crash,

zxcr9999 151 Jan 3, 2023
io-ts Typed Event Bus for the runtime of your Node.js application. A core for any event-driven architecture based app.

Typed Event Bus Based on io-ts types, this bus provides a handy interface to publish and consume events in the current runtime of the Node.js process.

Konstantin Knyazev 3 May 23, 2022
'event-driven' library aims to simplify building backends in an event driven style

'event-driven' library aims to simplify building backends in an event driven style(event driven architecture). For message broker, light weight Redis Stream is used and for event store, the well known NoSQL database, MongoDB, is used.

Sihoon Kim 11 Jan 4, 2023
🚀 Final Esports project based on Rocketseat's Next Level Week.

The evolved version of the Esports project on the Next Level Week by Rocketseat. The web client is deployed on Vercel here: https://nlw-esports-ignite

João Martinho 35 Oct 26, 2022
🍰 An extensible, layer based shader material for ThreeJS

lamina ?? An extensible, layer based shader material for ThreeJS These demos are real, you can click them! They contain the full code, too. ?? More ex

Poimandres 811 Jan 6, 2023
Allo Caching for Deno

Simple caching solution in Typescript.

Adam Josefus 5 Feb 14, 2022
A fast, safe and easy caching mechanism similar to Redis written in typescript

Viper Viper is a memory based caching mechanism which is aimed towards ease of use and speed. It's in a very early stage right now and not meant to us

Japroz Saini 1 Jan 24, 2022
A caching middeware library for Deno HTTP framework, Oak.

Caching middleware library for Oak Table of Contents Description Getting Started Middleware and caching Authors License Description Zoic is an easy-to

OSLabs Beta 64 Sep 28, 2022
Skip a job if it already succeeded for the same repo state. Uses S3 for caching.

?? ♻️ S3 Cache Action First Run, Cold Cache Next Run, Cache Hit Description Allows to skip a job if it already succeeded for the same repo state. Uses

Pleo Open Source 8 Jul 22, 2022