🗃️ An in-memory JS database optimized for large scale storage on the frontend.

Overview

BlinkDB Logo

BlinkDB is a in-memory JS database optimized for large scale storage on the frontend.


import { many } from 'blinkdb';

const items = await many(userTable, {
  where: {
    name: { $in: ['Alice', 'Charlie'] },
    age: { $gt: 24 }
  },
});

BlinkDB is not yet mature. If you aren't convinced with the current feature set, check back in version 1.0.0.

Motivation

Today, SPAs and modern UI frameworks are more complex than ever before, and in the biggest codebases, gigantic amounts of data are used in JS directly. Unfortunately, while user-friendly, the existing solutions - stores like Redux or MobX - are not optimized for performance with large quantities of entities.

Description

BlinkDB provides an alternative - a strongly optimized in-memory database for your frontend. With database features such as indexes, query optimization and support for both relational & non-relational data, BlinkDB allows you to query your data with filters, sort and paginate in one go - just like a real database, and just as performant (as far as a database can be in Javascript).

Getting started

You might also like...

javascript embeddable / in-memory database

LokiJS The super fast in-memory javascript document oriented database. Enable offline-syncing to your SQL/NoSQL database servers with SyncProxy !! Cod

Jan 9, 2023

ClientDB is an open source in-memory database for enabling real-time web apps.

ClientDB ClientDB is an open source in-memory database for enabling real-time web apps. Build fast, scalable apps that feel silky smooth for users. In

Dec 24, 2022

ClientDB is an open source in-memory database for enabling real-time web apps.

ClientDB ClientDB is an open-source in-memory database for enabling real-time web apps. Build fast, scalable apps that feel silky smooth for users. In

Aug 27, 2022

aka Scaletor, take screenshots of a piece of a map and scale/compare with other parts of the map

scale-a-tron A quick-and-dirty map that lets you compare one area to another. Draw a shape around a region, zoom in to another place on the map, and c

Nov 7, 2022

🍎 A simple application which enables you to sync readings from your Withings scale to a Notion page.

🍎 A simple application which enables you to sync readings from your Withings scale to a Notion page.

weight-logger weight-logger is a simple application which enables you to sync readings from your Withings scale to a Notion page. Preview Installation

Jan 14, 2022

College project done for the CG Artwork lecture in 2022. Used HTML for the indexes and mainly JavaScript (using to THREE.js). Ended with the final A grade (17.3 in scale grade).

CG Artwork Project 2022 This project was done by a group of 3 in 2022 with an educational purpose for the CG Artwork lecture in Instituto Superior Téc

Sep 19, 2022

Dump of a 5 day small scale data scraping project on the io game "arras.io"

Arras DB A bot built to scan all arras.io game servers (besides sandbox) in search of bulk server and player data. This bot was run 5 days straight, a

Jul 19, 2022

multi-channel fulfillment at scale

Openship is an operations platform that enables multi-channel fulfillment Website · Documentation · Openship Cloud Running locally To get Openship run

Dec 31, 2022

Web Scale™ Anarchy SponsorBlock Server

Web Scale SponsorBlock Server Joke Fork of ajayyy/SponsorBlockServer Now backed by Web Scale MangoDB (thanks @grub for the native nc port) Now with no

Sep 28, 2022
Comments
  • Avoid type assertions in update.ts

    Avoid type assertions in update.ts

    Hi.

    I've looked at your source code and noticed that you're heavily using type assertions, as well as such code smells as "as any".

    I want to use this pull request as an encouragement and example to show that type hacks are unnecessary in your codebase and you can do better. If you want to call your library "type-safe" please make sure your internal code is type-safe first and foremost.

    Unfortunately I don't have enough time to fully refactor all of your code, so only took liberty of doing it in a couple of files as a proof of concept.

    Cheers.

    opened by grumd 3
  • Limit performance increase + Bugfix for `many`

    Limit performance increase + Bugfix for `many`

    depends on #3 for additional benchmarking (therefore this PR contains the other changes as well until it's merged).

    I have changed limitItems to need way less array operations when limiting/paginating a large item data set.

    I also fixed a bug with many which didn't return any items if filter options are defined but don't contain a where (e.g. when paginating the whole data set).

    many query with limit:

    Before

      blinkdb   took 0.41430ms, high 16.08050ms, low 0.40960ms
    

    After

      blinkdb   took 0.19070ms, high 2.21160ms, low 0.17670ms
    
    opened by Valerionn 3
  • Improve write performance for operations on tables with indexes + Bugfix for `remove`

    Improve write performance for operations on tables with indexes + Bugfix for `remove`

    I have changed some create/remove operations to now edit the array in-memory instead of allocating a new array which should be a lot faster:

    grafik

    This should not change the behavior in any observable way (unless an index array contains the same item multiple times, which should not happen as far as I know) but provides a significant performance boost especially when inserting entries.

    I added some benchmarks below, using a table with 1 index. However, the performance increase is even bigger when using multiple indexes (e.g. creating 10.000 items with 5 indexes takes 40ms now instead of 78ms, using clone in both cases).

    Additionally, I fixed a small bug in the remove implementation that I stumbled upon.

    Create 10.000 items:

    Before

      blinkdb with index      took 24.31350ms, high 95.36340ms, low 22.92340ms
      blinkdb with no index   took 12.05970ms, high 106.57190ms, low 9.89640ms
    

    After

      blinkdb with index      took 18.07080ms, high 74.83170ms, low 15.48300ms
      blinkdb with no index   took 11.05330ms, high 98.95320ms, low 10.08770ms
    

    Update one item:

    Before

      blinkdb with index      took 0.00510ms, high 1.11310ms, low 0.00200ms
      blinkdb with no index   took 0.00250ms, high 0.37020ms, low 0.00200ms
    

    After

      blinkdb with index      took 0.00300ms, high 0.67560ms, low 0.00200ms
      blinkdb with no index   took 0.00220ms, high 0.99160ms, low 0.00190ms
    

    Remove one item:

    Before

    No before available due to a bug in the current implementation which leads to faster execution time (but a wrong result)

    After

      blinkdb with index      took 0.00580ms, high 2.15120ms, low 0.00270ms
      blinkdb with no index   took 0.00290ms, high 1.33420ms, low 0.00230ms
    
    opened by Valerionn 3
  • How to count the entries effectively?

    How to count the entries effectively?

    I can't find any example how to get the total rows in collections effectively.

    Currently, I am using

    const total = await many(users).then(x => x.length)
    

    But obviously it is not a good approach.

    enhancement 
    opened by shtse8 2
Releases(v0.9.0)
  • v0.9.0(Nov 26, 2022)

    What's Changed

    • BlinkDB introduces bulk methods! You can now use insertMany(), updateMany(), and removeMany() to create, update & delete multiple items at once. See the docs for more details.

    • These bulk methods are now used in the benchmarks against lokiJS, which shows the massive speed increase:

      BEFORE ( with insert() ):
        comparison for creating 10.000 items:
          lokijs    took 8.61030ms, high 19.76770ms, low 7.18500ms
          blinkdb   took 14.00180ms, high 142.01530ms, low 12.76930ms
      
      AFTER ( with insertMany() ):
        comparison for creating 10.000 items:
          lokijs    took 7.46930ms, high 28.76890ms, low 7.20460ms
          blinkdb   took 3.43980ms, high 25.70130ms, low 3.20480ms
      
    • New updateWhere()/removeWhere() functions that update/remove all items that match a given filter.

    • New count() method that returns the total number of items in a table (or matching a given filter, if a filter is supplied).

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Nov 7, 2022)

    What's Changed

    • The query types for BlinkDB have been reworked, and now produce correct type output if the interface of the entity passed to createTable() contains complex union types.
    • Indexes can now be built on properties marked as undefined/null.

    Breaking changes

    • The $ prefix for filters has been removed, and object matching is done with a new where subfilter.

    The prefix was used for distinction in object matching, where

    const users = await many(userTable, {
      age: { lt: 30 }
    });
    

    could mean both "retrieve all items with an age < 30" or "retrieve all items where age is an object matching { lt: 30 }".

    This confusion has been fixed by requiring usage of the eq filter or a new where subfilter for object matching. The two queries below produce the same result, namely all items where nested === { prop: "a" }:

    const users = await many(userTable, {
      nested: {
        eq: { 
          prop: "a" 
         }
      }
    });
    
    const users = await many(userTable, {
      nested: {
        where: {
          prop: "a" // usage of { eq: a } or other filters is allowed here
        }
      }
    });
    
    • The equals filter has been renamed to eq.

    Full Changelog: https://github.com/blinkdb-js/blinkdb/commits/v0.8.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Oct 13, 2022)

    What's Changed

    BlinkDB now uses a more refined query engine which will make it easier to implement sort & limit-based optimizations in the future. Additionally, $and now performs better by choosing the most performant sub query for execution.

    Full Changelog: https://github.com/blinkdb-js/blinkdb/commits/v0.7.0

    Source code(tar.gz)
    Source code(zip)
  • v0.6.4(Sep 17, 2022)

    What's Changed

    • Updated types of database methods
    • Fixed bug with limit in watch
    • Removed ability to use properties of types null / undefined as indexes or primary keys
    • Added a section regarding observing changes to the docs
    • Added a table with all options/parameter for each function in the reference

    Full Changelog: https://github.com/blinkdb-js/blinkdb/commits/v0.6.4

    Source code(tar.gz)
    Source code(zip)
  • v0.6.3(Sep 13, 2022)

    What's Changed

    • updated domain blinkdb.io by @k69420s in https://github.com/blinkdb-js/blinkdb/pull/1
    • Fix some minor documentation issues by @Valerionn in https://github.com/blinkdb-js/blinkdb/pull/2
    • Improve write performance for operations on tables with indexes + Bugfix for remove by @Valerionn in https://github.com/blinkdb-js/blinkdb/pull/3
    • Limit performance increase + Bugfix for many by @Valerionn in https://github.com/blinkdb-js/blinkdb/pull/4
    • Fix many not returning a clone of the elements if no filter is set by @Valerionn in https://github.com/blinkdb-js/blinkdb/pull/5

    New Contributors

    • @k69420s made their first contribution in https://github.com/blinkdb-js/blinkdb/pull/1
    • @Valerionn made their first contribution in https://github.com/blinkdb-js/blinkdb/pull/2

    Full Changelog: https://github.com/blinkdb-js/blinkdb/commits/v0.6.3

    Source code(tar.gz)
    Source code(zip)
Owner
BlinkDB
In-memory JS database optimized for large scale storage on the frontend.
BlinkDB
A large scale simulation which pits millions of space ships against each other in a virtual universe all running directly in SingleStore.

Wasm Space Program In this demo we simulate a fake universe full of thousands of solar systems. In each solar system there are many space ships and en

SingleStore Labs 11 Nov 2, 2022
T3 is a client-side JavaScript framework for building large-scale web applications

Box has migrated using react, webpack, and the latest version of ECMAScript for our frontend projects as of 2018. We no longer support chan

Box 1.6k Dec 8, 2022
Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

JavaScript-med-Ramverk-Laboration-3 Grupprojektet för kurserna Javascript med Ramverk och Agil Utveckling. Utvecklingsguide För information om hur utv

Svante Jonsson IT-Högskolan 3 May 18, 2022
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
Interactive To Do List, draggable tasks, optimized for desktop and mobile screen sizes. It reserves data in local storage. Built with HTML, CSS, and JavaScript.

To Do List Interactive daily To-Do List Built With Major languages: html, css, javascript Frameworks: Technologies used: Webpack Live Demo Live Demo L

Jafar Yousef 6 Aug 4, 2022
Boost is a Microsoft Excel Add-in to help developers import large excel workbooks into their database using SQL queries

Microsoft Excel Add-in for Developers About Us Boost is a Microsoft Excel Add-in to help developers import large excel workbooks into their database u

OSLabs Beta 30 Sep 30, 2022
startupDB is an Express middleware function implementing a high-performance in-memory database

startupDB startupDB is a database designed to create REST APIs. It is implemented as an Express middleware function and allows for easy implementation

Jeroen de Vries 8 Jul 26, 2022
An exercise in building a very minimal (and very stupid) in-memory SQL-like database for educational purposes.

Stupid Database This is an exercise in building a very minimal (and very stupid) in-memory SQL-like database for educational purposes. None of this co

Fabio Akita 196 Dec 20, 2022
In-memory abstract-level database for Node.js and browsers.

memory-level In-memory abstract-level database for Node.js and browsers, backed by a fully persistent red-black tree. The successor to memdown and lev

Level 14 Dec 27, 2022