💾 Data persistence plugin for Lyra

Overview

Data Persistence Plugin

Tests

This plugin aims to provide data persistence capabilities to Lyra.

Runtime Status
Node.js  available
Bun  available
V8 isolates  available
Major browsers  available
Deno 🚧  under construction

Usage

For the complete usage guide, please refer to the official plugin documentation.

License

Apache-2.0

Comments
  • TypeError: Cannot use 'in' operator to search for 'name' in undefined

    TypeError: Cannot use 'in' operator to search for 'name' in undefined

    Describe the bug With the new release of Lyra is not possible to restore an instance from a file.

    /home/mateonunez/source/lyra/just/node_modules/.pnpm/@[email protected]/node_modules/@lyrasearch/lyra/dist/cjs/src/lyra.js:403
            if (!(index in lyra.tokenOccurrencies))
                        ^
    
    TypeError: Cannot use 'in' operator to search for 'name' in undefined
        at search (/home/mateonunez/source/lyra/just/node_modules/.pnpm/@[email protected]/node_modules/@lyrasearch/lyra/dist/cjs/src/lyra.js:403:21)
        at Object.<anonymous> (/home/mateonunez/source/lyra/just/just.js:12:18)
        at Module._compile (node:internal/modules/cjs/loader:1159:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
        at Module.load (node:internal/modules/cjs/loader:1037:32)
        at Module._load (node:internal/modules/cjs/loader:878:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
        at node:internal/main/run_main_module:23:47
    
    Node.js v18.12.1
    

    To Reproduce Steps to reproduce the behavior:

    1. Create a new project, add lyra and plugin-data-persistence dependencies.
    2. Create a new instance of Lyra and save it to a file.
    3. Restore that file.
    4. Make a search.

    or just copy this code:

    const { create, insert, search } = require('@lyrasearch/lyra')
    const { persistToFile, restoreFromFile } = require('@lyrasearch/plugin-data-persistence')
    
    const db = create({ schema: { name: 'string', age: 'string' }})
    insert(db, { name: 'john doe', age: '27' })
    insert(db, { name: 'jenny doe', age: '27' })
    
    const filePath = persistToFile(db, 'json', './db.json');
    const results = search(db, { term: 'joh' });
    const db2 = restoreFromFile('json', filePath);
    const results2 = search(db2, { term: 'jen' });
    
    console.log(results, filePath, results2)
    

    Expected behavior The plugin should restore a Lyra instance from a file.

    Desktop:

    • Node v18.12.1
    • Lyra v0.3.1
    • plugin-data-persistence v0.0.3
    opened by mateonunez 2
  • Update documentation link on README

    Update documentation link on README

    This PR updates the official documentation link on the README.

    As reported on #5 is necessary to update the website voice into the repository details by removing the docs prefix.

    opened by mateonunez 1
  • Restore defaultLanguage from file

    Restore defaultLanguage from file

    DefaultLanguage is not properly restored. As such, when you set the indexer to Arabic and save to disk, searches won't work on restore. Unless you go out of your way and manually change the defaultLanguage prop of the restoredInstance to Arabic. This change is dependent on having the lyra save also change (I will open another pull request for that):

    
    export function save<S extends PropertiesSchema>(lyra: Lyra<S>): Data<S> {
      return {
        index: lyra.index,
        defaultLanguage: lyra.defaultLanguage, <--- this needs to be added.
        docs: lyra.docs,
        nodes: lyra.nodes,
        schema: lyra.schema,
        frequencies: lyra.frequencies,
        tokenOccurrencies: lyra.tokenOccurrencies,
      };
    }
    
    opened by YusufCelik 0
  • Should throw an error when restoring an unsupported format

    Should throw an error when restoring an unsupported format

    This PR adds a new test, bringing the coverage to 98.38% (current is 97.58%).

    The test checks that the exception raised when trying to retrieve a database using an unsupported format is correct.

    opened by mateonunez 0
  • Module not found: Can't resolve 'fs'

    Module not found: Can't resolve 'fs'

    I wanna use this memory persistence feature for my app. But, can't import this plugin into Next.js, always getting below error

    ./node_modules/@lyrasearch/plugin-data-persistence/dist/node/esm/node-bun/index.js:1:0
    Module not found: Can't resolve 'fs'
    
    opened by raitucarp 0
  • RangeError: Extra byte(s) found at buffer

    RangeError: Extra byte(s) found at buffer

    Hi, I'm not sure that this issue depends on this plugin, but I want to know if there is a way to avoid it so we can track similar issues.

    This problem is generated when I've tried to restore a previously saved Lyra instance.

    This is the code:

    import { create, insert } from '@lyrasearch/lyra'
    import { restoreFromFile, persistToFile } from '@lyrasearch/plugin-data-persistence'
    
    const db = create({
      schema: {
        author: 'string',
        quote: 'string'
      }
    })
    
    insert(db, {
      author: 'Poseidon',
      quote: 'Sea'
    })
    
    persistToFile(db, 'binary', './test.msp')
    
    const db2 = restoreFromFile('binary', './test.msp')
    

    The error is present only with the binary format which uses msgpack. There's another way to avoid it keep using binary formats?

    opened by mateonunez 1
  • Update docs link in repository details

    Update docs link in repository details

    opened by mateonunez 0
  • Automatic persistence

    Automatic persistence

    Previously posted in the lyra repo:


    From an API perspective, if I was creating a persistence package, I would want the ability to persist on each change to the database. It might be nice if a lyra DB was an event emitter for that reason. I could imagine a disk persistence package working like:

    const persist = (db, format) => {
      saveToDisk(db, format);
    
      db.addEventListener('change', () => {
        saveToDisk(db, format);
      });
    };
    

    That way a user would only need to call persist once and not manage calling it every time they inserted, removed, etc.

    opened by matthewp 2
  • Make plugin available on Deno, Bun, and browsers

    Make plugin available on Deno, Bun, and browsers

    As for now, the plugin runs on Node.js, but it will need to run on any browser/runtime. We should probably create different builds for different runtimes and environments to support a broader range of possibilities.

    Things to know:

    • dpack is a strategic dependency and it's supported in browsers, Node.js and Bun. We need to bundle it in some way to make the support available in Deno
    enhancement good first issue help wanted 
    opened by micheleriva 5
Releases(0.0.4)
Owner
Lyra
The edge-ready full-text search engine
Lyra
Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

JavaScript-med-Ramverk-Laboration-3 Grupprojektet för kurserna Javascript med Ramverk och Agil Utveckling. Utvecklingsguide För information om hur utv

Svante Jonsson IT-Högskolan 3 May 18, 2022
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
🚀 A (still experimental) Lyra integration for Astro

Lyra's Astro Plugin This package is a (still experimental) Lyra integration for Astro. Usage Configuring the Astro integration // In `astro.config.mjs

Lyra 23 Dec 13, 2022
Create a Lyra database from anywhere.

?? ☄️ ️ Impact Create a Lyra database from anywhere. Installation You can install Lyra using npm, yarn, pnpm: npm i @mateonunez/lyra-impact yarn add @

Mateo Nunez 10 Dec 19, 2022
JSON Visio is data visualization tool for your json data which seamlessly illustrates your data on graphs without having to restructure anything, paste directly or import file.

JSON Visio is data visualization tool for your json data which seamlessly illustrates your data on graphs without having to restructure anything, paste directly or import file.

Aykut Saraç 20.6k Jan 4, 2023
It shows an effective way to correct bus arrival information using data analytics based on Amazon Serverless such as Kiness Data Stream, Kinesis Data Firehose, S3, and Lambda.

Amazon Serverless를 이용한 실시간 버스 정보 수집 및 저장 본 github repository는 버스 정보를 주기적으로 수집하여 분석할 수 있도록, Amazon Serverless인 Amazon Kinesis Data Stream, Kinesis Data

John Park 4 Nov 13, 2022
A table component for your Mantine data-rich applications, supporting asynchronous data loading, column sorting, custom cell data rendering, row context menus, dark theme, and more.

Mantine DataTable A "dark-theme aware" table component for your Mantine UI data-rich applications, featuring asynchronous data loading support, pagina

Ionut-Cristian Florescu 331 Jan 4, 2023
Tool to sign data with a Cardano-Secret-Key // verify data with a Cardano-Public-Key // generate CIP-8 & CIP-36 data

Tool to sign data with a Cardano-Secret-Key // verify data with a Cardano-Public-Key // generate CIP-8 & CIP-36 data

Martin Lang 11 Dec 21, 2022
LunaSec - Open Source Security Software built by Security Engineers. Scan your dependencies for Log4Shell, or add Data Tokenization to prevent data leaks. Try our live Tokenizer demo: https://app.lunasec.dev

Our Software We're a team of Security Engineers on a mission to make awesome Open Source Application Security tooling. It all lives in this repo. Here

LunaSec 1.2k Jan 7, 2023
Pull sensitive data from users on windows including discord tokens and chrome data.

⭐ For a ?? Pegasus Pull sensitive data from users on windows including discord tokens and chrome data. Features ?? Discord tokens ?? Geolocation data

Addi 43 Dec 24, 2022
The Covid-19 data-app collects information from API and present data for the novel coronavirus.

COVID-19-data-app This project was bootstrapped with Create React App. The Covid-19 data-app collects information from API and present data for the no

Sentayhu berhanu 8 Jun 23, 2022
This project is built with JavaScript, Webpack, HTML & CSS, Leaderboard api. When user clicks on Refresh button it hits the api and responds with the data, The user can also post data to the api

leaderboad Description the project. this project is about the leaderboad i did during Microverse to build a website for adding Data to the API and fet

Emmanuel Moombe 4 May 30, 2022
A mobile web application to check the data on the total covid19 confirmed cases and deaths, check data for all countries with recorded cases.

This is a mobile web application to check the data on the total covid19 confirmed cases and deaths, check data for all countries with recorded cases. It also has a details page to check for the statistics for each region/state if available.

Solomon Hagan 7 Jul 30, 2022
🧙 Mage is an open-source data management platform that helps you clean data and prepare it for training AI/ML models.

Intro Mage is an open-source data management platform that helps you clean data and prepare it for training AI/ML models. What does this do? The curre

Mage 2.5k Jan 4, 2023
Python based web application to import, connect and analyze manufacturing data from multiple data sources.

Analysis Platform Analysis Platform is an open source web application to import, connect and visualize factory IoT data. It helps to collect, link and

Analysis Platform +DN7 7 Dec 1, 2022
✏️ A small jQuery extension to turn a static HTML table into an editable one. For quickly populating a small table with JSON data, letting the user modify it with validation, and then getting JSON data back out.

jquery-editable-table A small jQuery extension to turn an HTML table editable for fast data entry and validation Demo ?? https://jsfiddle.net/torrobin

Tor 7 Jul 31, 2022
Random Fractals stash of Observable Data Tools 🛠️ and Notebooks 📚 in ES Modules .js, .nb.json, .ojs, .omd, .html and .qmd document formats for Data Previews

Random Fractals stash of Observable Data Tools ??️ and Notebooks ?? in ES Modules .js, .nb.json, .ojs, .omd, .html and .qmd document formats for Data Previews in a browser and in VSCode IDE with Observable JS extension, Quarto extension, and new Quarto publishing tools.

Taras Novak 14 Nov 25, 2022