A Fetch API-compatible PlanetScale database driver

Overview

PlanetScale Serverless Driver for JavaScript

A Fetch API-compatible PlanetScale database driver for serverless and edge compute platforms that require HTTP external connections, such as Cloudflare Workers or Vercel Edge Functions

Installation

npm install @planetscale/database

Usage

import { connect } from '@planetscale/database'

const config = {
  host: '<host>',
  username: '<user>',
  password: '<password>'
}

const conn = connect(config)
const results = await conn.execute('select 1 from dual where 1=?', [1])
console.log(results)

Database URL

A single database URL value can be used to configure the host, username, and password values.

import { connect } from '@planetscale/database'

const config = {
  url: process.env['DATABASE_URL'] || 'mysql://user:pass@host'
}

const conn = connect(config)

Connection factory

Use the Client connection factory class to create fresh connections for each transaction or web request handler.

import { Client } from '@planetscale/database'

const client = new Client({
  host: '<host>',
  username: '<user>',
  password: '<password>'
})

const conn = client.connection()
const results = await conn.execute('select 1 from dual')
console.log(results)

Transactions

Use the transaction function to safely perform database transactions. If any unhandled errors are thrown during execution of the transaction, the transaction will be rolled back.

The following example is based on the Slotted Counter Pattern.

import { connect } from '@planetscale/database'

const config = {
  host: '<host>',
  username: '<user>',
  password: '<password>'
}

const conn = connect(config)
const results = await conn.transaction(async (tx) => {
  const whenBranch = await tx.execute('INSERT INTO branches (database_id, name) VALUES (?, ?)', [42, "planetscale"])
  const whenCounter = await tx.execute('INSERT INTO slotted_counters(record_type, record_id, slot, count) VALUES (?, ?, RAND() * 100, 1) ON DUPLICATE KEY UPDATE count = count + 1', ['branch_count', 42])
  return [whenBranch, whenCounter]
})
console.log(results)

Custom fetch function

Node.js version 18 includes a built-in global fetch function. When using an older version of Node.js, you can provide a custom fetch function implementation. We recommend the undici package on which Node's built-in fetch is based.

import { connect } from '@planetscale/database'
import { fetch } from 'undici'

const config = {
  fetch,
  host: '<host>',
  username: '<user>',
  password: '<password>'
}

const conn = connect(config)
const results = await conn.execute('select 1 from dual')
console.log(results)

Custom query parameter format function

Query replacement parameters identified with ? are replaced with escaped values. Named replacement parameters are supported with a colon prefix.

const results1 = await conn.execute('select 1 from dual where 1=?', [42])
const results2 = await conn.execute('select 1 from dual where 1=:id', { id: 42 })

Providing a custom format function overrides the built-in escaping with an external library, like sqlstring.

import { connect } from '@planetscale/database'
import SqlString from 'sqlstring'

const config = {
  format: SqlString.format,
  host: '<host>',
  username: '<user>',
  password: '<password>'
}

const conn = connect(config)
const results = await conn.execute('select 1 from dual where 1=?', [42])
console.log(results)

Custom type casting function

Column values are converted to their corresponding JavaScript data types. This can be customized by providing a cast function.

import { connect, cast } from '@planetscale/database'

function inflate(field, value) {
  if (field.type === 'INT64' || field.type === 'UINT64') {
    return BigInt(value)
  }
  return cast(field, value)
}

const config = {
  cast: inflate,
  host: '<host>',
  username: '<user>',
  password: '<password>'
}

const conn = connect(config)

Development

npm install
npm test

License

Distributed under the Apache 2.0 license. See LICENSE for details.

Comments
  • Question: Error when trying to use the module in jest

    Question: Error when trying to use the module in jest

    Expected Behavior

    "@planetscale/database" should be able to import in Jest tests.

    Current Behavior

    An error occurs and jest is not able to run.

     FAIL  ./test.test.ts
      ● Test suite failed to run
    
        Jest encountered an unexpected token
    
        Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.
    
        Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.
    
        By default "node_modules" folder is ignored by transformers.
    
        Here's what you can do:
         • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
         • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
         • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
         • If you need a custom transformation specify a "transform" option in your config.
         • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.
    
        You'll find more details and examples of these config options in the docs:
        https://jestjs.io/docs/configuration
        For information about custom transformations, see:
        https://jestjs.io/docs/code-transformation
    
        Details:
    
        /foo/node_modules/@planetscale/database/dist/index.js:1
        ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { format } from './sanitization.js';
                                                                                          ^^^^^^
    
        SyntaxError: Cannot use import statement outside a module
    

    What I've tried

    I'm not sure what the root cause is but it seems Jest cannot transform the ESM(?) file included in @planetscale/database/dist/index.js.

    I assumed this error is a well-known jest misconfiguration and tried some of these solutions:

    • support esm in ts-node
      • https://kulshekhar.github.io/ts-jest/docs/guides/esm-support/
    • tried the same settings as this repository(planetscale/database-js)

    The same error occurred each time.

    Steps to Reproduce

    I've created a minimum code base as follows: https://github.com/kenfdev/planetscale-jest-poc

    You can reproduce it with npm test.

    I might totally be missing something from the beginning but am pretty much stuck right now. Any advice would be highly appreciated! Thank you in advance!

    opened by kenfdev 6
  • feat: support `insecure` flag on Config

    feat: support `insecure` flag on Config

    • Allows passing 'insecure' bool. This has no production ramifications, if someone were to do this, we don't even listen on port 80. The application is for local dev, or in other unique environments. This is a common pattern to have secure-by-default, but opt into insecure with an explicit flag, see grpc-go APIs.
    • Unifies a bit of url building, and adopt a more service/method pattern

    If you can guide me on how to write a test for this, that'd be great. This is all foreign to me. :innocent:

    opened by mattrobenolt 3
  • Prisma-like JSON Query Constructor

    Prisma-like JSON Query Constructor

    Any plans to make a Prisma-like JSON query constructor? The idea is to make conditional queries easier, for example:

    import { NextApiRequest, NextApiResponse } from "next";
    import { pscale } from @planetscale/database
    
    export default async function handler(
      req: NextApiRequest,
      res: NextApiResponse
    ) {
        const { param1, param2, param3 } = req.query as {
          param1?: string;
          param2?: string;
          param3?: string;
        };
        const response = await pscale.table.findMany({
            where: {
                ...(param1 && {
                      attr1: param1
                 }),
                ...(param2 && {
                      attr2: param2
                 }),
                ...(param3 && {
                      attr3: param3
                 }),
            }
        })
        res.status(200).json(response);
    }
    
    opened by steven-tey 3
  • Inserting multiple rows at once

    Inserting multiple rows at once

    Hey guys, not sure if I'm missing something, but does this client not support inserting more than one row at a time? Having trouble finding any mention in planetscale docs, readme, and the code in this library

    connection.execute(
      `
    	  INSERT INTO test (id)
    	  VALUES (?)
      `,
      [[1], [2]]
    )
    

    I've also tried:

    VALUES (??)
    VALUES ??
    VALUES ?
    

    etc

    opened by atjeff 2
  • NULL types are mishandled

    NULL types are mishandled

    It seems, due to a quirk in how a protobuf is translated into JSON, the NULL type is left out from the field because it's integer value is 0: https://github.com/vitessio/vitess/blob/main/proto/query.proto#L111-L112

    What happens here is an HTTP response that looks like this for a query such as select null, 1.

    "result": {
        "fields": [
          {
            "name": "null"
          },
          {
            "name": ":vtg1",
            "type": "INT64"
          }
        ],
        "rows": [
          {
            "lengths": [
              "-1",
              "1"
            ],
            "values": "MQ=="
          }
        ]
      }
    }
    

    The quirk here is that type is entirely left off of the fields, resulting in us decoding the row incorrectly.

    The equivalent of after decoded by the driver is:

    {
      headers: [ 'null', ':vtg1' ],
      types: { null: undefined, ':vtg1': 'INT64' },
      fields: [ { name: 'null' }, { name: ':vtg1', type: 'INT64' } ],
      rows: [ { null: null, ':vtg1': '1' } ],
      rowsAffected: null,
      insertId: null,
      size: 1,
      statement: 'select null, 1',
      time: 194
    }
    

    As we can see, the null column has an undefined type, rather than something such as "NULL" as I'd expect.

    Even worse, but albeit an extremely edge case, on a query like select null, we actually throw an error:

    DOMException [InvalidCharacterError]: The string to be decoded is not correctly encoded.
        at new DOMException (node:internal/per_context/domexception:53:5)
        at __node_internal_ (node:internal/util:505:10)
        at atob (node:buffer:1288:11)
        at decodeRow (file:///xxx/node_modules/@planetscale/database/dist/index.js:161:20)
        at parseObjectRow (file:///xxx/node_modules/@planetscale/database/dist/index.js:149:17)
        at file:///xxx/node_modules/@planetscale/database/dist/index.js:158:88
        at Array.map (<anonymous>)
        at parse (file:///xxx/node_modules/@planetscale/database/dist/index.js:158:17)
        at Connection.execute (file:///xxx/node_modules/@planetscale/database/dist/index.js:86:31)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
    

    The HTTP response we get is:

      "result": {
        "fields": [
          {
            "name": "null"
          }
        ],
        "rows": [
          {
            "lengths": [
              "-1"
            ]
          }
        ]
      }
    

    My assumption here again is a similar quirk, and that values is just a missing key entirely since it'd be an empty string representing no data. It's stemming from within atob, so didn't check exactly what's going on, but my assumption is we're passing undefined in there, rather than anything else.

    My hunch here though is we need to explicitly handle the NULL type better, and can probably fix both issues together.

    opened by mattrobenolt 2
  • Throw errors returned from the API instead of returning

    Throw errors returned from the API instead of returning

    We should keep one pattern for error handling within our system. Rather than returning errors that we get returned back to us from the API, we can encapsulate them in a DatabaseError, which allows users to handle them all in one way.

    Closes https://github.com/planetscale/database-js/issues/61.

    opened by iheanyi 2
  • Reject promise if query fails

    Reject promise if query fails

    Hey,

    I would currently expect that when a query fails, the promise will be rejected. Maybe with a new config rejectPromiseIfQueryFails: boolean

    Thanks for building this. Works awesome with Cloudflare Workers

    opened by shyim 2
  • fetch config types not compatible with node-fetch v2 or undici

    fetch config types not compatible with node-fetch v2 or undici

    import { connect } from '@planetscale/database'
    import { fetch } from 'undici'
    
    export const connection = connect({
      url: process.env.DATABASE_URL,
      fetch,
    })
    
    

    Comes with a type error

    Type '(input: RequestInfo, init?: RequestInit | undefined) => Promise<Response>' is not assignable to type '(input: string, init?: ReqInit | undefined) => Promise<Pick<Response, "ok" | "json" | "status" | "statusText" | "text">>'.
      Types of parameters 'init' and 'init' are incompatible.
        Type 'ReqInit | undefined' is not assignable to type 'RequestInit | undefined'.
          Type 'ReqInit' is not assignable to type 'RequestInit | undefined'.
            Type 'ReqInit' is not assignable to type 'RequestInit'.
              Types of property 'headers' are incompatible.
                Type 'HeadersInit | undefined' is not assignable to type 'import("/Users/wizard/src/telley/telley-api/node_modules/.pnpm/[email protected]/node_modules/undici/types/fetch").HeadersInit | undefined'.
                  Type 'Headers' is not assignable to type 'HeadersInit | undefined'.
                    Type 'Headers' is missing the following properties from type 'Headers': keys, values, entries, [Symbol.iterator]ts(2322)
    index.d.ts(34, 5): The expected type comes from property 'fetch' which is declared here on type 'Config'
    

    A very similar error comes with the node-fetch v2 types.

    opened by reconbot 2
  • Json Datatypes are not natively supported

    Json Datatypes are not natively supported

    When sending a JSON object or JSON stringified string I get the following errors. I can't tell if I should be doing something different.

    JSON Object

    target: readcted.-.primary: vttablet: rpc error: code = Unknown desc = Invalid JSON text: \"Invalid value.\" at position 1 in value for `data`) values (:vtg1,)\", BindVars: {vtg1: \"type:VARCHAR value:\\\"[object Object]\\\"\}
    

    JSON Stringified

    target: redacted.-.primary: vttablet: rpc error: code = Unknown desc = Invalid JSON text: \"Invalid value.\" at position 0 in value for column 'Redacted.data'. (errno 3140) (sqlstate 22032) (CallerID: planetscale-admin): Sql: \"insert into Redacted(`data`) values (:vtg1)\", BindVars: {vtg1: \"type:VARCHAR value:\\\"{\\\\\\\"foo\\\\\\\":\\\\\\\"bar\\\\\\\"}\\\"\}
    

    Table Schema

    CREATE TABLE `Redacted` (
    	`id` varchar(191) NOT NULL,
    	`createdAt` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
    	`data` json NOT NULL,
    	PRIMARY KEY (`id`)
    ) ENGINE InnoDB,
      CHARSET utf8mb4,
      COLLATE utf8mb4_unicode_ci;
    
    opened by CaptainYarb 2
  • Question: How to import in node.js/express server

    Question: How to import in node.js/express server

    How can I import the js driver in node using require? Thanks!

    Edit:

    This:

    const { Client } = require("@planetscale/database");
    

    Results in this: Failed to load function definition from source: FirebaseError: Failed to load function definition from source: Failed to generate manifest from function source: Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/raphael/code/test/node_modules/@planetscale/database/dist/index.js from /Users/raphael/code/test/https/api.js not supported. Instead change the require of index.js in /Users/raphael/code/test/https/api.js to a dynamic import() which is available in all CommonJS modules

    opened by osejrafael 1
  • Remove timezone from sanitized date strings

    Remove timezone from sanitized date strings

    Including the Z timezone works in select statements but throws an "incorrect datetime value" error during insert. Strip the timezone from the string before using it as a parameter replacement value.

    opened by dgraham 1
  • Retrieved binary is converted to a string

    Retrieved binary is converted to a string

    AFAICT, this library does not support retrieving binary as binary.

    I added some code inspired by custom type casting function:

    function inflate(field, value) {
      console.log("field.type", field.type)
      console.log("value", value)
      console.log("value type", typeof value)
      return cast(field, value)
    }
    

    When I do so, here is some representative output of some UUIDs:

    field.type BINARY
    value ø0sô¼A¤ÿçÐsô
    value type string
    field.type BINARY
    value AD3ADFB038724973
    value type string
    field.type BINARY
    value type string
    

    Yes, the third value is missing - it is seemingly unprintable and silently killed that console.log. This isn't surprising because not all binary is valid utf-8.

    Some code, if you want gory details.
    create table Media_User
    (
        mediaId binary(16)  not null,
        userId  varchar(21) not null,
        primary key (mediaId, userId)
    )
        collate = utf8mb4_unicode_ci;
    
    INSERT INTO Media_User (mediaId, userId) VALUES (0x11f830737ff4bc41a4ffe792d073f41f, 'ff')
    INSERT INTO Media_User (mediaId, userId) VALUES (0x41443341444642303338373234393733, 'ff')
    INSERT INTO Media_User (mediaId, userId) VALUES (0x737295711B1121479411906BBD2F7A34, 'ff')
    
    
        import { connect, cast } from "@planetscale/database"
    
        const config = {
          cast: inflate,
          url: somethngSecret,
        }
    
        const conn = connect(config)
        const results = await conn.execute("select mediaId from planetScaleDb.Media_User")
        console.log("------------------")
        console.log(results.rows[0])
        console.log(results.rows[1])
        console.log(results.rows[2])
    

    This occurs for me on cloudflare workers (and miniflare).

    opened by AlexErrant 1
  • Wrong path to type definitions

    Wrong path to type definitions

    Hi, I’m using the @planetscale/database npm package in a TypeScript project (using Deno). It works but I had trouble getting the type definitions to work. package.json specifies dist/index.d.js as types file but this file does not exist. It should be dist/index.d.ts. Can you please fix the path in your package.json?

    For the record, here is my PoC with working types:

    // main.ts
    
    // @deno-types="./node_modules/@planetscale/database/dist/index.d.ts"
    import { connect } from "npm:@planetscale/database";
    
    const connection = connect({
      host: "xx",
      password: "xx",
      username: "xx",
    });
    
    const start = Date.now();
    const _result = await connection.execute("SELECT 1");
    console.log(`Execute: ${Date.now() - start} ms`);
    

    Then run deno run --allow-net --node-modules-dir main.ts

    Environment:

    deno --version
    deno 1.28.2 (release, aarch64-apple-darwin)
    v8 10.9.194.1
    typescript 4.8.3
    
    opened by ChristianSiegert 1
  • New as array/object option breaks Typescript

    New as array/object option breaks Typescript

    Hi! 👋

    I updated my Planetscale driver version today and Typescript started generating errors.

    After a bit of investigation, it seems that Typescript is seeing that a row can now be an Array or an Object and defaulting to Array (so things like query.rows[0].row_name break, even when specifying as).

    It seems to be this line. A row can either be an array or an object and array is preferred so it's underlined red. https://github.com/planetscale/database-js/blob/2ca73f8444c8867b690f778c4fb3090057f83217/src/index.ts#L7

    opened by isaac-mcfadyen 0
  • Doesn't work with jest

    Doesn't work with jest

    Please see #68 It was never fixed and using the minimal example provided it still errors. This is blocking me really hard if anyone could help me that would be great! Thanks

    opened by OultimoCoder 1
  • Performance difference

    Performance difference

    I did a small performance comparison with the standard MySQL connection and the new driver in a lambda function. (using this)

    I found that when using the serverless driver the cold start times and the execution times in a warm lambda take at least twice as long, often longer. For example, for a cold start with MySQL it's 3-4s, with the HTTP driver, it's more like 10s. Warm starts are 1s compared to 3 or 4s

    I haven't yet added any logging of what's taking the additional time, but before I dig into things further I was curious to find out what others' experiences were in terms of performance?

    opened by rcoundon 11
  • Send Request-Id header on requests

    Send Request-Id header on requests

    Our API supports sending along a Request-Id header, and this will be traced through on our side for logging and whatnot.

    A simple UUID or something unique would be sufficient.

    Note that this header should already be allowed by our CORS, but a manual verification would be nice.

    opened by mattrobenolt 0
Remix Stack for deploying to Vercel with remix-auth, Planetscale, Radix UI, TailwindCSS, formatting, linting etc. Written in Typescript.

Remix Synthwave Stack Learn more about Remix Stacks. npx create-remix --template ilangorajagopal/synthwave-stack What's in the stack Vercel deploymen

Ilango 56 Dec 25, 2022
This is a project manager X note taker built with Next js, Planetscale and Firebase.

Blaze is a project developed with NextJs, PlanetScale and Firebase. It's a task manager that doubles as a note taker To contribute to this project, Ki

ABDULLAH ABDULFATAH 35 Aug 8, 2022
Fetch publication metadata from the INSPIRE-HEP database

Zotero INSPIRE Metadata Updater This is an add-on for the excellent open-source reference manager Zotero. It is useful as most papers from arXiv will

null 20 Nov 27, 2022
Framework agnostic CLI tool for routes parsing and generation of a type-safe helper for safe route usage. 🗺️ Remix driver included. 🤟

About routes-gen is a framework agnostic CLI tool for routes parsing and generation of a type-safe helper for safe route usage. Think of it as Prisma,

Stratulat Alexandru 192 Jan 2, 2023
The high efficent browser driver on top of puppeteer, ready for production scenarios.

browserless is an efficient driver for controlling headless browsers built on top of puppeteer developed for scenarios where performance matters. High

microlink.io 1.2k Jan 6, 2023
Portuguese version of the Cassandra driver javascript node.js workshop

Versão em Português do workshop Cassandra driver javascript node.js Olá e bem-vindo! Este é o repositório complementar para a apresentação prática dos

DataStax Developers 2 Mar 17, 2022
🚀 Macaca Playwright driver

macaca-playwright Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API. Macaca P

Macaca 13 Nov 8, 2022
a simple wrapper nestjs dynamic module on top of surrealdb.js driver, with a consumer app to show case library in action, nothing fancy

README README Project Components Dynamic Module Consumer App Install SurrealDb Starts SurrealDb Init surrealDb Database Run App from Source Code Launc

Mário Monteiro 0 Oct 3, 2022
This simple project aims to connect to an API to fetch score data and display it on a LeaderBoard box, as well as provide the tool to submit a new score.

Leader Board: Hit the API! This simple project aims to connect to an API to fetch score data and display it on a LeaderBoard box, as well as provide t

Andrés Felipe Arroyave Naranjo 12 Apr 6, 2022
"Choose your Pokemon" is a Webpack project meant to fetch data from two different APIs: PokéAPI and Involvement API

"Choose your Pokemon" is a Webpack project meant to fetch data from two different APIs: PokéAPI and Involvement API. Here we display a list of 20 Pokemons for whom one can like, display more info, and comment; all based on the data from these two external resources.

Carlos HerverSolano 19 Mar 31, 2022
A script for Obsidian's QuickAdd plugin, to fetch books data using Google Books API.

Books script for Obsidian's Quickadd plugin Demo If this script helped you and you wish to contribute :) Description This script allows you to easily

Elaws 10 Dec 31, 2022
API and CLI tool to fetch and query Chome DevTools heap snapshots.

Puppeteer Heap Snapshot Capture heap snapshots and query the snapshot for objects matching a set of properties. Read more about it in this blog post.

Adrian Cooney 858 Jan 3, 2023
Script to fetch all NFT owners using moralis API. This script output is a data.txt file containing all owner addresses of a given NFT and their balances.

?? Moralis NFT Snapshot Moralis NFT API will only return 500 itens at a time when its is called. For that reason, a simple logic is needed to fetch al

Phill Menezes 6 Jun 23, 2022
Wraps postgres API in a pg compatible API.

postgres/pg compatibility layer Wraps postgres API in a pg compatible API.

Gajus Kuizinas 18 Oct 16, 2022
Interplanetary Database: A Database built on top of IPFS and made immutable using Ethereum blockchain.

IPDB IPDB (Interplanetary Database) is a key/value store database built on top of IPFS (Interplanetary File System). Project is intended to be an MVP

turinglabs 8 Oct 6, 2022
Visualize, modify, and build your database with dbSpy! An open-source data modeling tool to facilitate relational database development.

Visualize, modify, and build your database with dbSpy! dbSpy is an open-source data modeling tool to facilitate relational database development. Key F

OSLabs 115 Dec 22, 2022
portfolio-project is a npm package to automatically update your projects section in your portfolio website. It will fetch the selected repositories directly from your GitHub account.

portfolio-project Those days of manually updating portfolio website after every new project made are gone ⚡ Yesss . . . you read that right. ?? portfo

Gaurav Gulati 15 Aug 3, 2021
This simple project, show how work with async Fetch, function component and class component

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

DaliyaAsel 2 Feb 17, 2022
Easily fetch infos from Goyabu.com

goyabu A easy and simple way to get data from Goyabu.com. Note that the package only catches the first result, so if you want the second season of Shi

Luis Gabriel Araújo 2 Jan 28, 2022