PostgreSQL interface for Node.js

Overview

pg-promise

PostgreSQL interface for Node.js

Build Status Join the chat at https://gitter.im/vitaly-t/pg-promise



About

Built on top of node-postgres, this library adds the following:

  • Automatic connections
  • Automatic transactions
  • Powerful query-formatting engine + query generation
  • Declarative approach to handling query results
  • Global events reporting for central handling
  • Extensive support for external SQL files
  • Support for all promise libraries

At its inception in 2015, this library was only adding promises to the base driver, hence the name pg-promise. And while the original name was kept, the library's functionality was vastly extended, with promises now being only its tiny part.

Support & Sponsorship

I do free support here and on StackOverflow.

And if you want to help this project, I can accept Bitcoin: 1yki7MXMkuDw8qqe5icVdh1GJZSQSzKZp

Documentation

Chapter Usage below explains the basics you need to know, while the Official Documentation gets you started, and provides links to all other resources.

Contributing

Please read the Contribution Notes before opening any new issue or PR.

Usage

Once you have created a Database object, according to the steps in the Official Documentation, you get access to the methods documented below.

Methods

All query methods of the library are based off generic method query.

You should normally use only the derived, result-specific methods for executing queries, all of which are named according to how many rows of data the query is expected to return, so for each query you should pick the right method: none, one, oneOrNone, many, manyOrNone = any. Do not confuse the method name for the number of rows to be affected by the query, which is completely irrelevant.

By relying on the result-specific methods you protect your code from an unexpected number of data rows, to be automatically rejected (treated as errors).

There are also a few specific methods that you will often need:

  • result, multi, multiResult - for verbose and/or multi-query results;
  • map, each - for simpler/inline result pre-processing/re-mapping;
  • func, proc - to simplify execution of SQL functions/procedures;
  • stream - to access rows from a query via a read stream;
  • connect, task, tx + txIf - for shared connections + automatic transactions, each exposing a connected protocol that has additional methods batch, page and sequence.

The protocol is fully customizable / extendable via event extend.

IMPORTANT:

The most important methods to understand from start are task and tx/txIf (see Tasks and Transactions). As documented for method query, it acquires and releases the connection, which makes it a poor choice for executing multiple queries at once. For this reason, Chaining Queries is a must-read, to avoid writing the code that misuses connections.

Learn by Example is a beginner's tutorial based on examples.

Query Formatting

This library comes with embedded query-formatting engine that offers high-performance value escaping, flexibility and extensibility. It is used by default with all query methods, unless you opt out of it entirely via option pgFormatting within Initialization Options.

All formatting methods used internally are available from the formatting namespace, so they can also be used directly when needed. The main method there is format, used by every query method to format the query.

The formatting syntax for variables is decided from the type of values passed in:

ATTENTION: Never use ES6 template strings or manual concatenation to generate queries, as both can easily result in broken queries! Only this library's formatting engine knows how to properly escape variable values for PostgreSQL.

Index Variables

The simplest (classic) formatting uses $1, $2, ... syntax to inject values into the query string, based on their index (from $1 to $100000) from the array of values:

await db.any('SELECT * FROM product WHERE price BETWEEN $1 AND $2', [1, 10])

The formatting engine also supports single-value parametrization for queries that use only variable $1:

await db.any('SELECT * FROM users WHERE name = $1', 'John')

This however works only for types number, bigint, string, boolean, Date and null, because types like Array and Object change the way parameters are interpreted. That's why passing in index variables within an array is advised as safer, to avoid ambiguities.

Named Parameters

When a query method is parameterized with values as an object, the formatting engine expects the query to use the Named Parameter syntax $*propName*, with * being any of the following open-close pairs: {}, (), <>, [], //.

// We can use every supported variable syntax at the same time, if needed:
await db.none('INSERT INTO users(first_name, last_name, age) VALUES(${name.first}, $<name.last>, $/age/)', {
    name: {first: 'John', last: 'Dow'},
    age: 30
});

IMPORTANT: Never use the reserved ${} syntax inside ES6 template strings, as those have no knowledge of how to format values for PostgreSQL. Inside ES6 template strings you should only use one of the 4 alternatives - $(), $<>, $[] or $//. In general, you should either use the standard strings for SQL, or place SQL into external files - see Query Files.

Valid variable names are limited to the syntax of open-name JavaScript variables. And name this has special meaning - it refers to the formatting object itself (see below).

Keep in mind that while property values null and undefined are both formatted as null, an error is thrown when the property does not exist.

this reference

Property this refers to the formatting object itself, to be inserted as a JSON-formatted string.

await db.none('INSERT INTO documents(id, doc) VALUES(${id}, ${this})', {
    id: 123,
    body: 'some text'    
})
//=> INSERT INTO documents(id, doc) VALUES(123, '{"id":123,"body":"some text"}')

Nested Named Parameters

Named Parameters support property name nesting of any depth.

Example
const obj = {
    one: {
        two: {
            three: {
                value1: 123,
                value2: a => {
                    // a = obj.one.two.three
                    return 'hello';
                },
                value3: function(a) {
                    // a = this = obj.one.two.three
                    return 'world';
                },
                value4: {
                    toPostgres: a => {
                        // Custom Type Formatting
                        // a = obj.one.two.three.value4
                        return a.text;
                    },
                    text: 'custom'
                }                
            }
        }
    }
};
await db.one('SELECT ${one.two.three.value1}', obj); //=> SELECT 123
await db.one('SELECT ${one.two.three.value2}', obj); //=> SELECT 'hello'
await db.one('SELECT ${one.two.three.value3}', obj); //=> SELECT 'world'
await db.one('SELECT ${one.two.three.value4}', obj); //=> SELECT 'custom'

The last name in the resolution can be anything, including:

i.e. the resolution chain is infinitely flexible, and supports recursion without limits.

Please note, however, that nested parameters are not supported within the helpers namespace.

Formatting Filters

By default, all values are formatted according to their JavaScript type. Formatting filters (or modifiers), change that, so the value is formatted differently.

Note that formatting filters work only for normal queries, and are not available within PreparedStatement or ParameterizedQuery, because those are, by definition, formatted on the server side.

Filters use the same syntax for Index Variables and Named Parameters, following immediately the variable name:

With Index Variables
await db.any('SELECT $1:name FROM $2:name', ['price', 'products'])
//=> SELECT "price" FROM "products"
With Named Parameters
await db.any('SELECT ${column:name} FROM ${table:name}', {
    column: 'price',
    table: 'products'    
});
//=> SELECT "price" FROM "products"

The following filters are supported:

SQL Names

When a variable name ends with :name, or shorter syntax ~ (tilde), it represents an SQL name or identifier, to be escaped accordingly:

Using ~ filter
await db.query('INSERT INTO $1~($2~) VALUES(...)', ['Table Name', 'Column Name']);
//=> INSERT INTO "Table Name"("Column Name") VALUES(...)
Using :name filter
await db.query('INSERT INTO $1:name($2:name) VALUES(...)', ['Table Name', 'Column Name']);
//=> INSERT INTO "Table Name"("Column Name") VALUES(...)

Typically, an SQL name variable is a text string, which must be at least 1 character long. However, pg-promise supports a variety of ways in which SQL names can be supplied:

  • A string that contains only * (asterisks) is automatically recognized as all columns:
await db.query('SELECT $1:name FROM $2:name', ['*', 'table']);
//=> SELECT * FROM "table"
  • An array of strings to represent column names:
await db.query('SELECT ${columns:name} FROM ${table:name}', {
    columns: ['column1', 'column2'],
    table: 'table'
});
//=> SELECT "column1","column2" FROM "table"
  • Any object that's not an array gets its properties enumerated for column names:
const obj = {
    one: 1,
    two: 2
};

await db.query('SELECT $1:name FROM $2:name', [obj, 'table']);
//=> SELECT "one","two" FROM "table"

In addition, the syntax supports this to enumerate column names from the formatting object:

const obj = {
    one: 1,
    two: 2
};

await db.query('INSERT INTO table(${this:name}) VALUES(${this:csv})', obj);
//=> INSERT INTO table("one","two") VALUES(1, 2)

Relying on this type of formatting for sql names and identifiers, along with regular variable formatting protects your application from SQL injection.

Method as.name implements the formatting.

Alias Filter

An alias is a simpler, less-strict version of :name filter, which only supports a text string, i.e. it does not support *, this, array or object as inputs, like :name does. However, it supports other popular cases that are less strict, but cover at least 99% of all use cases, as shown below.

  • It will skip adding surrounding double quotes when the name is a same-case single word:
await db.any('SELECT full_name as $1:alias FROM $2:name', ['name', 'table']);
//=> SELECT full_name as name FROM "table"
  • It will automatically split the name into multiple SQL names when encountering ., and then escape each part separately, thus supporting auto-composite SQL names:
await db.any('SELECT * FROM $1:alias', ['schemaName.table']);
//=> SELECT * FROM "schemaName".table

For more details see method as.alias that implements the formatting.

Raw Text

When a variable name ends with :raw, or shorter syntax ^, the value is to be injected as raw text, without escaping.

Such variables cannot be null or undefined, because of the ambiguous meaning in this case, and those values will throw error Values null/undefined cannot be used as raw text.

const where = pgp.as.format('WHERE price BETWEEN $1 AND $2', [5, 10]); // pre-format WHERE condition
await db.any('SELECT * FROM products $1:raw', where);
//=> SELECT * FROM products WHERE price BETWEEN 5 AND 10

Special syntax this:raw / this^ is supported, to inject the formatting object as raw JSON string.

WARNING:
This filter is unsafe, and should not be used for values that come from the client side, as it may result in SQL injection.

Open Values

When a variable name ends with :value, or shorter syntax #, it is escaped as usual, except when its type is a string, the trailing quotes are not added.

Open values are primarily to be able to compose complete LIKE/ILIKE dynamic statements in external SQL files, without having to generate them in the code.

i.e. you can either generate a filter like this in your code:

const name = 'John';
const filter = '%' + name + '%';

and then pass it in as a regular string variable, or you can pass in only name, and have your query use the open-value syntax to add the extra search logic:

SELECT * FROM table WHERE name LIKE '%$1:value%')

WARNING:
This filter is unsafe, and should not be used for values that come from the client side, as it may result in SQL injection.

Method as.value implements the formatting.

JSON Filter

When a variable name ends with :json, explicit JSON formatting is applied to the value.

By default, any object that's not Date, Array, Buffer, null or Custom-Type (see Custom Type Formatting), is automatically formatted as JSON.

Method as.json implements the formatting.

CSV Filter

When a variable name ends with :csv or :list, it is formatted as a list of Comma-Separated Values, with each value formatted according to its JavaScript type.

Typically, you would use this for a value that's an array, though it works for single values also. See the examples below.

Using :csv filter
const ids = [1, 2, 3];
await db.any('SELECT * FROM table WHERE id IN ($1:csv)', [ids])
//=> SELECT * FROM table WHERE id IN (1,2,3)
Using :list filter
const ids = [1, 2, 3];
await db.any('SELECT * FROM table WHERE id IN ($1:list)', [ids])
//=> SELECT * FROM table WHERE id IN (1,2,3)

Using automatic property enumeration:

Enumeration with :csv filter
const obj = {first: 123, second: 'text'};

await db.none('INSERT INTO table($1:name) VALUES($1:csv)', [obj])
//=> INSERT INTO table("first","second") VALUES(123,'text')

await db.none('INSERT INTO table(${this:name}) VALUES(${this:csv})', obj)
//=> INSERT INTO table("first","second") VALUES(123,'text')
Enumeration with :list filter
const obj = {first: 123, second: 'text'};

await db.none('INSERT INTO table($1:name) VALUES($1:list)', [obj])
//=> INSERT INTO table("first","second") VALUES(123,'text')

await db.none('INSERT INTO table(${this:name}) VALUES(${this:list})', obj)
//=> INSERT INTO table("first","second") VALUES(123,'text')

Method as.csv implements the formatting.

Custom Type Formatting

The library supports dual syntax for CTF (Custom Type Formatting):

  • Explicit CTF - extending the object/type directly, for ease of use, while changing its signature;
  • Symbolic CTF - extending the object/type via Symbol properties, without changing its signature.

The library always first checks for the Symbolic CTF, and if no such syntax is used, only then it checks for the Explicit CTF.

Explicit CTF

Any value/object that implements function toPostgres is treated as a custom-formatting type. The function is then called to get the actual value, passing it the object via this context, and plus as a single parameter (in case toPostgres is an ES6 arrow function):

const obj = {
    toPostgres(self) {
        // self = this = obj
        
        // return a value that needs proper escaping
    }
}

Function toPostgres can return anything, including another object with its own toPostgres function, i.e. nested custom types are supported.

The value returned from toPostgres is escaped according to its JavaScript type, unless the object also contains property rawType set to a truthy value, in which case the returned value is considered pre-formatted, and thus injected directly, as Raw Text:

const obj = {
    toPostgres(self) {
        // self = this = obj
        
        // return a pre-formatted value that does not need escaping
    },
    rawType: true // use result from toPostgres directly, as Raw Text
}

Example below implements a class that auto-formats ST_MakePoint from coordinates:

class STPoint {
    constructor(x, y) {
        this.x = x;
        this.y = y;
        this.rawType = true; // no escaping, because we return pre-formatted SQL
    }
    
    toPostgres(self) {
        return pgp.as.format('ST_MakePoint($1, $2)', [this.x, this.y]);
    }
}

And a classic syntax for such a class is even simpler:

function STPoint(x, y){
    this.rawType = true; // no escaping, because we return pre-formatted SQL
    this.toPostgres = () => pgp.as.format('ST_MakePoint($1, $2)', [x, y]);
}

With this class you can use new STPoint(12, 34) as a formatting value that will be injected correctly.

You can also use CTF to override any standard type:

Date.prototype.toPostgres = a => a.getTime();

Symbolic CTF

The only difference from Explicit CTF is that we set toPostgres and rawType as ES6 Symbol properties, defined in the ctf namespace:

const {toPostgres, rawType} = pgp.as.ctf; // Global CTF symbols

const obj = {
    [toPostgres](self) {
        // self = this = obj
        
        // return a pre-formatted value that does not need escaping
    },
    [rawType]: true // use result from toPostgres directly, as Raw Text
};

As CTF symbols are global, you can also configure objects independently of this library:

const ctf = {
    toPostgres: Symbol.for('ctf.toPostgres'),
    rawType: Symbol.for('ctf.rawType')
};

Other than that, it works exactly as the Explicit CTF, but without changing the object's signature.

If you do not know what it means, read the ES6 Symbol API and its use for unique property names. But in short, Symbol properties are not enumerated via for(name in obj), i.e. they are not generally visible within JavaScript, only through specific API Object.getOwnPropertySymbols.

Query Files

Use of external SQL files (via QueryFile) offers many advantages:

  • Much cleaner JavaScript code, with all SQL kept in external files;
  • Much easier to write large and well-formatted SQL, with many comments and whole revisions;
  • Changes in external SQL can be automatically re-loaded (option debug), without restarting the app;
  • Pre-formatting SQL upon loading (option params), automating two-step SQL formatting;
  • Parsing and minifying SQL (options minify + compress), for early error detection and compact queries.
Example
const {join: joinPath} = require('path');

// Helper for linking to external query files:
function sql(file) {
    const fullPath = joinPath(__dirname, file);
    return new pgp.QueryFile(fullPath, {minify: true});
}

// Create a QueryFile globally, once per file:
const sqlFindUser = sql('./sql/findUser.sql');

db.one(sqlFindUser, {id: 123})
    .then(user => {
        console.log(user);
    })
    .catch(error => {
        if (error instanceof pgp.errors.QueryFileError) {
            // => the error is related to our QueryFile
        }
    });

File findUser.sql:

/*
    multi-line comments are supported
*/
SELECT name, dob -- single-line comments are supported
FROM Users
WHERE id = ${id}

Every query method of the library can accept type QueryFile as its query parameter. Type QueryFile never throws any error, leaving it for query methods to gracefully reject with QueryFileError.

Use of Named Parameters within external SQL files is recommended over the Index Variables, because it makes the SQL much easier to read and understand, and because it also allows Nested Named Parameters, so variables in a large and complex SQL file can be grouped in namespaces for even easier visual separation.

Tasks

A task represents a shared connection for executing multiple queries:

db.task(t => {
    // execute a chain of queries against the task context, and return the result:
    return t.one('SELECT count(*) FROM events WHERE id = $1', 123, a => +a.count)
        .then(count => {
            if(count > 0) {
                return t.any('SELECT * FROM log WHERE event_id = $1', 123)
                    .then(logs => {
                        return {count, logs};
                    })
            }
            return {count};
        });    
})
    .then(data => {
        // success, data = either {count} or {count, logs}
    })
    .catch(error => {
        // failed    
    });

Tasks provide a shared connection context for its callback function, to be released when finished, and they must be used whenever executing more than one query at a time. See also Chaining Queries to understand the importance of using tasks.

You can optionally tag tasks (see Tags), and use ES7 async syntax:

With ES7 async
db.task(async t => {
    const count = await t.one('SELECT count(*) FROM events WHERE id = $1', 123, a => +a.count);
    if(count > 0) {
        const logs = await t.any('SELECT * FROM log WHERE event_id = $1', 123);
        return {count, logs};
    }
    return {count};
})
    .then(data => {
        // success, data = either {count} or {count, logs}
    })
    .catch(error => {
        // failed    
    });
With ES7 async + tag
db.task('get-event-logs', async t => {
    const count = await t.one('SELECT count(*) FROM events WHERE id = $1', 123, a => +a.count);
    if(count > 0) {
        const logs = await t.any('SELECT * FROM log WHERE event_id = $1', 123);
        return {count, logs};
    }
    return {count};
})
    .then(data => {
        // success, data = either {count} or {count, logs}
    })
    .catch(error => {
        // failed    
    });

Conditional Tasks

Method taskIf creates a new task only when required, according to the condition.

The default condition is to start a new task only when necessary, such as on the top level.

With default condition
db.taskIf(t1 => {
    // new task has started, as the top level doesn't have one
    return t1.taskIf(t2 => {
        // Task t1 is being used, according to the default condition
        // t2 = t1
    });
})
With a custom condition - value
db.taskIf({cnd: false}, t1 => {
    // new task is created, i.e. option cnd is ignored here,
    // because the task is required on the top level
    return t1.taskIf({cnd: true}, t2 => {
        // new task created, because we specified that we want one;
        // t2 != t1
    });
})
With a custom condition - callback
const cnd = c => {
    // c.ctx - task/tx context (not available on the top level)
    // default condition: return !c.ctx;
    return someValue;
};

db.taskIf({cnd}, t1 => {
    // new task is always created, because it is required on the top level
    return t1.taskIf({cnd}, t2 => {
        // if someValue is truthy, a new task is created (t2 != t1);
        // otherwise, we continue with the containing task (t2 = t1).
    });
})

Transactions

Transaction method tx is like task, which also executes BEGIN + COMMIT/ROLLBACK:

db.tx(t => {
    // creating a sequence of transaction queries:
    const q1 = t.none('UPDATE users SET active = $1 WHERE id = $2', [true, 123]);
    const q2 = t.one('INSERT INTO audit(entity, id) VALUES($1, $2) RETURNING id', ['users', 123]);

    // returning a promise that determines a successful transaction:
    return t.batch([q1, q2]); // all of the queries are to be resolved;
})
    .then(data => {
        // success, COMMIT was executed
    })
    .catch(error => {
        // failure, ROLLBACK was executed
    });

If the callback function returns a rejected promise or throws an error, the method will automatically execute ROLLBACK at the end. In all other cases the transaction will be automatically closed by COMMIT.

The same as tasks, transactions support Tags and ES7 async:

With ES7 async
db.tx(async t => {
    await t.none('UPDATE users SET active = $1 WHERE id = $2', [true, 123]);
    await t.one('INSERT INTO audit(entity, id) VALUES($1, $2) RETURNING id', ['users', 123]);
})
    .then(data => {
        // success, COMMIT was executed
    })
    .catch(error => {
        // failure, ROLLBACK was executed
    });
With ES7 async + tag
db.tx('update-user', async t => {
    await t.none('UPDATE users SET active = $1 WHERE id = $2', [true, 123]);
    await t.one('INSERT INTO audit(entity, id) VALUES($1, $2) RETURNING id', ['users', 123]);
})
    .then(data => {
        // success, COMMIT was executed
    })
    .catch(error => {
        // failure, ROLLBACK was executed
    });

Nested Transactions

Nested transactions automatically share the connection between all levels. This library sets no limitation as to the depth (nesting levels) of transactions supported.

Example
db.tx(t => {
    const queries = [
        t.none('DROP TABLE users;'),
        t.none('CREATE TABLE users(id SERIAL NOT NULL, name TEXT NOT NULL)')
    ];
    for (let i = 1; i <= 100; i++) {
        queries.push(t.none('INSERT INTO users(name) VALUES($1)', 'name-' + i));
    }
    queries.push(
        t.tx(t1 => {
            return t1.tx(t2 => {
                return t2.one('SELECT count(*) FROM users');
            });
        }));
    return t.batch(queries);
})
    .then(data => {
        // success
    })
    .catch(error => {
        // failure
    });

If you want to avoid automatic occurrence of nested transactions, see Conditional Transactions.

Limitations

It is important to know that PostgreSQL does not support full/atomic nested transactions, it only supports savepoints inside top-level transactions, to allow partial rollbacks.

Postgres uses BEGIN with COMMIT / ROLLBACK for top-level transactions, and SAVEPOINT name with RELEASE / ROLLBACK TO name for inner save-points.

This library automatically executes all such transaction and savepoint commands, with unique savepoint names, based on the transaction level, plus index within the current level, in the form of sp_x_y.

In the name, x is the transaction level, starting with 1 (because 0 is the top-level transaction that does not use savepoints). And y represents sub-transaction order/index within the current level, starting with 1. So the first savepoint on the top level will be named sp_1_1.

Configurable Transactions

TransactionMode type can extend your BEGIN command with transaction configuration:

const {TransactionMode, isolationLevel} = pgp.txMode;
 
// Create a reusable transaction mode (serializable + read-only + deferrable):
const mode = new TransactionMode({
    tiLevel: isolationLevel.serializable,
    readOnly: true,
    deferrable: true
});

db.tx({mode}, t => {
    // do transaction queries here
})
    .then(() => {
        // success;
    })
    .catch(error => {
        // failure    
    });

Instead of the default BEGIN, such transaction will open with the following command:

BEGIN ISOLATION LEVEL SERIALIZABLE READ ONLY DEFERRABLE

Transaction Mode is set via option mode, preceding the callback function. See methods tx and txIf.

This is the most efficient and best-performing way of configuring transactions. In combination with Transaction Snapshots you can make the most out of transactions in terms of performance and concurrency.

Conditional Transactions

Method txIf executes a transaction / tx when a specified condition is met, or else it executes a task.

When no condition is specified, the default is to start a transaction, if currently not in one, or else it starts a task. It is useful when you want to avoid Nested Transactions - savepoints.

With default condition
db.txIf(t => {
    // transaction is started, as the top level doesn't have one
    return t.txIf(t2 => {
        // a task is started, because there is a parent transaction        
    });
})
With a custom condition - value
db.txIf({cnd: someValue}, t => {
    // if condition is truthy, a transaction is started
    return t.txIf(t2 => {
        // a task is started, if the parent is a transaction
        // a transaction is started, if the parent is a task
    });
})
With a custom condition - callback
const cnd = c => {
    // c.ctx - task/transaction context (not available on the top level)
    // default condition: return !c.ctx || !c.ctx.inTransaction;
    return someValue;
};

db.txIf({cnd}, t => {
    // if condition is truthy, a transaction is started
    return t.txIf(t2 => {
        // a task is started, if the parent is a transaction
        // a transaction is started, if the parent is a task
    });
})

Library de-initialization

This library manages all database connections via the connection pool, which internally caches them.

Connections in the cache expire due to inactivity after idleTimeoutMillis number of milliseconds, which you can set only when creating the Database object.

While there is a single open connection in the pool, the process cannot terminate by itself, only via process.exit(). If you want the process to finish by itself, without waiting for all connections in the pool to expire, you need to force the pool to shut down all the connections it holds:

db.$pool.end(); // shuts down the connection pool associated with the Database object

For example, if you are using the Bluebird library, you can chain the last promise in the process like this:

.finally(db.$pool.end);

IMPORTANT: Note that if your app is an HTTP service, or generally an application that does not feature any exit point, then you should not do any de-initialization at all. It is only if your app is a run-through process/utility, then you might want to use it, so the process ends without delays.

In applications that either use multiple databases or execute a multi-pool strategy for balanced query loads, you would end up with multiple Database objects, each with its own connection pool. In this scenario, in order to exit the process normally, at a particular point, you can call pgp.end to shut down all connection pools at once:

pgp.end(); // shuts down all connection pools created in the process

or promise-chained to the last query block in the process:

.finally(pgp.end);

Once you have shut down the pool associated with your Database object, you can no longer use the object, and any of its query methods will be rejecting with Error = Connection pool of the database object has been destroyed.

See the relevant API: pgp.end, Database.$pool

History

Although this project formally maintains a CHANGELOG, for a short list of the top-level changes, for detailed changes between versions you should see the corresponding release notes.

Comments
  • Access to ctx.db after disconnect()

    Access to ctx.db after disconnect()

    Not quite sure about the circumstances, but I'm getting TypeError: Cannot read property 'client' of null in index.js:430.

    ctx.db is null, probably because disconnect() was already called.

    improvement 
    opened by oliversalzburg 65
  • Question re: WARNING: Creating a duplicate database object for the same connection

    Question re: WARNING: Creating a duplicate database object for the same connection

    Is the right way to set this up in an Express app to create a separate db module per the example and then require that in each set of routes? I've done that, but am still getting the warnings. Do I need a separate database object for each set of routes or can you recommend otherwise? Thanks,

    improvement question fixed 
    opened by CalebEverett 59
  • Export queryResult

    Export queryResult

    Query Result Mask mentions that queryResult can be used to construct a bitmask to qualify expectations about a query result.

    I know it's trivial to just use the integer values directly, but I think it would be nice if queryResult was exported from the library, so that it can be used when calling query().

    question 
    opened by oliversalzburg 57
  • support for nested named parameters

    support for nested named parameters

    Hey @vitaly-t, thanks for the great library!

    While using named parameters like:

    query('INSERT into users (name) VALUES ($[name])', { name: 'Fred' })
    

    There are more complex cases I run across where I'd really love to be able to reference nested values, like many similar syntax in other libraries allow for, like:

    '$[user.firstName]'
    

    Or even:

    '$[snapshots.0.data]'
    

    I think this would feel very intuitive, and make dealing with some complex cases a lot easier. Right now I'm having to write "flattening" helpers to achieve the same affect while keeping things in the variables object namespaced from each other.

    If we used lodash/get for accessing, I think this might be really easy to add support for, and be done in a fairly standardized way.

    Thanks for reading!

    improvement 
    opened by ianstormtaylor 39
  • Setting application_name to current application user on global db object

    Setting application_name to current application user on global db object

    I'm trying to achieve something similar to #71, #100 —  setting the application_name to include the current application user's id for use in audit tables (via trigger functions).

    I haven't been able to set the application_name on a configured db object (despite this SO post) and the furthest I've got is this example from Vitaly of using event extend:

    // with extra parameter:
    var options = {
        extend: obj=> {
            obj.myMethodWithParam = (query, values, context)=> {
                // append/pre-pend query parameters, using the context;
                return obj.query(query, values);
            }
        }
    };
    

    How can I set application_name and would use of extend mean having to use db.myMethodWithParam vs. db.any, db.one etc. or does it 'extend' the base methods?

    Thanks, Lloyd

    question 
    opened by lloydh 36
  • Prepend/append a statement before/after each query

    Prepend/append a statement before/after each query

    I would like to prepend/append a statement to each query issued that sets and then clears a GUC variable containing the session_id of the logged in web user.

    If the use case is unclear, it's to enable the following pattern: https://gist.github.com/jmealo/4dc1c59b8d31009f9262

    question 
    opened by jmealo 36
  • formatting columns as a different type

    formatting columns as a different type

    what i mean by this, is that if you need to format a variable in a query as something other than what pg-promise guesses that data is, it's difficult.

    say you have a table with a json/jsonb column named "names", and you'd like to store an array of strings to that column. you can't do INSERT INTO "table" ("names") VALUES ($[names_param]) because pg-promise formats the array as a type of text[] rather than json, which for the majority of cases is probably correct. however, if you're storing an array in a json column it will fail because postgres can't coerce text[] to json.

    you can work around this by either running JSON.stringify() on the names_param first, which then causes the query to pass the value as text and postgres will coerce that to json without issue. you can also use the pg.as.json method to format the parameter, also ahead of time. it's annoying, but it works.

    what i would be interested in seeing, is some way of passing the format to be used for a column as part of the query. we already have $[param] for standard formatting, and $[param^] for raw values, what if we had something like $[param:json] to tell pg-promise that this value should be formatted as json?

    improvement question 
    opened by nlf 36
  • Support named parameters

    Support named parameters

    It'd be nice to specify named parameters in the query that references properties on an object passed in place of the array of values that would normally map to $1, $2, etc.

    db.one("select * from users where id=:id", {id: 123}) // find the user from id;
        .then(function(data){
            // find 'login' records for the user found:
            return db.query("select * from audit where event=:event and userId=:userId",
            {event: "login", userId: data.id});
        })
        .then(function(data){
            // display found audit records;
            console.log(data);
        }, function(reason){
            console.log(reason); // display reason why the call failed;
        })
    

    I created a small wrapper for the plain postgres module to do this for my internal project. I really like how your library looks and think this would be an excellent addition to essentially make my project obsolete.

    https://github.com/joeandaverde/tinypg

    improvement suggestion 
    opened by dynajoe 32
  • [help wanted] Query formatting issue

    [help wanted] Query formatting issue

    hi, I have a normal sql insert statement where i am trying to insert UUID value. I have been struggling with this error for hours. What is the problem in the Query format? Here is the below code that i use to execute the query from Queryfile. I am using pg-promise named parameters passed with object. I'm not sure it also throws the error twice.

    Source code

    pgdb.one(postgresMapper.Registry.TransactionRequest.insert,{
            transaction_request_guid: transactionRequestGuid,
            transaction_request_attributes_id: docId.toString(),
          })
            .then((data) => {
              request.app.transaction_id = data.transaction_request_id;
      });
    

    Where postgresMapper.Registry.TransactionRequest.insert is the QueryFile

    Console error QueryFile { file: "D:\aud-plugin\build\sqls\insert-transaction-request.sql" options: {"debug":true,"minify":true,"compress":false,"noWarnings":false} query: "INSERT INTO public.transaction_request (transaction_request_guid,transaction_reques t_attributes_id,createdon_utc) VALUES(${transaction_request_guid},${transaction_request_attribu tes_id},CURRENT_TIMESTAMP) RETURNING transaction_request_id" } QueryFile { file: "D:\aud-plugin\build\sqls\insert-transaction-request.sql" options: {"debug":true,"minify":true,"compress":false,"noWarnings":false} query: "INSERT INTO public.transaction_request (transaction_request_guid,transaction_reques t_attributes_id,createdon_utc) VALUES(${transaction_request_guid},${transaction_request_attribu tes_id},CURRENT_TIMESTAMP) RETURNING transaction_request_id" } (node:6164) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 4): Ty peError: Invalid query format. (node:6164) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 6): Ty peError: Invalid query format.

    question 
    opened by keerthivasan-r 30
  • Nested transaction behavior - maybe incorrect?

    Nested transaction behavior - maybe incorrect?

    I have a situation that kind of looks like this... (stole this from the pg-promise tests)

    db.tx(function (t1) {
        context1 = this;
        return this.none('update users set login=$1', ['External']).then(function () {
            return context1.tx(function (t2) {
                return t2.none('update users set login=$1', ['External']) 
            });
        }).then(function () {
            return context1.one('select * from unknowntable') // emulating a bad query;
        })
    })
    

    The premise - it creates a transaction, then does something, then creates an inner transaction that does something else, then finishes the inner transaction, then does something else which causes the outer transaction to rollback. I know it's a contrived situation, but when you have situations involving other dependant methods the composition does make sense. This is just a simplified form.

    The pg-monitor outlook looks like this...

    12:21:30 tx/start 12:21:30 tx: begin 12:21:30 tx: update users set login='External' 12:21:30 tx/start 12:21:30 tx: begin 12:21:30 tx: select * from users 12:21:30 tx: commit 12:21:30 tx/end; duration: .004, success: true 12:21:30 tx: select * from unknowntable 12:21:30 error: relation "unknowntable" does not exist tx: select * from unknowntable 12:21:30 tx: rollback 12:21:30 tx/end; duration: .013, success: false

    You can see in the bolded line that pg-promise finishes the inner transaction and issues a commit. This is incorrect behaviour as there is an outer scope in play, and PG does not support nested transactions. This commit actually commits the initial outer scope update, as well as commiting the inner scope. Shouldn't it hold the commit until the outer transaction completes?

    Any advice on this situation? Thanks!

    bug low frequency 
    opened by vertiman 30
  • How to set intervalStyle globally?

    How to set intervalStyle globally?

    I would like to tell Postgres to return intervals as ISO 8601 strings always and on all connections. To do this I must set intervalStyle as follows:

    SET intervalStyle = iso_8601
    

    What's the best way to do this? I tried db.none('SET intervalStyle = iso_8601'); but that's not working. I suspect it is setting the value on only one connection.

    question 
    opened by nareshbhatia 29
Releases(11.0.2)
Owner
Vitaly Tomilov
Full-Stack Developer
Vitaly Tomilov
Execute one command (or mount one Node.js middleware) and get an instant high-performance GraphQL API for your PostgreSQL database!

PostGraphile Instant lightning-fast GraphQL API backed primarily by your PostgreSQL database. Highly customisable and extensible thanks to incredibly

Graphile 11.7k Jan 4, 2023
PostgreSQL client for node.js.

node-postgres Non-blocking PostgreSQL client for Node.js. Pure JavaScript and optional native libpq bindings. Monorepo This repo is a monorepo which c

Brian C 10.9k Jan 9, 2023
A simple Node.js ORM for PostgreSQL, MySQL and SQLite3 built on top of Knex.js

bookshelf.js Bookshelf is a JavaScript ORM for Node.js, built on the Knex SQL query builder. It features both Promise-based and traditional callback i

Bookshelf.js 6.3k Jan 2, 2023
TypeScript ORM for Node.js based on Data Mapper, Unit of Work and Identity Map patterns. Supports MongoDB, MySQL, MariaDB, PostgreSQL and SQLite databases.

TypeScript ORM for Node.js based on Data Mapper, Unit of Work and Identity Map patterns. Supports MongoDB, MySQL, MariaDB, PostgreSQL and SQLite datab

MikroORM 5.4k Dec 31, 2022
Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server & SQLite

Prisma Quickstart • Website • Docs • Examples • Blog • Slack • Twitter • Prisma 1 What is Prisma? Prisma is a next-generation ORM that consists of the

Prisma 28k Jan 2, 2023
A Node.js ORM for MySQL, SQLite, PostgreSQL, MongoDB, GitHub and serverless service like Deta, InspireCloud, CloudBase, LeanCloud.

Dittorm A Node.js ORM for MySQL, SQLite, PostgreSQL, MongoDB, GitHub and serverless service like Deta, InspireCloud, CloudBase, LeanCloud. Installatio

Waline 21 Dec 25, 2022
A Node.js library for retrieving data from a PostgreSQL database with an interesting query language included.

RefQL A Node.js library for retrieving data from a PostgreSQL database with an interesting query language included. Introduction RefQL is about retrie

Rafael Tureluren 7 Nov 2, 2022
ORM for TypeScript and JavaScript (ES7, ES6, ES5). Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.

TypeORM is an ORM that can run in NodeJS, Browser, Cordova, PhoneGap, Ionic, React Native, NativeScript, Expo, and Electron platforms and can be used

null 30.1k Jan 3, 2023
A query builder for PostgreSQL, MySQL and SQLite3, designed to be flexible, portable, and fun to use.

knex.js A SQL query builder that is flexible, portable, and fun to use! A batteries-included, multi-dialect (MSSQL, MySQL, PostgreSQL, SQLite3, Oracle

knex 16.9k Jan 4, 2023
A PostgreSQL client with strict types, detailed logging and assertions.

Slonik A battle-tested PostgreSQL client with strict types, detailed logging and assertions. (The above GIF shows Slonik producing query logs. Slonik

Gajus Kuizinas 3.6k Jan 3, 2023
NodeJS PostgreSQL database performance insights. Locks, index usage, buffer cache hit ratios, vacuum stats and more.

Node Postgres Extras NodeJS port of Heroku PG Extras with several additions and improvements. The goal of this project is to provide powerful insights

Paweł Urbanek 68 Nov 14, 2022
A simple url shorter API built with nodejs running on Kubernetes in Google Cloud, using PostgreSQL for storage and cloud sql proxy.

Simple URL Shorter - Google Cloud - Kubernetes A simple url shorter API built with nodejs running on Kubernetes in Google Cloud, using PostgreSQL for

null 3 Nov 25, 2021
Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application

Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application

DbGate 2k Dec 30, 2022
just a graphql example created by typescript + fastify + mikro-orm(postgresql) + mercurius(graphql adaptor) + type-graphql

fastify-mikro-orm-mercurius-graphql-example A MikroORM boilerplate for GraphQL made with Fastify, Mercurius, Typescript using TypeGraphQL ?? Packages

Vigen 10 Aug 28, 2022
A tiny wrapper around pg that makes PostgreSQL a lot of fun to use. Written in TypeScript.

A tiny wrapper around pg that makes PostgreSQL a lot of fun to use. Written in TypeScript.

Mojolicious 8 Nov 29, 2022
Validate and auto-generate TypeScript types from raw SQL queries in PostgreSQL.

SafeQL Write SQL Queries With Confidence • Get started Install I would first recommend follow the instructions in the documentation. npm install --sav

null 747 Dec 28, 2022
Common Database Interface for Node

database-js Wrapper for multiple databases with a JDBC-like connection Database-js implements a common, promise-based interface for SQL database acces

null 57 Dec 3, 2022
A web client port-scanner written in GO, that supports the WASM/WASI interface for Browser WebAssembly runtime execution.

WebAssembly Port Scanner Written in Go with target WASM/WASI. The WASM main function scans all the open ports in the specified range (see main.go), vi

Avi Lumelsky 74 Dec 27, 2022
A web SQL interface to your Stripe account using Datasette.

Datasette, Stripe and tdog Or: Stripe Sigma Alternative Datasette is a web GUI for exploring SQLite datasets. Stripe handles online payments. Sigma is

Table Dog 20 Nov 27, 2022