Bree is the best job scheduler for Node.js and JavaScript with cron, dates, ms, later, and human-friendly support.

Overview

bree

chat build status code coverage code style styled with prettier made with lass license

Bree is the best job scheduler for Node.js and JavaScript with cron, dates, ms, later, and human-friendly support.

Works in Node v10+ and browsers (thanks to bthreads polyfill), uses worker threads (Node.js) and web workers (browsers) to spawn sandboxed processes, and supports async/await, retries, throttling, concurrency, and cancelable jobs with graceful shutdown. Simple, fast, and lightweight. Made for Forward Email and Lad.

❤️ Love this project? Support @niftylettuce's FOSS on Patreon or PayPal 🦄

Table of Contents

Foreword

Before creating Bree, I was a core maintainer (and financially invested in development) of Agenda. I have been with the Node.js community for a very, very long time, and have tried literally every solution out there (see Alternatives that are not production-ready). I have found that all existing solutions are subpar, as I have filed countless issues; discovered memory leaks, found functionality not working as described, unresolved core bugs have persisted over time, etc.

Previous to creating this, I was relying heavily on bull; having created @ladjs/bull – but due to core issues (and being Redis-backed) it was not the best tool for the job. Bull might have been okay if the core issues were fixed, however since it uses Redis it should not be used for a job queue. From my experience, Redis should only be used for caching and session storage purposes (e.g. CDN or managing user log in state in your application). As of the time of this writing, it has been months and the core bugs with Bull are still unresolved; as more people are continuing to reproduce and comment on the known issues.

Since workers are now readily available in LTS versions of Node, I thought it would be a great time to implement them in a job scheduler environment. Additionally, my research and development of a better anti-spam and anti-phishing classifier with Spam Scanner gave me some necessary insight to using workers.

Bree was created to give you fine-grained control with simplicity, and has built-in support for workers, sandboxed processes, graceful reloading, cron jobs, dates, human-friendly time representations, and much more. We recommend you to query a persistent database in your jobs, to prevent specific operations from running more than once. Bree does not force you to use an additional database layer of Redis or MongoDB to manage job state. In doing so, you should manage boolean job states yourself using queries. For instance, if you have to send a welcome email to users, only send a welcome email to users that do not have a Date value set yet for welcome_email_sent_at.

Install

npm:

npm install bree

yarn:

yarn add bree

Usage and Examples

The example below assumes that you have a directory jobs in the root of the directory from which you run this example. For example, if the example below is at /path/to/script.js, then /path/to/jobs/ must also exist as a directory. If you wish to disable this feature, then pass root: false as an option.

Inside this jobs directory are individual scripts which are run using Workers per optional timeouts, and additionally, an optional interval or cron expression. The example below contains comments, which help to clarify how this works.

The option jobs passed to a new instance of Bree (as shown below) is an Array. It contains values which can either be a String (name of a job in the jobs directory, which is run on boot) OR it can be an Object with name, path, timeout, and interval properties. If you do not supply a path, then the path is created using the root directory (defaults to jobs) in combination with the name. If you do not supply values for timeout and/nor interval, then these values are defaulted to 0 (which is the default for both, see index.js for more insight into configurable default options).

We have also documented all Instance Options and Job Options in this README below. Be sure to read those sections so you have a complete understanding of how Bree works.

Node

Since we use bthreads, Node v10+ is supported. For versions prior to Node v11.7.0, a ponyfill is provided for workers that uses child_process. For versions greater than or equal to Node v11.7.0, it uses workers directly. You can also pass --experimental-worker flag for older versions to use worker_threads (instead of the child_process polyfill). See the official Node.js documentation for more information.

NOTE: If you are using Node versions prior to Node v11.7.0, then in your worker files – you will need to use bthreads instead of workers. For example, you will const thread = require('bthreads'); at the top of your file, instead of requiring worker_threads. This will also require you to install bthreads in your project with npm install bthreads or yarn add bthreads.

const path = require('path');

// optional
const ms = require('ms');
const dayjs = require('dayjs');
const Graceful = require('@ladjs/graceful');
const Cabin = require('cabin');

// required
const Bree = require('bree');

//
// NOTE: see the "Instance Options" section below in this README
// for the complete list of options and their defaults
//
const bree = new Bree({
  //
  // NOTE: by default the `logger` is set to `console`
  // however we recommend you to use CabinJS as it
  // will automatically add application and worker metadata
  // to your log output, and also masks sensitive data for you
  // 
  //
  logger: new Cabin(),

  //
  // NOTE: instead of passing this Array as an option
  // you can create a `./jobs/index.js` file, exporting
  // this exact same array as `module.exports = [ ... ]`
  // doing so will allow you to keep your job configuration and the jobs
  // themselves all in the same folder and very organized
  //
  // See the "Job Options" section below in this README
  // for the complete list of job options and configurations
  //
  jobs: [
    // runs `./jobs/foo.js` on start
    'foo',

    // runs `./jobs/foo-bar.js` on start
    {
      name: 'foo-bar'
    },

    // runs `./jobs/some-other-path.js` on start
    {
      name: 'beep',
      path: path.join(__dirname, 'jobs', 'some-other-path')
    },

    // runs `./jobs/worker-1.js` on the last day of the month
    {
      name: 'worker-1',
      interval: 'on the last day of the month'
    },

    // runs `./jobs/worker-2.js` every other day
    {
      name: 'worker-2',
      interval: 'every 2 days'
    },

    // runs `./jobs/worker-3.js` at 10:15am and 5:15pm every day except on Tuesday
    {
      name: 'worker-3',
      interval: 'at 10:15 am also at 5:15pm except on Tuesday'
    },

    // runs `./jobs/worker-4.js` at 10:15am every weekday
    {
      name: 'worker-4',
      cron: '15 10 ? * *'
    },

    // runs `./jobs/worker-5.js` on after 10 minutes have elapsed
    {
      name: 'worker-5',
      timeout: '10m'
    },

    // runs `./jobs/worker-6.js` after 1 minute and every 5 minutes thereafter
    {
      name: 'worker-6',
      timeout: '1m',
      interval: '5m'
      // this is unnecessary but shows you can pass a Number (ms)
      // interval: ms('5m')
    },

    // runs `./jobs/worker-7.js` after 3 days and 4 hours
    {
      name: 'worker-7',
      // this example uses `human-interval` parsing
      timeout: '3 days and 4 hours'
    },

    // runs `./jobs/worker-8.js` at midnight (once)
    {
      name: 'worker-8',
      timeout: 'at 12:00 am'
    },

    // runs `./jobs/worker-9.js` every day at midnight
    {
      name: 'worker-9',
      interval: 'at 12:00 am'
    },

    // runs `./jobs/worker-10.js` at midnight on the 1st of every month
    {
      name: 'worker-10',
      cron: '0 0 1 * *'
    },

    // runs `./jobs/worker-11.js` at midnight on the last day of month
    {
      name: 'worker-11',
      cron: '0 0 L * *'
    },

    // runs `./jobs/worker-12.js` at a specific Date (e.g. in 3 days)
    {
      name: 'worker-12',
      // 
      date: dayjs().add(3, 'days').toDate()
      // you can also use momentjs
      // 
      // date: moment('1/1/20', 'M/D/YY').toDate()
      // you can pass Date instances (if it's in the past it will not get run)
      // date: new Date()
    },

    // runs `./jobs/worker-13.js` on start and every 2 minutes
    {
      name: 'worker-13',
      interval: '2m'
    },

    // runs `./jobs/worker-14.js` on start with custom `new Worker` options (see below)
    {
      name: 'worker-14',
      // 
      worker: {
        workerData: {
          foo: 'bar',
          beep: 'boop'
        }
      }
    },

    // runs `./jobs/worker-15.js` **NOT** on start, but every 2 minutes
    {
      name: 'worker-15',
      timeout: false, // <-- specify `false` here to prevent default timeout (e.g. on start)
      interval: '2m'
    },

    // runs `./jobs/worker-16.js` on January 1st, 2022
    // and at midnight on the 1st of every month thereafter
    {
      name: 'worker-16',
      date: dayjs('1-1-2022', 'M-D-YYYY').toDate(),
      cron: '0 0 1 * *'
    }
  ]
});

// handle graceful reloads, pm2 support, and events like SIGHUP, SIGINT, etc.
const graceful = new Graceful({ brees: [bree] });
graceful.listen();

// start all jobs (this is the equivalent of reloading a crontab):
bree.start();

/*
// start only a specific job:
bree.start('foo');

// stop all jobs
bree.stop();

// stop only a specific job:
bree.stop('beep');

// run all jobs (this does not abide by timeout/interval/cron and spawns workers immediately)
bree.run();

// run a specific job (...)
bree.run('beep');

// add a job array after initialization:
bree.add(['boop']);
// this must then be started using one of the above methods

// add a job after initialization:
bree.add('boop');
// this must then be started using one of the above methods

// remove a job after initialization:
bree.remove('boop');
*/

Browser

NOTE: Browser support is currently unstable until this GitHub issue is resolved. Contributions are welcome!

If you are using Bree in the browser, then please reference the Web Workers API (since it does not use Node.js worker threads). If the Web Workers API is not yet available, then it will be polyfilled accordingly.

VanillaJS

This is the solution for you if you're just using