Pack all your node_modules and other files you want inside your project to a zip file.

Last update: Dec 1, 2022

Overview

🚀 Node Modules Packer

This is a library to package all your node_modules and other files you want inside your project to a zip file. It's like using npm prune --production but without all the heavy I/O operations.

You can use this library to deploy applications in serverless environments, for example, without having to do lots of crazy configurations with webpack.

I personally created this library inspired by an internal library I created for my company to deploy our NestJS apps for AWS Lambda, with this guy I improve deployment time by up to 432% (M2 SSD) with the benefit that my Webstorm doesn't go crazy with dependency indexing every time I implement because I no longer need to run npm prune --production just to get a descending build size.

Wait, you're asking me why I build and deploy the APIs on my computer instead of using CI/CD?

Well the answer is pretty simple, I want to be in control of these things ~~and we didn't have a CI/CD until a few months ago.~~

Use cases

In which cases this library might suit you:

If you deploy your nodejs apps in zip file with node_modules.
If you want to keep the directory structure (eg typeorm).
If you don't like dealing with webpack and just want to get things done.
If you use terraform to deploy in serverless environments, just point the output file to terraform and that's it.
If you like to deploy your app manually.
- This library can give you more control over how you compress your files without having to write a lot of code to do so.

In which cases this library might not fit you:

If you already have some webpack configuration to package your app.
If you don't need to maintain the directory structure.
If your nodejs app is very simple (see serverless-bundle).
If you use serverless.com.
- I've personally never used it, but it looks pretty good to use, see their packaging docs.

Usage

First, install the library with global flag:

npm i -g @h4ad/node-modules-packer

Then, enter inside the project you want to pack and run:

node-modules-packer run ./

By default, this command will:

Find all production dependencies and will package them all.
Ignore all development/peer/optional dependencies
Ignore many common unused file extensions (eg: .md, .d.ts, .js.map, ..etc)
- See the full list here.
Output the final zip with the name deploy.zip.

Examples

Include more files in the deploy.zip:

node-modules-packer run ./ -i dist -i package.json -i package-lock.json

Exclude unwanted file extensions from node_modules:

# This will exclude all json files from `deploy.zip`.
node-modules-packer run ./ -e .json

Include development/peer/optional dependencies (why?):

node-modules-packer run ./ --dev
node-modules-packer run ./ --peer
node-modules-packer run ./ --optional
# or have all at once
node-modules-packer run ./ --dev --peer --optional

Disable default ignored extensions (too much or we ignore something you want?):

See the full list here. to know what you will leave inside your zip file if you run with this flag.

node-modules-packer run ./ --disable-default-ignore-file-ext

Ignore some node folders/paths that you know that should be OUT of your zip file:

node-modules-packer run ./ --ignore-node-path="typeorm/browser" --ignore-node-path="aws-sdk"

Or include some node folders/paths that you know that should be INSIDE of your zip file, this is particulary usefull if you have some dependency with some misconfiguration of their dependencies.

# the path will be concatenated with `node_modules`, so this became
# `node_modules/some-dependency-you-want`
node-modules-packer run ./ --include-node-path="some-dependency-you-want"

We don't include sub-dependencies of these folders, so if that dependency has another dependency, that dependency might be outside your zip.

You can change the output path and the output filename with:

node-modules-packer run ./ --output-path ./deploy --output-file result.zip

Headless

You can use this library in headless mode, for example, for cases where there is a lot of customization.

import Run from '@h4ad/node-modules-packer/lib/commands/run';

// this is my configuration to deploy my NestJS APIs
// to AWS Lambda
const result = await Run.headless({
  dir: './',
  ignoreNodePath: ['typeorm/browser', 'aws-crt/dist/bin', 'aws-crt/dist.browser', 'sqlite3', 'aws-sdk'],
  include: ['dist', 'ormconfig.js'],
  outputPath: './deploy',
  outputFile: 'deploy.zip',
});

console.log(result.size);
console.log(result.file);
console.log(result.path);

Benchmarks

See here more about.

Reference

See commands reference

node-modules-packer autocomplete [SHELL]
node-modules-packer commands
node-modules-packer help [COMMAND]
node-modules-packer plugins
node-modules-packer plugins:install PLUGIN...
node-modules-packer plugins:inspect PLUGIN...
node-modules-packer plugins:install PLUGIN...
node-modules-packer plugins:link PLUGIN
node-modules-packer plugins:uninstall PLUGIN...
node-modules-packer plugins:uninstall PLUGIN...
node-modules-packer plugins:uninstall PLUGIN...
node-modules-packer plugins:update
node-modules-packer run [DIR]
node-modules-packer version

`node-modules-packer autocomplete [SHELL]`

display autocomplete installation instructions

USAGE
  $ node-modules-packer autocomplete [SHELL] [-r]

ARGUMENTS
  SHELL  shell type

FLAGS
  -r, --refresh-cache  Refresh cache (ignores displaying instructions)

DESCRIPTION
  display autocomplete installation instructions

EXAMPLES
  $ node-modules-packer autocomplete

  $ node-modules-packer autocomplete bash

  $ node-modules-packer autocomplete zsh

  $ node-modules-packer autocomplete --refresh-cache

See code: @oclif/plugin-autocomplete

`node-modules-packer commands`

list all the commands

USAGE
  $ node-modules-packer commands [--json] [-h] [--hidden] [--tree] [--columns <value> | -x] [--sort <value>]
    [--filter <value>] [--output csv|json|yaml |  | [--csv | --no-truncate]] [--no-header | ]

FLAGS
  -h, --help         Show CLI help.
  -x, --extended     show extra columns
  --columns=<value>  only show provided columns (comma-separated)
  --csv              output is csv format [alias: --output=csv]
  --filter=<value>   filter property by partial string matching, ex: name=foo
  --hidden           show hidden commands
  --no-header        hide table header from output
  --no-truncate      do not truncate output to fit screen
  --output=<option>  output in a more machine friendly format
                     <options: csv|json|yaml>
  --sort=<value>     property to sort by (prepend '-' for descending)
  --tree             show tree of commands

GLOBAL FLAGS
  --json  Format output as json.

DESCRIPTION
  list all the commands

See code: @oclif/plugin-commands

`node-modules-packer help [COMMAND]`

Display help for node-modules-packer.

USAGE
  $ node-modules-packer help [COMMAND] [-n]

ARGUMENTS
  COMMAND  Command to show help for.

FLAGS
  -n, --nested-commands  Include all nested commands in the output.

DESCRIPTION
  Display help for node-modules-packer.

See code: @oclif/plugin-help

`node-modules-packer plugins`

List installed plugins.

USAGE
  $ node-modules-packer plugins [--core]

FLAGS
  --core  Show core plugins.

DESCRIPTION
  List installed plugins.

EXAMPLES
  $ node-modules-packer plugins

See code: @oclif/plugin-plugins

`node-modules-packer plugins:install PLUGIN...`

Installs a plugin into the CLI.

USAGE
  $ node-modules-packer plugins:install PLUGIN...

ARGUMENTS
  PLUGIN  Plugin to install.

FLAGS
  -f, --force    Run yarn install with force flag.
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Installs a plugin into the CLI.

  Can be installed from npm or a git url.

  Installation of a user-installed plugin will override a core plugin.

  e.g. If you have a core plugin that has a 'hello' command, installing a user-installed plugin with a 'hello' command
  will override the core plugin implementation. This is useful if a user needs to update core plugin functionality in
  the CLI without the need to patch and update the whole CLI.

ALIASES
  $ node-modules-packer plugins:add

EXAMPLES
  $ node-modules-packer plugins:install myplugin 

  $ node-modules-packer plugins:install https://github.com/someuser/someplugin

  $ node-modules-packer plugins:install someuser/someplugin

`node-modules-packer plugins:inspect PLUGIN...`

Displays installation properties of a plugin.

USAGE
  $ node-modules-packer plugins:inspect PLUGIN...

ARGUMENTS
  PLUGIN  [default: .] Plugin to inspect.

FLAGS
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Displays installation properties of a plugin.

EXAMPLES
  $ node-modules-packer plugins:inspect myplugin

See code: @oclif/plugin-plugins

`node-modules-packer plugins:install PLUGIN...`

Installs a plugin into the CLI.

USAGE
  $ node-modules-packer plugins:install PLUGIN...

ARGUMENTS
  PLUGIN  Plugin to install.

FLAGS
  -f, --force    Run yarn install with force flag.
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Installs a plugin into the CLI.

  Can be installed from npm or a git url.

  Installation of a user-installed plugin will override a core plugin.

  e.g. If you have a core plugin that has a 'hello' command, installing a user-installed plugin with a 'hello' command
  will override the core plugin implementation. This is useful if a user needs to update core plugin functionality in
  the CLI without the need to patch and update the whole CLI.

ALIASES
  $ node-modules-packer plugins:add

EXAMPLES
  $ node-modules-packer plugins:install myplugin 

  $ node-modules-packer plugins:install https://github.com/someuser/someplugin

  $ node-modules-packer plugins:install someuser/someplugin

See code: @oclif/plugin-plugins

`node-modules-packer plugins:link PLUGIN`

Links a plugin into the CLI for development.

USAGE
  $ node-modules-packer plugins:link PLUGIN

ARGUMENTS
  PATH  [default: .] path to plugin

FLAGS
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Links a plugin into the CLI for development.

  Installation of a linked plugin will override a user-installed or core plugin.

  e.g. If you have a user-installed or core plugin that has a 'hello' command, installing a linked plugin with a 'hello'
  command will override the user-installed or core plugin implementation. This is useful for development work.

EXAMPLES
  $ node-modules-packer plugins:link myplugin

See code: @oclif/plugin-plugins

`node-modules-packer plugins:uninstall PLUGIN...`

Removes a plugin from the CLI.

USAGE
  $ node-modules-packer plugins:uninstall PLUGIN...

ARGUMENTS
  PLUGIN  plugin to uninstall

FLAGS
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Removes a plugin from the CLI.

ALIASES
  $ node-modules-packer plugins:unlink
  $ node-modules-packer plugins:remove

`node-modules-packer plugins:uninstall PLUGIN...`

Removes a plugin from the CLI.

USAGE
  $ node-modules-packer plugins:uninstall PLUGIN...

ARGUMENTS
  PLUGIN  plugin to uninstall

FLAGS
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Removes a plugin from the CLI.

ALIASES
  $ node-modules-packer plugins:unlink
  $ node-modules-packer plugins:remove

See code: @oclif/plugin-plugins

`node-modules-packer plugins:uninstall PLUGIN...`

Removes a plugin from the CLI.

USAGE
  $ node-modules-packer plugins:uninstall PLUGIN...

ARGUMENTS
  PLUGIN  plugin to uninstall

FLAGS
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Removes a plugin from the CLI.

ALIASES
  $ node-modules-packer plugins:unlink
  $ node-modules-packer plugins:remove

`node-modules-packer plugins:update`

Update installed plugins.

USAGE
  $ node-modules-packer plugins:update [-h] [-v]

FLAGS
  -h, --help     Show CLI help.
  -v, --verbose

DESCRIPTION
  Update installed plugins.

See code: @oclif/plugin-plugins

`node-modules-packer run [DIR]`

Pack files and node dependencies to zip file.

USAGE
  $ node-modules-packer run [DIR] [--json] [-i <value>] [-e <value>] [--disable-default-ignore-file-ext]
    [--include-node-path <value>] [--ignore-node-path <value>] [--prod] [--peer] [--dev] [--optional] [--output-path
    <value>] [--output-file <value>] [-q]

ARGUMENTS
  DIR  [default: ./] Project root directory

FLAGS
  -e, --ignore-file-ext=<value>...        Force ignore specific file extension.
  -i, --include=package.json...           Include more files during packing (eg: -i dist).
  -q, --quiet                             Run without logging.
  --[no-]dev                              Include development dependencies when pack node dependencies.
  --[no-]disable-default-ignore-file-ext  Disable including default ignored extensions that we consider as useless.
  --ignore-node-path=typeorm/browser...   Force exclude folders starting with specified path (eg: -n "typeorm/browser"
                                          will exclude node_modules/typeorm/browser).
  --include-node-path=dev-dependency...   Force include folders starting with the specified path (eg --include-node-path
                                          "dev-dependency" will include node_modules/dev-dependency), but you need to
                                          MANUALLY add your sub-dependencies if dev-dependency has production
                                          dependencies.
  --[no-]optional                         Include optional dependencies when pack node dependencies.
  --output-file=<value>                   [default: deploy.zip] Specify output file name for the zip file.
  --output-path=<value>                   [default: ./] Specify output path for the zip file.
  --[no-]peer                             Include peer dependencies when pack node dependencies.
  --[no-]prod                             Include production dependencies when pack node dependencies.

GLOBAL FLAGS
  --json  Format output as json.

DESCRIPTION
  Pack files and node dependencies to zip file.

EXAMPLES
  $ node-modules-packer run /project/path -i dist

See code: src/commands/run/index.ts

`node-modules-packer version`

USAGE
  $ node-modules-packer version [--json] [--verbose]

FLAGS
  --verbose  Show additional information about the CLI.

GLOBAL FLAGS
  --json  Format output as json.

FLAG DESCRIPTIONS
  --verbose  Show additional information about the CLI.

    Additionally shows the architecture, node version, operating system, and versions of plugins that the CLI is using.

See code: @oclif/plugin-version

See code: src/commands/run/index.ts

Comments

feat(npm-v9): bump version of dependency extractor
Description of change

Bump depency-extrator to be able to process npm v9 lock file.

Pull-Request Checklist

[x] Code is up-to-date with the main branch

[x] npm run lint passes with this change

[x] npm run test passes with this change

[x] This pull request links relevant issues as Fixes #0000

[x] There are new or updated unit tests validating the change

[ ] Added documentation inside www/docs/main folder.

[ ] Included new files inside index.doc.ts.

[x] The new commits follow conventions outlined in the conventional commit spec

enhancement released
opened by H4ad 2
fix(graceful-fs): added to solve issues with too many open files
Description of change

closes #7

Pull-Request Checklist

[x] Code is up-to-date with the main branch

[x] npm run lint passes with this change

[x] npm run test passes with this change

[x] This pull request links relevant issues as Fixes #0000

[x] There are new or updated unit tests validating the change

[x] Added documentation inside www/docs/main folder.

[x] Included new files inside index.doc.ts.

[x] The new commits follow conventions outlined in the conventional commit spec

bug released
opened by H4ad 2
fix: windows paths
Description of change

Fixed issue with paths in Windows.

fix #6

Pull-Request Checklist

[x] Code is up-to-date with the main branch

[x] npm run lint passes with this change

[x] npm run test passes with this change

[x] This pull request links relevant issues as Fixes #0000

[x] There are new or updated unit tests validating the change

[x] Added documentation inside www/docs/main folder.

[x] Included new files inside index.doc.ts.

[x] The new commits follow conventions outlined in the conventional commit spec

bug released
opened by H4ad 2
feat: minify code
Description of change

This PR started with me trying to add support for uglify-js, then I switch for terser and in the final I changed to esbuild because is way way faster than the other options.

This PR will add support to flag --minify and --minify-keep-names, which integrate with esbuild to minify all .js files including .mjs and .cjs.

This PR fix #2

Pull-Request Checklist

[x] Code is up-to-date with the main branch

[x] npm run lint passes with this change

[x] npm run test passes with this change

[x] This pull request links relevant issues as Fixes #0000

[x] There are new or updated unit tests validating the change

[x] Added documentation inside www/docs/main folder.

[x] Included new files inside index.doc.ts.

[x] The new commits follow conventions outlined in the conventional commit spec

documentation released
opened by H4ad 2
feat: path mapping
Description of change

This PR closes #1, and also include a fix for bin/run file permissions which had a bug to not execute when install -g flag.

Pull-Request Checklist

[x] Code is up-to-date with the main branch

[x] npm run lint passes with this change

[x] npm run test passes with this change

[x] This pull request links relevant issues as Fixes #0000

[x] There are new or updated unit tests validating the change

[x] Added documentation inside www/docs/main folder.

[x] Included new files inside index.doc.ts.

[x] The new commits follow conventions outlined in the conventional commit spec

enhancement released
opened by H4ad 2
Add support for graceful-fs
Feature Request

Is your feature request related to a problem? Please describe. Add support for https://www.npmjs.com/package/graceful-fs to solve issues with TOO_MANY_OPEN_FILES.

Describe the solution you'd like

Install and mockey-patch the fs lib.

Are you willing to resolve this issue by submitting a Pull Request?

[x] Yes, I have the time, and I know how to start.

[ ] Yes, I have the time, but I don't know how to start. I would need guidance.

[ ] No, I don't have the time, although I believe I could do it if I had the time...

[ ] No, I don't have the time and I wouldn't even know how to start.

enhancement released
opened by H4ad 1
Zip on Windows
Current Behavior

When running this package on Windows, some packages like @h4ad/serverless-adapter are missing from final bundle.

Related: https://github.com/H4ad/serverless-adapter-examples/issues/2

Expected Behavior

The packages are included.

Steps to Reproduce the Problem

Clone: https://github.com/H4ad/serverless-adapter-examples/tree/master/examples/trpc-digital-ocean

Run zip command on Windows

See inside packages/api/prod/deploy.zip, the dependency @h4ad/serverless-adapter is missing on node_modules.

Environment

Version: 1.1.0

Platform: Win10

Node.js Version: 14.20.0

bug released
opened by H4ad 1
[Feature Request] Add support for uglify-js
Feature request

Is your feature request related to an issue? Please describe.

Sometimes we see inside node_modules libraries built with comments and lots of things not optimized. One way to reduce package size is to process these files to remove comments and make other optimizations.

Describe the desired solution

Something to pipe to transform code, like pipe code to be transformed by uglify-js.

Describe the alternatives you considered

Can I wrap uglify-js or do something else so I can expose a way to use integration with your own solution to transform code.

Are you willing to solve this problem by sending a Pull Request?

[x] Yes, I have time and I know how to start.

[ ] Yes, I have time, but I don't know how to start. I would need guidance.

[ ] No, I don't have time, although I believe I could do it if I had the time...

[ ] No, I don't have the time and I wouldn't even know how to start.

enhancement released
opened by H4ad 1
[Feature Request] Support for path mapping
Feature request

Is your feature request related to an issue? Please describe.

I want to be able to map a file from: ex /asset/image.png to /image.png.

Describe the desired solution

We can include another type of include like: string | { path: string, mappedPath: string }.

For CLI, we can do like: -i asset/image.png:image.png.

Are you willing to solve this problem by sending a Pull Request?

[x] Yes, I have time and I know how to start.

[ ] Yes, I have time, but I don't know how to start. I would need guidance.

[ ] No, I don't have time, although I believe I could do it if I had the time...

[ ] No, I don't have the time and I wouldn't even know how to start.

enhancement released
opened by H4ad 1

Releases(v1.3.0)

v1.3.0(Dec 4, 2022)
1.3.0 (2022-12-04)

Features

nodejs14: dropped support for nodejs 12.x (e9056f7)

npm-v9: bump version of dependency extractor (c42e7aa)

Source code(tar.gz)
Source code(zip)
v1.2.2(Oct 23, 2022)
1.2.2 (2022-10-23)

Bug Fixes

graceful-fs: added to solve issues with too many open files (58609ca)

Source code(tar.gz)
Source code(zip)
v1.2.1(Sep 11, 2022)
1.2.1 (2022-09-11)

Bug Fixes

index: issues with win32 paths (3ddcf0e)

Source code(tar.gz)
Source code(zip)
v1.2.0(Aug 14, 2022)
1.2.0 (2022-08-14)

Features

transformer: added initial draft for transformer uglify (c505b7e)

zip: added implementation with promises (4634c31)

Source code(tar.gz)
Source code(zip)
v1.1.0(Aug 14, 2022)
1.1.0 (2022-08-14)

Bug Fixes

bin: fixed problem with executable permission of dev and run (84dcb6b)

Features

run: added path remapping for --include flag (f199462)

Source code(tar.gz)
Source code(zip)
v1.0.2(Jul 28, 2022)

1.0.2 (2022-07-28)
Source code(tar.gz)
Source code(zip)
v1.0.1(Jul 28, 2022)

1.0.1 (2022-07-28)
Source code(tar.gz)
Source code(zip)
v1.0.0(Jul 28, 2022)
1.0.0 (2022-07-28)

Features

extensions: added more useless file extensions (0a3b5c7)

node-modules-packer: added initial version of the cli (185bed6)

Performance Improvements

archiver: removed archiver for yazl & fixed memory leak in tests (3dbc585)

zip: refactored to use another lib instead archiver (d4d06f5)

Source code(tar.gz)
Source code(zip)