My reason:

Installing larger packages (this one in this scenario) fails consistently from verdaccio with an unexpected EOF. The package is published to the local verdaccio server, not a redirection to the public registry. It installs fine from the public registry at https://registry.npmjs.org.

Even a simple wget ends abruptly

HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  X-Powered-By: verdaccio/2.3.4
  Access-Control-Allow-Origin: *
  Content-Type: application/octet-stream
  Content-Length: 137211454
  X-Status-Cat: http://flic.kr/p/aVuVsF
  Date: Mon, 21 Aug 2017 11:43:35 GMT
  Connection: keep-alive
Length: 137211454 (131M) [application/octet-stream]
Saving to: ‘react-native-awesome-card-io-0.6.6.tgz.2’

react-native-awesome-card-io-0.6.6.tgz.2        65%[======================================================================>                                      ]  85.73M  13.6MB/s    in 6.1s    

2017-08-21 17:15:23 (14.1 MB/s) - Connection closed at byte 89897191. Retrying.

Steps to reproduce:

• Publish the package to Verdaccio
• Try installing it with yarn or npm

Issue is occurring when running the latest docker image, or on the host system with verdaccio installed through npm. Issue has persisted on both a Linux host and a macOS host.

It seems to occur on an erratic timeout basis (~7-10 seconds) because the download completes successfully if the download speed is high enough. The byte at which the connection closes is also not constant.

App Version:

2.3.4

Config file:

Default config file with only max_body_size increased to 300mb allow publishing packages of this size.

This is a big refactoring of the core, fetching dependencies, improve code, more tests and better stability. This is essential for the next release, will take some time but would allow modularize more the core.

Notes

• Remove deprecated request by other got, retry improved, custom Agent ( got does not include it built-in)
• Remove async dependency from storage (used by core) it was linked with proxy somehow safe to remove now
• Refactor with promises instead callback wherever is possible
• ~Document the API~
• Improve testing, integration tests
• Bugfix
• Clean up old validations
• Improve performance
• Several left overts, naming, skipped test will be addressed in following PRs

💥 Breaking changes

• Plugin API methods were callbacks based are returning promises, this will break current storage plugins, check documentation for upgrade.
• Write Tarball, Read Tarball methods parameters change, a new set of options like AbortController signals are being provided to the addAbortSignal can be internally used with Streams when a request is aborted. eg: addAbortSignal(signal, fs.createReadStream(pathName));
• @verdaccio/streams stream abort support is legacy is being deprecated and soon will be removed
• Remove AWS and Google Cloud packages for future refactoring future https://github.com/verdaccio/verdaccio/pull/2574

Fix #1807 and #1225

I could not fix all possible regresions on this PR, I'll continue with small ones :-) otherwise this get for ever.

Describe the bug Verdaccio keeps only package.json in local storage and not package tarballs. Verdaccio is run on Windows Server 2019.

To Reproduce

1. Install verdaccio on Windows Server 2019
2. Apply provided configuration file
3. Run verdaccio
4. Use client to download any npm package passing verdaccio as registry
5. Package tarballs are not being cached

Expected behavior Verdaccio should cache package tarballs on local storage.

Configuration File (cat ~/.config/verdaccio/config.yaml)

config.yaml:

# This is the default config file. It allows all users to do anything,
# so don't use it on production systems.
#
# Look here for more config file examples:
# https://github.com/verdaccio/verdaccio/tree/master/conf
#

# path to a directory with all packages
storage: C:\node_packages
# path to a directory with plugins to include
plugins: ./plugins

web:
  title: Verdaccio
  # comment out to disable gravatar support
  # gravatar: false
  # by default packages are ordercer ascendant (asc|desc)
  # sort_packages: asc

auth:
  htpasswd:
    file: ./htpasswd
    # Maximum amount of users allowed to register, defaults to "+inf".
    # You can set this to -1 to disable registration.
    # max_users: 1000

# a list of other known repositories we can talk to
uplinks:
  npmjs:
    url: https://registry.npmjs.org/
    cache: true

packages:
  '@*/*':
    # scoped packages
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: npmjs

  '**':
    # allow all users (including non-authenticated users) to read and
    # publish all packages
    #
    # you can specify usernames/groupnames (depending on your auth plugin)
    # and three keywords: "$all", "$anonymous", "$authenticated"
    access: $all

    # allow all known users to publish/publish packages
    # (anyone can register by default, remember?)
    publish: $all

    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: npmjs

# You can specify HTTP/1.1 server keep alive timeout in seconds for incoming connections.
# A value of 0 makes the http server behave similarly to Node.js versions prior to 8.0.0, which did not have a keep-alive timeout.
# WORKAROUND: Through given configuration you can workaround following issue https://github.com/verdaccio/verdaccio/issues/301. Set to 0 in case 60 is not enough.
server:
  keepAliveTimeout: 60

middlewares:
  audit:
    enabled: true

# log settings
logs:
  - { type: stdout, format: pretty, level: debug }
  #- {type: file, path: verdaccio.log, level: info}
#experiments:
#  # support for npm token command
#  token: false

listen: 0.0.0.0:4873

Environment information

C:\node_packages>verdaccio --info

Environment Info:

  System:
    OS: Windows 10 10.0.17763
    CPU: (4) x64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
  Binaries:
    Node: 12.14.0 - C:\Program Files\nodejs\node.EXE
    npm: 6.13.4 - C:\Program Files\nodejs\npm.CMD`

C:\node_packages>verdaccio --version
4.4.0

Debugging output Local storage listing:

C:\node_packages>dir /s /b
C:\node_packages\.verdaccio-db.json
C:\node_packages\npm
C:\node_packages\underscore
C:\node_packages\npm\package.json
C:\node_packages\underscore\package.json

Verdaccio output:

 warn --- config file  - C:\Users\szymon.wilczek\AppData\Roaming\verdaccio\config.yaml
 warn --- Verdaccio started
 debug--- [local-storage/_sync]: init sync database
 debug--- [local-storage/_sync]: folder C:\node_packages created succeed
 debug--- [local-storage/_sync/writeFileSync]: sync write succeed
 debug--- [local-storage/_sync]: init sync database
 debug--- [local-storage/_sync]: folder C:\node_packages created succeed
 debug--- [local-storage/_sync/writeFileSync]: sync write succeed
 warn --- Plugin successfully loaded: verdaccio-htpasswd
 warn --- Plugin successfully loaded: verdaccio-audit
 warn --- http address - http://0.0.0.0:4873/ - verdaccio/4.4.0
 info <-- 10.80.57.156 requested 'GET /underscore'
 debug--- [local-storage/readPackage] read a package: underscore
 info --> making request: 'GET https://registry.npmjs.org/underscore'
 http --> 304, req: 'GET https://registry.npmjs.org/underscore' (streaming)
 http --> 304, req: 'GET https://registry.npmjs.org/underscore', bytes: 0/0
 debug--- [local-storage/readPackage] read a package: underscore
 debug-=- updating package underscore info
 debug--- [local-storage/savePackage] save a package: underscore
 http <-- 200, user: null(10.80.57.156), req: 'GET /underscore', bytes: 0/6705
 http <-- 200, user: null(10.80.57.156), req: 'GET /underscore', bytes: 0/6705
 info <-- 10.80.57.156 requested 'POST /-/npm/v1/security/audits/quick'
 http <-- 200, user: null(10.80.57.156), req: 'POST /-/npm/v1/security/audits/quick', bytes: 11747/151
 http <-- 200, user: null(10.80.57.156), req: 'POST /-/npm/v1/security/audits/quick', bytes: 11747/151

npm -ddd prints: (please note I removed the verdaccio server address from the logs and changed it to '*')

npm -ddd install underscore
npm info it worked if it ends with ok
npm verb cli [ 'C:\\Program Files (x86)\\Nodist\\v\\11.13.0\\node.exe',
npm verb cli   'C:\\Program Files (x86)\\Nodist\\npmv\\6.9.0\\bin\\npm-cli.js',
npm verb cli   '-ddd',
npm verb cli   'install',
npm verb cli   'underscore' ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session 185704a84643ebeb
npm sill install loadCurrentTree
npm sill install readLocalPackageData
npm http fetch GET 304 http://*:4873/underscore 233ms (from cache)
npm sill pacote tag manifest for underscore@latest fetched in 240ms
npm timing stage:loadCurrentTree Completed in 445ms
npm sill install loadIdealTree
npm sill install cloneCurrentTreeToIdealTree
npm timing stage:loadIdealTree:cloneCurrentTree Completed in 1ms
npm sill install loadShrinkwrap
npm timing stage:loadIdealTree:loadShrinkwrap Completed in 110ms
npm sill install loadAllDepsIntoIdealTree
npm sill resolveWithNewModule [email protected] checking installable status
npm sill removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 90ms
npm timing stage:loadIdealTree Completed in 244ms
npm sill currentTree temp
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree | `-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree +-- [email protected]
npm sill currentTree `-- [email protected]
npm sill idealTree temp
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree | `-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree +-- [email protected]
npm sill idealTree `-- [email protected]
npm sill install generateActionsToTake
npm timing stage:generateActionsToTake Completed in 13ms
npm sill diffTrees action count 1
npm sill diffTrees update [email protected]
npm sill decomposeActions action count 10
npm sill decomposeActions unbuild [email protected]
npm sill decomposeActions remove [email protected]
npm sill decomposeActions fetch [email protected]
npm sill decomposeActions extract [email protected]
npm sill decomposeActions preinstall [email protected]
npm sill decomposeActions build [email protected]
npm sill decomposeActions install [email protected]
npm sill decomposeActions postinstall [email protected]
npm sill decomposeActions finalize [email protected]
npm sill decomposeActions refresh-package-json [email protected]
npm sill install executeActions
npm sill doSerial global-install 10
npm verb correctMkdir C:\Users\szymon.wilczek\AppData\Roaming\npm-cache\_locks correctMkdir not in flight; initializing
npm verb lock using C:\Users\szymon.wilczek\AppData\Roaming\npm-cache\_locks\staging-7d368c8e4ed0da7d.lock for C:\temp\node_modules\.staging
npm sill doParallel extract 1
npm sill extract [email protected]
npm sill tarball trying underscore@latest by hash: sha512-5/4etnCkd9c8gwgowi5/om/mYO5ajCaOgdzj/oW+0eQV9WxKBDZw5+ycmKmeaTXjInS/W0BzpGLo2xR2aBwZdg==
npm sill extract underscore@latest extracted to C:\temp\node_modules\.staging\underscore-f5db5b84 (14ms)
npm timing action:extract Completed in 17ms
npm sill doReverseSerial unbuild 10
npm sill unbuild [email protected]
npm info lifecycle [email protected]~preuninstall: [email protected]
npm info lifecycle [email protected]~uninstall: [email protected]
npm verb unbuild rmStuff [email protected] from C:\temp\node_modules
npm info lifecycle [email protected]~postuninstall: [email protected]
npm timing action:unbuild Completed in 2ms
npm sill doSerial remove 10
npm sill remove C:\temp\node_modules\underscore
npm timing action:remove Completed in 3ms
npm sill doSerial move 10
npm sill doSerial finalize 10
npm sill finalize C:\temp\node_modules\underscore
npm timing action:finalize Completed in 1ms
npm sill doParallel refresh-package-json 1
npm sill refresh-package-json C:\temp\node_modules\underscore
npm timing action:refresh-package-json Completed in 6ms
npm sill doParallel preinstall 1
npm sill preinstall [email protected]
npm info lifecycle [email protected]~preinstall: [email protected]
npm timing action:preinstall Completed in 0ms
npm sill doSerial build 10
npm sill build [email protected]
npm info linkStuff [email protected]
npm sill linkStuff [email protected] has C:\temp\node_modules as its parent node_modules
npm timing action:build Completed in 0ms
npm sill doSerial global-link 10
npm sill doParallel update-linked 0
npm sill doSerial install 10
npm sill install [email protected]
npm info lifecycle [email protected]~install: [email protected]
npm timing action:install Completed in 0ms
npm sill doSerial postinstall 10
npm sill postinstall [email protected]
npm info lifecycle [email protected]~postinstall: [email protected]
npm timing action:postinstall Completed in 0ms
npm verb unlock done using C:\Users\szymon.wilczek\AppData\Roaming\npm-cache\_locks\staging-7d368c8e4ed0da7d.lock for C:\temp\node_modules\.staging
npm timing stage:executeActions Completed in 44ms
npm timing stage:rollbackFailedOptional Completed in 1ms
npm timing stage:runTopLevelLifecycles Completed in 790ms
npm sill saveTree temp
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree |   `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree |   `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   `-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree |   `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree |   `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | |   `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | +-- [email protected]
npm sill saveTree | | | | `-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   +-- [email protected]
npm sill saveTree | |   `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | | `-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | |   `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | |   `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | | `-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree |   `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree | `-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree +-- [email protected]
npm sill saveTree `-- [email protected]
npm sill install saveToDependencies
npm WARN saveError ENOENT: no such file or directory, open 'C:\temp\package.json'
npm info lifecycle undefined~preshrinkwrap: undefined
npm info lifecycle undefined~shrinkwrap: undefined
npm verb shrinkwrap skipping write for package-lock.json because there were no changes.
npm info lifecycle undefined~postshrinkwrap: undefined
npm WARN enoent ENOENT: no such file or directory, open 'C:\temp\package.json'
npm verb enoent This is related to npm not being able to find a file.
npm verb enoent 
npm WARN temp No description
npm WARN temp No repository field.
npm WARN temp No README data
npm WARN temp No license field.

npm sill install printInstalled
npm timing audit submit Completed in 565ms
npm http fetch POST 200 http://*:4873/-/npm/v1/security/audits/quick 639ms
npm timing audit body Completed in 75ms
+ [email protected]
updated 1 package and audited 2748 packages in 1.389s
found 0 vulnerabilities

npm verb exit [ 0, true ]
npm timing npm Completed in 1696ms
npm info ok 

My reason:

I setup a verdaccio 2.4 server yesterday and found many projects failed to build after reinstall dependencies.After several hours digging,i found verdaccio returns wrong package.json file which remove some info from original package.json, e.g, "main" field. So it cause nodejs to throw many "can not find module' errors. Finally i revert to verdaccio 2.2, and all works fine.

Steps to reproduce:

App Version: 2.4

If I want to run verdaccio on my unraid server, I can't map the folders /config/ to a host folder. If I want to map them, the container won't start. There are no logs.

The same occurs when I want to start the start the container on my Windows 10 system with WSL2 by following command:

docker run -it --rm --name verdaccio -p 4873:4873 -v C:\temp\verdaccio\conf:/verdaccio/conf -v C:\temp\verdaccio\storage:/verdaccio/storage -v C:\temp\verdaccio\plugins:/verdaccio/plugins  verdaccio/verdaccio

If I run the container without the port mapping, it starts flawlessly by this command:

docker run -it --rm --name verdaccio -p 4873:4873  verdaccio/verdaccio

The container should start with mapped /config/ folder. Unfortunatly there is currently nothing more I could share with you.

I guess there is something strange with the permissions itself. IMHO there are missing permission mappings for the /config at this line:

https://github.com/verdaccio/verdaccio/blob/b02dab1c2f8fc9c2e7cafe6510535ff491a7b1aa/Dockerfile#L44

Unfortunatly I'm not that docker expert to debug this.

Log-in Bug Hi Guys,

Maybe you can assist me regarding the following issue. I tried to find the solution but found nothing. Please review.

Verdaccio - 3.5.1.

1. When the existing user is trying to log in via npm, it gets an error "username is already registered".

2. If I disable registration with option "max_users: -1" in config.yaml, then I get an error "maximum amount of users reached".

Terminal output

$ npm login --registry=https://registry-npm.example.com
Username: test.user
Password:
Email: (this IS public) [email protected]
npm ERR! code E409
npm ERR! Registry returned 409 for PUT on https://registry-npm.example.com/-/user/org.couchdb.user:test.user: username is already registered

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/test.user/.npm/_logs/2018-08-23T12_45_12_403Z-debug.log

$ npm login --registry=https://registry-npm.example.com
Username: test.user
Password:
Email: (this IS public)
Email: (this IS public) [email protected]
npm ERR! code E403
npm ERR! Registry returned 403 for PUT on https://registry-npm.example.com/-/user/org.couchdb.user:test.user: maximum amount of users reached

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/test.user/.npm/_logs/2018-08-23T12_43_19_284Z-debug.log

Expected behavior

Log-in to Verdaccio server with a registered user.

Configuration File

web:
  # WebUI is enabled as default, if you want to disable it, just uncomment this line
  enable: true
  title: Verdaccio NPM
  logo: logo.png

auth:
  htpasswd:
    file: ./htpasswd
    # Maximum amount of users allowed to register, defaults to "+inf".
    # You can set this to -1 to disable registration.
    max_users: -1

# a list of other known repositories we can talk to
uplinks:
  npmjs:
    url: https://registry.npmjs.org/

packages:
  '@*/*':
    # scoped packages
    access: $authenticated
    publish: $authenticated
    proxy: npmjs

  '**':
    # allow all users (including non-authenticated users) to read and
    # publish all packages
    #
    # you can specify usernames/groupnames (depending on your auth plugin)
    # and three keywords: "$all", "$anonymous", "$authenticated"
    access: $authenticated

    # allow all known users to publish packages
    # (anyone can register by default, remember?)
    publish: $authenticated

    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: npmjs

# To use `npm audit` uncomment the following section
middlewares:
  audit:
    enabled: true

# log settings
logs:
  - {type: stdout, format: pretty, level: http}
  #- {type: file, path: verdaccio.log, level: info}

Debugging output

$ npm login --registry=https://registry-npm.example.com -ddd
npm info it worked if it ends with ok
npm verb cli [ '/usr/local/Cellar/node/10.9.0/bin/node',
npm verb cli   '/usr/local/bin/npm',
npm verb cli   'login',
npm verb cli   '--registry=https://registry-npm.example.com',
npm verb cli   '-ddd' ]
npm info using [email protected]
npm info using [email protected]
npm verb config Skipping project config: /Users/test.user/.npmrc. (matches userconfig)
npm verb npm-session 821b0c437be02ea2
npm verb web login before first POST
npm http request → POST https://registry-npm.example.com/-/v1/login
npm http 404 ← Not Found (https://registry-npm.example.com/-/v1/login)
npm verb web login not supported, trying couch
Username: test.user
Password:
Email: (this IS public) [email protected]
npm verb login before first PUT { _id: 'org.couchdb.user:test.user',
npm verb login   name: 'test.user',
npm verb login   password: 'XXXXX',
npm verb login   type: 'user',
npm verb login   roles: [],
npm verb login   date: '2018-08-23T13:02:26.562Z' }
npm http request → PUT https://registry-npm.example.com/-/user/org.couchdb.user:test.user
npm http 409 ← Conflict (https://registry-npm.example.com/-/user/org.couchdb.user:test.user)
npm http request → GET https://registry-npm.example.com/-/user/org.couchdb.user:test.user?write=true
npm http 200 ← OK (https://registry-npm.example.com/-/user/org.couchdb.user:test.user?write=true)
npm http request → PUT https://registry-npm.example.com/-/user/org.couchdb.user:test.user/-rev/undefined
npm http 409 ← Conflict (https://registry-npm.example.com/-/user/org.couchdb.user:test.user/-rev/undefined)
npm verb adduser before first PUT { _id: 'org.couchdb.user:test.user',
npm verb adduser   name: 'test.user',
npm verb adduser   password: 'XXXXX',
npm verb adduser   email: '',
npm verb adduser   type: 'user',
npm verb adduser   roles: [],
npm verb adduser   date: '2018-08-23T13:02:26.622Z' }
npm http request → PUT https://registry-npm.example.com/-/user/org.couchdb.user:test.user
npm http 409 ← Conflict (https://registry-npm.example.com/-/user/org.couchdb.user:test.user)
npm verb stack Error: Registry returned 409 for PUT on https://registry-npm.example.com/-/user/org.couchdb.user:test.user: username is already registered
npm verb stack     at fetch.defaults.catch.then.then.result (/usr/local/lib/node_modules/npm/node_modules/npm-profile/index.js:376:13)
npm verb stack     at process._tickCallback (internal/process/next_tick.js:68:7)
npm verb statusCode 409
npm verb cwd /Users/test.user
npm verb Darwin 17.6.0
npm verb argv "/usr/local/Cellar/node/10.9.0/bin/node" "/usr/local/bin/npm" "login" "--registry=https://registry-npm.example.com" "-ddd"
npm verb node v10.9.0
npm verb npm  v6.2.0
npm ERR! code E409
npm ERR! Registry returned 409 for PUT on https://registry-npm.example.com/-/user/org.couchdb.user:test.user: username is already registered
npm verb exit [ 1, true ]
npm timing npm Completed in 17112ms

Temporary solution

Only possible to log-in with the registered user is allowed in config.yaml access and publish to all, anonymous.

access: $all, $anonymous
publish: $all

But it's not secure.

1. The PR adds windows to the CI matrix
2. The PR fixes some tests that were failing on windows by using a pattern instead of hardcoded strings.
3. Some tiny snapshots got converted to inline snapshots (imho makes it easier to understand tests)
4. The lint job didn't run on GitHub's windows image due to line ending issues. I fixed this by adding a .gitattributes with lf as the default line ending for text
5. The PR sets one test on skip because it doesn't exit normally on windows, apparently due to a node.js streaming issue or something
6. Fixes website/gatsby-node.js on windows and simplifies it

Describe the bug

I tried to look at #1359 and #1405 but couldn't find a solution. I'm using ~~https://github.com/Remitly/verdaccio-s3-storage~~ https://registry.verdaccio.org/-/web/detail/verdaccio-aws-s3-storage as a storage plugin, but you can comment the whole storage key in configuration and it will be the same (except for one thing that is pointed out below).

npm publish --force

If I try to overwrite an existing package version with a new one:

npm publish --force --registry MY_REPO

The result is:

• old package is removed from the remote storage and the web UI doesn't show it
• the npm command fails with:

npm ERR! code EPUBLISHCONFLICT
npm ERR! publish fail Cannot publish over existing version.
npm ERR! publish fail Update the 'version' field in package.json and try again.
npm ERR! publish fail 
npm ERR! publish fail To automatically increment version numbers, see:
npm ERR! publish fail     npm help version

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/marco/.npm/_logs/2019-08-10T13_46_04_312Z-debug.log

• running the same exact command again publishes successfully

npm unpublish --force SPECIFIC_VERSION

If I try to unpublish the package first to avoid errors:

npm unpublish "@test/[email protected]" --force --registry MY_REPO

The result is:

• npm exits with no errors and says that the package has been removed
• the package still shows up on the AWS S3 storage and on the web UI
• trying to publish the same package from now on will result in failures (this doesn't happen when using local storage: publishing again works as with the command above)

The only way I found to recover from this situation was:

• removing the .tgz archive manually from the storage
• restart the verdaccio server

These commands don't work as expected.

To Reproduce

I'm using the following Dockerfile to speed up testing:

FROM verdaccio/verdaccio:latest

USER root

ENV NODE_ENV=production

# Install "envsubst" to replace ENV_VARS in config
RUN apk add gettext libintl

# Install AWS S3 Plugin
RUN yarn add verdaccio-aws-s3-storage --production=true --no-lockfile --registry https://registry.verdaccio.org

# Copy configuration and .htpasswd
COPY ./config.yml  $VERDACCIO_APPDIR/conf/config.yaml

# Fix permissions
RUN chown -R $VERDACCIO_USER_UID:root $VERDACCIO_APPDIR/conf/config.yaml \
    && chmod -R g=u $VERDACCIO_APPDIR/conf/config.yaml

USER $VERDACCIO_USER_UID

# Set user credentials, replace ENV_VARS in config and finally run Verdaccio
CMD echo "$HTPASSWD" > /verdaccio/storage/htpasswd \
    && envsubst \$AWS_S3_BUCKET,\$AWS_S3_PATH_PREFIX,\$AWS_DEFAULT_REGION,\$AWS_S3_ENDPOINT,\$AWS_S3_PATH_STYLE,\$AWS_ACCESS_KEY_ID,\$AWS_SECRET_ACCESS_KEY < $VERDACCIO_APPDIR/conf/config.yaml > $VERDACCIO_APPDIR/conf/config.yaml \
    && $VERDACCIO_APPDIR/bin/verdaccio --config $VERDACCIO_APPDIR/conf/config.yaml --listen $VERDACCIO_PROTOCOL://0.0.0.0:$VERDACCIO_PORT

This docker-compose.yml should run your registry:

version: '2.1'

services:
  verdaccio:
    image: testing/private-verdaccio-s3:latest
    environment:
      AWS_S3_BUCKET: 
      AWS_S3_PATH_PREFIX: 
      AWS_ACCESS_KEY_ID: 
      AWS_SECRET_ACCESS_KEY: 
      AWS_DEFAULT_REGION:
      AWS_S3_ENDPOINT: 
      AWS_S3_PATH_STYLE: "true"
      # User is demo:demo
      HTPASSWD: |
        demo:$apr1$uq2Uw53i$dS7GoGvNuupumWvXjOJ.R1
    ports:
      - "4874:4873"

Configuration file config.yml:

#
# This is the config file used for the docker images.
# It allows all users to do anything, so don't use it on production systems.
#
# Do not configure host and port under `listen` in this file
# as it will be ignored when using docker.
# see https://verdaccio.org/docs/en/docker#docker-and-custom-port-configuration
#
# Look here for more config file examples:
# https://github.com/verdaccio/verdaccio/tree/master/conf
#

# path to a directory with all packages
storage: /verdaccio/storage/data

# comment this section entirely to disable AWS S3
store:
  aws-s3-storage:
    bucket: $AWS_S3_BUCKET
    keyPrefix: $AWS_S3_PATH_PREFIX
    region: $AWS_DEFAULT_REGION
    endpoint: $AWS_S3_ENDPOINT
    s3ForcePathStyle: $AWS_S3_PATH_STYLE
    accessKeyId: $AWS_ACCESS_KEY_ID
    secretAccessKey: $AWS_SECRET_ACCESS_KEY

web:
  title: Verdaccio

auth:
  htpasswd:
    file: /verdaccio/storage/htpasswd
    max_users: -1

security:
  api:
    jwt:
      sign:
        expiresIn: 60d
        notBefore: 1
  web:
    sign:
      expiresIn: 7d
      notBefore: 1

# a list of other known repositories we can talk to
uplinks:
  npmjs:
    url: https://registry.npmjs.org/

packages:
  '**':
    access: $all
    publish: $all
    unpublish: $all

    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: npmjs

middlewares:
  audit:
    enabled: true

# log settings
logs:
  - { type: stdout, format: pretty, level: http }

listen:
  - 0.0.0.0:4873

Put these 3 files in a folder called verdaccio-test, then build and run everything:

cd /path/to/verdaccio-test
docker build -t testing/private-verdaccio-s3:latest .
docker-compose up -d

# Attach to logs
docker logs -f verdaccio-test_verdaccio_1

# use http://127.0.0.1:4874

Expected behavior

Packages should be deleted correctly.

Conclusion

I hope that everything is clear enough, please feel free to ask for further information.

Thank you for your time.

Currently verdaccio/sinopia has an internal cache of which packages exists. It does not reparse the .sinopia-db, therefore if you run sinopia in a cluster, there will be nodes that are out of sync and even more problematic, you may overwrite the .sinopia-db with different states.

Therefore we need to add the capability to either: Create a communication channel (EventBus, MQ, whatsoever) or just a central configuration, like a database.

We have to review the Dockerfile and improve the size of each layers. Some examples of things to improve are:

• Remove dumb-init
• Remove the addition and elimination of openssl package
• Remove test on Dockerfile

My reason:

https://circleci.com/blog/sunsetting-1-0/

Additional information:

I'll explain step by step what we are doing in a nutshell on circleci.ylm

• YARN_VERSION: 1.6.0
• verdacciobot user will perform all task
• Testing with node 6, 8, 9 and 10
• Default node should be 9
• yarn run pre:ci first step before running anything else
• - yarn run test:e2e should be run once using default Node
• Publish docusaurus after (master branch) is being successful (right now it publish even from branches and that is wrong)
• Deployment: This is really important
  • After a new tag is being detected we should deploy on npmjs /(v)?[0-9]+(\.[0-9]+)*/
  • l10n_master master branch should be ignored for builds (used by crowdin and it is nog being ignored currently).
  • gh-pages is being used by docusaurus and also should be ignored.

Upgrade

• We should keep the same configuration
• Fix what is not working properly
• Deployment is essential
• Wish list:
  • Use of artifacts ( to measure the size of the tarball in each PR)
• Any another improvement that improves the workflow and maintainability

Testing

• We should test in a separated repository (we cannot publish it accidentally on npmjs)

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

The problem is in 5.x/src/api/endpoint/api/v1/search.ts (I checked and the same logical issue exists also in master 6.x branch master/packages/api/src/v1/search.ts).

Basically, search returns always total equal or less than size query param.

For example, direct search of @babel packages on npmjs (https://registry.npmjs.org/-/v1/search?text=@babel&size=20&from=0) returns total of 192 and I can do pagination by using size and from params. Same call via verdaccio returns always total of 20 (equal to size) and pagination is not possible at all.

Here is the diff that solved my problem:

diff --git a/node_modules/verdaccio/build/api/endpoint/api/v1/search.js b/node_modules/verdaccio/build/api/endpoint/api/v1/search.js
index 27b03a0..8b1939e 100644
--- a/node_modules/verdaccio/build/api/endpoint/api/v1/search.js
+++ b/node_modules/verdaccio/build/api/endpoint/api/v1/search.js
@@ -106,11 +106,12 @@ async function sendResponse(resultBuf, resultStream, auth, req, from, size) {
   const checkAccessPromises = await Promise.all(removeDuplicates(resultsCollection).map(pkgItem => {
     return checkAccess(pkgItem, auth, req.remote_user);
   }));
-  const final = checkAccessPromises.filter(i => !_lodash.default.isNull(i)).slice(from, size);
+  const totalAllowed = checkAccessPromises.filter(i => !_lodash.default.isNull(i));
+  const final = totalAllowed.slice(from, from + size);
   _logger.logger.debug(`search results ${final === null || final === void 0 ? void 0 : final.length}`);
   const response = {
     objects: final,
-    total: final.length,
+    total: totalAllowed.length,
     time: new Date().toUTCString()
   };
   _logger.logger.debug(`total response ${final.length}`);

This issue body was partially generated by patch-package.

Discussed in https://github.com/verdaccio/verdaccio/discussions/3536

Version: 5.15.4

This happened when we are trying to change the version of a specific package. Our older package is already cached, and when verdaccio tries to get the tarball from the cache it gets 404 because of the version change.

The solution is to manually delete the folder of the package from the cache and try again. That means that Verdaccio can't support multiple versions in the cache. This is a big blocker.

Config:

storage: /verdaccio/storage/data
plugins: /verdaccio/plugins
web:
  enable: true
  title: Verdaccio
url_prefix: '/'
uplinks:
  npmjs:
    url: https://registry.npmjs.org/
    cache: true
    maxage: 259200m
  yarnpkg:
    url: https://registry.yarnpkg.com/
    cache: true
    maxage: 259200m
packages:
  '@*/*':
    access: $all
    publish: $authenticated
    proxy: yarnpkg npmjs
  '**':
    access: $all
    publish: $authenticated
    proxy: yarnpkg npmjs
middlewares:
  audit:
    enabled: true
logs:
  - {type: stdout, format: pretty, level: debug}

To support multiple version, do we have to enable this somehow in the configuration?

Your Environment

• verdaccio version: 5.18.0
• node version: from official docker image verdaccio/verdaccio:5.18.0
• package manager: npm, from official docker image verdaccio/verdaccio:5.18.0
• os: docker image verdaccio/verdaccio:5.18.0
• platform: kubernetes (helm)

Describe the bug

After installing chart into kubernetes cluster with preprovisioned aws-s3-storage plugin, application cannot list packages, but can reach them via direct URL.

To Reproduce

1. Configuration block from mostly base helm values.
2. Install into cluster with following values,
3. Upload packages into S3 Verdaccio/storage/ (backup from working old version 4.x.x). Couple GBs.
4. Proceed into UI.

Expected behavior Everything ok on old 4.x.x set up, but...

Screenshots, server logs, package manager log UI reachable, but 0 packages (changed host due security reasons):

Can reach a package (changed host and package name due security reasons):

Configuration files Part of very basic values:

 configMap: |
  # necessary (see https://github.com/verdaccio/verdaccio/issues/673)
  storage: /verdaccio/storage/data
  
  store:
    aws-s3-storage:
      keyPrefix: Verdaccio/storage/
      bucket: ${bucket}
      region: ${region}
      accessKeyId: ${access_key_id}
      secretAccessKey: ${secret_access_key}

  auth:
    htpasswd:
      file: /verdaccio/storage/htpasswd
      max_users: 4
      algorithm: bcrypt
      rounds: 10

  uplinks:
    npmjs:
      url: https://registry.npmjs.org/
      max_fails: 100

  packages:
    '@*/*':
      access: $all
      publish: $authenticated
      proxy: npmjs

    '**':
      access: $all
      publish: $authenticated
      proxy: npmjs

  middlewares:
    audit:
      enabled: true

  max_body_size: 100m

  server: 
    keepAliveTimeout: 60

  logs: {type: stdout, format: pretty, level: debug}

Dockerfile

FROM verdaccio/verdaccio:5.18.0

ENV NODE_ENV=production
USER root
RUN npm i -g [email protected]

Environment information EKS 1.22, Helm 3.8.2, Chart 4.10.3, [email protected], [email protected]

Debugging output

• $ NODE_DEBUG=request verdaccio display request calls (verdaccio <--> uplinks)
• [x] $ DEBUG=verdaccio* verdaccio enable extreme verdaccio debug mode (verdaccio api)
• $ npm -ddd prints:
• $ npm config get registry prints:

Plugin init starts (changed bucket name due security reasons):

verdaccio:plugin:loader loading plugin /usr/local/lib/node_modules/verdaccio/build/plugins/aws-s3-storage
verdaccio:plugin:loader plugin /usr/local/lib/node_modules/verdaccio/build/plugins/aws-s3-storage not found
verdaccio:plugin:loader loading plugin verdaccio-aws-s3-storage
verdaccio:plugin:loader plugin aws-s3-storage is an npm package
erdaccio:plugin:loader plugin is ES6
{"level":30,"time":1670257501552,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","content":"aws-s3-storage","prefix":"verdaccio","msg":"plugin successfully loaded: @{prefix}-@{content}"}
{"level":20,"time":1670257501552,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","keyPrefix":"Verdaccio/storage/","bucket":"MY_BUCKET","msg":"s3: [_getData] bucket: @{bucket} prefix: @{keyPrefix}"}
{"level":20,"time":1670257502155,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","keyPrefix":"Verdaccio/storage/","bucket":"MY_BUCKET","msg":"s3: [_sync] bucket: @{bucket} prefix: @{keyPrefix}"}
{"level":20,"time":1670257502285,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","msg":"s3: [_sync] sucess"}

Also found empty response from /-/verdaccio/data/packages:

verdaccio version: 5.16.0 and upper node version: v16.14.0 package manager: [[email protected], [email protected]] os: linux platform: docker

We had Verdaccio 5.15.4 with Minio storage (RELEASE.2022-10-24T18-35-07Z). config.yml contains the lines:

store:
  minio:
    port: 9000
    region: us-east-1
    endPoint: 10.10.10.10
    accessKey: minioadmin
    secretKey: minioadmin
    useSSL: false

It worked fine.

After upgrading to 5.18.0 we get errors on "yarn add @our/package":

yarn add v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://our-verdaccio-site.com/commander/-/commander-9.4.1.tgz: Request failed \"404 Not Found\""

If you try to get tarball file with the direct link - https://our-verdaccio-site.com/commander/-/commander-9.4.1.tgz - the same error appears: 404 "file not found". The file commander-9.4.1.tgz exists in Minio though.

The error appears only since v5.16.0 (I tried 5.16.0, 5.16.1, 5.16.2, 5.16.3, 5.17.0, 5.18.0)

Debug logs:

01.12.2022, 21:09:59 debug--- [Minio] ReadStream<commander/commander-9.4.1.tgz> is opened
01.12.2022, 21:09:59 debug--- [Minio] ReadStream<commander/commander-9.4.1.tgz> received 28174 bytes
01.12.2022, 21:09:59 debug--- [Minio] ReadStream<commander/commander-9.4.1.tgz> is finishing