Pipeable javascript. Quickly filter, map, and reduce from the terminal

Last update: Dec 10, 2022

Related tags

Overview

Pipeable JavaScript - another utility like sed/awk/wc... but with JS! Quickly filter, map and reduce from the command line. Features a streaming API. Inspired by pipeable ruby.

Overview

pjs is a cli tool that can accept input on stdin, or read from a list of files. Its filter, map and reduce options take expressions to be run, in that order, and applies them to the supplied input. The expressions themselves can contain identifiers used by keys in String.prototype, which will automatically be bound to the given line. This lets you save a bit of typing with your one-liners, while still giving you access to all your JS string functions! Check out some of the examples below to see how they translate.

# Return all lines longer than 5 chars
# => lines.filter(function(line) { return line.length > 5; });
ls -1 | pjs -f 'length > 5'

# Count characters in each line
# => lines.map(function(line) { return line.length; });
ls -1 | pjs -m 'length'

# Uppercase and pad each line
# => lines.map(function(line) { return '  ' + line.toUpperCase()"; });
ls -1 | pjs -m '"  " + toUpperCase()'

# Return lines longer than 5 chars, and remove any digits
# => lines
#      .filter(function(line) { return line.length > 5; })
#      .map(function(line) { return line.replace(/\d/g, ''); });
ls -1 | pjs -f 'length > 5' -m 'replace(/\d/g, "")'

The current line and value can also be accessed via the $ variable, and the tool supports json output.

(echo 'foo' && echo 'foobar') | pjs -jm '{name: $, length: length}'
[
{"name":"foo","length":3},
{"name":"foobar","length":6}
]

pjs also includes lodash functions, which can be accessed via the _ object, and chained using $$

echo 'hello' | pjs -m '_.upperFirst($)'
# Hello

echo 'please-titleize-this-sentence' | \
pjs -m '$$.lowerCase().split(" ").map(_.upperFirst).join(" ")'
# Please Titleize This Sentence

as well as Ramda and point-free style

echo 'please-titleize-this-sentence' | \
pjs -m "R.compose(R.replace(/(^|\s)\w/g, R.toUpper), R.replace(/-/g, ' '))"
# Please Titleize This Sentence

Installation

It can be installed via npm using:

npm install -g pipeable-js

Usage

Usage: pjs [options] [files ...]

Functions and expressions are invoked in the following order:
filter, map, reduce

All functions are passed the line ($) and index (i)
Built-in reduce functions: length, min, max, sum, avg, concat
Custom reduce expressions accept: prev, curr, i, array
Includes lodash (_), and can be chained using $$
Supports Ramda (R) and point-free style

Options:

  -h, --help               output usage information
  -V, --version            output the version number
  -i, --ignore             ignore empty lines
  -j, --json               output as json
  -f, --filter <exp>       filter by a boolean expression
  -m, --map <exp>          map values using the expression
  -r, --reduce <func|exp>  reduce using a function or expression

Examples

filter

# Print all odd lines
# awk 'NR % 2 == 1' file
pjs -f 'i % 2 == 0' file

# Print all lines greater than 80 chars in length
# awk 'length($0) > 80' file
pjs -f 'length > 80' file

map

# Remove all digits
# tr -d 0-9 < file
pjs -m "replace(/\d/g, '')" file

# Get second item of each line in csv
# awk -F "," '{print $2}' file
pjs -m 'split(",")[1]' file

reduce

# Count lines in file
# wc -l file
# awk 'END { print NR }' file
pjs -r length file

# Sum all decimal numbers in a file
# awk '{ sum += $1 } END { print sum }' file
# perl -nle '$sum += $_ } END { print $sum' file
pjs -r 'Number(prev) + Number(curr)' file
pjs -r '(+prev) + (+curr)' file
pjs -r sum file

# Concatenate all lines in multiple files
# awk '{printf $0;}' file1 file2
# cat file1 file2 | tr -d '\n'
pjs -r concat file1 file2

mixed

# Print the length of the longest line
# awk '{ if (length($0) > max) max = length($0) } END { print max }' file
pjs -m 'length' -r max file

Comparison

Features	pjs	pythonpy	pru
Language	JavaScript	Python	Ruby
Streaming	Yes	Limited ^[1]	Yes
Implementation	Streams	Iterables	Generators
Easy JSON output	Yes	No	No
Webscale^TM	YES	No	No

_{[1] Can't perform "tail -f logfile | py -x x"}

Comments

templates

@danielstjules, what do you think of a template feature? Something where you can define CommonJS-style modules for repeating filter/map/reduce tasks, like:

~/.config/pjs/templates/debug-log.js

module.exports = function template (pjs) {
    return pjs
        .filter(function (item) {
            return !!~item.indexOf('DEBUG');
        })
        .map(function (item) {
            return "Found debug entry: " + item;
        });

        // omitting "reduce" here ...
};

could be used in pjs as:

tail -f ~/logs/*.log | pjs -t debug-log

opened by akoenig 7

Including support for Lodash with $$ wrapper object in map transform
This pull request includes:

support for lodash functions with the _ object

wrapper object with the $$ object

tests for the new functionality

documentation
opened by thalesmello 5
Access the index from map/filter/...

Ok, first thing first pjs is AWESOME.

pjs should allow us to access to the second argument from .map/.filter/... Maybe it could be something like $1 or i (since curr and prev are already available in -r reduce).

opened by FGRibreau 4
Include support for Lodash?
Hello.

I was wondering if it would be possible to include support for lodash. It includes all sorts of useful functions, which I think work great with your command line tool.

I've create a reference implementation, in which I just required the library.

With it, it's possible to camelize words. For example

echo "foo_bar\nbee-bop" | pjs -m '_.camelCase($)'
opened by thalesmello 3

Point-free support through ramda

Ramda takes a functional approach, which allows tacit programming and very robust data-less scripts.

$ echo "Returns the elements of the given \
list or string (or object with a slice method) \
from fromIndex (inclusive) to toIndex (exclusive)." | \
pjs -m 'R.slice(0, 48)'
# Returns the elements of the given list or string

echo 'please-titleize-this-sentence' | \
pjs -m "R.compose(R.replace(/(^|\s)\w/g, R.toUpper), R.replace(/-/g, ' '))"
# Please Titleize This Sentence

And since R.compose only cares about unary functions, we can use any custom transform on the stack

echo 'please-titleize-this-sentence' | \
pjs -m "R.compose(encodeURIComponent, R.replace(/(^|\s)\w/g, R.toUpper), R.replace(/-/g, ' '))"
# Please%20Titleize%20This%20Sentence

opened by davidmh 1

Clarify access of line variable from the help menu

I couldn't figure out how to reference the line rather than the index from the help documentation.

Thought this might help make things clearer maybe.

Love this tool btw :-)

opened by halhenke 1
Remove leading ./ from bin path

Removing the leading ./ from the pjs bin path allows npm to copy the script to my bin directory when using fish shell and fish node manager (fnm). Otherwise, no script was installed in bin.

opened by jboxman 0

Releases(0.10.0)

0.10.0(Mar 1, 2017)
Add Ramda support

Source code(tar.gz)
Source code(zip)
0.9.0(Mar 24, 2016)
Add built-in lodash support

Source code(tar.gz)
Source code(zip)
0.8.0(Mar 5, 2015)
Improved binding support using with

Removed the --explicit option as it's no longer necessary

Improved object literal handling in https://github.com/danielstjules/pjs/commit/4af10a5c9e1ea425535efdd4e5adf98ca04b271b

Source code(tar.gz)
Source code(zip)
0.7.0(Mar 5, 2015)
Added support for object literals being passed to map

Fixed handling of input with leading spaces

Source code(tar.gz)
Source code(zip)
0.6.0(Oct 15, 2014)
Can now handle null values in streams

Source code(tar.gz)
Source code(zip)
0.5.1(Aug 17, 2014)
Additional tests

Improved stream interface: only reduce needs to be piped with {end: false}

Source code(tar.gz)
Source code(zip)
0.5.0(Aug 16, 2014)
The line's index is now passed to each filter and map function via the variable i

Source code(tar.gz)
Source code(zip)
0.4.1(Aug 15, 2014)
Fix missing dependency in previous release

Source code(tar.gz)
Source code(zip)
0.4.0(Aug 15, 2014)
Rewrote module to use a streaming API. It uses streams for all actions except reduce

Source code(tar.gz)
Source code(zip)
0.3.1(Aug 11, 2014)
Added handling of syntax/reference errors in expression

Source code(tar.gz)
Source code(zip)
0.3.0(Aug 11, 2014)
Added json flag

Source code(tar.gz)
Source code(zip)
0.2.0(Aug 10, 2014)
Added length reduce function

Reduce now takes an expression, and evaluates it with prev, curr, i and array variables

Source code(tar.gz)
Source code(zip)
0.1.0(Aug 10, 2014)
Fixed getBoundExpression regex

Added min and max reduce functions

Source code(tar.gz)
Source code(zip)
0.0.1(Aug 10, 2014)
First unstable release

Source code(tar.gz)
Source code(zip)

Owner

Daniel St. Jules

GitHub

Uses marked-terminal to render a README.md for any npm module in the terminal.

modhelp Uses marked-terminal to render a README.md for any npm module in the terminal. Now with built-in pager! Page up/down, arrow keys to scroll lin

23 Feb 8, 2022

Terminal ui for discord with interactive terminal

dickord why No fucking clue i was bored or something. why does it look dogshit Try and find a node module that supports terminal functions like trauma

3 Nov 7, 2022

Terminal table for Windows, Linux, and MacOS. Written in nodejs. Also works in browser console. Word wrap, padding, alignment, colors, Asian character support, per-column callbacks, and you can pass rows as objects or arrays. Backwards compatible with Automattic/cli-table.

tty-table 端子台 Display your data in a table using a terminal, browser, or browser console. Examples See here for complete example list To view all exam