rtail(1) - Terminal output to the browser in seconds, using UNIX pipes.

Related tags

Command Line rtail
Overview

rtail(1)

Wercker CI Coveralls NPM version NPM downloads GitHub Stars License Gitter

Terminal output to the browser in seconds, using UNIX pipes.

rtail is a command line utility that grabs every line in stdin and broadcasts it over UDP. That's it. Nothing fancy. Nothing complicated. Tail log files, app output, or whatever you wish, using rtail broadcasting to an rtail-server – See multiple streams in the browser, in realtime.

Installation

$ npm install -g rtail

Web app

Rationale

Whether you deploy your code on remote servers using multiple environments or simply have multiple projects, you must ssh to each machine running your code, in order to monitor the logs in realtime.

There are many log aggregation tools out there, but few of them are realtime. Most other tools require you to change your application source code to support their logging protocol/transport.

rtail is meant to be a replacement of logio, which isn't actively maintained anymore, doesn't support node v0.12., and uses TCP. (TCP requires strict client / server handshaking, is resource-hungry, and very difficult to scale.)

The rtail approach is very simple:

  • pipe something into rtail using UNIX I/O redirection [2]
  • broadcast every line using UDP
  • rtail-server, if listening, will dispatch the stream into your browser, using socket.io.

rtail is a realtime debugging and monitoring tool, which can display multiple aggregate streams via a modern web interface. There is no persistent layer, nor does the tool store any data. If you need a persistent layer, use something like loggly.

Examples

In your app init script:

$ node server.js 2>&1 | rtail --id "api.myproject.com"

$ mycommand | rtail > server.log

$ node server.js 2>&1 | rtail --mute

Supports JSON5 lines:

$ while true; do echo [1, 2, 3, "hello"]; sleep 1; done | rtail
$ echo { "foo": "bar" } | rtail
$ echo { format: 'JSON5' } | rtail

Using log files (log rotate safe!):

$ node server.js 2>&1 > log.txt
$ tail -F log.txt | rtail

For fun and debugging:

$ cat ~/myfile.txt | rtail
$ echo "Server rebooted!" | rtail --id `hostname`

Params

$ rtail --help
Usage: cmd | rtail [OPTIONS]

Options:
  --host, -h     The server host                 [string] [default: "127.0.0.1"]
  --port, -p     The server port                        [string] [default: 9999]
  --id, --name   The log stream id                 [string] [default: (moniker)]
  --mute, -m     Don't pipe stdin with stdout                          [boolean]
  --tty          Keeps ansi colors                     [boolean] [default: true]
  --parse-date   Looks for dates to use as timestamp   [boolean] [default: true]
  --help         Show help                                             [boolean]
  --version, -v  Show version number                                   [boolean]

Examples:
  server | rtail > server.log         localhost + file
  server | rtail --id api.domain.com  Name the log stream
  server | rtail --host example.com   Sends to example.com
  server | rtail --port 43567         Uses custom port
  server | rtail --mute               No stdout
  server | rtail --no-tty             Strips ansi colors
  server | rtail --no-date-parse      Disable date parsing/stripping

rtail-server(1)

rtail-server receives all messages broadcast from every rtail client, displaying all incoming log streams in a realtime web view. Under the hood, the server uses socket.io to pipe every incoming UDP message to the browser.

There is little to no configuration – The default UDP/HTTP ports can be changed, but that's it.

Examples

Use default values:

$ rtail-server

Always use latest, stable webapp:

$ rtail-server --web-version stable

Use custom ports:

$ rtail-server --web-port 8080 --udp-port 9090

Set debugging on:

$ DEBUG=rtail:* rtail-server

Open your browser and start tailing logs!

Params

$ rtail-server --help
Usage: rtail-server [OPTIONS]

Options:
--udp-host, --uh  The listening UDP hostname            [default: "127.0.0.1"]
--udp-port, --up  The listening UDP port                       [default: 9999]
--web-host, --wh  The listening HTTP hostname           [default: "127.0.0.1"]
--web-port, --wp  The listening HTTP port                      [default: 8888]
--web-version     Define web app version to serve                     [string]
--help, -h        Show help                                          [boolean]
--version, -v     Show version number                                [boolean]

Examples:
rtail-server --web-port 8080         Use custom HTTP port
rtail-server --udp-port 8080         Use custom UDP port
rtail-server --web-version stable    Always uses latest stable webapp
rtail-server --web-version unstable  Always uses latest develop webapp
rtail-server --web-version 0.1.3     Use webapp v0.1.3

UDP Broadcasting

To scale and broadcast on multiple servers, instruct the rtail client to stream to the broadcast address. Every message will then be delivered to all servers in your subnet.

Authentication layer

For the time being, the webapp doesn't have an authentication layer; it assumes that you will run it behind a VPN or reverse proxy, with a simple Authorization header check.

How to contribute

This project follows the awesome Vincent Driessen branching model.

  • You must add a new feature on its own branch
  • You must contribute to hot-fixing, directly into the master branch (and pull-request to it)

This project uses JSCS to enforce a consistent code style. Your contribution must be pass jscs validation.

The test suite is written on top of mochajs/mocha. Use the tests to check if your contribution breaks some part of the library and be sure to add new tests for each new feature.

$ npm test

Contributors

Roadmap (aka where you can help)

  • Write a rock solid test suite
  • Allow use of DTLS (waiting for node to support this https://github.com/joyent/node/pull/6704)
  • Add GitHub OAuth and basic auth for teams (join proposal convo here: https://github.com/kilianc/rtail/issues/44)
  • Implement infinite-scroll like behavior in the webapp to support bigger backlogs and make it future proof.
  • Publish base rtail docker image to DockerHub
  • Create a catch all docker logs image
  • Rewrite webapp using ng2

Sponsors

rTail? Consider sponsoring this project to keep it alive and free for the community.

  • Lukibear (domain)
  • ? (wildcard TLS cert)
  • ? (.io domain)

PayPal donate button

Professional support or ad-hoc is also available.

License

This software is released under the MIT license cited below.

Copyright (c) 2014 Kilian Ciuffolo, [email protected]. All Rights Reserved.

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the 'Software'), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
Comments
  • make backlog size configurable

    make backlog size configurable

    I was using this for a java log and a stacktrace was longer than the buffer. If I have the available RAM could there be an option to control the buffer size?

    In the mean time is there a place I can hack this in?

    enhancement help wanted webapp server 
    opened by rmchale 12
  • Howto reverse proxy via nginx

    Howto reverse proxy via nginx

    How does one reverse proxy rtail?

        server {
                server_name irc.dabase.com;
                location / {
                    proxy_pass http://localhost:8888;
                }
        }
    
    opened by kaihendry 10
  • Syntax highlighting / colorize support

    Syntax highlighting / colorize support

    Being able to colorize/ syntax highlight logs would be useful.

    Use case I envision is highlighting lines in red when ERROR shows up, and so on and so forth for other log levels.

    opened by jdrews 6
  • XSS Issue

    XSS Issue

    Any code in the content field get evaluated.

    [  
       "line",
       {  
          "timestamp":0000000000000,
          "streamid":"rtail.org",
          "host":"XXX.XXX.XXX.XXX",
          "port":50049,
          "content":"<script>window.top.location.href=\"http://www.google.com\"</script>",
          "type":"string"
       }
    ]
    

    This would redirect the user to http://www.google.com

    bug webapp 
    opened by MSylvia 6
  • Colors and front-end refactoring

    Colors and front-end refactoring

    @kilianc there you go, sorry, I was making changes in main repo., I know that feature branches are not our thing here :)

    New javascript files (in webapp/public/js) for front-end should go to S3.

    enhancement 
    opened by s-silva 6
  • Not getting all streamed info

    Not getting all streamed info

    I am streaming an IRC session with ii to my rtail server like so:

    sg:~/irc/irc.freenode.net/#hackerspacesg$ tail -F out | rtail --id hsg --host sg.dabase.com
    2015-07-09 15:33 -!- iilogbot([email protected]) has joined #hackerspacesg
    2015-07-09 15:34 <hendry> another test
    2015-07-09 15:34 <hendry> http://sg.dabase.com:8888/#hsg
    2015-07-09 15:34 <WantonMee> [ rTail : loading ... ] - sg.dabase.com:8888
    

    However it looks like this:

    It's missing the nick info <hendry> and duplicating the timestamp. Am I not using the right option?

    opened by kaihendry 5
  • Support grep color format

    Support grep color format

    Hi!

    I'm having fun playing with rtail but I want to use it to monitor scripts that use grep to colorize their output and it doesn't seem to work well with rtail. To reproduce, the following command should display 'lol' in red in the terminal but not in the rtail UI.

    $ echo lol | grep --color=always lol | rtail --id test_grep
    

    This is the format that grep uses:

    $ echo lol | grep --color=always lol | cat -e
    ^[[01;31m^[[Klol^[[m^[[K$
    

    I'm not sure if this is more of an ansi_up issue, or if there are some GREP_COLORS options I could set to fix it.

    Thank you for your help and for building something cool!

    question client 
    opened by baptistemarchand 5
  • Incomplete log

    Incomplete log

    missing-whole-log

    Not entirely sure how to test which version I am running: http://sg.dabase.com:8888/#/streams/hsg

    The last line where I say "tes" is what I expected to see on the browser!

    opened by kaihendry 4
  • Add date format in the settings menu

    Add date format in the settings menu

    Hi,

    This is an idea for a feature. It would be great if we could chose the date/time format in the settings menu. I'm from France and the current format "07/13/15 02:32:41" is not really intuitive. I would like to be able to select something like "13/07/15 14:32:41".

    Thanks!

    enhancement webapp 
    opened by baptistemarchand 4
  • rtail.org rick roll?

    rtail.org rick roll?

    rtail.org seems to be redirecting to youtube (rick roll 10 hours) on page load. It's a great track, but er, I'd kinda like to see the thing in action.

    bug webapp 
    opened by Malet 4
  • Message count

    Message count

    I would like UI to display message count somewhere at the top.

    It would be nice to be able to filter a log and see how many times a particular event happened.

    Is this a feature rtail UI could have at some point?

    opened by eush77 3
  • The term '/bin/sh.exe' is not recognized as a name of a cmdlet

    The term '/bin/sh.exe' is not recognized as a name of a cmdlet

    os

    • windows
    • shell: powershell v7 >
    • terminal env: windows terminal

    code

     echo "hello there" | rtail
    &: C:\Users\aschu\AppData\Roaming\npm\rtail.ps1:22
    Line |
      22 |      $input | & "/bin/sh$exe"  "$basedir/pnpm-global/5/node_modules/rt …
         |                 ~~~~~~~~~~~~~
         | The term '/bin/sh.exe' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the
         | name, or if a path was included, verify that the path is correct and try again.
    

    tried

    modify rtail.ps1

    replaced shebang #!/bin/sh with: #!/usr/bin/env node

    replaced line22

    replaced "/bin/sh$exe" with "powershell.exe" & "cmd.exe"

    opened by alexn-s 0
  • Limitations to amount of logs broadcasted by rtail ?

    Limitations to amount of logs broadcasted by rtail ?

    Hi rtail team! I have a question about the working of the rtail utility: When rtail get its stdin and start broadcasting its log lines, are connection timeouts likely ? Is there a limitation to the number of log lines it can broadcast? or anything else that would cause incomplete reception of logs ? If so, how can I configure each of those categories to allow for flexibility.

    opened by txt-bmukum 0
  • Display formatted hyperlinks

    Display formatted hyperlinks

    When displaying stripe logs, there are hyperlinks that opens the stripe dashboard page to the specific event id, for example. In the terminal, the hyperlinks show properly https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

    In rtail however, I'm seeing the unformatted text version of this.

    using stripe listen for example

    in terminal, displayed as a formatted link:

    Screen Shot 2021-09-29 at 8 11 55 PM

    in rtail, displayed without formatting:

    Screen Shot 2021-09-29 at 8 11 46 PM
    opened by angieology 0
  • Update README.md

    Update README.md

    Hi Kilian,

    I would like to make this proposal to replace broken link to a refer document at the following address: http://www.westwind.com/reference/os-x/commandline/pipes.html

    This document is no longer available for years.

    I've made a new one (https://bytebitebit.com/pipes/) that's the same as the old one to help readers have a good experience when looking for the explanation about 'UNIX I/O redirection'.

    Regards, Jane

    opened by janesmithjs 0
  • Not working with pm2 ecosystem files

    Not working with pm2 ecosystem files

    Unable to find any documentation which mentions how to use it ecosystem files of pm2. How do I use this with ecosystem files?

            name: appName,
            script: "./app.js",
            args: "one two",
            instances: 1,
            watch: true,
            ignore_watch: ["node_modules", "logs"],
            env: {
                NODE_ENV,
            },
            log_date_format: "ddd, DD-MMM-YYYY, hh:mm:ss A UTCZ",
            output: logLocation.output,
            error: logLocation.error,
    
    opened by dibyajyotiron 0
Releases(v0.2.1)
  • v0.2.1(Jul 11, 2015)

  • v0.2.0(Jul 11, 2015)

    New

    • Collapsable timestamp column [@kilianc]
    • Deep linking to a stream [@kilianc]
    • Smart timestamp parsing option to avoid double timestamps, also added --no-date-parse to disable it. [@kilianc]
    • Resizable sidebar and ellipsis for long stream names [@kilianc]

    Fixes

    • Multiple spaces support, have fun with ASCII art ! [@kilianc]
    • Fixed XSS vulnerability allowing to inject html / js code in a stream [@kilianc]
    • Fixed popup triagle color [@timriot]
    • Renamed -not-tty in --no-tty for consistency [@kilianc]
    • Updated --help output [@kilianc]
    • Updated README.md output [@timriot]
    • Updated deps [@kilianc]
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Jul 11, 2015)

  • v0.1.0(Jul 11, 2015)

Owner
Kilian Ciuffolo
Kilian Ciuffolo
Uses marked-terminal to render a README.md for any npm module in the terminal.

modhelp Uses marked-terminal to render a README.md for any npm module in the terminal. Now with built-in pager! Page up/down, arrow keys to scroll lin

Jason Livesay 23 Feb 8, 2022
Terminal ui for discord with interactive terminal

dickord why No fucking clue i was bored or something. why does it look dogshit Try and find a node module that supports terminal functions like trauma

Hima 3 Nov 7, 2022
DataENV is a cli tool that allows you to save data temporarily using your terminal.

DataEnv CLI Instllation npm install -g dataenv Usage Table of Contents LocalStorage npx dataenv save Parameters npx dataenv show Parameters npx dataen

PGamerX 2 Feb 5, 2022
Sublime-like terminal-based text editor

slap ?? slap is a Sublime-like terminal-based text editor that strives to make editing from the terminal easier. It has: first-class mouse support (ev

slap 6.1k Jan 1, 2023
A terminal-to-gif recorder minus the headaches.

ttystudio A terminal-to-gif recorder minus the headaches. Record your terminal and compile it to a GIF or APNG without any external dependencies, bash

Christopher Jeffrey (JJ) 3.2k Dec 23, 2022
Pipeable javascript. Quickly filter, map, and reduce from the terminal

Pipeable JavaScript - another utility like sed/awk/wc... but with JS! Quickly filter, map and reduce from the command line. Features a streaming API.

Daniel St. Jules 410 Dec 10, 2022
Translations with speech synthesis in your terminal as a node package

Normit Normit is an easy way to translate stuff in your terminal. You can check out its Ruby gem version termit. Installation npm install normit -g Us

Paweł Urbanek 234 Jan 1, 2023
Terminal recorder: Record your termial session into HTML

terminal-recorder Terminal recorder allows you to record your bash session, and export it to html so then you can share it with your friends. GitHub P

Cristian Cortez 104 Mar 3, 2022
Terminal task list

listr Terminal task list Install $ npm install --save listr Usage const execa = require('execa'); const Listr = require('listr'); const tasks = new

Sam Verschueren 3.1k Jan 3, 2023
📜 Create mutable log lines into the terminal, and give life to your logs!

Because Logging can be pretty and fun Installation $ npm install draftlog What it does It allows you to re-write a line of your log after being writt

Ivan Seidel 1.2k Dec 31, 2022
Display images in the terminal

terminal-image Display images in the terminal Works in any terminal that supports colors. In iTerm, the image will be displayed in full resolution, si

Sindre Sorhus 905 Dec 25, 2022
Create clickable links in the terminal

terminal-link Create clickable links in the terminal Install $ npm install terminal-link Usage import terminalLink from 'terminal-link'; const link

Sindre Sorhus 539 Dec 31, 2022
Reliably get the terminal window size

term-size Reliably get the terminal window size Because process.stdout.columns doesn't exist when run non-interactively, for example, in a child proce

Sindre Sorhus 132 Oct 11, 2022
Truncate a string to a specific width in the terminal

cli-truncate Truncate a string to a specific width in the terminal Gracefully handles ANSI escapes. Like a string styled with chalk. It also supports

Sindre Sorhus 78 Oct 10, 2022
Execute shell commands in terminal

Execute shell commands in terminal

skanehira 9 Dec 11, 2021
Add a hungry turtle to your terminal and feed it every time you mistype 'npm' as 'nom'

Nom Does this ever happen to you? You happily code away on a project, navigating the command line like a pro, testing, error logging, installing packa

Meike Hankewicz 5 Apr 26, 2022
Just a minimal library to do some terminal stuff.

Termctl A simple library to do some basic terminal stuff. Usage const termctl = require("termctl"); Note: We have tested this on Linux Mint and Window

Biraj 4 Sep 28, 2021
A terminal for a more modern age

Downloads: Latest release Repositories: Debian/Ubuntu-based, RPM-based Latest nightly build This README is also available in: Korean 简体中文 Tabby (forme

null 41.8k Dec 30, 2022