pino-clf

    1.0.6 • Public • Published

    pino-clf    stability

    npm version build status test coverage dependencies freshness js-standard-style

    Transform Pino HTTP logs into Common Log Format.

    This CLI tool and module is a transform transport for the pino which outputs Common Log Format.

    demo

    Supports

    Usage

    $ npm install -g pino-clf
    $ pino-clf -h
      
        pino-clf [-d] [-k] [-a] type
     
        type                common | combined (default)
     
        -d | --dest |       stderr | stdout (default) or Number or Array of Numbers
        --destination       Specify file descriptor(s) to send log(s) to [access, referral, agent]
        
        -a | --ancillary    stderr | stdout or Number. Specify JSON logs fd
        
        -k | --keep         true | false (default) Retain transformed logs in ancillary output
     

    Common Log Format

    The Common Log Format is a frequently used log format, that has a very mature and well established ecosystem built around it.

    It takes two forms: "Combined" and "Common".

    When the type is "Common" this can break down into (up to) three separate log streams:

    • Access log (also known as Common log)
    • Referall log
    • Agent log

    Access log example:

    127.0.0.1 - Aladdin [21/Jul/2016:17:34:52 -0060] "GET /api/activity/component HTTP/1.1" 200 -
    

    Referral log example:

    [21/Jul/2016:17:34:52 -0060] "http://localhost:20000/"
    

    Agent log example:

    [21/Jul/2016:17:34:52 -0060] "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
    

    The "Combined" format combines all three formats into one, for instance:

    127.0.0.1 - Aladdin [21/Jul/2016:17:34:52 -0060] "GET /api/activity/component HTTP/1.1" 200 - "http://localhost:20000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
    

    Example

    Spin up a server that uses a pino http logger (see the Supports section), pipe it to pino-clf and desribe the format in tokenized form

    $ node server | pino-clf
    127.0.0.1 - Aladdin [21/Jul/2016:17:34:52 -0060] "GET /api/activity/component HTTP/1.1" 200 - "http://localhost:20000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"

    The default output is Combined. To use three log output use the -d flag and specify up to three file descriptors (first is access log, second referral log, third agent log):

    node server | pino-clf -d [3,4,5] 3>> ./access.log 4>> ./referral.log 5>> ./agent.log

    The above will append the Access, Agent, and Referral logs to each corresponding log file.

    We can skip the middle log (referall) by using an file desriptor of 0 or null as the middle element:

    node server | pino-clf -d [3,0,4] 3>> ./access.log 4>> ./agent.log

    Destination (-d)

    By default, logs are output to STDOUT, however we can set the -d (alias, --dest, --destination), flag to a a stderr, or a number (1 for stdout, 2 for stderr, 3 or more for custom file descriptor):

    $ node server | pino-clf -d stderr

    The above is equivalent to:

    $ node server | pino-clf -d 2 combined

    We can also direct formatted log output to custom file descriptors, but we must use bash redirection (in some form) from that file descriptor, otherwise the process will most likely immediately crash (this is to do with how unix works).

    $ node server | pino-clf -d 8 8> ./combined.log

    Ancillary Output (-a)

    By default, any logs which aren't an HTTP log (meaning, they don't have req and res properties and the msg isn't "request complete") are filtered out.

    However, we can specify an ancillary (secondary) output for other log messages, using the -a (alias --ancillary) flag.

    The following will write reformatted HTTP logs to STDOUT and original JSON logs which are not HTTP logs to STDERR.

    $ node server | pino-clf -a 2 -d 1

    Keep Original HTTP JSON Logs (-k)

    The -a (--ancillary) flag can be coupled with the -k (--keep) flag so that raw HTTP JSON logs are also piped to the ancillary output stream, along with any filtered output.

    The following will pipe all formatted logs to the 4 file descriptor which is redirected to a file, while all original JSON logs (instead of non-HTTP logs) are written to STDOUT.

    $ node server | pino-clf -k -a 1 -d 4 4> ./combined.logs
    

    Programmatic API

    clf(type?, destination?, ancillary?)

    Returns a stream that we write Pino JSON logs to.

    type (String | Object)
    • combined (default)
    • common

    If object form type is {type, keep}

    • keep (Boolean, false) - preserve original HTTP Logs, as well as other JSON logs when writing to ancillary stream
    destination (Stream | Array)

    Stream or streams to write output to.

    If the type is common destination can be an array of up to three streams, to enable the three log output format.

    • destination0 - access log
    • destination1 - referral log
    • destination2 - agent log
    ancillary (Stream)

    Stream to output original JSON logs to.

    LICENSE

    MIT

    Acknowledgements

    Install

    npm i pino-clf

    DownloadsWeekly Downloads

    19

    Version

    1.0.6

    License

    MIT

    Unpacked Size

    1.41 MB

    Total Files

    11

    Last publish

    Collaborators

    • davidmarkclements
    • matteo.collina
    • jsumners
    • watson