Parse curl --trace option output to raw HTTP message

The curl --trace parser

The story

Did you know that you can record raw HTTP communication of Curl command-line tool with the --trace and --trace-ascii option? It's the only way I know to get raw HTTP communication without using the tcpdump or wireshark. For example, this trick is very useful for the proper introspection into HTTP communication of an undocumented RESTful API.

The only glitch is that cURL --trace saves data in its custom format, far from human-friendly, saving chunks as they are being received and splitting them by packets. If you want a human readable form then this parser is what you need. Delivered as a Node.js package.

$ curl --trace - | curl-trace-parser

We will be using this sample API created with the mock server to demonstrate tracing an HTTP communication and the use of the cURL trace parser.

$ npm install -g curl-trace-parser
$ curl --trace tracefile --header "Content-Type: application/json" \
--request POST \
--data-binary "{ \"product\":\"1AB23ORM\", \"quantity\": 2 }" \

Note this cURL example is copied and pasted from Apiary interactive API documentation.

The output is ASCII representation of a raw HTTP message with few modifications:

  • Request line begins with >
  • Response line begins with <
  • Request and Response is delimited by CR+LF
  • Both Request and Response are terminated by an extra trailing LF

Note: This is little bit tricky because HTTP RFC does not have declared delimiter for Request and Response, for obvious reasons.

$ cat tracefile | curl-trace-parser --raw
> POST /shopping-cart HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5
> Host:
> Accept: */*
> Content-Type: application/json
> Content-Length: 39
> { "product":"1AB23ORM", "quantity": 2 }
< HTTP/1.1 201 Created
< Content-Type: application/json
< Date: Tue, 30 Jul 2013 11:32:30 GMT
< X-Apiary-Ratelimit-Limit: 120
< X-Apiary-Ratelimit-Remaining: 119
< Content-Length: 50
< Connection: keep-alive
< { "status": "created", "url": "/shopping-cart/2" }

The output is HTTP Request and Response in the API blueprint format which is the superset of markdown.

$ cat tracefile | ./bin/curl-trace-parser --blueprint
# POST /shopping-cart
+ Request
    + Headers
            User-Agent:curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5
    + Body
            { "product":"1AB23ORM", "quantity": 2 }
+ Response 201
    + Headers
            Date:Tue, 30 Jul 2013 11:32:30 GMT
    + Body
            { "status": "created", "url": "/shopping-cart/2" }

Note: This format does not expect any CR+LF in body

var fs = require('fs');
var parser = require('curl-trace-parser');
fs.readFile('./tracefile', 'utf8', function (err,trace) {
var fs = require('fs');
var parser = require('curl-trace-parser');
fs.readFile('./tracefile', 'utf8', function (err,trace) {

parse(traceString) - parse string with trace to object with raw request and response

parseToString(traceString) - parse string with trace to output format

parseBack(outputString) - parse string with output format back to object with raw request an resposne