Incredibly unsafe way to free a Buffer
If this is so dangerous why should you use it? Because it can drastically increase throughput by relieving pressure from the garbage collector.
This is what's left of an attempt to allow users to manually free memory attached to a Buffer instance. It was too precarious to do in core, so now I'm making my efforts available here.
Note: This code is kept up with latest master. When v0.12 is released a 0.1 branch will be created to maintain stability.
First a note must be made that
Buffer has been rewritten in Node v0.11, and
SlowBuffer now has a new use. Where
Buffer will automatically return a
slice of data from a larger slab
SlowBuffer will always allocate the exact
amount requested, but
SlowBuffer now returns a normal instance of
instead of its own type.
For this reason the following example uses
buffer-dispose would only remove the reference to the slice it pointed to, but
not free the actual memory.
var dispose = require'buffer-dispose';var SlowBuffer = require'buffer'SlowBuffer;var buf = 5;console.logbuf;disposebuf;console.logbuf;// output:// <Buffer 00 00 00 00 00>// <Buffer >
The best use case is when receiving incoming data. Previously data returned from
'data' event was allocated from an internal slab. This
has been removed, and now all incoming allocations are
While this might seem to be a waste, understand that the overhead of tracking the slab along with the associated V8 calls necessary to do so, outweigh any performance benefit possibly gained.
The following is the simplest example of immediately disposing of incoming data.
Remember that all incoming buffers from a connection are discretely allocated,
allowing the data to be immediately
var dispose = require'buffer-dispose';// do some quick operations on the chunkdisposechunk;socketon'data' onData;require'net'createServeronConnectionlisten8000;
Though make sure all requests against the data are complete. This means you must be aware of any asynchronous events. In the following example a buffer is queued to be written to disk, but then memory is released before the asynchronous event is able to finish.
var dispose = require'buffer-dispose';var fs = require'fs';var buf = require'buffer'SlowBuffer10;buffill'a';fswriteFile'test.txt' buf ;// disposing here means nothing will be written to diskdisposebuf;
To install this from a non-globally installed build of master, use the following:
/path/to/build/node `which npm` --nodedir=/path/to/build install buffer-dispose
To build the library I do the following:
/path/to/build/node `which npm` --nodedir=/path/to/build install njsutil bindings/path/to/build/node `which node-gyp` rebuild --nodedir=/path/to/build
Probably a better way to do this, but eh. It works for now. Submit a ticket if you have something better.
To see how using this module can help your I/O I've included a performance test! It's easy to run (once you have the module built and all). Just run this:
/path/to/build/node ./speed/tcp.js <add "true" here to dispose buffers>
The below table shows performance differences cleaning up incoming Buffers at specific sizes. As we can see, the act of disposing has a performance cost. While in every case we save on memory usage, if performance is more imperative then tune your application accordingly.
64KB Writes Throughput Memory Usage-----------------------------------------Sad Ponies 26.2 Gb/s 243.2 MBMagic Unicorns 42.5 Gb/s 47.6 MB32KB Writes Throughput Memory Usage-----------------------------------------Sad Ponies 26.0 Gb/s 243.4 MBMagic Unicorns 37.6 Gb/s 47.7 MB16KB Writes Throughput Memory Usage-----------------------------------------Sad Ponies 25.3 Gb/s 243.5 MBMagic Unicorns 26.6 Gb/s 47.6 MB