Non Programming Manager

    binary-data-chunking

    3.0.1 • Public • Published

    binary-data-chunking

    A simple binary data chunking library that simplifies sending large amounts of chunked binary data. For example via. the WebRTC DataChannel.

    This is useful in the case that you would like to received chunks of data instead of one large file Blob at the end of the transfer (as in Firefox), and avoid the issues with sending large files in Chrome.

    Although this was designed for use in WebRTC, it could be used in other environments where you need a custom way to send chunks binary of information.

    Environment

    This can be used both in a node.js or browser environment (targeted browsers are the latest: Chrome, Firefox and eventually Edge)

    Usage

    Include

    Node

    var BDC = require('binary-data-chunking');

    Browser

    Include the either the minified, or non-minified files in the dist/ directory.

    dist/binary-data-chunking.js
    dist/binary-data-chunking.min.js

    Initialize

    You create a new BDC instance for each file you wish to transfer. The initialization parameters give the basic information about your file.

    Sending client

    var file = new BDC({ 
        name: 'sample.mp4',
        mimeType: 'video/mp4',
        arrayBuffer: ab, // must be in the ArrayBuffer format
        chunkSize: 16000 // chunkSize to use *before* meta-data additions.
    });
     
    file.fileSize;   // returns actual file size based on ArrayBuffer
    file.totalChunks; // total number of chunks that will be sent (based on file size and chunk size)
    file.checksum;   // checksum based on ArrayBuffer
     
    file.getMetadata();  // to be sent to other end to initialize the transfer, used to initialize a BDC instance on the receiving end.
     
    file.clearData();    // removes arrayBuffer data and all stored information
     
    var to = file.getTransferObject(); // unique instance to track the transfering of each chunk 
     
    to.getChunk([pos]); // will get next chunk, or optionally get the chunk based on the number (position) specific. 
                        // note, the chunk will have header info embedded in the first few bytes, to receiving client 
                        // must also have an instance created based on the information returned from `file.getMetadata()`
    to.currentIndex;    // returns the current number if chunks given from `getChunk()`
                          

    Receiving client

    When you receive the metadata sent from the sending client, you can initialize a new BDC instance

    // when metadata is received
    var file = new BDC(metadata);
     
    file.onChunkReceived(function (ab, pos) {
        // this handler will be called when a chunk that relates to this file instance is received    
    });
     
    file.onComplete(function (ab, metadata) {
       // here you receive the completed ArrayBuffer object along with it's metadata 
    });
     
    var to = file.getTransferObject();
    var payload = to.getUnpackedChunk(); // next ordered arrayBuffer without the binary metadata bits
     
    // when a binary chunk is received, we don't know yet which file object it belongs to, so we submit it to the Factory object and wait for the handler to be called
    BDC.submitChunk(chunk);
     
    file.generatedChecksum; // contains then generated checksum once a transfer is complete

    With your file object instance, you can access the methods like totalChunks, fileSize, getMetadata(), getChunk() and once completed getFile().

    Known Issues

    The generation of a uid does have some collision potential, we we are limited in space, and also we do not know what uids have already been generated on the receiving end(s) of the file transfer(s).

    Install

    npm i binary-data-chunking

    DownloadsWeekly Downloads

    24

    Version

    3.0.1

    License

    MIT

    Last publish

    Collaborators

    • silverbucket