Needful Program Management
    Share your code. npm Orgs help your team discover, share, and reuse code. Create a free org »

    node-hdfspublic

    node-hdfs

    This provides a JNI-implemented HDFS client allowing node.js applications to natively interoperate with the Hadoop FileSystem.

    Example Use

    There's a run script that will build + run the demo/demo.js sample.

    var HDFS = require('node-hdfs');
    var client = new HDFS({host:"default", port:0});
    
    client.list("/tmp/path", function(err, files) {
      console.log(files);
    });
    

    Compiling

    At the moment it's still a little tricky. At the least you'll need to make sure libhdfs is built and installed in a path accessible by ldconfig (i.e. /usr/local/lib).

    Ubuntu

    I (Paul) have had success with Cloudera's CDH distribution. Please see their instructions for installing from packages.

    Mac OSX

    cd [hadoop_uncompressed_src_path]/src/c++/libhdfs
    chmod +x ./configure
    ./configure
    

    To build libhdfs in Mac OSX a few changes need to be done in the generated 'Makefile' by './configure':

    Edit Makefile and search for a line that starts with 'CFLAGS = -g O2'

    • Remove the parameter "-m"
    • Add parameter "-framework JavaVM"

    Search for a line that starts with 'DEFS = -DPACKAGE_NAME=\"libhdfs\" -DPACKAGE_TARNAME=\"libhdfs\"'

    • Remove the parameter "-Dsize_t=unsigned\ int"

    Comment or remove include in 'hdfsJniHelper.c' :

    Search for '#include <error.h>' and remove (or comment it out).

    Then build

    make
    chmod +x ./install-sh
    make install
    

    Copy the libraries to a folder accessible to the compiler or add it to the env with dyld (man dyld for more information).

    cd [hadoop_uncompressed_src_path]/src/c++/install/lib
    cp libhdfs* /usr/local/lib
    

    Keywords

    none

    install

    npm i node-hdfs

    Downloadslast 7 days

    2

    version

    0.0.1

    license

    none

    repository

    github.com

    last publish

    collaborators

    • avatar