Have ideas to improve npm?Join in the discussion! »

    storacle

    0.2.15 • Public • Published

    Storacle [alpha] npm version Build status

    Storacle is a decentralized file storage based on the spreadable protocol.

    const Node = require('storacle').Node;
    
    (async () => {  
      try {
        const node = new Node({
          port: 4000,
          hostname: 'localhost'
        });
        await node.init();
      }
      catch(err) {
        console.error(err.stack);
        process.exit(1);
      }
    })();
    const Client = require('storacle').Client;
    
    (async () => {  
      try {
        const client = new Client({
          address: 'localhost:4000'
        });
        await client.init();
    
        // Store our file
        const hash = await client.storeFile('./my-file');
    
        // Get the direct file link
        const link = await client.getFileLink(hash);
    
        // Create the requested file link
        const requestedLink = client.createRequestedFileLink(hash);
        
        // Remove the file
        await client.removeFile(hash);
      }
      catch(err) {
        console.error(err.stack);
        process.exit(1);
      }
    })();

    The example above shows the simplest use of the library. But the server can be flexibly configured.

    Browser client

    You can also use the client in a browser. Look at the description of the spreadable library. In window you have window.ClientStoracle instead of window.ClientSpreadable. The prepared file name is storacle.client.js.

    How to use it via the command line

    Look at the description of the spreadable library. You only need to change everywhere spreadable word to storacle.

    How it works

    Nodes interact via the spreadable mechanism. The file can be added to the network through any node. Files are saved entirely, not splitted into parts. After saving you get a hash of the file. With this hash you can later get it again, delete it or do something else. If possible, links to files are cached so if you work with the same file and in some other cases you will receive it immediately without re-traversing the network. For better reliability files can be duplicated. How exactly you can customize yourself. By default, each file tends to have its copies in amount of Math.ceil(Math.sqrt(networkSize)).

    What are the limitations

    The number of files on one node is limited by the file system. The speed of receiving / saving in the network files is limited by spreadable protocol.

    What are the requirements

    Look at the spreadable requirements.

    Where to use it

    1. Wherever files need to be stored decentralized

    For example, you can create a network that stores books, invite new members, gather a large collection of books, and share links with others. Instead of books there could be anything else.

    2. For own needs

    Storing files of your own projects, websites, etc. The network can be made private and you will not need to figure out how to organize the file storage.

    3. Serverless solutions

    Since the library is written in javascript, you can receive / send / work with files in the browser and do not use server code at all. In some cases, it can be very convenient.

    Node configuration

    When you create an instance of the node you can pass options below. Only specific options of this library are described here, without considering the options of the parent classes.

    • {integer} [request.clientStoringConcurrency=20] - the maximum number of simultaneous client storing requests per endpoint.

    • {number|string} [request.fileStoringNodeTimeout="2h"] - file storing timeout.

    • {number|string} [request.cacheTimeout=250] - file cache link check timeout.

    • {number|string} [storage.dataSize="45%"] - amount of space for storing files. If indicated in percent, the calculation will be based on the maximum available disk space.

    • {number|string} [storage.tempSize="45%"] - amount of space for temporary files. If indicated in percent, the calculation will be based on the maximum available disk space.

    • {number|string} [storage.tempLifetime="2h"] - temporary files holding period.

    • {integer} [storage.autoCleanSize=0] - amount of space that should always be free. If indicated in percent, the calculation will be based on storage.dataSize.

    • {object} [file] - section that responds for a single file settings.

    • {number|string} [file.maxSize="40%"] - maximum size of one file. If indicated in percent, the calculation will be based on the maximum available disk space.

    • {number|string} [file.minSize=0] - minimum size of one file. If indicated in percent, the calculation will be based on the maximum available disk space.

    • {integer|string} [file.preferredDuplicates="auto"] - preferred number of file copies on the network. If indicated in percent, the calculation will be based on the network size. If the option is "auto" it will be calculated as Math.ceil(Math.sqrt(networkSize)).

    • {number|string} [file.responseCacheLifetime="7d"] - period of file caching after giving it to the client.

    • {string[]} [file.mimeWhitelist=[]] - whitelist for filtering a file by mime type.

    • {string[]} [file.mimeBlacklist=[]] - blacklist for filtering a file by mime type.

    • {string[]} [file.extWhitelist=[]] - whitelist for filtering a file by its extension.

    • {string[]} [file.extBlacklist=[]] - blacklist for filtering a file by its extension.

    • {object|false} [file.linkCache] - file link caching transport options.

    • {integer} [file.linkCache.limit=50000] - maximum cache links.

    • {number|string} [file.linkCache.lifetime="1d"] - cache link holding period.

    • {number|string} [task.cleanUpStorageInterval="30s"] - storage cleanup task interval.

    • {number|string} [task.cleanUpTempDirInterval="20s"] - temporary folder cleanup task interval.

    • {number|string} [task.calculateStorageInfoInterval="3s"] - storage information calculaion task interval.

    Client configuration

    When you create an instance of the client you can pass options below. Only specific options of this library are described here, without considering the options of the parent classes.

    • {number|string} [request.fileStoringTimeout="2.05h"] - file storing timeout.

    • {number|string} [request.fileGettingTimeout="1h"] - file getting timeout.

    • {number|string} [request.fileRemovalTimeout="10s"] - file removal timeout.

    • {number|string} [request.fileLinkGettingTimeout="10s"] - file link getting timeout.

    Client interface

    async Client.prototype.storeFile() - add file to the network.

    • {string|fs.ReadStream|Buffer|Blob} file - any file
    • {object} [options] - storing options
    • {number} [options.timeout] - storing timeout

    async Client.prototype.getFileLink() - get the file link by the hash.

    • {string} hash - file hash
    • {object} [options] - getting options
    • {number} [options.timeout] - getting timeout

    async Client.prototype.getFileLinks() - get all file links by the hash.

    • {string} hash - file hash
    • {object} [options] - getting options
    • {number} [options.timeout] - getting timeout

    async Client.prototype.getFileToBuffer() - download the file and return the buffer.

    • {string} hash - file hash
    • {object} [options] - getting options
    • {number} [options.timeout] - getting timeout

    async Client.prototype.getFileToPath() - download the file and write it to the specified path.

    • {string} hash - file hash
    • {object} [options] - getting options
    • {number} [options.timeout] - getting timeout

    async Client.prototype.getFileToBlob() - download the file and return the blob. For browser client only.

    • {string} hash - file hash
    • {object} [options] - getting options
    • {number} [options.timeout] - getting timeout

    async Client.prototype.removeFile() - Remove the file by hash.

    • {string} hash - file hash
    • {object} [options] - removal options
    • {number} [options.timeout] - removal timeout

    Client.prototype.createRequestedFileLink() - сreate a requested file link. This is convenient if you need to get the link without doing any asynchronous operations at the moment.

    • {string} hash - file hash
    • {object} [options] - options

    Contribution

    If you face a bug or have an idea how to improve the library, create an issue on github. In order to fix something or add new code yourself fork the library, make changes and create a pull request to the master branch. Don't forget about tests in this case. Also you can join the project on github.

    Install

    npm i storacle

    DownloadsWeekly Downloads

    49

    Version

    0.2.15

    License

    MIT

    Unpacked Size

    411 kB

    Total Files

    48

    Last publish

    Collaborators

    • avatar