Newline Pending Merge

    send-stream
    TypeScript icon, indicating that this package has built-in type declarations

    2.5.5 • Public • Published

    send-stream

    NPM Build Status Coverage Status

    send-stream is a library for streaming files from the file system or any other source of data.

    It supports partial responses (Ranges including multipart), conditional-GET negotiation (If-Match, If-Unmodified-Since, If-None-Match, If-Modified-Since) and precompressed content encoding negociation.

    It also have high test coverage, typescript typings and has multiple examples using Fastify, Koa, Express or pure NodeJS http/http2 modules.

    Installation

    This is a Node.js module available through the npm registry. Installation is done using the npm install command:

    npm install send-stream

    Getting start

    Serve all files from a directory (also serve index.html from directories on trailing slash) with Fastify, Koa or Express

    See examples for more advanced usages.

    Using Fastify (v3.x.x):

    const path = require('path');
    const fastify = require('fastify');
    const { FileSystemStorage } = require('send-stream');
    
    const app = fastify({ exposeHeadRoutes: true });
    const storage = new FileSystemStorage(
      path.join(__dirname, 'assets'),
      { onDirectory: 'serve-index' }
    );
    
    app.get('*', async (request, reply) => {
      const result = await storage.prepareResponse(
        request.url,
        request.raw
      );
      if (result.statusCode === 404) {
        reply.callNotFound(); // let fastify handle 404
        return;
      }
      await result.send(reply.raw);
    });
    
    app.listen(3000)
      .then(() => {
        console.info('listening on http://localhost:3000');
      });

    Using Koa (v2.x.x):

    const path = require('path');
    const Koa = require('koa');
    const { FileSystemStorage } = require('send-stream');
    
    const app = new Koa();
    const storage = new FileSystemStorage(
      path.join(__dirname, 'assets'),
      { onDirectory: 'serve-index' }
    );
    
    app.use(async (ctx, next) => {
      let result = await storage.prepareResponse(
        ctx.request.path,
        ctx.req
      );
      if (result.statusCode === 404) {
        await next(); // let koa handle 404
        return;
      }
      ctx.status = result.statusCode;
      ctx.set(result.headers);
      ctx.body = result.stream;
    });
    
    app.listen(3000, () => {
      console.info('listening on http://localhost:3000');
    });

    Using Express (v4.x.x):

    const path = require("path");
    const express = require("express");
    const { FileSystemStorage } = require('send-stream');
    
    const app = express();
    const storage = new FileSystemStorage(
      path.join(__dirname, 'assets'),
      { onDirectory: 'serve-index' }
    );
    
    app.get('*', async (req, res, next) => {
      try {
        let result = await storage.prepareResponse(
          req.url,
          req
        );
        if (result.statusCode === 404) {
          next(); // let express handle 404
          return;
        }
        await result.send(res);
      } catch (err) {
        next(err);
      }
    });
    app.listen(3000, () => {
      console.info('listening on http://localhost:3000');
    });

    API

    new FileSystemStorage(root, [options])

    Create a new FileSystemStorage which is a stream storage giving access to the files inside the given root folder.

    • The root parameter is a the absolute path from which the storage takes the files.

    • The options parameter let you add some addition options:

    mimeTypeLookup

    In order to return the content type, the storage will try to guess the mime type thanks to the mime-types module (see mime-types module documentation). This option override the mime-types lookup function which will be used for this purpose.

    Example:

    new FileSystemStorage(
      directory,
      {
        mimeTypeLookup: filename => {
          if (filename.endsWith('.abc')) {
            return 'text/abc';
          }
          return undefined;
        }
      }
    )

    mimeTypeDefaultCharset

    In order to return the content type, the storage will try to guess the mime type charset thanks to the mime-types module (see mime-types module documentation). This option override the mime-types charset function which will be used for this purpose.

    Example:

    new FileSystemStorage(
      directory,
      {
        mimeTypeDefaultCharset: mimeType => {
          if (mimeType === 'text/abc') {
            return 'UTF-8';
          }
          return undefined;
        }
      }
    )

    dynamicCompression

    Enable dynamic compression of file content. This can be a boolean or a list of encodings ordered by priority, ['br', 'gzip'] if true is used. Activating this option will automatically compress content as brotli or gzip if the content is detected as compressible and supported by the client.

    Note that this is highly recommended to use this option only if you can not use pre-compressed files with the contentEncodingMappings option.

    Also when dynamic compression is active, Content-Length header will be removed and range requests will be disabled as content length is unknown

    Defaults to false

    Example:

    new FileSystemStorage(
      directory,
      {
        dynamicCompression: true
      }
    )

    mimeTypeCompressible

    Function used to determine if a type is compressible (for dynamic compression only) compressible module will be used by default

    Example:

    new FileSystemStorage(
      directory,
      {
        dynamicCompression: true,
        mimeTypeCompressible: mimeType => mimeType === 'text/plain'
      }
    )

    dynamicCompressionMinLength

    Sets the minimum length of a response that will be dynamically compressed (only when the length is known)

    Defaults to 20

    Example:

    new FileSystemStorage(
      directory,
      {
        dynamicCompression: true,
        dynamicCompressionMinLength: 100
      }
    )

    defaultMimeType

    Configures the default content type (without charset) that will be used if the content type is unknown.

    undefined by default

    Example:

    new FileSystemStorage(
      directory,
      { defaultMimeType: 'application/octet-stream' }
    )

    maxRanges

    Configure how many ranges can be accessed in one request.

    Defaults to 200

    Setting it to 1 will disable multipart/byteranges

    Setting it to 0 or less will disable range requests

    Example:

    new FileSystemStorage(
      directory,
      { maxRanges: 10 }
    )

    weakEtags

    The storage will generate strong etags by default, when set to true the storage will generate weak etags instead.

    false by default

    Example:

    new FileSystemStorage(
      directory,
      { weakEtags: true }
    )

    contentEncodingMappings

    Configure content encoding file mappings.

    You can use this option to serve pre-compressed files. (see this this example to precompress files with NodeJS)

    This is a list of objects containing the following properties:

    • matcher: a regular expression used to detect which files have a pre-compressed version
    • encodings: a list of objets with a name property which is the encoding name and path property which is a string (or function returning a string) containing the path of the compressed file (the regular expression groups from matcher will be replaced in the string).

    undefined by default

    Example:

    new FileSystemStorage(
      directory,
      {
        contentEncodingMappings: [
          {
            matcher: /^(.+\.(?:html|js|css))$/,
            encodings: [
              { name: 'br', path: '$1.br' },
              { name: 'gzip', path: '$1.gz' }
            ]
          }
        ]
      }
    )

    ignorePattern

    The storage will ignore files which have any parts of the path matching this pattern.

    false will disable it

    Defaults to /^\./ (files/folders beginning with a dot)

    Example:

    new FileSystemStorage(
      directory,
      { ignorePattern: /^myPrivateFolder$/ }
    )

    fsModule

    Let you override the fs module that will be used to retrieve files.

    Example:

    const memfs = require('memfs');
    memfs.fs.writeFileSync('/hello.txt', 'world');
    new FileSystemStorage(
      directory,
      { fsModule: memfs }
    )

    onDirectory

    Determine what should happen on directory requests (trailing slash)

    • false to return an error
    • 'list-files' to list the files of directories
    • 'serve-index' to serve the index.html file of directories

    Defaults to false

    Note that you can customize the html template used for 'list-files' by overiding getDirectoryListing method.

    Example:

    new FileSystemStorage(
      directory,
      { onDirectory: 'list-files' }
    )

    storage.prepareResponse(path, req, [options])

    Create asynchronously a new StreamResponse for the given path relative to root ready to be sent to a server response.

    Note that you can use storage.send(path, req, res, [options]) instead if you don't need to treat the response before sending.

    • The path parameter is a urlencoded path (urlencoded) or an array of path parts (should always start with '').

      Query params will be ignored if present.

      For example, '/my%20directory/index.html?param=value' is the equivalent of ['', 'my directory', 'index.html'].

    • The req is the related request, it can be a http.IncomingMessage or a http2.Http2ServerRequest or a http2.IncomingHttpHeaders.

    • The options parameter let you add some addition options:

    cacheControl

    Custom cache-control header value, overrides storage value (public, max-age=0 by default)

    false to remove header

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { cacheControl: 'public, max-age=31536000' }
    )

    lastModified

    Custom last-modified header value, overrides storage value (defaults to mtimeMs converted to UTC)

    false to remove header

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { lastModified: 'Wed, 21 Oct 2015 07:28:00 GMT' }
    )

    etag

    Custom etag header value, overrides storage value (defaults to size + mtimeMs + content encoding)

    false to remove header

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { etag: '"123"' }
    )

    mimeType

    Custom mime type for content-type header value, overrides storage value (defaults to storage content type)

    false to remove header

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { mimeType: 'text/plain' }
    )

    mimeTypeCharset

    Custom content-type charset value, overrides storage value (defaults to storage content type charset mapping)

    false to remove charset

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { mimeTypeCharset: 'UTF-8' }
    )

    contentDispositionType

    Custom content-disposition header type value, overrides storage value

    false to remove header

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { contentDispositionType: 'attachment' }
    )

    contentDispositionFilename

    Custom content-disposition header filename value, overrides storage value

    false to remove filename from header

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { contentDispositionFilename: 'file.txt' }
    )

    statusCode

    Defines the statusCode that will be used in response (instead of 200/206) Setting this will disable conditional GET and partial responses

    undefined by default

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { statusCode: 404 }
    )

    allowedMethods

    By default GET and HEAD are the only allowed http methods, set this parameter to change allowed methods

    ['GET', 'HEAD'] by default

    Example:

    await storage.prepareResponse(
      req.url,
      req,
      { allowedMethods: ['POST'] }
    )

    storage.send(path, req, res, [options])

    The parameters are the same as for storage.prepareResponse(path, req, [options]) and streamResponse.send(res, [options]) and the result is the same as if you call the two methods consecutively.

    You may prefer to use this method if you don't need to use the stream response.


    streamResponse.statusCode

    The status code that match the required resource.

    For example, it can be 200 if the file is found, 206 for a range request, 404 if the file does not exists, ...


    streamResponse.headers

    The headers that match the required resource.


    streamResponse.error

    If an error occured while reading (e.g. if the file is not found), a 404 status is returned and this property will contains the error which have been returned by the storage. See errors


    streamResponse.send(res, [options])

    Send the current response through the response in parameter

    • The res parameter is the related response, it can be a http.ServerResponse or a http2.Http2ServerResponse or a http2.ServerHttp2Stream.

    • The options parameter let you add some addition options:

    ignorePrematureClose

    Ignore premature close errors, if any error occurs while sending, it will be ignored

    Defaults to true (since clients can stop transfer anytime, forwarding errors can be annoying)


    streamResponse.dispose()

    Disposes of resources within the stream response object


    Errors

    StorageError

    All errors inherits from this one.

    The following property is available:

    • reference: the storage reference linked to the error

    MethodNotAllowedStorageError (extends StorageError)

    The HTTP method is not allowed (405 HTTP code)


    PreconditionFailedStorageError (extends StorageError)

    The precondition set in headers have failed (412 HTTP code)


    RangeNotSatisfiableStorageError (extends StorageError)

    The ranges set in headers are not statisfiable (416 HTTP code)


    FileSystemStorageError (extends StorageError)

    All errors from FileSystemStorage inherits from this one.

    The following additional property is available:

    • pathParts: the path parts linked to the error

    MalformedPathError (extends FileSystemStorageError)

    The path cannot be parsed for some reason.


    NotNormalizedPathError (extends FileSystemStorageError)

    Storages only accept normalized paths for security reasons.

    For example, '/../index.html' access will be refused.

    The following additional property is available:

    • normalizedPath: the encoded normalized path (you can redirect to it if you want to)

    ForbiddenCharacterError (extends FileSystemStorageError)

    Storages refuse some forbidden characters like encoded slashes.


    ConsecutiveSlashesError (extends FileSystemStorageError)

    Storages refuse pathes like '/dir//index.html' because it should not contain two consecutive slashes.


    TrailingSlashError (extends FileSystemStorageError)

    Storages refuse pathes like '/dir/' because it is probably pointing to a directory.


    IgnoredFileError (extends FileSystemStorageError)

    Storages can ignore some files/folders composing the path (see ignorePattern).


    IsDirectoryError (extends FileSystemStorageError)

    Storages refuse pathes matching directories.

    The following additional property is available:

    • resolvedPath: the resolved file system path

    DoesNotExistError (extends FileSystemStorageError)

    When the file can not be found.

    The following additional property is available:

    • resolvedPath: the resolved file system path

    Other source of data

    You can extend the class Storage if you want to stream files from something else than the file system. Take a look at this MongoDB/GridFS storage example to have an example.

    You can also take a look to this this Buffer storage example or this JSON storage example to serve data directly from your current route.

    If you want to implement something nearer from file system you can also take a look at the GenericFileSystemStorage class. Take a look at this memory cache storage example to have an example.


    Examples

    See examples folder in this repository for full examples

    Serve files

    const storage = new FileSystemStorage(directory);
    
    ...
    
    let result = await storage.prepareResponse(
      req.url,
      req
    );
    await result.send(res);

    Serve files with directory index.html

    const storage = new FileSystemStorage(
      directory,
      { onDirectory: 'serve-index' }
    );
    
    ...
    
    let result = await storage.prepareResponse(
      req.url,
      req
    );
    await result.send(res);

    Serve files with directory listing

    const storage = new FileSystemStorage(
      directory,
      { onDirectory: 'list-files' }
    );
    
    ...
    
    let result = await storage.prepareResponse(
      req.url,
      req
    );
    await result.send(res);

    Serve files and add CORS headers

    const storage = new FileSystemStorage(directory);
    
    ...
    
    let result = await storage.prepareResponse(
      req.url,
      req
    );
    result.headers['Access-Control-Allow-Origin'] = '*';
    result.headers['Access-Control-Allow-Headers'] = 'Origin, X-Requested-With, Content-Type, Accept, Range';
    await result.send(res);

    Serve one file specifically

    const storage = new FileSystemStorage(directory);
    
    ...
    
    let result = await storage.prepareResponse(
      '/index.html',
      req
    );
    await result.send(res);

    Serve index.html instead of 404 for history.pushState applications

    const storage = new FileSystemStorage(directory);
    
    ...
    
    let result = await storage.prepareResponse(
      req.url,
      req
    );
    // if path is not found then rewrite to root index.html
    if (result.error instanceof FileSystemStorageError) {
      result = await storage.prepareResponse(
        '/index.html',
        req
      );
    }
    await result.send(res);

    Serve files and add CSP (Content-Security-Policy) header when content is html

    const storage = new FileSystemStorage(directory);
    
    ...
    
    let result = await storage.prepareResponse(
      req.url,
      req
    );
    if (
      !result.error
      && result.storageInfo?.mimeType === 'text/html'
    ) {
      result.headers['Content-Security-Policy'] = "script-src 'self'";
      // you can also add some other security headers:
      // result.headers['X-Frame-Options'] = "SAMEORIGIN";
      // result.headers['Referrer-Policy'] = "no-referrer";
      // result.headers['Feature-Policy'] = "...";
    }
    await result.send(res);

    License

    MIT

    Install

    npm i send-stream

    DownloadsWeekly Downloads

    15

    Version

    2.5.5

    License

    MIT

    Unpacked Size

    177 kB

    Total Files

    33

    Last publish

    Collaborators

    • icewil