Naughty Platypus Memorabilia

    @tsrt/mparty
    TypeScript icon, indicating that this package has built-in type declarations

    0.8.0 • Public • Published

    Typescript Reusable Tools: Mparty

    npm version GitHub license Size Downloads

    Mparty provides async/await API for handling multipart/form-data (in most cases for file uploading) using default or custom Adapter to store files (FileSystem, Memory, Aws or whatever is needed).

    It is build on top of busboy.

    Features

    • Ability to provide custom Adapter.
    • Async/await.
    • Typings.
    • Ability to use not only as middleware.
    • Framework independent.
    • Shipped w/ 2 default Adapters: FileSystem, Memory.
    • Customizable via options.
    • File validations (not only those Busboy provides).
    • Easy and convenient usage for multi-tenant systems.

    Plugins

    Usage

    For example in some controller (Here is used awesome TsED Framework):

    import { Mparty } from '@tsrt/mparty';
    
    ...
    
    @Post('/')
    public uploadFiles(@Request() req: Request) {
      try {
        const mparty = new Mparty({ destination: 'some/path' });
        const { fields, files, file } = mparty.upload(req); // `file` property will be available only if there were uploaded 1 file
        ...
      } catch (err) {
        ...
      }
    }

    Or in well known Express as middleware: (Note, you can use @tsrt/mparty-express)

    import { Request, Response, Next } from 'express';
    import { Mparty, FsAdapter, MemoryAdapter, IFileMetadata } from '@tsrt/mparty';
    
    export async function uploadMiddeware(req: Request, res: Response, next: Next): Promise<void> {
      try {
        const mparty = new Mparty({ destination: 'some/path' });
        const { fields, files, file } = mparty.upload(req); // `file` property will be available only if there were uploaded 1 file
        req.body = fields;
        req.files = files;
        req.file = file;
        ...
        next();
      } catch (err) {
        next(err);
      }
    }
    
    // Optionally update express typings if using Typescript
    declare module 'express' {
      interface Request {
        files: IFileMetadata[];
        file?: IFileMetadata; // `file` property will be available only if there were uploaded 1 file
      }
    }
    
    declare module 'express-serve-static-core' {
      interface Request {
        files: IFileMetadata[];
        file?: IFileMetadata; // `file` property will be available only if there were uploaded 1 file
      }
    }
    
    // And then
    ...
    router.post('/', uploadMiddeware, (req, res) => {
      // req.files
      // req.file
    });

    Use with default Adapters

    import { Mparty, FsAdapter, MemoryAdapter, IFileMetadata } from '@tsrt/mparty';
    
    const mparty = new Mparty({ destination: 'some/path' }); // If destination only provided the FsAdapter will be used by default.
    
    // Or
    const adapter = new FsAdapter({ destination: 'some/path' }) // const adapter = new MemoryAdapter()
    const mparty = new Mparty({ adapter });
    
    // It is also possible to provide options exactly to upload method
    mparty.upload(req, { adapter, limits: { ... }, ... });
    ... or create custom Adapter
    import { IncomingMessage } from 'http'; // Or import { Request } from 'express or other compatible
    import { IFileMetadata, IAdapter, IUploadResult } from '@tsrt/mparty';
    
    interface IMyFile extends IFileMetadata {
      someProperty: string;
    }
    
    export class MyAdapter implements IAdapter<IMyFile, IncomingMessage> {
      public async onUpload(
        req: IncomingMessage, file: NodeJS.ReadableStream,
        { fieldName, fileName, originalFileName, encoding, mimetype, extension }: IFileMetadata,
      ): Promise<IMyFile> {
        // ... logic for handling each file stream
        // Here you could use req, file stream and default file metadata
      }
    
      public async onRemove(req: IncomingMessage, uploadedResult: IUploadResult<IMyFile>): Promise<void> {
        // ... logic for deleting uploaded files
        // Here you could use req and already uplaodedResult: { fields, files, file? } 
      }
    }
    
    // And then
    
    const adapter = new MyAdapter();
    const mparty = new Mparty({ adapter });
    ...
    ... or imagine multi-tenant system, where you need to decide which config to use for each and separate request:

    (To simplify handling context in app i advise you consider using express-http-context or async_hooks)

    import { IncomingMessage } from 'http'; // Or import { Request } from 'express or other compatible
    import { IFileMetadata, IAdapter, IUploadResult } from '@tsrt/mparty';
    
    interface IMyFile extends IFileMetadata {
      someProperty: string;
    }
    
    interface IMyAdapterOptions {
      clientId: stirng;
      clientSecret: string;
      bucket: string;
    }
    
    /
    export class MyAdapter implements IAdapter<IMyFile, IncomingMessage> {
      constructor(private options: IMyAdapterOptions)
    
      public async onUpload(req: IncomingMessage, file: NodeJS.ReadableStream, fileMetadata: IFileMetadata): Promise<IMyFile> {
        // upload(this.options);
    
        // Or using, for example, express-http-context: upload(requestContext.get('config'));
      }
    
      public async onRemove(req: IncomingMessage, uploadedResult: IUploadResult<IMyFile>): Promise<void> {
        // remove(this.options);
      }
    }
    
    ...
    
    // Later in your middeware
    export async function mydMiddeware(req: Request, res: Response, next: Next): Promise<void> {
      try {
        const config: IMyAdapterOptions = await SomeService.getConfigByClinetIdFromRequest(req);
        const adapter = new MyAdapter({ config });
        const mparty = new Mparty({ adapter });
        const { fields, files, file } = mparty.upload(req);
        ...
        next();
      } catch (err) {
        next(err);
      }
    }

    Options

    // Options
    const options = {
      /** Adapter to be used for file upload */
      adapter?: IAdapter<T>;
    
      /** If no adapter provided and provided a destionation - FsAdapter will be used for provided destionation */
      destination?: string;
    
      /**
       *  Files filter, which is called before each file upload.
       *  Here it is recommended to filter files is case of default Adapter usage
       *  (in case of custom adapter you can encapsulate it there)
       *
       *  Inspired by multer's @see https://www.npmjs.com/package/multer#filefilter. Thx guys, you are awesome.
       */
      filesFilter?: FilesFilter<T, Req>;
    
      /** Function for generating fileName for file. __Note__ that you re responsible for naming collisions */
      fileNameFactory?: FileNameFactory<T, Req>;
    
      /** Whether to throw an error on requests with application/json Content-Type. Default: false  */
      failOnJson?: boolean;
    
      /**
       *  Whether to remove uploaded files from storage on Error occured. Default: true.
       *  If `false` - already upload files metadata (before error occured) will be attached to MpartyError in `uploadedResult` field
       */
      removeOnError?: boolean;
    
      /**
       *  Busboy option. If paths in the multipart 'filename' field shall be preserved. (Default: false).
       *
       *  @see https://www.npmjs.com/package/busboy#busboy-methods
       */
      preservePath?: boolean;
    
      /**
       *  Busboy option. Various limits on incoming data
       *
       *  @see https://www.npmjs.com/package/busboy#busboy-methods
       */
      limits?: {
        // ===> Busboy validations:
        /** Max field name size (in bytes) (Default: 100 bytes). */
        fieldNameSize?: number;
    
        /** Max field value size (in bytes) (Default: 1MB). */
        fieldSize?: number;
    
        /** Max number of non-file fields (Default: Infinity). */
        fields?: number;
    
        /** For multipart forms, the max file size (in bytes) (Default: Infinity). */
        fileSize?: number;
    
        /** For multipart forms, the max number of file fields (Default: Infinity). */
        files?: number;
    
        /** For multipart forms, the max number of parts (fields + files) (Default: Infinity). */
        parts?: number;
    
        /** For multipart forms, the max number of header key=>value pairs to parse Default: 2000 (same as node's http). */
        headerPairs?: number;
    
        // ===> Additional multipart validations:
        /** Allowed files' extensions. Example: ['.png', '.pdf'] */
        extensions?: string[];
    
        /** Required files' fieldNames in form data */
        requiredFiles?: string[];
    
        /** Allowed files' fieldNames in form data */
        allowedFiles?: string[];
      };
    }

    Todo

    • [ ] Update allowedFiles signature to be as next: upload(['file1', ['files', min, max], ['files', 1, 5]])

    Disclaimer

    This module initially was created due to the reason that i did not manage to found file-upload packages which provides convenient ability to stream files in a multi-tenant way into AWS S3.

    The main purpose was to provide ability to create custom Adapters w/ async save/remove interface and ability to use it not only as middleware.

    There were also added some useful validation utilities for files validation.

    In first version it was based on multiparty. Later on it was rewritten to use busboy internally.

    While struggling with error handling and adopting busboy for usage I discovered that awesome multer which I used sometimes previously provides ability to create custom storage engine which indeed was what i really needed at the very beginnig of my path )

    Still a lot of work was done, and there are some existing code which depends on this, so i decided to publish it.

    License

    This project is licensed under the terms of the MIT license.

    Install

    npm i @tsrt/mparty

    DownloadsWeekly Downloads

    9

    Version

    0.8.0

    License

    MIT

    Unpacked Size

    62.3 kB

    Total Files

    63

    Last publish

    Collaborators

    • mopc