lvr_bulk_downloader
TypeScript icon, indicating that this package has built-in type declarations

0.5.8 • Public • Published

LVR Bulk Downloader

Easy downloading...


Tested with 17000 files in input, about 17GB downloaded without an error.


Features

  • Single file or bulk download;

  • Retry download if failed;

  • Output directory flat or tree directories;

  • Sync or Async download;

  • Output response for all files downloaded;



Installation


   npm i lvr_bulk_downloader --save

Import


   import { Downloader, DownloaderTypes } from 'lvr_bulk_downloader';

Simple bulk downalod use


    const files =  [ 
        new DownloaderTypes.InputFile('http://host.domain/dir/file.ext'), 
        new DownloaderTypes.InputFile('http://host.domain/dir/file2.ext')
        ];
    const downloader = new Downloader();
    downloader.run(files);

Complex bulk download use


    const files =  [ 
        new DownloaderTypes.InputFile('http://host.domain/dir/file.ext', './download/file.ext'), 
        new DownloaderTypes.InputFile('http://host.domain/dir/file2.ext', './download/dir/file2.ext'), 
        new DownloaderTypes.InputFile('http://host.domain/dir/file3.ext', './download/dir/file3.ext')
        ];
    
    const options = new DownloaderTypes.Options();
    options.output_directory = './download/';
    options.debug_mode = DownloaderTypes.DebugMode.DEBUG;
    options.retry_times = 3;

    const downloader = new Downloader(options);
    downloader.run(files, function(output: DownloaderTypes.OutputFile[]){
        const downloadedCount = output.filter(x=>x.response?.status == DownloaderTypes.Status.OK).length;
        const errorCount = output.filter(x=>x.response?.status == DownloaderTypes.Status.KO).length;
    });

For sync download adding await to run function


    const files =  [ 
        new DownloaderTypes.InputFile('http://host.domain/dir/file.ext'), 
        new DownloaderTypes.InputFile('http://host.domain/dir/file2.ext')
        ];
    const downloader = new Downloader();
    await downloader.run(files);

Single file download use


    var file = new DownloaderTypes.InputFile('http://host.domain/dir/file.ext');
    var downloader = new Downloader();
    downloader.run(file);



Types definition

Types are contained in module DownloaderTypes


InputFile

url: string

output_path: string | null //This attribute overwrite output_directory defined on options only for this file.


OutputFile

url: string

path: string | null

retry_times: number

response: Response | null


Options

retry_times: number = 3

debug_mode: DebugMode = DebugMode.LOG

output_directory: string = './download/'


Response

status: Status;

message: any;


enum Status{
    OK,
    KO
}

enum DebugMode{
    NONE,
    DEBUG,
    LOG
}

Package Sidebar

Install

npm i lvr_bulk_downloader

Weekly Downloads

1

Version

0.5.8

License

ISC

Unpacked Size

37.5 kB

Total Files

11

Last publish

Collaborators

  • corradoamatore