readlines-iconv

Description

Handler that returns a file line by line synchronously or asynchronously.
It includes per default a evaluation of the end of line character of the file (CRLF, LF, CR). It supports a lot of possible encodings thankfully to the integrated iconv-lite module.

See for supported encodings on the iconv-lite module page or all supported encodings on their wiki.

Installation

npm i readlines-iconv

Basic Usage

The readlines-iconv module can be loaded using ESM:

import { ReadLinesSync, ReadLinesAsync } from 'readlines-iconv';

Synchronous example

It is synchronously, but it only reads in (per next()) either the specified bytes from minBuffer (default: 16384) or the multiples of these specified bytes, until it reaches a valid line ending or the end of the file. So, it is blocking for the minimum amount of time to evaluate at least one line.

First you need to integrate readlines-iconv into your application:

const lineHandler = new ReadLinesSync(options);
fileHandler.open('./directory/someFile.txt');

Each time when you execute next() it will return one line of the file using the Iterator Protocol:

let line = lineHandler.next();
console.log(line.value) // Returns line

The next() method will return a string for the property value for each line.
It will return { done: true } in case the end of file is reached.

It will autommatically close the file handle at the end of the file. But you can close the file handle at any time manually (And you should):

lineHandler.close();

Note: subsequent next() calls will return { done: true }

You can use the handler inside of a for loop:

for(const line of lineHandler) {
	console.log(line); // Line after each other
}

After you closed the file, you are able to open a new file without the need to create a new instance using open(). But this will only work if the options and line ending of all files opened are the same.

Promise/Asynchronous example

Each of the functions returns a promise, but handling is basically the same as for the synchronous version. It only reads in (per next()) either the specified bytes from minBuffer (default: 16384) or the multiples of these specified bytes, until it reaches a valid line ending or the end of the file.
It can be used using Promise prototype methods like .then() or .catch().
Or simply by using async and await. next()

First you need to integrate readlines-iconv into your application:

const lineHandler = new ReadLinesAsync(options);
await fileHandler.open('./directory/someFile.txt');

Each time when you execute next() it will return one line of the file using the Iterator Protocol:

let line = await lineHandler.next();
console.log(line.value) // Returns line

The next() method will return a string for the property value for each line.
It will return { done: true } in case the end of file is reached.

It will autommatically close the file handle at the end of the file.
But you can close the file handle at any time manually:

await lineHandler.close();

Note: subsequent next() calls will return { done: true }

You can use the handler inside of a for loop:

for await (const line of lineHandler) {
	console.log(line); // Line after each other
}

After you closed the file, you are able to open a new file without the need to create a new instance.
But this will only work if the options and line ending of all files opened are the same.

Configuring readlines-iconv

The configuration is optional. Without any manual configuration, readlines-iconv tries to use reasonable defaults.
However, sometimes you may need to change it's configuration.

You can apply a configuration when starting a new instance of readlines-iconv by providing an object.

const lineHandler = new ReadLinesSync(filePath, options);

Options

The object can have following options:

options.encoding

Type: string Default: utf8

See for supported encodings on the iconv-lite module page or all supported encodings on their wiki.

options.minBuffer

Type: number Default: 16384 (16 kiB)

Defines the minimum number of bytes to read from file per next execution.
The best fit for the amount of the minimum used memory and performance, would be the averange bytes each of the lines has.
If it is to small it will slow down the overall perofrmance.
16384 is the default from fs.read

options.newLineCharacter

Type: null | '\n' | '\r\n' | '\r' Default: null

readlines-iconv tries to evaluate the specific line ending by itself.
To explicitly set the line ending, use the newLineCharacter property (e.g. for Windows: \r\n, Unix: \n, legacy OSX: \n).
null means it will be automatically evaluated.

License

Copyright (c) 2023-2024 Marina Egner (sheepcs.de). This is free software and may be redistributed under the terms specified in the LICENSE file.