Ce plugin propose une série d'instructions transformer plusieurs formats text
(xml, json, cvs, etc.) en flux d'objets Javascript
npm install @ezs/basics
- BIBParse
- BUFObject
- CSVObject
- CSVParse
- CSVString
- FILELoad
- FILEMerge
- FILESave
- INIString
- JSONParse
- JSONString
- OBJCount
- OBJFlatten
- OBJNamespaces
- OBJStandardize
- TARDump
- TARExtract
- TXTConcat
- TXTInflection
- TXTObject
- TXTParse
- TXTSentences
- TXTZip
- URLConnect
- URLFetch
- URLPagination
- URLParse
- URLRequest
- URLStream
- URLString
- XMLConvert
- XMLParse
- XMLString
- ZIPExtract
Take a String
and split it at bibtext entry.
Input:
["@article{my_article,\ntitle = {Hello world},\n", "journal = \"Some Journal\"\n"]
Output:
["a", "b", "c", "d"]
Returns Object
Take Mixed
and produce Buffer.
For example, it's useful to send string to browser.
-
none
undefined
Returns Buffer
Take an Array
of arrays and transform rows into objects.
Each row (Array) is tranformed into an object where keys are the values of the first row.
See CSVParse.
Input:
[
["a", "b", "c"],
[1, 2, 3],
[4, 5, 6]
]
Output:
[{
"a": 1,
"b": 2,
"c": 3
}, {
"a": 4,
"b": 5,
"c": 6
}]
Tip: this is useful after a CSVParse, to convert raw rows into n array of Javascript objects
When several values of the first row are the same, produced keys are suffixed with a number.
Input:
[
["a", "a", "b", "b", "b"],
[1, 2, 3, 4, 5]
]
Output:
[{
"a1": 1,
"a2": 2,
"b1": 3,
"b2": 4,
"b3": 5
}]
-
none
undefined
Returns (Object | Array<Object>)
Take String
and parse it as CSV to generate arrays.
See:
Input:
"a,b,c\nd,e,d\n"
Output:
[
["a", "b", "c"],
["d", "e", "d"]
]
Tip: see CSVObject, to convert arrays of values to array of objects.
-
separator
String to indicate the CSV separator (optional, defaultauto
) -
quote
String to indicate the CSV quote. (optional, defaultauto
)
Take an array of objects and transform row into a string where each field is separated with a character.
The resulting string is CSV-compliant.
See CSVObject
Input:
[{
"a": 1,
"b": 2,
"c": 3
}, {
"a": 4,
"b": 5,
"c": 6
}]
Output:
a;b;c
1;2;3
4;5;6
-
format
String if set to "strict" the fields will be wrapped with double quote (optional, defaultstandard
) -
separator
String to indicate the CSV separator (optional, default";"
) -
header
Boolean first line contains key name (optional, defaulttrue
)
Returns String
Take Object
containing filename et throw content by chunk
[ fi1e1.csv, file2.csv ]
Script:
[use]
plugin = analytics
plugin = basics
[FILELoad]
location = /tmp
[CSVParse]
Output:
[
(...)
]
-
location
String Directory location (optional, defaultTMPDIR
) -
compress
Boolean Enable gzip compression (optional, defaultfalse
)
Returns Object
Take Object
or Buffer
and throw only one document
[ fi1e1.csv, file2.csv ]
Script:
[use]
plugin = basics
[FILELoad]
[FILEMerge]
[replace]
path = contentOfFile1AndFile2
value = self()
Output:
[
(...)
]
Returns Object
Take data, convert it to buffer and append it to file
Input:
[
{"a": "a"},
{"a": "b"},
{"a": "c" }
]
Script:
[FILESave]
location = /tmp
identifier = toto
Output:
[{ filename: "/tmp/toto", size: XXX, ... }]
-
location
String Directory location (optional, defaultTMPDIR
) -
identifier
String? File name -
content
String? Content to save instead of using input object -
jsonl
Boolean Save as json line (optional, defaultfalse
) -
compress
Boolean Enable gzip compression (optional, defaultfalse
)
Returns Object
Take Object
and generate INI
Take an array of ezs's statements in JSON, and yield an ezs script in a string.
Input:
[
{ "param": 1, "section": { "arg1": "a", "arg2": "b" } },
{ "param": 1, "section": { "arg1": "a", "arg2": "b" } },
{ "section": { "arg1": "a", "arg2": true } },
{ "sec1": { "arg1": "a", "arg2": [3, 4, 5] }, "sec2": { "arg1": "a", "arg2": { "x": 1, "y": 2 } } },
{ "secvide1": {}, "secvide2": {} },
]
Output:
param = 1
[section]
arg1 = a
arg2 = b
param = 1
[section]
arg1 = a
arg2 = b
[section]
arg1 = a
arg2 = true
[sec1]
arg1 = a
arg2 = [3,4,5]
[sec2]
arg1 = a
arg2 = {"x":1,"y":2}
[secvide1]
[secvide2]
Returns String
Parse a String
to JSON and generate objects.
See https://github.com/dominictarr/JSONStream
Input:
["{ \"a\": 1, \"b\": 3 }", "{ \"a\": 2, \"b\": 4 }"]
Script:
[JSONParse]
separator = b
Output:
[3, 4]
Input:
["{ \"a\": 1 }", "{ \"a\": 2 }"]
Output:
[1, 2]
-
separator
String to split at every JSONPath found (optional, default"*"
)
Returns Object
Take an Object
and generate a JSON string.
Input:
[{ "a": 1 }, { "b": 2 }]
Output:
"[{\"a\":1},{\"b\":2}]"
-
wrap
String every document is wrapped into an array (optional, defaulttrue
) -
indent
String indent JSON (optional, defaultfalse
)
Returns String
Count how many objects are received, and yield the total.
Input:
["a", "b", "c", "d"]
Output:
[4]
-
none
undefined
Returns Number
Flatten an Object
with a path delimiting character.
See https://www.npmjs.com/package/flat
Input:
[
{ "a": { "b": 1, "c": 2}},
{ "a": { "b": 3, "c": 4}}
]
Output:
[
{ "a/b": 1, "a/c": 2 },
{ "a/b": 3, "a/c": 4 }
]
-
separator
String choose a character to flatten keys (optional, default"/"
) -
reverse
Boolean unflatten instead of flatten keys (optional, defaultfalse
) -
safe
Boolean preserve arrays and their contents, (optional, defaultfalse
)
Returns Object
Take Object
and throw the same object, all keys parsed to replace namespaces with their prefixes
Note: You can also parse values for specific keys (keys containing references to other keys)
[
{
"http://purl.org/dc/terms/title": "Life is good",
"http://purl.org/ontology/places#Countryl": "France",
},
{
"http://purl.org/dc/terms/title": "The rising sun",
"http://purl.org/ontology/places#Country": "Japan",
},
{
"http://purl.org/dc/terms/title": "Dolce Vista",
"http://purl.org/ontology/places#Country": "Italy",
}
]
Script:
[use]
plugin = basics
[OBJNamespaces]
prefix = dc:
namespace = http://purl.org/dc/terms/
prefix = place:
namespace = http://purl.org/ontology/places#
Output:
[
{
"dc:title": "Life is good",
"place:Country": "France",
},
{
"dc:title": "The rising sun",
"place:Country": "Japan",
},
{
"dc:title": "Dolce Vista",
"place:Country": "Italy",
}
]
-
prefix
String? the alias for a namespace -
namespace
String? the namespace to substitute by a prefix -
reference
String a regex to find key that contains a namespace to substitute (optional, defaultnull
)
Returns Object
Standardize Object
s so that each object have the same keys.
Input:
[{ "a": 1, "b": 2},
{ "b": 2, "c": 3},
{ "a": 1, "c": 3}]
Output:
[{ "a": 1, "b": 2, "c": ""},
{ "b": 2, "b": "", "c": 3},
{ "a": 1, "b": "", "c": 3}]
-
none
undefined
Returns Object
Take all recevied objects and build a tar file
{
}
-
manifest
String? Location path to store files in the tarball -
location
String Location path to store files in the tarball (optional, defaultdata
) -
json
String Convert to JSON the content of each chunk (optional, defaulttrue
) -
extension
String Choose extension fo each file (optional, defaultjson
) -
additionalFile
String? Path to an additional file that will be add to tarball -
compress
Boolean Enable gzip compression (optional, defaultfalse
)
Take the content of a tar file, extract some files. The JSON object is sent to the output stream for each file. It returns to the output stream
{
"id": "file name",
"value": "file contents"
}
-
path
String Regex to select the files to extract (optional, default"**\/*.json"
) -
json
String Parse as JSON the content of each file (optional, defaulttrue
) -
text
Boolean The content of each file is converted to a string (otherwise it remains a buffer) (optional, defaulttrue
) -
compress
Boolean Enable gzip compression (optional, defaultfalse
)
Returns Array<{id: String, value: String}>
Concatenate all String
items into one string
Input:
["a", "b"]
Output:
["ab"]
-
none
undefined
Returns String
Take a String
and inflect it with or more transformers from this list
pluralize, singularize, camelize, underscore, humanize, capitalize,
dasherize, titleize, demodulize, tableize, classify, foreign_key, ordinalize
Input:
{ "id": 1, "value": "all job" }
Script:
[TXTInflection]
transform = pluralize
transform = capitalize
transform = dasherize
Output:
{ "id": 1, "value": "All-jobs" }
-
path
String path of the field to segment (optional, default"value"
) -
transform
String? name of a transformer
Meta
- deprecated: see https://inist-cnrs.github.io/ezs/#/plugin-strings?id=inflection see https://www.npmjs.com/package/inflection
Take an array of values and generate an array containing objects with the
given key
and matching value from the input array.
Input:
[1, "b"]
Output:
[{ "value": 1 }, { "value": "b" }]
-
key
String choose a the key name (optional, default"value"
)
Returns Object
Take a String
and split it at each separator found.
Input:
["a\nb\n", "c\nd\n"]
Output:
["a", "b", "c", "d"]
-
separator
String choose character which trigger the split (optional, default"\n"
)
Returns String
Take a String
and split it into an array of sentences.
Input:
{ "id": 1, "value": "First sentence? Second sentence. My name is Bond, J. Bond." }
Output:
{ "id": 1, "value": ["First sentence?", "Second sentence.", "My name is Bond, J. Bond."] }
-
path
String path of the field to segment (optional, default"value"
)
Meta
- deprecated: see https://inist-cnrs.github.io/ezs/#/plugin-strings?id=sentences
Take a String
and zip it.
Uses gzip algorithm to compress strings.
-
unzip
Boolean to Unzip input (optional, defaultfalse
)
Returns Buffer
Take an Object
and send it to an URL.
The output will be the returned content of URL.
Useful to send JSON data to an API and get results.
-
url
String? URL to fetch -
streaming
String Direct connection to the Object Stream server (disables the retries setting) (optional, defaultfalse
) -
json
String Parse as JSON the content of URL (optional, defaultfalse
) -
timeout
Number Timeout in milliseconds (optional, default1000
) -
noerror
Boolean Ignore all errors (optional, defaultfalse
) -
retries
Number The maximum amount of times to retry the connection (optional, default5
) -
encoder
String The statement to encode each chunk to a string (optional, defaultdump
)
Returns Object
Add a new field to an Object
, with the returned content of URL.
Or if no target is specified, the output will be the returned content of URL.
-
url
String? URL to fetch -
path
String? if present select value to send (by POST) -
target
String? choose the key to set -
json
String parse as JSON the content of URL (optional, defaultfalse
) -
timeout
Number timeout in milliseconds (optional, default1000
) -
mimetype
String mimetype for value of path (if presents) (optional, default"application/json"
) -
noerror
Boolean ignore all errors, the target field will remain undefined (optional, defaultfalse
) -
retries
Number The maximum amount of times to retry the connection (optional, default5
)
Returns Object
Take Object
and multiple it to make it one object per page
Input:
[{"q": "a"}]
Script:
[URLRequest]
url = https://api.search.net
[URLPagination]
total = get('total')
Output:
[
{
"q": "a",
"total": 22
"offset": 0,
"pageNumber": 1,
"totalPages", 3,
"maxPages": 1000,
"limit": 10
},
{
"q": "a",
"total": 22
"offset": 10,
"pageNumber": 2,
"totalPages", 3,
"maxPages": 1000,
"limit": 10
},
{
"q": "a",
"total": 22
"offset": 20,
"pageNumber": 3,
"totalPages", 3,
"maxPages": 1000,
"limit": 10
}
]
-
total
Number total to use for the pagination (optional, default0
) -
limit
Number limit to use to pagination (optional, default10
) -
maxPages
Number maxPages to use to pagination (optional, default1000
)
Returns Object
Take an URL String
, parse it and return Object
.
Fields of the returned object:
- href
- origin
- protocol
- username
- password
- host
- hostname
- port
- pathname
- search
- hash
URLString statement convert such an object to a string.
See:
Returns Object
Take Object
as parameters of URL, throw each chunk from the result
Input:
[{"q": "a"}]
Script:
[URLRequest]
url = https://api.search.net
Output:
[
{
"result": "a"
}
]
-
url
String? URL to fetch -
json
Boolean parse result as json (optional, defaulttrue
) -
target
String? choose the key to set -
timeout
Number Timeout in milliseconds (optional, default1000
) -
noerror
Boolean Ignore all errors, the target field will remain undefined (optional, defaultfalse
) -
retries
Number The maximum amount of times to retry the connection (optional, default5
) -
insert
String? a header response value in the result
Returns Object
Take String
as URL, throw each chunk from the result or
Take Object
as parameters of URL, throw each chunk from the result
Next examples use an API https://httpbin.org/get?a=n
returning
{ args: { "a": "n" }}
Input:
[{"a": "a"}, {"a": "b"}, {"a": "c" }]
Script:
[URLStream]
url = https://httpbin.org/get
path = .args
Output:
[{"a": "a"}, {"a": "b"}, {"a": "c" }]
Input:
[
"https://httpbin.org/get?a=a",
"https://httpbin.org/get?a=b",
"https://httpbin.org/get?a=c"
]
Script:
[URLStream]
path = .args
Output:
[{"a": "a"}, {"a": "b"}, {"a": "c" }]
-
url
String? URL to fetch (by default input string is taken) -
path
String choose the path to split JSON result (optional, default"*"
) -
timeout
Number Timeout in milliseconds (optional, default1000
) -
noerror
Boolean Ignore all errors, the target field will remain undefined (optional, defaultfalse
) -
retries
Number The maximum amount of times to retry the connection (optional, default5
)
Returns Object
Take an Object
representing an URL and stringify it.
See URLParse
Returns String
Convert each chunk as XML String to JSON Object
Input:
[
"<xml>A</xml>",
"<xml>B</xml>"
]
Output:
[
{ "xml": { "$t": "A" } },
{ "xml": { "$t": "B" } }
]
Input:
[
{ "x": { "a": 1 } },
{ "x": { "a": 2 } }
]
Output:
[
"<x a=\"1\"/>",
"<x a=\"2\"/>",
]
Input:
[
{ "x": { "a": 1 } },
{ "x": { "a": 2 } }
]
Output:
[
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<x a=\"1\"/>",
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<x a=\"2\"/>",
]
See https://www.npmjs.com/package/xml-mapping
-
invert
String change conversion (JSON to XML) (optional, defaultfalse
) -
prologue
String add XML prologue (optional, defaultfalse
)
Returns Object
Take String
as XML input, parse it and split it in multi document at each path found
Input:
<* ["<a><b>x</b><b>y</b></a>"]
Script:
[XMLParse]
separator: /a/b
Output:
["x", "y"]
See https://www.npmjs.com/package/xml-splitter
-
separator
String choose a character for flatten keys (optional, default"/"
)
Returns Object
Transform an Object
into an XML string.
Input:
[{ "$t": "a" }]
Output:
[
"<items><item>a</item></items>"
]
See XMLParse
-
rootElement
String Root element name for the tag which starts and close the feed (optional, default"items"
) -
contentElement
String Content element name for the tag which starts and closes each item (optional, default"item"
) -
rootNamespace
String? Namespace for the root tag (xmlns=) -
prologue
Boolean Add XML prologue<?xml
(optional, defaultfalse
)
Returns String
Take the content of a zip file, extract some files. The JSON object is sent to the output stream for each file. It returns to the output stream
{
"id": "file name",
"value": "file contents"
}
-
path
String Regex to select the files to extract (optional, default"**\/*.json"
)