- Supports Pilosa Server v0.4.0.
- Breaking Change: Changed default row ID label to
rowIDand default column ID to
- Updated the accepted values for index, frame names and labels to match with the Pilosa server.
Unionqueries accept 0 or more arguments.
Differencequeries accept 1 or more arguments.
- Inverse enabled status of frames is not checked on the client side.
httpsscheme is allowed.
- Initial version.
- Supports Pilosa Server v0.3.2.
- NodeJS 4 or later
- (Optional) Typescript 2.3 and higher
Pilosa client is available as an npm package. You can install the library using:
npm install --save pilosa
Assuming Pilosa server is running at
localhost:10101 (the default):
var pilosa = ;// Create the default clientvar client = ;// Create an Index objectvar myindex = "myindex";// Create a Frame objectvar myframe = myindexframe"myframe";// Make sure the index exists on the serverclient;
Quick overview (Typescript using async/await)
;main.catchconsole.log"ERROR: ", err;
Data Model and Queries
Indexes and Frames
Index and frames are the main data models of Pilosa. You can check the Pilosa documentation for more detail about the data model.
Index constructor is used to create an index object. Note that this does not create an index on the server; the index object simply defines the schema.
var repository = "repository"
Indexes support changing the column label and time quantum.
IndexOptions objects store that kind of data. In order to apply these custom options, pass an
IndexOptions object as the second argument to
var options =columnLabel: "repo_id"timeQuantum: pilosaTimeQuantumYEAR_MONTHvar repository = pilosa;
Frames are created with a call to
var stargazer = repositoryframe"stargazer";
Similar to index objects, you can pass custom options to the
var options =rowLabel: "stargazer_id"timeQuantum: pilosaTimeQuantumYEAR_MONTH_DAYvar stargazer = repositoryframe"stargazer" options;
Once you have indexes and frame objects created, you can create queries for them. Some of the queries work on the columns; corresponding methods are attached to the index. Other queries work on rows, with related methods attached to frames.
Bitmap queries work on rows; use a frame object to create those queries:
var bitmapQuery = stargazer; // corresponds to PQL: Bitmap(frame='stargazer', stargazer_id=1)
Union queries work on columns; use the index object to create them:
var query = repository;
In order to increase througput, you may want to batch queries sent to the Pilosa server. The
index.batchQuery method is used for that purpose:
var query = repository;
The recommended way of creating query objects is, using dedicated methods attached to index and frame objects. But sometimes it would be desirable to send raw queries to Pilosa. You can use the
index.rawQuery method for that. Note that, query string is not validated before sending to the server:
var query = repository;
Check Pilosa documentation for PQL details. Here is a list of methods corresponding to PQL calls:
union(...bitmaps: Array<PqlBitmapQuery>): PqlBitmapQuery
intersect(...bitmaps: Array<PqlBitmapQuery>): PqlBitmapQuery
difference(...bitmaps: Array<PqlBitmapQuery>): PqlBitmapQuery
count(bitmap: PqlBitmapQuery): PqlQuery
setColumnAttrs(columnID: number, attrs: AttributeMap): PqlBitmapQuery
bitmap(rowID: number): PqlBitmapQuery
inverseBitmap(columnID: number): PqlQuery
setBit(rowID: number, columnID: number, timestamp?: Date): PqlQuery
clearBit(rowID: number, columnID: number): PqlQuery
topN(n: number, bitmap?: PqlBitmapQuery, field?: string, ...values: Array<any>): PqlBitmapQuery
inverseTopN(n: number, bitmap?: PqlBitmapQuery, field?: string, ...values: Array<any>): PqlBitmapQuery
range(rowID: number, start: Date, end: Date): PqlBitmapQuery
inverseRange(columnID: number, start: Date, end: Date): PqlBitmapQuery
setRowAttrs(rowID: number, attrs: AttributeMap): PqlBitmapQuery
A Pilosa URI has the
- Scheme: Protocol of the URI. Default:
- Host: Hostname or ipv4/ipv6 IP address. Default: localhost.
- Port: Port number. Default:
All parts of the URI are optional, but at least one of them must be specified. The following are equivalent:
A Pilosa URI is represented by the
pilosa.URI class. Below are a few ways to create
// create the default URI:var uri1 =// create a URI from string addressvar uri2 = pilosaURI// create a URI with the given host and portvar URI uri3 = host="db1.pilosa.com" port=20202;
In order to interact with a Pilosa server, an instance of
pilosa.Client should be created. We recommend creating a single instance of the client and share it with other objects when necessary.
If the Pilosa server is running at the default address (
http://localhost:10101) you can create the default client with default options using:
var client =
To use a a custom server address, pass the address in the first argument:
var client = ""
If you are running a cluster of Pilosa servers, you can create a
pilosa.Cluster object that keeps addresses of those servers:
var cluster =pilosaURIpilosaURIpilosaURI;// Create a client with the clustervar client = cluster
Once you create a client, you can create indexes, frames and start sending queries. All client methods return a
Here is how you would create an index and frame:
// materialize repository index instance initialized beforeclient;
try// materialize repository index instance initialized beforeawait client;// materialize stargazer frame instance initialized beforeawait client;// actions on the frame...catch e// act on the error
If the index or frame exists on the server, you will receive a
PilosaError. You can use
ensureFrame methods to ignore existing indexes and frames.
You can send queries to a Pilosa server using the
query method of client objects.
var response = await client;// act on the response
query method accepts an optional argument of type
var queryOptions =columns: true // return column data in the responseclient;
When a query is sent to a Pilosa server, the server either fulfills the query or sends an error message. In the case of an error,
PilosaError is returned, otherwise a
QueryResponse object is returned.
QueryResponse object may contain zero or more results of
QueryResult type. You can access all results using the
results property of
QueryResponse (which returns a list of
QueryResult objects) or you can use the
result property (which returns either the first result or
null if there are no results):
QueryResponse object may include a number of column objects, if
columns=true query option was used:
// check that there's a column object and act on itvar column = responsecolumnif column// act on the column// iterate on all columnsresponsecolumns
QueryResult objects contain:
bitmapproperty to retrieve a bitmap result,
countItemsproperty to retrieve column count per row ID entries returned from
countproperty to retrieve the number of rows per the given row ID returned from
var bitmap = responsebitmapvar bits = bitmapbitsvar attributes = bitmapattributesvar countItems = responsecountItemsvar count = responsecount
- Fork this repo and add it as upstream:
git remote add upstream firstname.lastname@example.org:yuce/js-pilosa.git.
- Make sure all tests pass (use
make test-all) and be sure that the tests cover all statements in your code (we aim for 100% test coverage).
- Commit your code to a feature branch and send a pull request to the
masterbranch of our repo.
The sections below assume your platform has
make. Otherwise you can view the corresponding steps of the
You can run unit tests with:
And both unit and integration tests with:
Generating protobuf classes
Protobuf classes are already checked in to source control, so this step is only needed when the upstream
Before running the following step, make sure you have the Protobuf compiler installed:
Copyright 2017 Yuce Tekol Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.