@arkie-ai/milvus-client

Milvus Node SDK

This repository is a node client for milvus service. It uses gRPC to communicate with milvus server with protobuf which is a mechanism for serializing structured data.

Getting started

Assuming you are already familiar with milvus, this repository provides the some interfaces that you can use to communicate with the server of milvus.

Prerequisites

Before you start to use this package, it is recommended that you learn something about gRPC and Milvus so that you can figure out how the flow works.

Installation

Install the milvus node sdk package

npm install @arkie/milvus-client

Interface

There are some important interfaces that you need to know.

Status

Because it's a communication between client and server, every response comes from server will indicate that if the server is down or the operation is successful or not.

interface Status {
    error_code: string      // error code
    reason: string          // the reason why error happens, would be empty if there are no errors
}

Error Code

There are some error codes that indicates the error briefly. If the operation is failed, the status will carry error_code and reason to tell the user why this operation fails.

enum ErrorCode {
    SUCCESS = 0;				// success operation
    UNEXPECTED_ERROR = 1;
    CONNECT_FAILED = 2;
    PERMISSION_DENIED = 3;
    TABLE_NOT_EXISTS = 4;
    ILLEGAL_ARGUMENT = 5;
    ILLEGAL_RANGE = 6;
    ILLEGAL_DIMENSION = 7;
    ILLEGAL_INDEX_TYPE = 8;
    ILLEGAL_TABLE_NAME = 9;
    ILLEGAL_TOPK = 10;
    ILLEGAL_ROWRECORD = 11;
    ILLEGAL_VECTOR_ID = 12;
    ILLEGAL_SEARCH_RESULT = 13;
    FILE_NOT_FOUND = 14;
    META_FAILED = 15;
    CACHE_FAILED = 16;
    CANNOT_CREATE_FOLDER = 17;
    CANNOT_CREATE_FILE = 18;
    CANNOT_DELETE_FOLDER = 19;
    CANNOT_DELETE_FILE = 20;
    BUILD_INDEX_ERROR = 21;
    ILLEGAL_NLIST = 22;
    ILLEGAL_METRIC_TYPE = 23;
    OUT_OF_MEMORY = 24;
}

How to use

Notice that every function is async except clientVersion.

// import the package
// init the client with host and port
const Milvus = require('milvus-client')
const client = new Milvus.MilvusClient('localhost', 9000)

clientVersion

You can get the version of the client which is "0.6.0" for now.

Note: it's not the package version, but the client version.

const clientVersion = client.clientVersion()

serverStatus

Use this function to get the status of the server to see if the server is running or not.

// response would be look like this
// {
//     status: {
//         error_code: 'SUCCESS',
//         reason: ''
//     },
//     string_reply: 'OK'
// }
const serverStatus = await client.serverStatus()

serverVersion

Use this function to get the version of the server.

// response would be look like this
// {
//     status: {
//         error_code: 'SUCCESS',
//         reason: ''
//     },
//     string_reply: '0.6.0'
// }
const serverVersion = await client.serverVersion()

serverMode

Use this function to see if the server is running with CPU or GPU.

// response would be look like this
// {
//     status: {
//         error_code: 'SUCCESS',
//         reason: ''
//     },
//     string_reply: 'CPU'
// }
const serverMode = await client.serverMode()

createTable

Create a table in milvus named 'testTable' whose vector's dimension is 5 and indexFileSize is 1GB and metric_type is L2 (L2 or IP).

// response: Status
// { error_code: 'SUCCESS', reason: '' }
const createTableResponse = await client.createTable({
    table_name: 'testTable', // table's name
    dimension: 5,	         // dimension of table's vector
    index_file_size: 1,      // must be a positive value
    metric_type: 1, 		 // L2 = 1, IP = 2
})

showTables

Show all tables in milvus server

// response
// { status: Status, table_names: string[] }
const tables = await client.showTables()

dropTable

Drop a table

// response
// { error_code: 'SUCCESS', reason: '' }
const dropResponse = await client.dropTable('testTable')

hasTable

Check out if there is a table by the name of 'testTable' in the server

// response
// { error_code: 'SUCCESS', reason: '' }
const hasTable = await client.hasTable('testTable')

countTable

Get table row count

/**
 * @brief Get table row count
 *
 * This method is used to get table row count.
 *
 * @param table_name, target table's name.
 * @return Indicate if this operation is successful and row_count
 * 
 * response example:
 * { status: Status, table_row_count: number }
 */
const rowCountResponse = await client.countTable('testTable')

describeTable

Describe information about a table

// response
// {
//   status: Status,
//   table_name: string,
//   dimension: number,
//   index_file_size: number,
//   metric_type: number
// }
//
// 
const tableInformation = await client.describeTable('testTable')

createIndex

Create index in a table

Note: there is a default index method called 'idmap'

/**
 * @brief Create index method
 *
 * This method is used to create index for whole table(and its partitions).
 *
 *  @table_name: table name is going to be create index.
 *  @index_type: 0-invalid, 1-idmap, 2-ivflat, 3-ivfsq8, 4-nsgmix
 *  @nlist: index nlist
 *
 * @return Status Indicate if build index successfully.
 */
const createIndexResponse = await client.createIndex({
  table_name: 'testTable',
  index: {
    index_type: 2,
    nlist: 5,
  }
})

describeIndex

Describe the index of a table

// response
interface Index {
  index_type: number
  nlist: number
}
// { table_name: string, status: Status, index: Index, index_name: string}
const indexDescription = await client.describeIndex('testTable')

dropIndex

Drop the index of the table except default index idmap

// response
// { error_code: 'SUCCESS', reason: '' }
const dropIndexResponse = await client.dropIndex('testTable')

insert

Insert some vectors into a table

/**
 * @brief Insert vector to table
 *
 * This method is used to insert vector array to table.
 *
 * @param table_name, target table's name.
 * @param partition_tag, target partition's tag, keep empty if no partition.
 * @param row_record_array, vector array is inserted.
 * @param row_id_array, optional
 *  specify id for each vector,
 *  if this array is empty, milvus will generate unique id for each vector,
 *  and return all ids by this parameter.
 *
 * @return Indicate if vector array are inserted successfully
 *  and if the operation is successful, then the ids of vectors will be returned too
 *
 * resposne example:
 * {
 *   status: { error_code: 'SUCCESS', reason: '' },
 *   vector_id_array: [ 123891238129, 2543534523 ] // if two vectors are inserted
 * }
 */

const insertResponse = await client.insert({
    table_name: 'testTable',
    partition_tag: '',
    row_record_array: [{
        vector_data: [1, 2, 3, 4, 5],
    }, {
        vector_data: [5, 4, 3, 2, 1],
    }]
})

search

Query similar vectors in the table 'testTable'

/**
 * @brief Search vector
 *
 * This method is used to query vector in table.
 *
 * @param table_name, target table's name.
 * @param partition_tag_array, target partitions, keep empty if no partition.
 * @param query_record_array, all vector are going to be queried.
 * @param query_range_array, [deprecated] time ranges, if not specified, will search in whole table
 * @param topk, how many similarity vectors will be searched.
 * @param nprobe, the number of centroids choose to search.
 *
 * @return Indicate if query is successful and vectors' id will be returned
 *
 * response example:
 * {
 *   status: { error_code: 'SUCCESS', reason: '' },
 *   ids: [ 12839129314, 2384572385 ],
 *   distances: [ 123.54, 1234.54 ],
 *   row_num: 1
 * }
 */
const searchResponse = await client.search({
	table_name: 'testTable',
    query_record_array: [{
        vector_data: [1, 2, 3, 4, 5],
    }],
    topk: 2,
    nprobe: 2,
    partition_tag_array: [],
    query_range_array: [],
})

deleteByDate

Delete data between two date

/*
 * @table_name: table's name
 * @range: date range, use 'YYYY-MM-DD' format
 *
 * @response Status
 * { error_code: 'SUCCESS', reason: '' }
 */
const deleteResponse = await client.deleteByDate({
    table_name: 'testTable',
    range: {
        start_value: '2020-01-01',
        end_value: '2020-02-01'
    }
})

preloadTable

This method is used to preload the table so that you can insert and search quickly.

// response
// { error_code: 'SUCCESS', reason: '' }
const preloadResponse = await client.preloadTable('testTable')

createPartition

Create a partition for a table so that you can search in some partitions lately.

// response
// { error_code: 'SUCCESS', reason: '' }
// @param type PartitionParam
interface PartitionParam {
    table_name: string
    partition_name: string
    tag: string
}
const createPartitionResponse = await client.createPartition({
    table_name: 'testTable',
    partition_name: 'partition1',
    tag: 'tag1',
})

showPartitions

Show all partitions in a table

// response
// { status: Status, partition_array: PartitionParam[] }
const partitions = await client.showPartitions('testTable')

dropPartition

Drop the partition that you don't want any more in a table

// response
// { error_code: 'SUCCESS', reason: '' }
const dropPartitionResponse = await client.dropPartition({
    table_name: 'testTable',
    partition_name: 'partition1',
    tag: 'tag1'
})

cmd

You can go to milvus web site to see if there are some command that you can send to the server such as 'get_config', 'mode', 'status'...

// parameter is a string ---- command
// response
// { status: Status, stringReply: string }
const cmdResponse = await client.cmd('version')

Version

The SDK corresponds to version 0.6.0 of the service.

Keywords

Milvus gRPC ProtoBuf Typescript Vector Search

License