Have ideas to improve npm?Join in the discussion! »

    feathers-solr
    TypeScript icon, indicating that this package has built-in type declarations

    2.11.0 • Public • Published

    feathers-solr

    Build Status Coverage Status dependencies Status Known Vulnerabilities Download Status

    A Feathers Solr Adapter. Require >= Solr 5.x.

    See this adapter in action

    Installation

    $ npm install feathers-solr --save
    

    Important: feathers-solr implements the Feathers Common database adapter API and querying syntax.

    Install a supported HTTP Client Fetch, Undici or use a different HTTP Client.

    $ npm install node-fetch --save
    

    API

    service([options])

    Returns a new service instance initialized with the given options.

    const service = require('feathers-solr');
    
    app.use('/gettingstarted', service({ id, Model, events, paginate }));

    Options:

    • Model (required) - HTTP Client (fetch, undici, or your custom).
    • name - The name of the Solr Core / Colelction.
    • defaultParams - This params added to all Solr request.
    • id (optional, default: 'id') - The name of the id field property.
    • commitStrategy - (optional, default: { softCommit: true, commitWithin: 10000, overwrite: true }) - Define how Index changes are stored Solr Commits.
    • defaultSearch - (optional, default: { defType: 'edismax', qf: 'name^10 age^1 gender' }) - Search strategy if query contains the param $search The Extended DisMax Query Parser.
    • suggestHandler - (optional, default: suggest) - Request a Solr Suggest Handler instead reggular a search if query contains the param $suggest Suggester.
    • events (optional) - A list of custom service events sent by this service
    • paginate (optional) - A pagination object containing a default and max page size
    • whitelist (default: ['$search', '$params', '$facet', '$filter']) [optional] - The list of additional non-standard query parameters to allow, by default populated with all Solr specific ones. You can override, for example in order to restrict access to some queries (see the options documentation).
    • multi (optional) - Allow create with arrays and update and remove with id null to change multiple items. Can be true for all methods or an array of allowed methods (e.g. [ 'remove', 'create' ])

    Getting Started

    The following example will create a Service with the name and endpoint solr.

    const feathers = require('@feathersjs/feathers');
    const express = require('@feathersjs/express');
    const fetch = require('node-fetch');
    const undici = require('undici');
    const Service = require('feathers-solr');
    const { SolrClient } = require('feathers-solr');
    const solrServer = 'http://localhost:8983/solr/gettingstarted';
    
    // Create an Express compatible Feathers application instance.
    const app = express(feathers());
    // Turn on JSON parser for REST services
    app.use(express.json());
    // Turn on URL-encoded parser for REST services
    app.use(express.urlencoded({ extended: true }));
    // Enable REST services
    app.configure(express.rest());
    // Enable REST services
    
    // init Adapter witch Fetch or Undici
    const options = {
      Model: SolrClient(fetch, solrServer),
      paginate: {},
      events: ['testing']
    };
    app.use('gettingstarted', new Service(options));
    
    app.listen(3030, () => {
      console.log(`Feathers server listening on port http://127.0.0.1:3030`);
    });

    Install Solr

     bin/solr start -e gettingstarted
    

    Use feathers-solr/bin/install-solr.sh for a kickstart installation.

    Run the example with node app and go to localhost:3030/gettingstarted.

    Querying

    Feathers Docs Database Querying

    Supportet Solr specific queries

    This Adapter use the Solr JSON Request API.

    The following params passed in raw to Solr. This gives the full access to the Solr JSON Request API.

    • $search (alias to query)
    • $params (alias to params)
    • $facet (alias to facet)
    • $filter (alias to filter)

    To avoid full query (read) access, just whitelist only $search and add your query startegy into a Hook.

    $search

    An alias to Solr param query (string) - Solr Schemaless Mode

    Simple Search in default field _text_.

    query: {
      $search: 'John';
    }

    The Standard Query Parser - Some Search Examples:

    • Exact match: { $search: "John" }
    • Fuzzy match: { $search: "John~" }
    • Phrase match: { $search: "John Doe" }
    • Starts with: { $search: "Jo*" }
    • Ends with: { $search: "*n" }
    • Contains: { $search: "(John AND Doe)" }
    • Contain one: { $search: "(John OR Doe)" }

    Define a default search query.

    service.options.defaultSearch = {
      defType: 'edismax',
      qf: 'name^10 age^1 gender'
    };
    
    const response = await service.find({
      query: {
        $search: 'Doug 20 male'
      }
    });

    See $parmas example how query advanced search

    $facet

    A alias to Solr param facet

    Get Min, Max and a Range Facet

    query: {
        $facet: {
            age_min : "min(age)",
            age_max : "max(age)",
            age_ranges: {
                type: "range",
                field: "age",
                start: 0,
                end: 100,
                gap: 10
            }
        }
    }

    The response should look like this:

    {
        QTime: 0,
        total: 50,
        limit: 10,
        skip: 0,
        data: [...],
        facet: {
            age_min: 1,
            age_max: 104,
            count: 54,
            age_ranges: {
                buckets: [{
                    val: 0,
                    count: 4
                }, {
                    val: 25,
                    count: 17
                }, {
                    val: 50,
                    count: 15
                }, {
                    val: 75,
                    count: 14
                }]
            }
        }
    }

    Get a Range Multi Facet

    query:{
      $search:'blue',
      '{!tag=COLOR}color':'Blue',
      $facet:{
          sizes:{type:terms, field:size},
          colors:{type:terms, field:color, domain:{excludeTags:COLOR} },
          brands:{type:terms, field:brand, domain:{excludeTags:BRAND}
      }
    }

    $params

    An alias to Solr param params. Allows you to access all solr query (read) features like:

    Spellchecker - Solr Spell Checking

    const response = await service.find({
      query: {
        $search: 'John !Doe +age:[80 TO *]',
        $params: {
          'defType': 'edismax',
          'qf': 'name^10 city^5 age',
          'mm': '2<99% 7<80% 10<50%',
          'q.op': 'OR',
          'sow': true,
          'spellcheck': true,
          'spellcheck.accuracy': 0.7,
          'spellcheck.extendedResults': true,
          'spellcheck.collate': true,
          'spellcheck.count': 10,
          'spellcheck.maxCollations': 1,
          'spellcheck.maxCollationTries': 10,
          'spellcheck.collateExtendedResults': true,
          'spellcheck.onlyMorePopular': true,
          'spellcheck.dictionary': 'LANG_X_text_spell_token'
        }
      }
    });

    Suggester - Solr Suggester

    const response = await service.find({
      query: {
        $suggest: 'john'
      }
    });

    Grouping - Solr Result Grouping

    const response = await service.find({
      query: {
        $params: {
          'group': true,
          'group.field': 'gender',
          'group.format': 'simple'
        }
      }
    });

    Highlight - Solr Highlighting

    const response = await service.find({
      query: {
        $search: 'doug',
        $params: {
          'hl': true,
          'hl.field': 'name'
        }
      },
      paginate: { max: 10, default: 3 }
    });

    MoreLikeThis - Solr MoreLikeThis

    const response = await service.find({
      query: {
        $search: 'male',
        $params: {
          'mlt': true,
          'mlt.fl': 'gender'
        }
      },
      paginate: { max: 10, default: 3 }
    });

    Spartial - Solr Spatial Search

    const response = await service.find({
      query: {
        $select: ['*', 'score', '_dist_:geodist()'],
        $params: {
          'sfield': 'location_p',
          'pt': '40.649558, -73.991815',
          d: 50,
          distanceUnits: 'kilometers',
          sort: 'geodist() asc'
        }
      },
      paginate: { max: 10, default: 3 }
    });

    $filter

    An alias to Solr filter passed in raw. It's recomanded to go with the common Querying.

    See more query variants JSON Facet API,Solr Facet Functions and Analytics, Solr Subfacets, Multi-Select Faceting

    Service Methods

    All service methods provide the multi options.

    Service.create

    The options.commitStrategy.override is true in default. This allow to override an existing id by service.create. Add the field _version_ to the $select params will return the document content with it's version. Create with an existing id and _version_ for optimistic concurrency

    Service.update

    Will overide the complete Document. If the _version_ field is part of update content, it will be removed.

    Service.patch

    Use the Solr Updating Parts of Documents

    Simple usage

    service.patch(id, {age: 30});

    Atomic Updates - Increment field age + 1

    service.patch(id, {age: {inc:1}});

    All Solr methods provided:

    • set - Set or replace the field value(s) with the specified value(s), or remove the values if 'null' or empty list is specified as the new value. May be specified as a single value, or as a list for multiValued fields.
    • add - Adds the specified values to a multiValued field. May be specified as a single value, or as a list.
    • add-distinct - Adds the specified values to a multiValued field, only if not already present. May be specified as a single value, or as a list.
    • remove - Removes (all occurrences of) the specified values from a multiValued field. May be specified as a single value, or as a list.
    • removeregex - Removes all occurrences of the specified regex from a multiValued field. May be specified as a single value, or as a list.
    • inc - Increments a numeric value by a specific amount. Must be specified as a single numeric value.

    Service.remove

    Provide delete by id ans query Delete all documents at once:

    service.remove(null, {query: {id:'*'}});

    Performance considerations

    Most of the data mutating operations in Solr (create, update, remove) do not return the full resulting document, therefore I had to resolve to using get as well in order to return complete data. This solution is of course adding a bit of an overhead, although it is also compliant with the standard behaviour expected of a feathers database adapter.

    Use Raw Query service.Model.post('update/json', data) to avoid this overhead. The response is native Solr.

    Raw Solr Queries

    You can access all Solr API endpoints by using a direct model usage.

    Ping the Solr core/collection:

    service.Model.get('admin/ping');

    Get Solr schema information:

    service.Model.get('schema');

    Modify Solr schema:

    service.Model.post('schema',{"add-field":{name:"age",type:"pint"}});

    Get Solr config information:

    service.Model.get('config');

    Modify Solr config:

    service.Model.post('config',{"add-requesthandler":{...}});

    Use a different HTTP Client

    This Adapter offers a node-fetch interface for a maximum on HTTP Client comfort and undici for maximum (100% compared to node-fetch) in performance. If you like to go with your favorite one:

    class CustomClient {
      constructor(HTTPModule, conn) {}
      get(api, params = {}) {}
      post(api, data, params = {}) {}
    };
    
    const options = {
        Model: CustomClient(HTTPModule, solrServer),
        paginate: {},
        events: ['testing']
      };
    
    app.service('solr', new Service(options))

    Contributing

    If you find a bug or something to improve i will be happy to see your PR!

    When adding a new feature, please make sure you write tests for it with decent coverage as well.

    TODO

    • Hook Examples
      • Schema Migration Hook (drop,alter,safe)
      • Json Hook Store Data as JSON
      • Validation Hook
      • Spellcheck
      • Suggester
      • MoreLikeThis

    Changelog

    2.3.0

    2.2.0

    • complete refactoring
    • implement @feathers/adapter-tests

    License

    Copyright (c) 2015

    Licensed under the MIT license.

    Install

    npm i feathers-solr

    DownloadsWeekly Downloads

    132

    Version

    2.11.0

    License

    MIT

    Unpacked Size

    43.6 kB

    Total Files

    21

    Last publish

    Collaborators

    • avatar