Naboo's Podracing Misadventure

npm

Need private packages and team management tools?Check out npm Orgs. »

stopword

0.3.4 • Public • Published

stopword

stopword is a module for node and the browser that allows you to strip stopwords from an input text. In natural language processing, "Stopwords" are words that are so frequent that they can safely be removed from a text without altering its meaning.

NPM version NPM downloads MIT License Build Status

wonderful day stopword module

Live stopword browser demo.

Usage

Node.js

sw = require('stopword')
// sw.removeStopwords and sw.[language code] now available

Script tag method

<script src="stopword.js"></script>
 
<script>
// sw.removeStopwords and sw.[language code] now available
</script> 

Default (English)

By default, stopword will strip an array of "meaningless" English words

sw = require('stopword')
const oldString = 'a really Interesting string with some words'.split(' ')
const newString = sw.removeStopwords(oldString)
// newString is now [ 'really', 'Interesting', 'string', 'words' ]
 

Other languages

You can also specify a language other than English:

sw = require('stopword')
const oldString = 'Trädgårdsägare är beredda att pröva vad som helst för att bli av med de hatade mördarsniglarna åäö'.split(' ')
// sw.sv contains swedish stopwords
const newString = sw.removeStopwords(oldString, sw.sv)
// newString is now [ 'Trädgårdsägare', 'beredda', 'pröva', 'helst', 'hatade', 'mördarsniglarna', 'åäö' ]

Custom list of stopwords

And last, but not least, it is possible to use your own, custom list of stopwords:

sw = require('stopword')
const oldString = 'you can even roll your own custom stopword list'.split(' ')
// Just add your own list/array of stopwords
const newString = sw.removeStopwords(oldString, [ 'even', 'a', 'custom', 'stopword', 'list', 'is', 'possible']
// newString is now [ 'you', 'can', 'roll', 'your', 'own']

API

removeStopwords

Returns an Array that represents the text with the specified stopwords removed.

  • text An array of words
  • stopwords An array of stopwords
sw = require('stopword')
var text = sw.removeStopwords(text[, stopwords])
// text is now an array of given words minus specified stopwords

<language code>

Arrays of stopwords for the following 32 languages are supplied:

  • af - Afrikaans
  • ar - Modern Standard Arabic
  • bn - Bengali
  • br - Brazilian Portuguese
  • da - Danish
  • de - German
  • en - English
  • es - Spanish
  • fa - Farsi
  • fi - Finnish
  • fr - French
  • ha - Hausa
  • he - Hebrew
  • hi - Hindi
  • id - Indonesian
  • it - Italian
  • ja - Japanese
  • lgg - Lugbara (without diacritics)
  • lggo - Lugbara official (with diacritics)
  • nl - Dutch
  • no - Norwegian
  • pl - Polish
  • pt - Portuguese
  • pa - Punjabi Gurmukhi
  • ru - Russian
  • so - Somali
  • st - Sotho
  • sv - Swedish
  • sw - Swahili
  • vi - Vietnamese
  • yo - Yoruba
  • zh - Chinese Simplified
  • zu - Zulu
sw = require('stopword')
norwegianStopwords = sw.no
// norwegianStopwords now contains an Array of norwgian stopwords

Languages with no space between words

ja Japanese and zh Chinese Simplified and some of the other languages supported have no space between words. For these languages you need to split the text into an array of words in another way than just textString.split(' '). You can check out TinySegmenter for Japanese and chinese-tokenizer for Chinese.

Your language missing?

If you can't find a stopword file for your language, you can try creating one with stopword-trainer. We're happy to help you in the process.

install

npm i stopword

Downloadsweekly downloads

6,644

version

0.3.4

license

MIT

homepage

github.com

repository

Gitgithub

last publish

collaborators

  • avatar
  • avatar
Report a vulnerability