Cuckoo Filters
https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf
Abstract
In many networking systems, Bloom filters are used for highspeed set membership tests. They permit a small fraction of false positive answers with very good space efficiency. However, they do not permit deletion of items from the set, and previous attempts to extend “standard” Bloom filters to support deletion all degrade either space or performance. We propose a new data structure called the cuckoo filter that can replace Bloom filters for approximate set membership tests. Cuckoo filters support adding and removing items dynamically while achieving even higher performance than Bloom filters. For applications that store many items and target moderately low false positive rates, cuckoo filters have lower space overhead than space-optimized Bloom filters. Our experimental results also show that cuckoo filters outperform previous data structures that extend Bloom filters to support deletions substantially in both time and space.
Note
Potentially breaking changes for backward compatability in version 1.1 with prior versions. If you are using previously serialized cuckoo filters from other versions they may need to be rebuilt with the current version.
Install
npm install cuckoo-filter
Usage
const CuckooFilter = CuckooFilterconst ScalableCuckooFilter = ScalableCuckooFilter let cuckoo= 200 4 2 // (Size, Bucket Size, Finger Print Size) let cuckoo2 = 2000 4 2 2 // (Size, Bucket Size, Finger Print Size, Exponential Scale) console//(buffer|string|number) returns true if successfulconsole// true: She's definately in thereconsole // 1console//false He's not homeconsole // true less than 95% fulllet json = cuckoo // serialize to json objectlet cbor = cuckoo // serialize to cbor
Note
Size your buckets and fingerprints to avoid collisions. Scalable cuckoo filters scale exponentially to hold your data.