lambda-sketch

KO | EN

gitlite — search

#javascript #hacktoberfest #llm #golang #go #machine-learning #deep-learning #windows #cli #awesome-list #vue #pytorch #mcp #open-source

(★ 12)

Probabilistic data structures and sketching algorithms in Clojure

lambda-sketch 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# lambda-sketch

Probabilistic data structures and sketching algorithms in Clojure:

* Bloom filter - set membership
* Count-min sketch - frequency counts
* HyperLogLog - cardinality counts

### Installation

Add the following dependency to your `deps.edn` file:

io.github.cloudkj/lambda-sketch {:git/tag "v0.0.2" :git/sha "c2ff724"}

### Usage

```clojure
=> (use 'lambda-sketch.core)

;; Load tokenized texts
=> (def alice (map clojure.string/lower-case (clojure.string/split (slurp "https://www.gutenberg.org/cache/epub/11/pg11.txt") #"\W+")))
=> (def pride (map clojure.string/lower-case (clojure.string/split (slurp "https://www.gutenberg.org/cache/epub/1342/pg1342.txt") #"\W+")))

;; Initialize bloom filters as set membership predicates
=> (def alice? (into (bloom-filter 30000 7) alice))
=> (def pride? (into (bloom-filter 70000 7) pride))

;; Frequent words in one text that don't occur in the other
=> (filter (complement alice?) (map first (take-last 100 (sort-by second (frequencies pride)))))
("wickham" "jane" "bingley" "bennet" "mrs" "darcy" "elizabeth" "mr")
=> (filter (complement pride?) (map first (take-last 100 (sort-by second (frequencies alice)))))
("rabbit" "gryphon" "hatter" "mock" "turtle" "queen" "alice")

;; Initialize count-min sketch for frequency estimates
=> (def count-alice (into (count-min-sketch 200 5) alice))

;; Estimated frequencies for most frequent words in text
=> (take-last 10 (sort-by second (frequencies alice)))
(["said" 460] ["you" 471] ["i" 524] ["she" 549] ["it" 607] ["of" 637] ["a" 695] ["to" 811] ["and" 941] ["the" 1839])
=> (map (fn [[w _]] [w (count-alice w)]) (take-last 10 (sort-by second (frequencies alice))))
(["said" 519] ["you" 501] ["i" 550] ["she" 580] ["it" 654] ["of" 673] ["a" 723] ["to" 856] ["and" 1010] ["the" 1873])

;; Initialize HyperLogLog for cardinality estimates
=> (def distinct-alice (into (hyperloglog 128) alice))
=> (def distinct-pride (into (hyperloglog 128) pride))

;; Actual and estimated cardinality for text
=> (count (distinct alice))
3143
=> (count distinct-alice)
3743
=> (count (distinct pride))
7303
=> (count distinct-pride)
7702
```

lambda-sketch

lambda-sketch 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

lambda-sketch

lambda-sketch 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README