transductio

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 17 Open GitHub ↗

A bridge from I/O to transducers and back

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# transductio

[![clojars badge](https://badgen.net/badge/release/1.1.0/blue)](https://clojars.org/systems.thoughtfull/transductio/versions/1.1.0) [![cljdoc badge](https://badgen.net/badge/docs/1.1.0/blue)](https://cljdoc.org/d/systems.thoughtfull/transductio/1.1.0/doc/readme)

A bridge from I/O to transducers, and back!

This library provides two core functions to bridge I/O with transducers, and convenience functions
building on them.  `decoder` creates a function that can produce data from input for use with
transducers and `encoder` makes a function that takes data, possibly through a transducer, and
writes it to output.

`decoder` is analogous to a source collection in a transduction pipeline.  `encoder` is analogous to
`into`, it is a transduction context that takes a transducer and an input collection, but writes to
output instead of building a collection.

`decoder` takes four arguments: open, decode, done?, close.  It returns a function that takes a
source and returns a reducible.  This reducible calls open with the source, then decode with that
result, and eventually, when done? returns true, the reduction completes and it calls close.

For example, if you want to decode Clojure data:

```clojure
(def clojure-decoder
  (decoder (comp PushbackReader/new io/reader)
    #(read % false ::eof)
    #{::eof}
    AutoCloseable/.close))

(into [] (clojure-decoder (StringReader. "1 2 3")))
;;=> [1 2 3]
```

`encoder` takes three arguments: open, encode!, close.  It returns a function that acts like `into`.
Like `into` it has two arities.  The first takes a sink and a source and reduces the source into the
sink.  It calls open with the sink, then encode! with that result and each value from the source,
and eventually, when it exhausts the source, the reduction completes and it calls close.

The second arity takes a sink, a transduction, and a source. The process is the same as above, but
the data flows from the source, through the transduction, then into the sink.

For example, if you want to encode Clojure data:

```clojure
(def clojure-encoder
  (encoder io/writer
    (fn [^Writer out value] (.write out (prn-str value)))
    AutoCloseable/.close))

(clojure-encoder "/tmp/foo.clj" (map inc) [1 2 3])

(slurp "/tmp/foo.clj")
;;=> "2\n3\n4\n"
```

## Comparison to `clojure.core/iteration`

Unlike `clojure.core/iteration`, a decoder repeatedly calls `decode` with the return value from
`open`.  `open` should return a stateful, mutable object that produces a new value every time it is
given to `decode`, there is no succession of continuation tokens.  This is simpler to think about
and use in common situations like `java.io` streams and readers.

A decoder also cleans up resources when its reduction completes (early or otherwise).  This makes it
suitable for working with things like database connection, file descriptors, etc.  If an error
occurs during the reduction, or if the reduction returns early (i.e. someone returns a `reduced`),
there is still an opportunity to release resources when the decoder calls `close`.

## Clojure and EDN data

There are three functions for decoding/encoding Clojure and EDN data.

`clojure-decoder` decodes Clojure data.  It takes `:encoding` which it sends to
`clojure.java.io/reader`, `:features` and `:read-cond` as options which it passes along to
`clojure.core/read`, and `:data-readers`, `:default-data-reader-fn`, and `:read-eval` which it binds
to the appropriate dynamic variables for the duration of the reduction/transduction.

`clojure-encoder` encodes Clojure data.  It takes `:encoding` and `:append` as options which it
passes along to `clojure.java.io/writer`.  It encodes Clojure data using `prn-str`.

`edn-decoder` decodes EDN data.  It takes `:encoding` which it sends to `clojure.java.io./reader`
and `:features` and `:read-cond` as options which it passes along to `clojure.edn/read`.

There is no EDN encoder, because there isn't really such a thing in Clojure.  `clojure-encoder` can
substitute, but EDN is probably best hand written because `prn-str` isn't necessarily guaranteed to
produce valid EDN data.[^1]

[^1]: See https://nitor.com/en/articles/pitfalls-and-bumps-clojures-extensible-data-notation-edn

## License

> Copyright © technosophist
>
> This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of
> the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.
>
> This Source Code Form is "Incompatible With Secondary Licenses", as defined by the Mozilla Public
> License, v. 2.0.

transductio

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README