envlogger - GitLite

KO | EN

gitlite — search

#hacktoberfest #react #ai #typescript #llm #android #rust #deep-learning #windows #vue #awesome-list #security #mcp #ios

envlogger

(★ 134)

A tool for recording RL trajectories.

File Explorer

Download Latest Version (.zip)

- - ci.yml
  - release.yml
- build.sh
- Dockerfile
- - envlogger.png
  - timings.png
- - __init__.py
  - backend_reader.py
  - backend_type.py
  - backend_writer.py
  - BUILD.bazel
  - in_memory_backend.py
  - in_memory_backend_test.py
  - riegeli_backend_reader.py
  - riegeli_backend_writer.py
  - riegeli_backend_writer_test.py
  - schedulers.py
  - schedulers_test.py
- - __init__.py
  - BUILD.bazel
  - codec.py
  - codec_test.py
  - make_visitor.h
  - make_visitor_test.cc
  - spec_codec.py
  - spec_codec_test.py
  - xtensor_codec.cc
  - xtensor_codec.h
  - xtensor_codec_test.cc
- - __init__.py
  - BUILD.bazel
  - random_agent_catch.py
- - BUILD.bazel
  - bundle.h
  - filesystem.h
  - parse_text_proto.h
  - proto_testutil.h
  - riegeli_file_reader.h
  - riegeli_file_writer.h
  - status_macros.h
  - test_macros.h
- - __init__.py
  - BUILD.bazel
  - storage.proto
- - __init__.py
  - BUILD.bazel
  - catch_env.py
- - BUILD.bazel
  - convert_tfds_to_riegeli.py
  - convert_tfds_to_riegeli_test.py
- __init__.py
- BUILD.bazel
- environment_logger.py
- environment_logger_test.py
- environment_wrapper.py
- reader.py
- requirements.txt
- step_data.py
- BUILD.bazel
- crc32.BUILD.bazel
- gmp.BUILD.bazel
- highwayhash.BUILD.bazel
- net_zstd.BUILD.bazel
- pybind11_bazel.diff
- snappy.BUILD.bazel
- xtensor.BUILD.bazel
- xtl.BUILD.bazel
- zlib.BUILD.bazel
.bazelrc
.bazelversion
CONTRIBUTING.md
LICENSE
README.md
WORKSPACE.bazel

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# EnvironmentLogger

`EnvLogger` is a standard `dm_env.Environment` class wrapper that
records interactions between a real environment and an agent. These interactions
are saved on disk as trajectories and can be retrieved in whole, by individual
timesteps or by specific episodes.

![drawing](https://raw.githubusercontent.com/google-deepmind/envlogger/main/docs/images/envlogger.png "EnvironmentLogger Diagram")

## Metadata

To better categorize your logged data, you may want to add some tags in the
metadata when you construct the logger wrapper. The metadata is written once per
`EnvLogger` instance.

```python
env = envlogger.EnvLogger(
    env,
    data_directory='/tmp/experiment_logs',
    metadata={
        'environment_type': 'dm_control',
        'agent_type': 'D4PG'
    })
```
## How to use Envlogger

NOTE: Ensure that `data_directory` exists _before_ instantiating the wrapper.

Most of the time, it is just a one-liner wrapper, e.g.

```python
import envlogger
from envlogger.testing import catch_env
import numpy as np

env = catch_env.Catch()
with envlogger.EnvLogger(
    env, data_directory='/tmp/experiment_logs') as env:

env.reset()
  for step in range(100):
    action = np.random.randint(low=0, high=3)
    timestep = env.step(action)
```

Full example of random agent in Catch is available here:
[random_agent_catch.py](https://github.com/google-deepmind/envlogger/tree/main/envlogger/examples/random_agent_catch.py)

### Step metadata

Envlogger also allows to record custom metadata per step by defining a function
that can be passed to the wrapper. In this example, we want to record the
timestamp of when each step was produced:

```python

def step_fn(unused_timestep, unused_action, unused_env):
    return {'timestamp': time.time()}

...

with envlogger.Envlogger(
    env,
    data_directory='/tmp/experiment_logs',
    step_fn=step_fn) as env:

...

```

### Episode metadata

Recording custom episode metadata is also possible by providing a callback. This
callback is invoked at every step but only the last value returned that is not
`None` (if any) is stored.

In the following example, we only store the timestamp of the first step of the
episode.

```python

def episode_fn(timestep, unused_action, unused_env):
    if timestemp.first:
      return {'timestamp': time.time()}
    else:
      return None

...

with envlogger.Envlogger(
    env,
    data_directory=FLAGS.trajectories_dir,
    episode_fn=episode_fn) as env:

...

```

### Riegeli backend (Default & Recommended)

By default, `EnvLogger` uses the `RiegeliBackendWriter`, which efficiently
stores trajectories in compressed Riegeli format without requiring complex
dataset schema definitions or heavy external dependencies.

```python
with envlogger.EnvLogger(
    env,
    data_directory='/tmp/experiment_logs') as env:
```

### Recording human-generated data

Please see [RLDS Creator] if you want to record data from a human interacting
with an environment.

[RLDS Creator]: http://github.com/google-research/rlds-creator

## Reading stored trajectories

`Reader` can read stored trajectories. Example:

```python
from envlogger import reader

with reader.Reader(
    data_directory='/tmp/experiment_logs') as r:
  for episode in r.episodes:
    for step in episode:
       # step is a step_data.StepData.
       # Use step.timestep.observation, step.timestep.reward, step.action etc...
```

### Reading the dataset with TFDS/RLDS

Datasets generated with Envlogger are compatible with [RLDS].

If you have legacy TFDS dataset files, you can read your data directly
with `tfds.builder_from_directory`. Check the [RLDS] website for colabs and
tools to manipulate your datasets.

Otherwise, you can transform them into a [TFDS](http://www.tensorflow.org/datasets)
compatible dataset (learn how in the [RLDS] documentation).

[RLDS]: http://github.com/google-research/rlds

## Getting Started

> EnvLogger currently only supports Linux based OSes and Python 3.

You can install EnvLogger via `pip`:

```
pip install envlogger
```

> **Note**: `envlogger` requires the GNU Multiple Precision (GMP) library at
> runtime. On Debian-based systems (like Ubuntu), you must install
> `libgmpxx4ldbl` before importing `envlogger`:
> ```bash
> sudo apt-get install libgmpxx4ldbl
> ```

##### Compiling from source

For this option you will need to [install Bazel](https://docs.bazel.build/versions/main/install.html) and [GMP](https://gmplib.org/) (`libgmp-dev` on Debian-based systems).
We recommend using [bazelisk](https://github.com/bazelbuild/bazelisk) to manage Bazel versions, which will automatically use the pinned version in `.bazelversion` (currently 7.1.0). Then:

```
git clone https://github.com/deepmind/envlogger/
cd envlogger
bazel test --test_output=errors envlogger/...
```

To build and install the wheel package:

```
bazel build -c opt envlogger:envlogger_wheel
pip install bazel-bin/envlogger/*.whl
```

Note: `setup.py` has been removed from this project. Standard Python packaging
tools (like `pip install .` or `setup.py install`) are not supported. You must
use Bazel to build the wheel as shown above.

##### Running inside Docker

We provide a Docker image that can be used to run tests in a more controlled
environment. You can run it as follows:

```
sh docker/build.sh
docker run -it envlogger bash
bazel test --test_output=errors envlogger/...
```

## Benchmarks

Wrapping your environment with EnvLogger adds an approximately 2 millisecond
overhead per environment step.
See the following difference in distributions in the case of random agent on a
100 steps per second Catch environment (measured in milliseconds).

Percentiles      | 50th      | 95th       | 99th      | 99.9th     | mean (± std)
---------------- | --------- | -----------| ----------| -----------| -----------
w/o EnvLogger    | 10.15     | 10.23      | 11.51     | 14.70      | 10.19 (± 1.06)
w/ EnvLogger     | 12.65     | 14.39      | 15.94     | 19.43      | 12.88 (± 0.34)

<img src="docs/images/timings.png" width="40%">

## Acknowledgements

We greatly appreciate all the support from the
[TF-Agents](https://github.com/tensorflow/agents) team in setting up building
and testing for EnvLogger.