KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #codex #typescript #developer-tools #ai-agents #deepseek-harness #claude-code #ai-agent #cli #react #mcp

bettermdptools

★ 32 Open GitHub ↗

so much room for activities

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

![PyPI](https://img.shields.io/pypi/v/bettermdptools.svg)
![Python Versions](https://img.shields.io/pypi/pyversions/bettermdptools.svg)
![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)
![Code Style: Black](https://img.shields.io/badge/code%20style-black-000000.svg)
![Linter: Ruff](https://img.shields.io/badge/lint-ruff-blue.svg)
[![CircleCI](https://dl.circleci.com/status-badge/img/circleci/WH9eaoZnQRJ8SGFDrvqQAd/HURrQDZ2vzVYyU2QhPL29y/tree/master.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/circleci/WH9eaoZnQRJ8SGFDrvqQAd/HURrQDZ2vzVYyU2QhPL29y/tree/master)
# bettermdptools

Bettermdptools is a lightweight toolkit for working with Gymnasium environments using classic planning and tabular reinforcement learning methods.

It is designed to help users get up and running quickly, explore standard RL algorithms, and experiment with environments like FrozenLake, Taxi, Blackjack, and CartPole without heavy framework overhead.

---

## Getting started

### Install

Install from PyPI:

```bash
pip install bettermdptools
```
<details>
<summary><strong>pygame installation issues (Python 3.11+)</strong></summary>

If you encounter errors installing `pygame` on Python 3.11 or newer, try:

```bash
pip install pygame --pre
```

- https://stackoverflow.com/questions/74188013/python-pygame-not-installing

</details>

<details>
<summary><strong>Google Colab notes (NumPy + Gymnasium)</strong></summary>

Some Gymnasium-compatible environments may require a NumPy downgrade.

When using **Google Colab** - after installing numpy<2 **you must restart the Colab session** for the change to take effect.

Typical workflow:

```bash
pip install "numpy<2"
```
Then:

Runtime → Restart session

</details>

---

## Quick example (FrozenLake)

Below is a minimal example using value iteration on FrozenLake:

```python
import gymnasium as gym
from bettermdptools.algorithms.planner import Planner
from bettermdptools.utils.plots import Plots

env = gym.make("FrozenLake8x8-v1", render_mode=None)

V, V_track, pi = Planner(env.P).value_iteration()

Plots.values_heat_map(
    V,
    title="State Values",
    size=(8, 8),
)
```
![grid_state_values](https://user-images.githubusercontent.com/10093986/211906047-bc13956b-b8e6-411d-ae68-7a3eb5f2ad32.PNG)
---

## Example notebooks

The fastest way to explore the library is through the notebooks in [`/examples`](/examples) .

---

## Entrypoint APIs (optional)

Bettermdptools provides optional high-level entrypoints for quick experimentation and hyperparameter search.

See:
- [docs/api/experiments_api.md](docs/api/experiments_api.md)
- [docs/api/optuna_search_api.md](docs/api/optuna_search_api.md)

---

## Documentation and Dependency Management

The project supports standard `pip` workflows. For development, using Poetry is recommended to ensure reproducible environments.  API documentation is generated using **pdoc** (not pdoc3) and lives in the `docs/` directory.

To regenerate documentation locally:

```bash
pdoc --include-undocumented -d numpy -t docs-templates --output-dir docs bettermdptools
```

---

## Contributing

Pull requests are welcome.

Guidelines:
- Use numpy-style docstrings
- Add or update tests when introducing new functionality
- Prefer explicit, readable code over clever or showy abstractions
- Keep public APIs stable and avoid breaking changes unless clearly justified
- Prefer small, focused pull requests with a limited number of file changes

Basic workflow:
1. Fork the repository
2. Create a feature branch
3. Commit changes
4. Open a pull request

---

bettermdptools

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README