notebooks

KO | EN

gitlite — search

#claude-code #ai #ai-agents #llm #mcp #typescript #python #cli #developer-tools #agents #model-context-protocol #cybersecurity

★ 34 Open GitHub ↗

Notebook images for ODH

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# OpenDataHub Notebooks

[![GitHub Tag](https://img.shields.io/github/v/tag/opendatahub-io/notebooks?style=plastic)](https://github.com/opendatahub-io/notebooks/releases)
![GitHub License](https://img.shields.io/github/license/opendatahub-io/notebooks)

Welcome to the OpenDataHub Notebooks repository! This repository provides a collection of notebooks tailored for data analysis, machine learning, research and coding within the OpenDataHub ecosystem. Designed to streamline data science workflows, these notebooks offer an integrated environment equipped with the latest tools and libraries. These notebooks were created to be used with OpenDataHub ecosystem with the ODH Notebook Controller as the launcher.

These workbenches are available at: [quay.io/repository/opendatahub/workbench-images](https://quay.io/repository/opendatahub/workbench-images)

## Getting Started
For a deeper understanding of the architecture underlying this repository, please explore our wiki page [Wiki](https://github.com/opendatahub-io/notebooks/wiki/Workbenches)

### Prerequisites
Make sure the following tools are installed in your environment:
 - podman/docker
 - python 3.14
 - [uv](https://docs.astral.sh/uv/)
 - make (on macOS: `brew install make` and add `/opt/homebrew/opt/make/libexec/gnubin` to `PATH`)
 - curl
 - git, git-lfs (for `*.vsix` files)

### Installation
Clone this repository to your local machine:

```shell
git clone https://github.com/opendatahub-io/notebooks.git
cd notebooks
```

### Quick Start Guide

### Build a Notebook

To build a workbench image, you can execute the following command:

```shell
make ${WORKBENCH_NAME} -e  IMAGE_REGISTRY=quay.io/${YOUR_USER}/workbench-images  -e  RELEASE=2023x
```

Using  `IMAGE_REGISTRY` and `RELEASE` variables you can overwrite the default values and use a different registry or release tag

Using `CONTAINER_BUILD_CACHE_ARGS` (default: `--no-cache`), and `PUSH_IMAGES` variables you can further customize the build process.

### Hermetic prefetch (`prefetch-input/`)

Jupyter images that build **hermetically** (Konflux and local builds with Cachi2) share **repo-root** [`prefetch-input/`](prefetch-input/README.md): RPM lock inputs (`odh/`, `rhds/`), `repos/`, and the **mongocli** submodule. Each hermetic image directory under `jupyter/*/ubi9-python-3.12/` contains a **symlink** `prefetch-input` → `../../../prefetch-input`. **Code-server** keeps its own `codeserver/ubi9-python-3.12/prefetch-input/` (different dependency set). To prefetch dependencies locally, see [`scripts/lockfile-generators/README.md`](scripts/lockfile-generators/README.md).

### Local Execution

The notebook can be run as container on the local systems.

Use podman/docker to execute the workbench images as container.

```shell
podman  run -it -p  8888:8888  quay.io/opendatahub/workbench-images:jupyter-minimal-ubi9-python-3.9-2024a-20240317-6f4c36b
```

### Lock file generation

Image lock files (`pylock.toml` / `uv.lock.d/`) are regenerated with `make refresh-lock-files`.
For details on the dual `uv` version policy (dev vs image locks), see the
[Deploy & Test](#deploy--test) section below.

### Update base image versions

Use the root [`versions_config.yml`](versions_config.yml) file as the operator
entry point for base image updates.

The sync flow validates that config, rewrites managed `build-args/*.conf`
`BASE_IMAGE` and `RELEASE` values, updates root `Makefile` `RELEASE` defaults,
and resolves RHDS `channel: fast` tags from published Quay tags with `skopeo`.

For the full configuration contract, RHDS and ODH resolution rules, Mermaid flow
diagram, and worked examples, see
[`docs/base_image_versions_update_configuration.md`](docs/base_image_versions_update_configuration.md).

Quick commands:

```shell
make sync-build-args-from-versions
make sync-build-args-from-versions SYNC_BUILD_ARGS_ARGS=--dry-run
make sync-build-args-from-versions SYNC_BUILD_ARGS_ARGS=--check
```

If the version update also requires lockfile refreshes, run
`make refresh-lock-files` after the sync.

### Kick off a new release (maintainers)

Use this flow when starting a **new release stream** (for example, `3.5` → `3.6`).
For mid-release image bumps only, use `make sync-build-args-from-versions` instead
(see above).

1. Update [`versions_config.yml`](versions_config.yml) (`release.full_version` and any
   preferred base-image settings).
2. Run the orchestrated kickoff recipe locally **or** via GitHub Actions.

The kickoff recipe runs these steps in order:

1. `make sync-build-args-from-versions`
2. `make refresh-lock-files` (with `FORCE_LOCKFILES_UPGRADE=1`)
3. Generate ODH RPM lockfile (root prefetch input)
4. Generate ODH RPM lockfile (codeserver prefetch input)
5. Generate RHDS RPM lockfile (root prefetch input)
6. Generate RHDS RPM lockfile (codeserver prefetch input)
7. Roll out imagestream tags (`manifests/tools/rollout_tag_on_imagestreams.py`)
8. Validate with `make test`

#### Local kickoff

Export RHDS subscription credentials once per shell session, then run kickoff:

```shell
export SUBSCRIPTION_ACTIVATION_KEY='...'
export SUBSCRIPTION_ORG='...'
gmake kickoff-release
```

If a step fails, rerun the same command to auto-resume from
`.kickoff-release.state`, or force a step:

```shell
gmake kickoff-release KICKOFF_START_STEP=5
```

To clear cached RPM lockfile generator images between ODH/RHDS phases manually:

```shell
gmake clean-rpm-lockfile-cache
```

#### GitHub Actions kickoff

On `opendatahub-io/notebooks`, run **Actions → Release Kickoff Action → Run workflow**.

Optional workflow inputs:

- `release_full_version`, `release_rhds_os_base`, `release_python_version`
- `versions_overrides` (`dot.path=value`, one per line), for example:

```text
artifacts.base_image.cuda.tensorflow.acc_version=13.0
artifacts.base_image.cpu.rhds.channel=fast
artifacts.base_image.rocm.pytorch.rhds.channel=stable
```

The workflow applies overrides, runs `make kickoff-release`, and opens a PR titled
`GHA: Kick off new release <full_version>`.

Required repository secrets: `GIT_CRYPT_KEY`, `SUBSCRIPTION_ACTIVATION_KEY`,
`SUBSCRIPTION_ORG`, `GH_ACCESS_TOKEN`.

### Deploy & Test

#### Prepare Python + uv + pytest env

Root dev tooling uses `[tool.uv] required-version` in `pyproject.toml` (currently `>=0.11.8,<0.13`).
Install any uv in that range and use it directly for sync, tests, and pre-commit:

```shell
# Linux
sudo dnf install python3.14
pip install --user 'uv>=0.11.8,<0.13'
# macOS
brew install python@3.14 uv

uv venv --python $(which python3.14)
uv sync --locked
```

Per-image lock files (`pylock.toml` / `uv.lock.d/`) are generated with a **stricter** uv pin
(`dependencies/uv-image-lock-version`, currently `0.12.0`) via the `./uv` wrapper and
`make refresh-lock-files`. Do not use `./uv` for everyday dev commands unless you are
refreshing image locks.

<details>
<summary>Image lock generation with <code>./uv</code></summary>

```shell
make refresh-lock-files
# or: ./uv run scripts/pylocks_generator.py public-index <dir>
```

The `./uv` script runs `uv tool run "uv@$(cat dependencies/uv-image-lock-version)"` when your
system uv does not already match that exact version.

</details>

#### Running tests

For the full test catalog (types, markers, CI parity), see [docs/agents/testing.md](docs/agents/testing.md).

By completing configuration in the previous section, you can run tests using the following targets:

```bash
make test              # Quick static tests (pytest + Dockerfile alignment check)
make test-unit         # Python unit tests + doctests + Go tests (no container runtime needed)
make test-integration PYTEST_ARGS="--image=<image>"  # Container integration tests
```

##### Container selftests

We're using [Testcontainers.com](https://testcontainers.com/) to run containers from Python tests.

Ideally, these tests should allow switching between Kubernetes and Docker/Podman.
Running on Kubernetes (OpenShift) is the easiest way to manage GPU access for testing TensorFlow/Pytorch images.
This improvement is tracked as a future work.

We also considered [Dagger.io](https://dagger.io).
It has very nice verbose logging by default for every action that is running.
The main difference between the two is that Dagger creates more abstractions over the container engine.
Dagger limitations that stem from tight [BuildKit](https://github.com/moby/buildkit) integration discouraged us from using it.
Images present in a local store cannot be directly used in Dagger, they have to be imported into its BuildKit container first.
In addition, Dagger [does not allow bind-mounting local directories](https://docs.dagger.io/cookbook/#modify-a-copied-directory-or-remote-repository-in-a-container)
to the container but always copies files in and out.

#### Running testcontainers tests in Pytest
```
# Podman/Docker config
# Linux
sudo dnf install podman
systemctl --user start podman.service
systemctl --user status podman.service
systemctl --user status podman.socket
DOCKER_HOST=unix:///run/user/$UID/podman/podman.sock uv run pytest tests/containers -m 'not openshift and not cuda and not rocm' --image quay.io/opendatahub/workbench-images@sha256:e98d19df346e7abb1fa3053f6d41f0d1fa9bab39e49b4cb90b510ca33452c2e4

# Mac OS
brew install podman
podman machine init
podman machine set --rootful=false
sudo podman-mac-helper install
podman machine start
uv run pytest tests/containers -m 'not openshift' --image quay.io/opendatahub/workbench-images@sha256:e98d19df346e7abb1fa3053f6d41f0d1fa9bab39e49b4cb90b510ca33452c2e4
```

When using lima on macOS, it might be useful to give yourself access to rootful podman socket

```shell
lima sudo systemctl start podman
lima sudo dnf install acl
lima sudo setfacl -m u:${USER}:x /var/run/podman
lima sudo setfacl -m u:${USER}:rw /var/run/podman/podman.sock
```

#### Running Playwright tests

[tests/browser/README.md](tests/browser/README.md)

#### Notebooks

Deploy the notebook images in your Kubernetes environment using:
`deploy8-${NOTEBOOK_NAME} for ubi8 or deploy9-${NOTEBOOK_NAME} for ubi9`

```shell
make  deployX-${NOTEBOOK_NAME}
```

Run the test suite against this notebook:

```shell
make  test-${NOTEBOOK_NAME}
```

You can overwrite `NOTEBOOK_REPO_BRANCH_BASE` variable to use a different repository and branch for testing scripts. This is useful when you debug your changes.

```shell
make  test-${NOTEBOOK_NAME} -e  NOTEBOOK_REPO_BRANCH_BASE="https://raw.githubusercontent.com/${YOUR_USER}/notebooks/${YOUR_BRANCH}"
```

Clean up the environment when the tests are finished:

```shell
make  undeployX-${NOTEBOOK_NAME}
```

#### Runtimes

The runtimes image requires to have curl and python installed, so that on runtime additional packages can be installed.

Deploy the runtime images in your Kubernetes environment using: `deploy8-${WORKBENCH_NAME} for ubi8 or deploy9-${WORKBENCH_NAME} for ubi9`

```shell
make  deployX-${WORKBENCH_NAME}
```

Run the validate test suit for checking compatabilty of runtime images:

```shell
make  validate-runtime-image  image=<runtime-image>
```

Clean up the environment when the tests are finished:

```shell
make  undeployX-${WORKBENCH_NAME}
```

## Image Inventory List
This table provides a concise overview of the support status for various container images.

| Workbench | Version | Python | OS | x86_64 | aarch64 | ppc64le | s390x | Container |
|------------|----------|--------|------|---------|----------|----------|--------|-------------|
| CPU Minimal | - | 3.12 | UBI9 | ✅ | ✅ | ✅ | ✅ | [quay.io/repository/opendatahub/odh-workbench-jupyter-minimal-cpu-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-minimal-cpu-py312-ubi9?tab=tags) |
| CPU DataScience | - | 3.12 | UBI9 | ✅ | ✅ | ✅ | ✅ | [quay.io/repository/opendatahub/odh-workbench-jupyter-datascience-cpu-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-datascience-cpu-py312-ubi9?tab=tags) |
| CPU TrustyAI | - | 3.12 | UBI9 | ✅ | ✅ | ✅ | ✅ | [quay.io/repository/opendatahub/odh-workbench-jupyter-trustyai-cpu-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-trustyai-cpu-py312-ubi9?tab=tags) |
| CPU CodeServer | - | 3.12 | UBI9 | ✅ | ✅ | ✅ | ✅ | [quay.io/repository/opendatahub/odh-workbench-codeserver-datascience-cpu-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-codeserver-datascience-cpu-py312-ubi9?tab=tags) |
| CUDA Minimal | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-minimal-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-minimal-cuda-py312-ubi9?tab=tags) |
| CUDA PyTorch | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-pytorch-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-pytorch-cuda-py312-ubi9?tab=tags) |
| CUDA PyTorch LLMCompressor | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-pytorch-llmcompressor-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-pytorch-llmcompressor-cuda-py312-ubi9?tab=tags) |
| CUDA TensorFlow | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-tensorflow-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-tensorflow-cuda-py312-ubi9?tab=tags) |
| ROCM Minimal | 6.3 | 3.12 | UBI9/RHEL9.6 | ✅ | ❌ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-minimal-rocm-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-minimal-rocm-py312-ubi9) |
| ROCM PyTorch | 6.3 | 3.12 | UBI9/RHEL9.6 | ✅ | ❌ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-pytorch-rocm-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-pytorch-rocm-py312-ubi9?tab=tags) |
| ROCM TensorFlow | 6.3 | 3.12 | UBI9/RHEL9.6 | ✅ | ❌ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-workbench-jupyter-tensorflow-rocm-py312-ubi9](https://quay.io/repository/opendatahub/odh-workbench-jupyter-tensorflow-rocm-py312-ubi9?tab=tags) |

| Runtime | Version | Python | UBI | x86_64 | aarch64 | ppc64le | s390x | Container |
|----------|----------|--------|------|---------|----------|----------|--------|-------------|
| CPU Minimal | - | 3.12 | UBI9 | ✅ | ✅ | ✅ | ✅ | [quay.io/repository/opendatahub/odh-pipeline-runtime-minimal-cpu-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-minimal-cpu-py312-ubi9?tab=tags) |
| CPU DataScience | - | 3.12 | UBI9 | ✅ | ✅ | ✅ | ✅ | [quay.io/repository/opendatahub/odh-pipeline-runtime-datascience-cpu-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-datascience-cpu-py312-ubi9?tab=tags) |
| CUDA PyTorch | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-pipeline-runtime-pytorch-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-pytorch-cuda-py312-ubi9?tab=tags) |
| CUDA PyTorch LLMCompressor | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-pipeline-runtime-pytorch-llmcompressor-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-pytorch-llmcompressor-cuda-py312-ubi9) |
| CUDA TensorFlow | 12.8 | 3.12 | UBI9/RHEL9.6 | ✅ | ✅ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-pipeline-runtime-tensorflow-cuda-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-tensorflow-cuda-py312-ubi9?tab=tags) |
| ROCM PyTorch | 6.3 | 3.12 | UBI9/RHEL9.6 | ✅ | ❌ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-pipeline-runtime-pytorch-rocm-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-pytorch-rocm-py312-ubi9?tab=tags) |
| ROCM TensorFlow | 6.3 | 3.12 | UBI9/RHEL9.6 | ✅ | ❌ | ❌ | ❌ | [quay.io/repository/opendatahub/odh-pipeline-runtime-tensorflow-rocm-py312-ubi9](https://quay.io/repository/opendatahub/odh-pipeline-runtime-tensorflow-rocm-py312-ubi9?tab=tags) |

* ✅ supported
* ⚠️ under development
* ⏰ planned in the future
* ❌ not supported by vendor

## Contributing

Whether you're fixing bugs, adding new notebooks, or improving documentation, your contributions are welcome. Please refer to our [Contribution Guidelines](CONTRIBUTING.md).

### For AI Agents

If you're an AI agent working with this repository, please refer to our [Agents Guide](AGENTS.md) for comprehensive instructions on project structure, development workflows, and best practices.

## Acknowledgments

A huge thank you to all our contributors and the broader OpenDataHub community!

## License

This project is licensed under  the Apache License 2.0 - see the [LICENSE](https://github.com/opendatahub-io/notebooks/blob/main/LICENSE) file for details.

## Contact

Anything unclear or inaccurate? Please let us know by reporting an issue: [notebooks/issues](https://github.com/opendatahub-io/notebooks/issues/new)

notebooks

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README