gemma_penzai - GitLite

KO | EN

gitlite — search

#python #react #go #golang #android #deep-learning #linux #docker #nodejs #windows #cli #awesome-list #ios #open-source

gemma_penzai

(★ 99)

A JAX Research Toolkit for Visualizing, Manipulating, and Understanding Gemma Models with Multi-modal Support based on Penzai.

파일 탐색기

최종 버전 다운로드 (.zip)

- - __init__.py
  - load_gemma.py
  - sampling_mode.py
  - simple_decoding_loop.py
- - __init__.py
  - convolution.py
  - image_utils.py
  - siglip.py
  - vision_layers.py
  - vision_parts.py
- __init__.py
- - attention_and_logit_lens.ipynb
  - gemma_scope1.ipynb
  - gemma_scope2.ipynb
- gemma3_multimodal_penzai.ipynb
- gemma3_text_penzai.ipynb
- gemma3_vision_penzai.ipynb
CONTRIBUTING.md
LICENSE
pyproject.toml
README.md

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/google-deepmind/gemma_penzai

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd gemma_penzai

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. 공식 설치 스크립트

쉬움 추천

사전 준비물

Python 3 pip 명령어를 쓰려면 Python이 필요합니다.

pip install --upgrade pip

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

✅ 설치 후 새 터미널을 열고, 프로그램의 버전 확인 명령(예: --version)으로 정상 설치됐는지 확인하세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install --upgrade pip

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

pip install -e .

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Gemma in Penzai

---
`gemma_penzai` is JAX research toolkit for visualizing, manipulating and
understanding Gemma models with multi-modal support based on Penzai. The
original [Penzai](https://github.com/google-deepmind/penzai) mainly supports
text-only LLMs, including Gemma 1 and Gemma 2. Now we extend Penzai with vision
and multimodal support. Therefore, Gemma 3 is supported in this package for more
interpretability research. As a preliminary, the detailed documentation on
Penzai can be found at https://penzai.readthedocs.io.

---

[Gemma](https://ai.google.dev/gemma) is a family of open-weights Large Language
Model (LLM) by [Google DeepMind](https://deepmind.google/), based on Gemini
research and technology.

It has been implemented on different platforms:

*   [Gemma on JAX](https://github.com/google-deepmind/gemma)
*   [Gemma on PyTorch](https://github.com/google/gemma_pytorch)
*   [Gemma on Hugging Face](https://huggingface.co/docs/transformers/en/model_doc/gemma)

However, using the above implementations makes it difficult to visualize the
internal mechanism of Gemma. Therefore, we extend the implementation of Gemma
on [Penzai](https://github.com/google-deepmind/penzai), a JAX research toolkit
for building, editing, and visualizing neural networks.

Gemma 1 and Gemma 2 have been supported in original Penzai package, here we
mainly extend the support of Gemma 3 with following new features:

- Vision Transformers (ViTs) and basic components.
- Multi-modal Large Language Models (MLLMs) with both vision encoders and LLM
backbone. The new attention mask is implemented.
- Decoding algorithms for MLLMs.

## Getting Started

If you haven't already installed JAX with TPU support, you should do that first,
since the installation process depends on your platform. You can find
instructions in the
[JAX documentation](https://jax.readthedocs.io/en/latest/installation.html).
Afterwards, you can install our package as

```python
git clone https://github.com/google-deepmind/gemma_penzai.git
cd gemma_penzai
pip install --upgrade pip
pip install -e .
```

and import it and its dependence `penzai` using

```python
import penzai
from penzai import pz
from gemma_penzai import mllm, vision
```

(`penzai.pz` is an *alias namespace*, which makes it easier to reference
common Penzai objects.)

When working in a Colab or IPython notebook, we recommend also configuring
Treescope (Penzai's companion pretty-printer) as the default pretty printer, and
enabling some utilities for interactive use:

```python
import treescope
treescope.basic_interactive_setup(autovisualize_arrays=True)
```

## Using the package

We provide notebooks inside `./notebooks` about the basic usage of Gemma 3,
including the multimodal case. Inside the folder `./notebooks/mech_interp`, we also provide tutorials for [attention sink](https://arxiv.org/abs/2309.17453), [logit-lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens), [gemma scope 1](https://deepmind.google/models/gemma/gemma-scope/) and [2](https://deepmind.google/blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior/).

## Acknowledgements

Our codes are developed based on [Penzai](https://github.com/google-deepmind/penzai)
and [Gemma on JAX](https://github.com/google-deepmind/gemma).