model-memory-calculator

KO | EN

gitlite — search

#hacktoberfest #react #ai #go #machine-learning #rust #deep-learning #linux #windows #awesome-list #pytorch #security #openai #mcp

model-memory-calculator

(★ 84)

Simple model memory requirements calculator for GGUF

model-memory-calculator 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# GGUF Metadata Reader (Browser)

A single-file, static web app to read GGUF model metadata directly in the browser and estimate memory usage (RAM/VRAM) for a chosen context window and KV cache quantization.

- Works with remote URLs that support HTTP Range requests (e.g., many Hugging Face files)
- Works with local `.gguf` files (drag-and-drop via file picker)
- Detects sharded models (e.g., `-00001-of-00013`) and sums total size
- No server required; everything runs client-side

## Quick Start

Option A: Open the file directly
1. Open `index.html` in a modern browser (Chrome, Edge, Safari).
2. Paste a GGUF URL or choose a local `.gguf` file and click the corresponding button.

Option B: Serve locally (helps with some CORS setups)
```bash
cd path/to/model-memory-calculator
python -m http.server 8000
# Then open http://localhost:8000 in your browser
```

## Usage

- GGUF URL: Paste a direct link to a `.gguf` (e.g., a Hugging Face “resolve/main” URL). Many hosts allow partial download via HTTP Range.
- Or choose a local GGUF file: Uses the browser’s File API; no upload leaves your machine.
- Context size (tokens): Select a preset (e.g., 4K, 16K, 128K) or choose "Custom…" and enter any positive integer token length.
- KV cache quantization: Choose how keys/values are stored in memory. Options show approximate bytes per value.
- Verbose: Prints debug logs of what’s read and how size is determined.

Click “Read URL” or “Read File”. If successful, you’ll see:
- Extracted params: `attention_heads`, `kv_heads`, `hidden_layers`, `hidden_size`, `split_count` (if present)
- Memory estimate: model size + KV cache size at your chosen context/quantization

## How It Works

- GGUF parsing: Reads just enough of the GGUF header to extract:
  - `.attention.head_count`
  - `.attention.head_count_kv`
  - `.block_count`
  - `.embedding_length`
  - `split.count` (if present)
- Remote file size:
  - Tries `HEAD` to get `Content-Length`.
  - Falls back to a `Range: bytes=0-0` request and reads `Content-Range`.
- Sharded models:
  - Detects `-00001-of-000NN` style patterns in URLs or uses `split.count` metadata.
  - Sums sizes across parts (remote) or estimates total from a single shard (local) when possible.
- KV cache estimate:
  - Uses a simplified formula: `bytes_per_value × hidden_size × hidden_layers × context_tokens`.
  - Shows total as: `Model + KV` (MB/GB). Actual usage can vary by backend/implementation.

## Notes & Limitations

- GGUF versions: Supports GGUF v1–v3 headers for the fields listed above.
- CORS & Range: Remote hosts must allow cross-origin requests and HTTP Range. If not, size detection may fail; download the file and use the local option instead.
- Range ignored: Some servers respond `200` without honoring `Range`. The app avoids downloading the full body for size only; estimates can fail in this case.
- Sanity limits: Very long strings/arrays in metadata are bounded to avoid huge reads.
- Estimates only: KV cache math is intentionally simplified. Different runtimes store KV differently (e.g., layout, precision, per-head factors).

## Troubleshooting

- “Failed to read params.”
  - The file may not be GGUF or uses unsupported/unexpected metadata. Try another file or update the URL.
- “Could not determine file size or compute usage (CORS/Range?).”
  - The remote host may block CORS or not report size via `HEAD`/`Range`. Try serving the page locally, a different host, or the local file picker.
- Split detection issues
  - Ensure URLs use a stable pattern (e.g., `-00001-of-000xx`) or that `split.count` is present in metadata.

## Privacy

- The local file option never uploads your file; parsing happens entirely in your browser.
- For remote URLs, the app performs small range requests to read the header and determine file size. It aborts early once required metadata is read.

## Development

- No build step required. The app is a single page:
  - `index.html` — All logic and UI
- Open in a browser or serve with any static server.

## License

This project is licensed under the terms in `LICENSE`.

model-memory-calculator

model-memory-calculator 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

model-memory-calculator

model-memory-calculator 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README