deepseek-ocr-2-client

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

deepseek-ocr-2-client

★ 14 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# DeepSeek-OCR-2 Client

A high-performance OCR service based on vLLM, wrapping DeepSeek-OCR-2 for document recognition.

[中文文档](README_CN.md)

## Features

- 🚀 High-performance inference based on vLLM
- 📄 Document to Markdown conversion (preserving format and structure)
- 📑 PDF file parsing (automatic page handling)
- 🖼️ Automatic image region cropping with base64 embedding
- 🎯 Multiple OCR scenarios (document conversion, chart parsing, image description, etc.)
- 🐳 One-click Docker deployment
- 🔧 Multi-GPU parallel inference support

## Requirements

- NVIDIA GPU with at least 24GB VRAM (RTX 4090 or higher recommended)
- Docker 20.10+
- NVIDIA Container Toolkit

## Quick Start

### 1. Download Model

```bash
python download_model.py -d /path/to/models/
```

Or download manually:
```bash
huggingface-cli download deepseek-ai/DeepSeek-OCR-2 --local-dir /path/to/DeepSeek-OCR-2
```

### 2. Start Service

```bash
# If downloaded via download_model.py, model path is auto-detected
sh start.sh -gpu 0

# Specify port and memory utilization
sh start.sh -gpu 0 -port 8080 -mem 0.8

# Manually specify model path
sh start.sh -gpu 0 -model /path/to/DeepSeek-OCR-2

# Multi-GPU parallel
sh start.sh -gpu 0,1
```

### 3. Call API

```bash
# Single image OCR
curl -X POST http://localhost:20000/v1/ocr \
    -F "file=@your_image.jpg"

# Batch recognition
curl -X POST http://localhost:20000/v1/ocr/batch \
    -F "files=@image1.jpg" \
    -F "files=@image2.jpg"

# Document parsing (supports PDF and images)
curl -X POST http://localhost:20000/parse \
    -F "file=@document.pdf" \
    -F "type=json"

# Get Markdown file
curl -X POST http://localhost:20000/parse \
    -F "file=@document.pdf" \
    -F "type=file" \
    -o result.md
```

## API Documentation

After starting the service, visit: http://localhost:20000/docs

### GET /health

Health check endpoint, returns service status and model info.

### POST /v1/ocr

OCR recognition for a single image.

**Parameters:**
- `file`: Image file (required)
- `prompt`: OCR prompt (default: `<image>\n<|grounding|>Convert the document to markdown.`)
- `max_tokens`: Maximum generated tokens (default: 8192)
- `temperature`: Generation temperature (default: 0.0)
- `embed_images`: Whether to embed cropped images as base64 (default: true)

**Response:**
```json
{
    "markdown": "# Title\n\nContent...",
    "raw_output": "Raw model output (with coordinate tags)",
    "processing_time": 2.5
}
```

### POST /v1/ocr/batch

Batch OCR recognition for multiple images.

**Parameters:**
- `files`: List of image files (required)
- `prompt`: OCR prompt
- `max_tokens`: Maximum generated tokens
- `embed_images`: Whether to embed images

**Response:**
```json
{
    "results": [
        {
            "filename": "image1.jpg",
            "markdown": "...",
            "processing_time": 2.5
        }
    ]
}
```

### POST /parse

Universal document parsing endpoint, supports PDF and image files.

**Parameters:**
- `file`: Document file (PDF or image: jpg, jpeg, png, bmp, tiff, webp)
- `type`: Return type
  - `json`: Return JSON format (default)
  - `file`: Return Markdown file

**Response (JSON format):**
```json
{
    "filename": "document.pdf",
    "pages": 3,
    "markdown": "Complete Markdown content",
    "pages_detail": [
        {"page": 1, "markdown": "Page 1 content"},
        {"page": 2, "markdown": "Page 2 content"},
        {"page": 3, "markdown": "Page 3 content"}
    ],
    "processing_time": 15.2
}
```

## Available Prompts

| Scenario | Prompt |
|----------|--------|
| Document Conversion | `<image>\n<|grounding|>Convert the document to markdown.` |
| General OCR | `<image>\n<|grounding|>OCR this image.` |
| Layout-free OCR | `<image>\nFree OCR.` |
| Chart Parsing | `<image>\nParse the figure.` |
| Image Description | `<image>\nDescribe this image in detail.` |

## Configuration

Configure via environment variables or `start.sh` parameters:

| Parameter | Environment Variable | Default | Description |
|-----------|---------------------|---------|-------------|
| `-model` | `MODEL_PATH` | Auto-detected | Model path |
| `-len` | `MAX_MODEL_LEN` | `8192` | Maximum sequence length |
| `-mem` | `GPU_MEMORY_UTILIZATION` | `0.75` | GPU memory utilization |
| `-gpu` | - | `0` | GPU ID(s) to use (multi-GPU: `0,1`) |
| `-port` | - | `20000` | Service port |
| - | `TENSOR_PARALLEL_SIZE` | Auto-calculated | Tensor parallel size (based on GPU count) |

## Project Structure

```
deepseek-ocr-2-client/
 ├── app/
 │   └── main.py              # FastAPI main service
 ├── deepseek_ocr2_vllm/      # Official vLLM integration code
 │   ├── config.py            # Configuration file
 │   ├── deepseek_ocr2.py     # vLLM model definition
 │   ├── deepencoderv2/       # Vision encoder (Qwen2 + SAM)
 ▔   └── process/             # Image processing
 ├── model/                   # Model code copy
 ├── Dockerfile               # Docker image build file
 ├── docker-entrypoint.sh     # Container entry script
 ├── start.sh                 # Service startup script
 ├── download_model.py        # Model download tool
 └── requirements.txt         # Python dependencies
```

## Troubleshooting

### Out of Memory
Reduce `GPU_MEMORY_UTILIZATION`:
```bash
sh start.sh -gpu 0 -mem 0.6
```

Or use multi-GPU:
```bash
sh start.sh -gpu 0,1
```

### Model Loading Failed
Ensure you're using the complete model path and have installed dependencies required for `trust_remote_code=True`.

### PDF Processing Failed
Ensure PyMuPDF is installed:
```bash
pip install PyMuPDF
```

### View Logs
```bash
docker logs -f deepseek-ocr-2-service
```

### Stop Service
```bash
docker stop deepseek-ocr-2-service
```

## Dependencies

- **vLLM**: High-performance inference engine (pre-installed in Docker image)
- **FastAPI**: Web framework
- **Transformers**: Model loading
- **PyMuPDF**: PDF processing (optional)
- **Pillow**: Image processing

## References

- [DeepSeek-OCR-2 Official Repository](https://github.com/deepseek-ai/DeepSeek-OCR-2)
- [vLLM Documentation](https://docs.vllm.ai/)

## License

This project is for learning and research purposes only. Model weights are subject to DeepSeek license.

deepseek-ocr-2-client

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README