MarkItDownServer

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

MarkItDownServer

(★ 49)

WebServer exposing the markitdown functionality to convert files to markitdown

MarkItDownServer 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MarkItDown Web Server

A production-ready web server application built using FastAPI that receives binary data from various document formats and converts them to Markdown using the MarkItDown library.

> **💡 Quick Answer: Is there a concurrency limit?**  
> By default, the server runs with 1 worker and no rate limiting. You can configure workers and rate limits using environment variables.  
> See [docs/CONCURRENCY_SUMMARY.md](./docs/CONCURRENCY_SUMMARY.md) for a quick guide or [docs/CONCURRENCY.md](./docs/CONCURRENCY.md) for detailed information.

## 🚀 Features

- **Multiple Format Support**: Convert DOC, DOCX, PPT, PPTX, PDF, XLS, XLSX, ODT, ODS, ODP, and TXT files to Markdown
- **FastAPI Framework**: Modern, fast, and well-documented REST API
- **Health Checks**: Built-in health monitoring endpoints
- **Input Validation**: Comprehensive file size, type, and content validation
- **Error Handling**: Robust error handling with detailed error messages
- **CORS Support**: Configurable CORS for web client integration
- **Security Headers**: Built-in security headers middleware
- **Docker Support**: Containerized deployment ready
- **Azure Compatible**: Ready for Azure Container Apps deployment
- **AI Chat Web App Sample**: Full-featured .NET Aspire application with document upload and RAG chat capabilities (see [samples/AiChatWebApp](./samples/AiChatWebApp/README.md))

## 📋 Table of Contents

- [Quick Start](#quick-start)
- [Installation](#installation)
- [Usage](#usage)
- [Samples](#samples)
- [Configuration](#configuration)
- [API Documentation](#api-documentation)
- [Documentation](#documentation)
- [License](#license)

## ⚡ Quick Start

### Using Docker (Recommended)

```bash
# Build the Docker image
docker build -t markitdownserver .

# Run the container
docker run -d --name markitdownserver -p 8490:8490 markitdownserver

# Test the health endpoint
curl http://localhost:8490/health
```

### Using Python (Development)

```bash
# Install dependencies
pip install -r requirements.txt

# Run the server
python app.py
```

The server will be available at `http://localhost:8490`

## 📦 Installation

### Prerequisites

- **Python 3.12+** (for local development)
- **Docker** (for containerized deployment)
- **.NET 9.0 SDK** (for running C# client examples)

### Local Development

1. **Clone the repository**:
   ```bash
   git clone https://github.com/elbruno/MarkItDownServer.git
   cd MarkItDownServer
   ```

2. **Create a virtual environment** (recommended):
   ```bash
   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   ```

3. **Install dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

4. **Run the server**:
   ```bash
   python app.py
   ```

The server will start on `http://0.0.0.0:8490`

### Docker Deployment

1. **Build the Docker image**:
   ```bash
   docker build -t markitdownserver:latest .
   ```

2. **Run the container**:
   ```bash
   docker run -d \
     --name markitdownserver \
     -p 8490:8490 \
     markitdownserver:latest
   ```

3. **Verify the container is running**:
   ```bash
   docker ps | grep markitdownserver
   ```

4. **View logs**:
   ```bash
   docker logs markitdownserver
   ```

5. **Stop the container**:
   ```bash
   docker stop markitdownserver
   docker rm markitdownserver
   ```

## 📖 Usage

### API Endpoints

#### Root Endpoint
```http
GET /
```

Returns service information and available endpoints.

**Response**:
```json
{
  "service": "MarkItDown Server",
  "description": "API for converting documents to Markdown",
  "version": "1.0.0",
  "endpoints": {
    "health": "/health",
    "docs": "/docs",
    "process": "/process_file"
  }
}
```

#### Health Check
```http
GET /health
```

Returns the health status of the service.

**Response**:
```json
{
  "status": "healthy",
  "timestamp": "2025-01-07T12:00:00",
  "service": "MarkItDown Server",
  "version": "1.0.0"
}
```

#### Process File
```http
POST /process_file
```

Upload a document file and receive its content in Markdown format.

**Parameters**:
- `file`: The document file to convert (multipart/form-data)

**Supported File Types**:
- Microsoft Office: DOC, DOCX, XLS, XLSX, PPT, PPTX
- PDF: PDF
- OpenDocument: ODT, ODS, ODP
- Text: TXT

**File Size Limit**: 50MB

**Response**:
```json
{
  "markdown": "# Document Title\n\nContent in markdown format..."
}
```

**Error Responses**:
- `400 Bad Request`: Invalid file type or empty file
- `413 Payload Too Large`: File exceeds 50MB
- `500 Internal Server Error`: Conversion error

### Client Examples

#### Simple Console Application

Located in `samples/SimpleConsole/`, this is a basic example showing minimal code to use the API.

```bash
cd samples/SimpleConsole
dotnet run
```

**Code**:
```csharp
using System.Net.Http.Headers;

HttpClient client = new HttpClient();
string url = "http://127.0.0.1:8490/process_file";
string filePath = "Benefit_Options.pdf";

using (var content = new MultipartFormDataContent())
{
    byte[] fileBytes = File.ReadAllBytes(filePath);
    var fileContent = new ByteArrayContent(fileBytes);
    fileContent.Headers.ContentType = MediaTypeHeaderValue.Parse("application/pdf");
    content.Add(fileContent, "file", Path.GetFileName(filePath));

#### Detailed Console Application

Located in `samples/DetailedConsole/`, this includes comprehensive error handling, configuration, and features.

```bash
cd samples/DetailedConsole
dotnet run
```

**Features**:
- Configuration file support (`appsettings.json`)
- Comprehensive error handling
- Timeout configuration
- File validation
- Colored console output
- Automatic markdown file saving
- Content type detection

**Configuration** (`appsettings.json`):
```json
{
  "MarkItDownServer": {
    "Url": "http://127.0.0.1:8490/process_file",
    "FilePath": "Benefit_Options.pdf",
    "TimeoutMinutes": "5"
  }
}
```

#### AI Chat Web App (Full-Featured Sample)

Located in `samples/AiChatWebApp/`, this is a complete .NET Aspire application with:
- Blazor Server UI with modern chat interface
- Document upload with drag-and-drop support
- Integration with GitHub Models for AI chat
- Retrieval-Augmented Generation (RAG) with vector search
- Real-time document processing and ingestion

```bash
cd samples/AiChatWebApp
# See QUICKSTART.md for detailed setup instructions
dotnet run --project AiChatWebApp.AppHost
```

**Features**:
- Upload documents (PDF, Word, PowerPoint, Excel, Text) through the web UI
- Documents are automatically converted to Markdown via MarkItDown
- Chat with your documents using AI
- Semantic search with citations
- .NET Aspire orchestration with health monitoring

For complete documentation, see [samples/AiChatWebApp/README.md](./samples/AiChatWebApp/README.md) or [QUICKSTART.md](./samples/AiChatWebApp/QUICKSTART.md).

#### cURL Example

```bash
curl -X POST "http://localhost:8490/process_file" \
  -F "file=@document.pdf" \
  -H "Content-Type: multipart/form-data"
```

#### Python Example

```python
import requests

url = "http://localhost:8490/process_file"
files = {"file": open("document.pdf", "rb")}

response = requests.post(url, files=files)
if response.status_code == 200:
    markdown = response.json()["markdown"]
    print(markdown)
else:
    print(f"Error: {response.status_code}")
    print(response.json())
```

#### PowerShell Example

```powershell
$url = "http://localhost:8490/process_file"
$filePath = "document.pdf"

$fileContent = [System.IO.File]::ReadAllBytes($filePath)
$boundary = [System.Guid]::NewGuid().ToString()
$LF = "`r`n"

$bodyLines = (
    "--$boundary",
    "Content-Disposition: form-data; name=`"file`"; filename=`"$(Split-Path $filePath -Leaf)`"",
    "Content-Type: application/pdf$LF",
    [System.Text.Encoding]::UTF8.GetString($fileContent),
    "--$boundary--$LF"
) -join $LF

Invoke-RestMethod -Uri $url -Method Post -ContentType "multipart/form-data; boundary=$boundary" -Body $bodyLines
```

## ⚙️ Configuration

### Environment Variables

The server can be configured using environment variables:

- `PORT`: Server port (default: 8490)
- `HOST`: Server host (default: 0.0.0.0)
- `MAX_FILE_SIZE`: Maximum file size in bytes (default: 52428800 = 50MB)
- `LOG_LEVEL`: Logging level (default: INFO)
- `WORKERS`: Number of worker processes (default: 1)
- `ENABLE_RATE_LIMIT`: Enable rate limiting (default: false)
- `RATE_LIMIT`: Rate limit (default: 60/minute)

### Docker Environment

```bash
docker run -d \
  --name markitdownserver \
  -p 8490:8490 \
  -e PORT=8490 \
  -e MAX_FILE_SIZE=104857600 \
  -e WORKERS=4 \
  -e ENABLE_RATE_LIMIT=true \
  -e RATE_LIMIT=100/minute \
  markitdownserver:latest
```

## 🚦 Concurrency and Performance

### Default Behavior

By default, the server runs with:
- **1 worker process** (single worker)
- **Async request handling** via FastAPI
- **No rate limiting**

### Configuring Concurrency

**Multi-worker setup** for better performance:
```bash
# Run with 4 workers
docker run -d -p 8490:8490 -e WORKERS=4 markitdownserver:latest
```

**Enable rate limiting** to prevent abuse:
```bash
# Limit to 100 requests per minute per IP
docker run -d -p 8490:8490 \
  -e ENABLE_RATE_LIMIT=true \
  -e RATE_LIMIT=100/minute \
  markitdownserver:latest
```

**Note**: Rate limiting requires `slowapi` package. Install with:
```bash
pip install slowapi
```

### Performance Recommendations

- **Small scale** (< 100 req/min): 1-2 workers
- **Medium scale** (100-1000 req/min): 2-4 workers  
- **Large scale** (> 1000 req/min): Use horizontal scaling with load balancer

**📚 For detailed concurrency information**, see [docs/CONCURRENCY.md](./docs/CONCURRENCY.md)

## 🧪 Testing

### Test the Server

1. **Start the server**:
   ```bash
   python app.py
   ```

2. **Run health check**:
   ```bash
   curl http://localhost:8490/health
   ```

3. **Test file conversion**:
   ```bash
   curl -X POST "http://localhost:8490/process_file" \
     -F "file=@samples/SimpleConsole/Benefit_Options.pdf"
   ```

### Run Client Examples

**Simple Console**:
```bash
cd samples/SimpleConsole
dotnet run
```

**Detailed Console**:
```bash
cd samples/DetailedConsole
dotnet run
```

## 🚀 Deployment

### Local Deployment

For development and testing:

```bash
# Using Python
python app.py

# Using uvicorn directly
uvicorn app:app --host 0.0.0.0 --port 8490 --reload
```

### Docker Deployment

For production:

```bash
# Build
docker build -t markitdownserver:1.0.0 .

# Run
docker run -d \
  --name markitdownserver \
  -p 8490:8490 \
  --restart unless-stopped \
  markitdownserver:1.0.0

# View logs
docker logs -f markitdownserver
```

### Azure Container Apps

See the comprehensive [docs/CODE_QUALITY_IMPROVEMENTS.md](./docs/CODE_QUALITY_IMPROVEMENTS.md) document for detailed Azure deployment instructions, including:

- Multi-stage Dockerfile optimization
- Azure CLI deployment scripts
- Bicep templates for Infrastructure as Code
- Environment configuration
- Security best practices
- Cost estimation and optimization

**Quick Azure Deployment**:

```bash
# Set variables
RESOURCE_GROUP="rg-markitdown"
LOCATION="eastus"
CONTAINER_APP_NAME="markitdown-server"

# Create resource group
az group create --name $RESOURCE_GROUP --location $LOCATION

# Deploy (see docs/CODE_QUALITY_IMPROVEMENTS.md for complete script)
az containerapp up \
  --name $CONTAINER_APP_NAME \
  --resource-group $RESOURCE_GROUP \
  --location $LOCATION \
  --source .
```

## 👨‍💻 Development

### Project Structure

```
MarkItDownServer/
├── app.py                          # Main FastAPI application
├── requirements.txt                # Python dependencies
├── dockerfile                      # Docker configuration
├── README.md                       # This file
├── docs/                           # Comprehensive documentation
├── samples/
│   ├── SimpleConsole/              # Basic C# client example
│   │   ├── Program.cs
│   │   ├── SimpleConsole.csproj
│   │   └── Benefit_Options.pdf
│   └── DetailedConsole/            # Advanced C# client example
│       ├── Program.cs
│       ├── DetailedConsole.csproj
│       ├── appsettings.json
│       └── Benefit_Options.pdf
├── src/                            # Legacy client (preserved)
│   └── ...
└── utils/
    └── file_handler.py             # Utility functions
```

### Adding New Features

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

### Code Quality

The project follows Python best practices:
- Type hints for better code clarity
- Comprehensive error handling
- Input validation
- Security headers
- Structured logging

## 📚 API Documentation

The server provides automatic interactive API documentation:

- **Swagger UI**: http://localhost:8490/docs
- **ReDoc**: http://localhost:8490/redoc

These interfaces allow you to:
- Explore all available endpoints
- Test API calls directly from the browser
- View request/response schemas
- See example requests and responses

## 📦 Dependencies

### Python Dependencies

- **fastapi** (0.115.5): Modern web framework for building APIs
- **uvicorn** (0.32.1): ASGI server for FastAPI
- **python-multipart** (0.0.20): Multipart form data support
- **markitdown** (0.0.1a2): Document to Markdown conversion
- **pydantic** (2.10.3): Data validation using Python type hints

### System Requirements

- Python 3.12 or higher
- 512MB RAM minimum (1GB recommended)
- 100MB disk space

## 🔍 Troubleshooting

### Server won't start

**Issue**: Port already in use
```
Error: [Errno 48] Address already in use
```

**Solution**: Change the port or stop the conflicting service
```bash
# Find process using port 8490
lsof -i :8490

# Kill the process
kill -9 <PID>

# Or use a different port
python app.py --port 8491
```

### File conversion fails

**Issue**: "File type not allowed"

**Solution**: Ensure your file has a supported extension (doc, docx, pdf, etc.)

**Issue**: "File too large"

**Solution**: Files must be under 50MB. Compress or split large files.

### Docker issues

**Issue**: Cannot connect to Docker daemon

**Solution**: Ensure Docker Desktop is running
```bash
docker ps  # Test Docker connection
```

**Issue**: Container exits immediately

**Solution**: Check container logs
```bash
docker logs markitdownserver
```

## 📞 Support

For issues, questions, or contributions:

- **GitHub Issues**: [Create an issue](https://github.com/elbruno/MarkItDownServer/issues)
- **Documentation**: See [docs/](./docs/)

## 📄 License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- Built with [FastAPI](https://fastapi.tiangolo.com/)
- Powered by [MarkItDown](https://github.com/microsoft/markitdown)
- Developed by [El Bruno](https://github.com/elbruno)

## 📚 Documentation

Comprehensive documentation is available in the [docs](./docs/) directory:

- **[Quick Reference](./docs/QUICK_REFERENCE.md)** - Common commands and API usage
- **[Developer Manual](./docs/DEVELOPER_MANUAL.md)** - Integration guide for developers
- **[Concurrency Guide](./docs/CONCURRENCY.md)** - Performance and scaling information
- **[Code Quality](./docs/CODE_QUALITY_IMPROVEMENTS.md)** - Best practices and improvements
- **[Implementation Plans](./docs/plans/)** - Detailed feature implementation plans

### Sample Applications

- **[AI Chat Web App](./samples/AiChatWebApp/)** - Full .NET Aspire application with:
  - Document upload and conversion
  - RAG-based chat with semantic search
  - Vector store integration
  - Real-time markdown preview
  - [Quick Start Guide](./samples/AiChatWebApp/QUICKSTART.md)
  - [User Manual](./samples/AiChatWebApp/docs/USER_MANUAL.md) *(coming soon)*

## 📈 Version History

- **v1.0.0** (2025-01): Initial release with production-ready features
  - Multi-format document conversion
  - Comprehensive error handling
  - Health check endpoints
  - Docker support
  - Azure deployment ready
  - AI Chat Web App sample with .NET Aspire

---

**Ready to convert your documents to Markdown?** 🚀

Start the server and visit http://localhost:8490/docs to explore the interactive API documentation!

MarkItDownServer

MarkItDownServer 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

MarkItDownServer

MarkItDownServer 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README