forum-image-scraper

KO | EN

gitlite — search

#javascript #python #react #ai #machine-learning #rust #docker #java #nodejs #cli #kubernetes #pytorch #openai #open-source

forum-image-scraper

(★ 18)

Download images from forum threads

forum-image-scraper 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Forum Image Scraper

Download images from paginated forum threads with parallel processing and duplicate detection.

## Limitations

- No support for forums that require login or forums protected by services like Cloudflare.
## Quick Start

1. Install dependencies: `pip install -r requirements.txt`
2. **Configure `input.properties`** - This file is required and contains all settings (forum URL, threads, page range, filters)
3. Run: `python forum-image-scraper.py`

## Output Structure

Images are saved in a hierarchical folder structure matching the thread path:
```
../hostname/
  └── path/
      └── to/
          └── thread/
              ├── p1_abc12345.jpg
              ├── p1_def67890.jpg
              ├── p2_ghi11223.jpg
              └── ...
```

- `hostname`: Extracted domain name from the forum
- Full path hierarchy created from thread URL
- `pX_hash.jpg`: Page number and unique hash identifier

## Features

- Parallel downloads (10 workers) for speed
- Resolution filtering to skip small images
- Hash-based duplicate detection across all pages
- Progress feedback and statistics

forum-image-scraper

forum-image-scraper 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

forum-image-scraper

forum-image-scraper 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README