ShortVideoCrawl

KO | EN

gitlite — search

#hacktoberfest #react #ai #typescript #llm #android #linux #nodejs #cli #kubernetes #vue #ios #mcp #openai

ShortVideoCrawl

(★ 14)

Short video crawler based on scrapy

ShortVideoCrawl 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ShortVideoCrawl

[![GitHub](https://img.shields.io/github/license/dxsooo/ShortVideoCrawl)](https://raw.githubusercontent.com/dxsooo/ShortVideoCrawl/master/LICENSE)
[![CodeFactor](https://www.codefactor.io/repository/github/dxsooo/shortvideocrawl/badge)](https://www.codefactor.io/repository/github/dxsooo/shortvideocrawl)

Short video crawler based on [scrapy](https://github.com/scrapy/scrapy), crawling with search query of the target sites.

Supports:

> \*度小视/全民小视频官网已经下线，但是目前本项目仍可用（2024.6测试）

## Usage

requirements:

- python 3.10+
- poetry

### prepare

```bash
git clone https://github.com/dxsooo/ShortVideoCrawl
cd ShortVideoCrawl
poetry install --only main
poetry shell
```

### run

For example:

```bash
cd shortvideocrawl

# main parameters:
#   query: query word
#   count: target video count

# kuaishou
scrapy crawl kuaishou -a query='蔡徐坤' -a count=50

# xigua, with highest resolution and size smaller than 64 MB, duration smaller than 5 min
# scrapy crawl ixigua -a query='蔡徐坤' -a count=50

# xinpianchang, with highest resolution and size smaller than 64 MB, duration smaller than 5 min, but can only get a fixed number of video
scrapy crawl xinpianchang -a query='蔡徐坤'

# haokan, with highest resolution
# scrapy crawl haokan -a query='蔡徐坤' -a count=50

# quanmin
scrapy crawl quanmin -a query='蔡徐坤' -a count=50
```

videos are saved in `./videos`, named with video id of source platform.

ShortVideoCrawl

ShortVideoCrawl 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ShortVideoCrawl

ShortVideoCrawl 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README