Scrapling
π·οΈ λ¨μΌ μμ²λΆν° λκ·λͺ¨ ν¬λ‘€λ§κΉμ§ λͺ¨λ κ²μ μ²λ¦¬νλ μ μν Web Scraping νλ μμν¬μ λλ€!
νμΌ νμκΈ°
μ΅μ’ λ²μ λ€μ΄λ‘λ (.zip)- 01-bug_report.yml
- 02-feature_request.yml
- 03-other.yml
- 04-docs_issue.yml
- config.yml
- code-quality.yml
- docker-build.yml
- release-and-publish.yml
- tests.yml
- FUNDING.yml
- PULL_REQUEST_TEMPLATE.md
- 01_fetcher_session.py
- 02_dynamic_session.py
- 03_stealthy_session.py
- 04_spider.py
- README.md
- choosing.md
- dynamic.md
- static.md
- stealthy.md
- scrapy.md
- adaptive.md
- main_classes.md
- selection.md
- advanced.md
- architecture.md
- generic-templates.md
- getting-started.md
- platform-templates.md
- proxy-blocking.md
- requests-responses.md
- sessions.md
- mcp-server.md
- migrating_from_beautifulsoup.md
- LICENSE.txt
- SKILL.md
- README.md
- Scrapling-Skill.zip
- mcp-server.md
- custom-types.md
- fetchers.md
- mcp-server.md
- proxy-rotation.md
- response.md
- selector.md
- spiders.md
- cover_dark.png
- cover_dark.svg
- cover_light.png
- cover_light.svg
- favicon.ico
- logo.png
- main_cover.png
- scrapling_shell_curl.png
- spider_architecture.png
- star-history.png
- extract-commands.md
- interactive-shell.md
- overview.md
- adaptive_storage_system.md
- scrapling_custom_types.md
- choosing.md
- dynamic.md
- static.md
- stealthy.md
- scrapy.md
- main.html
- adaptive.md
- main_classes.md
- selection.md
- advanced.md
- architecture.md
- generic-templates.md
- getting-started.md
- platform-templates.md
- proxy-blocking.md
- requests-responses.md
- sessions.md
- extra.css
- migrating_from_beautifulsoup.md
- replacing_ai.md
- benchmarks.md
- donate.md
- index.md
- overview.md
- README_AR.md
- README_CN.md
- README_DE.md
- README_ES.md
- README_FR.md
- README_JP.md
- README_KR.md
- README_PT_BR.md
- README_RU.md
- requirements.txt
- BirdProxies.jpg
- coldproxy.png
- CoreClaw.jpg
- DataImpulse.png
- decodo.png
- evomi.png
- HyperSolutions.png
- niuproxy.png
- NodeMaven.jpg
- novada.jpg
- petrosky.png
- proxidize.png
- proxiware.png
- ProxyEmpire.png
- SerpApi.png
- SwiftProxy.png
- TikHub.jpg
- TWSC.png
- webshare.png
- __init__.py
- _shell.py
- _utils.py
- __init__.py
- _shell_signatures.py
- _types.py
- ai.py
- custom_types.py
- mixins.py
- shell.py
- storage.py
- translator.py
- __init__.py
- _base.py
- _config_tools.py
- _controllers.py
- _page.py
- _stealth.py
- _types.py
- _validators.py
- __init__.py
- ad_domains.py
- convertor.py
- custom.py
- fingerprints.py
- navigation.py
- proxy_rotation.py
- __init__.py
- constants.py
- static.py
- __init__.py
- chrome.py
- requests.py
- stealth_chrome.py
- __init__.py
- scrapy.py
- __init__.py
- _utils.py
- crawler.py
- feed.py
- shopify.py
- sitemap.py
- __init__.py
- cache.py
- checkpoint.py
- engine.py
- links.py
- request.py
- result.py
- robotstxt.py
- scheduler.py
- session.py
- spider.py
- throttle.py
- __init__.py
- cli.py
- parser.py
- py.typed
- __init__.py
- test_ai_mcp.py
- __init__.py
- test_cli.py
- test_shell_functionality.py
- __init__.py
- test_shell_core.py
- test_storage_core.py
- __init__.py
- test_dynamic.py
- test_dynamic_session.py
- test_requests.py
- test_requests_session.py
- test_stealth.py
- test_stealth_session.py
- __init__.py
- test_dynamic.py
- test_requests.py
- test_requests_session.py
- test_stealth_session.py
- __init__.py
- test_base.py
- test_constants.py
- test_impersonate_list.py
- test_merge_request_args.py
- test_pages.py
- test_proxy_rotation.py
- test_response_handling.py
- test_utils.py
- test_validator.py
- __init__.py
- test_scrapy.py
- __init__.py
- test_adaptive.py
- test_ancestor_navigation.py
- test_attributes_handler.py
- test_find_similar_advanced.py
- test_general.py
- test_parser_advanced.py
- test_selectors_filter.py
- __init__.py
- test_cache.py
- test_checkpoint.py
- test_engine.py
- test_feed.py
- test_force_stop_checkpoint.py
- test_links.py
- test_request.py
- test_result.py
- test_robotstxt.py
- test_scheduler.py
- test_session.py
- test_shopify.py
- test_sitemap.py
- test_spider.py
- test_templates.py
- test_throttle.py
- __init__.py
- requirements.txt
- .bandit.yml
- .dockerignore
- .gitignore
- .pre-commit-config.yaml
- .readthedocs.yaml
- AI_POLICY.md
- benchmarks.py
- cleanup.py
- CODE_OF_CONDUCT.md
- CONTRIBUTING.md
- Dockerfile
- LICENSE
- MANIFEST.in
- pyproject.toml
- pytest.ini
- README.md
- ROADMAP.md
- ruff.toml
- server.json
- setup.cfg
- tox.ini
- zensical.toml
π μ€μΉ κ°μ΄λ
git clone https://github.com/D4Vinci/Scrapling
κΉνλΈμμ νλ‘μ νΈ μ½λ μ 체λ₯Ό λ΄ μ»΄ν¨ν°λ‘ λ΄λ €λ°μ΅λλ€.
cd Scrapling
λ°©κΈ λ΄λ €λ°μ νλ‘μ νΈ ν΄λ μμΌλ‘ μ΄λν©λλ€.
2. 곡μ μ€μΉ μ€ν¬λ¦½νΈ
μ¬μ μΆμ²- Python 3 pip λͺ λ Ήμ΄λ₯Ό μ°λ €λ©΄ Pythonμ΄ νμν©λλ€.
pip install scrapling
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[fetchers]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[ai]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[shell]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
μ΄ λ ν¬μ READMEμ μ ν μ€μ λͺ λ Ήμ΄λ₯Ό κ·Έλλ‘ κ°μ Έμμ΅λλ€.
3. Docker
μ¬μ- Git GitHubμμ νλ‘μ νΈ μ½λλ₯Ό λ΄λ €λ°μΌλ €λ©΄ νμν©λλ€.
- Docker Desktop 컨ν μ΄λλ₯Ό λΉλνκ³ μ€ννλ €λ©΄ νμν©λλ€. μ€μΉ ν μ€νν΄μ λ°±κ·ΈλΌμ΄λμ μΌλμΈμ.
docker pull pyd4vinci/scrapling
μ΄ λͺ λ Ήμ΄λ₯Ό ν°λ―Έλμ κ·Έλλ‘ μ λ ₯ν΄ μ€ννμΈμ.
docker pull ghcr.io/d4vinci/scrapling:latest
μ΄ λͺ λ Ήμ΄λ₯Ό ν°λ―Έλμ κ·Έλλ‘ μ λ ₯ν΄ μ€ννμΈμ.
<a href="https://hub.docker.com/r/pyd4vinci/scrapling" target="_blank">
μ΄ λͺ λ Ήμ΄λ₯Ό ν°λ―Έλμ κ·Έλλ‘ μ λ ₯ν΄ μ€ννμΈμ.
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pyd4vinci/scrapling?labelColor=%20%23FDB062&logo=Docker&labelColor=%20%23528bff"></a>
μ΄ λͺ λ Ήμ΄λ₯Ό ν°λ―Έλμ κ·Έλλ‘ μ λ ₯ν΄ μ€ννμΈμ.
μ΄ λ ν¬μ READMEμ μ ν μ€μ λͺ λ Ήμ΄λ₯Ό κ·Έλλ‘ κ°μ Έμμ΅λλ€.
4. Python
μ¬μpip install scrapling
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[fetchers]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[ai]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[shell]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
pip install "scrapling[all]"
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
μ΄ λ ν¬μ READMEμ μ ν μ€μ λͺ λ Ήμ΄λ₯Ό κ·Έλλ‘ κ°μ Έμμ΅λλ€.
