webscraping-and-ai

KO | EN

gitlite — search

#javascript #python #typescript #llm #golang #android #rust #deep-learning #docker #awesome #cli #awesome-list #vue #ios

webscraping-and-ai

(★ 11)

presentations about web scraping and ai (agents)

webscraping-and-ai 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Is the Future of Web Scraping Agentic?
A comprehensive exploration of modern web scraping techniques that combine traditional approaches with AI-powered agents. This repository demonstrates the evolution from static scraping to intelligent, autonomous web data extraction using LLMs and agent frameworks.
# Overview
Traditional web scraping follows a linear pipeline: Discover → Crawl → Scrape → Extract. Agentic scraping introduces intelligent decision-making, adaptive strategies, and autonomous navigation through AI agents that can reason about web content and interactions.
## Content

- CODECON 2025 #BRATISLAVA  [Slides](/slides/CODECON%202025%20Web%20Scraping%20Agents.pdf) / [Code](/src/) / [Video](https://www.youtube.com/watch?v=l3dj5huaJwM)
- CODECON 2024 #ZILINA  [Slides](https://github.com/williambrach/webscraping-and-ai/blob/codecon-2024/slides/CODECON%202024%20AI%20Scraping.pdf) / [Code](https://github.com/williambrach/webscraping-and-ai/blob/codecon-2024/src/webscrape_examples.ipynb) / [Video](https://www.youtube.com/watch?v=fhTsBuzMmVI)

## Installation Guide

1. Create environment:
```bash
uv venv --python 3.12
```

2. Activate environment:
```bash
source .venv/bin/activate
```

3. Install dependencies:
```bash
uv sync
```

4. Install Playwright:
```bash
uv run playwright install chromium
```

## Environment Setup

Create a `.env` file with:
```plaintext
LITELLM_API_KEY=
LITELLM_URL=
FIRECRAWL_API_KEY=
```

webscraping-and-ai

webscraping-and-ai 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

webscraping-and-ai

webscraping-and-ai 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README