Web-Scraping-Projects

KO | EN

gitlite — search

#typescript #ai-agents #ai #dsh-plugin #deepseek-harness #open-source #claude-code #codex #cli #developer-tools #react #windows

Web-Scraping-Projects

★ 81 Open GitHub ↗

A collection of web scraping projects to practice your skills or build a portfolio

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Web-Scraping-Projects
A collection of web scraping projects to practice your skills or build a portfolio.

| Project | Description |
|---|---|
| [Amazon](https://github.com/israel-dryer/Amazon-Scraper) | product and price data |
| [Indeed.com](https://github.com/israel-dryer/Indeed-Job-Scraper) | job postings based on search criteria | 
| [Yahoo! Finance](https://github.com/israel-dryer/Yahoo-Finance-Scraper) | financial, company and historical stock data |
| [Salary.com](https://github.com/israel-dryer/Salary-Dot-Com-Scraper) | salary statistics based on specific search criteria |
| [Yahoo! News](https://github.com/israel-dryer/Yahoo-News-Scraper) | news article data including summmary based on search criteria |
| [Twitter](https://github.com/israel-dryer/Twitter-Scraper) | scrape twitter data |
| [Ebay](https://github.com/israel-dryer/Ebay-Scraper) | scrape ebay searches |

## Requirements
A few of these projects use browser automation, but most do not. For this, I use **Selenium** to automate the browser.  Other libraries that are used include:
- Requests
- BeautifulSoup
- lxml

## Expectations
These projects are designed to give you experience web scraping, but assume that you have some basic familiarity with at least Requests and BeautifulSoup. Selenium is not used extensively enough to need familiarity, but you will need to install it on the few projects that require it.

## Caution
While I will try to keep these projects updated, please keep in mind that websites can change at any time, rendering an existing scraper useless. This is unfortunately the nature of webscraping. Your production models will require constant attention and maintenance to ensure they are delivering the data and results that you expect.

Web-Scraping-Projects

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README