KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #codex #typescript #ai-agents #claude-code #dsh #dsh-plugin #deepseek-harness #cli #macos #ai-agent #open-source

scrapy-spider-example

★ 79 Open GitHub ↗

Scrapy spider example for Scrapy Tutorial Series

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This project is a Scrapy spider example collection, [Michael Yin](https://blog.michaelyin.info/) create this project to host the source code of [Scrapy Tutorial Series: Web Scraping Using Python](https://blog.michaelyin.info/scrapy-tutorial-series-web-scraping-using-python/)

You can find Scrapy spider example code which can help you:

1. A simple Scrapy spider shows you how to extract data from the web page.

2. How to handle pagination in Scrapy spider.

3. A simple script which can make your Scrapy shell more powerful.

4. How to define Scrapy item, and how to create a custom Item Pipeline to save the data of Item into Databases such as Mysql or PostgreSQL.

5. All the code can run without problem in Python2 and Python3

## Warning

When you run the code in your local env, make sure to edit `CONNECTION_STRING` in `scrapy_spider/settings.py`. If you want more detail about this project, just check the [Scrapy Tutorial Series: Web Scraping Using Python](https://blog.michaelyin.info/scrapy-tutorial-series-web-scraping-using-python/)

## Feedback

If you have any problem, feel free to fire issues in Github, I will reply ASAP.

## Contact

You can contact me `admin#michaelyin.info`

scrapy-spider-example

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README