image_scraper

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

(★ 8)

You do want to find high quality image with custom setting? here is what you need with Search Engines Image Scraper You can easily find and download image with custom size , color and ratio. Let's start and Start project if you want help to improve It's My honor to check Your pull request.

image_scraper 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Search Engines Image Scraper
You do want to find high quality image with custom setting? here is what you need with *Search Engines Image Scraper* You can easily find and download image with custom size , color and ratio.
Let's start and Start project if you want help to improve It's My honor to check Your pull request.

![wellcome.gif](https://raw.githubusercontent.com/moaminsharifi/image_scraper/master/wellcome.gif)

# Table of Content
1. requirement and installation
2. Usage  
    - Cli
    - Pythn module
3. Defindes
    - image ratio
    - image size
    - image color
    - table of paramters
    - Why DuckDuckGo?
4. helpful link

---
## requirement:
  1. python 3+ (it can use also for python 2 but need to change)
  2. python packges: selenium , numpy , cv2

### install requirement (freeze version):
- install with `pip install -r requirement.txt'` or `pip3 install -r requirement.txt` 
---

# Usage:
## 1. CLI 
open Your shell in repo dir and enter (python3 also for some cases)
##### *CLI version just accept two paramanter*
`python image_scraper.py  --qeury string --amount int `

for sample here I am download amazon logo

`python image_scraper.py  --qeury "amazon logo" --amount 10 `

for get help wirte:

`python image_scraper.py -h`

## 2. python module
1. Import it as normal module
```python 
from image_scraper import Scraper_Config , Image_Scraper
```
2. Create config (also you can use default config)
```python 
config = Scraper_Config(SEARCH_QUERY = 'duckduckgo',
                 NUMBER_OF_PICTURES = 5,
                 IMAGE_RATIO = (16, 9),
                 IMAGE_SIZE = (200 , 100),
                 CHECK_RATIO_AND_RESIZE = False,
                 EPSILON_RATIO_ERROR = 0.5 ,
                 MAKE_GRAY = False,
                 SEARCH_ENGINE_TYPE='duckduckgo',
                 MINIMUM_ELEMENT_ERROR = 0.2,
                 SLEEP_TIME = 0,
                 SCROLL_PAUSE_TIME = 2)
```
3. Create Image_Scraper isntace and call **scrap** function

```python 
scraper = Image_Scraper(config)
scraper.scrap(verbose=True)
```

---
# Defindes
## aspect ratios
#### what is  aspect ratio and why is important?
![aspect ratios](https://raw.githubusercontent.com/moaminsharifi/image_scraper/master/aspect-ratios_orig.png)
Maybe you known aspect ratio in images but why is important to us?

Let's check an example, our task is find great mohammad ali picture from google or duckduckgo.
![ratio.png](https://raw.githubusercontent.com/moaminsharifi/image_scraper/master/ratio.png)
in this system you can easily define image ratio and which size if ratio with some gap is neer to ratio accept it else next one. look at paramter table for more details.

## parameter table for better config

## Why DuckDuckGo?
[duckduckgo](https://duckduckgo.com/) say "Privacy, simplified ..." why here I am not talking about **Privacy** here we need Scrap web. Google Check ip, Behivaer in Browser and almost all thing with **Privacy** option you can even scrap it with [TOR network](https://www.torproject.org/)
---
# helpful link
## some issue I got maybe helpful for you:
- FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'
[link 1 stackoverflow](https://stackoverflow.com/questions/50461779/geckodriver-executable-needs-to-be-in-path) - [link 2 askubuntu](https://askubuntu.com/questions/851401/where-to-find-geckodriver-needed-by-selenium-python-package
)

- [install last Chrome on ubuntu](https://gist.github.com/mrtns/78d15e3263b2f6a231fe)

- [install Chrome web driver ubuntu](https://medium.com/@gerrysabar/scraping-linkedin-profile-using-python-selenium-88cb64888cf
)
--
# todo list
- [ ] add more example 
- [ ] add google search engine
- [ ] add more browser driver 
- [ ] add batch image downloader
- [ ] add image instant augment
- [x] add multi threading support
- [ ] add just resize without checking ratio
- [ ] add colab support and colab file exmaple

image_scraper

image_scraper 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

image_scraper

image_scraper 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README