hubscraper

KO | EN

gitlite — search

#python #react #typescript #llm #golang #machine-learning #rust #deep-learning #nodejs #windows #vue #awesome-list #ios #open-source

(★ 9)

Implementing Docker Hub Scraper using Python, Selenium and Docker Desktop

hubscraper Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

![stars](https://img.shields.io/github/stars/collabnix/hubscraper)
![forks](https://img.shields.io/github/forks/collabnix/hubscraper)
![Discord](https://img.shields.io/discord/1020180904129335379)
![issues](https://img.shields.io/github/issues/collabnix/hubscraper)
![Visitor count](https://shields-io-visitor-counter.herokuapp.com/badge?page=collabnix.hubscraper)
![Twitter](https://img.shields.io/twitter/follow/collabnix?style=social)

# Hubscraper - Scraping Docker Hub for Official Images and Extensions
A Docker image that scraps Docker Hub official and verfied Images.

A Handy Python script for web scraping dynamically the Docker Hub website. 
This script is capable of fetching a list of Docker Extensions from the Docker Hub

## Getting Started

## Pre-requisite

- Install Python 3.9+
- Download [Chrome Driver](https://chromedriver.storage.googleapis.com/index.html?path=108.0.5359.71/)

## Buiding it locally

### Clone the repository

```
git clone https://github.com/collabnix/hubscraper/
```

### Install the required modules

```
pip3 install -r requirements.txt
```

### Modify the script

Go to Line 17 and make the necessary changes:

```
# Change the base_dir with your path.
base_dir = '/Users/ajeetraina/Downloads' + os.sep

# MS Edge Driver
# driver = webdriver.Edge(service=Service(EdgeChromiumDriverManager().install()))

# Safari Driver

csv_file = open('results.csv', 'w')

# create the csv writer
writer = csv.writer(csv_file)

writer.writerow(['Image Name','Downloads','Stars'])

driver = webdriver.Chrome(executable_path = "/Users/ajeetraina/Downloads/chromedriver\ 3")
```

### Execute the script

```
python3 scraper.py
```

## Building with Docker

```
git clone https://github.com/collabnix/hubscraper/
docker build -t ajeetraina/hubscraper .
```

## Running the Hubscraper in a Docker container

```
docker run --platform=linux/amd64 -it -w /app -v $(pwd):/app ajeetraina/scraperhubb bash
root@960e8b9fa2c2:/usr/workspace# python scraper.py 
[WDM] - Downloading: 100%|███████████████████████████████████████████████████████████████| 6.96M/6.96M [00:00<00:00, 8.90MB/s]
```

hubscraper

hubscraper Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

hubscraper

hubscraper Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README