GhostHunter

KO | EN

gitlite — search

#python #llm #golang #android #deep-learning #linux #docker #nodejs #awesome #kubernetes #vue #pytorch #mcp #ios

(★ 22)

GhostHunter is a powerful and user-friendly tool designed to uncover hidden treasures from the Wayback Machine. It allows you to search for archived URLs (snapshots) of a specific domain, filter them by file extensions, and save the results in an organized manner.

GhostHunter Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# GhostHunter

GhostHunter is a program that finds and collects a list of web pages (URLs) from a website that were previously saved by the **Wayback Machine** — an internet archive service that keeps copies of web pages from the past, even ones that are no longer accessible today.

This program is useful for:
- Researchers who want to see what a website looked like in the past.
- Developers who want to recover files or pages that are no longer available.
- Anyone who wants to know which pages ever existed on a domain (website name).

Everything runs through commands in a terminal (Command Prompt or PowerShell), and the results are saved as neat text files.

---

## Terms You Should Know

To make things easier, here are a few simple terms:

| Term | Meaning |
|------|---------|
| Domain | A website name, for example: `example.com` |
| URL | The full address of a page or file on the internet, for example: `https://example.com/file.pdf` |
| File extension | The ending of a file name that shows its type, for example: `.pdf`, `.sql`, `.env`, `.json` |
| Snapshot | A copy of a web page that the Wayback Machine saved on a certain date |
| Wayback Machine | A digital library that stores copies of web pages from previous years |

---

## Main Features

1. **Find all URLs of a website** — gets the list of archived URLs from the Wayback Machine.
2. **Filter by file type** — keeps only URLs with certain extensions (for example `.pdf`, `.sql`, `.env`). The list of extensions can be changed in the `config.json` file.
3. **Save results neatly** — each file type is saved into its own file inside the `results/` folder.
4. **Find snapshots (optional)** — see older versions of each URL with their dates, and save them into one file.
5. **Automatic checks** — before working, the program makes sure the internet connection is active, the Wayback Machine is reachable, and the target site still exists.
6. **Friendly interface** — colors, a loading animation, and summary tables make the output easy to read.

---

## Screenshots

Here is what the program looks like when it runs:

---

## Requirements

- **Go** installed on your computer (version 1.25 or newer). If you don't have it, download it from https://go.dev/dl/
- An active internet connection.

---

## Installation

1. Download this code to your computer:

```bash
   git clone https://github.com/mysteriza/GhostHunter.git
   cd GhostHunter
   ```

2. Build the program into an executable file:

```bash
   go mod download
   go build -o GhostHunter
   ```

On Windows, use this command instead:

```powershell
   go build -o GhostHunter.exe
   ```

> Important: the `tablewriter` library version must stay at `v0.0.5`. Do not run `go get github.com/olekukonko/tablewriter@latest`, because the newest version is not compatible with this program.

---

## Usage

1. Run the program:

```bash
   ./GhostHunter
   ```

On Windows:

```powershell
   .\GhostHunter.exe
   ```

2. Type the domain you want to check, then press Enter. For example: `example.com`.

3. The program checks the internet connection, the Wayback Machine, and whether the site is available.

4. The program gets the list of URLs, filters them by file extension, and saves the results into the `results/example.com/` folder.

5. The program asks whether you want to find snapshots (old versions) of the discovered URLs.
   - Type `y` if you do, then choose the site and file types from the list shown.
   - Type `n` or just press Enter if you don't.

Example output:

```
Enter the domain to search (e.g., target.com): example.com
Connected to the Internet!
Wayback Machine is UP and running.
Domain is active!

Directory 'results\example.com' created successfully.
Fetching data from Wayback Machine ...

Total URLs found (filtered): 12

---

## Where Results Are Saved

All results are saved inside the `results/` folder:

```
results/
└── example.com/
    ├── example.com.pdf.txt      # list of URLs ending in .pdf
    ├── example.com.sql.txt      # list of URLs ending in .sql
    ├── example.com.env.txt      # list of URLs ending in .env
    └── example.com.snapshots.txt  # list of snapshots (if searched)
```

---

## Configuration (`config.json`)

The `config.json` file holds settings you can change:

| Setting | Purpose |
|---------|---------|
| `extensions` | The list of file extensions to keep. Only URLs ending with one of these extensions are collected. |
| `numWorkers` | The number of parallel processes used while searching for snapshots. The default value is `5`. |

Example file contents:

```json
{
  "extensions": ["sql", "env", "json", "pdf", "php", "js", "txt"],
  "numWorkers": 5
}
```

---

## Troubleshooting

| Problem | Cause / Solution |
|---------|------------------|
| `context deadline exceeded` or the process seems stuck | The Wayback Machine sometimes responds slowly. Just run the program again. |
| "0 URLs" result | The site may have no archived files with the extensions configured in `config.json`. |
| `tablewriter` build error | Make sure `go.mod` uses `tablewriter v0.0.5`. |
| The window closes immediately | Run the program from a terminal. On error, the program now waits for you to press Enter so the message can be read. |

---

## For Developers

Project file structure:

```
.
├── main.go        # Program entry point
├── config.go      # Reads and validates settings
├── network.go     # Internet connection and Wayback Machine data fetching
├── processor.go   # URL filtering and result saving
├── snapshots.go   # Snapshot discovery and saving
├── ui.go          # Welcome banner
├── config.json    # Extension and worker settings
└── results/       # Output folder (gitignored)
```

---

## Contributing

If you want to request a feature or fix the program, please leave a message or open a pull request. All input is welcome and will be considered.

---

## Acknowledgments

Thanks to the [Wayback Machine](https://web.archive.org/) for providing the API used to fetch data.

Special thanks to the creators of the Go libraries used in this project:
- [briandowns/spinner](https://github.com/briandowns/spinner)
- [fatih/color](https://github.com/fatih/color)
- [olekukonko/tablewriter](https://github.com/olekukonko/tablewriter)

---

## Responsible Use

GhostHunter is built for research and educational purposes. Use it only on sites you own or have explicit permission to check, and respect the Wayback Machine's usage limits.

GhostHunter

GhostHunter Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

GhostHunter

GhostHunter Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README