KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #claude-code #codex #typescript #ai-agents #dsh #dsh-plugin #deepseek-harness #ai-agent #mcp #agent-skills #macos

reverse-api-engineer

★ 987 Open GitHub ↗

The agent that turns websites into APIs!

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![Greptile: The War on Bugs](https://www.greptile.com/badge.svg)](https://www.greptile.com/?utm_source=oss_badge&utm_medium=readme&utm_campaign=greptile_for_open_source)

</div>

# Reverse API Engineer

> **Turn websites into APIs.** Browse (or let an agent browse), and get a clean, typed client for the endpoints the site actually uses.

<img src="https://raw.githubusercontent.com/kalil0321/reverse-api-engineer/main/assets/rae-autoscout.gif" alt="Agent Mode Demo">
 
 Agent mode

<img src="https://raw.githubusercontent.com/kalil0321/reverse-api-engineer/main/assets/reverse-api-engineer.gif" alt="Manual Mode Demo">
 
 Manual mode

---

## How it works

1. You give it a website and a goal ("fetch all Apple jobs").
2. A browser visits the site, either driven by you or by an AI agent.
3. Network traffic is captured to a HAR file.
4. Your configured model reads the traffic and writes a working API client in Python, JavaScript, TypeScript, Go, Java, C#, PHP, Ruby, or C.

No more manually opening DevTools, copying cURL commands, and gluing together a client.

## Install

```bash
uv tool install reverse-api-engineer   # or: pip install reverse-api-engineer
```

Agent mode (the default) captures through `npx`-launched browser tooling —
browser MCP servers (Playwright or Chrome DevTools) or the Vercel `agent-browser`
CLI — and needs no extra Python packages. **Manual mode** drives a local
Playwright browser, which ships in the optional `[manual]` extra:

```bash
uv tool install "reverse-api-engineer[manual]"   # or: pip install "reverse-api-engineer[manual]"
playwright install chromium
```

## Quick start

```bash
reverse-api-engineer
> fetch all apple jobs from their careers page

# Browser opens. Navigate, interact, close when done.
# → ./scripts/apple_jobs_api/  (api_client.py, README.md, example_usage.py)
```

Cycle modes with **Shift+Tab**:

| Mode | What it does |
|------|--------------|
| `manual` | You drive the browser; AI generates the client from captured traffic. |
| `agent` | An AI agent drives capture autonomously (Playwright or Chrome MCP, or Vercel agent-browser CLI). |
| `engineer` | Re-run generation on a previous capture (`engineer <run_id>`). |
| `collector` | Agent collects structured data (JSON/CSV) using web search + fetch. |

Agent mode providers:
- **auto** (default): Playwright MCP, single workflow for browsing + reverse engineering.
- **chrome-mcp**: drives your real Chrome so you keep existing sessions/cookies. Requires Chrome 146+ and Node.js 20.19+.
- **agent-browser**: [Vercel agent-browser](https://github.com/vercel-labs/agent-browser) **CLI** (not a Reverse API Engineer browser MCP server). At session start RAE uses whatever `agent-browser` is already on `PATH`, otherwise runs **`npm install -g <pin>`** (same pin as config / `RAE_AGENT_BROWSER_PACKAGE`), prints a yellow notice, validates with **`--help`**, and only then falls back to **`npx -y <pin>`** if npm cannot install. Prompts embed the resolved shell prefix alongside **`skills get core --full`**, **`skills list`**, HAR phases, cloud notes from `agent_browser_notes`. Tune with `agent_browser_npx_package` (optional), env `RAE_AGENT_BROWSER_*`. First Chromium fetch: **`agent-browser install`** (add `--with-deps` on trimmed Linux).

Optional sanity checks:

```bash
agent-browser doctor --offline --quick || true
agent-browser skills list >/dev/null
```

## Configuration

Settings live in `~/.reverse-api/config.json` and can be edited via `/settings` in the CLI:

```json
{
  "agent_provider": "auto",
  "agent_browser_npx_package": "agent-browser@0",
  "agent_browser_notes": "",
  "claude_code_model": "claude-sonnet-4-6",
  "collector_model": "claude-sonnet-4-6",
  "ollama_auto_start": true,
  "ollama_base_url": "http://127.0.0.1:11434",
  "opencode_auto_start": true,
  "opencode_base_url": "http://127.0.0.1:4096",
  "opencode_model": "big-pickle",
  "opencode_npx_package": "opencode-ai@latest",
  "opencode_provider": "opencode",
  "copilot_model": "gpt-5",
  "cursor_model": "composer-2.5",
  "output_dir": null,
  "output_language": "python",
  "real_time_sync": true,
  "sdk": "claude"
}
```

- **Models**: Sonnet 4.6 (default), Opus 4.6 (most capable), Haiku 4.5 (fastest). For OpenCode see [models.dev](https://models.dev).
- **SDK**: `claude` (default), `opencode`, `cursor`, or `copilot` (GitHub Copilot).
- **OpenCode setup**: with `sdk: "opencode"`, RAE reuses an existing server or downloads/starts `opencode-ai@latest` through `npx`; a global OpenCode installation is not required. Fresh configurations default to the free `opencode/big-pickle` model. `/settings` shows a loading spinner, then offers a searchable **OpenCode Provider / Model** picker populated from the server's connected, tool-capable catalog and marks free OpenCode options. Before creating a session, RAE validates the saved pair again and suggests currently available free models when configuration is invalid. Compatible older servers are reused with a version warning. Node.js 20+ is required for automatic startup. Password-protected servers use `OPENCODE_SERVER_PASSWORD` and optional `OPENCODE_SERVER_USERNAME`. Override startup with `OPENCODE_BASE_URL`, `RAE_OPENCODE_PACKAGE`, or `RAE_OPENCODE_AUTO_START=0`.
- **Ollama through OpenCode**: choose provider `ollama` in `/settings`; RAE starts an installed Ollama daemon if needed, lists only installed models with tool calling and 64k+ context, and supplies OpenCode's provider config inline. Models are never downloaded silently. Override with `RAE_OLLAMA_BASE_URL` or `RAE_OLLAMA_AUTO_START=0`.
- **Output language**: `python`, `javascript`, `typescript`, `go`, `java`, `csharp`, `php`, `ruby`, or `c`. C needs a POSIX toolchain (`cc`, libcurl headers) — macOS/Linux, or WSL/MSYS2 on Windows.

## CLI

Slash commands inside the CLI:
- `/settings`: configure model, SDK, agent provider, and sync settings.
- `/history`: list past runs with timestamps, costs, and status.
- `/messages <run_id>`: view detailed message logs for a run.
- `/help` (alias: `/commands`): show the command list.
- `/exit` (alias: `/quit`): leave the CLI.

Scriptable subcommands (pipe to `jq`):

```bash
reverse-api-engineer agent --prompt "capture the public jobs api" \
  --url https://example.com/jobs --json | jq

reverse-api-engineer list --json
reverse-api-engineer show <run_id> --json
reverse-api-engineer run <run_id> --file api_client.py \
 --no-interactive --auto-install -- --org acme
```

Pass `--no-interactive` (and/or `--json`) to skip prompts. With `--json`, stdout is one JSON document and logs go to stderr.

### `agent --json` schema

| Field            | Type                | Notes                                                                  |
|------------------|---------------------|------------------------------------------------------------------------|
| `schema_version` | `int`               | Currently `1`.                                                         |
| `status`         | `"ok"` \| `"error"` | Top-level result.                                                      |
| `run_id`         | `string` \| `null`  | Use with `show` / `engineer` / `run`.                                  |
| `prompt`         | `string`            |                                                                        |
| `url`            | `string` \| `null`  |                                                                        |
| `mode`           | `string` \| `null`  | `"auto"`, `"chrome-mcp"`, or `"agent-browser"`.                                        |
| `har_path`       | `string` \| `null`  | Captured HAR.                                                          |
| `script_path`    | `string` \| `null`  | Generated client.                                                      |
| `usage`          | `object`            | `{input_tokens, output_tokens, total_cost}`.                           |
| `error`          | `string` \| `null`  | When `status == "error"`.                                              |

### Exit codes

| Code | Meaning |
|------|---------|
| `0` | Success. |
| `1` | Runtime error. |
| `2` | Missing required arg under `--no-interactive` / `--json`. |

For `run`, the exit code is the underlying script's return code on success, `1` if no script was found, or non-zero if `--no-interactive` would have had to prompt.

## Output locations

- `~/.reverse-api/runs/scripts/{run_id}/`: permanent storage
- `./scripts/{descriptive_name}/`: local copy with a readable name
- Collector: `./collected/{folder_name}/` (`items.json`, `items.csv`, `README.md`)

## Caveats

- Generated code runs locally via Claude Code, so review before executing.
- Sites with aggressive bot detection may block capture or require manual interaction.

## Development

```bash
git clone https://github.com/kalil0321/reverse-api-engineer.git
cd reverse-api-engineer
uv sync
uv run reverse-api-engineer
```

Build: `./scripts/clean_build.sh`. Requires Python 3.11+ and an API key for agent mode. Manual mode additionally needs the `[manual]` extra plus `playwright install chromium`.

## License

MIT. See [LICENSE](LICENSE).

reverse-api-engineer

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README