skydiscover

KO | EN

gitlite — search

#ai #typescript #llm #go #golang #android #rust #deep-learning #macos #nodejs #awesome #cli #vue #openai

(★ 603)

AI-Driven Scientific, Algorithmic, and Systems Discovery

skydiscover 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">
  <img src="assets/logo_vector.png" height="80" alt="SkyDiscover logo" style="vertical-align: middle;">&nbsp;

SkyDiscover
</h1>

A Framework for AI-Driven System Synthesis and Scientific Discovery
 
  <a href="https://skydiscover-ai.github.io/blog.html"><img src="https://img.shields.io/badge/blog-SkyDiscover-orange?style=flat-square" alt="Blog" /></a>
 <a href="https://arxiv.org/abs/2605.23109"><img src="https://img.shields.io/badge/paper-IDS-blueviolet?style=flat-square" alt="IDS Paper" /></a>
 <a href="https://arxiv.org/abs/2605.24096"><img src="https://img.shields.io/badge/paper-Just--in--Time%20Systems-yellow?style=flat-square" alt="Jitski Paper" /></a>
 <a href="https://arxiv.org/abs/2602.20133"><img src="https://img.shields.io/badge/paper-AdaEvolve-red?style=flat-square" alt="AdaEvolve Paper" /></a>
 <a href="https://arxiv.org/abs/2602.23413"><img src="https://img.shields.io/badge/paper-EvoX-lightblue?style=flat-square" alt="EvoX Paper" /></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-green?style=flat-square" /></a>

SkyDiscover is an open source project from UC Berkeley providing state-of-the-art tooling for scientific discovery and end-to-end system synthesis.

SkyDiscover provides tools for two broad classes of tasks:
 1. **Optimization**: given an evaluator and problem description, apply evolutionary techniques to incrementally discover better solutions.
 2. **Synthesis**: starting from a formal specification or text description of a system, full end-to-end synthesis of compliant and highly-performant systems.

See the getting started page for quickstart guides for each.

> 🚧 This project is under active development.

## SkyDiscover Optimize

**SkyDiscover Optimize** is a modular framework for AI-driven scientific and algorithmic discovery, providing a unified interface for implementing, running, and fairly comparing discovery algorithms across 200+ optimization tasks.

We ship with two SOTA adaptive optimization algorithms built by the SkyDiscover team, [AdaEvolve](https://arxiv.org/abs/2602.20133) and [EvoX](https://arxiv.org/abs/2602.23413). SkyDiscover Optimize also supports OpenEvolve, ShinkaEvolve and GEPA to quickly benchmark these algorithms using their own source code.

SkyDiscover Optimize natively supports [Harbor](https://harborframework.com/)-format benchmarks, so you can run external benchmark suites out of the box, including [AlgoTune](https://github.com/oripress/AlgoTune), [EvoEval](https://github.com/evo-eval/evoeval), [HumanEvalFix](https://github.com/bigcode-project/octopack), [BigCodeBench](https://github.com/bigcode-project/bigcodebench), [LiveCodeBench](https://livecodebench.github.io/), [USACO](https://usaco.org/), [CRUSTBench](https://github.com/AInfinity/CRUSTBench), and [CodePDE](https://github.com/).

Get started with our [quick start guide here](link-to-quickstart).

## SkyDiscover Synthesize

![Test-Driven Synthesis](https://raw.githubusercontent.com/skydiscover-ai/skydiscover/main/assets/test_driven_synthesis.png)

**SkyDiscover Synthesize** is a set of tooling that enables full, end-to-end synthesis of compliant and highly-performant systems, such as LLM inference systems, model routing, or key-value stores.

Our synthesis tooling is built on two published techniques introduced by the SkyDiscover team:
1. [Inductive-Deductive Synthesis](https://arxiv.org/abs/2605.23109): when a formal specification in Rocq for the target system is provided, Synthesize jointly generates implementation and corresponding proofs in stages, optimizing for performance in the process.
2. [Test-Driven Synthesis](https://arxiv.org/abs/2605.24096): when the system is described by natural language and tests, Synthesize iterates on implementations with a collection of planning, coding, critiquing, and auditing agents.

SkyDiscover Synthesize is provided as a Claude Code and Codex-compatible skill, and has been used to produce a wide range of systems.

See the [Quick Start guide here](link-to-quick-s).

## Optimization Results

SkyDiscover Optimize has been used across industry including Google, Uber, and more; the algorithms released by the SkyDiscover team, AdaEvolve and EvoX, achieve the strongest open-source results across ~200 optimization benchmarks: matching or exceeding AlphaEvolve and human SOTA, and outperforming OpenEvolve, GEPA, and ShinkaEvolve under identical generation budgets.

- **Frontier-CS (172 problems)**: ~34% median score improvement over OpenEvolve, GEPA, and ShinkaEvolve  
- **Math + Systems Optimization (14 tasks evaluated)**: Matches or exceeds AlphaEvolve and human-designed SOTA on 6/6 systems and 6/8 math tasks
- **Real-world systems impact**: 41% lower cross-cloud transfer cost, 14% better GPU load balance for MoE serving, and 29% lower KV-cache pressure via GPU model placement

<details>
<summary>📊 Complete results of AdaEvolve and EvoX (100 iterations)</summary>

> AdaEvolve and EvoX are **complementary**: AdaEvolve adapts search *parameters* for fast early gains; EvoX evolves the search *strategy itself* for stronger long-horizon gains. Both are built on SkyDiscover.

</details>

<details>
<summary>📈 Scaling behavior of AdaEvolve and EvoX</summary>

The scaling behavior of AdaEvolve and EvoX shows a **complementary crossover**. AdaEvolve's per-iteration parameter adaptation yields fast early gains in low-budget runs (T≤50), while EvoX's demand-driven strategy evolution unlocks step-change improvements in longer runs (T≥50).

<img src="assets/scaling_comparison.png" width="900" alt="Scaling behavior of AdaEvolve vs EvoX across 500 iterations">
 Best-so-far score vs. iteration for Signal Processing, Heilbronn Convex, Prism, and Cloudcast (500 iterations, GPT-5).

</details>

<details>
<summary>🔗 Evolving AdaEvolve's policy with EvoX (coming soon)</summary>

The two methods are **composable**: EvoX can evolve using AdaEvolve as its starting strategy, achieving the best results on 3 out of 4 benchmarks (100 iterations, GPT-5). This combined mode will be available in SkyDiscover soon.

| Benchmark | AdaEvolve | EvoX (Random Init) | EvoX (AdaEvolve Init) |
|:--|--:|--:|--:|
| Signal Proc. (↑) | 0.718 | 0.721 | **0.760** |
| Heilbronn Cvx. (↑) | 0.0290 | 0.0270 | **0.0291** |
| Cloudcast (↓) | 640.5 | 637.1 | **623.4** |
| Prism (↑) | 26.37 | **30.52** | 26.27 |

</details>

<details>
<summary>Task breakdown across math, systems, and programming challenges</summary>

| | Benchmark | Domain | Tasks | Description |
|-|-----------|--------|------:|-------------|
| 🔢 | [math/](benchmarks/math/) | Math | 14 | Circle packing, Erdos problems, geometric optimization |
| 🖥️ | [ADRS/](benchmarks/ADRS/) | Systems | 5 | Cloud scheduling, load balancing, MoE expert placement |
| ⚡ | [gpu_mode/](benchmarks/gpu_mode/) | Systems | 4 | GPU kernel optimization |
| 🔧 | [kernelbench/](benchmarks/kernelbench/) | Systems | 250+ | [KernelBench](https://github.com/ScalingIntelligence/KernelBench) GPU kernel speedup optimization |
| 🧩 | [frontier-cs-eval/](benchmarks/frontier-cs-eval/) | Algorithms | 172 | [Frontier-CS](https://frontier-cs.org/) competitive programming |
| 🧠 | [arc_benchmark/](benchmarks/arc_benchmark/) | Reasoning | — | ARC-AGI visual reasoning |
| 💻 | [ale_bench/](benchmarks/ale_bench/) | Algorithms | 10 | Algorithmic programming contests |
| 🎨 | [image_gen/](benchmarks/image_gen/) | Creative | 1 | AI image generation evolution |
| 💬 | [prompt_optimization/](benchmarks/prompt_optimization/) | NLP | 1 | HotPotQA prompt evolution |

See [Dependency extras](#dependency-extras) for install commands per benchmark.

</details>

## 🚀 Optimization Quick Start

**Prerequisites:** Python >= 3.10, [uv](https://docs.astral.sh/uv/)

```bash
# Install
uv sync
export OPENAI_API_KEY="<your-key>"

# Try the circle packing benchmark
uv sync --extra math
uv run skydiscover-run benchmarks/math/circle_packing/initial_program.py \
  benchmarks/math/circle_packing/evaluator.py \
  --config benchmarks/math/circle_packing/config.yaml \
  --search evox \
  --iterations 100

uv run skydiscover-run benchmarks/math/circle_packing/initial_program.py \
  benchmarks/math/circle_packing/evaluator.py \
  --config benchmarks/math/circle_packing/config.yaml \
  --search adaevolve \
  --iterations 100

# Or run on your own problem
# algo can be "evox", "adaevolve", "openevolve", "gepa", "shinkaevolve"
uv run skydiscover-run initial_program.py evaluator.py \
 --search <algo> \
 --model gpt-5 \
 --iterations 100

# initial_program is optional — omit it to let the LLM start from scratch
uv run skydiscover-run evaluator.py \
 --search <algo> \
 --model gpt-5 \
 --iterations 100

# Run a Harbor benchmark (e.g. AlgoTune) — no seed program needed
pip install harbor
harbor datasets download algotune@1.0 -o /tmp/algotune
uv run skydiscover-run /tmp/algotune/<id>/algotune-set-cover \
 --model anthropic/claude-sonnet-4-6 \
 --search best_of_n -i 10
```

Or use the Python API:

```python
from skydiscover import run_discovery

result = run_discovery(
    initial_program="initial_program.py",
    evaluator="evaluator.py",
    search=[algo], # algo can be "adaevolve", "evox", "openevolve", "gepa", "shinkaevolve"
    model="gpt-5",
    iterations=100,
)

print(result.best_score, result.best_solution)
```

## ✏️ What You Write

### Scoring Function (required)

SkyDiscover supports three evaluator formats — pick whichever fits your use case:

| Format | When to use | What you point `evaluation_file` at |
|:---|:---|:---|
| **Python function** | Simple tasks, no system deps | `evaluator.py` |
| **Containerized** | Custom deps, data files, isolation | `evaluator/` directory (must contain `Dockerfile` + `evaluate.sh`) |
| **Harbor task** | External benchmark suites (AlgoTune, EvoEval, HumanEvalFix, BigCodeBench, LiveCodeBench, USACO, CRUSTBench, CodePDE, and more) | Task directory (must contain `instruction.md` + `tests/` + `environment/Dockerfile`) |

SkyDiscover auto-detects the format. See [`benchmarks/README.md`](benchmarks/README.md#adding-a-benchmark) for full setup instructions.

**Python evaluator** — a file with an `evaluate(program_path)` function:

```python
def evaluate(program_path):
    score = run_and_grade(program_path)
    return {
        "combined_score": score,       # primary optimization target (maximized)
        "artifacts": {                 # optional — stored with the solution for future context
            "feedback": "Off by one in the loop boundary",
        },
    }
```

**Containerized evaluator** — a directory with a `Dockerfile` and `evaluate.sh` that writes JSON to stdout. Runs in Docker, so it can have arbitrary dependencies.

**Harbor task** — a directory following the [Harbor](https://harborframework.com/) format (`instruction.md`, `environment/Dockerfile`, `tests/test.sh`). Works out of the box with 8+ tested benchmark suites (see [benchmarks/README.md](benchmarks/README.md#tested-harbor-datasets) for the full list).

- **combined_score** drives evolution. If omitted, SkyDiscover averages all numeric values in the dict.
- **artifacts** is optional — entries are injected into the next LLM prompt as context.

For `search.type: adaevolve`, you can also enable explicit Pareto optimization by configuring `search.database.pareto_objectives` and returning those objective metrics directly from the evaluator. In that mode, `combined_score` becomes optional and is only used as a scalar fallback/proxy when configured.

### Starting Solution (optional)

The initial program is **optional**. When omitted, the LLM generates a solution from scratch. If provided, it marks the region to mutate with EVOLVE-BLOCK markers. Everything outside is left untouched.

```python
# EVOLVE-BLOCK-START
def solve(input_data):
    return input_data  # baseline — SkyDiscover will improve this
# EVOLVE-BLOCK-END
```

If no markers are present, the entire file is treated as mutatable.

## 🧬 Pick an Algorithm

See [Benchmark Performance](#-benchmark-performance) for a detailed comparison of AdaEvolve and EvoX against other algorithms.

| Algorithm | Flag | Description |
|:---|:---|:---|
| ⭐&nbsp;**AdaEvolve** | `--search adaevolve` | Multi-island adaptive search with UCB, migration, and paradigm breakthroughs |
| 🧠&nbsp;**EvoX** | `--search evox` | Self-evolving paradigm that co-adapts solution generation and experience management |
| 📊&nbsp;**Top-K** | `--search topk` | Selects top-K solutions to refine |
| 🔍&nbsp;**Beam&nbsp;Search** | `--search beam_search` | Breadth-first expansion of a beam of top solutions |
| 🎲&nbsp;**Best-of-N** | `--search best_of_n` | Generates N variants per iteration, keeps the best |
| 🧪&nbsp;**GEPA&nbsp;Native** | `--search gepa_native` | Pareto-efficient search with reflective prompting and LLM-mediated merge |
| 🗺️&nbsp;**OpenEvolve&nbsp;Native** | `--search openevolve_native` | MAP-Elites + island-based evolutionary search |

### External backends

Install with `uv sync --extra external`, then use the corresponding flag:

| Backend | Flag | Source |
|:---|:---|:---|
| **OpenEvolve** | `--search openevolve` | [codelion/openevolve](https://github.com/codelion/openevolve) |
| **GEPA** | `--search gepa` | [gepa-ai/gepa](https://github.com/gepa-ai/gepa) |
| **ShinkaEvolve** | `--search shinkaevolve` | [SakanaAI/ShinkaEvolve](https://github.com/SakanaAI/ShinkaEvolve) (manual install) |

<details>
<summary>ShinkaEvolve manual install</summary>

```bash
git clone --depth 1 https://github.com/SakanaAI/ShinkaEvolve.git external_repos/ShinkaEvolve
uv pip install -e external_repos/ShinkaEvolve
```

</details>

## ⚙️ Configuration

Pass a YAML config with `-c`. See [configs/](configs/) for full annotated templates.

```yaml
max_iterations: 100
llm:
  models: [{ name: "gemini/gemini-3-pro-preview", weight: 1.0 }]
search:
  type: "adaevolve"                  # or "evox", "topk", "beam_search", "best_of_n"
prompt:
  system_message: |
    You are an expert at optimizing algorithms.
```

API keys (OPENAI_API_KEY, GEMINI_API_KEY, etc.) are resolved from environment variables automatically.

### 📊 Live Monitor & Human Feedback

Add `monitor: { enabled: true }` to your config. The dashboard URL prints at run start — scatter plot of all programs, code diffs, metrics, and AI summaries. A **Human Feedback** panel lets you steer evolution in real time.
Replay a completed run:

```bash
uv run skydiscover-viewer /path/to/checkpoints/checkpoint_100
```

## 📖 Reference

<details>
<summary>CLI flags</summary>

```
uv run skydiscover-run [INITIAL_PROGRAM] EVALUATOR [options]
```

| Flag | Description |
|:---|:---|
| `-c, --config FILE` | Config YAML |
| `-i, --iterations N` | Number of iterations |
| `-m, --model MODEL` | LLM model (overrides config) |
| `-s, --search TYPE` | Search algorithm |
| `-o, --output DIR` | Output directory |
| `--api-base URL` | Override LLM API endpoint |
| `--checkpoint DIR` | Resume from checkpoint |
| `--agentic` | Enable agentic mode (LLM can read your files) |
| `-l, --log-level LEVEL` | DEBUG, INFO, WARNING, or ERROR |

</details>

<details>
<summary>Python API — discover_solution() (convenience wrapper)</summary>

`discover_solution()` is a convenience wrapper around `run_discovery()` (shown in [Quick Start](#-quick-start)) for inline string solutions and callable evaluators:

```python
from skydiscover import discover_solution

result = discover_solution(
    initial_solution="def solve(x): return x",  # optional — omit to start from scratch
    evaluator=lambda path: {"combined_score": run_tests(path)},
    iterations=50,
    search="evox",
)
```

</details>

<details>
<summary>Model providers</summary>

Any [LiteLLM](https://docs.litellm.ai/)-compatible model works using `provider/model` format:

```bash
--model gpt-5                                               # OpenAI (default)
--model gemini/gemini-3-pro-preview                          # Gemini
--model anthropic/claude-sonnet-4-20250514                   # Anthropic
--model ollama/llama3 --api-base http://localhost:11434/v1   # Local (Ollama, vLLM, etc.)
```

Multi-model pools with weighted sampling are supported in config:

```yaml
llm:
  models:
    - name: "gpt-5-mini"
      weight: 0.7
    - name: "gemini/gemini-2.0-flash"
      weight: 0.3
```

</details>

<details id="dependency-extras">
<summary>Benchmark dependency extras</summary>

```bash
uv sync                              # Base install
uv sync --extra math                 # Math benchmarks (SciPy, JAX, PyWavelets, …)
uv sync --extra adrs                 # ADRS systems benchmarks
uv sync --extra frontier-cs          # Frontier-CS benchmark tooling
uv sync --extra external             # OpenEvolve / GEPA / ShinkaEvolve backends
uv sync --extra prompt-optimization  # HotPotQA prompt optimization
```

Combine extras as needed: `uv sync --extra external --extra math`

If a benchmark ships its own `requirements.txt`, also run: `uv pip install -r path/to/requirements.txt`

</details>

---

## 🛠️ Extending SkyDiscover

- **New benchmark** → [`benchmarks/README.md`](benchmarks/README.md#adding-a-benchmark)
- **New search algorithm** → [`skydiscover/search/README.md`](skydiscover/search/README.md)
- **New context builder** → [`skydiscover/context_builder/README.md`](skydiscover/context_builder/README.md)

---

## 🔗 Related Work
SkyDiscover is inspired by [AlphaEvolve](https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/) and incorporates useful code components from open-source efforts such as [OpenEvolve](https://github.com/codelion/openevolve). Its interface is compatible with the [optimize_anything](https://gepa-ai.github.io/gepa/blog/2026/02/18/introducing-optimize-anything/) API.

## ✍️ Citation

```bibtex
@inproceedings{liu2026skydiscover,
  author    = {Liu, Shu and Cemri, Mert and Agarwal, Shubham and Krentsel, Alexander and Naren, Ashwin and Mang, Qiuyang and Li, Zhifei and Gupta, Akshat and Maheswaran, Monishwaran and Cheng, Audrey and Pan, Melissa and Boneh, Ethan and Ramchandran, Kannan and Sen, Koushik and Zaharia, Matei and Dimakis, Alexandros G. and Stoica, Ion},
  title     = {SkyDiscover: A Flexible, Adaptive Framework for AI-Driven Scientific and Algorithmic Discovery},
  booktitle = {Proceedings of the ACM Conference on AI and Agentic Systems},
  series    = {CAIS '26},
  year      = {2026},
  pages     = {1223--1227},
  publisher = {Association for Computing Machinery},
  doi       = {10.1145/3786335.3813221},
  url       = {https://doi.org/10.1145/3786335.3813221}
}
```

## 📬 Contact Us
For questions or feedback, reach out to us:
[lshu@berkeley.edu](mailto:lshu@berkeley.edu) · [akrentsel@berkeley.edu](mailto:akrentsel@berkeley.edu) · [mert_cemri@berkeley.edu](mailto:mert_cemri@berkeley.edu) · [shubham3@berkeley.edu](mailto:shubham3@berkeley.edu)

skydiscover

skydiscover 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

skydiscover

skydiscover 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README