DeepScholar

KO | EN

gitlite — search

#javascript #react #typescript #go #android #rust #deep-learning #java #windows #kubernetes #vue #awesome-list #ios #open-source

(★ 10)

Automated agentic research loop for vision-language-action (VLA) robot policy self-improvement

DeepScholar 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">DeepScholar</h1>

An agentic research loop for improving vision-language-action and robot policy models.

<a href="README.md">English</a> &nbsp;·&nbsp;
 <a href="README.zh.md">简体中文</a> &nbsp;·&nbsp;
 <a href="#start-from-a-baseline-policy">Start From A Baseline</a> &nbsp;·&nbsp;
 <a href="#research-loop">Research Loop</a> &nbsp;·&nbsp;
 <a href="#quick-start">Quick Start</a> &nbsp;·&nbsp;
 <a href="#configuration-guide">Configuration</a> &nbsp;·&nbsp;
 <a href="#how-results-are-organized">Artifacts</a>

---

DeepScholar turns model-improvement work into a reproducible research loop. Give it a baseline policy, an evaluation protocol, and an execution budget. It proposes candidate ideas, debates them, generates isolated configs, runs proxy and final evaluations, records failures, and writes a technical report for the next round.

The goal is not to run one more experiment. The goal is to make the improvement process itself repeatable: every hypothesis, patch, log, checkpoint, metric, and lesson is preserved.

<div align="center">
 <table>
 <tr>
 <td align="center" bgcolor="#eff6ff">
 <h1>0</h1>
 Manual intervention during the loop
 </td>
 <td align="center" bgcolor="#fef2f2">
 <h1>&gt;24h</h1>
 Long-running autonomous studies
 </td>
 <td align="center" bgcolor="#f0fdf4">
 <h1>+2.0 pp</h1>
 Observed gains on selected LIBERO runs
 </td>
 </tr>
 </table>
</div>

## Start From A Baseline Policy

Starting from a known checkpoint is often faster than manually designing the next experiment.

DeepScholar can take a baseline VLA or robot policy study and turn it into a structured self-improvement run:

1. **Define the research target**: model family, dataset, task suite, metric, and eval protocol.
2. **Let the loop generate candidates**: conservative hyperparameter changes plus more experimental module ideas.
3. **Run proxy training and evaluation**: cheap gates reject weak candidates before spending full budget.
4. **Promote promising patches**: final training and evaluation confirm whether a change beats the calibrated baseline.
5. **Write the memory**: successful recipes and failure cards seed the next iteration.

```text
"Improve this SmolVLA LIBERO baseline. Keep the task suite, metric, horizon, and camera protocol fixed unless a candidate explicitly targets evaluation."
```

What you get back is not an untracked pile of shell logs. You get:

- **Candidate configs** with isolated YAML patches.
- **Aligned evaluations** that keep task, metric, horizon, cameras, and protocol fields consistent.
- **Experiment records** linking patches to logs, checkpoints, eval videos, metrics, and reports.
- **Long-term memory** so later rounds reuse what worked and avoid repeated failures.

## What DeepScholar Handles

- **End-to-end research loop**: idea generation, debate, planning, config generation, execution, evaluation, memory, and reporting.
- **Agent and non-agent modes**: use real Cursor SDK agents when available, or fall back to deterministic local heuristics.
- **Safe candidate generation**: supports conservative tuning and frontier module or architecture experiments.
- **GPU-aware execution**: assigns candidate experiments to idle GPUs and queues the rest automatically.
- **Reproducible artifacts**: stores agent traces, train/eval logs, eval videos, checkpoints, registries, manifests, and reports.

## Current Results

```mermaid
---
config:
  themeVariables:
    xyChart:
      plotColorPalette: "#2563eb, #dc2626"
---
xychart-beta
  title "DeepScholar - Base Model vs Self-Improve"
  x-axis ["SmolVLA Base", "SmolVLA Improve", "Pi0.5 Base", "Pi0.5 Improve"]
  y-axis "Success Rate" 0.50 --> 1.00
  bar "Base model" [0.64, 0, 0.92, 0]
  bar "Self-improve" [0, 0.66, 0, 0.94]
```

- **SmolVLA**: starting from the `lerobot/smolvla_libero` base model, improves the LIBERO-10 subset success rate from 64.0% to 66.0% (+2.0 pp).
- **Pi0.5**: starting from the `lerobot/pi05_libero_finetuned` base model, improves the LIBERO-10 subset success rate from 92.0% to 94.0% (+2.0 pp).

Legend: **blue** = calibrated baseline; **red** = DeepScholar best result.

## Research Loop

DeepScholar separates scientific reasoning, candidate generation, execution, evaluation, and long-term memory so each experiment remains reproducible.

```mermaid
%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, sans-serif", "primaryColor": "#f8fafc", "primaryTextColor": "#0f172a", "primaryBorderColor": "#cbd5e1", "lineColor": "#64748b", "tertiaryColor": "#f1f5f9"}}}%%
flowchart TB
 Goal["Research Goal baseline model / dataset / eval protocol / target metric"]:::hero

subgraph Loop["DeepScholar Self-Improvement Loop"]
    direction LR

subgraph Think["1. Reason"]
 direction TB
 R["Researcher papers, logs, prior failures"]:::agent
 I["Idea Generator safe hparams + frontier modules"]:::agent
 D["Debate Panel reviewer, mentor, engineer, skeptic"]:::agent
 P["Execution Planner budget, gates, rollback"]:::agent
 end

subgraph Build["2. Build"]
 direction TB
 X["Candidate Builder isolated YAML patches"]:::build
 Y["Candidate Configs workspace/configs/*.yaml"]:::build
 S["Sandbox Apply whitelist + rollback boundary"]:::build
 end

subgraph Run["3. Execute"]
 direction TB
 T["Proxy Train LeRobot / custom command"]:::run
 V["Proxy Eval aligned metric parser"]:::run
 F["Final Eval more episodes / seeds"]:::run
 end

subgraph Decide["4. Decide"]
 direction TB
 B0["Baseline Eval calibrated success rate"]:::eval
 G1{"Stage-A Proxy Gate"}:::gate
 G2{"Stage-B Final Gate"}:::gate
 end
 end

subgraph Memory["Memory and Reporting"]
 direction LR
 Mem["Long-Term Memory recipes + failure cards"]:::memory
 Reg["Experiment Registry experiments.jsonl"]:::memory
 Man["Session Manifest logs, evals, checkpoints"]:::memory
 Rep["Technical Report round summary + best result"]:::report
 end

Goal --> R
  Goal --> B0
  R --> I --> D --> P --> X --> Y --> S --> T --> V --> G1
  B0 --> G1
  G1 -- pass --> F --> G2
  G1 -- fail --> Mem
  G2 -- accept --> Reg --> Rep
  G2 -- reject --> Mem
  T -. trace .-> Man
  V -. trace .-> Man
  F -. trace .-> Man
  Reg -. index .-> Man
  Mem -. reuse lessons .-> R
  Rep -. next prompt .-> R

classDef hero fill:#020617,stroke:#020617,color:#ffffff,stroke-width:2px;
  classDef agent fill:#eef2ff,stroke:#6366f1,color:#312e81,stroke-width:1.5px;
  classDef build fill:#faf5ff,stroke:#a855f7,color:#581c87,stroke-width:1.5px;
  classDef run fill:#ecfeff,stroke:#06b6d4,color:#164e63,stroke-width:1.5px;
  classDef eval fill:#f0fdf4,stroke:#22c55e,color:#14532d,stroke-width:1.5px;
  classDef gate fill:#fff7ed,stroke:#f97316,color:#7c2d12,stroke-width:2px;
  classDef memory fill:#fdf2f8,stroke:#ec4899,color:#831843,stroke-width:1.5px;
  classDef report fill:#fffbeb,stroke:#f59e0b,color:#78350f,stroke-width:1.5px;
```

## Repository Layout

```text
src/research_loop/
  agent_mode/                 # Cursor SDK agent mode
    prompts/*.md              # Role definitions: researcher, idea, debate, planner, executor, reflector
    roles.py                  # Load Markdown prompts and call agents
  non_agent_mode/             # Local fallback mode
    roles.py                  # Deterministic heuristic roles
  loop.py                     # Main research loop
  runner.py                   # LeRobot train/eval command runner
  evaluation.py               # Stage-A / Stage-B gates
  registry.py                 # Experiment registry and memory
  report.py                   # Final report writer
src/sitecustomize.py           # Project-local LeRobot compatibility patches loaded by runner subprocesses

workspace/
 configs/ # Generated candidate patches
 artifacts/
 <session_id>/ # Same name as artifacts/agent_runs/<session_id>
 runs/ # train/eval logs
 eval/ # eval_info + videos
 checkpoints/ # trained checkpoints

artifacts/
  agent_runs/                 # Agent call records grouped by session
  reports/                    # Markdown reports
  registry/                   # experiments.jsonl
  memory/                     # failure and recipe memory
```

## Quick Start

### 1. Install

Create a Python environment and install DeepScholar in editable mode:

```bash
cd /path/to/DeepScholar
python -m venv .venv
source .venv/bin/activate
pip install -e .
```

For the SmolVLA/LIBERO example, install LeRobot and verify the commands are available:

```bash
lerobot-train --help
lerobot-eval --help
```

### 2. Configure

Use one of the example configs:

```text
configs/smolvla-self-improve.json
configs/pi0fast-self-improve.json
configs/pi05-self-improve.json
```

You can extend DeepScholar to other model families by copying one of these configs and updating the baseline model, dataset, train/eval commands, and eval protocol fields.

### 3. Run

Start the full loop without agents:

```bash
source .venv/bin/activate
vla-loop --config configs/smolvla-self-improve.json
```

To run with Cursor SDK agents, provide an API key:

```bash
export CURSOR_API_KEY="your_cursor_api_key"
vla-loop --config configs/smolvla-self-improve.json
```

If `CURSOR_API_KEY` is missing, DeepScholar still runs in non-agent mode.

## Configuration Guide

The config file controls the research target, baseline, execution budget, agents, and GPU scheduling. In most runs, these are the fields you will edit:

### Key Parameters

- `goal.description`: research objective shown to agents and reports.
- `goal.baseline_model.framework`: execution backend, usually `lerobot` or `command`.
- `goal.baseline_model.checkpoint` / `model_id`: baseline checkpoint used as the starting point.
- `goal.baseline_model.eval_protocol`: task, metric, horizon, cameras, and other eval settings that must stay aligned across candidates.
- `goal.baseline_success_rate`: calibrated baseline used by improvement gates.
- `goal.baseline_model.command_templates.train`: train command template.
- `goal.baseline_model.command_templates.eval`: eval command template that emits parseable metrics.
- `runner.proxy_train_steps`: cheap Stage-A training budget for filtering candidates.
- `runner.final_train_steps`: larger Stage-B training budget for promising candidates.
- `agent.enabled`: whether to use Cursor SDK agents instead of local heuristic roles.
- `agent.model`: Cursor SDK model id passed to `Agent.prompt()`.
- `agent.enable_code_agent`: whether agents may edit code in addition to generated YAML configs.
- `gpu.ids`: local GPU allowlist; `[]` means all GPUs visible to `nvidia-smi`.
- `gpu.max_parallel_experiments`: max concurrent candidate runs; `0` or `"auto"` uses idle GPUs automatically.

For non-LeRobot models, set `goal.baseline_model.framework` to `command` and provide train/eval shell commands that produce parseable metrics.

Set a readable session tag when comparing multiple runs:

```bash
export CURSOR_API_KEY="your_cursor_api_key"
export DEEPSCHOLAR_AGENT_RUN_TAG="smolvla_libero_trial_001"
vla-loop --config configs/smolvla-self-improve.json
```

The agent model is configured in JSON via `agent.model`; there is no CLI flag. Session metadata is recorded in:

```text
artifacts/agent_runs/<session_id>/_session_meta.json
```

## How Results Are Organized

Each run writes a manifest and a set of linked artifacts. Start with:

```text
artifacts/agent_runs/<session_id>/manifest.json
```

`manifest.json` is the best entry point for reproducing a run. It links the agent session, baseline record, candidate patch IDs, train/eval logs, eval videos, checkpoints, registry rows, and final report.

Then inspect the supporting files:

```text
artifacts/agent_runs/<session_id>/
artifacts/reports/*.md
artifacts/registry/experiments.jsonl
workspace/artifacts/<session_id>/runs/<patch_id>/proxy_train.log
workspace/artifacts/<session_id>/runs/<patch_id>/proxy_eval.log
workspace/artifacts/<session_id>/eval/<patch_id>_proxy/
workspace/artifacts/<session_id>/checkpoints/<patch_id>/
```

For LeRobot LIBERO logs, use aggregated metrics as the source of truth:

```text
Overall Aggregated Metrics:
{'pc_success': 90.0, 'n_episodes': 30, ...}
```

Do not confuse `running_success_rate` in a progress bar with final experiment success.

DeepScholar

DeepScholar 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

DeepScholar

DeepScholar 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README