physical-ai-studio

KO | EN

gitlite — search

#llm #typescript #deep-learning #golang #react #claude #machine-learning #javascript #cli #python #claude-code #java

physical-ai-studio

★ 71 Open GitHub ↗

Physical AI Studio is an end-to-end framework for training robots to perform tasks through imitation learning from human demonstrations.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**Train and deploy Vision-Language-Action (VLA) models for robotic imitation learning**

[Key Features](#key-features) •
[Quick Start](#quick-start) •
[Documentation](#documentation) •
[Contributing](#contributing)

</div>

---

## What is Physical AI Studio?

Physical AI Studio is an end-to-end framework for teaching robots to perform tasks through imitation learning from human demonstrations.

## Key Features

- **End-to-End Pipeline** - From demonstration recording to robot deployment
- **State-of-the-Art Policies** - Native policy implementations such as [ACT](https://arxiv.org/abs/2304.13705), [Pi0](https://www.physicalintelligence.company/download/pi0.pdf), [SmolVLA](https://huggingface.co/lerobot/smolvla_base), [GR00T](https://arxiv.org/abs/2503.14734) and [Pi0.5](https://arxiv.org/pdf/2504.16054), plus full [LeRobot](https://github.com/huggingface/lerobot) policy zoo
- **Flexible Interface** - Use Python API, CLI, or GUI
- **Production Export** - Deploy to [OpenVINO](https://docs.openvino.ai/), [ONNX](https://onnx.ai/), or [Torch](https://docs.pytorch.org/executorch/stable/index.html) for any hardware
- **Standardized Benchmarks** - Evaluate on benchmarks such as [LIBERO](https://libero-project.github.io/) and [PushT](https://diffusion-policy.cs.columbia.edu/)
- **Built on Lightning** - [PyTorch Lightning](https://lightning.ai/docs/pytorch/stable/) for distributed training, mixed precision, and more

## Quick Start

### Application (GUI)

For users who prefer a visual interface for end-to-end workflow:

<p align="center">
  <a href="docs/assets/physical_ai_studio_full_overview.mp4">
    <img src="docs/assets/physical_ai_studio_overview.gif" alt="Application demo" width="100%">
  </a>
</p>

[Download the full demo video →](docs/assets/physical_ai_studio_full_overview.mp4)

[Application Documentation →](./application/README.md)

#### Docker

Run the full application (backend + UI) in a single container (using [Docker](https://docs.docker.com/engine/install/ubuntu/)):

```bash
# Clone the repository
git clone https://github.com/open-edge-platform/physical-ai-studio.git
cd physical-ai-studio

# Setup and run docker services
cd application/docker
./setup-devices.sh --xpu # or use --cuda, --cpu
docker compose up -d
```

Application runs at <http://localhost:7860>. See the [Docker README](./application/docker/README.md) for
hardware configuration (Intel XPU, NVIDIA CUDA) and device setup.

If you plan to train Hugging Face Hub-backed policies (for example, SmolVLA, Pi0,
and others), configure `HF_TOKEN` to avoid unauthenticated Hub access warnings. See
[Hugging Face Integration](./application/backend/docs/huggingface_integration.md).

#### Native: installation & running

Run the application in development mode, using [uv package manager](https://docs.astral.sh/uv/getting-started/installation/) and [node v24](https://nodejs.org/en/download) (we recommend using nvm)

Note: native setup requires additional OS-level libraries (OpenCV/video/USB and Python
build dependencies). See the **Prerequisites** section in
[Application Installation](./application/docs/01-installation.md#prerequisites).

```bash
# Clone the repository
git clone https://github.com/open-edge-platform/physical-ai-studio.git
cd physical-ai-studio

# Install and run backend
cd application/backend

# Start the backend, or use --extra cpu, --extra cuda
uv run --extra xpu physicalai-studio serve  # or: ./run.sh
```

```bash
# In a new terminal: install and run UI
cd application/ui
npm install

# Start the UI
npm run start
```

Open <http://localhost:3000> in your browser.

If you plan to train Hugging Face Hub-backed policies (for example, SmolVLA, Pi0,
and others), configure `HF_TOKEN` in your backend environment. See
[Hugging Face Integration](./application/backend/docs/huggingface_integration.md).

### Library (Python/CLI)

For programmatic control over training, benchmarking, and deployment with both API and CLI

```bash
pip install physicalai-train
```

<details open>
<summary>Training</summary>

```python test="skip" reason="requires dataset download"
from physicalai.data import LeRobotDataModule
from physicalai.policies import ACT
from physicalai.train import Trainer

datamodule = LeRobotDataModule(repo_id="lerobot/aloha_sim_transfer_cube_human")
model = ACT()
trainer = Trainer(max_epochs=100)
trainer.fit(model=model, datamodule=datamodule)
```

</details>

<details>
<summary>Benchmark</summary>

```python test="skip" reason="requires checkpoint and libero"
from physicalai.benchmark import LiberoBenchmark
from physicalai.policies import ACT

policy = ACT.load_from_checkpoint("experiments/lightning_logs/version_0/checkpoints/last.ckpt")
benchmark = LiberoBenchmark(task_suite="libero_10", num_episodes=20)
results = benchmark.evaluate(policy)
print(f"Success rate: {results.aggregate_success_rate:.1f}%")
```

</details>

<details>
<summary>Export</summary>

```python test="skip" reason="requires checkpoint"
from physicalai.export import get_available_backends
from physicalai.policies import ACT

# See available backends
print(get_available_backends())  # ['onnx', 'openvino', 'torch', 'executorch']

# Export to OpenVINO
policy = ACT.load_from_checkpoint("experiments/lightning_logs/version_0/checkpoints/last.ckpt")
policy.export("./policy", backend="openvino")
```

</details>

<details>
<summary>Inference</summary>

```python test="skip" reason="requires exported model and environment"
from physicalai.inference import InferenceModel

policy = InferenceModel("./policy")
obs, info = env.reset()
done = False

while not done:
    action = policy.select_action(obs)
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated
```

</details>

<details>
<summary>CLI Usage</summary>

```bash
# Train
physicalai fit --config configs/physicalai/act.yaml

# Evaluate
physicalai benchmark --config configs/benchmark/libero.yaml --ckpt_path model.ckpt

# Export (Python API only - CLI coming soon)
# Use: policy.export("./policy", backend="openvino")
```

</details>

[Library Documentation →](./library/README.md)

## Documentation

| Resource                                    | Description                         |
| ------------------------------------------- | ----------------------------------- |
| [Library Docs](./library/README.md)         | API reference, guides, and examples |
| [Application Docs](./application/README.md) | GUI setup and usage                 |
| [Contributing](./CONTRIBUTING.md)           | Contributing and development setup  |

## Contributing

We welcome contributions! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines.

physical-ai-studio

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README