KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #deepseek-harness #typescript #ai-agent #codex #developer-tools #claude-code #windows #mcp #python #desktop-app

CodeClash

★ 200 Open GitHub ↗

Benchmarking Goal-Oriented Software Engineering

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<hr />

## 👋 Overview

CodeClash is a benchmark for evaluating AI systems on **goal-oriented software engineering**.

Today's AI coding evals are *task*-oriented (e.g.,
<a href="https://github.com/openai/human-eval">HumanEval</a>, <a href="https://swebench.com">SWE-bench</a>).
Models are given explicit instructions.
We then verify correctness with unit tests.

But building software is fundamentally driven by goals ("improve user retention", "reduce costs", "increase revenue").
Reaching our goals via code is a self-directed, iterative, and often competitive process.
To capture this dynamism of real software development, we introduce CodeClash!

Check out our [arXiv paper](https://arxiv.org/abs/2511.00839) and [website](https://codeclash.ai/) for the full details!

## 🏎️ Quick Start

### Prerequisites

- **Python 3.11+**
- **[uv](https://docs.astral.sh/uv/)** - Fast Python package manager
- **Docker** - For running games in containers
- **Git**

### Installation

```bash
# Clone the repository
git clone https://github.com/CodeClash-ai/CodeClash.git
cd CodeClash

# Install uv (if you haven't already)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install dependencies and create virtual environment
uv sync --extra dev

# Set up your environment variables
cp .env.example .env  # Then edit .env with your GITHUB_TOKEN

# Run a test battle
uv run codeclash run configs/test/battlesnake.yaml
```

> [!TIP]
> CodeClash requires Docker to create execution environments. CodeClash was developed and tested on Ubuntu 22.04.4 LTS.
> The same instructions should work for Mac. If not, check out [#81](https://github.com/CodeClash-ai/CodeClash/issues/81) for an alternative solution.

<details>
<summary>Alternative: Using pip (not recommended)</summary>

```bash
pip install -e '.[dev]'
codeclash run configs/test/battlesnake.yaml
```
</details>

Once this works, you should be set up to run a real tournament!
To run *Claude Sonnet 4.5* against *o3* in a *BattleSnake* tournament with *5 rounds* and *1000 competition simulations* per round, run:
```bash
uv run codeclash run configs/pvp/BattleSnake__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml
```

## ⚔️ How It Works

In CodeClash, 2+ coding agents compete in a **code arena** over the course of a multi-round tournament.

For the duration of the tournament, each agent is iteratively improving their own codebase to win a high-level, competitive objective (e.g., accumulate resources, survive the longest, etc).

Each round consists of two phases:

* Edit phase: LM agents make whatever changes they want to their codebase.
* Competition phase: The modified codebases are pitted against each other in the arena.

Critically, *LMs don't play the game directly*.
Their code serves as their competitive proxy.
The winner is the LM agent who wins the most rounds.

## 🧩 Available Arenas

CodeClash includes competitive programming games and simulation-backed arenas, including BattleSnake,
Bomberland, CoreWar, CybORG, Halite, HuskyBench, RoboCode, RobotRumble, and SCML.

## 🚀 Get Involved

- Check out our [docs](https://docs.codeclash.ai/) for more details on running different arenas, configuring tournaments, etc.
- Explore [2000+ tournaments](https://viewer.codeclash.ai/) via our viewer.
- See our [contribution guide](CONTRIBUTING.md) for what we're excited about!
- Have a big idea? Open an issue, and let's turn it into an [insight](https://codeclash.ai/insights/)!

## 💫 Contributions
We're actively working on several follow ups!
Check out the [Contributing Guide](CONTRIBUTING.md) for more.

Contact Person: [John Yang](https://john-b-yang.github.io/), [Kilian Lieret](https://lieret.net)
(Email: [johnby@stanford.edu](mailto:johnby@stanford.edu), [kl5675@princeton.edu](mailto:kl5675@princeton.edu))

## 🪪 License
MIT. Check `LICENSE` for more information.

## ✍️ Citation

```bibtex
@misc{yang2025codeclashbenchmarkinggoalorientedsoftware,
    title={CodeClash: Benchmarking Goal-Oriented Software Engineering},
    author={John Yang and Kilian Lieret and Joyce Yang and Carlos E. Jimenez and Ofir Press and Ludwig Schmidt and Diyi Yang},
    year={2025},
    eprint={2511.00839},
    archivePrefix={arXiv},
    primaryClass={cs.SE},
    url={https://arxiv.org/abs/2511.00839},
}
```

## 📕 Our Other Projects
<div align="center">
  <a href="https://github.com/SWE-bench/SWE-bench"><img src="docs/assets/swebench_logo_text_below.svg" alt="SWE-bench" height="120px"></a>
  &nbsp;&nbsp;
  <a href="https://github.com/SWE-agent/SWE-agent"><img src="docs/assets/sweagent_logo_text_below.svg" alt="SWE-agent" height="120px"></a>
  &nbsp;&nbsp;
  <a href="https://github.com/SWE-agent/Mini-SWE-Agent"><img src="docs/assets/mini_logo_text_below.svg" alt="Mini-SWE-Agent" height="120px"></a>
  &nbsp;&nbsp;
  <a href="https://github.com/SWE-agent/SWE-ReX"><img src="docs/assets/swerex_logo_text_below.svg" alt="SWE-ReX" height="120px"></a>
  &nbsp;&nbsp;
  <a href="https://github.com/SWE-bench/SWE-smith"><img src="docs/assets/swesmith_logo_text_below.svg" alt="SWE-smith" height="120px"></a>
</div>

CodeClash

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README