bears-benchmark

KO | EN

gitlite — search

#react #typescript #golang #deep-learning #nodejs #macos #java #windows #awesome #cli #kubernetes #awesome-list #mcp #open-source

bears-benchmark

(★ 38)

An Extensible Java Bug Benchmark for Automatic Program Repair Studies

bears-benchmark 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# The Bears-Benchmark

The Bears-Benchmark, or just Bears, is a benchmark of bugs for automatic program repair studies in Java 8.
The bugs are collected from open-source projects hosted on GitHub through a process that scans pairs of builds from Travis Continuous Integration and reproduces bugs (by test failure) and their patches (passing test suite).

If you use Bears, please cite our paper:

```bibtex
@inproceedings{Madeiral2019,
  author = {Fernanda Madeiral and Simon Urli and Marcelo Maia and Martin Monperrus},
  title = {{Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies}},
  booktitle = {Proceedings of the 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER '19)},
  year = {2019},
  url = {https://arxiv.org/abs/1901.06024}
}
```

## Bug Repository Design

Each bug is stored in a branch.
The name of each branch follows the pattern `<GitHub project slug>-<buggy Travis build id>-<patched Travis build id>`.
The files and data on a bug are organized in commits in the branch of the bug as follows:

- Commit #1 contains the version of the program with the bug
- Commit #2 contains the changes in the tests (the one to execute to get a failure)
- Commit #3 contains the version of the program with the human-written patch
- Commit #4 contains the metadata file `bears.json`, which is a gathering of information collected during the bug reproduction process. It contains information about the bug (e.g. test failure names), the patch (e.g. patch size), and the bug reproduction process (e.g. duration).

## Prerequisites

Ensure you use Java 8.

```
$export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/ 
$ mvn -version
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 1.8.0_382, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-79-generic", arch: "amd64", family: "unix"
```

## How to use

In the folder [scripts](scripts/) you can find scripts to check out bugs, to compile them, and to run tests on them.

You can list the bugs in the current version of the benchmark with jq

```
jq ".[].bugId" scripts/data/bug_id_and_branch.json
```

You can check out a single bug by given a bug ID (see [bug IDs](https://bears-bugs.github.io/bears-benchmark)) as argument with

```bash
python scripts/checkout_bug.py --bugId <bug ID>
```

For example,

```bash
python scripts/checkout_bug.py --bugId Bears-1
python scripts/run_tests_bug.py --bugId Bears-1 # this should fail because the checkout_bug.py has checked out the third, buggy commit
```

In the same way as you can check out one or all bugs you can compile the checked out buggy program versions and run tests on them using the scripts `compile_all.py`, `compile_bug.py`, `run_tests_all.py`, `run_tests_bug.py`.

## Data

The original benchmark contained 251 bugs (see file `scripts/data/bug_id_and_branch_2019.json`). However, many bugs have rotten with dependencies having disappeared (in particular snaphots) and version problems.

The benchmark as of Sep 2023 contains 118 bugs (`jq length scripts/data/bug_id_and_branch.json`).

## Java 11

The following 17 bugs can also be reproduced with Java 11: Bears-150, Bears-179, Bears-194, Bears-206, Bears-207, Bears-211, Bears-212, Bears-220, Bears-225, Bears-231, Bears-232, Bears-236, Bears-238, Bears-240, Bears-242, Bears-246, Bears-250

## Contact

Feel free to create issues in this repository with questions.

## Credits

Thanks to Hohyeon Jeong for contributing to saving old bugs with workarounds.

bears-benchmark

bears-benchmark 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

bears-benchmark

bears-benchmark 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README