LiveResearchBench

(★ 121)

A live benchmark and evaluation framework for open-ended deep research in the wild.

File Explorer

Download Latest Version (.zip)

- batch_config.yaml
- batch_config.yaml.example
- - qid_infoco2F10Hm9IIrzZsKJR_report.md
  - qid_infocoAHb8t0CYTuzTCznB_report.md
  - qid_infocoBp2TDUwPt62Iqzak_report.md
  - qid_infocoK5coAaUHa2SScywL_report.md
  - qid_infocomcw7CBtHAmVbvEYm_report.md
  - qid_infocoTspDptS95JzUHtfS_report.md
  - qid_infocoUaNfGu8KK80qMmpg_report.md
  - qid_infocoXlRepzUFvAfGkcSn_report.md
  - qid_lifesp2DJmlulFMDdLb7hA_report.md
  - qid_lifesp3dNgib6HFZgsHUFR_report.md
  - qid_lifesp3KjIoCKZxJ10IgVh_report.md
  - qid_lifespjtHZwYQLi8NU6SdU_report.md
  - qid_lifespOPMqhxL5pPFJsWd8_report.md
  - qid_lifespUycUsYiR4GtRfrKQ_report.md
  - qid_lifespwgR2QorAuHgkzask_report.md
  - qid_lifespX6WTipV09EAveX19_report.md
  - qid_market6VWmPyxptfK47civ_report.md
  - qid_marketCb3JeqbQuoNtoO6e_report.md
  - qid_marketcmQtxteZNgUdSpUE_report.md
  - qid_marketDEJL1uDRvx5UqKKm_report.md
  - qid_marketJg4461KKm2uAPb7X_report.md
  - qid_marketONPr2EPqd6edkLv7_report.md
  - qid_marketOZ8DcbjNbI7D7gkQ_report.md
  - qid_marketY2oyfQ0PA24CMjb4_report.md
  - qid_policy0nSp8LBMkdYR1dN4_report.md
  - qid_policy6ukPGIQni5apWCMS_report.md
  - qid_policyDAYZvZSnnBh32igN_report.md
  - qid_policyfX8vyl8vSaEC7Jue_report.md
  - qid_policyGkZ8xushXFtVcZHF_report.md
  - qid_policymqmG9gpR6KDuxp5F_report.md
  - qid_policyRtVmcbZHm8DO6rmR_report.md
  - qid_policytDVXqY4eQXxyrtLC_report.md
  - qid_procon2EqeEXA31JZn71e7_report.md
  - qid_proconh2ZdDUYEw7hj6yTY_report.md
  - qid_proconhy0bVrN2JhZ2Y4GE_report.md
  - qid_proconoaEUC24gzbaHqsB9_report.md
  - qid_proconX7zAJLOPgcyZRDpZ_report.md
  - qid_procony5ewSmxa6EiwByts_report.md
  - qid_proconyXgOxCYl7uWArBmZ_report.md
  - qid_prodcp1BJglRaf2jODTVcl_report.md
  - qid_prodcp5SZd3x5Kd51QEyJC_report.md
  - qid_prodcpGSsSCKpQAvSZhczP_report.md
  - qid_prodcpJ5QMHZrBMvFuCKbY_report.md
  - qid_prodcpk7qXxlpmrrTy0GyZ_report.md
  - qid_prodcponmUtiKpSBpdwMUq_report.md
  - qid_prodcpQVPUCCeIbgfrg01Y_report.md
  - qid_prodcpslATZmS6ESv2yDln_report.md
  - qid_rankli824vjs9DMOPbb7HD_report.md
  - qid_ranklidfdj3RVVH31L5VPo_report.md
  - qid_rankliDwyJ6ePJgm8znsQh_report.md
  - qid_rankliFmBTzbMqx2kaw22n_report.md
  - qid_rankliNmeXNx8zhLK17tZm_report.md
  - qid_rankliqLKYjYMVUuUxPsQk_report.md
  - qid_rankliQW3nT9D69Y9z2IwY_report.md
  - qid_rankliscQtnu8LeS5DQPDu_report.md
  - qid_review5q6iS0uJfnrUKg5h_report.md
  - qid_review7WR9DY30D57MosM2_report.md
  - qid_reviewDuKImcS9AcUMBZGy_report.md
  - qid_reviewes8I18LtI6ioOQum_report.md
  - qid_reviewK0nAC9ggA3ijafYR_report.md
  - qid_reviewkF4ySyG6kBGDm8gm_report.md
  - qid_reviewUSC6Z2qYt8BXkwBa_report.md
  - qid_reviewvl0BjYy6KRNoqDNQ_report.md
  - qid_techsp5dgyRDlZ9iuOOEpT_report.md
  - qid_techsp9FOJRdQzS5k3RUpc_report.md
  - qid_techspH03dNoqnVQ9MahGu_report.md
  - qid_techspkCUk3fQGVjhIjenV_report.md
  - qid_techspnypcYIutGeysr17X_report.md
  - qid_techsppQVl7qtQ5HVWUJKp_report.md
  - qid_techspsK4Re3Jq5v6Zj5AB_report.md
  - qid_techspSZIG8XzmicGFYvHD_report.md
  - qid_topic0wgEjhAfQW51cyiZ_report.md
  - qid_topicBSSGEhfWR3DVNsA2_report.md
  - qid_topicd2PZ70chNQZVYofJ_report.md
  - qid_topicFUrl0vQ0wz5kkSHx_report.md
  - qid_topicnMbVhT4zxQSZHUx2_report.md
  - qid_topicqwjvYGldTBetIKqZ_report.md
  - qid_topicU5Om4qeJWDc9Pn2A_report.md
  - qid_topicxQpauRxmf4wtVEVl_report.md
  - qid_topiczL71JengsGofPr0A_report.md
- DATASET.md
- task_domain_dist.png
- - __init__.py
  - io_utils.py
  - model_clients.py
  - reference_reports.py
- - __init__.py
  - citation.py
  - consistency.py
  - coverage.py
  - depth.py
  - presentation.py
- - __init__.py
  - base_grader.py
  - checklist_grader.py
  - pairwise_grader.py
  - pointwise_grader.py
- __init__.py
- batch_evaluator.py
- batch_grade.sh
- batch_grade_multi_provider.sh
- preprocess_all.sh
- test_grading.py
.env.example
.gitignore
AI_ETHICS.md
average_results.py
CODE_OF_CONDUCT.md
CODEOWNERS
CONTRIBUTING.md
LICENSE.txt
main.py
preprocess.py
pyproject.toml
README.md
SECURITY.md
uv.lock

# Installation Guide

1. Get the code

git clone https://github.com/SalesforceAIResearch/LiveResearchBench

Downloads the entire project code from GitHub to your computer.

cd LiveResearchBench

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install .

Installs the package published on PyPI directly — no need to clone the source.

python main.py

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.