big-ann-benchmarks

(★ 446)

Framework for evaluating ANNS algorithms on billion scale datasets.

File Explorer

Download Latest Version (.zip)

- - neurips21.yml
  - neurips23.yml
- - base.py
  - base_runner.py
  - bbann.py
  - buddy_t1.py
  - cuanns_ivfpq.py
  - cuanns_multigpu.py
  - definitions.py
  - diskann-t2.py
  - diskann-v0_3.py
  - elastiknn.py
  - faiss_inmem.py
  - faiss_t1.py
  - faiss_t3.py
  - gemini.py
  - httpann.py
  - httpann_example.py
  - kota-t2.py
  - kst_t1.py
  - puck_t1.py
  - team11.py
- - __init__.py
  - eval_range_search.py
  - metrics.py
  - plot_variants.py
  - utils.py
- - power_capture.py
- - __init__.py
  - compute_gt.py
  - download_gt.py
  - load_runbook.py
- - __init__.py
  - helper.py
- __init__.py
- dataset_io.py
- datasets.py
- distances.py
- main.py
- results.py
- runner.py
- - distance_histogram.png
  - IoU.png
  - pr_compression.png
  - result_stats.png
- caselaw_dataset.md
- dino10b_dataset.md
- FB_ssnpp_dataset.md
- filter_query_set_by_num_matches.py
- filter_query_set_by_num_predicates.py
- filter_query_set_by_selectivity.py
- filter_query_set_utils.py
- filter_utils.md
- jsonl_filter_utils.py
- make_filtered_groundtruth.py
- make_groundtruth.py
- make_sparse_groundtruth.py
- multi_vector_utils.py
- prepare_bigann.py
- prepare_fb_ssnpp.py
- prepare_yfcc100m.py
- sparse_dataset.md
- yfcc_filtered_dataset.md
- yfccimages_dataset.md
- show_operating_points.py
- Dockerfile
- Dockerfile.bbann
- Dockerfile.diskann
- Dockerfile.elastiknn
- Dockerfile.faiss
- Dockerfile.faissconda
- Dockerfile.httpann_example
- Dockerfile.kota
- Dockerfile.kst_ann_t1
- Dockerfile.pqbuddy
- Dockerfile.puck
- requirements_conda.txt
- - README.md
- - LEADERBOARDS.md
  - LEADERBOARDS_PRIVATE.md
  - LEADERBOARDS_PRIVATE_REJECT_ANOMALIES.md
  - LEADERBOARDS_PUBLIC.md
  - LEADERBOARDS_PUBLIC_REJECT_ANOMALIES.md
  - LEADERBOARDS_REJECT_ANOMALIES.md
  - RANKING.md
  - README.md
  - TASKS_ISSUES_RESOLUTIONS.md
- - __init__.py
  - baseline_faiss.py
  - baseline_faiss_filtered.py
  - parse_results.py
  - README.md
  - run_baselines.bash
  - test_bow_id_selector.py
- - gpu_baseline_faiss.py
  - README.md
- README.md
- - base.py
  - install_neurips23.sh
  - operating_points_private_queries_AzureD8lds_v5.csv
  - operating_points_public_queries_AzureD8lds_v5.txt
  - plot_public_queries_AzureD8lds_v5.png
  - res_private_queries_AzureD8lds_v5.csv
  - res_public_queries_AzureD8lds_v5.csv
  - run.py
- - README.md
- - Azure_D8lds_v5_table.md
  - leaderboard.md
- - base.py
  - install_neurips23.sh
  - operating_points_public_queries_AzureD8lds_v5.txt
  - plot_public_queries_AzureD8lds_v5.png
  - res_public_queries_AzureD8lds_v5.csv
  - run.py
- - clustered_data_gen.py
  - clustered_replace_runbook.yaml
  - clustered_runbook.yaml
  - delete_runbook.yaml
  - final_runbook.yaml
  - final_runbook_gen.py
  - gen_expiration_time_runbook.py
  - gen_replace_runbooks.py
  - generate_msturing10m_runbooks.py
  - msmarco-100M_expirationtime_runbook.yaml
  - msturing-10M_slidingwindow_runbook.yaml
  - random_replace_runbook.yaml
  - simple_replace_runbook.yaml
  - simple_runbook.yaml
  - wikipedia-1M_expiration_time_replace_delete_runbook.yaml
  - wikipedia-1M_expiration_time_replace_only_runbook.yaml
  - wikipedia-1M_expiration_time_runbook.yaml
  - wikipedia-35M_expiration_time_replace_delete_runbook.yaml
  - wikipedia-35M_expiration_time_replace_only_runbook.yaml
  - wikipedia-35M_expirationtime_runbook.yaml
- - base.py
  - install_neurips23.sh
  - operating_points_private_queries_AzureD8lds_v5.csv
  - operating_points_public_queries_AzureD8lds_v5.txt
  - plot_public_queries_AzureD8lds_v5.png
  - res_private_queries_AzureD8lds_v5.csv
  - res_public_queries_AzureD8lds_v5.csv
  - run.py
- - __init__.py
  - base.py
  - README.md
  - res_final_runbook_AzureD8lds_v5.csv
  - run.py
- __init__.py
- Azure_D8lds_v5_table.md
- common.py
- Dockerfile
- ec2_c6i.2xlarge_res.csv
- ec2_c6i.2xlarge_table.md
- leaderboard.md
- README.md
- - parse_filtered_results.ipynb
- recall_tests.py
- tests.sh
.dockerignore
.gitignore
algos-2021.yaml
create_dataset.py
data_export.py
install.py
LICENSE
logging.conf
plot.py
README.md
requirements.txt
requirements_py3.10.txt
requirements_py3.12.txt
requirements_py38.txt
run.py
run_algorithm.py
setup_links.sh

# Installation Guide

1. Get the code

git clone https://github.com/harsha-simhadri/big-ann-benchmarks

Downloads the entire project code from GitHub to your computer.

cd big-ann-benchmarks

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Docker Desktop Needed to build and run containers. Install it and keep it running in the background.

docker build -f install/Dockerfile -t big-ann-benchmarks .

Builds a runnable image based on the Dockerfile.

docker run -p 8080:80 big-ann-benchmarks

Runs the built image as an actual container.

✅ Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

3. Python

Easy

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r neurips21/t3/gemini/requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python run.py

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.