KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agent #developer-tools #codex #typescript #cli #deepseek-harness #claude-code #ai-agents #mcp #rust

GAR

★ 74 Open GitHub ↗

Code and resources for papers "Generation-Augmented Retrieval for Open-Domain Question Answering" and "Reader-Guided Passage Reranking for Open-Domain Question Answering", ACL 2021

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repo provides the code and resources of the following papers:

**GAR**

["Generation-Augmented Retrieval for Open-domain Question Answering"](https://arxiv.org/abs/2009.08553), ACL 2021

TLDR: GAR augments a question with relevant contexts generated by seq2seq learning, with the question as input and target outputs such as the answer, the sentence where the answer belongs to, and the title of a passage that contains the answer. With the generated contexts appended to the original questions, GAR achieves state-of-the-art OpenQA performance with a simple BM25 retriever.

**RIDER**

["Reader-Guided Passage Reranking for Open-Domain Question Answering"](https://arxiv.org/abs/2101.00294), Findings of ACL 2021.

TLDR: RIDER is a simple and effective passage reranker, which reranks retrieved passages by reader predictions without any training. RIDER achieves 10\~20 gains in top-1 retrieval accuracy, 1\~4 gains in Exact Match (EM), and even outperforms supervised transformer-based rerankers.

## Data & Model Output
**[update 2022-02]** The [retrieval results of GAR on TriviaQA](https://drive.google.com/file/d/1pUvGCv8et7SrXsJNqZkzTp1O6Dw2SH3y/view?usp=sharing) have been uploaded.

**[update 2022-01]** The [data](https://drive.google.com/file/d/15JYV5nnZNSMJNVV7OwZioc4NIljvWgHq/view?usp=sharing) for GAR training/testing on TriviaQA is uploaded.

**[update 2021-08]** We provide the [data](https://drive.google.com/file/d/1T1YN4-UZGF_UN0N6XWDjOmxJrd90CwDW/view?usp=sharing) for GAR training/testing on NaturalQuestions (NQ).

We now provide the [generation-augmented queries](https://drive.google.com/file/d/1OstUyyh6n9otQj7TqiShDjlJxHVQ5Uv0/view?usp=sharing) (in case you wonder what they look like and/or perform retrieval yourself) and the retrieval results of [GAR](https://drive.google.com/file/d/1HmgS1TOx7cKh-RixbXRKGSqQWh5V_DmN/view?usp=sharing) and [GAR+](https://drive.google.com/file/d/1eF5Eb4cEhs2hZK6RNIKGCYA-hme9tdtM/view?usp=sharing) (same format as DPR, easily replacable) on the test set of NaturalQuestions. You may achieve performance improvements by simply replacing the retrieval results of DPR to GAR/GAR+ during inference. If not, you may need to re-train the reader using GAR/GAR+ results as well. More outputs will be released in the future.

For seq2seq learning, use {train/val/test}.source as the input and {train/val/test}.target as the output, where each line is one example.
We provide the data files in such formats for GAR training/testing above.

In the same folder, save the list of ground-truth answers with name {val/test}.target.json if you want to evaluate EM during training (of the generative reader).

Please refer to [DPR](https://github.com/facebookresearch/DPR#resources--data-formats) for the original dataset downloading.

## Code

### Generation

The codebase of seq2seq models is based on [huggingface](https://github.com/huggingface)/[transformers](https://github.com/huggingface/transformers) (version==2.11.0) examples.

**See  `train_gen.yml` for the package requirements and example commands to run the models.**

`train_generator.py`: training of seq2seq models.

`conf.py`: configurations for `train_generator.py`.  There are some default parameters but it might be easier to set e.g., `--data_dir` and `--output_dir` directly.

`test_generator.py`: test of seq2seq models (if not already done in `train_generator.py`).

### Retrieval

We use [pyserini](https://github.com/castorini/pyserini) for BM25 retrieval. Please refer to its [document](https://github.com/castorini/pyserini/#how-do-i-index-and-search-my-own-documents) for indexing and searching wiki passages (wiki passages can be downloaded [here](https://github.com/facebookresearch/DPR#resources--data-formats)). Alternatively, you may take a look at its [effort to reproduce DPR results](https://github.com/castorini/pyserini/blob/master/docs/experiments-dpr.md), which gives more detailed instructions and incorporates the passage-level span voting in GAR.

### Reranking

Please see the instructions in `rider/rider.py`.

### Reading

We experiment with one extractive reader and one generative reader.

For the extractive reader, we take the one used by dense passage retrieval. Please refer to [DPR](https://github.com/facebookresearch/DPR) for more details.

For the generative reader, we reuse the codebase in the generation stage above, with [question; top-retrieved passages] as the source input and one ground-truth answer as the target output. Example script is provided in `train_gen.yml`.

## Citation

Please cite our papers if you find them useful.

```
@inproceedings{mao-etal-2021-generation,
    title = "Generation-Augmented Retrieval for Open-Domain Question Answering",
    author = "Mao, Yuning  and
      He, Pengcheng  and
      Liu, Xiaodong  and
      Shen, Yelong  and
      Gao, Jianfeng  and
      Han, Jiawei  and
      Chen, Weizhu",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.316",
    doi = "10.18653/v1/2021.acl-long.316",
    pages = "4089--4100",
}

@inproceedings{mao-etal-2021-reader,
    title = "Reader-Guided Passage Reranking for Open-Domain Question Answering",
    author = "Mao, Yuning  and
      He, Pengcheng  and
      Liu, Xiaodong  and
      Shen, Yelong  and
      Gao, Jianfeng  and
      Han, Jiawei  and
      Chen, Weizhu",
    booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.findings-acl.29",
    doi = "10.18653/v1/2021.findings-acl.29",
    pages = "344--350",
}

```

GAR

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README