RAGchain - GitLite

KO | EN

gitlite — search

#javascript #python #ai #typescript #go #golang #android #rust #docker #java #windows #awesome-list #mcp #openai

RAGchain

(★ 282)

Extension of Langchain for RAG. Easy benchmarking, multiple retrievals, reranker, time-aware RAG, and so on...

File Explorer

Download Latest Version (.zip)

- - conf.py
  - index.rst
  - modules.rst
  - RAGchain.benchmark.answer.rst
  - RAGchain.benchmark.dataset.rst
  - RAGchain.benchmark.extra.rst
  - RAGchain.benchmark.retrieval.rst
  - RAGchain.benchmark.rst
  - RAGchain.DB.rst
  - RAGchain.pipeline.rst
  - RAGchain.preprocess.loader.rst
  - RAGchain.preprocess.rst
  - RAGchain.preprocess.text_splitter.rst
  - RAGchain.reranker.importance.rst
  - RAGchain.reranker.llm.rst
  - RAGchain.reranker.pygaggle.model.rst
  - RAGchain.reranker.pygaggle.rst
  - RAGchain.reranker.rst
  - RAGchain.reranker.tart.rst
  - RAGchain.reranker.time.rst
  - RAGchain.reranker.upr.rst
  - RAGchain.retrieval.rst
  - RAGchain.rst
  - RAGchain.schema.rst
  - RAGchain.utils.compressor.rst
  - RAGchain.utils.embed.rst
  - RAGchain.utils.linker.rst
  - RAGchain.utils.rst
  - RAGchain.utils.vectorstore.rst
  - RAGchain.utils.websearch.rst
- .nojekyll
- index.html
- make.bat
- Makefile
- - __init__.py
  - auto.py
  - base.py
- - __init__.py
  - base.py
  - mongo_db.py
  - pickle_db.py
- - __init__.py
  - base.py
  - basic.py
  - google_search.py
  - rerank.py
  - visconde.py
- - __init__.py
- - __init__.py
  - base.py
  - bm25.py
- - __init__.py
  - base.py
  - bm25_retrieval.py
  - hybrid.py
  - hyde.py
  - vectordb_retrieval.py
- - __init__.py
  - db_origin.py
  - evaluate_result.py
  - passage.py
  - prompt.py
  - retrieval_result.py
- - __init__.py
  - evidence_extractor.py
  - file_cache.py
  - query_decompose.py
  - rede_search_detector.py
  - semantic_clustering.py
  - util.py
- __init__.py
- - rem_sample.sqlite3
  - sample_passages.pkl
  - sample_test_document.txt
- conftest.py
- README.md
.env.template
.gitignore
CONTRIBUTING.md
dev_requirements.txt
LICENSE
pytest_template.ini
README.md
setup.py
tox.ini

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# RAGchain

RAGchain is a framework for developing advanced RAG(Retrieval Augmented Generation) workflow powered by LLM (Large Language Model).
While existing frameworks like Langchain or LlamaIndex allow you to build simple RAG workflows, they have limitations when it comes to building complex and high-accuracy RAG workflows.

RAGchain is designed to overcome these limitations by providing powerful features for building advanced RAG workflow easily.
Also, it is partially compatible with Langchain, allowing you to leverage many of its integrations for vector storage,
embeddings, document loaders, and LLM models.

[Docs](https://nomadamas.gitbook.io/ragchain-docs/) | [API Spec](https://marker-inc-korea.github.io/RAGchain/build/html/index.html) | [QuickStart](https://nomadamas.gitbook.io/ragchain-docs/quick-start)

# Quick Install
```bash
pip install RAGchain
```

# Why RAGchain?
RAGchain offers several powerful features for building high-quality RAG workflows:

## OCR Loaders
Simple file loaders may not be sufficient when trying to enhance accuracy or ingest real-world documents. OCR models can scan documents and convert them into text with high accuracy, improving the quality of responses from LLMs.

## Reranker
Reranking is a popular method used in many research projects to improve retrieval accuracy in RAG workflows. Unlike LangChain, which doesn't include reranking as a default feature, RAGChain comes with various rerankers.

## Great to use multiple retrievers
In real-world scenarios, you may need multiple retrievers depending on your requirements. RAGchain is highly optimized for using multiple retrievers. It divides retrieval and DB. Retrieval saves vector representation of contents, and DB saves contents. We connect both with Linker, so it is really easy to use multiple retrievers and DBs.

## pre-made RAG pipelines
We provide pre-made pipelines that let you quickly set up RAG workflow. We are planning to make much complex pipelines, which hard to make but powerful. With pipelines, you can build really powerful RAG system quickly and easily.

## Easy benchmarking

It is crucial to benchmark and test your RAG workflows. We have easy benchmarking module for evaluation. Support your
own questions and various datasets.

# Installation
## From pip

simply install at pypi.

```bash
pip install RAGchain
```

## From source
First, clone this git repository to your local machine.

```bash
git clone https://github.com/Marker-Inc-Korea/RAGchain.git
cd RAGchain
```

Then, install RAGchain module.
```bash
python3 setup.py develop
```

For using files at root folder and test, run dev requirements.
```bash
pip install dev_requirements.txt
```

# Supporting Features

## Advanced RAG features

- [Time-Aware RAG](https://nomadamas.gitbook.io/ragchain-docs/advanced-rag/time_aware_rag)
- [Importance-Aware RAG]()

## Retrievals
- BM25
- Vector DB
- Hybrid ([rrf](https://www.elastic.co/guide/en/elasticsearch/reference/current/rrf.html) and [cc](https://arxiv.org/abs/2210.11934))
- [HyDE](https://arxiv.org/abs/2212.10496)

## OCR Loaders

- [Nougat](https://github.com/facebookresearch/nougat)
- [Deepdoctection](https://github.com/deepdoctection/deepdoctection)

## Rerankers
- [UPR](https://github.com/DevSinghSachan/unsupervised-passage-reranking)
- [TART](https://github.com/facebookresearch/tart)
- BM25
- LLM
- [MonoT5](https://huggingface.co/castorini/monot5-3b-msmarco-10k)

## Web Search
- Google Search
- Bing Search

## Workflows (pipeline)
- Basic
- [Visconde](https://arxiv.org/abs/2212.09656)
- Rerank
- Google Search

## Extra utils
- Query Decomposition
- Evidence Extractor
- [REDE](https://arxiv.org/pdf/2109.08820.pdf) Search Detector
- Semantic Clustering
- Cluster Time Compressor

## Dataset Evaluators

- [MS-MARCO](https://paperswithcode.com/dataset/ms-marco)
- [Mr. Tydi](https://arxiv.org/abs/2108.08787)
- [Qasper](https://paperswithcode.com/dataset/qasper)
- [StrategyQA](https://allenai.org/data/strategyqa)
- [KoStrategyQA](https://huggingface.co/datasets/NomaDamas/Ko-StrategyQA)
- [ANTIQUE](https://paperswithcode.com/dataset/antique)
- [ASQA](https://arxiv.org/abs/2204.06092)
- [DSTC11-Track5](https://github.com/alexa/dstc11-track5)
- [Natural QA](https://paperswithcode.com/dataset/natural-questions)
- [NFCorpus](http://www.cl.uni-heidelberg.de/statnlpgroup/nfcorpus/)
- [SearchQA](https://arxiv.org/abs/1704.05179)
- [TriviaQA](https://arxiv.org/abs/1705.03551)
- [ELI5](https://huggingface.co/datasets/Pakulski/ELI5-test)

# Contributing
We welcome any contributions. Please feel free to raise issues and submit pull requests.

# Acknowledgement

This project is an early version, so it can be unstable. The project is licensed under the Apache 2.0 License.