KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agents #developer-tools #typescript #codex #claude-code #mcp #rust #deepseek-harness #ai-agent #cli

extralit

★ 46 Open GitHub ↗

Fast and accurate systematic literature data extraction with LLM assistance

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h3>
<p align="center">
<a href="https://docs.extralit.ai/latest/" target="_blank">📄 Documentation</a> | </span>
<a href="https://docs.extralit.ai/latest/getting_started/quickstart/" target="_blank">🚀 Quickstart</a> <span> | </span>
<a href="https://docs.extralit.ai/latest/community/developer/" target="_blank">🛠️ Architecture</a> <span>
</p>
</h3>

## What is Extralit?

Extralit (EXTRAct LITerature) is a data extraction workflow with user-friendly UI, designed for **LLM-assisted scientific data extraction** and other **unstructured document intelligence** tasks. It focuses on data accuracy above all else, and further integrates human feedback loops for continuous LLM refinement and collaborative data extraction.

Why Use Extralit?

- Precision First – Built for high data accuracy, ensuring reliable results.
- Human-in-the-Loop – Seamlessly integrate human annotations to refine LLM outputs and collaborate on data validation.
- Flexible & Scalable – Available as a Python SDK, CLI, and Web UI with multiple deployment options to fit your workflow.

Key Features:

- Schema-Driven Extraction – Define structured schemas for context-aware, high-accuracy data extraction across scientific domains.
- Advanced PDF Processing – AI-powered OCR detects complex table structures in both digital and scanned PDFs.
- Built-in Validation – Automatically verify extracted data for accuracy in both the annotation UI and the data pipeline outputs.
- User-Friendly Interface – Easily review, edit, and validate data with team-based consensus workflows.
- Data Flywheel – Collect human annotations to monitor performance and build fine-tuning datasets for continuous improvement.

Start extracting smarter with Extralit! 🚀

## Project background

Extralit began at the [Institute for Disease Modeling](https://www.idmod.org/) at the Gates Foundation, where disease models depend on structured evidence extracted from thousands of papers. See [NOTICE](NOTICE) for attribution details.

It builds on [Argilla](https://github.com/argilla-io/argilla) (Apache-2.0), whose original authors have stepped back from active feature development. Extralit carries that codebase forward for scientific literature extraction, and has since migrated it to Vue 3 / Nuxt 4 with versioned schema storage. API consolidation is in progress.

## Recent News

- **May 2025**: Extralit selected for Google Summer of Code 2025 under the Open Science Labs umbrella organization! We're working on Scientific PDF Data Extraction and Interactive Schema Editor UI projects.
- **Looking to contribute?** Check out our GSoC projects or open issues to get started!

## Getting started

### Installation
Install the client package

```bash
pip install extralit
```

If you already have a server deployed and login credentials, obtain your API key in the User Settings. You can manage your extraction workspace through the CLI with:

```bash
extralit login --api-url http://<extralit_server_instance>
# You will be prompted an API key to login to your account
```

### Server setup

See [https://docs.extralit.ai/latest/getting_started/quickstart/](https://docs.extralit.ai/latest/getting_started/quickstart/)

## 🛠️ Project Architecture

Extralit is built on top of Argilla, extending its capabilities with enhanced data extraction, validation, and human-in-the-loop workflows, with these 5 core components:

- **Python SDK**: A Python SDK which is installable with `pip install extralit` to interact with the web server and provides an API to manage the data extraction workflows.
- **FastAPI Server**: The backbone of Extralit, handling users, storage, and API interactions. It manages application data using a relational database (PostgreSQL by default).
- **Web UI**: A web application to visualize and annotate your data, users and teams. It is built with *Vue.js* and *Nuxt.js* and is directly deployed alongside the FastAPI Server within our Docker image.
- **Vector Database**: A vector database to store the records data and perform scalable vector similarity searches and basic document searches. We currently support *ElasticSearch* and *AWS OpenSearch* and they can be deployed as separate Docker images.

## Repo Activity

![Alt](https://repobeats.axiom.co/api/embed/3837e8b3ea152454b555ace7b6675060649004b7.svg "Repobeats analytics image")

extralit

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README