llmdantic

KO | EN

gitlite — search

#typescript #react #python #nodejs #hacktoberfest #reactjs #csharp #javascript #nextjs #android #python3 #api

★ 59 Open GitHub ↗

Structured Output Is All You Need!

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h3 style="font-size: 5em" align="center">
   Structured Output Is All You Need!  
</h3>

<br>

LLMdantic is a powerful and efficient Python library that simplifies the integration of Large Language Models (LLMs) into your projects. Built on top of the incredible [Langchain](https://github.com/hwchase17/langchain) package and leveraging the power of [Pydantic](https://github.com/pydantic/pydantic) models, LLMdantic provides a seamless and structured approach to working with LLMs.

## Features 🚀

- 🌐 Wide range of LLM support through Langchain integrations
- 🛡️ Ensures data integrity with Pydantic models for input and output validation
- 🧩 Modular and extensible design for easy customization
- 💰 Cost tracking and optimization for OpenAI models
- 🚀 Efficient batch processing for handling multiple data points
- 🔄 Robust retry mechanism for smooth and uninterrupted experience

## Getting Started 🌟

### Requirements

Before using LLMdantic, make sure you have set the required API keys for the LLMs you plan to use. For example, if you're using OpenAI's models, set the `OPENAI_API_KEY` environment variable:

```bash
export OPENAI_API_KEY="your-api-key"
```

If you're using other LLMs, follow the instructions provided by the respective providers in Langchain's documentation.

### Installation

```bash
pip install llmdantic
```

### Usage

#### 1. Define input and output schemas using Pydantic:

- Use Pydantic to define input and output models with custom validation rules.

> [!IMPORTANT]
>
> Add docstrings to validation rules to provide prompts for the LLM. This will help the LLM understand the validation rules and provide better results

```python
from pydantic import BaseModel, field_validator

class SummarizeInput(BaseModel):
    text: str

class SummarizeOutput(BaseModel):
    summary: str

@field_validator("summary")  
    def summary_must_not_be_empty(cls, v) -> bool:
        """Summary cannot be empty"""  # Add docstring that explains the validation rule. This will be used as a prompt for the LLM.
        if not v.strip():
            raise
        return v

@field_validator("summary")
    def summary_must_be_short(cls, v) -> bool:  
        """Summary must be less than 100 words"""  # Add docstring that explains the validation rule. This will be used as a prompt for the LLM.
        if len(v.split()) > 100:
            raise  
        return v
```

#### 2. Create an LLMdantic client:

- Provide input and output models, objective, and configuration.

> [!TIP]
>
> The `objective` is a prompt that will be used to generate the actual prompt sent to the LLM. It should be a high-level description of the task you want the LLM to perform.
>
> The `inp_schema` and `out_schema` are the input and output models you defined in the previous step.
> 
> The `retries` parameter is the number of times the LLMdantic will retry the request in case of failure.

```python
from llmdantic import LLMdantic, LLMdanticConfig  
from langchain_openai import ChatOpenAI

llm = ChatOpenAI()

config: LLMdanticConfig = LLMdanticConfig(
    objective="Summarize the text", 
    inp_schema=SummarizeInput,
    out_schema=SummarizeOutput, 
    retries=3,
)

llmdantic = LLMdantic(llm=llm, config=config)
```

Here's the prompt template generated based on the input and output models:

```text
Objective: Summarize the text

Input 'SummarizeInput': 
{input}

Output 'SummarizeOutput''s fields MUST FOLLOW the RULES:
SummarizeOutput.summary:
• SUMMARY CANNOT BE EMPTY
• SUMMARY MUST BE LESS THAN 100 WORDS

{format_instructions}
```

#### 3. Generate output using the LLMdantic:

> [!TIP]
>
> The `invoke` method is used for single requests, while the `batch` method is used for batch processing.
>
> The `invoke` method returns an instance of `LLMdanticResult`, which contains the generated text, parsed output, and other useful information such as cost and usage stats such as the number of input and output tokens. Check out the [LLMdanticResult](#LLMdanticResult) model for more details.
>

```python
from llmdantic import LLMdanticResult

data = SummarizeInput(text="A long article about natural language processing...")
result: LLMdanticResult = llmdantic.invoke(data)

output: Optional[SummarizeOutput] = result.output

if output:
    print(output.summary)
```

Here's the actual prompt sent to the LLM based on the input data:

```text
Objective: Summarize the text

Input 'SummarizeInput': 
{'text': 'A long article about natural language processing...'}

Output 'SummarizeOutput''s fields MUST FOLLOW the RULES:
SummarizeOutput.summary:
• SUMMARY CANNOT BE EMPTY
• SUMMARY MUST BE LESS THAN 100 WORDS

The output should be formatted as a JSON instance that conforms to the JSON schema below.

As an example, for the schema {"properties": {"foo": {"title": "Foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]}
the object {"foo": ["bar", "baz"]} is a well-formatted instance of the schema. The object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted.

Here is the output schema:
{"properties": {"summary": {"title": "Summary", "type": "string"}}, "required": ["summary"]}
```

- For batch processing, pass a list of input data.

> [!IMPORTANT]
>
> The `batch` method returns a list of `LLMdanticResult` instances, each containing the generated text, parsed output, and other useful information such as cost and usage stats such as the number of input and output tokens. Check out the [LLMdanticResult](#LLMdanticResult) model for more details.
>
> The `concurrency` parameter is the number of concurrent requests to be made. Please check the usage limits of the LLM provider before setting this value.
>

```python
data: List[SummarizeInput] = [
    SummarizeInput(text="A long article about natural language processing..."),
    SummarizeInput(text="A long article about computer vision...")  
]
results: List[LLMdanticResult] = llmdantic.batch(data, concurrency=2)

for result in results:
    if result.output:
        print(result.output.summary)
```

#### 4. Monitor usage and costs:

> [!IMPORTANT]
>
> The cost tracking feature is currently available for OpenAI models only.
>
> The `usage` attribute returns an instance of `LLMdanticUsage`, which contains the number of input and output tokens, successful requests, cost, and successful outputs. Check out the [LLMdanticUsage](#LLMdanticUsage) model for more details.
>
> Please note that the usage is tracked for the entire lifetime of the `LLMdantic` instance.

- Use the `cost` attribute of the LLMdanticResult to track the cost of the request (currently available for OpenAI models).

- Use the `usage` attribute of the LLMdantic to track the usage stats overall.

```python
from llmdantic import LLMdanticResult

data: SummarizeInput = SummarizeInput(text="A long article about natural language processing...")  
result: LLMdanticResult = llmdantic.invoke(data)

if result.output:
    print(result.output.summary)

# Track the cost of the request (OpenAI models only)
print(f"Cost: {result.cost}")

# Track the usage stats
print(f"Usage: {llmdantic.usage}")
```

```bash
Cost: 0.0003665
Overall Usage: LLMdanticUsage(
  inp_tokens=219,
  out_tokens=19,
  total_tokens=238,
  successful_requests=1,
  cost=0.000367,
  successful_outputs=1
)
```

## Advanced Usage 🛠

`LLMdantic` is built on top of the langchain package, which provides a modular and extensible framework for working with LLMs. You can easily switch between different LLMs and customize your experience.

Switching LLMs

> [!IMPORTANT]
>
> Make sure to set the required API keys for the new LLM you plan to use.
>
> The `llm` parameter of the `LLMdantic` class should be an instance of `BaseLanguageModel` from the langchain package.
>

> [!TIP]
>
> You can use the `langchain_community` package to access a wide range of LLMs from different providers.
>
> You may need to provide model_name, api_key, and other parameters based on the LLM you want to use. Check out the documentation of the respective LLM provider for more details.
>

```python
from llmdantic import LLMdantic, LLMdanticConfig
from langchain_community.llm.ollama import Ollama
from langchain.llms.base import BaseLanguageModel

llm: BaseLanguageModel = Ollama()

config: LLMdanticConfig = LLMdanticConfig(
    objective="Summarize the text",
    inp_schema=SummarizeInput, 
    out_schema=SummarizeOutput,
    retries=3,
)

llmdantic = LLMdantic(
    llm=llm,
    config=config
)
```

## Contributing 🤝

Contributions are welcome! Whether you're fixing bugs, adding new features, or improving documentation, your help makes
**LLMdantic** better for everyone. Feel free to open an issue or submit a pull request.

## License 📄

**LLMdantic** is released under the [MIT License](LICENSE). Feel free to use it, contribute, and spread the word!

llmdantic

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README