gpt-json - GitLite

KO | EN

gitlite — search

#javascript #react #llm #machine-learning #rust #windows #cli #awesome-list #vue #security #mcp #ios #openai #open-source

gpt-json

(★ 734)

Structured and typehinted GPT responses in Python

파일 탐색기

최종 버전 다운로드 (.zip)

- - deploy.yml
  - lint.yml
- example.py
- function_example.py
- hint_example.py
- poetry.lock
- pyproject.toml
- README.md
- stream_example.py
- template_example.py
- truncation_example.py
- - __init__.py
  - conftest.py
  - shared.py
  - test_api.py
  - test_fixtures.py
  - test_fn_calling.py
  - test_gpt.py
  - test_models.py
  - test_parsers.py
  - test_prompts.py
  - test_streaming.py
  - test_transformations.py
  - test_truncation.py
- __init__.py
- enums.py
- exceptions.py
- fn_calling.py
- gpt.py
- models.py
- parsers.py
- prompts.py
- py.typed
- streaming.py
- transformations.py
- truncation.py
- types_oai.py
.git-blame-ignore-revs
.gitignore
LICENSE
lint.sh
poetry.lock
pyproject.toml
README.md

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/piercefreeman/gpt-json

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd gpt-json

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. 공식 설치 스크립트

쉬움 추천

사전 준비물

Python 3 pip 명령어를 쓰려면 Python이 필요합니다.

pip install gpt-json

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

✅ 설치 후 새 터미널을 열고, 프로그램의 버전 확인 명령(예: --version)으로 정상 설치됐는지 확인하세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install gpt-json

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

poetry install

pyproject.toml에 명시된 라이브러리를 설치합니다.

poetry run pytest .

이 명령어를 터미널에 그대로 입력해 실행하세요.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# gpt-json

`gpt-json` is a wrapper around GPT that allows for declarative definition of expected output format. Set up a schema, write a prompt, and get results back as beautiful typehinted objects.

This library introduces the following features:

- 🏗️ Pydantic schema definitions for type casting and validations
- 🧵 Templating of prompts to allow for dynamic content
- 🔎 Supports Vision API, Function Calling, and standard chat prompts
- 🚕 Lightweight transformations of the output to fix broken json
- ♻️ Retry logic for the most common API failures
- 📋 Predict single-objects and lists of objects
- ✈️ Lightweight dependencies: only OpenAI, pydantic, and backoff

## Getting Started

```bash
pip install gpt-json
```

Here's how to use it to generate a schema for simple tasks:

```python
import asyncio

from gpt_json import GPTJSON, GPTMessage, GPTMessageRole
from pydantic import BaseModel

class SentimentSchema(BaseModel):
    sentiment: str

SYSTEM_PROMPT = """
Analyze the sentiment of the given text.

Respond with the following JSON schema:

{json_schema}
"""

async def runner():
    gpt_json = GPTJSON[SentimentSchema](API_KEY)
    payload = await gpt_json.run(
        messages=[
            GPTMessage(
                role=GPTMessageRole.SYSTEM,
                content=SYSTEM_PROMPT,
            ),
            GPTMessage(
                role=GPTMessageRole.USER,
                content="Text: I love this product. It's the best thing ever!",
            )
        ]
    )
    print(payload.response)
    print(f"Detected sentiment: {payload.response.sentiment}")

asyncio.run(runner())
```

```bash
sentiment='positive'
Detected sentiment: positive
```

The `json_schema` is a special keyword that will be replaced with the schema definition at runtime. You should always include this in your payload to ensure the model knows how to format results. However, you can play around with _where_ to include this schema definition; in the system prompt, in the user prompt, at the beginning, or at the end.

You can either typehint the model to return a BaseSchema back, or to provide a list of multiple BaseSchema. Both of these work:

```python
from gpt_json.gpt import GPTJSON, ListResponse

gpt_json_single = GPTJSON[SentimentSchema](API_KEY)
gpt_json_multiple = GPTJSON[ListResponse[SentimentSchema]](API_KEY)
```

If you want to get more specific about how you expect the model to populate a field, add hints about the value through the "description" field. This helps the model understand what you're looking for, and will help it generate better results.

```python
from pydantic import BaseModel, Field

class SentimentSchema(BaseModel):
    sentiment: int = Field(description="Either -1, 0, or 1.")
```

```
sentiment=1
Detected sentiment: 1
```

## Prompt Variables

In addition to the `json_schema` template keyword, you can also add arbitrary variables into your messages. This allows you to more easily insert user generated content or dynamically generate prompts based on the results of previous messages.

```python
class QuoteSchema(BaseModel):
    quotes: list[str]

SYSTEM_PROMPT = """
Generate fictitious quotes that are {sentiment}.

{json_schema}
"""

gpt_json = GPTJSON[QuoteSchema](API_KEY)
response = await gpt_json.run(
    messages=[
        GPTMessage(
            role=GPTMessageRole.SYSTEM,
            content=SYSTEM_PROMPT,
        ),
    ],
    format_variables={"sentiment": "happy"},
)
```

When calling the `.run()` function you can pass it the values that should be filled in this template. This also extends to field descriptions as well, so you can specify custom behavior on a per-field basis.

```python
class QuoteSchema(BaseModel):
    quotes: list[str] = Field(description="Max quantity {max_items}.")

SYSTEM_PROMPT = """
Generate fictitious quotes that are {sentiment}.

{json_schema}
"""

gpt_json = GPTJSON[QuoteSchema](API_KEY)
response = await gpt_json.run(
    messages=[
        GPTMessage(
            role=GPTMessageRole.SYSTEM,
            content=SYSTEM_PROMPT,
        ),
    ],
    format_variables={"sentiment": "happy", "max_items": 5},
)
```

## Function Calls

`gpt-3.5-turbo-0613` and `gpt-4-0613` were fine-tuned to support a specific syntax for function calls. We support this syntax in `gpt-json` as well. Here's an example of how to use it:

```python
class UnitType(Enum):
    CELSIUS = "celsius"
    FAHRENHEIT = "fahrenheit"

class GetCurrentWeatherRequest(BaseModel):
    location: str = Field(description="The city and state, e.g. San Francisco, CA")
    unit: UnitType | None = None

class DataPayload(BaseModel):
    data: str

def get_current_weather(request: GetCurrentWeatherRequest):
    """
    Get the current weather in a given location
    """
    weather_info = {
        "location": request.location,
        "temperature": "72",
        "unit": request.unit,
        "forecast": ["sunny", "windy"],
    }
    return json_dumps(weather_info)

async def runner():
    gpt_json = GPTJSON[DataPayload](API_KEY, functions=[get_current_weather])
    response = await gpt_json.run(
        messages=[
            GPTMessage(
                role=GPTMessageRole.USER,
                content="What's the weather like in Boston, in F?",
            ),
        ],
    )

assert response.function_call == get_current_weather
    assert response.function_arg == GetCurrentWeatherRequest(
        location="Boston", unit=UnitType.FAHRENHEIT
    )
```

The response provides the original function alongside a formatted Pydantic object. If users want to execute the function, they can run response.function_call(response.function_arg). We will parse the get_current_weather function and the GetCurrentWeatherRequest parameter into the format that GPT expects, so it is more likely to return you a correct function execution.

GPT makes no guarantees about the validity of the returned functions. They could hallucinate a function name or the function signature. To address these cases, the run() function may now throw two new exceptions:

`InvalidFunctionResponse` - The function name is incorrect.
`InvalidFunctionParameters` - The function name is correct, but doesn't match the input schema that was provided.

## Other Configurations

The `GPTJSON` class supports other configuration parameters at initialization.

| Parameter                   | Type                   | Description                                                                                                                                                                                                                     |
| --------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| model                       | GPTModelVersion \| str | (default: GPTModelVersion.GPT_4) - For convenience we provide the currently supported GPT model versions in the `GPTModelVersion` enum. You can also pass a string value if you want to use another more specific architecture. |
| auto_trim                   | bool                   | (default: False) - If your input prompt is too long, perhaps because of dynamic injected content, will automatically truncate the text to create enough room for the model's response.                                          |
| auto_trim_response_overhead | int                    | (default: 0) - If you're using auto_trim, configures the max amount of tokens to allow in the model's response.                                                                                                                 |
| \*\*kwargs                  | Any                    | Any other parameters you want to pass to the underlying `GPT` class, will just be a passthrough.                                                                                                                                |

## Transformations

GPT (especially GPT-4) is relatively good at formatting responses at JSON, but it's not perfect. Some of the more common issues are:

- _Response truncation_: Since GPT is not internally aware of its response length limit, JSON payloads will sometimes exhaust the available token space. This results in a broken JSON payload where much of the data is valid but the JSON object is not closed, which is not valid syntax. There are many cases where this behavior is actually okay for production applications - for instance, if you list 100 generated strings, it's sometimes okay for you to take the 70 that actually rendered. In this case, `gpt-json` will attempt to fix the truncated payload by recreating the JSON object and closing it.
- _Boolean variables_: GPT will sometimes confuse valid JSON boolean values with the boolean tokens that are used in other languages. The most common is generating `True` instead of `true`. `gpt-json` will attempt to fix these values.

When calling `gpt_json.run()`, we return a tuple of values:

```python
payload = await gpt_json.run(...)

print(transformations.fix_transforms)
```

```bash
FixTransforms(fixed_truncation=True, fixed_bools=False)
```

The first object is your generated Pydantic model. The second object is our correction storage object `FixTransforms`. This dataclass contains flags for each of the supported transformation cases that are sketched out above. This allows you to determine whether the response was explicitly parsed from the GPT JSON, or was passed through some middlelayers to get a correct output. From there you can accept or reject the response based on your own business logic.

_Where you can help_: There are certainly more areas of common (and not-so-common failures). If you see these, please add a test case to the unit tests. If you can write a handler to help solve the general case, please do so. Otherwise flag it as a `pytest.xfail` and we'll add it to the backlog.

## Testing

We use poetry for package management. To run the bundled tests, clone the package from github.

```bash
poetry install
poetry run pytest .
```

Our focus is on making unit tests as robust as possible. The variability with GPT should be in its language model, not in its JSON behavior! This is still certainly a work in progress. If you see an edge case that isn't covered, please add it to the test suite.

## Comparison to Other Libraries

A non-exhaustive list of other libraries that address the same problem. None of them were fully compatible with my deployment (hence this library), but check them out:

[jsonformer](https://github.com/1rgs/jsonformer) - Works with any Huggingface model, whereas `gpt-json` is specifically tailored towards the GPT-X family. GPT doesn't output logit probabilities or allow fixed decoder templating so the same approach can't apply.

## Formatting

We use black and mypy for formatting. You can set up a pre-commit git hook to do this automatically via the `./lint.sh` helper file.

If you perform a bulk reformatting to the codebase, you should add your most recent commit to the `.git-blame-ignore-revs` file and run:

```
git config blame.ignoreRevsFile .git-blame-ignore-revs
```