awesome-eval-driven-development

KO | EN

gitlite — search

#javascript #python #hacktoberfest #golang #go #android #machine-learning #docker #nodejs #java #awesome #kubernetes #awesome-list #open-source

awesome-eval-driven-development

(★ 37)

A curated list of resources, projects, and products to help implement Eval-Driven-Development (EDD) for LLM-backed apps.

awesome-eval-driven-development 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Awesome Eval Driven Development (EDD)

Eval-Driven-Development (EDD) is a methodology for guiding the development of LLM-backed apps via a set of task-specific evals (i.e. prompt, context, expected outputs as references).*

These evals guide prompt engineering, model selection, fine-tuning, and so on. We can then run these evals to quickly measure improvements or regressions as the app changes.

It's Test Driven Development (TDD) for LLM-backed apps.

## Open-source LLM-backed app evaluation products

| Name | Description|
| --- | --- |
 [Auto Evaluator](https://github.com/rlancemartin/auto-evaluator) | Evaluation tool for LLM QA chains |
 [DeepEval](https://github.com/confident-ai/deepeval) | Evaluation and Unit Testing for LLMs |
 [Evals](https://github.com/openai/evals) | A framework for evaluating LLMs and LLM systems |
 |[Phoenix](https://github.com/Arize-ai/phoenix) | Evaluate, troubleshoot, and fine tune your LLM in a notebook |
 [Ragas](https://github.com/explodinggradients/ragas ) | Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines |
 [Uptrain](https://github.com/uptrain-ai/uptrain) | Your open-source LLM evaluation toolkit |

## Paid LLM-backed app evaluation products

| Name | Distribution | Maturity | Self-service signup |
| --- | --- | --- | --- |
| [Freeplay](https://freeplay.ai/) | SaaS | Private Beta | No |
| [Patronus AI](https://www.patronus.ai/) | SaaS | Released | No |

## References

*- Definition adapted from _[Patterns for Building LLM-based Systems & Products](https://eugeneyan.com/writing/llm-patterns/#evals-to-measure-performance)_ by Eugene Yan.

awesome-eval-driven-development

awesome-eval-driven-development 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

awesome-eval-driven-development

awesome-eval-driven-development 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README