Behavior-Alignment

KO | EN

gitlite — search

#python #hacktoberfest #ai #llm #go #android #rust #linux #docker #macos #nodejs #cli #awesome-list #openai

Behavior-Alignment

(★ 69)

Repository for the paper:

Behavior-Alignment 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Repository for paper "Behavior Alignment: A New Perspective of Evaluating LLM-based
Conversational Recommendation Systems" published at The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval.

Update: Repo under meantainance, more content is to-be updated.

The checkpoint of Binary Classifier for computing intrinct Behavior Alignment can be found under huggingface model repos:
Original: `Dylan1999/Behavior_Alignment_Origin_Binary_Classifier`
Mixed-hard: `Dylan1999/Behavior_Alignment_MixedHard_Binary_Classifier`

There are 13 distinct behavior types in CRS defined in INSPIRED dataset. We replaced them to IDs, and the mapping table is:
```
    id2label = {
        0: "acknowledgment",
        1: "credibility",
        2: "encouragement",
        3: "experience_inquiry",
        4: "offer_help",
        5: "opinion_inquiry",
        6: "personal_experience",
        7: "personal_opinion",
        8: "preference_confirmation",
        9: "rephrase_preference",
        10: "self_modeling",
        11: "similarity",
        12: "transparency"  
    }

```

Under `evaluation` folder, you can find the code to output evaluation results under Section 5.2.

For OOD eval data, `Redial` dataset is considered as Out-of-distribution. Since our model is trained using data from INSPIRED dataset.

- `OOD_data_creation`: Code to create `redial_eval_dataset.json`
- `inference.py`: file to create two evaluation result `.csv` file under OOD_inference_result, which load the trained checkpoint and inference the predictions.

Behavior-Alignment

Behavior-Alignment 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Behavior-Alignment

Behavior-Alignment 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README