cs336_assignment-5

KO | EN

gitlite — search

#react #ai #llm #golang #go #android #machine-learning #deep-learning #macos #nodejs #windows #awesome #cli #ios

cs336_assignment-5

(★ 40)

CS336 作业 5 实现, 附加作业里面的 dpo/rlhf 也完成了, 消融实验分析也放在飞书文档里面了, 仅供参考

cs336_assignment-5 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# CS336 Spring 2025 Assignment 5: Alignment
experiments record doc: https://tdu8is5la6.feishu.cn/docx/CksDd8zrHomlARx56MxctVfUncf
## sft result
![alt text](https://raw.githubusercontent.com/heng380/cs336_assignment-5/main/image.png)

## ei result
![alt text](https://raw.githubusercontent.com/heng380/cs336_assignment-5/main/image-1.png)

## grpo 
### grpo lr tuning
![alt text](https://raw.githubusercontent.com/heng380/cs336_assignment-5/main/image-2.png)

### grpo vs reinforce_baseline vs reinforce
![alt text](https://raw.githubusercontent.com/heng380/cs336_assignment-5/main/image-3.png)

For a full description of the assignment, see the assignment handout at
[cs336_spring2025_assignment5_alignment.pdf](./cs336_spring2025_assignment5_alignment.pdf)

We include a supplemental (and completely optional) assignment on safety alignment, instruction tuning, and RLHF at [cs336_spring2025_assignment5_supplement_safety_rlhf.pdf](./cs336_spring2025_assignment5_supplement_safety_rlhf.pdf)

If you see any issues with the assignment handout or code, please feel free to
raise a GitHub issue or open a pull request with a fix.

## Setup

As in previous assignments, we use `uv` to manage dependencies.

1. Install all packages except `flash-attn`, then all packages (`flash-attn` is weird)
```
uv sync --no-install-package flash-attn
uv sync
```

2. Run unit tests:

``` sh
uv run pytest
```

Initially, all tests should fail with `NotImplementedError`s.
To connect your implementation to the tests, complete the
functions in [./tests/adapters.py](./tests/adapters.py).

cs336_assignment-5

cs336_assignment-5 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

cs336_assignment-5

cs336_assignment-5 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README