SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING

KO | EN

gitlite — search

#python #typescript #llm #golang #deep-learning #linux #awesome #kubernetes #security #pytorch #mcp #ios #openai #open-source

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING

(★ 58)

in this project, I've implemented the Facebook paper about fine tuning RoBERTa with contrastive loss.

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING"

in this code, I've implemented sentiment analysis task with sst-2 dataset.

the below results are for 100 training samples:

cross entropy loss:

![My Image](https://github.com/sl-93/SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING/blob/main/result/cross%20loss.png)

cross entropy + contrastive loss:

![My Image](https://github.com/sl-93/SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING/blob/main/result/cross%20%2B%20contrastive%20loss.png)

cross entropy heatmap on test dataset:

![My Image](https://github.com/sl-93/SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING/blob/main/result/cross%20heatmap.png)

Accuracy on test dataset: 90.13

cross entropy + contrastive loss heatmap:

![My Image](https://github.com/sl-93/SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING/blob/main/result/cross%20%2B%20contrastive%20heatmap.png)

Accuracy on test dataset: 92.20

paper:
https://arxiv.org/abs/2011.01403

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README