CSKB-Population

KO | EN

gitlite — search

#javascript #hacktoberfest #typescript #go #android #rust #docker #macos #windows #cli #vue #ios #mcp #open-source

CSKB-Population

(★ 26)

Codes for the EMNLP2021 paper: Benchmarking Commonsense Knowledge Base Population (https://aclanthology.org/2021.emnlp-main.705.pdf). An updated version CKBP v2 (https://arxiv.org/pdf/2304.10392.pdf)

CSKB-Population 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# CSKB-Population
Codes for the EMNLP2021 paper: [Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset](https://arxiv.org/abs/2109.07679)

We have an expert-annotated and adversarially constructed evaluation set [CKBP v2](https://arxiv.org/abs/2304.10392).

For data preprocessing and the codes released upon EMNLP 2021, checkout the branch `emnlp2021` (https://github.com/HKUST-KnowComp/CSKB-Population/tree/emnlp2021).

## Data

CKBP v1: [training data](https://github.com/HKUST-KnowComp/CSKB-Population/tree/emnlp2021), [evaluation set](https://github.com/HKUST-KnowComp/CSKB-Population/blob/main/data/evaluation_set.csv).

CKBP v2: [training data (updated)](https://drive.google.com/file/d/14imdF7yeKPf0_pNyj6Fbv_EhlvH3OXDx/view?usp=sharing), [evaluation data](https://github.com/HKUST-KnowComp/CSKB-Population/blob/main/ckbp_v2/ckbp2.0.csv)

## Environment

We tested our codes on `python 3.8.5`. Here are some packages their corresponding versions.

```
networkx                 2.5
numpy                    1.19.2
pandas                   1.2.2
scikit-learn             0.24.1
scipy                    1.6.3
sklearn                  0.0
spacy                    3.2.0
stanfordnlp              0.2.0
torch                    1.7.1
torch-geometric          1.7.2
torch-scatter            2.0.7
torch-sparse             0.6.9
torchsummary             1.5.1
torchtext                0.8.1
tqdm                     4.56.0
transformers             3.4.0
```

## Dataset Preprocess

See the branch [`emnlp2021`](https://github.com/HKUST-KnowComp/CSKB-Population/tree/emnlp2021)

## Model Training

### Download the data

```
kaggle datasets download -d tianqingfang/ckbp-emnlp2021
```

Or download at https://www.kaggle.com/datasets/tianqingfang/ckbp-emnlp2021.

Put the `train.csv` under `data/ckbp_csv/emnlp2021/`. The `dev.csv` and `test.csv` are automatically constructed dev and test sets, sampled from ASER and CSKBs (CSKB ground truth triples are labeled 1, negative sampled examples are labeled 0.)

The annotated evaluation set is at `data/evaluation_set.csv`. Please use this data set to test.

### Train w/ KG-BERT (BERT-base)

KG-BERT baseline is provided here.

```
CUDA_VISIBLE_DEVICES=0 python models/train_kgbert_baseline.py \
    --ptlm bert-base-uncased \
    --lr 1e-5 \
    --epochs 1 \
    --output_dir results \
    --train_csv_path data/ckbp_csv/emnlp2021/train.csv \
    --relation_as_special_token \
    --save_best_model \
    --seed 100 --batch_size 64 --test_batch_size 128 --save_best_model --experiment_name ""
```

### Access the whole graph data

Check out `Model Training` section on the branch [`emnlp2021`](https://github.com/HKUST-KnowComp/CSKB-Population/tree/emnlp2021#model-training).

## Results

(Note that the evaluation metric is a bit different from that in the paper. In the paper we use a grouped AUC where AUC scores within different groups are calculated and averaged. Here we report the overall AUC across all relations.)

|Model|All|Ori Test Set|CSKB head|ASER edge|
|:--:|:--:|:--:|:--:|:--:|
|KG-BERT (BERT-base) | 62.5 | 74.2 | 51.9 | 54.7|
|KG-BERT (RoBERTa-large) | 70.9 | 78.0 | 63.4 | 64.6|

## Reproduce DISCOS using current version of ASER\_norm

Checkout the [`DISCOS-reproduce`](https://github.com/HKUST-KnowComp/CSKB-Population/tree/emnlp2021/DISCOS-reproduce) folder under the branch [`emnlp2021`](https://github.com/HKUST-KnowComp/CSKB-Population/tree/emnlp2021).

CSKB-Population

CSKB-Population 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

CSKB-Population

CSKB-Population 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README