MoRT_NMI

KO | EN

gitlite — search

#typescript #ai-agents #ai #dsh-plugin #deepseek-harness #open-source #claude-code #codex #cli #developer-tools #react #windows

★ 18 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Pre-Trained Language Models Mirror Human-like Moral Norms
Code repository and Supplemental Material for the "Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do" 
publication.

## 1. Structure
The code is structured in:

* /MoRT: the moral choice machine and moral direction source code
* /LAMA: forked repo of https://github.com/facebookresearch/LAMA
* /realtoxicityprompts forked repo of https://github.com/allenai/real-toxicity-prompts
* /Supplemental_Material additional supplement (see below)

The Supplemental Material is divided in:

* /MoralScore: tables with results to reproduce the manuscript figures
* /TextGeneration: results of text generation. Read the contrained README.md for download links.
* /UserStudy: material of conducted user studies

## 2. Dependencies
    see MoRT/requirements.txt
    see LAMA/requirements.txt
    see realtoxicityprompts/environment.yml

## 3. Before you start 
* create an virtual enviroment and install the requirements
* download requiered data for LAMA (see README https://github.com/facebookresearch/LAMA) by executing LAMA/download_models.sh
* download testbed data from https://open.quiltdata.com/b/ai2-datasets/tree/realtoxicityprompts/ and place it in /real-toxicity-prompts/data/
* download text generation with the moral direction approach from https://hessenbox.tu-darmstadt.de/public?folderID=MjR2QVhvQmc0blFpdWd1YjViNHpz

## 4. Reproducing Results
Scripts and pipeline to reproduce results. Steps with (optional) are only required to reproduce data which is already contained in this repository.
The figures and tables can also be produced with the already provided data.

### LAMA
Create conda or virtual environment and install requirements (also see instruction of https://github.com/facebookresearch/LAMA)

cd LAMA
    python lama/eval_generation_moral.py --lm "bert"

### PCA MoralDirection (optional)
Create and save transformation
Create conda or virtual environment and install requirements

cd MoRT
    python mort/pca.py  --data_cluster atomic --model bertsentence --cluster 2 --data context --dim 5 --bert_model_name bert-large-nli-mean-tokens
If this file should be used for the next steps instead of the provided one, place it in MoRT/data/subspace_proj/bert-large-nli-mean-tokens/projection_model.p

### Compute Moral Scores (optional)
    cd MoRT
    export PYTHONPATH=.

Universal Sentence Encoder:

python mort/plot_corr/compute_mcm_cossim_scores.py --data userstudy --model use_hub
    python mort/plot_corr/compute_mcm_cossim_scores.py --data pos50 --model use_hub
    python mort/plot_corr/compute_mcm_cossim_scores.py --data neg50 --model use_hub
GloVe and BERT:

python mort/plot_corr/computeBERTScorePosNeg.py
    python mort/plot_corr/computeGloveScorePosNeg.py
    python mort/plot_corr/computeBERTAndGloVeScoreOfUserStudyActions.py

### Correlation between human and PLM's moral scores

python mort/plot_corr/plt_UserStudyCorr.py

By default the correlation is calculated with the regional study. 
To apply the global AMT study replace 'regional' with 'globalAMT' in line 181.

### Realtoxicityprompts text generation (optional)
Run scripts contained in realtoxicityprompts/scripts_mort

### Realtoxicityprompts Evaluation
Before you run the notebooks change the path to this repository.

- Run notebook for figures and result table: realtoxicityprompts/notebooks_mcm/realtoxicityprompts_results_moral.ipynb .

- Run notebook for inspecting selected examples: realtoxicityprompts/notebooks_mcm/example_generaions.ipynb .

## 5. Try MD
You can compute moral score of queries by running following script and typing your query, e.g. "Should I kill", "Should I kill time".

python mort/runMCM.py --model orig
Type stop to exit.

## 6. Citing & Authors

If you find this repository helpful, feel free to cite our publication: TODO

MoRT_NMI

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README