awesome-machine-psychology

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

awesome-machine-psychology

★ 15 Open GitHub ↗

A curated list of awesome papers, datasets, and models relevant to machine psychology.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Awesome Machine Psychology

[![Awesome](https://cdn.jsdelivr.net/gh/sindresorhus/awesome@d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)

🌟 **A curated collection of standout papers, datasets, and models in machine psychology**—the fascinating study of artificial intelligence (AI) systems, especially large language models (LLMs), using experimental and theoretical methods traditionally applied in human psychology.

🎓 This project was created at [OMNILab](https://github.com/OMNILab), Shanghai Jiao Tong University, by [Xiangtiange Li](https://github.com/tranquiltina), [Qiyuan Gu](https://github.com/phoeniiix1203), [Siyu Pan](https://github.com/lesleypan0327), and [Xinyue Zhang](https://github.com/Adazhangdada), under the guidance of Professor Yaohui Jin and Dr. Binglei Zhao. OMNILab is now a part of the [BaiYuLan Open AI community](https://github.com/Bai-Yu-Lan).

💡 We welcome contributions to this collection! Please review the [Contribution Guidelines](CONTRIBUTING.md) to make sure your entries fit the criteria.

## Table of Contents

- [awesome-machine-psychology](#awesome-machine-psychology)
    - [Papers](#papers)
        - [By Year](#by-year)
            - [2024](#2024)
            - [2023](#2023)
            - [2022](#2022)
        - [By Topic](#by-topic)
            - [Clinical Psychology](#clinical-psychology)
            - [Cognitive Psychology](#cognitive-psychology)
            - [Developmental Psychology](#developmental-psychology)
            - [Group Psychology](#group-psychology)
            - [Intelligence Assessment](#intelligence-assessment)
            - [Moral Psychology](#moral-psychology)
            - [Psychology of Creativity](#psychology-of-creativity)
            - [Psychology of Personality](#psychology-of-personality)
            - [Social Psychology](#social-psychology)
    - [Datasets](#datasets)
    - [Models](#models)

## Papers

Note: To keep paragraphs concise, we only include essential details when sorting papers by topic. Full information is provided when papers are sorted by year.

### By Year

#### 2024

**LLMs achieve adult human performance on higher-order theory of mind tasks**

- PDF: https://arxiv.org/abs/2405.18870
- Authors: Winnie Street, John Oliver Siy, Geoff Keeling, Adrien Baranes, Benjamin Barnett, Michael McKibben, Tatenda Kanyere, Alison Lentz, Blaise Aguera y Arcas, Robin I. M. Dunbar
- [x] Grouped by topic

**Machine Psychology: Investigating Emergent Capabilities and Behavior in Large Language Models Using Psychological Methods**

- PDF: https://arxiv.org/abs/2303.13988
- Authors: Thilo Hagendorff, Ishita Dasgupta, Marcel Binz, Stephanie C.Y. Chan, Andrew Lampinen, Jane X. Wang, Zeynep Akata, Eric Schulz
- Code: https://osf.io/w5vhp/
- [ ] Grouped by topic

**Testing theory of mind in large language models and humans**

- Published in: Nature Human Behavior
- PDF: https://www.nature.com/articles/s41562-024-01882-z
- Authors: James W. A. Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, Michael S. A. Graziano & Cristina Becchio
- [x] Grouped by topic

**Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View**

- Published in: ACL 2024
- PDF: https://arxiv.org/abs/2310.02124
- Authors: 
- Code: https://github.com/zjunlp/MachineSoM
- [x] Grouped by topic

**InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews**

- Published in: ACL 2024
- PDF: https://arxiv.org/abs/2310.17976
- Authors: 
- Code: https://github.com/neph0s/incharacter
- [x] Grouped by topic

**PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety**

- Published in: ACL 2024
- PDF: https://arxiv.org/abs/2401.11880
- Authors: 
- Code: https://github.com/AI4Good24/PsySafe
- [ ] Grouped by topic

**PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents**

- PDF: https://arxiv.org/abs/2402.12326
- Published in: ACL 2024
- Authors: 
- [x] Grouped by topic

**HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy**

- PDF: https://arxiv.org/abs/2403.05574
- Published in: ACL 2024
- Authors: 
- Code: https://github.com/elsa66666/healme
- [x] Grouped by topic

**CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling**

- Published in: ACL 2024
- PDF: https://arxiv.org/abs/2405.16433
- Authors: 
- Code: https://github.com/CAS-SIAT-XinHai/CPsyCoun
- [x] Grouped by topic

**Using Artificial Populations to Study Psychological Phenomena in Neural Models**

- Published in: AAAI 2024
- PDF: https://arxiv.org/abs/2308.08032
- Authors: 
- Code: https://github.com/JesseTNRoberts/Using-Artificial-Populations-to-Study-Psychological-Phenomena-in-Language-Models
- [ ] Grouped by topic

**Working Memory Capacity of ChatGPT: An Empirical Study**

- Published in: AAAI 2024
- PDF: https://doi.org/10.1609/aaai.v38i9.28868
- Authors: 
- Code: https://github.com/Daniel-Gong/ChatGPT-WM
- [x] Grouped by topic

**PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals**

- PDF: https://arxiv.org/abs/2405.19660
- Authors: Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M. Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Zoey Chen
- Code: https://github.com/ruiyiw/patient-psi
- [x] Grouped by topic

<!-- 
The Following are papers from 2024 that don't meet the citation criteria

**Regulating AI: Applying insights from behavioural economics and psychology to the application of article 5 of the EU AI Act**

- Published in: AAAI 2024
- PDF: https://arxiv.org/abs/2308.02041
- Authors: 
cited by 1

-->

#### 2023

**Playing repeated games with Large Language Models**

- PDF: https://arxiv.org/abs/2305.16867
- Authors: Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, Eric Schulz
- [x] Grouped by topic

**Inductive reasoning in humans and large language models**

- PDF: https://arxiv.org/abs/2306.06548
- Authors: Simon J. Han, Keith Ransom, Andrew Perfors, Charles Kemp
- [x] Grouped by topic

**Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?**

- PDF: https://arxiv.org/abs/2301.07543
- Authors: John J. Horton

**Deception abilities emerged in large language models**

- Published in: Proceedings of the National Academy of Sciences of the United States of America
- PDF: https://arxiv.org/abs/2307.16513
- Authors: Thilo Hagendorff

**Using cognitive psychology to understand GPT-3**

- Published in: Proceedings of the National Academy of Sciences. Vol. 120 | No. 6.
- PDF: https://www.pnas.org/doi/10.1073/pnas.2218523120
- Authors: Marcel Binz, Eric Schulz
- Code: https://github.com/marcelbinz/GPT3goesPsychology
- [x] Grouped by topic

**Inducing anxiety in large language models increases exploration and bias**

- PDF: https://arxiv.org/abs/2304.11111
- Authors: Julian Coda-Forno, Kristin Witte, Akshay K. Jagadish, Marcel Binz, Zeynep Akata, Eric Schulz
- [x] Grouped by topic

**Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT**

- Published in: Nature Computer Science
- PDF: https://www.nature.com/articles/s43588-023-00527-x
- Authors: Thilo Hagendorff, Sarah Fabi, Michal Kosinski
- [x] Grouped by topic

**A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do?**

- Published in: Social Science Research Network
- PDF: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4380365
- Authors: Yang Chen, Samuel Kirshner, Anton Ovchinnikov, Meena Andiappan, Tracy Jenkin
- [x] Grouped by topic

**Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks**

- PDF: https://arxiv.org/abs/2302.08399
- Authors: Tomer Ullman
- [x] Grouped by topic

**Sparks of Artificial General Intelligence: Early experiments with GPT-4**

- PDF: https://arxiv.org/abs/2303.12712
- Authors: Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
- [x] Grouped by topic

**Evaluating the Moral Beliefs Encoded in LLMs**

- Published in: NeurIPS 2023
- PDF: https://arxiv.org/abs/2307.14324
- Authors: Nino Scherrer, Claudia Shi, Amir Feder, David M. Blei
- [x] Grouped by topic

<!-- 
The Following are papers from 2023 that don't meet the citation criteria

**Detection of Multiple Mental Disorders from Social Media with Two-Stream Psychiatric Experts**

- Published in: EMNLP 2023
- PDF: https://aclanthology.org/2023.emnlp-main.562.pdf
- Authors: 
cited by 3

**Challenging the appearance of machine intelligence: Cognitive  bias in LLMs.**

- PDF: https://arxiv.org/abs/2304.01358
- Authors: Alaina N. Talboy, Elizabeth Fuller
cited by 19

-->

#### 2022

**Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies**

- Published in: ICML 2023
- PDF: https://arxiv.org/abs/2208.10264
- Authors: Gati Aher, Rosa I. Arriaga, Adam Tauman Kalai
- [ ] Grouped by topic

**Towards Reasoning in Large Language Models: A Survey**

- Published in: ACL 2023
- PDF: https://arxiv.org/abs/2212.10403
- Authors: Jie Huang, Kevin Chen-Chuan Chang
- [x] Grouped by topic

**Evaluating Psychological Safety of Large Language Models**

- PDF: https://arxiv.org/abs/2212.10529
- Authors: Xingxuan Li, Yutong Li, Lin Qiu, Shafiq Joty, Lidong Bing
- [x] Grouped by topic

**Language models show human-like content effects on reasoning tasks**

- PDF: https://arxiv.org/abs/2207.07051
- Authors: Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, Felix Hill
- [x] Grouped by topic

**Capturing Failures of Large Language Models via Human Cognitive Biases**

- Published in: NeurIPS 2022
- PDF: https://arxiv.org/abs/2202.12299
- Authors: Erik Jones, Jacob Steinhardt
- [x] Grouped by topic

**Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs**

- Published in: EMNLP 2022
- PDF: https://arxiv.org/abs/2210.13312
- Authors: Maarten Sap, Ronan LeBras, Daniel Fried, Yejin Choi
- [x] Grouped by topic

**Do Large Language Models know what humans know?**

- PDF: https://pubmed.ncbi.nlm.nih.gov/37401923/
- Authors: Sean Trott, Cameron Jones, Tyler Chang, James Michaelov, Benjamin Bergen
- Code: https://osf.io/hu865/
- [x] Grouped by topic

**Who is GPT-3? An Exploration of Personality, Values and Demographics**

- Published in: NLPCSS
- PDF: https://arxiv.org/abs/2209.14338
- Authors: Marilù Miotto, Nicola Rossberg, Bennett Kleinberg
- [x] Grouped by topic

**Emergent Analogical Reasoning in Large Language Models**

- Published in: Nature Human Behaviour
- PDF: https://arxiv.org/abs/2212.09196
- Authors: Taylor Webb, Keith J. Holyoak, Hongjing Lu
- [x] Grouped by topic

**Putting GPT-3's Creativity to the (Alternative Uses) Test**

- PDF: https://arxiv.org/abs/2206.08932
- Authors: Claire Stevenson, Iris Smal, Matthijs Baas, Raoul Grasman, Han van der Maas
- [x] Grouped by topic

**When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment**

- Published in: NeurIPS 2022
- PDF: https://arxiv.org/abs/2210.01478
- Authors: Zhijing Jin, Sydney Levine, Fernando Gonzalez, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, Bernhard Schölkopf
- [x] Grouped by topic

<!-- 
The Following are papers from 2022 that don't meet the citation criteria

-->

#### Clinical Psychology

HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy

CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling

PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents

Inducing anxiety in large language models increases exploration and bias

Evaluating Psychological Safety of Large Language Models

#### Cognitive Psychology

Using cognitive psychology to understand GPT-3

Towards Reasoning in Large Language Models: A Survey

Capturing Failures of Large Language Models via Human Cognitive Biases

Language models show human-like content effects on reasoning tasks

Working Memory Capacity of ChatGPT: An Empirical Study

Inductive reasoning in humans and large language models

Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT

A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do?

#### Developmental Psychology

LLMs achieve adult human performance on higher-order theory of mind tasks

Testing theory of mind in large language models and humans

Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks

Do Large Language Models know what humans know?

Sparks of Artificial General Intelligence: Early experiments with GPT-4

#### Group Psychology

Playing repeated games with Large Language Models

#### Intelligence Assessment

Emergent Analogical Reasoning in Large Language Models

#### Moral Psychology

Evaluating the Moral Beliefs Encoded in LLMs

When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment

#### Psychology of Creativity

Putting GPT-3's Creativity to the (Alternative Uses) Test

#### Psychology of Personality

InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews

Who is GPT-3? An Exploration of Personality, Values and Demographics

#### Social Psychology

Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View

## Datasets

Note: Most of the datasets listed below are free, however, some are not.

[HealMe](https://github.com/elsa66666/healme): Please refer to HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy

[InCharacter](https://github.com/Neph0s/InCharacter): Please refer to InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews

[PsySafe](https://github.com/AI4Good24/PsySafe): Please refer to PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

[CPsyCoun](https://github.com/CAS-SIAT-XinHai/CPsyCoun): Please refer to CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling

[ChatGPT-WM](https://github.com/Daniel-Gong/ChatGPT-WM): Please refer to Working Memory Capacity of ChatGPT: An Empirical Study

[Using Artificial Populations to Study Psychological Phenomena in Language Models](https://github.com/JesseTNRoberts/Using-Artificial-Populations-to-Study-Psychological-Phenomena-in-Language-Models)

[GPT3goesPsychology](https://github.com/marcelbinz/GPT3goesPsychology): Please refer to Using cognitive psychology to understand GPT-3

[Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT](https://osf.io/w5vhp/)

[Do Large Language Models Know What They Don’t Know?](https://osf.io/hu865/)

## Models

[HealMe](https://github.com/elsa66666/healme): Please refer to HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy

[Patient Psi](https://github.com/ruiyiw/patient-psi): Please refer to PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

[MachineSoM](https://github.com/zjunlp/MachineSoM): Please refer to Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View

[InCharacter](https://github.com/Neph0s/InCharacter): Please refer to InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews

[CPsyCoun](https://github.com/CAS-SIAT-XinHai/CPsyCoun): Please refer to CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling

[Using Artificial Populations to Study Psychological Phenomena in Language Models](https://github.com/JesseTNRoberts/Using-Artificial-Populations-to-Study-Psychological-Phenomena-in-Language-Models)

awesome-machine-psychology

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README