Awesome-LLM-in-Social-Science

KO | EN

gitlite — search

#javascript #python #typescript #go #deep-learning #linux #docker #nodejs #macos #windows #cli #pytorch #ios #open-source

Awesome-LLM-in-Social-Science

(★ 645)

Awesome papers involving LLMs in Social Science.

파일 탐색기

최종 버전 다운로드 (.zip)

LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Awesome-LLM-in-Social-Science

> **🔗 Recommended Resource:**  
> Check out [Awesome-LLM-Psychometrics](https://github.com/ValueByte-AI/Awesome-LLM-Psychometrics) for a comprehensive collection of papers and resources on LLM psychometrics, including evaluation, validation, and enhancement.  
>

Below we compile *awesome* papers that  
- **evaluate** Large Language Models (LLMs) from a perspective of Social Science.
- **align** LLMs from a perspective of Social Science.
- employ LLMs to **facilitate research, address issues, and enhance tools** in Social Science.
- contribute **surveys**, **perspectives**, and **datasets** on the above topics.

The above taxonomies are by no means orthogonal. For example, evaluations require simulations. We categorize these papers based on our understanding of their focus. This collection has **a special focus on Psychology and intrinsic values**.

Welcome to contribute and discuss!

---

🤩 Papers marked with a ⭐️ are contributed by the maintainers of this repository. If you find them useful, we would greatly appreciate it if you could give the repository a star and cite our paper.

```bibtex
@article{ye2025large,
  title={Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement},
  author={Ye, Haoran and Jin, Jing and Xie, Yuhang and Zhang, Xin and Song, Guojie},
  journal={arXiv preprint arXiv:2505.08245},
  year={2025},
  note={Project website: \url{https://llm-psychometrics.com}, GitHub: \url{https://github.com/ValueByte-AI/Awesome-LLM-Psychometrics}}
}
```

---

## Table of Contents

* 1. [📚 Survey](#Survey)
* 2. [🗂️ Dataset](#Dataset)
* 3. [🔎 Evaluating LLM](#EvaluatingLLM)
	* 3.1. [❤️ Value](#Value)
	* 3.2. [🩷 Personality](#Personality)
	* 3.3. [🔞 Morality](#Morality)
	* 3.4. [🎤 Opinion](#Opinion)
	* 3.5. [💚 General Preference](#GeneralPreference)
	* 3.6. [🧠 Ability](#Ability)
	* 3.7. [⚠️ Risk](#Risk) 
* 4. [⚒️ Tool enhancement](#Toolenhancement)
* 5. [⛑️ Alignment](#Alignment)
	* 5.1. [🌈 Pluralistic Alignment](#PluralisticAlignment)
* 6. [🚀 Simulation](#Simulation)
* 7. [👁️‍🗨️ Perspective and Position](#Perspective)

---

##  1. <a name='Survey'></a>📚 Survey 
- ⭐️ **Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement**, 2025.05, [[paper]](https://arxiv.org/abs/2505.08245).
- **Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs**, ACL 2025, [[paper]](https://aclanthology.org/2025.findings-acl.1246/), [[collection]](https://github.com/Indiiigo/LLM_rep_review).
- **Large language models (LLM) in computational social science: prospects, current state, and challenges**, 2025.03, Social Network Analysis and Mining, [[paper]](https://link.springer.com/article/10.1007/s13278-025-01428-9).
- **Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents**, 2025.03, [[paper]](https://arxiv.org/abs/2503.24047).
- **On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective**, 2025.02, [[paper]](https://arxiv.org/abs/2502.14296).
- **The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment**, 2024.12, [[paper]](https://arxiv.org/abs/2412.16468).
- **Large Language Model Safety: A Holistic Survey**, 2024.12, [[paper]](https://arxiv.org/abs/2412.17686).
- **Political-LLM: Large Language Models in Political Science**, 2024.12, [[paper]](https://arxiv.org/abs/2412.06864), [[website]](https://political-llm.org/).
- **LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods**, 2024.12, [[paper]](https://arxiv.org/abs/2412.05579).
- **From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents**, 2024.12, [[paper]](https://arxiv.org/abs/2412.03563), [[repo]](https://github.com/FudanDISC/SocialAgent).
- **A Survey on Human-Centric LLMs**, 2024.11, [[paper]](https://arxiv.org/abs/2411.14491).
- **Survey of Cultural Awareness in Language Models: Text and Beyond**, 2024.11, [[paper]](https://arxiv.org/pdf/2411.00860).
- **How developments in natural language processing help us in understanding human behaviour**, 2024.10 Nature Human Behavior, [[paper]](https://www.nature.com/articles/s41562-024-01938-0.pdf).
- **How large language models can reshape collective intelligence**, 2024.09, Nature Human Behavior, [[paper]](https://www.nature.com/articles/s41562-024-01959-9).
- **Automated Mining of Structured Knowledge from Text in the Era of Large Language Models**, 2024.08, KDD 2024, [[paper]](https://dl.acm.org/doi/pdf/10.1145/3637528.3671469).
- **Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective**, 2024.07, [[paper]](https://arxiv.org/abs/2408.04638).
- **Perils and opportunities in using large language models in psychological research**, 2024.07, [[paper]](https://academic.oup.com/pnasnexus/article/3/7/pgae245/7712371).
- **The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models**, 2024.06, [[paper]](https://arxiv.org/abs/2406.11096).
- **Can Generative AI improve social science?**, 2024.05, PNAS, [[paper]](https://www.pnas.org/doi/pdf/10.1073/pnas.2314021121).
- **Foundational Challenges in Assuring Alignment and Safety of Large Language Models**, 2024.04, [[paper]](https://arxiv.org/abs/2404.09932).
- **Large Language Model based Multi-Agents: A Survey of Progress and Challenges**, 2024.01, [[paper]](https://arxiv.org/abs/2402.01680), [[repo]](https://github.com/taichengguo/LLM_MultiAgents_Survey_Papers).
- **The Rise and Potential of Large Language Model Based Agents: A Survey**, 2023, [[paper]](https://arxiv.org/abs/2309.07864), [[repo]](https://github.com/WooooDyy/LLM-Agent-Paper-List).
- **A Survey on Large Language Model based Autonomous Agents**, 2023, [[paper]](https://arxiv.org/abs/2308.11432), [[repo]](https://github.com/Paitesanshi/LLM-Agent-Survey).
- **AI Alignment: A Comprehensive Survey**, 2023.11, [[paper]](https://arxiv.org/abs/2310.19852), [[website]](https://alignmentsurvey.com/).
- **Aligning Large Language Models with Human: A Survey**, 2023, [[paper]](https://arxiv.org/abs/2307.12966), [[repo]](https://github.com/GaryYufei/AlignLLMHumanSurvey).
- **Large Language Model Alignment: A Survey**, 2023, [[paper]](https://arxiv.org/abs/2309.15025).
- **Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives**, 2023.12, Nature humanities and social sciences communications, [[paper]](https://arxiv.org/abs/2312.11970).
- **A Survey on Evaluation of Large Language Models**, 2023.07, [[paper]](https://arxiv.org/abs/2307.03109), [[repo]](https://github.com/MLGroupJLU/LLM-eval-survey).
- **From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models**, 2023.08, [[paper]](https://arxiv.org/abs/2308.12014), [[repo]](https://github.com/ValueCompass/Alignment-Goal-Survey).
- **Concerns on the use of generative AI in social science research**, [[website]](https://uh-dcm.github.io/genai-concerns/), [[repo]](https://github.com/uh-dcm/genai-concerns/)

##  2. <a name='Dataset'></a>🗂️ Dataset
- **[AI Job Displacement Tracker](https://github.com/noahaust2/ai-displacement-tracker)** — Structured, source-backed dataset tracking 96 AI-attributed workforce reductions (457K workers affected, 13 countries, 13 sectors). Every entry includes source URLs, attribution tier, and job functions. Available in JSON/CSV under CC-BY-4.0.
- **[Mental Health Datasets](https://github.com/kharrigian/mental-health-datasets)**
- **[Datasets for depression detection using data posted on online platforms](https://github.com/bucuram/depression-datasets-nlp)**
- **Benchmarking Multi-National Value Alignment for Large Language Models**, 2025.04, [[paper]](https://arxiv.org/abs/2504.12911).
- **COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values**, 2025.04, [[paper]](https://arxiv.org/abs/2504.05535).
- **Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?**, 2025.01, [[paper]](https://arxiv.org/abs/2501.15463).
- **The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models**, NeurIPS 2024 D&B Track best paper, [[paper]](https://arxiv.org/abs/2404.16019).
- ⭐️ **ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models**, ACL 2024, [[paper]](https://arxiv.org/abs/2406.04214), [[code]](https://github.com/Value4AI/ValueBench).
- **Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models**, NeurIPS 2024, [[paper]](https://arxiv.org/abs/2402.11894).
- https://github.com/CLUEbenchmark/CLUEDatasetSearch
- **HATEDAY: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter**, ACL 2025 Outstanding Paper, [[paper]](https://arxiv.org/abs/2411.15462).
- https://lit.eecs.umich.edu/downloads.html
- **COMPO: Community Preferences for Language Model Personalization**, 2024.10, [[paper]](https://arxiv.org/pdf/2410.16027).
- **Cultural Commonsense Knowledge for Intercultural Dialogues**, CIKM 2024, [[paper]](https://dl.acm.org/doi/pdf/10.1145/3627673.3679768), [[dataset]](https://mango.mpi-inf.mpg.de/).

##  3. <a name='EvaluatingLLM'></a>🔎 Evaluating LLM

###  3.1. <a name='Value'></a>❤️ Value
- **A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptive**, ACL 2025 Best Paper, [[paper]](https://arxiv.org/abs/2402.11005).
- **Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs**, ACL 2025 Best Paper, [[paper]](https://arxiv.org/abs/2502.01926).
- **Towards Better Value Principles for Large Language Model Alignment: A Systematic Evaluation and Enhancement**, ACL 2025 SAC Highlight, [[paper]](https://aclanthology.org/2025.acl-long.1408/).
- ⭐️ **Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models**, 2025.02, [[paper]](https://arxiv.org/abs/2502.02444).
- ⭐️ **Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models**, AAAI 2025, [[paper]](https://arxiv.org/abs/2409.12106), [[code]](https://github.com/Value4AI/gpv).
- ⭐️ **ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models**, ACL 2024, [[paper]](https://arxiv.org/abs/2406.04214), [[code]](https://github.com/Value4AI/ValueBench).
- **Value Profiles for Encoding Human Variation**, 2025.03, [[paper]](https://arxiv.org/abs/2503.15484).
- **Simulating Human-like Daily Activities with Desire-driven Autonomy**, ICLR 2025, [[paper]](https://openreview.net/forum?id=3ms8EQY7f8), [[arXiv]](https://arxiv.org/abs/2412.06435), [[code]](https://github.com/zfw1226/D2A).
- **Beyond Self-Interest: Modeling Social-Oriented Motivation for Human-like Multi-Agent Interactions**, AAMAS 2026 Oral, [[paper]](https://arxiv.org/abs/2603.13890), [[code]](https://github.com/jingzhe-lin/ASVO).
- **Value Compass Leaderboard: A Platform for Fundamental and Validated Evaluation of LLMs Values**, 2025.01, [[paper]](https://arxiv.org/abs/2501.07071).
- **Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective**, ACL 2025 Findings, [[paper]](https://aclanthology.org/2025.findings-acl.1188/), [[arXiv]](https://arxiv.org/abs/2501.00581).
- **NORMAD: A Framework for Measuring the Cultural Adaptability of Large Language Models**, 2024.10, [[paper]](https://arxiv.org/abs/2404.12464).
- **ValueCompass: A Framework of Fundamental Values for Human-AI Alignment**, 2024.09, [[paper]](https://arxiv.org/abs/2409.09586).
- **LOCALVALUEBENCH: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models**, 2024.08, [[paper]](https://arxiv.org/abs/2408.01460).
- **Stick to your role! Stability of personal values expressed in large language models**, 2024.08, [[paper]](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0309114).
- **Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing**, 2024.07, [[paper]](https://arxiv.org/abs/2406.14230).
- **Do LLMs have Consistent Values?**, 2024.07, [[paper]](https://arxiv.org/abs/2407.12878).
- **CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses**, 2024.07, [[paper]](https://arxiv.org/abs/2407.10725).
- **Are Large Language Models Consistent over Value-laden Questions?**, 2024.07, [[paper]](https://arxiv.org/abs/2407.02996).
- **Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches**, 2024.04, [[paper]](https://arxiv.org/abs/2404.12744).
- **Heterogeneous Value Evaluation for Large Language Models**, 2023.03, [[paper]](https://arxiv.org/abs/2305.17147), [[code]](https://github.com/zowiezhang/A2EHV).
- **Measuring Value Understanding in Language Models through Discriminator-Critique Gap**, 2023.10, [[paper]](https://arxiv.org/abs/2310.00378).
- **Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values**, 2023.11, [[paper]](https://arxiv.org/abs/2311.10766).
- **Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties**, AAAI24, [[paper]](https://arxiv.org/abs/2309.00779), [[code]](https://github.com/tsor13/kaleido).
- **High-Dimension Human Value Representation in Large Language Models**, 2024.04, [[paper]](https://arxiv.org/abs/2404.07900), [[code]](https://github.com/HLTCHKUST/UniVaR).

###  3.2. <a name='Personality'></a>🩷 Personality
- **A psychometric framework for evaluating and shaping personality traits in large language models**, Nature Machine Intelligence, 2025, [[paper]](https://www.nature.com/articles/s42256-025-01115-6).
- **R.U.Psycho? Robust Unified Psychometric Testing of Language Models**, 2025.03, [[paper]](https://arxiv.org/abs/2503.10229).
- **Evaluating the ability of large language models to emulate personality**, 2025.01, Nature Scientific Reports, [[paper]](https://www.nature.com/articles/s41598-024-84109-5).
- **Evaluating the efficacy of LLMs to emulate realistic human personalities**, AAAI 2024, [[paper]](https://ojs.aaai.org/index.php/AIIDE/article/view/31867).
- **Quantifying ai psychology: A psychometrics benchmark for large language models**, 2024.07, [[paper]](https://arxiv.org/abs/2406.17675).

- **Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews**, ACL 2024, [[paper]](https://aclanthology.org/2024.acl-long.102/), [[code]](https://github.com/Neph0s/InCharacter)

- [*MBTI*] **Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models**, 2024.01, [[paper]](https://arxiv.org/abs/2401.07115)

- **Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench**, ICLR 2024, [[paper]](https://arxiv.org/abs/2310.01386), [[code]](https://github.com/CUHK-ARISE/PsychoBench)

- [*BFI*] **AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories**, Journal, 2024.01, [[paper]](https://journals.sagepub.com/doi/full/10.1177/17456916231214460)

- **Does Role-Playing Chatbots Capture the Character Personalities? Assessing Personality Traits for Role-Playing Chatbots**, 2023.10, [[paper]](https://arxiv.org/abs/2310.17976)

- [*MBTI*] **Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models**, 2023.07, [[paper]](https://arxiv.org/abs/2307.16180)

- [*MBTI*] **Can ChatGPT Assess Human Personalities? A General Evaluation Framework**, 2023.03, EMNLP 2023, [[paper]](https://arxiv.org/abs/2303.01248), [[code]](https://github.com/Kali-Hac/ChatGPT-MBTI).

- [*BFI*] **Personality Traits in Large Language Models**, 2023.07, [[paper]](https://arxiv.org/abs/2307.00184)

- [*BFI*] **Revisiting the Reliability of Psychological Scales on Large Language Models**, 2023.05, [[paper]](https://arxiv.org/abs/2305.19926)

- [*BFI*] **Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimation**, ACL 2023 workshop, [[paper]](https://arxiv.org/abs/2306.01183)

- [*BFI*] **Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs**, 2023.05, [[paper]](https://arxiv.org/abs/2305.14693)

- [*BFI*] **Evaluating and Inducing Personality in Pre-trained Language Models**, NeurIPS 2023 (spotlight), [[paper]](https://arxiv.org/abs/2206.07550)

- [*BFI*] **Identifying and Manipulating the Personality Traits of Language Models**, 2022,12, [[paper]](https://arxiv.org/abs/2212.10276)

- **Who is GPT-3? An Exploration of Personality, Values and Demographics**, 2022.09, [[paper]](https://arxiv.org/abs/2209.14338)

- **Does GPT-3 Demonstrate Psychopathy? Evaluating Large Language Models from a Psychological Perspective**, 2022.12, [[paper]](https://arxiv.org/abs/2212.10529)

###  3.3. <a name='Morality'></a>🔞 Morality
- **Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral**, ACL 2025 Best Resource Paper, [[paper]](https://arxiv.org/abs/2502.14083).
- **How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation**, ACL 2026 Main, [[paper]](https://arxiv.org/abs/2603.13876), [[code]](https://github.com/MoralAgentSim/RoleModel-Moral-Sim).
- **Why Are We Moral? An LLM-based Agent Simulation Approach to Study Moral Evolution**, ACL 2026 Main (Oral), [[paper]](https://arxiv.org/abs/2509.17703), [[code]](https://github.com/MoralAgentSim/Simulation-Engine).
- **Revealing the Pragmatic Dilemma for Moral Reasoning Acquisition in Language Models**, 2025.02, [[paper]](https://arxiv.org/abs/2502.16600).
- **Normative Evaluation of Large Language Models with Everyday Moral Dilemmas**, 2025.02, [[paper]](https://arxiv.org/abs/2501.18081).
- **Investigating machine moral judgement through the Delphi experiment**, Nature Machine Intelligence, 2025.01, [[paper]](https://www.nature.com/articles/s42256-024-00969-6).
- **The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions**, 2026.08, [[paper]](https://arxiv.org/abs/2608.05583).
- **Moral Foundations of Large Language Models**, 2023.10, [[paper]](https://arxiv.org/abs/2310.15337).
- **Exploring the psychology of GPT-4's Moral and Legal Reasoning**, 2023.08, [[paper]](https://arxiv.org/abs/2308.01264).
- **Probing the Moral Development of Large Language Models through Defining Issues Test**
- **Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity**, 2023.06, [[paper]](https://arxiv.org/abs/2209.12106)
- **Evaluating the Moral Beliefs Encoded in LLMs**, 2023.07, [[paper]](https://arxiv.org/abs/2307.14324)

###  3.4. <a name='Opinion'></a>🎤 Opinion
- **The AI Gap: How Socioeconomic Status Affects Language Technology Interactions**, ACL 2025 Best Social Impact Paper, [[paper]](https://arxiv.org/abs/2505.12158).
- **Persuading voters using human–artificial intelligence dialogues**, Nature, 2025.12, [[paper]](https://www.nature.com/articles/s41586-025-09771-9).
- **Generative language models exhibit social identity biases**, Nature Computational Science, 2025.01, [[paper]](https://www.nature.com/articles/s43588-024-00741-1).
- **Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models**, ACL 2025, [[paper]](https://aclanthology.org/2025.acl-long.1529.pdf).
- **Generative AI Meets Open-Ended Survey Responses: Research Participant Use of AI and Homogenization**, 2025.05, Sociological Methods & Research, [[paper]](https://journals.sagepub.com/doi/10.1177/00491241251327130).
- **Machine Bias. How Do Generative Language Models Answer Opinion Polls?**, 2025.04, Sociological Methods & Research, [[paper]](https://doi.org/10.1177/00491241251330582).
- **Large language models that replace human participants can harmfully misportray and flatten identity groups**, Nature Machine Intelligence, 2025, [[paper]](https://www.nature.com/articles/s42256-025-00986-z).
- **Questioning the Survey Responses of Large Language Models**, NeurIPS 2024 Oral, [[paper]](https://arxiv.org/abs/2306.07951).
- **Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models**, ACL 2024, [paper](https://arxiv.org/abs/2402.16786).
- **More human than human: measuring ChatGPT political bias**, 2023, [[paper]](https://link.springer.com/article/10.1007/s11127-023-01097-2).
- **Towards Measuring the Representation of Subjective Global Opinions in Language Models**, 2023.07, [[paper]](https://arxiv.org/abs/2306.16388), [[website]](https://llmglobalvalues.anthropic.com/).

###  3.5. <a name='GeneralPreference'></a>💚 General Preference

- **Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)**, NeurIPS 2025 D&B Track Best Paper, [[paper]](https://arxiv.org/abs/2510.22954).
- **Diverging Preferences: When do Annotators Disagree and do Models Know?**, 2024.10, [[paper]](https://arxiv.org/abs/2410.14632).

###  3.6. <a name='Ability'></a>🧠 Ability 
- **Large language models surpass human experts in predicting neuroscience results**, Nature Human Behaviour, 2025.02, [[paper]](https://www.nature.com/articles/s41562-024-02046-9).
- **Quantifying large language model usage in scientific papers**, Nature Human Behaviour, 2025, [[paper]](https://www.nature.com/articles/s41562-025-02273-8).
- **Large language models without grounding recover non-sensorimotor but not sensorimotor features of human concepts**, Nature Human Behaviour, 2025, [[paper]](https://www.nature.com/articles/s41562-025-02203-8).
- **Visual cognition in multimodal large language models**, Nature Machine Intelligence, 2025.01, [[paper]](https://www.nature.com/articles/s42256-024-00963-y).
- **Testing theory of mind in large language models and humans**, Nature Human Behaviour, 2024.07, [[paper]](https://www.nature.com/articles/s41562-024-01882-z).
- **Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation**, 2025, [[paper]](https://arxiv.org/abs/2509.08825), [[post]](https://x.com/joabaum/status/1966454537922793979).
- **Improving LLM Leaderboards with Psychometrical Methodology**, 2025.01, [[paper]](https://arxiv.org/abs/2501.17200).
- **AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios**, NAACL 2025, [[paper]](https://arxiv.org/abs/2410.19346).
- **A Novel Psychometrics-Based Approach to Developing Professional Competency Benchmark for Large Language Models**, 2024.11, [[paper]](https://arxiv.org/abs/2411.00045).
- **Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates**, NeurIPS 2024, [[paper]](https://arxiv.org/abs/2410.07137).
- **Replication for Language Models: Problems, Principles, and Best Practice for Political Science**, 2024.10, [[paper]](https://arthurspirling.org/documents/BarriePalmerSpirling_TrustMeBro.pdf).
- **Can Language Models Reason about Individualistic Human Values and Preferences?**, 2024.10, [[paper]](https://arxiv.org/abs/2410.03868).
- **Language Models in Sociological Research: An Application to Classifying Large Administrative Data and Measuring Religiosity**, 2021, [[paper]](https://journals.sagepub.com/doi/full/10.1177/00811750211053370).
- **Can Large Language Models Transform Computational Social Science?**, 2023, [[paper]](https://arxiv.org/abs/2305.03514), [[code]](https://github.com/SALT-NLP/LLMs_for_CSS).
- **SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents**, 2023, [[paper]](https://arxiv.org/pdf/2310.11667.pdf), [[code]](https://www.sotopia.world/).
- **Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View**, 2023, [[paper]](https://arxiv.org/abs/2310.02124), [[code]](https://github.com/zjunlp/MachineSoM).
- **Playing repeated games with large language models**, Nature Human Behaviour, 2025.05, [[paper]](https://www.nature.com/articles/s41562-025-02172-y), [[arXiv]](https://arxiv.org/abs/2305.16867).
- **Machine Psychology: Investigating Emergent Capabilities and Behavior in Large Language Models Using Psychological Methods**, 2023, [[paper]](https://arxiv.org/abs/2303.13988).
- **Using cognitive psychology to understand GPT-3**, 2023.02, PNAS, [[paper]](https://www.pnas.org/doi/full/10.1073/pnas.2218523120?doi=10.1073%2Fpnas.2218523120).
- **Large language models as a substitute for human experts in annotating political text**, 2024.02, [[paper]](https://journals.sagepub.com/doi/10.1177/20531680241236239).

### 3.7. <a name='Risk'></a>❤️ Risk and Safety
- **From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents**, 2024.12, [[paper]](https://arxiv.org/abs/2412.07951).
- **Exploring Social Desirability Response Bias in Large Language Models: Evidence from GPT-4 Simulations**, 2024.10, [[paper]](https://arxiv.org/abs/2410.15442).
- **AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies**, 2024.07, [[paper]](https://arxiv.org/abs/2407.17436).

##  4. <a name='Toolenhancement'></a>⚒️ Tool enhancement
- **Past Meets Present: Creating Historical Analogy with Large Language Models**, ACL 2025 Outstanding Paper, [[paper]](https://arxiv.org/abs/2409.14820), [[code]](https://github.com/Nianqi-Li/Historical-Analogy-of-LLMs).
- **A reporting checklist for large language models in behavioural science**, Nature Human Behaviour, 2026, [[paper]](https://www.nature.com/articles/s41562-026-02492-7).
- **Durably reducing conspiracy beliefs through dialogues with AI**, Science, 2024.09, [[paper]](https://www.science.org/doi/10.1126/science.adq1814).
- ⭐️ **Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models**, 2025.02, [[paper]](https://arxiv.org/abs/2502.02444).
- ⭐️ **Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models**, AAAI 2025, [[paper]](https://arxiv.org/abs/2409.12106), [[code]](https://github.com/Value4AI/gpv).
- **STAMPsy: Towards SpatioTemporal-Aware Mixed-Type Dialogues for Psychological Counseling**, AAAI 2025, [[paper]](https://arxiv.org/abs/2412.16674).
- **Psychogat: A novel psychological measurement paradigm through interactive fiction games with llm agents**, ACL 2024, [[paper]](https://arxiv.org/abs/2402.12326).
- **ValueScope: Unveiling Implicit Norms and Values via Return Potential Model of Social Interactions**, 2024.10, [[paper]](https://arxiv.org/abs/2407.02472).
- **AI can help humans find common ground in democratic deliberation**, 2024.10, Science, [[paper]](https://www.science.org/doi/10.1126/science.adq2852).
- **PsyDI: Towards a Personalized and Progressively In-depth Chatbot for Psychological Measurements**, 2024, [[paper]](https://arxiv.org/abs/2408.03337), [[code]](https://github.com/opendilab/PsyDI).
- **ChatFive: Enhancing User Experience in Likert Scale Personality Test through Interactive Conversation with LLM Agents**, CUI 2024, [[paper]](https://dl.acm.org/doi/abs/10.1145/3640794.3665572)
- **LLM Agents for Psychology: A Study on Gamified Assessments**, 2024.02, [[paper]](https://arxiv.org/abs/2402.12326).
- **Generative Social Choice**, 2023.09, [[paper]](https://arxiv.org/abs/2309.01291)
- **Leveraging large language models to map triggers of contamination-related obsessive-compulsive symptoms**, Communications Psychology, 2026.08, [[paper]](https://www.nature.com/articles/s44271-026-00503-x).

##  5. <a name='Alignment'></a>⛑️ Alignment
- **On the Alignment of Large Language Models with Global Human Opinion**, AAAI 2026 Best Paper (AI Alignment Track), [[paper]](https://arxiv.org/abs/2509.01418).
- **Language Models Resist Alignment: Evidence From Data Compression**, ACL 2025 Best Paper, [[paper]](https://arxiv.org/abs/2406.06144).
- **Simple Role Assignment is Extraordinarily Effective for Safety Alignment**, ACL 2026 Findings, [[paper]](https://arxiv.org/abs/2602.00061).
- **Communication-Efficient Desire Alignment for Proactive Embodied Human–Agent Interaction**, ACL 2026 Main (Oral), [[paper]](https://arxiv.org/abs/2505.22503).
- **Implicit Behavioral Alignment of Language Agents in High-Stakes Crowd Simulations**, EMNLP 2025, [[paper]](https://aclanthology.org/2025.emnlp-main.1562/).
- **COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values**, 2025.04, [[paper]](https://arxiv.org/abs/2504.05535).
- ⭐️ **Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models**, 2025.02, [[paper]](https://arxiv.org/abs/2502.02444).
- **Optimizing generative AI by backpropagating language model feedback**, Nature, 2025.03, [[paper]](https://www.nature.com/articles/s41586-025-08661-4).
- **Position: We Need An Adaptive Interpretation of Helpful, Honest, and Harmless Principles**, 2025.02, [[paper]](https://arxiv.org/abs/2502.06059).
- **Scopes of Alignment**, 2025.01, AAAI 2025 workshop, [[paper]](https://arxiv.org/abs/2501.12405)
- **Aligning Large Language Models with Human Opinions through Persona Selection and Value-Belief-Norm Reasoning**, 2024.11, [[paper]](https://arxiv.org/pdf/2311.08385).
- **SafetyAnalyst: Interpretable, transparent, and steerable LLM safety moderation**, 2024.10, [[paper]](https://arxiv.org/abs/2410.16665).
- **Moral Alignment for LLM Agents**, 2024.10, [[paper]](https://arxiv.org/abs/2410.01639).
- **ProgressGym: Alignment with a Millennium of Moral Progress**, NeurIPS 2024 D&B Track Spotlight, [[paper]](https://arxiv.org/abs/2406.20087), [[code]](https://github.com/PKU-Alignment/ProgressGym).
- **Strong and weak alignment of large language models with human values**, 2024.08, Nature Scientific Reports, [[paper]](https://www.nature.com/articles/s41598-024-70031-3).
- **STELA: a community-centred approach to norm elicitation for AI alignment**, 2024.03, Nature Scientific Reports, [[paper]](https://www.nature.com/articles/s41598-024-56648-4).
- **A Roadmap to Pluralistic Alignment**, ICML 2024, [[paper]](https://arxiv.org/abs/2402.05070), [[code]](https://github.com/jfisher52/AI_Pluralistic_Alignment).
- [*Value*] **What are human values, and how do we align AI to them?**, 2024.04, [[paper]](https://arxiv.org/abs/2404.10636).
- **Agent Alignment in Evolving Social Norms**, 2024.01, [[paper]](https://arxiv.org/abs/2401.04620).
- [*Norm*] **Align on the Fly: Adapting Chatbot Behavior to Established Norms**, 2023.12, [[paper]](https://arxiv.org/abs/2312.15907), [[code]](https://github.com/GAIR-NLP/OPO).
- [*MBTI*] **Machine Mindset: An MBTI Exploration of Large Language Models**, 2023.12, [[paper]](https://arxiv.org/abs/2312.12999), [[code]](https://github.com/PKU-YuanGroup/Machine-Mindset).
- **Training Socially Aligned Language Models in Simulated Human Society**, 2023, [[paper]](https://arxiv.org/abs/2305.16960), [[code]](https://github.com/agi-templar/Stable-Alignment).
- **Fine-tuning language models to find agreement among humans with diverse preferences**, 2022, [[paper]](https://arxiv.org/abs/2211.15006).
- **ValueNet: A New Dataset for Human Value Driven Dialogue System**, AAAI 2022, [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/21368), [[dataset]](https://liang-qiu.github.io/ValueNet/).
- **Towards Pluralistic Value Alignment: Aggregating Value Systems through ℓp-Regression**, AAMAS 2022 workshop, [[paper]](https://openaccess.city.ac.uk/id/eprint/31381/).

###  5.1. <a name='PluralisticAlignment'></a>🌈 Pluralistic Alignment
- [*Benchmark*] **Benchmarking Distributional Alignment of Large Language Models**, 2024.11, [[paper]](https://arxiv.org/abs/2411.05403).
- **Legal Theory for Pluralistic Alignment**, 2024.10, [[paper]](https://arxiv.org/abs/2410.17271).
- **Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models**, 2024.10, [[paper]](https://arxiv.org/abs/2410.12880), [[code and data]](https://github.com/NeuralSentinel/CulturalKaleidoscope).
- **PAD: Personalized Alignment at Decoding-Time**, 2024.10, [[paper]](https://arxiv.org/abs/2410.04070).
- **Policy Prototyping for LLMs: Pluralistic Alignment via Interactive and Collaborative Policymaking**, 2024.09, [[paper]](https://arxiv.org/abs/2409.08622).
- **Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration**, 2024.06, [[paper]](https://arxiv.org/abs/2406.15951).
- **Self-Pluralising Culture Alignment for Large Language Models**, 2024.10, [[paper]](https://arxiv.org/abs/2410.12971).
- **Cultural Palette: Pluralising Culture Alignment via Multi-agent Palette**, 2024.12, [[paper]](https://arxiv.org/abs/2412.11167).
- **PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs**, 2026.08, [[paper]](https://arxiv.org/abs/2608.01458).

##  6. <a name='Simulation'></a>🚀 Simulation
- **CityReal: Human-Aligned Urban Behavior and City Dynamics Simulation with Large-Scale LLM Agents**, arXiv, 2026.07, [[paper]](https://arxiv.org/abs/2608.16897).
- **Large language models can predict the results of social science experiments**, Nature, 2026.07, [[paper]](https://www.nature.com/articles/s41586-026-10742-x).
- **A foundation model to predict and capture human cognition**, Nature, 2025.07, [[paper]](https://www.nature.com/articles/s41586-025-09215-4).
- **Frame Entrepreneurs in an AI Agent Community: Concentrated Identity-Claim Production on Moltbook**, ASA CITAMS Roundtable 2026, [[paper]](https://arxiv.org/abs/2604.27271).
- **Beyond Self-Interest: Modeling Social-Oriented Motivation for Human-like Multi-Agent Interactions**, AAMAS 2026 Oral, [[paper]](https://arxiv.org/abs/2603.13890), [[code]](https://github.com/jingzhe-lin/ASVO).
- **How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation**, ACL 2026 Main, [[paper]](https://arxiv.org/abs/2603.13876), [[code]](https://github.com/MoralAgentSim/RoleModel-Moral-Sim).
- **EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation**, ICML 2026, [[paper]](https://edumirror.net/assets/EduMirror.pdf), [[website]](https://edumirror.net/).
- **Why Are We Moral? An LLM-based Agent Simulation Approach to Study Moral Evolution**, ACL 2026 Main (Oral), [[paper]](https://arxiv.org/abs/2509.17703), [[code]](https://github.com/MoralAgentSim/Simulation-Engine).
- **TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets**, 2025.02, [[paper]](https://arxiv.org/abs/2502.01506).
- **AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society**, 2025.02, [[paper]](https://arxiv.org/abs/2502.08691), [[code]](https://github.com/tsinghua-fib-lab/agentsociety/).
- **Political Actor Agent: Simulating Legislative System for Roll Call Votes Prediction with Large Language Models**, AAAI 2025, [[paper]](https://arxiv.org/abs/2412.07144).
- **TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System**, 2024.12, [[paper]](https://arxiv.org/abs/2412.12196).
- **Can Large Language Model Agents Simulate Human Trust Behavior?**, NeurIPS 2024, [[paper]](https://arxiv.org/abs/2402.04559).
- **OASIS: Open Agents Social Interaction Simulations on One Million Agents**, 2024.11, [[paper]](https://arxiv.org/abs/2411.11581), [[code]](https://github.com/camel-ai/oasis).
- **Generative Agent Simulations of 1,000 People**, 2024.11, [[paper]](https://arxiv.org/abs/2411.10109).
- **Social Science Meets LLMs: How Reliable Are Large Language Models in Social Simulations?**, 2024.11, [[paper]](https://arxiv.org/abs/2410.23426).
- **Multi-expert Prompting Improves Reliability, Safety, and Usefulness of Large Language Models**, EMNLP 2024, [[paper]](https://arxiv.org/pdf/2411.00492).
- **Simulating Opinion Dynamics with Networks of LLM-based Agents**, NAACL Findings 2024, [[paper]](https://aclanthology.org/2024.findings-naacl.211.pdf) [[code]](https://github.com/yunshiuan/llm-agent-opinion-dynamics)
- **Beyond demographics: Aligning role-playing llm-based agents using human belief networks**, EMNLP Findings 2024, [[paper]](https://arxiv.org/pdf/2406.17232)
- **The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents**, CogSci 2024, [[paper]](https://escholarship.org/content/qt3k67x8s5/qt3k67x8s5_noSplash_f34c019b5fef5ecab5b70e30108f787c.pdf)
- **Large Language Models can Achieve Social Balance**, 2024.10, [[paper]](https://arxiv.org/abs/2410.04054).
- **On the limits of agency in agent-based models**, 2024.09, [[paper]](https://arxiv.org/abs/2409.10568), [[code]](https://github.com/AgentTorch/AgentTorch).
- **United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections**, 2024.09, [[paper]](https://arxiv.org/abs/2409.09045).
- **Out of One, Many: Using Language Models to Simulate Human Samples**, 2022, [[paper]](https://arxiv.org/abs/2209.06899).
- **Social Simulacra: Creating Populated Prototypes for Social Computing Systems**, 2022, [[paper]](https://dl.acm.org/doi/abs/10.1145/3526113.3545616).
- **Generative Agents: Interactive Simulacra of Human Behavior**, 2023, [[paper]](https://arxiv.org/abs/2304.03442), [[code]](https://github.com/joonspk-research/generative_agents).
- **Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies**, 2023, [[paper]](https://proceedings.mlr.press/v202/aher23a.html), [[code]](https://github.com/GatiAher/Using-Large-Language-Models-to-Replicate-Human-Subject-Studies).
- **Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?**, 2023 [[paper]](https://www.nber.org/papers/w31122), [[code]](https://github.com/johnjosephhorton/homo_silicus).
- **$S^3$: Social-network Simulation System with Large Language Model-Empowered Agents**, 2023, [[paper]](https://arxiv.org/abs/2307.14984).
- **Rethinking the Buyer's Inspection Paradox in Information Markets with Language Agents**, 2023, [[paper]](https://openreview.net/forum?id=6werMQy1uz).
- **SocioDojo: Building Lifelong Analytical Agents with Real-world Text and Time Series**, 2023, [[paper]](https://openreview.net/forum?id=s9z0HzWJJp).
- **Humanoid Agents: Platform for Simulating Human-like Generative Agents**, 2023, [[paper]](https://arxiv.org/abs/2310.05418), [[code]](https://github.com/HumanoidAgents/HumanoidAgents).
- **When Large Language Model based Agent Meets User Behavior Analysis: A Novel User Simulation Paradigm**, 2023, [[paper]](https://arxiv.org/abs/2306.02552), [[code]](https://github.com/RUC-GSAI/YuLan-Rec).
- **Large Language Model-Empowered Agents for Simulating Macroeconomic Activities**, 2023, [[paper]](https://arxiv.org/abs/2310.10436).
- **Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence**, 2023, [[paper]](https://arxiv.org/abs/2309.11456).
- **Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models**, 2023.06, NeurIPS 2023, [[paper]](https://arxiv.org/abs/2306.04746).
- **Epidemic Modeling with Generative Agents**, 2023.07, [[paper]](https://arxiv.org/abs/2307.04986), [[code]](https://github.com/bear96/GABM-Epidemic).
- **Emergent analogical reasoning in large language models**, 2023.08, nature human behavior, [[paper]](https://www.nature.com/articles/s41562-023-01659-w).
- **MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents**, 2023.10, [[paper]](https://arxiv.org/abs/2310.06500).
- **War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars**, 2023.11, [[paper]](https://arxiv.org/abs/2311.17227), [[code]](https://github.com/agiresearch/WarAgent).
- **Emergence of Social Norms in Large Language Model-based Agent Societies**, 2024.03, [[paper]](https://arxiv.org/abs/2403.08251), [[code]](https://github.com/sxswz213/CRSEC).
- **Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior**, ICLR-2024, [[paper]](https://openreview.net/forum?id=TrKq4Wlwcz)
- **Representational Equality in Cross-country Value Simulation: A Systematic Analysis of Large Language Models**, 2026.08, [[paper]](https://arxiv.org/abs/2608.08058).
- **Role Steering of Language Models for Social Simulations**, 2026.08, [[paper]](https://arxiv.org/abs/2608.00023).

##  7. <a name='Perspective'></a>👁️‍🗨️ Perspective and Position
- **Large AI models are cultural and social technologies**, Science, 2025.03, [[paper]](https://www.science.org/doi/10.1126/science.adt9819).
- **Position: Simulating Society Requires Simulating Thought**, 2025.06, [[paper]](https://arxiv.org/abs/2506.06958).
- **The benefits, risks and bounds of personalizing the alignment of large language models to individuals**, 2024.04, Nature Machine Intelligence, [[paper]](https://www.nature.com/articles/s42256-024-00820-y).
- **A social path to human-like artificial intelligence**, 2023.11, Nature Machine Intelligence, [[paper]](https://www.nature.com/articles/s42256-023-00754-x).
- **Using large language models in psychology**, 2023.10, Nature reviews psychology, [[paper]](https://www.nature.com/articles/s44159-023-00241-5).

Awesome-LLM-in-Social-Science

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Awesome-LLM-in-Social-Science

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README