LLM_based_Synthetic_Data_Generation

KO | EN

gitlite — search

#python #typescript #android #machine-learning #linux #macos #java #windows #awesome #cli #kubernetes #ios #openai #open-source

LLM_based_Synthetic_Data_Generation

(★ 15)

A curated and continuously updated collection of papers, tools, and datasets on synthetic data generation using LLMs and agentic workflows.

LLM_based_Synthetic_Data_Generation Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center"> A Survey of LLM-Based Methods for Synthetic Data Generation and the Rise of Agentic Workflows </h1>

<p align="center">
  <a href="https://link.springer.com/chapter/10.1007/978-3-031-93418-6_9" target="_blank">📄Paper</a> •
</p>

As AI systems become increasingly data-hungry, the need for high-quality datasets has never been greater. However, real-world data collection faces major challenges: scarcity, privacy constraints, and high acquisition costs. Synthetic Data Generation (SDG) offers a compelling alternative—creating artificial data that mimics real-world patterns without the associated drawbacks. Beyond training, synthetic data also plays a crucial role in evaluation and benchmark creation, enabling controlled, diverse, and scalable testing environments that are often impractical to build with real data.

> 📌 This repository complements the paper [*A Survey of LLM-Based Methods for Synthetic Data Generation and the Rise of Agentic Workflows*](https://link.springer.com/chapter/10.1007/978-3-031-93418-6_9). It serves as a continuously updated resource, curating references on LLM-based synthetic data generation to support ongoing learning, exploration, and collaboration within the research community.  
> 💡 If you’d like to contribute or suggest additions, feel free to open a pull request or issue!

<a name="top"></a>
## Table of Contents

- 📚 **[Surveys](#surveys)**
- 🧪 **[SDG Methods](#sdg-methods)**
  - 🧠 **[Traditional Architectures: Single LLM without External Tools](#traditional-architectures-single-llm-without-external-tools)**
  - 🤖 **[Agentic Workflows](#agentic-workflows)**
- 📖 **[Further Reading](#further-reading)**
- 🔗 **[Related Repositories](#related-repositories)**
- 📝 **[Citation](#citation)**

## 📚 Surveys <a name="surveys"></a>

[🔝 Back to Top](#top)

## 🧪 SDG Methods <a name="sdg-methods"></a>
### 🧠 Traditional Architectures: Single LLM without External Tools <a name="traditional-architectures-single-llm-without-external-tools"></a>

| **Title**                                                                                                                                                                                                                 | **Publication Date** | **Overview**                                                                                                                                       |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematics](https://arxiv.org/abs/2608.01522) | 08-2026 | Introduces Question-begets-Question, a self-evolving curriculum where a teacher LLM generates variants of problems the current checkpoint can mostly solve, breaking apparent performance ceilings on competition math. |
| [Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning](https://arxiv.org/abs/2607.17043) | 07-2026 | Shows collapse in iterative instruction tuning appears as polarization of competence, then proposes KITE, combining failure-guided synthetic data generation with knowledge-boundary-aware uncertainty curation. |
| [CodeAlchemy: Synthetic Code Rewriting at Scale](https://arxiv.org/abs/2606.10087) | 06-2026 | Presents CodeAlchemy, which rewrites publicly sourced code into 500B+ synthetic tokens via five strategies spanning quality rewriting, QA, developer tasks, dialogues, and execution traces. |
| [Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation](https://arxiv.org/abs/2606.18389) | 06-2026 | Shows that activation steering of open LLMs with language-identity and quality directions improves diversity and downstream utility of synthetic classification data for 11 low-resource languages over few-shot prompting. |
| [Domain-Specific Data Synthesis for LLMs via Minimal Sufficient Representation Learning](https://arxiv.org/abs/2605.30039) | 05-2026 | Introduces DOMINO, which synthesizes domain-specific data from reference examples by learning minimal domain representations via prompt tuning with a contrastive disentanglement objective separating domain patterns from sample noise. |
| [Make LLM Learn to Synthesize from Streaming Experiences through Feedback](https://arxiv.org/abs/2605.29940) | 05-2026 | Introduces StreamSynth, a setting where synthesis tasks arrive sequentially, and SynLearner, a framework letting a model accumulate reusable experience and feedback across tasks to improve later synthetic data generation. |
| [Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejection](https://arxiv.org/abs/2605.14062) | 05-2026 | Proposes MSIFR, a training-free framework that terminates low-quality generation trajectories at intermediate checkpoints using fast rule-based validators, cutting token consumption while preserving accuracy in LLM synthetic data generation. |
| [WRAP++: Web discoveRy Amplified Pretraining](https://arxiv.org/abs/2604.06829) | 04-2026 | Proposes WRAP++, which discovers cross-document relationships from web hyperlinks and synthesizes joint QA over document pairs, amplifying ~8.4B Wikipedia tokens into 80B relational pretraining tokens. |
| [How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data](https://arxiv.org/abs/2604.13977) | 04-2026 | Systematically studies prompt design, generator model, and source data across ~90 experiments, deriving a recipe for generating trillions of synthetic pretraining tokens by rewriting web documents with a single small LLM (companion [FinePhrase Space](https://huggingface.co/spaces/HuggingFaceFW/finephrase)). |
| [Data-efficient pre-training by scaling synthetic megadocs](https://arxiv.org/abs/2603.18534) | 03-2026 | Shows that stitching or stretching multiple synthetic rephrases of a web document into long "megadocs" improves loss scaling and data efficiency by up to 1.8x in data-constrained pretraining. |
| [Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts](https://arxiv.org/abs/2602.03545) | 02-2026 | Introduces Persona Generators, which evolve persona-generating code with LLM mutation operators to produce diverse synthetic populations maximizing coverage of opinions and rare trait combinations. |
| [FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale](https://arxiv.org/abs/2601.22146) | 01-2026 | Proposes FineInstructions, which transforms internet-scale pretraining documents into billions of synthetic instruction–answer pairs via ~18M instruction templates, enabling pretraining from scratch with an instruction-tuning objective. |
| [MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy](https://arxiv.org/abs/2508.05592) | 08-2025 | Synthesizes hard mathematical problems from scratch using PlanetMath concept-explanation pairs and a reinforcement-learning policy that optimizes problem validity, complexity, and reasoning consistency. |
| [CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks](https://arxiv.org/abs/2507.23751) | 07-2025 | Proposes CoT-Self-Instruct, which prompts an LLM to reason via Chain-of-Thought over seed tasks before generating new synthetic examples, then filters them with automatic quality metrics for training. |
| [Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models](https://arxiv.org/abs/2506.04689) | 06-2025 | Proposes REWIRE, a guided-rewriting method that transforms low-quality discarded web documents into useful synthetic pre-training data, improving downstream performance over training on filtered web data alone. |
| [NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communities](https://arxiv.org/abs/2505.18383) | 05-2025 | Proposes a methodology to generate synthetic and retrieval-based pre-training data tailored to a community's language, cultural heritage, and values, demonstrated by building NileChat for Egyptian and Moroccan Arabic. |
| [A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment](https://arxiv.org/abs/2505.10717) | 05-2025 | Introduces a modular framework adapting small language models into clinical experts via pre-instruction tuning, model merging, and alignment, producing the MediFlow synthetic dataset of 2.5 million medical instructions. |
| [O1 Replication Journey – Part 2: Surpassing O1-preview through Simple Distillation Big Progress or Bitter Lesson?](https://arxiv.org/abs/2411.16489)                                                                     | 11-2024              | A base model can outperform O1-preview on mathematical reasoning through simple knowledge distillation from O1's API.                               |
| [Self-Judge: Selective Instruction Following with Alignment Self-Evaluation](https://arxiv.org/abs/2409.00935)                                                                                                            | 09-2024              | Introduces a framework for a large language model to self-evaluate and decide whether to respond to a given prompt.                                 |
| [Automated test generation to evaluate tool-augmented LLMs as conversational AI agents](https://arxiv.org/abs/2409.15934)                                                                                                 | 09-2024              | Presents a method for automatically generating tests to evaluate how well tool-augmented LLMs perform as conversational agents.                     |
| [Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources](https://arxiv.org/abs/2409.08239) | 09-2024 | Introduces a scalable pipeline that generates synthetic examples with intermediate reasoning grounded in real data sources, then curates them by discarding low-quality generations to raise dataset quality. |
| [Is Child-Directed Speech Effective Training Data for Language Models?](https://arxiv.org/abs/2408.03617)                                                                                                                 | 08-2024              | Investigates the effectiveness of training language models on child-directed speech for better language acquisition.                                |
| [Case2Code: Learning Inductive Reasoning with Synthetic Data](https://arxiv.org/abs/2407.12504)                                                                                                                           | 07-2024              | Teaches large language models inductive reasoning by training them on synthetic input-output examples and their corresponding code.                 |
| [Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language Models via Inherent Capability](https://arxiv.org/abs/2407.00454)                                                                               | 07-2024              | Improves the cross-lingual abilities of large language models by using their own translation capabilities to generate training data.                |
| [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://arxiv.org/abs/2406.20094)                                                                                                                           | 06-2024              | Presents a method for generating vast amounts of diverse synthetic data by prompting a large language model with a billion different personas.      |
| [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://arxiv.org/abs/2406.08464)                                                                                                  | 06-2024              | Proposes generating large-scale alignment data by feeding simple pre-query templates to an already aligned model.                                   |
| [ToolCoder: Teach Code Generation Models to use API search tools](https://arxiv.org/abs/2305.04032)                                                                                                                       | 05-2024              | Teaches code-generating models to use API search tools for finding relevant APIs for specific tasks.                                                |
| [DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data](https://arxiv.org/abs/2405.14333)                                                                                                 | 05-2024              | Proposes a framework and a large synthetic dataset to improve theorem-proving abilities of large language models in natural language.               |
| [Phi-3 Technical Report](https://arxiv.org/abs/2404.14219)                                                                                                                                                                | 04-2024              | Details the architecture, training, and capabilities of the Phi-3 family of small, powerful, open-source multimodal models.                         |
| [Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks](https://arxiv.org/abs/2304.10145)                                                                                                       | 04-2024              | Investigates how well ChatGPT can replicate human-generated data labels for various social computing tasks.                                         |
| [OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset](https://proceedings.neurips.cc/paper_files/paper/2024/hash/3d5aa9a7ce28cdc710fbd044fd3610f3-Abstract-Datasets_and_Benchmarks_Track.html)              | 02-2024              | Introduces a dataset of 1.8 million math problems and solutions to improve the mathematical reasoning of open-source language models.               |
| [Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling](https://arxiv.org/abs/2401.16380)                                                                                                         | 01-2024              | Presents a method for more efficient language model training by rephrasing web documents into different styles like Wikipedia articles.             |
| [Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://arxiv.org/abs/2401.01335)                                                                                                         | 01-2024              | Shows that a language model can improve its reasoning and problem-solving skills by "playing" against itself to solve problems.                     |
| [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://arxiv.org/abs/2312.06585)                                                                                                     | 12-2023              | Explores a self-training method where a model generates and filters its own training data to improve problem-solving skills.                        |
| [Orca 2: Teaching Small Language Models How to Reason](https://arxiv.org/abs/2311.11045)                                                                                                                                  | 11-2023              | Presents a method to teach smaller language models different reasoning strategies for various tasks, enhancing their overall capabilities.          |
| [Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models](https://arxiv.org/abs/2311.00287)                                                                         | 11-2023              | Examines using large language models and techniques like retrieval-augmented generation to create structured medical text data.                     |
| [HELPSTEER: Multi-attribute Helpfulness Dataset for STEERLM](https://arxiv.org/abs/2311.09528)                                                                                                                            | 11-2023              | Introduces a dataset annotating model response helpfulness across multiple attributes (e.g., correctness, coherence) to improve training.           |
| [CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation](https://arxiv.org/abs/2310.15638)                                                                          | 10-2023              | Uses a large language model's uncertainty to decide whether a human or the model should handle data annotation for greater efficiency.              |
| [ULTRAFEEDBACK: Boosting Language Models with Scaled AI Feedback](https://arxiv.org/abs/2310.01377)                                                                                                                       | 10-2023              | Explores using large-scale AI-generated feedback to align language models with human preferences and presents a corresponding dataset.              |
| [Textbooks Are All You Need II: phi-1.5 technical report](https://arxiv.org/abs/2309.05463)                                                                                                                               | 09-2023              | Describes phi-1.5, a 1.3B parameter model trained on high-quality "textbook" data for common sense reasoning.                                       |
| [MAMMOTH: BUILDING MATH GENERALIST MODELS THROUGH HYBRID INSTRUCTION TUNING](https://arxiv.org/abs/2309.05653)                                                                                                            | 09-2023              | Introduces open-source language models trained on a diverse set of math problems to improve general math problem-solving skills.                    |
| [METAMATH: BOOTSTRAP YOUR OWN MATHEMATICAL QUESTIONS FOR LARGE LANGUAGE](https://arxiv.org/abs/2309.12284)                                                                                                                | 09-2023              | Improves mathematical reasoning in language models by rewriting existing math questions from various perspectives for a more diverse training set.  |
| [DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation](https://arxiv.org/abs/2308.14346)                                                                                               | 08-2023              | Presents a language model trained on high-quality medical data for more accurate and reliable healthcare-related conversations.                     |
| [AlpaGasus: Training A Better Alpaca with Fewer Data](https://arxiv.org/abs/2307.08701)                                                                                                                                   | 07-2023              | Shows that filtering low-quality examples from the Alpaca dataset allows for training a better model with significantly less data.                  |
| [BEAVERTAILS: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://proceedings.neurips.cc/paper_files/paper/2023/hash/4dbb61cb68671edc4ca3712d70083b9f-Abstract-Datasets_and_Benchmarks.html) | 07-2023              | Introduces a dataset that separately annotates helpfulness and harmlessness of model responses for better safety alignment.                         |
| [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://arxiv.org/abs/2306.08568)                                                                                                                 | 06-2023              | Introduces Evol-Instruct, a method for generating complex instruction data to train more capable code-generating language models.                   |
| [Textbooks Are All You Need](https://arxiv.org/abs/2306.11644)                                                                                                                                                            | 06-2023              | Argues that training language models on high-quality, "textbook-style" data yields better performance than training on unfiltered web data.         |
| [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://arxiv.org/abs/2306.02707)                                                                                                                   | 06-2023              | Introduces Orca, a model that learns to imitate the reasoning process of larger models like GPT-4 by training on their detailed explanation traces. |
| [TinyStories: how Small Can Language Models Be and Still Speak Coherent English?](https://arxiv.org/abs/2305.07759)                                                                                                       | 05-2023              | Explores the minimum size for language models to generate coherent English by training them on synthetic children's stories.                        |
| [WizardLM: Empowering Large Language Models to Follow Complex Instructions](https://arxiv.org/abs/2304.12244)                                                                                                             | 04-2023              | Creates large amounts of instruction-following data with varying complexity to improve how language models follow complex instructions.             |
| [Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data](https://arxiv.org/abs/2304.01196)                                                                                                    | 04-2023              | Creates a multi-turn chat dataset by having ChatGPT converse with itself, then uses this data to train an open-source chat model.                   |
| [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://arxiv.org/abs/2304.06975)                                                                                                                             | 04-2023              | Details fine-tuning the LLaMA model with a large Chinese medical dataset to create a specialized medical language model.                            |
| [Instruction Tuning with GPT-4](https://arxiv.org/abs/2304.03277)                                                                                                                                                         | 04-2023              | Demonstrates using GPT-4 to generate a large dataset of instruction-following examples for fine-tuning other language models.                       |
| [Alpaca: A Strong, Replicable Instruction-Following Model](https://crfm.stanford.edu/2023/03/13/alpaca.html)                                                                                                              | 03-2023              | Presents a language model fine-tuned on 52,000 instruction-following demonstrations generated by a larger model.                                    |
| [CHATGPT OUTPERFORMS CROWD WORKERS FOR TEXT-ANNOTATION TASKS](https://www.pnas.org/doi/abs/10.1073/pnas.2305016120)                                                                                                       | 03-2023              | Shows that for several text annotation tasks, ChatGPT performs better and is more cost-effective than human crowd workers.                          |
| [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://ieeexplore.ieee.org/abstract/document/10858342/)                                                                                                          | 02-2023              | Proposes using ChatGPT to rephrase training sentences to create a more diverse and larger dataset for model training.                               |
| [SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions](https://arxiv.org/abs/2212.10560)                                                                                                              | 12-2022              | Introduces a method for a language model to generate its own instruction-following data for self-improvement through fine-tuning.                   |
| [DISCO: Distilling Counterfactuals with Large Language Models](https://arxiv.org/abs/2212.10534)                                                                                                                          | 12-2022              | Proposes generating high-quality, large-scale counterfactual data by using a syntactic parser to guide a large language model.                      |
| [CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation](https://arxiv.org/abs/2210.04873)                                                                                                               | 10-2022              | Presents a "retrieve-then-edit" framework where a model finds and modifies text to create counterfactual examples.                                  |
| [STaR: Self-Taught Reasoner: Bootstrapping Reasoning With Reasoning](https://research.google/pubs/star-self-taught-reasoner-bootstrapping-reasoning-with-reasoning/)                                                      | 03-2022              | Introduces a method for a language model to learn reasoning by generating its own rationales for problems and then fine-tuning on the correct ones. |

[🔝 Back to Top](#top)

### 🤖 Agentic Workflows <a name="agentic-workflows"></a>

[🔝 Back to Top](#top)

## 📖 Further Reading <a name="further-reading"></a>

[🔝 Back to Top](#top)

## 🔗 Related Repositories <a name="related-repositories"></a>

| [Awesome Synthetic Datasets](https://github.com/davanstrien/awesome-synthetic-datasets) | Practical resources for building synthetic text and vision datasets. |
| --------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| [LLM Synthetic Data](https://github.com/wasiahmad/Awesome-LLM-Synthetic-Data/tree/main) | Papers, tools, and blogs on LLM-generated data.                      |
| [LLM-Datasets](https://github.com/mlabonne/llm-datasets/tree/main):                     | Curated datasets and tools for LLM post-training.                    |
| [DataArc-SynData-Toolkit](https://github.com/DataArcTech/DataArc-SynData-Toolkit) | Open-source, configuration-driven framework (visual UI + CLI) for multi-source, multimodal, and multilingual synthetic data generation via a modular, quality-controllable closed-loop pipeline. |
| [GraphGen](https://github.com/InternScience/GraphGen) | Knowledge-graph-driven framework that builds fine-grained graphs from source text, locates an LLM's knowledge gaps via calibration error, and generates targeted QA, chain-of-thought, and multi-hop data for supervised fine-tuning. |

[🔝 Back to Top](#top)

## 📝 Citation <a name="citation"></a>

If this work is helpful, please kindly cite as:

```bibtex
@inproceedings{alismail2025survey,
  title={A Survey of LLM-Based Methods for Synthetic Data Generation and the Rise of Agentic Workflows},
  author={Alismail, Ahmad and Lanquillon, Carsten},
  booktitle={International Conference on Human-Computer Interaction},
  pages={119--135},
  year={2025},
  organization={Springer}
}
```

LLM_based_Synthetic_Data_Generation

LLM_based_Synthetic_Data_Generation Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

LLM_based_Synthetic_Data_Generation

LLM_based_Synthetic_Data_Generation Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README