Text2GraphQuery-DataGen

KO | EN

gitlite — search

#dotfiles #javascript #java #php #vim #ruby #mantisbt-plugin #nodejs #vim-configuration #shell #unmaintained #c

Text2GraphQuery-DataGen

★ 102 Open GitHub ↗

Fine-Tuning Dataset Auto-Generation for Graph Query Languages.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Awesome-Text2GQL

[![Star](https://shields.io/github/stars/tugraph-family/Awesome-Text2GQL?logo=startrek&label=Star&color=yellow)](https://github.com/TuGraph-family/Awesome-Text2GQL/stargazers)
[![Fork](https://shields.io/github/forks/tugraph-family/Awesome-Text2GQL?logo=forgejo&label=Fork&color=orange)](https://github.com/TuGraph-family/Awesome-Text2GQL/forks)
[![Contributor](https://shields.io/github/contributors/tugraph-family/Awesome-Text2GQL?logo=actigraph&label=Contributor&color=abcdef)](https://github.com/TuGraph-family/Awesome-Text2GQL/contributors)
[![Commit](https://badgen.net/github/last-commit/tugraph-family/Awesome-Text2GQL/master?icon=git&label=Commit)](https://github.com/TuGraph-family/Awesome-Text2GQL/commits/master)
[![License](https://shields.io/github/license/tugraph-family/Awesome-Text2GQL?logo=apache&label=License&color=blue)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![Release](https://shields.io/github/v/release/tugraph-family/Awesome-Text2GQL.svg?logo=stackblitz&label=Version&color=red)](https://github.com/TuGraph-family/Awesome-Text2GQL/releases)

## Introduction

Awesome-Text2GQL is an AI-assisted framework for Text2GQL dataset construction. The framework supports translation, generalization and generation of Text2GQL corpus and corresponding database instance. With the assistance of this framework, users are able to construct a high quality Text2GQL dataset on any graph query language(ISO-GQL, Cypher, Gremlin, SQL/PGQ, etc.) with a significantly lower cost than pure human labor annotation.

![image](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/overview.jpg)

### Key Features

+ **Translation**

+ [x] **Question translation** from graph query languages to questions in different natural languages to accelerate question annotation.

+ [x] **Query translation** between different graph query languages to gather corpus from all existing graph query languages.

+ **Generalization**

+ [x] **Question generalization** for getting natural language questions in different language style but with same semantic meaning to increase the diversity of corpus.

+ [x] **Query generalization** for instantiating similar query patterns on different graph schemas to increase the diversity of corpus.

+ **Generation**

+ [x] **Schema generation** for automatically generating complex graph database schemas from natural language domain descriptions.

+ [x] **Data generation** for generating realistic simulation data that follows statistical distributions (power-law, long-tail, normal distribution, etc.).

+ [x] **Corpus generation** for producing high-quality Question-Query pairs with complex queries (multi-hop, nested queries) through iterative enhancement strategies.

### Generated Benchmark Dataset

The [Text2GQL-Bench](https://arxiv.org/abs/2602.11745)'s dataset is generated by Awesome-Text2GQL framework. It contains 178,184 (Question, Query) pairs spanning 13 domains. The dataset is available at [Text2GQL-Bench_dataset](https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/text2gql/Text2GraphQueryBenchmark/Text2GQL-Bench_dataset.zip). The dataset including the Oracle SQL/PGQ translated queries is available at [Dataset-with-SQL/PGQ](https://objectstorage.us-ashburn-1.oraclecloud.com/p/8dIkuVGsfnRQlP3ifxVDjQP0pmidpadEY18ltEbkPC4PrZyLTxjJdqDjbtWIEYUW/n/ogcs/b/Text2GQL-Bench_dataset/o/Text2GQL-Bench_dataset.zip), it includes 19633 out of 22407 existing queries. To run Text2GQL test, please refer to our [Text2GraphQuery-Driver](https://github.com/TuGraph-family/text2graphquery-driver/tree/main).

## Demo: TuGraph-DB ChatBot

TuGraph-DB ChatBot is a demo demonstrates the effect of an agent trained by the corpus generated by Awesome-Text2GQL. It can intercract with the TuGraph-DB taking how you want to operate the db in Chinese as input, such as querying or creating data. And The ChatBot will also help you to excute the cypher too.

![demo](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/demo.gif)

Refer to [demo](./chatbot_demo/README.md) for more information.

## Quick Start

### Environment Preparation

It is recommended to use poetry to manage your python environment while other tools may also work.

```
# install poetry and poetry shell
pip install poetry
pip install poetry-plugin-shell

# install environment
poetry install

# activate virtual environment
poetry shell
```

### LLM Setup

We offer two platforms for your LLM needs: DashScope (by Alibaba Cloud) and OpenAI.
#### Remote LLM Setup
##### Remote LLM Setup For DashScope (recommended)

1. Set API-KEY via environment variables 
   
    It is based on the Qwen Inference Service served by Aliyun,you can refer to [Aliyun](https://help.aliyun.com/zh/dashscope/developer-reference/acquisition-and-configuration-of-api-key?spm=a2c4g.11186623.0.0.4e202a9dXlz5vH#1e6311202fthe) to apply the API-KEY.

```
# replace YOUR_DASHSCOPE_API_KEY with your API-KEY
echo "export DASHSCOPE_API_KEY='YOUR_DASHSCOPE_API_KEY'" >> ~/.bashrc
source ~/.bashrc
echo $DASHSCOPE_API_KEY
```
##### Remote LLM Setup For OpenAI
1. Set API-KEY via environment variables
```
# replace OPENAI_API_KEY with your API-KEY
echo "export OPENAI_API_KEY='YOUR_OPENAI_API_KEY'" >> ~/.bashrc
source ~/.bashrc
echo $OPENAI_API_KEY
```
2. Set BASE_URL via environment variables
```
# replace OPENAI_BASE_URL with your OPENAI_BASE_URL
echo "export OPENAI_BASE_URL='YOUR_OPENAI_BASE_URL'" >> ~/.bashrc
source ~/.bashrc
echo OPENAI_BASE_URL
```
##### Change The Platform 
   
you can change the platform by setting the evenvironment variables

```
    # use openai platform for LLM
    # echo "export LLM_PLATFORM='dashscope'" >> ~/.bashrc
    echo "export LLM_PLATFORM='openai'" >> ~/.bashrc
    source ~/.bashrc
    echo LLM_PLATFORM
```
or set the platform and model when instantiating an object of the LlmClient
```python
from app.core.llm.llm_client import LlmClient
    
llm_client_for_dashscope = LlmClient(model="qwen-plus-0723",platform = "dashscope")
llm_client_for_openai = LlmClient(model="gpt-4",platform = "openai")
```
   
#### Local LLM Setup

Awesome-Text2GQL's local LLM client is based on transformers library, use model id from HuggingFace model hub if you can access HuggingFace or use the related local file path where the LLM model is. Add model_path when initializing llm client if you want to use local LLM instead of remote LLM.

### Run Example

#### Generate Schema

`python ./examples/generate_schema.py`

This example shows how to use Awesome-Text2GQL Framework to generate a graph schema for use in corpus construction.

#### Generate Data

`python ./examples/generate_data.py`

This example shows how to use Awesome-Text2GQL Framework to generate data instances based on a given graph schema file.

#### Generate Corpus

This example shows how to use the Awesome-Text2GQL framework to generate a corpus. Before running it, ensure you have a running database instance and update the database connection and output configuration in examples/generate_corpus.py. Alternatively, you can import our provided test dataset into TuGraph. Download link: [*test dataset*](https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/text2gql/awesome-text2gql/example_graph.zip).

Example TuGraph import command:

```shell
# lgraph_import -u admin -p 73@TuGraph -c import_config.json --dir /var/lib/lgraph/data/ --overwrite true -v 3
```

After all, run:

`python ./examples/generate_corpus.py`

When the script finishes, the generated corpus will be saved to the output directory specified in the script.

#### Generate Corpus by Template

This example shows how to use the Awesome-Text2GQL framework to generate a corpus based on predefined query templates. Before running it, ensure you have a running database instance and update the database connection and output configuration in examples/generate_corpus_by_template.py.

The template-based generation process works as follows:
1. Connects to the database to fetch real node/relationship data through exploration queries
2. Fills the predefined query templates (from examples/corpus_templates/corpus_templates.json) with real data
3. Uses LLM to translate the filled queries into natural language questions
4. Validates the generated pairs and saves them to the output directory

#### Convert TuGraph-DB Schema to Google Spanner

`python ./examples/tugraph_to_spanner.py`

This example demonstrates how to convert a TuGraph-DB graph schema to Google Spanner compatible format. It generates:

- **Spanner_SchemaDDL_*.sql**: Complete DDL statements including:
  - Vertex table definitions with primary keys
  - Edge table definitions with foreign key constraints
  - Spanner Property Graph definitions
  
- **spanner_import_config.json**: Data import configuration for loading CSV data into Spanner

The conversion handles:
- Schema transformation from TuGraph format to Spanner DDL
- Type mapping (INT64, FLOAT64, STRING, BOOL, DATE)
- Graph relationships and constraints
- Property definitions for both vertices and edges

**Usage:**
```python
from examples.tugraph_to_spanner import convert_tugraph_to_spanner

# Convert TuGraph schema to Spanner format
convert_tugraph_to_spanner(
    input_json_path="examples/generated_schemas/example_schema.json",
    output_dir="examples/Spanner_Instance"
)
```

After all, run:

`python ./examples/generate_corpus_by_template.py`

When the script finishes, the generated corpus will be saved to examples/generated_corpus/{graph_name}_template_corpus.json.

#### Oracle SQL Property Graphs (SQL/PGQ)

Awesome-Text2GQL includes Oracle SQL/PGQ support for schema conversion, graph setup, query translation, corpus generation, validation, and benchmark dataset preparation.

For detailed workflows, see:

- [Oracle SQL/PGQ data generation workflow](./doc/en-us/development/oracle_sqlpgq_data_generation_workflow.md): convert framework/TuGraph-style schemas into Oracle SQL/PGQ artifacts, create local Oracle property graphs, generate deterministic and LLM-based corpora, validate generated queries, and combine corpus outputs.
- [Dataset preparation utilities](./dataset_prep/README.md): translate benchmark Cypher/GQL-like records to Oracle SQL/PGQ, optionally validate them against Oracle, analyze failures, compare Oracle SQL/PGQ results with Neo4j, and export validated datasets.

#### Cypher2GQL

`python ./examples/cypher2gql.py`

This example shows how to use AwesomeText2GQL Framework to translate neo4j's Text2Cypher corpus into Text2GQL corpus with queries aligned to ISO-GQL grammar.

#### Generalize Cypher Corpus

`python ./examples/generalize_corpus_cypher.py`

This example shows how to use AwesomeText2GQL Framework to generalize from one cypher corpus pair to construct a Text2GQL corpus dataset.

#### Generalize GQL Corpus

`python ./examples/generalize_corpus_gql.py`

This example shows how to use AwesomeText2GQL Framework to generalize from one ISO-GQL corpus pair to construct a Text2GQL corpus dataset.

#### English to Chinese

`python ./examples/english_to_chinese.py`

This example shows how to use AwesomeText2GQL Framework to translate English question into Chinese question with the same semantic meaning.

#### Hierarchical Question Translation

`python ./examples/hierarchical_question_translation.py`

This example shows how to use `HierarchicalQuestionTranslator` to translate GQL queries into hierarchical natural language questions (`level_1`, `level_2`, `level_3`) and optionally enrich them with `external_knowledge`.

#### AST Printing

`python ./examples/print_ast.py`

This example shows how to use AwesomeText2GQL Framework to print the ast of a query. Visualizing the AST is helpful for IR and other AST related development.

#### SQL to Cypher

`python ./examples/sql2cypher.py`

This example demonstrates how to translate SQL queries to Cypher queries and execute them on Neo4j using the SQL-to-Cypher pipeline.

**Important Notes:**

1. **Create Neo4j Database First**: Before running this script, you must create the target Neo4j database. The database name cannot contain special characters like underscores (`_`).

```cypher
   # Connect to Neo4j and create the database
   CREATE DATABASE sql2cypherexample;
   ```

2. **Java Dependency**: This module uses Neo4j's JDBC driver for SQL-to-Cypher translation. The Java JAR file is included at:
   - `app/impl/neo4j_cypher/neo4j-jdbc-full-bundle-6.9.1.jar`

3. **Java CLI Tool**: The translation is performed by `Sql2CypherCLI.class` which wraps the Neo4j JDBC translator. The Python module calls this Java tool via subprocess.

**Usage Example:**

```python
from app.impl.neo4j_cypher.db_client.neo4j_db_client import Neo4jDBClient
from app.impl.neo4j_cypher.translator.neo4j_cypher_query_translator import Neo4jCypherQueryTranslator

# Create Neo4j DB client
db_client_params = {
    "uri": "neo4j://localhost:7687",
    "user": "neo4j",
    "password": "password",
    "database": "sql2cypherexample",
}
neo4j_client = Neo4jDBClient(db_client_params)

# Create SQL to Cypher translator
translator = Neo4jCypherQueryTranslator()

# Translate SQL to Cypher
sql = "SELECT name, age FROM users WHERE id > 5"
cypher = translator.translate(sql)
# Output: MATCH (users:users) WHERE users.id > 5 RETURN users.name AS name, users.age AS age

# Execute Cypher query
result = neo4j_client.execute_query(cypher)
if result.is_success():
    print(result.data)
```

#### SQLite to Neo4j Migration

`python ./examples/sqlite2neo4j.py`

This example demonstrates how to migrate SQLite database data and schema to Neo4j graph database.

**Important Notes:**

1. **Create Neo4j Database First**: Before running this script, create the target Neo4j database.

```cypher
   CREATE DATABASE sql2cypherexample;
   ```

2. **Migration Strategy**:
   - **Nodes**: Each SQLite table becomes a node label in Neo4j
   - **Relationships**: Foreign keys become relationships
   - **Join Tables**: Tables with 2+ foreign keys are "flattened" into direct relationships (M:N)

3. **Batch Processing**: Large datasets are processed in batches (default: 5000 records per batch)

**Usage Example:**

```python
from app.impl.sqlite_sql.db_client.sqlite_db_client import SQLiteDBClient
from app.impl.neo4j_cypher.db_client.neo4j_db_client import Neo4jDBClient
from app.impl.sqlite_sql.migrator.sqlite_to_neo4j_migrator import SQLiteToNeo4jMigrator

# Create SQLite client
sqlite_client = SQLiteDBClient("path/to/database.sqlite")
sqlite_client.connect()

# Create Neo4j client
neo4j_client = Neo4jDBClient({
    "uri": "neo4j://localhost:7687",
    "user": "neo4j",
    "password": "password",
    "database": "mydb"
})

# Create migrator and execute migration
migrator = SQLiteToNeo4jMigrator(
    sqlite_db_client=sqlite_client,
    neo4j_db_client=neo4j_client,
    batch_size=5000
)
migrator.migrate(clear_before=True)

# Cleanup
neo4j_client.close()
sqlite_client.disconnect()
```

**Module Architecture:**

```
app/impl/
├── neo4j_cypher/          # Neo4j Cypher support
│   ├── db_client/         # Neo4jDBClient - Neo4j database client
│   └── translator/        # Neo4jCypherQueryTranslator - SQL to Cypher translator
└── sqlite_sql/            # SQLite support
    ├── db_client/         # SQLiteDBClient - SQLite database client
    └── migrator/          # SQLiteToNeo4jMigrator - SQLite to Neo4j migrator
```

## Modules

Awesome-Text2GQL use Translator, Generalizer and Generator to assit the entire process of Text2GQL dataset construction.

### Translator

Translator supports multilingual tranlsation for question translation and multi-graph-query-language translation for query translation. Users can use translator to translate existing corpus in different natural language and graph query language into target natural language and graph query language.

#### Question Translator

Question translator currently has the ability to translate a query into a natural language question(English) with a similar query template and corresponding question template. In the future, we will support multilingual translation of natural language question.

``` python
from app.core.llm.llm_client import LlmClient
from app.core.translator.question_translator import QuestionTranslator

llm_client = LlmClient(model="qwen-plus-0723")
query_template="MATCH (n:Person)-[:HAS_CHILD*1]->(n) WHERE n.name = 'Vanessa Redgrave' RETURN n"
question_template="Who are Roy Redgrave's second generations?"
query_list = [
    "MATCH (n1:person)-[e1:acted_in]->{1,1}(n2:movie) WHERE n1.id = 'Neo' RETURN n2.`duration` AS `DURATION`",
    "MATCH (n1:person)-[e1:directed]->{1,1}(n2:movie) WHERE n1.name = 'MacQUeen' RETURN n2.id AS ID",
    "MATCH (n1:person)-[e1:produce]->{1,1}(n2:movie) WHERE n1.name = 'Hans' RETURN n2.rated AS RATED"
    ]

# translate query into question
question_translator = QuestionTranslator(llm_client=llm_client, chunk_size=5)
question_list = question_translator.translate(
    query_template=query_template,
    question_template=question_template,
    query_list = query_list
)
```

### Hierachical Question Translator

Hierarchical question translator can generate `level_1`, `level_2`, and `level_3` questions for each GQL query, and optionally append `external_knowledge`.

``` python
from app.core.llm.llm_client import LlmClient
from app.core.translator.question_translator import HierarchicalQuestionTranslator

llm_client = LlmClient(model="qwen-plus")
question_translator = HierarchicalQuestionTranslator(
    llm_client=llm_client,
    need_external_knowledge=True
)

query_list = [
    "MATCH (n:Person)-[:ACTED_IN]->(m:Movie) WHERE n.name = 'Tom Hanks' RETURN m.title",
    "MATCH (m:Movie)<-[:DIRECTED]-(d:Person) WHERE m.title = 'The Matrix' RETURN d.name"
]
data_schema_list = [
    "Node labels: Person(name), Movie(title); Relationship: (:Person)-[:ACTED_IN|DIRECTED]->(:Movie)",
    "Node labels: Person(name), Movie(title); Relationship: (:Person)-[:ACTED_IN|DIRECTED]->(:Movie)"
]

sample_list = question_translator.translate_hierachical_questions(
    query_list=query_list,
    data_schema_list=data_schema_list
)
```
#### Query Translator

![query_translator](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/query_translator.png)

Query translator has the ability to translate queries in one query language into another, like cypher to gql. To achieve this, Awesome-Text2GQL designed and implemented a set of intermediate representation for commonly used graph query languages(ISO-GQL, Cypher, Gremlin, SQL/PGQ, etc.) and their dialects. With ast vistitor's implementations, different graph query language can be translated into the intermediate representation. With the query translator's implementations, intermediate representation can be translated into different graph query language.

#### Query Grader

Query grader uses LLM to analyze and grade the difficulty level of graph queries. It evaluates query complexity based on factors like path length, nesting depth, aggregation complexity, and condition complexity.

**Grading Standards:**

+ **Easy**: Single node/edge matching, no aggregation/complex conditions (e.g., `MATCH (n) RETURN n`)
+ **Medium**: One-hop path, simple aggregation/filtering (e.g., `MATCH (a)-[]->(b) RETURN COUNT(b)`)
+ **Hard**: Multi-hop (≤2), multiple conditions, non-nested aggregation
+ **Extra Hard**: Complex path (≥3), multi-step MATCH, nested aggregation

```python
from app.core.llm.llm_client import LlmClient
from app.core.translator.query_grader import QueryGrader

llm_client = LlmClient(model="qwen-plus-0723")
query_grader = QueryGrader(llm_client)

# Grade a query
query = [
    {"question": "Find all actors", "query": "MATCH (n:Person) RETURN n"},
    {"question": "Find movies by actor", "query": "MATCH (a:Person)-[:ACTED_IN]->(m:Movie) RETURN m"}
]
graded_query = query_grader.grade_query_sync(query)
# Each item will have a 'difficulty' field added
```

``` python
from app.impl.iso_gql.translator.iso_gql_query_translator import IsoGqlQueryTranslator as GQLTranslator
from app.impl.tugraph_cypher.ast_visitor.tugraph_cypher_query_visitor import TugraphCypherAstVisitor

query_visitor = TugraphCypherAstVisitor()
gql_translator = GQLTranslator()
cypher = "MATCH (n:Person)-[:HAS_CHILD*1]->(n) WHERE n.name = 'Vanessa Redgrave' RETURN n"

# translate cypher to gql
success, query_pattern = query_visitor.get_query_pattern(cypher)
if success:
    gql = gql_translator.translate(query_pattern)
```

### Generalizer

Generalizer supports the corpus generalization based on the given query template and question template. Users can use generalizer to construct a large scale corpus dataset across multiple database instance from a limited number of existing corpus templates.

#### Question Generalizer

Question generalizer has the ability to generalize the given natural language question into similar questions with different language styles, and the symantic similarity is ensured with the given corresponding query. This generalization aims to increase the linguistic diversity of corpus to simulate the real world Text2GQL scenario.

``` python
from app.core.generalizer.question_generalizer import QuestionGeneralizer
from app.core.llm.llm_client import LlmClient

llm_client = LlmClient(model="qwen-plus-0723")
question_generalizer = QuestionGeneralizer(llm_client)
corpus_pair_list = [
    [
        "MATCH (n:Person)-[:HAS_CHILD*1]->(n) WHERE n.name = 'Vanessa Redgrave' RETURN n",
        "Who are Roy Redgrave's second generations?"
    ]
]

# generalize question
generalized_corpus_pair_list = []
for corpus_pair in corpus_pair_list:
    query = corpus_pair[0]
    question = corpus_pair[1]
    generalized_question_list = question_generalizer.generalize(
        query=query,
        question=question
    )
    for generalized_question in generalized_question_list:
        generalized_corpus_pair_list.append((query, generalized_question))
    generalized_corpus_pair_list.append((query, question))
```

#### Query Generalizer

![query_generalizer](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/query_generalizer.png)

Query generalizer has the ability to generalize the given query into queries with similar query pattern on the given schema. With the intermediate representation for graph query languages, Awesome-Text2GQL can translate a query into intermediate query pattern, and the similar query pattern can be constructed with different variables on different schema. This generalization aims to migrate existing query patterns onto new database instance efficiently.

``` python
from app.core.generalizer.query_generalizer import QueryGeneralizer
from app.impl.tugraph_cypher.ast_visitor.tugraph_cypher_query_visitor import TugraphCypherAstVisitor

db_id = "movie"
instance_path = "../app/impl/tugraph_cypher/generalizer/base/db_instance/movie"
query_visitor = TugraphCypherAstVisitor()
query_generalizer = QueryGeneralizer(db_id, instance_path)
query_template="MATCH (n {name: 'Carrie-Anne Moss'}) RETURN n.born AS born"

# generalize cypher query
query_list = query_generalizer.generalize_from_cypher(query_template=query_template)
```

``` python
from app.core.generalizer.query_generalizer import QueryGeneralizer
from app.impl.iso_gql.translator.iso_gql_query_translator import IsoGqlQueryTranslator as GQLTranslator
from app.impl.tugraph_cypher.ast_visitor.tugraph_cypher_query_visitor import TugraphCypherAstVisitor

db_id = "movie"
instance_path = "../app/impl/tugraph_cypher/generalizer/base/db_instance/movie"
query_generalizer = QueryGeneralizer(db_id, instance_path)
query_visitor = TugraphCypherAstVisitor()
gql_translator = GQLTranslator()
query_template="MATCH (n:Person)-[:HAS_CHILD*1]->(n) WHERE n.name = 'Vanessa Redgrave' RETURN n"

# generalize gql query
query_list = []
success, query_pattern = query_visitor.get_query_pattern(query_template)
if success:
    query_pattern_list = query_generalizer.generalize(query_pattern=query_pattern)
    for query_pattern in query_pattern_list:
        query = gql_translator.translate(query_pattern)
        query_list.append(query)
```

### Generator

The framework implements a full-chain automated generation pipeline: **Schema Generation → Graph Database Instance Construction → Complex Corpus Generation**. This transforms the traditional "manual template design & semi-automatic corpus generation" model into a new paradigm of fully automated generation.

![Architecture Diagram](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/Architecture.drawio.svg)

The system consists of four highly cohesive, loosely coupled core modules:

| **Module** | **Function Description** |
|------------|--------------------------|
| **Schema Generator** | Understands natural-language domain and subdomain descriptions and generates corresponding graph schemas (nodes, edges, properties) for the Data Generator |
| **Data Generator** | Generates simulated node and edge data based on Schema, supporting large-scale complex relationship network construction |
| **Corpus Generator** | Uses LLM to generate high-quality Question-Query pairs containing multi-hop queries, nested queries |
| **Validator** | Checks the correctness of generated Schema, and the grammatical/semantic correctness of Query |

#### Schema Generator

The Schema Generator module automatically creates complex graph database schemas from natural language domain descriptions. It introduces Domain and Subdomain concepts to enhance semantic hierarchy and implements a quantifiable graph structure generation strategy based on a 5-level complexity model.

**Key Features:**

+ Generates SchemaGraph format schemas that can be converted into TuGraph modeling files or adapted for other database engines
+ Supports quantitative control over schema complexity through predefined node and relationship ranges
+ Uses LLM to generate Schema Descriptions and corresponding Schema JSON
+ Ensures polymorphism through SchemaGraph class for future database adapters

![Schema Generator Architecture](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/Schema.drawio.svg)

#### Data Generator

The Data Generator module creates realistic simulation data based on generated schemas, following real-world statistical distributions like power-law, long-tail, and normal distributions.

**Key Features:**

+ Generates node and edge CSV files with property constraints
+ Creates TuGraph-compatible import_config.json for batch importing via lgraph_import
+ Handles common import errors (type parsing failures, missing delimiters, null value handling)
+ Generated 488 CSV files containing ~3,716,332 rows of data during development

![Data Generator Architecture](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/Data.drawio.svg)

#### Corpus Generator

The Corpus Generator produces high-quality Question-Query pairs through a hierarchical generation strategy that balances complexity and diversity.

**Key Features:**

+ Layered generation strategy: first generates simple seed corpus, then complex corpus based on seeds
+ Real validation: all queries are executed and verified on actual graph databases
+ Context-aware generation: uses query execution results as context for LLM enhancement
+ Iterative enhancement: controllable iteration rounds for gradually increasing complexity
+ Query grading: automatically grades query difficulty (easy, medium, hard, extra hard) using LLM-based analysis
+ Generated 800+ high-quality corpus pairs across multiple domains

![Corpus Generator Architecture](https://raw.githubusercontent.com/ldbc/Text2GraphQuery-DataGen/master/images/Corpus.drawio.svg)

## Experimental Results

We used the framework’s automated generation pipeline to construct multi-dimensional test datasets `Geography_World`, `Movie_Movielens`, `Healthcare_Donor` and training datasets `Banking_Financial`, `Game Olympics`, `Retail_RetailWorld`. They can be downloaded here: [*test set*](https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/text2gql/awesome-text2gql/test.zip), [*training set*](https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/text2gql/awesome-text2gql/train.zip).

Then, we conducted zero-shot experiments using three test database instances and their generated corpora. We evaluated Qwen-Plus, Qwen-8B Base, and Qwen-8B fine-tuned with LoRA on different datasets using four metrics: **Grammar**, **Similarity**, **Google BLEU**, and **EA**.

| Test Set                       | Schema Complexity | Grammer | Similarity | Google BLEU | EA    |
| ------------------------------ | --------- | ------- | ---------- | ----------- | ----- |
| geography              | 5 |  83.7    | 85.8       | 64.3        | 16.28 |
| geography_seeds        | 5 |  100     | 84.1       | 58.7        | 14.29 |
| healthcare_donor       | 3 |  84.8    | 87.8       | 63.2        | 27.85 |
| healthcare_donor_seeds | 3 |  95.2    | 86.7       | 53.6        | 42.86 |
| movie                  | 3 |  86.5    | 88.3       | 63.4        | 14.86 |
| movie_seeds            | 3 |  96.4    | 86.8       | 50.8        | 50.00 |

<p align="center"\>Table 1. Qwen-Plus Experiment Results on LLM-Synthesis Dataset  </p\>

| Test Set               | Schema Complexity |  Grammar | Similarity | Google BLUE | EA     |
|------------------------|--------- |---------|------------|-------------|--------|
| geography              | 5 |  0.884   | 0.856      | 0.564       |  26.74% |
| geography_seeds        | 5 |  0.929   | 0.861      | 0.551       | 21.43% |
| healthcare_donor       | 3 |  0.949   | 0.885      | 0.652       | 29.11% |
| healthcare_donor_seeds | 3 |  1.000   | 0.861      | 0.510       | 42.86% |
| movie                  | 3 |  0.946   | 0.877      | 0.598       | 6.76%  |
| movie_seeds            | 3 |  0.893   | 0.864      | 0.512       | 39.29% |

<p align="center"\>Table 2. Qwen-8B Base Experiment Results on LLM-Synthesis Dataset  </p\>

| Test Set               |  Schema Complexity | Grammar | Similarity | Google BLUE | EA     |
|------------------------|--------- |---------|------------|-------------|--------|
| geography              | 5 |  0.942   | 0.878      | 0.683       | 20.93% |
| geography_seeds        | 5 |  1.000   | 0.899      | 0.747       | 71.43% |
| healthcare_donor       | 3 |  0.987   | 0.870      | 0.578       | 27.85% |
| healthcare_donor_seeds | 3 |  1.000   | 0.901      | 0.641       | 71.43% |
| movie                  | 3 |  1.000   | 0.893      | 0.600       | 28.38% |
| movie_seeds            | 3 |  1.000   | 0.927      | 0.633       | 64.29% |

<p align="center"\>Table 3. Qwen-8B Base fine-tuned with LoRA Experiment Results on LLM-Synthesis Dataset  </p\>

Testing on LLM-synthesized datasets shows that the framework successfully generates complex corpora that challenge current LLMs, with execution accuracy below 30% on complex iterated corpora. Fine-tuning experiments demonstrate that models trained on framework-generated data show improved performance.

## Development Guide

To make Awesome-Text2GQL supports the corpus construction of more types of graph query languages, we welcome contribution to the implementation of ast visitor, query translator, and schema parser of new graph query languages. If you find the compatibility of current intermediate representation is not enough for the new graph query language, we also welcome contribution to the intermediate representation.

### Introduction to Intermediate Representation

The clause class is the core of Awesome-Text2GQL's intermediate representation for graph query languages. Currently clause class has match clause, return clause, where clause and with clause as subclasses. The design of subclasses might be updated in the future for the compatibility of more graph query languages

+ Match Clause: the intermediate representation for pattern match

+ Return Clause: the intermediate representation for item return.

+ Where Clause: the intermediate representation for condition expression.

+ With Clause: the intermediate representation for variable control.

### Graph Query Language Implementation Guideline

#### Implement AST Visitor

The ast visitor class is a virtual class and should be implemented for different graph query language, like cypher ast visitor or gql ast visitor. Implementation on each graph query language should be able to parse the given query, visit the abstarct syntax tree, then return the graph pattern(a list of clauses) of the given query as the intermediate representation for further translation or generalization. See `app/impl/tugraph_cypher/ast_visitor/tugraph_cypher_ast_visitor.py` as an example.

#### Implement Query Translator

The query translator class is a virtual class and should be implemented for different graph query language, like cypher query translator or gql query translator. Implementation on each graph query language should implement the translate function to turn a list of clauses(the intermediate representation) into an actual query aligned to the grammar of corresponding language, and implement the grammar check function to check if a query is grammatically correct. See `app/impl/iso_gql/translator/iso_gql_query_translator.py` as an example.

#### Implement Schema Parser

The schema parser class is a virtual class and should be implemented for different DBMS, like neo4j schema parser or tugraph schema parser. Implementation on each DBMS should be able to parse the correspongding schema file, whether it's a set of queries or a json file, then return a in memory schema graph for query generalization. See `app/impl/tugraph_cypher/schema/schema_parser.py` as an example.

## Future Plan

Awesome-Text2GQL will continue to enhance the quality and diversity of generated corpora and improve the framework's usability and performance.

## Contribution

Do the following steps before submitting your code.

### Code Formatting

```
poetry run ruff format .
```

### Code Checking

```
poetry run ruff check ./app ./examples --fix
```

if all check passed, you can submit your code.

### Submitting Code

create a pull request, link it to a related issue, then wait for the project maintainer to review your changes and provide feedback. If your pull request is finally approved by our maintainer, we will merge it. Other details can reference to our [contributing document](https://github.com/TuGraph-family/community/blob/master/docs/CONTRIBUTING.md).

### Contributors Wall

## Attention

This project is still under development, suggestions, issues or pull requests are welcome.

Text2GraphQuery-DataGen

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README