dbt_faker

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 20 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ***Archival Notice***
This repository has been archived.

As a result all of its historical issues and PRs have been closed.

Please *do not clone* this repo without understanding the risk in doing so:
- It may have unaddressed security vulnerabilities
- It may have unaddressed bugs

<details>
   <summary>Click for historical readme</summary>

# dbt faker 
Generate fake/test/demo/sample data directly from dbt. dbt_faker is a python model generator for generating data within a [dbt](https://docs.getdbt.com/docs/introduction) project using the Python [Faker](https://faker.readthedocs.io/en/master/) project.

![Welcome to the dbt faker project!](https://i.imgflip.com/4cfh9t.jpg)

## Install
Include in `packages.yml`:

```yaml
packages:
  - git: &quot;https://github.com/dbt-labs/dbt_faker.git&quot;
    revision: main 
```

### Requirements 
- dbt version &gt;= 1.3

## How to use it 
### 1. Add a source override macro in your project 
Create the file `macro/dbt_faker_source_override.sql` that looks like this:

```
{% macro source(source_name, table_name) %}
{{ return(dbt_faker.dbt_faker_source(source_name, table_name)) }}
{% endmacro %}
```
Activate the faker_enabled variable in your project.yml 
```
vars: 
  faker_enabled: true
```
### 2. Declare your sources.yml
including columns and [faker_providers](#providers), and add the meta config `faker_enabled:true`.

```yaml
version: 2

sources:
  - name: tpch
    meta:
      faker_enabled: true
  - name: fake_tpch
    tables:
      - name: orders
        meta:
          faker_enabled: true
          faker_rows: 250
        columns:
          - name: o_orderkey
            meta:
              faker_provider: pyint

- name: o_order_date
            meta:
              faker_provider: date

# an example of using faker provider parameters
          # see docs for this example at https://faker.readthedocs.io/en/master/providers/faker.providers.lorem.html#faker.providers.lorem.Provider.word
          - name: o_customer_name
            meta:
              faker_provider: word
              faker_params:
                ext_word_list: [&quot;customer A&quot;, &quot;customer B&quot;, &quot;customer C&quot;]
```

### 3. Generate your python model 
Execute the command `dbt run-operation generate_faker_model`

### 4. Copy the output of your terminal and create a python model 
Create a file (e.g. dbt_faker.py) with the code generated from step #2

### 5. Execute your newly created python model
For example `dbt run -m dbt_faker.py`. This will create a table called fake__source_table for each source you have defined as fake-able

### 6. Use your fake data! 
Run the models depending on the fake sources and be amazed

## Providers
dbt_faker relies on [Faker's](https://faker.readthedocs.io/en/master/) robust data providers. In order to use them, simply include the name of the provider in the `faker_provider` meta tag. A full list of providers is [here]([url](https://faker.readthedocs.io/en/master/providers.html)). Some examples you can use:

- [faker_provider.address](https://faker.readthedocs.io/en/master/providers/faker.providers.address.html) (48764 Howard Forge Apt. 421 Vanessaside, PA 19763)
- [faker_provider.name](https://faker.readthedocs.io/en/master/providers/faker.providers.person.html) ( Diego Maradona)
- [faker_provider.pyint](https://faker.readthedocs.io/en/master/providers/faker.providers.python.html) (1234)

If a fake_provider has not been defined for a column, dbt faker will generate a string by default.

## FAQ

### `generate_faker_model` is skipping my sources   
    You should check that your sources have: 
        - Columns defined in the sources.yml
        - the meta field faker_enabled: true either at the source name level or source table name level
        - the meta field faker_enabled:false not defined at the source table level

### `dbt run -m dbt_faker.py` gives me a warning that the selector haven't found the model
    You may not be running dbt 1.3, needed to be able to execute dbt python models

dbt_faker

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README