etl

KO | EN

gitlite — search

#javascript #react #typescript #llm #golang #android #rust #deep-learning #macos #awesome #cli #kubernetes #pytorch #mcp

(★ 20)

Convenience tooling to perform database interactions from CI, bash scripts and automation processes.

etl Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ETL - SQL-First CLI, API & Web Development Framework

[![Go Report Card](https://goreportcard.com/badge/github.com/titpetric/etl)](https://goreportcard.com/report/github.com/titpetric/etl)
[![Testing coverage](https://img.shields.io/badge/coverage-54.2%25-yellowgreen)](https://raw.githubusercontent.com/titpetric/etl/main/docs/testing-coverage.md)
[![GoDoc](https://pkg.go.dev/badge/github.com/titpetric/etl.svg)](https://pkg.go.dev/github.com/titpetric/etl)

**ETL** started out as a CLI tool to interface databases with JSON from
the terminal. It became something more, with a server, where you can define
API endpoints using parametrized SQL queries in a yaml configuration.

Currently, it provides:

- A CLI for writing/reading from databases using JSON
- A server that uses a .yaml definition to provide APIs
- Web development capabilities with templating from API responses (SSR)
- CQRS support by composing definitions with `include`.

## Installation

```bash
go install github.com/titpetric/etl/cmd/etl@main
```

## Quick Navigation

- **[CLI Reference](./docs/cli.md)** – Command-line tool usage, database operations, and examples
- **[Configuration](./docs/config.md)** – Server configuration, endpoints, handlers, and options
- **[Server Development](./docs/server.md)** – API configuration, web development, and template rendering

## Overview

ETL supports these major use cases:

1. **CLI Database Interface** - Query and manipulate databases from shell scripts
2. **SQL-First API Development** - Expose database queries as json based REST/RPC API endpoints with zero boilerplate
3. **Web Development** - Responses can be templated, reusing existing APIs defined in the system as the data source
4. **CQRS Development** - Composable configuration allows trivial separation of read/write statements

## Supported Databases

- SQLite
- PostgreSQL
- MySQL

The ETL CLI aims to be database agnostic. It leans into `json` as a
portable data format, and implements compatibility drivers to talk to
each database, storing and returning JSON with a common API.

The SQL you define on your own is not portable, as it depends on the
database server in use. This allows you to use server specific syntax.

## Testing

Several test suites demonstrate core functionality:

- **[tests/git-tags](./tests/git-tags)** – CLI usage examples
- **[tests/petstore](./tests/petstore)** – SQL-as-API examples
- **[tests/users](./tests/users)** – CQRS and integrated testing

## Configuration

Set environment variables with your connection/storage details:

| Database    | Example DSN                                                         |
|-------------|---------------------------------------------------------------------|
| SQLite      | `export ETL_DB_DSN="sqlite://file:myapp.db"`                        |
| PostgreSQL  | `export ETL_DB_DSN="postgres://user:pass@localhost:5432/dbname"     |
| MySQL       | `export ETL_DB_DSN="mysql://user:pass@tcp(localhost:3306)/dbname"`  |

## Why ETL?

**Standard database clients** (`psql`, `mysql -e`) require mastering database-specific syntax and CLI arguments and flags.

**ETL provides:**

- Unified interface across SQLite, PostgreSQL, and MySQL
- First-class JSON support for input/output
- Simple insert/update operations via JSON or arguments
- Custom SQL query capability for complex operations
- Rapid API development without boilerplate
- Web application rendering alongside API servers

**Use with** `jq`, `yq`, or other standard tools for data transformation pipelines.

## License

[See LICENSE file](./LICENSE)

## References

- [go-bridget/mig](https://github.com/go-bridget/mig) – For database migrations
- [titpetric/vuego](https://github.com/titpetric/vuego) – Template engine
- [titpetric/platform](https://github.com/titpetric/platform) – HTTP server framework

etl

etl Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

etl

etl Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README