dig-etl-engine

(β˜… 105)

Download DIG to run on your laptop or server.

File Explorer

Download Latest Version (.zip)
  • .dockerignore
  • .env.example
  • .gitattributes
  • .gitignore
  • config.py
  • config_docker_production.py
  • config_docker_sandbox.py
  • diagnose.sh
  • docker-compose-production.yml
  • docker-compose.ache.yml
  • docker-compose.dev.yml
  • docker-compose.kafka-manager.yml
  • docker-compose.rss.yml
  • docker-compose.yml
  • Dockerfile
  • engine.sh
  • etk_worker.py
  • LICENSE
  • manager.py
  • README.md
  • release_docker.sh
  • requirements.txt
  • reset_offset.py
  • VERSION

# Installation Guide

1. Get the code
git clone https://github.com/usc-isi-i2/dig-etl-engine

Downloads the entire project code from GitHub to your computer.

cd dig-etl-engine

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
sudo mv /var/lib/docker /path_with_more_space

Type this command into your terminal and run it.

sudo ln -s /path_with_more_space /var/lib/docker

Type this command into your terminal and run it.

docker build -t uscisii2/nginx:auth-1.0 nginx/.

Builds a runnable image based on the Dockerfile.

βœ… Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

Pulled directly from this repo's README.

3. Python

Easy
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.
pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <μ‹€ν–‰ν•  파일λͺ…>.py # READMEμ—μ„œ μ •ν™•ν•œ μ‹€ν–‰ 파일λͺ…을 ν™•μΈν•˜μ„Έμš”

Runs the Python script (or module).

βœ… If it runs without errors and prints output in the terminal, it worked.

Today's picks

langflow β˜… 153,529

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

Python
opencode β˜… 199,829

The open source coding agent.

TypeScript
Web-Dev-For-Beginners β˜… 96,420

24 Lessons, 12 Weeks, Get Started as a Web Developer

JavaScript
Understand-Anything β˜… 80,004

Graphs that teach > graphs that impress. Turn any code into an interactive knowledge graph

TypeScript
transformers β˜… 164,305

πŸ€— Transformers: the model-definition framework for state-of-the-art machine learning model

Python
langchain β˜… 144,721

The agent engineering platform.

Python
// repository documentation