serverless-streaming-data-pipeline

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #typescript #golang #machine-learning #deep-learning #nodejs #java #awesome #kubernetes #ios #openai

serverless-streaming-data-pipeline

(★ 9)

A serverless streaming data pipeline running on LocalStack

serverless-streaming-data-pipeline 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# LocalStack Streaming Data Pipeline Demo

A serverless ETL for demonstrating streaming structured analytics events from Lambda to Tinybird using CloudWatch and Kinesis.

## Requirements
* LocalStack Pro
* AWS [CDK](https://aws.amazon.com/cdk/)
* [cdklocal](https://github.com/localstack/aws-cdk-local)
* Python 3

## Running Locally

### Deploy Under LocalStack

1. Start LocalStack Pro: `LOCALSTACK_API_KEY="your_key" localstack start -d`
2. Install python dependencies: `make install`
3. Deploy locally: `make deploy-local`

### Testing
After following the local deployments steps above, you can run the sample integration test with `make test-integration`.
Alternatively, you can test end-to-end manually:
* Start the local Tinybird server mock with `make start-request-recorder`
* Emit an event by invoking the logger Lambda: `make invoke-test-logger-local`
* Observe the event payload arrive in the mock server output.

## Further Reading

* [Develop and Test Real-time Data Pipelines with LocalStack](https://localstack.cloud/blog/2022-04-04-develop-and-test-data-analytics-pipelines-on-localstack/)

serverless-streaming-data-pipeline

serverless-streaming-data-pipeline 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

serverless-streaming-data-pipeline

serverless-streaming-data-pipeline 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README